Skip to content

Do not attempt to process files when there are none#65

Open
set321go wants to merge 10000 commits intoapache:trunkfrom
set321go:trunk
Open

Do not attempt to process files when there are none#65
set321go wants to merge 10000 commits intoapache:trunkfrom
set321go:trunk

Conversation

@set321go
Copy link
Copy Markdown

I tried to create a ticket for this but I can't create tickets in ASF jira.

Fileinstall was surprisingly at the top of the hotspot list while profiling today while running fileinstall 3.5.0 after upgrading to 3.5.4 there were some improvements. Even after stretching the polling from the default 2sec to 60sec it still used a lot of cpu. My file install directory contains 4 config files and no subdirectories.

This change prevents process being called when files is empty which is most of the time.

I also noticed locally that during a poll this line of code was called multiple times (usually 4).

Pierre De Rop and others added 30 commits February 14, 2016 13:17
- added a configType attribute in FactoryConfigurationAdapterService annotation.
- when using a config type with ConfigurationDependency, then assume that pid is set to the fqdn of the provided config type, in case
no pid has already been set using setPid method.
- code cleanup.
- adapted samples to use configuration type.
- added javadocs.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1730934 13f79535-47bb-0310-9956-ffa450edef68
…library:

- renamed the DependencyManagerActivator.activate() method to  init(BundleContext ctx, DependencyManager dm) 
- Removed abbreviated names like "cb()" or "cbi" methods, and replaced them with niced method names like "add()/changed()/removed()" ...
- Added a system property "org.apache.felix.dependencymanager.lambda.dependencymode" that allows to control the default
mode of dependency (still to be discussed).
- added full support or new configuration types.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731147 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731238 13f79535-47bb-0310-9956-ffa450edef68
…adapter component for the factory pid adapter.

git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731456 13f79535-47bb-0310-9956-ffa450edef68
…oxy() method has been added in order to allow to create

configuration types from any DM custom dependencies.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731457 13f79535-47bb-0310-9956-ffa450edef68
…stopped.

FELIX-5193: Factory Pid Adapter race condition when component is stopped.
Removed Eclipse warnings. Reworked the way configuration exception is handled.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731471 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731614 13f79535-47bb-0310-9956-ffa450edef68
…to ComponentContext.createConfigurationProxy.createConfigurationType.

Fixed some javadocs.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731979 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731989 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1731994 13f79535-47bb-0310-9956-ffa450edef68
…exception

Fixed using 'active' flag. Unit test included.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1732445 13f79535-47bb-0310-9956-ffa450edef68
…exception

Improved the thread safety of this fix.


git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1732453 13f79535-47bb-0310-9956-ffa450edef68
cziegeler and others added 26 commits May 11, 2016 06:55
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743317 13f79535-47bb-0310-9956-ffa450edef68
…SlingFilter order element. Apply patch from Konrad Windszus

git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743324 13f79535-47bb-0310-9956-ffa450edef68
…clipse. Apply second patch from Stefan Seifert

git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743325 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743459 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743468 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743472 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743473 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/felix/trunk@1743477 13f79535-47bb-0310-9956-ffa450edef68
@CodingFabian
Copy link
Copy Markdown
Member

This is a semantics change, as previously the call would cause failures to be re-tried. should failures be empty and files be empty the code would be actually fast.

as I have profiled fileinstall and was responsible for some of the improvements in 3.5.4, i would like to see the profile, maybe i am missing something.
in our case the code is no longer hot and we poll every second hundreds of files.

@set321go
Copy link
Copy Markdown
Author

set321go commented May 14, 2016

I'm not sure what you mean. Looking at Scanner.scan it news up a TreeSet as the return type, the only way i'm getting anything other than a Set is an Exception which scan does not catch, tbh you could probably remove the null check entirely.
https://github.com/set321go/felix/blob/7a6bdb0895674f2359a2a05ee50f802baaa612c2/fileinstall/src/main/java/org/apache/felix/fileinstall/internal/Scanner.java#L131

I can get some of the profiling data next week, I don't have it here.

@CodingFabian
Copy link
Copy Markdown
Member

@set321go out of coincidence my profiling arrived now also here as optimization candidate, so I had a look at it.
What i meant is that doProcess will not only work on files but also on processingFailures. So while there would be no changes on file system, the mechanism would retry the processing failures.

so a change I would approve is

                     if (!files.isEmpty() || !processingFailures.isEmpty()) {
                          process(files);
                      }

which would not do much, but avoid locking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants