GPII-1839: Acceptance/Integration tests must use the dynamic device reporter#494
GPII-1839: Acceptance/Integration tests must use the dynamic device reporter#494javihernandez wants to merge 20 commits intoGPII:masterfrom
Conversation
We don't want to include some things from the original klown's feature branch.
Also: * updated the config that needs to be used for all test definitions * marked "com.ilunion.cloud4chrome" to not report itself as installed in linux as it was made for windows in GPII#482 (GPII-2109)
Also, updated the config that needs to be used for all test definitions
* Windows & Linux now always use the dynamic device reporter * Android keeps using the static device reporter - re-added gpii.tests.acceptance.localInstall.config.json * Removed the old dynamic reporter tests in both windows & linux platforms * Removed all different and no longer needed config files for each test suites - All tests use the same config * Renamed config names for consistency
Now these tests also use the dynamic device reporter.
* upstream/master: (167 commits) GPII-1884: Updated provisioning/requirements.yml to use the master branches of gpii-ops/ansible-preferences-server and gpii-ops/ansible-flow-manager with the pull requests on those repos for fixing GPII-1884 have been merged. GPII-1274: Removing fluid.logObjectRenderChars directives from outlying tests GPII-1274: Updated to use the latest gpii-pouchdb changes to hide the interruptive pouchdb document update conflict errors at running acceptance tests. GPII-1274: Updated to use the latest gpii-pouchdb changes. GPII-1809: Fixing up KASPARNET's barbarous comments GPII-1809: Added explanation for why we're loading all solution registry entries when inferring common terms GPII-1809: Addressed most of antranigs comments GPII-1274: Moved GPII production VM from "examples/production-components-vm/" to "vagrant-configs/" directory. GPII-1274: Ensure the VM that runs GPII in the production mode installs node.js 6.x which is now required by GPII/universal code base. GPII-1274: Included json5.js into in-browser tests for the need of the kettle dataSource-core.js dependency. GPII-1274: Moved gpii-pouchdb from npm devDependencies to dependencies to allow platform specific repos, e.g. windows and linux, to use universal development environment. GPII-1274: Improve the save test for gpii.dataSource.pouchDB to use "writable: true" option hooking up the writable grade instead of using the writable grade itself directly. GPII-1274: Added readOnlyGrade option in gpii.dataSource.pouchDB component. GPII-1274: Improved scripts/browserifyTestDependency.js accroding to the review comments. GPII-2074: Now supports specialized per-ontology ontological matching function GPII-1274: Removed unneeded grunt packages. GPII-1274: Updated the npm postinstall step to make sure the script browserifying only runs when the devDependency "browserify" module exists. GPII-1809: Got the remaining tests working properly again GPII-1274: Improved the pouch manager tests to also test pouchDB queries by views. GPII-1274: Addressed pull request comments. ...
|
This is not ready yet for merging since I still need to make the platform sides of this pull request (will be ready soon), but the review of this universal portion can begin. |
|
CI job passed. |
|
CI job failed. Please visit http://lists.gpii.net/pipermail/ci/ for more details. |
|
ok to test |
|
CI job passed. |
|
CI job passed. |
* upstream/master: (29 commits) GPII-2235: Minor tidying GPII-2235: Unit test improvements GPII-2235: Added single instance detection to flowManager lifecycle GPII-2259: Updated the extraneous copies of Infusion client code that the tests depend on to the matching release of Infusion GPII-2259: Minor dependency update for universal to latest infusion, express, oauth2orize etc. GPII-2235: Added component destruction GPII-2235: Tarted up the unit tests. GPII-2235: Added clarity for 0 pid. GPII-2235: Moved pidFile path generation code into function, and disposing component. GPII-2235: Ensures there's only one instance of GPII TT-5: Added inverseCapabilitiesTransformations for JAWS TT-5: Adressed review comments TT-5: Minor cleanup and MDs added TT-5: All tests passing. Modified the acceptance test framework to allow a single acceptance test to define the maxTimeouts setting for when checking exec commands. This was done because Jaws was slow on my machine and wasn't shut down within the 6 sec. allowed time limit TT-5: Refixed all integration tests to work properly again TT-5: Updated solutions registry format and valuemapper entries for jaws minor changes add tests updated solution win32 changes ...
|
This is ready again for another round of review. I had a concern related to what tests must run and whether we should improve our filtering options when running the Acceptance/Integration tests. i.e.: right now we can indicate the tests we want to run (node AcceptanceTests.js builtIn) but I wasn't sure if we really want to improve such thing. To me, right now it's enough - if you call the tests with no arguments all the tests will run. And after having a conversation with Kasper in Maryland, the question is, do we want to improve our current filtering in order to, i.e.:
@GPII/core-architecture-committers, I do know that right now it's enough, but I just wanted to hear your opinions on this. If you prefer, we can create a separate issue for this topic and have the discussion there. The review of this pull request can continue either case. |
|
CI job failed. Please visit http://lists.gpii.net/pipermail/ci/ for more details. |
|
@javihernandez - the current implementation we have here https://github.com/GPII/universal/blob/master/gpii/node_modules/testing/src/Testing.js#L495 does a simple OR between all strings - feel free to update it to apply AND instead and/or NOT filtering with syntax such as !JAWS, etc. |
|
I checked out the branch and ran the web tests just fine. I think there must be a fault with the CI machine that has left some dirty files from a previous run - @gtirloni ? |
|
I can consistently reproduce this issue on my machine (from @amb26 #501 - one which is not failing on the CI) My environment is IOS as host, I'm using the idrc 1.0.0 box (default from universal) and testing from the universal branch only. I've tried:
and various combinations thereof... everything fails in the same place.... Note that this is not based on this pull request, but rather #501 |
|
ok to test |
|
CI job passed. |
|
There was a VirtualBox process for universal that was stuck running for a long time. I don't know how VirtualBox shares resources but I have a hunch the OS-level processes separation is not the whole story since it has a kernel component. I killed that process (which terminated fine, it was not ignoring signals) and re-ran this PR through Jenkins and then it worked. Unfortunately, I forgot to collect the VM logs from this process and now we're only left with the logs from the VM that worked. I suggest we keep an eye on this kind of issue, collect more data and maybe address it upstream with the VirtualBox team. Sorry for the inconvenience. |
* upstream/master: GPII-2283: Updates to logging payloads following review GPII-2283: Adjustment of logging payloads in advance of HST GPII-2146: Linting fix GPII-2160: Updates to line lengths following review GPII-2160: Updates to line length and commenting following review GPII-2259: Pushing further updates through GPII-2160: Fix and test case for failure to continue with journal restore actions if one results in a failure GPII-252: Add support for highContrastTheme GPII-2146 Addressing comments from PR. Factoring out querying of kettle.configs, changing API casing, and fixing up js docs. GPII-2146 Adding optional Options block to gpii.start to allow changing the configName and configPath. Also adding a gpii.stop function... because if you start it you might want to stop it! GPII-2151: Removed now superfluous material from README.md GPII-2151: Allow universal's test cases to run from an isolated checkout
|
CI job failed. Please visit http://lists.gpii.net/pipermail/ci/ for more details. |
|
looks like we've hit the canopy's mm multilayer support problem again, this time with maavis 😢
Both of the proposed solutions are temporal and can be reverted once https://issues.gpii.net/browse/GPII-1998 is solved. My preference is to disable the failing test, but I'd like to hear your thoughts on this. |
…d on their priority
…is. Fixed maavis solution to reference all the supported common terms
* kasper/GPII-1839_GPII-2393: GPII-1839: Added priorities to the maavis NP set to actually use maavis. Fixed maavis solution to reference all the supported common terms GPII-2393: Now tiebreaking equally fit non-conflicting solutions based on their priority GPII-2393: Failing test to show illustrate the issue
|
CI job passed. |
|
After solving the problems with conflicting solutions, this is ready again for a new round of review. |
|
hi @javihernandez - there is a minor conflict with master! Also, after merging the two, I'm experiencing GPII-1880 (which I have made a fix for here: #532). Other than that, this looks pretty good! Once GPII-1880 is merged with master and this pull request have been updated, I'll give it a final lookover |
|
@javihernandez could you merge this with master and have CI run through it again - then I'll give it another round of review |
Also, removed unnecessary config file and updated some txt files
|
CI job passed. |
|
@kaspermarkus ready again. Let me know if you need anything else regarding this pull. 😉 |
|
sorry @javihernandez ... new conflicts showed up, but I promise (scouts honor) that I'll review again as soon as you've merged it with master |
* upstream/master: (86 commits) GPII-2435: Typo in CBFM comment GPII-2435: Replaced the programmatically constructed paths in the oauth acceptance tests with the "termMap" option that is now supported by "kettle.test.request.http". GPII-2435: Fixed the issue that, when a wrong user id/password combination is provided at the auth server login page, the error message is not displayed inline on the page but is displayed on a separate page. GPII-2435: Fixed the VM in "vagrant-configs" directory. GPII-2435: Work around issues with spinning up the production mode VM in /vagrant-configs directory. Add split node_modules workaround GPII-2435: Improved the documentation. GPII-2435: Addressed more review comments. GPII-2435: Removed gpii.oauth2.getExpiresIn() that is no longer in use. GPII-2435: Modified oauth2 date processing utility functions to received the current timestamp as an argument instead of generating it inside functions. GPII-2612: Correcting ToC transformation for uio+ solution GPII-2583: Removed definition of gpii.metrics namespace. There's a component with the same name. GPII-2583: Removed testing of "duration" field. GPII-2435: Addressed more review comments. GPII-2586: Logging solutions that did not get applied. GPII-2435: Removed one more unneeded require() block. GPII-2435: Removed unneeded require() blocks. GPII-2435: Addressed review comments. NOJIRA: Added net.gpii.uioPlus into installedSolutions.json GPII-2297: Added filtering of common terms ...
* Updated uioPlus tests (and removed related files that are no longer needed in order to run these tests) * Updated ContextIntegrationTests * Updated os_common and maavis_selfVoicing np sets in order to workaround the current problems with the canopy matchmaker
|
CI job failed: https://ci.gpii.net/job/universal-tests/545/ |
|
@kaspermarkus, this failure is expected since I still need to update a few more things.
|
|
Hi @javihernandez Is the conversation with me still relevant? If you, we should have that soon, so we can get this pull out of the way :) |
5 participants
Supersedes #461.
Also, includes the fix for https://issues.gpii.net/browse/GPII-2173.
This pull request make the integration/acceptance tests use the dynamic device reporter. When running the acceptance tests, it will fail when a solution can't be found.