Implement import and export

Pull request type

Feature
What is the current behavior?

Issue Number: fixes #111, fixes #114, fixes #115
What is the new behavior?

The following formats can be imported and exported using the Pepper library and a simple graphical wizard

EXMARaLDA
Paula XML

The Pepper framework is added as a regular dependency, but the plugins are JAR-files loaded and activated by the new bundle.
This allows us to use a fixed version and don't depend on the Pepper update mechanism.
Does this introduce a breaking change?

N/A
Other information

Pull request checklist
Please check if your PR fulfills the following requirements:

[X] Tests for the changes have been added or the PR is neither a bug fix nor a feature
[x] Docs have been reviewed and added / updated if needed or the PR is neither a bug fix nor a feature
[x] The Unreleased section in CHANGELOG.md has been amended to reflect the changes in this PR if needed
[x] Build (mvn verify) was run locally and any changes were pushed
[x] The pull request is against the correct branch (master for bug fixes, develop for new functionality)
[x] Dependencies and citation templates have been updated where necessary or there are no new dependencies

Checklist for maintainers
Releases

Do NOT push the green merge button on GitHub.
Follow the latest version of the developer/maintainer docs for making releases.
Check that license and citation information for dependencies are complete (each dependency has a folder in /THIRD-PARTY/).

I'm wondering whether it'd make sense to include some minimal test data for the formats we support for import & export.
From a code perspective, this would be outside the scope, as it would mostly test code in the pepper-plugins.
From a functionality perspective, though, it'd make sense in integration tests to check whether the correct info has made it into the UI, e.g., the project manager, and possibly even one of the editors.
What do you think, @thomaskrause?

Also just tried importing the EXMARaLDA test corpus (https://doi.org10.25592/uhhfdm.8364), which failed for 77 documents (perhaps that's all of them, I haven't counted). Trying to import different sub-folders also fails.
This is technically in the scope of the EXMARaLDA modules, but we should think about whether it is perhaps a requirement that importing the test corpus works?

1. (Re-)export doesn't work (should be fixed in the scope of this PR):

   1. Open example corpus (Salt)
   2. Export to PAULA
   3. Import from PAULA (this is where the gif starts)
   4. Export menu item is not active

This is actually intended and documented behavior right now. If you import a corpus, it will not have a location on disk (only a temporary one which will vanish when Hexatomic is closed), which is indicated by the "unsaved" marker. The export only works if the project is saved to a non-temporary disk (see "A project must be saved to the disk and must not have any unsaved changes before you can export it." in the documentation). We must tell Pepper from where to convert the files from, hence this restriction. This is also true for unsaved projects, even if they have a location: when the changes are not saved yet, Pepper has no way of converting the actual non-saved data. The current solution is to disable the export menu entry, but we could also enable it and ask the user to save the project files before she/he can actually perform the export.
For the other issue when not-autodetecting a format, this is actually already a strong indicator that something is wrong, since both formats currently have a quite good autodection. The user would also be free to manually select another format and click next, but I thought the conscious choice of the user is "warning" enough. If the user tells us that she/he is really sure this is this format, they should expect errors on the way. How to best report errors is a different topic though, since it is really difficult to extract this information from Pepper.
We could distinguish the error message if no document was imported at all from the one were only some documents have an error and indicate to the user, that maybe the wrong format was selected. I will try to implement this "complete fail".

Also just tried importing the EXMARaLDA test corpus (https://doi.org/10.25592/uhhfdm.8364), which failed for 77 documents (perhaps that's all of them, I haven't counted). Trying to import different sub-folders also fails.
This is technically in the scope of the EXMARaLDA modules, but we should think about whether it is perhaps a requirement that importing the ("official") test corpus works?
Options:
* Adapt EXMARaLDA modules to work with the test corpus

* Scope out better what "EXMARaLDA import" means for Hexatomic in this PR (i.e., which dataset does actually import successfully? Use that as test data and in the docs?)

* Leave out EXMARaLDA import for now

Weird, I will check if these affects all files and is a systematic error (non-implemented feature of Exmaralda?) and then we can discuss the options.

We could distinguish the error message if no document was imported at all from the one were only some documents have an error and indicate to the user, that maybe the wrong format was selected. I will try to implement this "complete fail".

This has been implemented in 340db9e2.

Also just tried importing the EXMARaLDA test corpus (https://doi.org/10.25592/uhhfdm.8364), which failed for 77 documents (perhaps that's all of them, I haven't counted). Trying to import different sub-folders also fails.
This is technically in the scope of the EXMARaLDA modules, but we should think about whether it is perhaps a requirement that importing the ("official") test corpus works?
Options:
* Adapt EXMARaLDA modules to work with the test corpus

* Scope out better what "EXMARaLDA import" means for Hexatomic in this PR (i.e., which dataset does actually import successfully? Use that as test data and in the docs?)

* Leave out EXMARaLDA import for now

Weird, I will check if these affects all files and is a systematic error (non-implemented feature of Exmaralda?) and then we can discuss the options.

Ok, so several things are happening here.

The corpus is in the "corpus manager format" of Exmaralda, with a lot of extra files in addition to *.exb files. These files and directories containing only filtered files should be ignored by the Exmaralda Importer or it should detect that this is not a "EXB" corpus
The actual *.exb files have generic meta-information with no name: <ud-information attribute-name="" />. Trying to add these empty named annotations to Salt fails with an exception. This must be fixed in the Exmaralda Importer to make it more robust.

The current solution is to disable the export menu entry, but we could also enable it and ask the user to save the project files before she/he can actually perform the export.

I would actually prefer that the menu item is active and the user is asked to save as you suggest.

For the other issue when not-autodetecting a format, this is actually already a strong indicator that something is wrong, since both formats currently have a quite good autodection.

I see, but the user may not know this.

The user would also be free to manually select another format and click next, but I thought the conscious choice of the user is "warning" enough. If the user tells us that she/he is really sure this is this format, they should expect errors on the way. How to best report errors is a different topic though, since it is really difficult to extract this information from Pepper.

What about informing users that auto-detection wasn't possible with a status update in the wizard? Is cheap enough and makes this clearer.

We could distinguish the error message if no document was imported at all from the one were only some documents have an error and indicate to the user, that maybe the wrong format was selected. I will try to implement this "complete fail".

:+1:

1. The corpus is in the "corpus manager format" of Exmaralda, with a lot of extra files in addition to `*.exb` files. These files and directories containing only filtered files should be ignored by the Exmaralda Importer or it should detect that this is not a "EXB" corpus. This confusion around what the "EXMARaLDA format" is was the reason I originally wrote something like "EXMARaLDA Partitur Editor format" in the documentation but then simplified it because I was afraid no one knows what I'm talking about (except maybe Thomas Schmidt). In addition to the "EXMARAaLDA Corpus Manager format" (Coma) there is also a specialized transcription format with the ending "exs".

I see. Yes, ignoring files or detecting as "non-EXB" corpus would be good to have in the import. Either way, I think the respective info should probably be patched through to the Hexatomic user via the wizard (e.g., let it say some files in the import are ignored as they are not .exb files, or Corpus is not a pure EXB corpus (former would be better I think, as it doesn't signal "I can' do this" but "I can do this, but I won't be able to do everything").

2. The actual `*.exb` files have generic meta-information with no name: `<ud-information attribute-name="" />`. Trying to add these empty named annotations to Salt fails with an exception. This must be fixed in the Exmaralda Importer to make it more robust.

:+1:
On the topic of whether the changes to the EXB Importer (shold it be renamed to EXMARaLDA_EXB_Modules?) should be in the scope of this PR: I don't see it. I'd rather approve the fact that import / export works with the discussed changes. The changes to the importer should be made and integrated asap, but that's a different scope.
What do you think?

The current solution is to disable the export menu entry, but we could also enable it and ask the user to save the project files before she/he can actually perform the export.

I would actually prefer that the menu item is active and the user is asked to save as you suggest.

Is implemented that way, was a little bit difficult to actually test it because it both needs mocked objects and the real SWT shell for the dialogs.

For the other issue when not-autodetecting a format, this is actually already a strong indicator that something is wrong, since both formats currently have a quite good autodection.

I see, but the user may not know this.

The user would also be free to manually select another format and click next, but I thought the conscious choice of the user is "warning" enough. If the user tells us that she/he is really sure this is this format, they should expect errors on the way. How to best report errors is a different topic though, since it is really difficult to extract this information from Pepper.

What about informing users that auto-detection wasn't possible with a status update in the wizard? Is cheap enough and makes this clearer.

There is now an error message indicating this.

I'm assuming no, but is there a way to get rid of the lengthy error messages logged from the PepperTest-Thread (whose name feels a bit awkward to have here as well)?
22:13:47.894 [PepperTest-Thread] INFO org.corpus_tools.pepper.core.PepperOSGiRunner - ************************************************************************************************************************
*                                         ____                                                                         *
*                                        |  _ \ ___ _ __  _ __   ___ _ __                                              *
*                                        | |_) / _ \ '_ \| '_ \ / _ \ '__|                                             *
*                                        |  __/  __/ |_) | |_) |  __/ |                                                *
*                                        |_|   \___| .__/| .__/ \___|_|                                                *
*                                                  |_|   |_|                                                           *
*                                                                                *
************************************************************************************************************************
* Pepper is a Salt model based converter for a variety of linguistic formats.                                          *
* For further information, visit: http://u.hu-berlin.de/saltnpepper                                                    *
* For contact write an eMail to<EMAIL_ADDRESS>                                                     *
************************************************************************************************************************

22:13:47.908 [PepperTest-Thread] INFO org.corpus_tools.pepper.core.PepperOSGiRunner - +----------------------------------------------------------------------------------------------------------------------+
|  configuration for Pepper                                                                                |
+----------------------------------------------------------------------------------------------------------------------+
| pepper.detailedStatusReport             : true                                                                       |
| pepper.gcAfterDocumentSleep             : true                                                                       |
| pepper.memPolicy                        : MODERATE                                                                   |
| pepper.maxAmountOfProcessedSDocuments   : 10                                                                         |
+----------------------------------------------------------------------------------------------------------------------+

22:13:47.919 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DoNothingImporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.919 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DoNothingImporter'.
22:13:47.920 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'SaltXMLImporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.920 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'SaltXMLImporter'.
22:13:47.921 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'TextImporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.921 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'TextImporter'.
22:13:47.921 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DoNothingImporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.921 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DoNothingImporter'.
22:13:47.921 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'SaltXMLImporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.922 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'SaltXMLImporter'.
22:13:47.922 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'TextImporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.922 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'TextImporter'.
22:13:47.923 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DOTManipulator'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.924 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DOTManipulator'.
22:13:47.925 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DoNothingManipulator'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.925 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DoNothingManipulator'.
22:13:47.927 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'SaltValidator'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.927 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'SaltValidator'.
22:13:47.927 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DOTManipulator'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.927 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DOTManipulator'.
22:13:47.928 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DoNothingManipulator'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.928 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DoNothingManipulator'.
22:13:47.928 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'SaltValidator'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.928 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'SaltValidator'.
22:13:47.930 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DOTExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.930 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DOTExporter'.
22:13:47.931 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DoNothingExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.931 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DoNothingExporter'.
22:13:47.932 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'SaltXMLExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.932 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'SaltXMLExporter'.
22:13:47.933 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'TextExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.933 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'TextExporter'.
22:13:47.933 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DOTExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.933 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DOTExporter'.
22:13:47.934 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'DoNothingExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.934 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'DoNothingExporter'.
22:13:47.934 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'SaltXMLExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.934 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'SaltXMLExporter'.
22:13:47.935 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot find location of resource folder for Pepper module 'TextExporter'. This could cause problems running the module. Tried to detect the location via system property 'osgi.bundles' which was 'reference:file:org.eclipse.osgi.compatibility.state_1.1.800.v20200511-1223.jar,reference:file:org.apache.felix.scr_2.1.16.v20200110-1820.jar@1:start,reference:file:org.eclipse.equinox.simpleconfigurator_1.3.500.v20200211-1505.jar@1:start'.
22:13:47.935 [PepperTest-Thread] WARN org.corpus_tools.pepper.core.ModuleResolverImpl - Cannot set resource for Pepper module 'TextExporter'.
22:13:47.936 [PepperTest-Thread] INFO org.corpus_tools.pepper.core.PepperOSGiRunner - +-----+---------------------+----------------+------------+--------------------------------+---------------------------+
| no. | module-name         | module-version | module-type| formats                        | website                   |
+-----+---------------------+----------------+------------+--------------------------------+---------------------------+
| 1   | DoNothingImporter   | 3.5.3          | IMPORTER   | doNothing, 0.0                 | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 2   | SaltXMLImporter     | 3.5.3          | IMPORTER   | SaltXML, 1.0                   | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 3   | TextImporter        | 3.5.3          | IMPORTER   | txt, 0.0                       | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 4   | DOTManipulator      | 3.5.3          | MANIPULATOR|                                | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 5   | DoNothingManipulator| 3.5.3          | MANIPULATOR|                                | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 6   | SaltValidator       | 3.5.3          | MANIPULATOR|                                | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 7   | DOTExporter         | 3.5.3          | EXPORTER   | dot, 1.0                       | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 8   | DoNothingExporter   | 3.5.3          | EXPORTER   | doNothing, 0.0                 | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 9   | SaltXMLExporter     | 3.5.3          | EXPORTER   | SaltXML, 1.0                   | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
| 10  | TextExporter        | 3.5.3          | EXPORTER   | txt, 1.0                       | http://corpus-tools.      |
|     |                     |                |            |                                | org/pepper/               |
+-----+---------------------+----------------+------------+--------------------------------+---------------------------+

22:13:47.936 [PepperTest-Thread] ERROR org.corpus_tools.pepper.core.PepperOSGiRunner - An exception in Test occured. Cannot start pepper-osgi-runner, please set environment variable 'PEPPER_TEST_WORKFLOW_FILE' to workflow description file which is supposed to be used for conversion.
22:13:47.936 [PepperTest-Thread] INFO org.corpus_tools.pepper.core.PepperOSGiRunner - Conversion ended, and needed: 47 ms
22:13:47.936 [PepperTest-Thread] INFO org.corpus_tools.pepper.core.PepperOSGiRunner - ************************************************************************************************************************
22:13:48.417 [main] INFO  o.c.h.c.ApplicationLifecycle - Logging configured from logback.xml in the Hexatomic root folder
22:13:48.451 [main] INFO  o.c.h.f.Activator - Using /home/stephan/src/hexatomic/releng/org.corpus_tools.hexatomic.product/target/products/org.corpus_tools.hexatomic.product/linux/gtk/x86_64/configuration/org.eclipse.osgi/31/0/.cp/pepper-plugins as Pepper module resource folder
22:13:48.452 [main] INFO  o.c.h.f.Activator - Installing Pepper module from file:/home/stephan/src/hexatomic/releng/org.corpus_tools.hexatomic.product/target/products/org.corpus_tools.hexatomic.product/linux/gtk/x86_64/configuration/org.eclipse.osgi/31/0/.cp/pepper-plugins/exmaralda-emf-api-1.2.2.jar
22:13:48.474 [main] INFO  o.c.h.f.Activator - Installing Pepper module from file:/home/stephan/src/hexatomic/releng/org.corpus_tools.hexatomic.product/target/products/org.corpus_tools.hexatomic.product/linux/gtk/x86_64/configuration/org.eclipse.osgi/31/0/.cp/pepper-plugins/pepperModules-EXMARaLDAModules-1.3.1.jar
22:13:48.486 [main] INFO  o.c.h.f.Activator - Installing Pepper module from file:/home/stephan/src/hexatomic/releng/org.corpus_tools.hexatomic.product/target/products/org.corpus_tools.hexatomic.product/linux/gtk/x86_64/configuration/org.eclipse.osgi/31/0/.cp/pepper-plugins/pepperModules-PAULAModules-1.3.0.jar

Suggest to remove the VSCode-specific files in docs/.vscode/....

Suggest to remove the VSCode-specific files in docs/.vscode/....

Removed and added an ignore entry, so it won't be added again. There are some nice plugins for spellchecking markdown files with https://languagetool.org in VS Code, that's why they were added the first place.

I'm assuming no, but is there a way to get rid of the lengthy error messages logged from the PepperTest-Thread (whose name feels a bit awkward to have here as well)?

Nope, Pepper does not really care about any other environment it is running in. We could try to silence all messages, but this could also lead to problems when we have to debug real-life conversion problems. Proper error handling and better integration as a library are the main reasons why I prototyped a new implementation of the framework (https://github.com/korpling/pepper-ng). But this is future tech we can't rely on yet :)

I'm assuming no, but is there a way to get rid of the lengthy error messages logged from the PepperTest-Thread (whose name feels a bit awkward to have here as well)?

Nope, Pepper does not really care about any other environment it is running in. We could try to silence all messages, but this could also lead to problems when we have to debug real-life conversion problems. Proper error handling and better integration as a library are the main reasons why I prototyped a new implementation of the framework (https://github.com/korpling/pepper-ng). But this is future tech we can't rely on yet :)

Would've thought so 😉. Guess people will learn to live with it, as we all have, haha.


Document Dashboard UI controls related to release pinning policy

We currently don't have docs that explain the policy regarding our recently deployed UI update that allows pinning releases through the dashboard.

[pdcastro] This issue has attached support thread https://jel.ly.fish/#/support-thread~697ac73f-044e-4e6f-b8bc-483b24174a32

[lucianbuzzo] This issue has attached support thread https://jel.ly.fish/#/support-thread~d696d5c4-ec5c-4b50-9683-d375d79a1aa1


Alpine doesn't work with vmType: vz on macOS 13.2.1 / aarch64 (anymore?)

I know this used to work for me, but now I can no longer use Alpine with the vz vmType. I observed this in the current 0.15 release, so I went back to 0.14.2 and 0.14.1, and all fail the same way.
I know this used to work on the same machine before, so I wonder if this is due to a macOS update (I may have been on 13.1 the last time I tested).
Failure looks like this (all I'm doing is adding a line with vmType: vz to the template):
$ l --debug start template://alpine
DEBU[0000] interpreting argument "template://alpine" as a template name "alpine"
? Creating an instance "alpine" Open an editor to review or modify the current configuration
DEBU[0003] opening editor "/opt/homebrew/bin/emacsclient" for a file "/var/folders/h6/d884sl8915j_xt2fs42bztb00000gn/T/lima-editor-1708077455"
Waiting for Emacs...
WARN[0014] `vmType: vz` is experimental
DEBU[0014] Make sure "shared" network is stopped
DEBU[0014] Make sure "bridged" network is stopped
DEBU[0014] Make sure "host" network is stopped
INFO[0014] Attempting to download the image from "https://github.com/lima-vm/alpine-lima/releases/download/v0.2.27/alpine-lima-std-3.17.0-aarch64.iso"  digest="sha512:b8028c96385ea5be499e37c142cb9e9d9c861f2647ff7f8cceaf867961ef8878e34a8ba0e8cfe2e01ac082191048e7acc30534782b13c5ec4bea53dad5919c33"
DEBU[0014] file "/Users/jan/.lima/alpine/basedisk" is cached as "/Users/jan/Library/Caches/lima/download/by-url-sha256/c53b1c3f6137c7378edfd6d113f3a63f758b8af6090d8611445331f907966aac/data"
DEBU[0014] Comparing digest "sha512:b8028c96385ea5be499e37c142cb9e9d9c861f2647ff7f8cceaf867961ef8878e34a8ba0e8cfe2e01ac082191048e7acc30534782b13c5ec4bea53dad5919c33" with the cached digest file "/Users/jan/Library/Caches/lima/download/by-url-sha256/c53b1c3f6137c7378edfd6d113f3a63f758b8af6090d8611445331f907966aac/sha512.digest", not computing the actual digest of "/Users/jan/Library/Caches/lima/download/by-url-sha256/c53b1c3f6137c7378edfd6d113f3a63f758b8af6090d8611445331f907966aac/data"
DEBU[0014] res.ValidatedDigest=true
INFO[0014] Using cache "/Users/jan/Library/Caches/lima/download/by-url-sha256/c53b1c3f6137c7378edfd6d113f3a63f758b8af6090d8611445331f907966aac/data"
DEBU[0014] [hostagent] Creating iso file /Users/jan/.lima/alpine/cidata.iso
DEBU[0014] [hostagent] Using /var/folders/h6/d884sl8915j_xt2fs42bztb00000gn/T/diskfs_iso1838456468 as workspace
DEBU[0014] [hostagent] Failed to detect CPU features. Assuming that AES acceleration is available on this Apple silicon.
DEBU[0014] [hostagent] OpenSSH version 9.0.1 detected
DEBU[0014] [hostagent] AES accelerator seems available, prioritizing<EMAIL_ADDRESS>and<EMAIL_ADDRESS>INFO[0014] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/jan/.lima/alpine/serial.log")
DEBU[0014] [hostagent] Start udp server listening on: <IP_ADDRESS>:49855
DEBU[0014] [hostagent] Start tcp server listening on: <IP_ADDRESS>:50037
DEBU[0015] [hostagent] Using search domains: [home]
DEBU[0015] [hostagent] [VZ] - vm state change: "VirtualMachineStateStarting"
DEBU[0015] received an event                             event="{2023-03-01 17:23:29.62845 -0800 PST {false false false [] 50036}}"
INFO[0015] SSH Local Port: 50036
INFO[0015] [hostagent] new connection from  to
INFO[0015] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
INFO[0015] [hostagent] [VZ] - vm state change: running
DEBU[0015] [hostagent] executing script "ssh"
DEBU[0015] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/jan/.lima/_config/user" -o<EMAIL_ADDRESS>-o<EMAIL_ADDRESS>-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm<EMAIL_ADDRESS>-o User=jan -o ControlMaster=auto -o ControlPath="/Users/jan/.lima/alpine/ssh.sock" -o ControlPersist=5m -p 50036 <IP_ADDRESS> -- /bin/bash]
INFO[0018] [hostagent] 2023/03/01 17:23:32 tcpproxy: for incoming conn <IP_ADDRESS>:50038, error dialing "<IP_ADDRESS>:22": connect tcp <IP_ADDRESS>:22: no route to host
DEBU[0018] [hostagent] stdout="", stderr="kex_exchange_identification: read: Connection reset by peer\r\nConnection reset by <IP_ADDRESS> port 50036\r\n", err=failed to execute script "ssh": stdout="", stderr="kex_exchange_identification: read: Connection reset by peer\r\nConnection reset by <IP_ADDRESS> port 50036\r\n": exit status 255
INFO[0028] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
DEBU[0028] [hostagent] executing script "ssh"
DEBU[0028] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/jan/.lima/_config/user" -o<EMAIL_ADDRESS>-o<EMAIL_ADDRESS>-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm<EMAIL_ADDRESS>-o User=jan -o ControlMaster=auto -o ControlPath="/Users/jan/.lima/alpine/ssh.sock" -o ControlPersist=5m -p 50036 <IP_ADDRESS> -- /bin/bash]
INFO[0031] [hostagent] 2023/03/01 17:23:45 tcpproxy: for incoming conn <IP_ADDRESS>:50039, error dialing "<IP_ADDRESS>:22": connect tcp <IP_ADDRESS>:22: no route to host
DEBU[0031] [hostagent] stdout="", stderr="kex_exchange_identification: read: Connection reset by peer\r\nConnection reset by <IP_ADDRESS> port 50036\r\n", err=failed to execute script "ssh": stdout="", stderr="kex_exchange_identification: read: Connection reset by peer\r\nConnection reset by <IP_ADDRESS> port 50036\r\n": exit status 255
INFO[0041] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
DEBU[0041] [hostagent] executing script "ssh"
DEBU[0041] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/jan/.lima/_config/user" -o<EMAIL_ADDRESS>-o<EMAIL_ADDRESS>-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm<EMAIL_ADDRESS>-o User=jan -o ControlMaster=auto -o ControlPath="/Users/jan/.lima/alpine/ssh.sock" -o ControlPersist=5m -p 50036 <IP_ADDRESS> -- /bin/bash]
^C

Can anybody else repro?
Any thoughts on how to debug this, given we have no video support?
$ ssh -o 'IdentityFile="/Users/jan/.lima/_config/user"' -o<EMAIL_ADDRESS>-o<EMAIL_ADDRESS>-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o 'Ciphers="^aes128-gcm<EMAIL_ADDRESS>-o User=jan -o ControlMaster=auto -o 'ControlPath="/Users/jan/.lima/alpine/ssh.sock"' -o ControlPersist=5m -o Hostname=<IP_ADDRESS> -o Port=49360 lima-alpine
kex_exchange_identification: read: Connection reset by peer
Connection reset by <IP_ADDRESS> port 49360

There doesn't seem to be anything helpful in ha.stderr.log:
$ cat ~/.lima/alpine/ha.stderr.log
{"level":"debug","msg":"Creating iso file /Users/jan/.lima/alpine/cidata.iso","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"Using /var/folders/h6/d884sl8915j_xt2fs42bztb00000gn/T/diskfs_iso1838456468 as workspace","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"Failed to detect CPU features. Assuming that AES acceleration is available on this Apple silicon.","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"OpenSSH version 9.0.1 detected","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing<EMAIL_ADDRESS>and aes256-gcm@openssh.com","time":"2023-03-01T17:23:29-08:00"}
{"level":"info","msg":"Starting VZ (hint: to watch the boot progress, see \"/Users/jan/.lima/alpine/serial.log\")","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"Start udp server listening on: <IP_ADDRESS>:49855","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"Start tcp server listening on: <IP_ADDRESS>:50037","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"Using search domains: [home]","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"[VZ] - vm state change: \"VirtualMachineStateStarting\"","time":"2023-03-01T17:23:29-08:00"}
{"level":"info","msg":"new connection from  to ","time":"2023-03-01T17:23:29-08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2023-03-01T17:23:29-08:00"}
{"level":"info","msg":"[VZ] - vm state change: running","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2023-03-01T17:23:29-08:00"}
{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/jan/.lima/_config/user\" -o<EMAIL_ADDRESS>-o<EMAIL_ADDRESS>-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm<EMAIL_ADDRESS>-o User=jan -o ControlMaster=auto -o ControlPath=\"/Users/jan/.lima/alpine/ssh.sock\" -o ControlPersist=5m -p 50036 <IP_ADDRESS> -- /bin/bash]","time":"2023-03-01T17:23:29-08:00"}
2023/03/01 17:23:32 tcpproxy: for incoming conn <IP_ADDRESS>:50038, error dialing "<IP_ADDRESS>:22": connect tcp <IP_ADDRESS>:22: no route to host
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\nConnection reset by <IP_ADDRESS> port 50036\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\nConnection reset by <IP_ADDRESS> port 50036\\r\\n\": exit status 255","time":"2023-03-01T17:23:32-08:00"}
[...]

Both experimental/vz and default (with added vmType: vz) work fine on the same machine with the same Lima version.

Just verified alpine alpine-lima-std-3.16.0-aarch64.iso is working fine. Its not working with latest alpine image.
So i guess it is not related to macOS version
Will check further and update

Thank you! I can confirm that it works for me with
- location: "https://github.com/lima-vm/alpine-lima/releases/download/v0.2.23/alpine-lima-std-3.16.0-aarch64.iso"
  arch: "aarch64"
  digest: "sha512:f55eb91500b5665eb2000ee71083d3912a2284585bb8f0be5488c805e0527db700b76e823c4d4560407077a4cace566a883a78a7fb7412789d4f8a4a7094f519"

So indeed it is not the macOS version. I thought I had tested with at least 0.14.1, but that already uses an Alpine 3.17 image.

Interestingly 3.17 alpine image is working fine in my intel machine but not on my M1.

On M1, VM stuck with Booting Linux Virt

I suspect its related to https://gitlab.alpinelinux.org/alpine/aports/-/issues/14407
But here no crashes as well just stuck on booting
given we have no video support
We can enable this but vz requires

to run on main thread
it will will be a blocking call
we should use runtimer.LockOSThread()
Doing from go routine crashes the application.

I have debugged this by enabling video support. If interested to try refer here
https://github.com/lima-vm/lima/compare/master...balajiv113:lima:display

I suspect its related to https://gitlab.alpinelinux.org/alpine/aports/-/issues/14407

That looks very likely; unfortunately there is little progress on this issue in the last 2 months. Since the reporter mentions that switching to GRUB from AlmaLinux made it work, I wonder if that is an option for us. I'm not sure how to do this ISO surgery though; any ideas?
I tried your display branch, and it works for me, in that I get the same result. Is there any way to pass additional boot parameters when using UEFI boot? I would like to add console=hvc0 to hopefully see the console, to see if we get the same failures as in the Gitlab issue.
I also tried to boot the image with UTM, but also without success (it does boot our 3.16 image). I didn't find a way to view the console while the system was booting either.

Is there any way to pass additional boot parameters when using UEFI boot
On grub window, you can press e to change the boot parameters. But even after adding console=hvc0 no change for me. No panic errors shown

Able to reproduce that similar panic logs with UTM
Steps

Added a tty serial in vm configuration
Start the VM, select e and add console=hvc0 at last
Press F10
In UTM, vm screen they show tty value
Do screen /dev/ttys006 and notice there is a panic

I don't see the tty value, but I see the panic directly in the UTM window (cannot select & copy, just screenshot):

Which looks identical to the stack trace in https://gitlab.alpinelinux.org/alpine/aports/-/issues/14407

I remember having issues with alpine 3.17 a while back and had to revert to 3.16 https://github.com/abiosoft/alpine-lima/commit/320e3d5348ba0e48aa564265cbe5d632de803480.

Working in macOS 13.3 beta.
Looks like the issue is with macOS EFI loader logic only.

@balajiv113 given the UI call is blocking, how did you get your startup sequence to continue? I get a black window and nothing continues from there.

Checking the GUI output with that image, the VM seems to start, but SSH fails to connect.
INFO[0081] [hostagent] 2023/03/31 12:04:11 tcpproxy: for incoming conn <IP_ADDRESS>:61411, error dialing "<IP_ADDRESS>:22": connect tcp <IP_ADDRESS>:22: connection was refused

I've tested the Alpine 3.18 ISO with vmType=vz on 13.1 and 13.4. It does not boot on 13.1, but seems to work fine on 13.4.
This seems to be due to Apple changes in 13.3, but I no longer have a 13.3 machine for testing, and nested virtualization of macOS doesn't seem to be supported by the Virtualization framework.
Not sure if we want to keep this issue open, just in case Alpine is changing the boot process. Or maybe we just close the issue and add a comment to the alpine.lima file denoting the requirement to run on 13.4 or later (I don't want to claim 13.3 unless somebody tests it).

@jandubois i do have 13.3 i will do a check today.
I have already verified in 13.3 beta but will do a recheck again

add a comment to the alpine.lima file denoting the requirement to run on 13.4 or later

SGTM

Verified in 13.3 as well

#1564 adds the comment to alpine.yaml.


PDCL-6567 Removed deprecation warning for result.decisions.

Description
Removed deprecation warning when accessing result.decisions.

Related Issue
https://jira.corp.adobe.com/browse/PDCL-6567

Motivation and Context
The SDK itself was accessing result.decisions, causing a warning to be logged in the console for customers who were not accessing result.decisions. This is fixable, but after discussions with the team, we've decided to remove the warning entirely.

Screenshots (if appropriate):
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Improvement (non-breaking change which does not add functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA or I'm an Adobe employee.
[x] I have made any necessary test changes and all tests pass.
[x] I have run the Sandbox successfully.

I believe so.


Different spacing behaviour with wrap layout between mobile and desktop

Steps to Reproduce

Run flutter create bug.
Update the files as follows:

create main.dart like this

complete code sample
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'app demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'app demo'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _purpose = 0;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Wrap(
        direction: Axis.horizontal,
        spacing: 10,
        children: [
          ChoiceChip(
            label: Text("purpose1"),
            selected: _purpose == 0,
            onSelected: (isSelected) {
              if (isSelected) {
                setState(() {
                  _purpose = 0;
                });
              }
            },
          ),
          ChoiceChip(
            label: Text("purpose2"),
            selected: _purpose == 1,
            onSelected: (isSelected) {
              if (isSelected) {
                setState(() {
                  _purpose = 1;
                });
              }
            },
          ),
          ChoiceChip(
            label: Text("purpose3"),
            selected: _purpose == 2,
            onSelected: (isSelected) {
              if (isSelected) {
                setState(() {
                  _purpose = 2;
                });
              }
            },
          ),
          ChoiceChip(
            label: Text("purpose4"),
            selected: _purpose == 3,
            onSelected: (isSelected) {
              if (isSelected) {
                setState(() {
                  _purpose = 3;
                });
              }
            },
          ),
          ChoiceChip(
            label: Text("purpose5"),
            selected: _purpose == 4,
            onSelected: (isSelected) {
              if (isSelected) {
                setState(() {
                  _purpose = 4;
                });
              }
            },
          ),
        ],
      ),
    );
  }
}

Run the app on an Android simulator and web server(I use Opera and Safari).

Expected results:
The same UI on Android simulator and web browser
Actual results:
Web browser is on the left. Android simulator is on the right.

Logs

[✓] Flutter (Channel master, 1.23.0-8.0.pre.253, on Mac OS X 10.14.4 18E226 x86_64, locale zh-Hans-CN)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[!] Xcode - develop for iOS and macOS (Xcode 7.3)
    ✗ Flutter requires a minimum Xcode version of 11.0.0.
      Download the latest version or update via the Mac App Store.
    ✗ CocoaPods not installed.
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your
        plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To install:
        sudo gem install cocoapods
[✗] Chrome - develop for the web (Cannot find Chrome executable at /Applications/Google
    Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.
[✓] Android Studio (version 4.0)
[✓] IntelliJ IDEA Community Edition (version 2019.3.5)
[✓] Connected device (2 available)

This is probably due to using a different MaterialTapTargetSize

https://github.com/flutter/flutter/blob/master/packages/flutter/lib/src/material/chip.dart#L1928

Hi @yankai-victor
I tried to track the commits to find this regression on the master channel

My guess is regression happened here https://github.com/flutter/flutter/issues/66754
Issue on  commit https://github.com/flutter/flutter/commit/9efdcf61d2eaa7d7126b8e15a23ce097c28449f5

No Issue on  commit https://github.com/flutter/flutter/commit/3063182e7775e9824a33e41b535961a30400f70c

flutter doctor -v
[✓] Flutter (Channel unknown, 1.23.0-8.0.pre.42, on Mac OS X 10.15.7 19H2
    x86_64, locale en-GB)
    • Flutter version 1.23.0-8.0.pre.42 at /Users/tahatesser/Code/flutter_master
    • Framework revision 9efdcf61d2 (2 weeks ago), 2020-09-28 18:22:02 -0400
    • Engine revision 77701d303e
    • Dart version 2.11.0 (build 2.11.0-174.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/tahatesser/Code/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/tahatesser/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    • CocoaPods version 1.10.0.rc.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 50.0.1
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.50.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.15.0

[✓] Connected device (4 available)
    • Taha’s iPhone (mobile) • 00008020-001059882212002E • ios            • iOS
      14.0.1
    • macOS (desktop)        • macos                     • darwin-x64     • Mac
      OS X 10.15.7 19H2 x86_64
    • Web Server (web)       • web-server                • web-javascript •
      Flutter Tools
    • Chrome (web)           • chrome                    • web-javascript •
      Google Chrome 85.0.4183.121

• No issues found!

@gspencergoog , @goderbauer density seems wrong for desktop ?
Not sure if TapTargetSize should be set lower but not zero for desktop.
Otherwise, users will have to compensate in all multiplat apps.

Density will change on the web based on the platform that it thinks it is running on: there is code to check the browser and see if it's running on desktop or mobile, and pick different densities by default based on that.
However, the density isn't the problem here, it's the change in shrink-wrapping behavior of the touch targets for desktop platforms. I think that part is WAI, except that [Chip]s are a little odd in that they don't have any intrinsic padding, because they often need to be presenting in a very compact way. Perhaps Chips should keep their touch target size a minimum of 2dp outside of their bounding box instead of shrink wrapping. @willlarche from a Material perspective, what do you think the guidance should be?

@willlarche, any guidance on this? Is 2dp a good solution? Otherwise every chip may need to be wrapped inside a padding to make your app multiplat.

Could you elaborate on the proposed solution? Is it altering behavior of shrinkWrap for chips, or changing the default?
Or could it be to add 2dp of padding to the child of _ChipRedirectingHitDetectionWidget when the MaterialTapTargetSize is shrinkWrap?: https://github.com/flutter/flutter/blob/8874f21e79d7ec66d0457c7ab338348e31b17f1d/packages/flutter/lib/src/material/chip.dart#L1941

@clocksmith , its about default, is guidance using 2dp?

@ferhatb The guidance is that there should be 8dp above and below the chip top and chip bottom

@ferhatb The guidance is that there should be 8dp above and below the chip top and chip bottom

@gspencergoog given guidance above , does the shrinkwrap default for desktop need to be changed?

@gspencergoog given guidance above , does the shrinkwrap default for desktop need to be changed?

Yes, probably so.

Yes, probably so.

I've just encountered this as well 😊
Wrap(   spacing: 4,   runSpacing: 4,   children: List<Widget>.generate(arrLength, (int index) {     final item = arr[index];     return ChoiceChip(       shape: RoundedRectangleBorder(),       label: Text(item.name),       selected: _selectedItemId == item.id,       onSelected: (bool selected) {         setState(() {           _selectedItemId = item.id;         });       },     );   }), ),
I'm actually only developing my app for the 'Web' & I debug the app in Chrome, then do 'flutter build web' => firebase deploy :)
When I open the website on my PC, the spacing is fine, but when I open the same link on my Android device, the spacing is .. well, it looks at least 4x the size :D

I'm encountering this issue as well. The spacing on mobile is too much.

found same issue as well

Any progress about this question? The bug has been here for almost one year now.

Not sure it is a wrong hunch but I'm going to risk it nonetheless.
My workaround was something like the below:
@override
  Widget build(BuildContext context) {
    return Container(
        padding: EdgeInsets.symmetric(
          /// Only necessary because [Wrap] is misbehaving between platforms.
          // See: https://github.com/flutter/flutter/issues/67797.
          // FIXME: monitor the aforementioned issue to remove this workaround.
          vertical: _platform == PlatformType.desktop
              ? verticalPadding
              : verticalPadding - tagPadding,
        ),
        child: Wrap(
          /// Only necessary because [Wrap] is misbehaving between platforms.
          // See: https://github.com/flutter/flutter/issues/67797.
          // FIXME: monitor the aforementioned issue to remove this workaround.
            runSpacing: _platform == PlatformType.desktop
                ? tagPadding
                : -tagPadding,
            spacing: tagPadding,
            children: <Widget>[
              ...tags.map((t) => _buildTagChip(
                  context: context,
                  name: t.name,
                  icon: t.icon,
                  color: t.color
              )),
              _buildTagChip(context: context, name: 'Add tag', isAddTag: true)
            ]
        )
    );
  }

I haven't investigated more thoroughly but, at first glance, it almost seems like the value of spacing is being added to both the padding around the whole Wrap and the runSpacing. Subtracting spacing from both seems to do the trick.
This can turn out to be a misleading coincidence or it may point to the root cause of the problem. Again, I haven't investigated enough to check which.

Desktop vs mobile

Preview

This seems to be working as intended, see https://github.com/flutter/flutter/pull/108436

@TahaTesser what was the reason for reopening this? (Desktop Triage)

@gspencergoog
It seems MaterialTapTargetSize is not valid for clicks? https://github.com/flutter/flutter/pull/108436#issuecomment-1197221499

@TahaTesser Can you close this and file another bug that describes the remaining issue, if any?  It sounds like the original reported issue here should be closed as "working as intended", but that perhaps there is a change that could help make it easier to use?

I will close this for WAI (working as intended).
AFAIK, we can set density for chips, if you want spacings for desktop as well.
If there is a specific proposal to make it easier, please file a new issue.


Graphical Interpolation is weird

Review the way that interpolation takes place for graphical frequency functions. While some discontinuity is expected, what we see in the image is beyond what is expected. In particular, we're concerned about the flat zones that are followed by jumps.

This effort could take 32hrs.

this was resolved in a dedicated PR by re-arranging the algorithm


Bump @babel/traverse from 7.21.4 to 7.23.2 (#120)

Updated the professor card (All the data syncs up still; nothing is hard coded except for course review).
Note: We still need to add the percentage of people who completed a course review.

See #137


Need a server instance

Create EC2 instance with elastic IP address
Install<EMAIL_ADDRESS>Install and configure nginx
Create certificate for the domain name PICO.picolabs.io
Make a child pico of the root pico to represent the entire network

I would recommend the instructions I created for the pico engine I'm hosting on EC2. It's working well.

Let me know when you need the domain name.

The instructions are Pico Engine on EC2, and useful. Particularly the use of yum to keep the instance up to date.
I've decided to clone the existing instance, to keep everyone who opted in to byname as a founding member of the PLAN. So, first step is to remove those who had been loaded in as part of the fully-sharded database experiment.

The EC2 instance has been created (free tier for now at least) and associated with the elastic IP address <IP_ADDRESS> to which Phil will soon point the domain name PLAN.picolabs.io.
I have copied two of the HTML files in the docs folder to .nvm/versions/node/v16.20.1/lib/node_modules/pico-engine/public/ so the the pico engine will serve them up, once port 80 (upgraded to post 443) reverse proxies to port 3000.

Done. plan.picolabs.io. -> <IP_ADDRESS>


Static analysis to find unnecessary locations

I don't know why this didn't work last time. Appears to be issues with CI.
@sjoerdvisscher any clue?
Pre-submit checklist:

Branch

[x] Tests are provided (if possible)
[x] Commit sequence broadly makes sense
[x] Key commits have useful messages
[x] Relevant tickets are mentioned in commit messages
[x] Formatting, materialized Nix files, PNG optimization, etc. are updated

PR

[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

If you try running nix-build -A plutus-apps.haskell.packages.plutus-use-cases.checks.plutus-use-cases-test locally you'll see that the tests fail.
I think that it's probably because the plutus commit hash has not been update in the sha256map.nix file.

If you try running nix-build -A plutus-apps.haskell.packages.plutus-use-cases.checks.plutus-use-cases-test locally you'll see that the tests fail.
I think that it's probably because the plutus commit hash has not been update in the sha256map.nix file.

Oups sorry, seems like you updated the commit hash there as well.

Everything works fine when running tests with cabal test plutus-use-cases.
However, when running the tests with Nix using: nix-build -A plutus-apps.haskell.packages.plutus-use-cases.checks.plutus-use-cases-test, I'm getting error with the golden files.
multisig
    2 out of 5:                                                                    OK (0.06s)
    3 out of 5:                                                                    OK (0.07s)
    PIR:                                                                           FAIL
      Test output was different from 'test/Spec/multisig.pir'. It was:
      (program
        (let
          (nonrec)
          (datatypebind
            (datatype
              (tyvardecl Monoid (fun (type) (type)))
              (tyvardecl a (type))
              Monoid_match
              (vardecl
                CConsMonoid
                (fun [ (lam a (type) (fun a (fun a a))) a ] (fun a [ Monoid a ]))
              )
            )
          )
          (termbind
            (strict)
            (vardecl
              p1Monoid
              (all a (type) (fun [ Monoid a ] [ (lam a (type) (fun a (fun a a))) a ]))
            )
            (abs
              a
              (type)
              (lam
                v
                [ Monoid a ]
                [
                  { [ { Monoid_match a } v ] [ (lam a (type) (fun a (fun a a))) a ] }
                  (lam v [ (lam a (type) (fun a (fun a a))) a ] (lam v a v))
                ]
              )
            )
          )
          (termbind
            (strict)
            (vardecl mempty (all a (type) (fun [ Monoid a ] a)))
            (abs
              a
              (type)
              (lam
                v
                [ Monoid a ]
                [
                  { [ { Monoid_match a } v ] a }
                  (lam v [ (lam a (type) (fun a<truncated>
      Use --accept or increase --size-cutoff to see full output.
      Use -p '/multisig.PIR/' to rerun this test only.

@michaelpj Seems like something on the plutus side no?

@koslambrou I can't reproduce this. But I get:
building
use cases
  crowdfunding
    Expose 'contribute' and 'scheduleCollection' endpoints:                        OK
    make contribution:                                                             OK (0.02s)
    make contributions and collect:                                                OK (0.14s)
    cannot collect money too late:                                                 OK (0.13s)
    cannot collect unless notified:                                                OK (0.13s)
    can claim a refund:                                                            OK (0.09s)
    PIR:                                                                           OK
    script size is reasonable:                                                     OK
      Script size: 3533
    renders the log of a single contract instance sensibly:                        OK (0.14s)
    renders the emulator log sensibly:                                             OK (0.11s)
    renders an error sensibly:                                                     OK
    QuickCheck ContractModel:                                                      OK (0.10s)
      +++ OK, passed 10 tests.

      Actions (19 in total):
      74% CContribute
      16% CStart
      11% WaitUntil

      Wait interval (2 in total):
      100% <10

      Wait until (2 in total):
      50% 10-19
      50% <10
  vesting
    secure some funds with the vesting script:                                     OK (0.02s)
    retrieve some funds:                                                           OK (0.03s)
    cannot retrieve more than allowed:                                             OK (0.01s)
    can retrieve everything at the end:                                            OK (0.05s)
    PIR:                                                                           OK (0.04s)
    script size is reasonable:                                                     OK
      Script size: 4661
    prop_Vesting:                                                                  OK (1.43s)
      +++ OK, passed 20 tests.

      Actions (87 in total):
      43% WaitUntil
      34% Vest
      23% Retrieve

      Actions rejected by precondition (221 in total):
      54.3% Retrieve
      45.7% Vest

      Wait interval (37 in total):
      62% <10
      27% 10-19
       8% 20-29
       3% 30-39

      Wait until (37 in total):
      35% <10
      14% 10-19
      14% 20-29
      11% 30-39
       8% 100-199
       5% 50-59
       3% 40-49
       3% 60-69
       3% 70-79
       3% 80-89
       3% 90-99
    prop_CheckNoLockedFundsProof:                                                  OK (7.71s)
      +++ OK, passed 20 tests.

      Actions (452 in total):
      49.6% WaitUntil
      32.7% Retrieve
      13.7% Vest
       4.0% Unilateral

      Wait interval (224 in total):
      48.2% <10
      34.8% 10-19
      15.2% 20-29
       1.8% 30-39

      Wait until (224 in total):
      21.0% 100-199
      17.4% 10-19
      15.6% 20-29
      10.3% <10
       8.9% 30-39
       6.2% 60-69
       4.9% 80-89
       4.5% 40-49
       4.0% 90-99
       3.1% 70-79
       2.2% 50-59
       1.8% 200-299
  error handling
    throw an error:                                                                OK
    catch an error:                                                                OK
  futures
    setup tokens:                                                                  OK (0.05s)
    can initialise and obtain tokens:                                              OK (0.21s)
    can increase margin:                                                           OK (0.30s)
    can settle early:                                                              OK (0.20s)
    can pay out:                                                                   OK (0.24s)
    PIR:                                                                           OK (0.02s)
    script size is reasonable:                                                     OK
      Script size: 9313
  multisig
    2 out of 5:                                                                    OK (0.03s)
    3 out of 5:                                                                    OK (0.04s)
    PIR:                                                                           OK
  multi sig state machine tests
    lock, propose, sign 3x, pay - SUCCESS:                                         OK (0.23s)
    lock, propose, sign 2x, pay - FAILURE:                                         OK (0.17s)
    lock, propose, sign 3x, pay x2 - SUCCESS:                                      OK (0.44s)
    lock, propose, sign 3x, pay x3 - FAILURE:                                      OK (0.48s)
    PIR:                                                                           OK (0.06s)
    script size is reasonable:                                                     OK
      Script size: 8364
  currency
    can create a new currency:                                                     OK (0.05s)
    script size is reasonable:                                                     OK (0.01s)
  pubkey
    works like a public key output:                                                OK (0.03s)
  escrow
    can pay:                                                                       OK (0.02s)
    can redeem:                                                                    OK (0.12s)
    can redeem even if more money than required has been paid in:                  OK (0.11s)
    can refund:                                                                    OK (0.08s)
    script size is reasonable:                                                     OK (0.04s)
      Script size: 5166 (0.04s)
    QuickCheck ContractModel:                                                      OK (0.77s)
      +++ OK, passed 10 tests:
      50% Redeemable
      30% Contains Redeem

      Actions (42 in total):
      50% Pay
      21% BadRefund
      19% WaitUntil
       10% Redeem

      Actions rejected by precondition (22 in total):
      77% Redeem
       9% BadRefund
       9% Refund
       5% Pay

      Bad refund attempts (9 in total):
      89% steal refund
      11% early refund

      Wait interval (8 in total):
      62% <10
      25% 10-19
      12% 20-29

      Wait until (8 in total):
      50% 10-19
      12% 20-29
      12% 30-39
      12% 50-59
      12% <10
    QuickCheck NoLockedFunds:                                                      OK (2.86s)
      +++ OK, passed 10 tests:
      80% Redeemable
      60% Contains Redeem

      Actions (162 in total):
      43.8% Pay
      20.4% WaitUntil
      13.6% Refund
       9.9% Redeem
       6.8% Unilateral
       5.6% BadRefund

      Bad refund attempts (9 in total):
      78% steal refund
      22% early refund

      Wait interval (33 in total):
      45% 30-39
      36% <10
      18% 20-29

      Wait until (33 in total):
      64% 40-49
      24% 10-19
      12% <10
  simple-escrow
    can lock some value in the contract:                                           OK
    can lock and redeem:                                                           OK (0.02s)
    can lock and refund:                                                           OK (0.06s)
    only locking wallet can request refund:                                        OK (0.05s)
    can't redeem if you can't pay:                                                 OK
  game with secret arguments tests
    run a successful game trace:                                                   OK (0.03s)
    run a failed trace:                                                            OK (0.03s)
    PIR:                                                                           OK
    script size is reasonable:                                                     OK
      Script size: 2301
  game state machine with secret arguments tests
    run a successful game trace:                                                   OK (0.21s)
    run a 2nd successful game trace:                                               OK (0.25s)
    run a successful game trace where we try to leave 1 Ada in the script address: OK (0.14s)
    run a failed trace:                                                            OK (0.09s)
    PIR:                                                                           OK (0.09s)
    script size is reasonable:                                                     OK
      Script size: 9750
    can always get the funds out:                                                  OK (1.45s)
      +++ OK, passed 10 tests (70% Unlocking funds).

      Actions (50 in total):
      52% Guess
      32% GiveToken
      14% Lock
       2% WaitUntil

      Wait interval (1 in total):
      100% <10

      Wait until (1 in total):
      100% <10
    sanity check the contract model:                                               OK
      +++ OK, failed as expected. Falsified (after 2 tests and 14 shrinks):
      Actions
       [Lock (Wallet 2) "*******" 2000000]
    game state machine crash tolerance:                                            OK (3.31s)
      +++ OK, passed 20 tests.

      Actions (170 in total):
      41.8% GiveToken
      28.2% Guess
      11.2% WaitUntil
       8.8% Lock
       6.5% Crash
       3.5% Restart

      Actions rejected by precondition (28 in total):
      64% Restart
      36% Guess

      Wait interval (19 in total):
      84% <10
      16% 10-19

      Wait until (19 in total):
      37% 10-19
      37% <10
      21% 20-29
       5% 40-49
  showBlockchain
    renders a crowdfunding scenario sensibly:                                      OK (0.11s)
    renders a game guess scenario sensibly:                                        FAIL (0.23s)
      Test output was different from 'test/Spec/renderGuess.txt'. It was:
      ==== Slot #0, Tx #0 ====
      TxId:       ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958
      Fee:        -
      Mint:       Ada:      Lovelace:<PHONE_NUMBER>
      Signatures  -
      Inputs:

      Outputs:
        ---- Output 0 ----
        Destination:  PaymentPubKeyHash: 2e0ad60c3207248cecd47dbde3d752e0aad141d6... (Wallet c30efb78b4e272685c1f9f0c93
787fd4b6743154)
        Value:
          Ada:      Lovelace:  10000000

        ---- Output 1 ----
        Destination:  PaymentPubKeyHash: 2e0ad60c3207248cecd47dbde3d752e0aad141d6... (Wallet c30efb78b4e272685c1f9f0c93
787fd4b6743154)
        Value:
          Ada:      Lovelace:  10000000

        ---- Output 2 ----
        Destination:  PaymentPubKeyHash: 2e0ad60c3207248cecd47dbde3d752e0aad141d6... (Wallet c30efb78b4e272685c1f9f0c93
787fd4b6743154)
        Value:
          Ada:      Lovelace:  10000000

        ---- Output 3 ----
        Destination:  PaymentPubKeyHash: 2e0ad60c3207248cecd47dbde3d752e0aad141d6... (Wallet c30efb78b4e272685c1f9f0c93
787fd4b6743154)
        Value:
          Ada:      Lovelace:  10000000

        ---- Output 4 ----
        Desti<truncated>
      Use --accept or increase --size-cutoff to see full output.
      Use -p '/renders a game guess scenario sensibly/' to rerun this test only.
    renders a vesting scenario sensibly:                                           OK (0.06s)
  token account
    Create a token account:                                                        OK (0.10s)
    Pay into the account:                                                          OK (0.07s)
    Transfer & redeem all funds:                                                   OK (0.10s)
  pingpong
    activate endpoints:                                                            OK (0.12s)
    Stop the contract:                                                             OK (0.08s)
  PRISM
    withdraw:                                                                      OK (0.21s)
    QuickCheck property:                                                           OK (1.39s)
      +++ OK, passed 15 tests.

      Actions (45 in total):
      36% Call
      36% Revoke
      24% Issue
       4% WaitUntil

      Wait interval (2 in total):
      100% <10

      Wait until (2 in total):
      50% 10-19
      50% 30-39
  Stablecoin
    mint reservecoins:                                                             OK (0.13s)
    mint reservecoins and stablecoins:                                             OK (0.21s)
    mint reservecoins, stablecoins and redeem stablecoin at a different price:     OK (0.29s)
    Cannot exceed the maximum reserve ratio:                                       OK (0.20s)
  auction
    run an auction:                                                                OK (0.38s)
    run an auction with multiple bids:                                             OK (0.61s)
    QuickCheck property:                                                           OK (2.56s)
      +++ OK, passed 10 tests.

      Actions (31 in total):
      39% Bid
      39% WaitUntil
      23% Init

      Wait interval (12 in total):
      75% 90-99
       8% 10-19
       8% 70-79
       8% <10

      Wait until (12 in total):
      83% 100-199
       8% 20-29
       8% <10
    NLFP fails:                                                                    Expected funds of W[2] to change by
  Value (Map [(,Map [("",-87240078)]),(363d3944282b3d16b239235a112c0f6e2f1195de5067f61c0dfc0f5f,Map [("token",1)])])
but they changed by
  Value (Map [(,Map [("",-89240078)])])
a discrepancy of
  Value (Map [(,Map [("",-2000000)]),(363d3944282b3d16b239235a112c0f6e2f1195de5067f61c0dfc0f5f,Map [("token",-1)])])
Expected funds of W[1] to change by
  Value (Map [(,Map [("",87240078)]),(363d3944282b3d16b239235a112c0f6e2f1195de5067f61c0dfc0f5f,Map [("token",-1)])])
but they changed by
  Value (Map [(,Map [("",-2000000)]),(363d3944282b3d16b239235a112c0f6e2f1195de5067f61c0dfc0f5f,Map [("token",-1)])])
a discrepancy of
  Value (Map [(,Map [("",-89240078)])])
Test failed.
Emulator log:
OK (0.96s)
      +++ OK, failed as expected. Assertion failed (after 2 tests):
      DLScript
        [Do $ Init,
         Do $ Bid (Wallet 2) 89240078]

      The ContractModel's Unilateral behaviour for Wallet 2 does not match the actual behaviour for actions:
      Actions
       [Var 0 := Init,
        Var 1 := Bid (Wallet 2) 89240078,
        Var 2 := Unilateral (Wallet 2),
        Var 3 := WaitUntil (Slot {getSlot = 100})]
    prop_Reactive:                                                                 OK (0.11s)
      +++ OK, passed 1000 tests.
  sealed bid auction
    packInteger is injective:                                                      OK
      +++ OK, passed 100 tests; 12 discarded.
    prop_AuctionModelCorrect:                                                      OK (2.52s)
      +++ OK, passed 20 tests.

      Actions (94 in total):
      40% WaitUntil
      18% Init
      17% Bid
      15% Reveal
       10% Payout

      Actions rejected by precondition (183 in total):
      40.4% Bid
      39.3% Reveal
      20.2% Payout

      Reveals (14 in total):
      71% no bid
      14% dishonest
      14% honest

      Wait interval (38 in total):
      71% <10
       8% 30-39
       8% 50-59
       5% 20-29
       3% 10-19
       3% 40-49
       3% 60-69

      Wait until (38 in total):
      29% 70-79
      21% 60-69
      16% 40-49
      11% 80-89
       8% 30-39
       8% <10
       3% 20-29
       3% 50-59
       3% 90-99
  governance tests
    vote all in favor, 2 rounds - SUCCESS:                                         OK (1.51s)
    vote 60/40, accepted - SUCCESS:                                                OK (0.83s)
    vote 50/50, rejected - SUCCESS:                                                OK (0.81s)
    PIR:                                                                           OK (0.06s)
    script size is reasonable:                                                     Script size: 9246
OK
  uniswap
    can create a liquidity pool and add liquidity:                                 OK (0.52s)
    prop_UniswapAssertions:                                                        OK (0.56s)
      +++ OK, passed 1000 tests.

      Actions (25269 in total):
      19.724% Bad
      15.822% AddLiquidity
      15.810% PerformSwap
      14.718% RemoveLiquidity
      13.515% WaitUntil
       7.978% SetupTokens
       7.745% CreatePool
       3.376% Start
       1.314% ClosePool

      Actions rejected by precondition (8299 in total):
      53.26% ClosePool
      17.04% CreatePool
      15.57% Bad
      14.13% Start

      Bad actions (4984 in total):
      34.95% RemoveLiquidity
      28.71% AddLiquidity
      28.19% PerformSwap
       8.15% BadRemoveLiquidity

      Wait interval (3415 in total):
      31.07% <10
      29.19% 10-19
      25.27% 20-29
      11.95% 30-39
       2.43% 40-49
       0.09% 50-59

      Wait until (3415 in total):
      22.78% 100-199
      14.64% 200-299
       9.75% 300-399
       6.06% 400-499
       4.57% 60-69
       3.95% 500-599
       3.66% 50-59
       3.63% 30-39
       3.57% 40-49
       3.51% 90-99
       3.28% 70-79
       3.28% 80-89
       2.87% 1000-1999
       2.75% 600-699
       2.72% 10-19
       2.69% <10
       2.02% 700-799
       1.55% 20-29
       1.46% 800-899
       1.02% 900-999
       0.20% 2000-2999
    prop_NLFP:                                                                     OK (0.67s)
      +++ OK, passed 250 tests.

Argh. Maybe this is the same issue that we had in plutus: somehow what you did has introduced instability in the compiler output between the Nix and non-Nix builds. We let it slide there but if it's popping up more here that's a problem.

We should probably bump the core dep even further here as we just merged a relevant bugfix there today. I'll get to that next week.

This is now stuck on cached OOM failure.

@koslambrou I can not reproduce these issues on my machine. Does this happen both when you run it through nix and when you just do cabal test plutus-use-cases?

@koslambrou I can not reproduce these issues on my machine. Does this happen both when you run it through nix and when you just do cabal test plutus-use-cases?

Thes issues happen when I run the tests through Nix. Surprisingly, everything works fine when doing cabal test plutus-use-cases.

@michaelpj this seems like a nix issue to me. Any clue what's going on?

Also, can @michaelpj maybe do something about all these cached failures?

@michaelpj this seems like a nix issue to me. Any clue what's going on?

No. I presume it's the same thing that caused this issue on your PR to plutus. I have asked @zliu41 to try and figure out what's going on there, so maybe we'll know something in a bit.

Also, can @michaelpj maybe do something about all these cached failures?

I restarted all the failing jobs, let's see what happens.

Ok, no idea what's up with "wrong ELF type"...

Remaining failures appear to be actual test failures.

@michaelpj see above about these failures. They don't reproduce with cabal test so we can't update the golden tests.

I've bumped the core dependency here to:

Include the latest bugfix
Try to magically fix this weird non-reproducability problem

@michaelpj restart build plx?

Maybe pulling in the latest changes from main to get GHC 8.10.7 might help? My latest PR using that went smoothly, but that could have been a fluke.

@sjoerdvisscher I rebased this morning - has it changed since?
Also, does that solve the OOM issues in hydra?

Also, does that solve the OOM issues in hydra?

Apparently not 😞

@michaelpj could you please restart the failing CI jobs here?

I bumped the dep on core to include some bugfixes, maybe that could magically solve the flaky-inconsistent-PIR-golden-tests issue?

@MaximilianAlgehed I'm getting the same PIR errors as before when running the tests in Nix. Latest commit is d1441914629f509a6eaed678578927c04522e5b7

@koslambrou Ok. I don't know what to make of this issue. Probably it's a good idea to file this as an issue in some issue tracker somewhere and try to get some eyes on it because as far as I can tell this is an issue with the compiler / nix / reproducibility. I've spoken a bit to @michaelpj about it but I don't know what the right bug report to file is, or even where to file it.

We have a ticket for it already and someone is going to look at it.

@michaelpj any progress on this reproducibility issue?

@michaelpj that's commit 6d9ac7c2f89363d574dbc10be5c2db4b661c9a43 no? Which is the one we are on? Am I missing something?

@michaelpj restart hydra plz...

done

@michaelpj PIR reproducibility problem unfortunately wasn't fixed :(

Though this one fails when I run CI locally too! Improvement! But cabal test doesn't give us the failure 🤕

@koslambrou because progress on this reproducibility issue is a bit slow I think the right thing to do is to turn off the tests here, get this merged, and fix things in a separate push later. If this turns green, could you merge this?

@MaximilianAlgehed Yes I agree.

Sorry I just saw this. It's probably a similar issue. I'll open a ticket which hopefully I (or someone else) can work on in the next few days.


Use includes?

As far as I can see, the demos have a lot of repeated shader code.
How about doing this something like this?
<script src="three.js"></script>
<script src="three-pathtracing.js"></script>

three-pathtracing.js would have this inside:
THREE.ShaderChunk[ 'pathtracing_functions' ] = `
//-----------------------------------------------------------------------
float SphereIntersect( float rad, vec3 pos, Ray r )
//-----------------------------------------------------------------------
{
	vec3 op = pos - r.origin;
	float b = dot(op, r.direction);
	float det = b * b - dot(op,op) + rad * rad;
       	if (det < 0.0)
		return INFINITY;

	det = sqrt(det);
	float t1 = b - det;
	if( t1 > 0.0 )
		return t1;

	float t2 = b + det;
	if( t2 > 0.0 )
		return t2;
	return INFINITY;
}
// etc
`;

Then all the demos can include that code by adding this in the glsl:
#include <pathtracing_functions>

Hi @mrdoob ,
What a great idea!  Yes, as the number of demos has grown,  I was thinking that there is a lot of repeated boiler-plate shader code as well as init code in the init() function (the javascript portion located later in the demo .html files).  I didn't want to have to enclose every shader code line in strings "", but from what you posted as an example, maybe I can just turn it into a THREE.ShaderChunk .
This would significantly reduce the demos code length, as well as make the main path tracing renderer more reusable, especially when I make small changes to it.
Thanks for the suggestion, I will try to get a pathtracingRenderer.js file up and running.
-Erich

Hi @mrdoob ,
Just a follow up - I took your suggestion to heart and refactored the pathtracing engine functions using a more modular approach with #includes.
I have updated the Cornell box demo as the first demo to use this new code base:
Cornell Box Demo using #includes
and here is the new source code:
Demo source code
and the file with common pathtracing functions, setup uniforms, and defines:
common pathtracing functions
Thanks again for the suggestion!  It cut down the amount of demo .html code significantly.  Plus, it's nice having the common pathtracing utility functions all located in 1 file.  This will make small changes and maintenance much easier.  Now, I have to convert all the other demo .html files 1 by 1, but the updates will be worth it later on down the line.
p.s. Can't wait for WebGL 2.0 support in three.js (it will allow me to add robust LBVH acceleration structures using bit manipulations and Morton codes in the WebGL shaders, all on the GPU!) :-)
Take care,
-Erich

Looking good!! 👌


File includes are not working on bitbucket

Hey,
We are evaluating writing our doc files with asciidoc instead of markdown.
One feature I am excited about is the include possibility.
In asciidoc I could write:
[source,perl]
----
include::lib/dirlist.pl[lines=9..17]
----

Which gets processed and resolved in the IDEA Asciidoc support, but when viewed on our bitbucket server, it gets translated to:
link:lib/dirlist.pl[]

I learned that for example github also does this, probably due to the so called safe mode:

http://asciidoctor.org/docs/user-manual/#running-asciidoctor-securely

But at least the link should work I guess (meaning I'd like to have the actual link not the text "link").
We are running version 2.3.1 of the plugin.
Could you please suggest a workaround?

Hi @dbalakirev,
I will look into it the next days.

I finally found some time to look into your request.
Unfortunately it is not just changing the security level but additional changes are needed to parse the links and map them to bitbucket resources. This is somehow doable, but is not as as simply as it seemed in the first place.
When you say "I learned that for example github also does this" does this mean that github also doesn't render the include or that it support this (I could also try this but as you obviously tried it, would be great if you can clarify this question.

I meant that to my knowledge includes won't work on github either.
What works there though, is it successfully replaces the include with a link.
From this page, I ended up at this example:
include::include.adoc[]
This include gets translated to a clickable link.

Hi,
I am facing the same issue.
The include feature of ascciiDoc with tags is one of the silver bullet that make me choice asciiDoc over MarkDown.
Unfortunately my knowledge on this part is too low to help fixing this issue.
But as you guess, I will be more than interested if you fix this issue ;-)
Thanks
Yann

Adding to this issue as well as we are affected, but using include:file[tags=myTags] for source code documentation also does not work.
README.adoc
= Include Bug
:toc:

.Full source include
[source, groovy]
----
include::source-code.groovy[]
----

.Tagged source Include
[source, groovy]
----
include::source-code.groovy[tags=code]
----

source-code.groovy
def myVar = 'variable'
// tag::code[]
println(myVar)
println('We are in the tag')
// end::code[]

Would it be possible to change the rendering from client side rendering to server side rendering?

I think where the rendering is done if not relevant for getting the support for includes.
I hope fully have some time in the next days and will look into it again what is possible here.

I now had some time to look into. Here a first overview how different includes are rendered.
include::include/ad.adoc[]
include::{includedir}/ad.adoc[]
are both rendered into a correct HTML link.

That sounds promising. I am happy to give us a try, should you have a release. Of course I guess it will take a couple of days until I get them update the plugin version in our company bitbucket installation.

I tested it with the latest release version actually.

Ah ok. I misunderstood. I thought you actually had to fix something. Then I will test on our side to see if the problem is still around. Thanks for looking into it.

Add-on 2.4.0 and 3.1.0 are now released supporting file includes.


Error !

after   ./install.sh
----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-jwtsck9s/mechanize/
and
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-jwtsck9s/mechanize/
root@kali:~/Spectre# python spectre.py --help
Traceback (most recent call last):
File "spectre.py", line 11, in
from lib.scrape import Scraper
File "/root/Spectre/lib/scrape.py", line 7, in
from dryscrape import Session
ImportError: No module named dryscrape
root@kali:~/Spectre#

Are you using Python 2.7

root@kali:~# python -V
Python 2.7.14+

I will take a look at it when I get the chance. Thanks for letting me know
instead of calling my bad.

hello there !


[autoTVM][BUG] bitserial dense arm_cpu

The bitserial dense operation for ARM CPU's is not auto-tune-able for larger matrix sizes. For small sizes N=64,125,256,512 it works, but for N=768,1024,2048 it crashes with a broadcast shape error. I reproduced this with TVM v0.7 branch and the current head 80ca598e271c1a93eadc26e31e5bef00b8d86899 of main branch.
error message:
list of tasks:
task =  bitserial_dense.arm_cpu
tasks =  [Task(func_name=bitserial_dense.arm_cpu, args=(('TENSOR', (768, 768), 'uint8'), ('TENSOR', (768, 768), 'uint8'), 1, 1, 'uint8', 'int16', 1), kwargs={}, workload=('bitserial_dense.arm_cpu', ('TENSOR', (768, 768), 'uint8'), ('TENSOR', (768, 768), 'uint8'), 1, 1, 'uint8', 'int16', 1))]
Tuning...
[Task  1/ 1]  Current/Best:    0.00/   0.00 GFLOPS | Progress: (0/108) | 0.00 sTraceback (most recent call last):
  File "autotuning.py", line 239, in <module>
    main( sys.argv[1:] )
  File "autotuning.py", line 174, in main
    autotune_bunchOfTinyNets( device_config, botn, timestamp )
  File "autotuning.py", line 91, in autotune_bunchOfTinyNets
    timestamp )
  File "autotuning.py", line 65, in autotune
    autotuner.tune( device_config, ops, network )
  File "./autotuner.py", line 173, in tune
    tune_tasks(tasks, **tuning_opt)
  File "./autotuner.py", line 139, in tune_tasks
    autotvm.callback.log_to_file(tmp_log_file)])
  File "tvm/python/tvm/autotvm/tuner/xgboost_tuner.py", line 103, in tune
    super(XGBTuner, self).tune(*args, **kwargs)
  File "tvm/python/tvm/autotvm/tuner/tuner.py", line 169, in tune
    self.update(inputs, results)
  File "tvm/python/tvm/autotvm/tuner/model_based_tuner.py", line 270, in update
    self.cost_model.fit(self.xs, self.ys, self.plan_size)
  File "tvm/python/tvm/autotvm/tuner/xgboost_cost_model.py", line 184, in fit
    x_train = self._get_feature(xs)
  File "tvm/python/tvm/autotvm/tuner/xgboost_cost_model.py", line 348, in _get_feature
    ret[i, :] = t if t is not None else 0
ValueError: could not broadcast input array from shape (1854) into shape (1278)```

At the first glance it looks like the problem in XGBTuner, so trying RandomTuner might workaround it. Let's move the future discussions to https://discuss.tvm.apache.org/.


Enable sccache hits for local builds

This change passes through the value of SCCACHE_S3_NO_CREDENTIALS to our conda builds, enabling devs to utilize the sccache cache that's populated by CI when they are reproducing build issues locally as per these instructions.

/merge


improved sitemap and robots.txt files

updates & news features from @SamSaffron's comment

There is no built in extensibility to robots.txt generation, this needs to be added so it does not overwrite it completely (instead plugin should inject the instructions). Otherwise, each time we touch robots.txt plugin needs to be updated.
The way it iterates using the object model is inefficient for so many records, instead this should be carefully plucking the data it needs or even possibly using raw sql.
Nothing is cached which is risky, instead ... something should be cached. Perhaps site map should be re-generated daily or something along those lines.
Sitemap is most useful for gigantic sites, it should use a sitemap index and multiple sitemap files (splitting site into 50k chunks)

I would remove, sitemap actually exposes information, robots does not.
Also, if you require login I would not even inject the sitemap into robots,
pointless.
On Thu, Feb 16, 2017 at 5:36 PM, Vinoth Kannan<EMAIL_ADDRESS>wrote:

@vinothkannans commented on this pull request.
In plugin.rb
https://github.com/discoursehosting/discourse-sitemap/pull/1#discussion_r101644831
:

@@ -20,63 +20,80 @@ class Engine < ::Rails::Engine
require_dependency "application_controller"

class DiscourseSitemap::SitemapController < ::ApplicationController

def generate

 prepend_view_path "plugins/discourse-sitemapcrap/app/views/"

 # this code is never called somehow

end
end

layout false
skip_before_filter :preload_json, :check_xhr, :redirect_to_login_if_required

RobotsTxtController
https://github.com/discourse/discourse/blob/master/app/controllers/robots_txt_controller.rb#L3
skipping the redirect_to_login_if_required filter. And sitemap.xml file's
reference included in robots.txt file. So after looking into robots.txt
file automatically bots will be looking for sitemap.xml file I guess.
In this case still I can remove it? I am not sure about how bot things
will work.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/discoursehosting/discourse-sitemap/pull/1#discussion_r101644831,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAUXfsOs0cgf_fabK23U2MBxHVXL4Uuks5rdM9egaJpZM4MC2Nz
.

I would remove, sitemap actually exposes information, robots does not.

Yes. It should be. Very good explanation.

if you require login I would not even inject the sitemap into robots, pointless.

I got it. I will do the implementation like that :+1:

Thanks a million @vinothkannans  and @SamSaffron !


Exclude folders/files from linting

Is your proposal related to a problem?

It would be great to have the ability to exclude some folders/files. For example, being able to run something like
python3 -m curlylint . --exclude some_folder

Describe the solution you’d like

Add an option exclude that will remove folders/files from the linter
Describe alternatives you’ve considered

(Write your answer here.)
Additional context

(Write your answer here.)

Hi @jkevingutierrez, this looks a lot like the existing --exclude. Have you tried that first?