read_csv usecols cannot find columns

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
Modin installed from (source or binary): Using pip install git+https://github.com/modin-project/modin
Modin version: 0.5.2
Python version: 3.6.7
Exact command to reproduce:

import modin.pandas as pdm
import pandas as pd

d = pdm.DataFrame([{'a': 'test'}])
d.to_csv('test.csv', index=False)

# pandas output
u = pd.read_csv('test.csv', usecols=['a'])
print(u)

# modin output
u = pdm.read_csv('test.csv', usecols=['a'])
print(u)

Expected output:
No errors should be raised
Actual output:
ValueError: Usecols do not match columns, columns expected but not found: ['a']
The usecol argument would be nice to have because it makes reading large CSVs even faster.

Hi @mar-muel, thanks for the report!
This is a regression, we will get it resolved for the next release. It should be supported, thank you for the report.
It wasn't caught in the tests because we test for the integer index of the column, not the column name. We will add it so this regression doesn't occur in the future. Thanks again for the report!


Password field on sign up page

The first password field label on the sign up form does not have a translation and is listed as "Password1" instead of "Password"

😬


chore: new atPlatform logo

chore: new atPlatform logo

@gkc looks like this one slipped through the net


Internal Error

-- [hunter] Initializing Hunter workspace (8fe144d9490501d398777532e0c0f56bc7bb0ec7)
-- [hunter]   https://github.com/ruslo/hunter/archive/v0.19.71.tar.gz
-- [hunter]   -> C:/.hunter/_Base/Download/Hunter/0.19.71/8fe144d
The system cannot find the file specified
CMake Error: Generator: execution of make failed. Make command was: "MSBuild.exe" "ALL_BUILD.vcxproj" "/p:Configuration=Debug" "/p:VisualStudioVersion=14.0"
[hunter ** INTERNAL **] Build project failed
[hunter ** INTERNAL **] [Directory:D:/Work/Stuff/CPPEthereum]
------------------------------ WIKI -------------------------------
https://github.com/ruslo/hunter/wiki/error.internal
CMake Error at cmake/HunterGate.cmake:82 (message):
Call Stack (most recent call first):
cmake/HunterGate.cmake:92 (hunter_gate_wiki)
cmake/HunterGate.cmake:346 (hunter_gate_internal_error)
cmake/HunterGate.cmake:499 (hunter_gate_download)
CMakeLists.txt:54 (HunterGate)

Apparently this was caused by me trying to generate VS2015 project files when I have VS2017 (with the VS2015 build tools) installed.


Latest improvements to APIM 3.0.0 under update 2.0

Describe your problem(s)
This ticket is created to track the following issues and will be created updates on new update 2.0 model for the APIM 3.0.0.
https://github.com/wso2/product-apim/issues/9300
https://github.com/wso2/product-apim/issues/9349
https://github.com/wso2/product-apim/issues/8079
https://github.com/wso2/product-apim/issues/9380
https://github.com/wso2/product-apim/issues/9444
https://github.com/wso2/product-apim/issues/9451
https://github.com/wso2/product-apim/issues/9400
https://github.com/wso2/product-apim/issues/9469
https://github.com/wso2/product-apim/issues/9097

Closing the issue since the updates are created.


[tokenizers] Upgrade huggingface tokenizer to 0.13.3

Description
Brief description of what this PR is about

If this change is a backward incompatible change, why must this change be made?
Interesting edge cases to note here

Codecov Report
Patch coverage: 54.72% and project coverage change: +0.07 :tada:

Comparison is base (bb5073f) 72.08% compared to head (1fe852f) 72.16%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2697      +/-   ##
============================================
+ Coverage     72.08%   72.16%   +0.07%
- Complexity     5126     7024    +1898
============================================
  Files           473      698     +225
  Lines         21970    31254    +9284
  Branches       2351     3225     +874
============================================
+ Hits          15838    22553    +6715
- Misses         4925     7165    +2240
- Partials       1207     1536     +329

Impacted Files
Coverage Δ

api/src/main/java/ai/djl/modality/cv/Image.java
69.23% <ø> (-4.11%)
:arrow_down:

...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java
76.00% <ø> (ø)

.../main/java/ai/djl/modality/cv/output/Landmark.java
100.00% <ø> (ø)

...djl/modality/cv/transform/RandomFlipLeftRight.java
25.00% <0.00%> (-25.00%)
:arrow_down:

...djl/modality/cv/transform/RandomFlipTopBottom.java
25.00% <0.00%> (-25.00%)
:arrow_down:

...i/djl/modality/cv/translator/BigGANTranslator.java
21.42% <0.00%> (-5.24%)
:arrow_down:

.../modality/cv/translator/ImageFeatureExtractor.java
0.00% <0.00%> (ø)

.../ai/djl/modality/cv/translator/YoloTranslator.java
27.77% <0.00%> (+18.95%)
:arrow_up:

...ain/java/ai/djl/modality/cv/util/NDImageUtils.java
67.10% <0.00%> (+7.89%)
:arrow_up:

api/src/main/java/ai/djl/modality/nlp/Decoder.java
63.63% <ø> (ø)

... and 226 more

... and 368 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add support for scale value function of dnd5e 1.6.0

Is your feature request related to a problem? Please describe.
No, There is no issue with how it currently functions, this would just be a quality of life improvement.
Describe the solution you'd like
dnd5e 1.6.0 added a function for class advancements to add/change features by class level for characters. The relevant bit is this part right here about scaling values. This allows, for example, adding an identifier for the scale of certain functions, such as @scale.paladin.aura-of-protection for paladin's aura of protection radius.
Describe alternatives you've considered
The current method for achieving this is simply editing the aura's radius in the effect manually upon hitting the level for it to change. Really, this isn't the biggest issue, but this change would be a small quality of life change.
Additional context
An example of the scale value function for aura of protection

yes please!

Renewing the call for this feature. A scale value for Paladin auras is a perfect example.

This is in v0.5.6


feat(tcgc): use @server description for default endpoint client parameter

Clear and concise description of the problem
This is a minor issue.
See cadl-ranch server definition: https://github.com/Azure/cadl-ranch/blob/2fb006d5b4609647138382bf5d248a2edbde7742/packages/cadl-ranch-expect/src/decorators.ts#L241
There is no parameter defined for endpoint, instead it's a hard-coded value.

context.call($server, target, "http://localhost:3000", "TestServer endpoint");

TCGC will give a default endpoint parameter with description Service host. Maybe we should use the description defined in @server? e.g. for this case, it's TestServer endpoint.
Checklist

[X] Follow our Code of Conduct
[X] Check that this issue is about the Azure libraries for typespec. For feature request in the typespec language or core libraries file it in the TypeSpec repo
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Maybe we should differentiate a constant endpoint URL case from an expression endpoint URL case. For endpoint which is an expression, it would be better that each template argument description is undefined? (it doesn't make sense to assign service description to them)
@server(
    "{firstTestTypeSpecUrl}",
    "Endpoint Service",
    {
        firstTestTypeSpecUrl: string,
    }
)

Another example of why we might need to leave uri template parameter as-is:

We have further discussion, and we thought it might be better to just leave description empty if it's not defined. And let each language emitter to decide what they want to add.


Bump deployment target for iOS due to Xcode 14.3 error

Summary

Bumps deployment target for iOS to 11.0
Unblocks building depending SDKs on Xcode versions 14.3 and above

Details
Currently building an SDK that depends on GZIP v1.3.0 is blocked on Xcode versions above 14.3
For the top-level apps using GZIP as a dependency, there is a workaround. It is possible to override the build_settings['IPHONEOS_DEPLOYMENT_TARGET'] in the post_install actions in the Podfile.
Unfortunately, the above-described solution does not work when building an SDK. pod lib lint will fail with an error (not a warning). So this is a complete blocker that makes building impossible. For such cases, it is necessary to update the podspec to specify iOS 11 as the minimum deployment target.

Thank you @ARiezn  and @nicklockwood  !!!!!


Java 17 support?

Are we gonna get Java 17 support for free, or do we need to do something? I tried setting java-version to 17 in v1 and it says:

Error: No valid download found for version 17.x and package jdk. Check https://static.azul.com/zulu/bin/ for a list of valid versions or download your own jdk file and add the jdkFile argument

In the meanwhile, you may resort to download JDK 17 via https://www.oracle.com/java/technologies/jdk-script-friendly-urls/  and use it as an input of setup-java (@sormuras/bach/.github/workflows/ci.yml#L39-L50).

setup-java@v2

      - name: 'Download latest JDK 17'
        run: |
          wget \
            --no-verbose \
            --directory-prefix $RUNNER_TEMP \
            https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.tar.gz
      - name: 'Set up JDK'
        uses: actions/setup-java@v2
        with:
          java-version: 17
          distribution: jdkfile
          jdkFile: ${{ runner.temp }}/jdk-17_linux-x64_bin.tar.gz

setup-java@v1

Same as above without the distribution: jdkfile input line.

Personally, I hope add openjdk and oracle as alternative providers.
And provide early access build of OpenJDK for eager developers.

Personally, I hope add openjdk and oracle as alternative providers.
And provide early access build of OpenJDK for eager developers.

There's 17-ea version of the default jdk just FYI.

17 is GA...

Hello @ice1000. Thank you for your report. It looks like only 17-ea version is available for the provided distributions by the setup-java action. For v1 It's Zulu OpenJDK. For v2 you can see in the Supported distributions section. When stable version becomes available in the API, you'll be able to use it.
For now you can use early access versions or use a solution from @sormuras above.

v1

- name: Setup java
  uses: actions/setup-java@v1
  with:
    java-version: 17-ea

v2

- uses: actions/setup-java@v2
  with:
    java-version: 17-ea
    distribution: adopt # zulu, temurin,

Thank you for the fast response @sormuras !
For now I'm closing the issue. If you have any concerns feel free to ask us.

Cool

Just in case someone needs it for Windows or Mac as well, we use the following snippet:
    strategy:
      matrix:
        java: [ '8', '11', '17' ]
    runs-on: macos-latest
    steps:
      - name: Setup correct Java version
        uses: actions/setup-java@v2
        if: ${{ matrix.java != '17' }}
        with:
          distribution: adopt
          java-version: ${{ matrix.java }}
      - name: Fetch JDK 17
        if: ${{ matrix.java == '17' }}
        run: |
          aria2c -d ${{ runner.temp }} -o openjdk-17_macos-x64_bin.tar.gz https://download.java.net/java/GA/jdk17/0d483333a00540d886896bac774ff48b/35/GPL/openjdk-17_macos-x64_bin.tar.gz
      - name: Setup JDK 17
        uses: actions/setup-java@v2
        if: ${{ matrix.java == '17' }}
        with:
          java-version: ${{ matrix.java }}
          distribution: jdkfile
          jdkFile: ${{ runner.temp }}/openjdk-17_macos-x64_bin.tar.gz

    strategy:
      matrix:
        java: [ '8', '11' , '17' ]
    runs-on: windows-latest
    steps:
      - name: Setup correct Java version
        uses: actions/setup-java@v2
        if: ${{ matrix.java != '17' }}
        with:
          distribution: adopt
          java-version: ${{ matrix.java }}
      - name: Fetch JDK 17
        if: ${{ matrix.java == '17' }}
        run: |
          aria2c -d ${{ runner.temp }} -o openjdk-17_windows-x64_bin.zip https://download.java.net/java/GA/jdk17/0d483333a00540d886896bac774ff48b/35/GPL/openjdk-17_windows-x64_bin.zip
      - name: Setup JDK 17
        uses: actions/setup-java@v2
        if: ${{ matrix.java == '17' }}
        with:
          java-version: ${{ matrix.java }}
          distribution: jdkfile
          jdkFile: ${{ runner.temp }}/openjdk-17_windows-x64_bin.zip

Have zulu released 17 yet?

Have zulu released 17 yet?

Yes. See https://www.azul.com/downloads/?version=java-17-lts&package=jdk for packages and an example PR with it here

zulu 17 is available in Github actions.
Hantsy Bai
Self-employed consultant, fullstack developer, agile coach
GitHub: https://github.com/hantsy
Twitter: @.***
Medium: @.***
On Mon, Sep 20, 2021 at 1:50 PM Zac Sweers @.***> wrote:

Have zulu released 17 yet?
Yes. See https://www.azul.com/downloads/?version=java-17-lts&package=jdk
for packages and an example PR with it here
https://github.com/square/kotlinpoet/pull/1153
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/actions/setup-java/issues/229#issuecomment-922651969,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGQT6AKWAODLEYT3ZELI7DUC3DT5ANCNFSM5EBICJNA
.


Parameters for deployment are poorly documented

The parameters for deployment are almost entirely useless/redundant currently.

All of these are clearly named, but the description just explains the name, not what it is. I know that groupLocation is the location of the Azure resource group, but what does that mean?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9d5bd35d-3178-42cd-2893-3b6a4b7a94ce
Version Independent ID: 1545fd44-d471-d947-f94d-4b5e8d6e853f
Content: Tutorial to create and deploy a basic bot - Bot Service
Content Source: articles/v4sdk/bot-builder-tutorial-basic-deploy.md
Service: bot-service
Sub-service: sdk
GitHub Login: @ivorb
Microsoft Alias: v-ivorb

These are standard Azure parameters that we are using in the CLI for deployment - nothing specific to bots. This topic has a link at the bottom to the terminology  that should help.

Sure, but the job of the documentation isn't to be technically correct, it's to make development easier for developers. Adding explanations would make it easier, even if it is also documented elsewhere. At the very least, a link to that page would go a long way.
Remember: People reading the documentation don't know it as well as your team does. They don't know that they can just go to that other page and find what they need. Anything we can do to help, we should be doing.


Image Modal With Zoom In/Out - jQuery image-popup.js

Are you accessing the project from the GitHub Codespaces?

[ ] No, I am running the project on my local machine (I will fill in the OS section below)
[x] Yes, I am using GitHub Codespaces.

Note: The project is optimized to run on Codespaces. We welcome contributions to enhance it for cross-platform local development, but it's not something we will prioritize right now. Thank you!

Describe the bug
Image Modal With Zoom In/Out - jQuery image-popup.js
Bascially When User Go through Featured Listings , Click into Preview Image , The Image  should be Zoom In/Out feature
To Reproduce
Steps to reproduce the behavior:

Go to Featured Listings
Select any component listed
Trying to click on Image
Haven't any response

Expected behavior
Implementing the Image Modal With Zoom In/Out - jQuery image-popup.js feature
Screenshots
BEFORE :

AFTER :

Please Assign Me in Under Hack Together , Good First Issue So I can work on it
Thank You
Sahil (MLSA )

HI @thestarsahil that is a very nice UI feature however we may want to steer away from additional 3rd party dependencies that can't be tree-shaken or are not modular and that need to be loaded before other JS and therefore may impact performance. Maybe we can take a look at an Angular specific library that is small in size and would help us achieve the same results?

We also suggest this is tackled together with https://github.com/Azure-Samples/contoso-real-estate/issues/253

@anfibiacreativa Thank You For Reviewing 😃

@anfibiacreativa Thank You For Reviewing 😃

You're welcome! Should we assume you will teamup with @john0isaac to fix it together with #253 , or do you prefer trying with an Angular library?

@anfibiacreativa Thank You For Reviewing 😃

You're welcome! Should we assume you will teamup with @john0isaac to fix it together with #253 , or do you prefer trying with an Angular library?

Yes I would love to work together @john0isaac to fix this bug out 🤘🧑💻


Broken Link

There is a broken link on this line - For model-driven apps, see: Supported web browsers and mobile devices linking to https://docs.microsoft.com/dynamics365/customer-engagement/admin/supported-web-browsers-and-mobile-devices

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c8fbf2eb-d0a2-3f2e-651b-bbec8cfe4d2c
Version Independent ID: 0e271da6-8cea-f9d1-376e-2432b6a63759
Content: Run apps in a web browser - Power Apps
Content Source: powerapps-docs/user/run-app-browser.md
Service: powerapps
GitHub Login: @mduelae
Microsoft Alias: mkaur

@pratapladhani thanks for catching this. I have updated the link and republished the topic - which will go live in about an hour.
Thanks,
Mints


Not compatible with latest version of Token Action HUD Core

Module is only compatible with Token Action HUD Core 1.5.X, but current Token Action HUD Core version is 2.0.6

I second that bug report.

@r-valyar
You probably is using the old version. Please remove it and re-install (do not update, need to remove it since the module's id has been changed)

That worked. Thanks!


Package repository seems to need more quality control / consistency checking

I have created a little WinForms Tool which allows the user to browse through all of the (currently more than 400 packages) in a gridview: https://github.com/donid/WingetRepoBrowser
This tool makes it  easy to spot various quality / consistency issues. Some Examples:

VisualStudio packages use 'devenv' or 'devenv.exe' as AppMoniker
Nutstore has a comma separated list as AppMoniker
firefox-esr has an AppMoniker, normal doesn't
all older git packages have an AppMoniker, 2.62.2 doesn't
OneDrive's AppMoniker changes casing between version (and Installer Arch and more)
wireshark's licenseurl is hhttps://...
the version property is not used in the recommended way (Foldit: 20180312-1714f56737-win_x86 / Windscribe: 1.83 Build 18)

I know that I could / should open an issue / PR for each (most) of those problems, but having found so many inconsistencies in just 400 packages hints that something should be changed on a larger scale. I don't know if those checks could be automated (with ML/AI?), or if the manual review process of PRs in the package repo has to be improved.

I don't think you can avoid straying off the recommended version format because software simply doesn't use a consistent numbering system. Some software uses the standard 1.5.2, others use something date related like 2020.1.15 and then you got others that include build numbers and other stuff.

@MarNicGit That's not what donid is saying. He's pointing out obvious inconsistencies in how the manifest field contents are maintained, i.e.:

missing file extensions
field separators
missing fields
case mismatches

And I agree, this should be handled by some automated process to ensure a decent level of consistency across all packages.

@MarNicGit I partly agree with you, and that is probably the reason why the manifest spec says 'should' instead of 'must'. But if versions don't use the recommended rule, it is very hard for a tool to sort packages by this property - e.g. to offer the latest version. That is one of the main reasons why I have created an issue proposing a 'ReleaseDate' property: https://github.com/microsoft/winget-cli/issues/345
But in the case of 'Foldit' it is probably not a good idea to mix the architecture into the version number

@donid If packages are to be managed, then reliable metadata is needed to enable packages to be managed. To be reliable and acceptable in a larger enterprise setting, I believe @megamorf is right that there must be a minimum standard (and some of those 'should's probably should be turned into 'musts'" to raise the bar so to speak)  There must be a high minimum level spec of the metadata when a package is pulled and rejected if it falls short of the standard.


Fix FhirContext initialisation race condition.

This looks great! I'm going to merge it now.

FYI I've added a double-check in FhirContext#validateInitialized just to keep performance up. That method gets called multiple times during a parse cycle so it needs to be pretty fast. Let me know if you have any concerns with the implementation! The test does still pass so that seems like a good sign.


Added images for background

Hey , Please try and explore about .svg files , from here

Hey @ShivamTyagi12345 The linked website downloads the images in jgep format only. Is there something else u were trying to convey?

Still waiting for PR #21 to get merged before merging this

Hey @ShivamTyagi12345 The linked website downloads the images in jgep format only. Is there something else u were trying to convey?

I tried to find a better alternative. but seems like we will have to work with  the larger files for now

Yup, i checked through. The only option is to compress these in case the size in a problem.


siteindex.xsd out of date

The siteindex.xsd file included is out of date. It only allows 1,000 sitemap URLs. New version from http://www.sitemaps.org/schemas/sitemap/siteindex.xsd will allow up to 50,000 sitemap URLs.

@jiwhiz Thanks for reporting. Could you please provide a pull request? Thanks!

sure. I will do that during the weekend.

Thanks!


Add mask adaptor

Given a sequence of values and a sequence of selectors, select_by(vals, selectors) yields those elements of vals for which the corresponding element of selectors is true (after conversion to bool).
Python itertools calls this "compress", but I'm not overly keen on that name. On the other hand, select_by isn't ideal either -- chunk_by takes a predicate, but this takes a second sequence, so there's an inconsistency there. I did consider just calling it select(), but then selection algorithms are a thing...
Fixes #84

The code looks nice. As for the name of the adapter, I just prefer select(), but here are some other names that came to mind:

mask: This name implies that the function applies a masking operation on the input iterator, retaining only the elements that meet the specified condition.

subset: This name suggests that the function creates a subset of the original iterator by selectively including or excluding elements based on a given criterion.

filter_true: This name reflects the behavior of the function, where it filters elements based on a truth value. It conveys the idea that it selects elements that evaluate to true.

Thanks for the name suggestions. I agree select is the nicest, but I quite like mask as well, and that doesn't have an existing meaning

I had a chat with a few folks at C++ on Sea about the name, and I was reminded that Posix select() is a thing, which might be confusing. Sean Parent suggested the name mask(), so that seems like a good one to go for.

The good news is that the Windows CI problems seem to have resolved themselves. The bad news is that now the MacOS CI isn't working because of problems with Homebrew sigh. Hopefully this will get sorted out too and we can merge this.

I also like the mask() version the most.


non-static method after generation

See https://github.com/barryvdh/laravel-ide-helper/issues/95


Fix OpenMind tests timing out

Current status: activations tests are taking way longer than they used to with the same amount of CPUs (5 sec to >1 hour)
Don't want to bypass test because it seems indicative of larger problem with the unit tests


Use the Location response in S3 uploads to properly escape URIs

As per #140 (see comment about this), use the Location response to get the image's URI instead of generating it from the Key value, so it escapes things properly. (Replace https://glowfic-constellation.s3.amazonaws.com with https://d1anwqy6ci9o1i.cloudfront.net so it still uses Cloudfront, though.)
I would appreciate it if you could test this, as I don't think I have a way to do so. Specific things you'd be looking for:

It uses the Cloudfront URL instead of the S3 URL
It actually works with any icons (and it doesn't just mysteriously fail)
It works with an icon where the filename of the icon includes a + (which I think fails at present – the icon won't display on the site, as the URL will be broken)

This is mostly untested.

Make sure icon deletion works if this gets merged, since it seems to rely on the URL for the key and this escapes the key.

Will hopefully be superseded by the "key" branch, where the key value is being separated from the image location.


[issue #1147] Replace lazy_static with once_cell for page server

Found issue #1147 and decided to contribute.
One note:
storage_sync.FATAL_TASK_FAILURES was unused. Do I need to remove it or (as I did) mark it as unused?

There's a similar PR for this https://github.com/neondatabase/neon/pull/2195. JFYI

huh, I didn't notice about it.
Can be closed because of this.


[improve][misc] Upgrade Lombok to 1.18.24

Motivation

attempt to fix StackoverflowError issue in compilation

Modifications

upgrade Lombok to 1.18.24 (release notes)

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository: https://github.com/lhotari/pulsar/pull/105

/pulsarbot rerun-failure-checks

Codecov Report

Merging #18604 (754a8b5) into master (598ca5d) will decrease coverage by 4.39%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #18604      +/-   ##
============================================
- Coverage     41.45%   37.05%   -4.40%
- Complexity     2393     6989    +4596
============================================
  Files           235      618     +383
  Lines         16660    58568   +41908
  Branches       1818     6093    +4275
============================================
+ Hits           6906    21701   +14795
- Misses         8998    34068   +25070
- Partials        756     2799    +2043

Flag
Coverage Δ

unittests
37.05% <ø> (-4.40%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.


Feature import annotations

Required by users: Functionality to import annotations that already exists in some other URLs (i.e. states stored in the database). The following APIs are provided by @eddyod
https://activebrainatlas.ucsd.edu/activebrainatlas/annotation/[id]/[layer_name], which returns the annotations in the corresponding layer, in JSON format.
https://activebrainatlas.ucsd.edu/activebrainatlas/annotations, which returns the list of all annotation layers that we have in the database.

Hi @Junjie-Cheng, I'm the author of the Braincogs neuroglancer fork. Ed and I have been discussing how to integrate our two forks into a single one since we are working on a lot of common features. I think you guys have done way more work than me so I am happy to start working off of your fork. I just started adopting the REST API, for example. I also found your recent histogram feature quite helpful.
I just tried to pull your latest commit: 34795b54db0967412533586cc9cfc79f0f432dd1 to test out the annotation fetching feature and found that I couldn't compile the code due to a missing file: neuroglancer/widget/fetch_annotation.ts. I'm happy to test out this and other features in the future, and I plan to make some PRs of some of my other features that we could both use to reduce the work on your end of cherry picking.
Austin

Hi @austinhoag ,
Thanks for contacting us! If you do not mind you can check this repository: git@github.com:Junjie-Cheng/neuroglancer.git. I have been working on a better organized repository in terms of commits and history, but unfortunately I did not find a good chance to get this version into the ActiveBrainAtlasPipeline repository. Please also note that there are slight differences between these two repositories, and they have totally different git commit tree.
You should be able to pull/merge any branch without any conflicts as they are based on the Google's commit tree, and they are all independent to each other. That being said, please be careful when dealing with the backend branch - we have integrated the API URLs in our code and there may or may not be documentations in the commit message. Also, we DID NOT upload the firebase.ts file to the repository as it contains credentials that may expose us to traffic attacks - this will result compilation errors if you simply pull/merge this branch and build without the firebase.ts file. For this branch, you can utilize git cherry-pick to get the functionalities that you may want.


put parse trees in database order

fixes #170

In some cases the grammar test checks the internal parse tree.
Could it be that those tests fail because that tree order has changed?

yes, the test changes are desired.


Callout bug

Fix for Issue #606 on UWP/WPF, at least.

Thank you for your help.
As I can see (did only a fast view), you only changed one of the samples for Forms. Could you please change the second too?

@charlenni, It has been 6 weeks since I pushed this, so could you point me toward the other sample?
The actual fix is in MapView and Pin, which are both common.    The edits I made to the MainPageLarge of the one sample were just to make it easier to use, and center on the host system's location.
Compare to March commit for MapView, and Feb commit for Pin to see the root issues that were fixed:

Callout created on non-UI thread (Pin.cs) is no-good on Windows because Callout is a UI element/control.
root of above is that the tap/click events are being raised on a background thread (MapView.cs), which is unexpected on Windows (or Android, for that matter).  UI events are generally on the UI/main thread.

@jasells It is very difficult for me to review a PR, which change more than one part at a time. It is the first time I ever review something.
You changed the Callout handling, changed event handling and different parts of half of the sample (consits of MainPageLarge and MainPage/MapPage for different sizes).
It would be nice, if you could make three PR, each for one of the parts, and merge the current master before.

it was current 6wks ago.  ;-)
I realized after I changed the demo app code, that it probably did more harm than good for the PR, BUT it does make repro-ing the issue and fix easier....   which is why I did it.

1. Callout created on non-UI thread (Pin.cs) is no-good on Windows because Callout is a UI element/control.

Did you checked your code on Android and iOS too?

2. root of above is that the tap/click events are being raised on a background thread (MapView.cs), which is unexpected on Windows (or Android, for that matter).  UI events are generally on the UI/main thread.

Did you checked all the other events too? There are much more events than tap/click like fling and longtap.

@jasells - yea the repo got a bit behind as paul is busy. I have completed a few pull requests in the past couple of days so we should be able to get these changes in if you split them up to make the pull request easier to test/understand.
Is there a specific reason you think these events need to be marshalled on the main thread? As your handler can handle the marshalling if a UI change was going to be made off handling of the event. Or is this internally breaking in the MapView itself?

@radderz
"Is there a specific reason you think these events need to be marshalled on the main thread?"
Yes.

From experience, UI events are raised on UI thread in Windows (I think also in Android, I'd have to check).  The fact that SkiaSharp is raising them on a background thread is not expected, and leads to a Cross thread exception when creating the default Callout in Pin (on Windows).
See below :

From Microsoft:
"Code running on a background thread cannot manipulate UI elements directly without raising a cross-thread exception. Instead, you must run UI code using a dispatcher"
I wish I had a better link to explain it, but I have run these circles in the past several times on different projects on Windows.
Xamarin.Forms < v 4 only gives you the option for Xamarin.Forms.Device.BeginInvokeOnMainThread(), which internally calls the Dispatcher (asynchronously) on WPF/UWP.
You can see my other reply above for some more info, but the BLUF is that with Xamarin.Forms v 3.x, this is the best sln I can come up with that isn't ridiculously complicated (invlolving catpuring the UI thread ID during init, and storing it to compare in the Pin.Callout get method to make sure the Callout is created on the Main thread while also avoiding a deadlock.
The changes to Pin and MapView are not seperable, and the changes
only affect UWP/WPF so that UI events behave as expected.  I intentionally wrote the code so that no other platforms will see a change from current.
private static bool PlatformRequiresUIcontext()          {              return Device.RuntimePlatform.Equals(Device.UWP)                   || Device.RuntimePlatform.Equals(Device.WPF);          }
So, the UI events only get routed to the Main/UI thread for Windows, all other platforms behave per status quo.

I'm with Mark: why call this events in MainThread by default. Sometimes you don't want to do anything in the main thread. But if you want, you can do this. If I understand you correct, than you say, that all events are called from the main thread in XF. If this is correct, than we should change all events in the XF MapControl/MapView, not only a few. But I couldn't find any informations on this up to now. Do you have some link (source code of XF?).
The real information is, that BeginInvokeOnMainThread() is an asynchron call. Than it is clear, why creating the callout crashes. With this in mind, your lazy creation should work fine, because the callout is first accessed, when the Pin is created.

The real information is, that BeginInvokeOnMainThread() is an asynchron call. Than it is clear, why creating the callout crashes.

Almost...  BeginInvokeOnMainThread() pushes an Action onto the Main thread's message (work to-do) queque, so any Action passed to BeginInvokeOnMainThread()  will be executed by the UI/main thread.... at some point in the future.  Using BeginInvokeOnMainThread() will prevent the cross-thread exception and resulting crash, BUT you'd need some sync mech to know when it is done, and finish the property getter, and return the resulting Callout's ref.  And, to compound this, async await is not available in a ctor, nor in a Property getter/setter, and BeginInvokeOnMainThread() is async in the sense that it returns immediately, not blocking the caller until the main thread completes the Action, but _neither _  is it awatiable, so any sync attempt would involve an Action with a mutex of some sort to signal to the calling thread that the Main thread is complete.  This means you also  need to know the Main thread's ID so that you can not try to sync a call from the main thread to the Action passed to BeginInvokeOnMainThread(), since that will obviously result in a deadlock (main thread is waiting for the main thread to complete the Action....)

With this in mind, your lazy creation should work fine, because the callout is first accessed, when the Pin is created

You are correct, if limited to only the ctor of Callout.  That was my hope was well, and why I put the lazy initializer in, BUT  Callout's ctor creates and manipulates grid(s), which are UI elements, and is the precise source of the first cross-thread exception in the demo, as written.
See here:             _grid = new Grid()
So, the bottom line is that a Callout, as currently implemented, can never be "newed" on a worker thread on Windows.  Full Stop.
But, the ctor is not the only member of Callout that can produce this exception, since it internally uses a Grid, and any manipulation of the internal Grid can throw the exception, including several properties that typically get set immediately after a Callout is created that ultimately call Callout.UpdateMargin():
_grid.Margin = margin;
This is not surprising behavior, for a UI element/control, on Windows.  (It is idiomatically allowed on Windows that UI elements are not guaranteed to be thread-safe).  Though, I would argue that a well designed control would use the dispatcher synchronously in the ctor, or any other method internally, as needed (or Xamarin.Forms.Device.Invoke() in the case of a Xamarin control).
Again, upgrading to Xforms v4+ would be the cleanest, as then you could use the synchronous InvokeOnMainThread().... either in Pin.Callout getter, or (preferably) internally to Callout anywhere Grid is manipulated.  But, if Windows is the only platform with this issue, then you'd probably still want to check the platform at runtime and only switch thread context for Windows, not other platforms.  Android especially is almost inverse in the fact that you want to do as much as possible on worker threads, and avoid the UI thread, except when absolutely necessary.  But, that doesn't consider the dependency/support issue for a package already released.
However, the UI events being fired on a background thread is  atypical behavior for a Windows app, and as such, any Windows apps using the MapView would be extremely prone to the same error, and make it difficult to track down (especially if it is an x-platform app, other platforms would work fine, it seems).   Popping them over to the UI thread internally, for Windows context only,  makes the MapView more idiomatically correct on Windows, while changing nothing for other platforms.  The Windows idiom is to use async await Task in UI event handlers (button clicks handlers, tap event handles, etc) to intentionally farm non-ui work to worker threads because it is an implicit expectation that the thread raising the event is the main thread.  In fact, this is one of the top use cases for which the async await Task API was created.
I concede, it may be a different issue, but it is very tightly coupled to the exception in the demo, and why it was thought to be resolved previously, but wasn't, at least on Windows.
And... yes, it is complicated. And why I structured the proposed solution to try to contain the complication internally to MapsUI and shield any app-devs potententially using MapsUI from this Windows-specifc mess.  =)

I'll try merging in current master today.

FYI, the Lazy discussion got me thinking.  I checked the mem profile and found that the Lazy  is ~ 10x smaller than the Callout instance, so it does help memory usage in the case of a large # of Pins where you're not likely to need the callout for all of them.
I got to thinking about the simpler code for just creating a default Callout in the Pin ctor.  That would avoid most of the x-thread issue in trade for more memory used.

I'm not happy with the Callout implementation. It is implementad as View. The background is a Skia bitmap and on top of this is a grid and some other Views. This whole thing has now to be moved around on the screen when you move the map. While doing this, you could see, that the Callout moves a little bit slower than the map.
I made it, because I thought, a callout belongs to a map. Users want to have one. But need they such a flexible one? The only advantage is, that you could add any View to the Callout, even an own designed.
But the question is, if we need such a horrible thing? Wouldn't it be better to add a native Mapsui callout, that is drawn direct to the map in the same way as other features? Or only a simple one with header, text and an image? Or as SVG feature? That all would be much faster an would use less memory than the used implementation.
Up to now I didn't discussed this with anybody, so that it would be helpful to think about it, before we try to improve the existing code.

Improving the implementation of the Callout is a different discussion.  Options you mention sound like ideas worth pursuing. I like the SVG idea myself for speed.
However, this PR is to address a bug in the as-is implementation, and would improve things regardless of future Callout implementation over-haul/refactoring.

It's worth to look at this, because if the 2.0 beta goes to 2.0 stable, than you couldn't change such a big thing.

Still contend that is separate from this.   This primarily an issue of MapView behaving as expected on Windows with respect to UI events on UI thread.  Would still be applicable, regardless of Callout impl.

Ok, no discussion.
Regarding this PR: this PR changes 3 different things (Callout problem, event on MainThread, change of sample), which isn‘t always not a good thing. So I would suggest to close this PR and create 3 PRs, each for an individual problem. This is much clearer and could be understand much better.
For the MainThread problem: if you change the behavior of one or two events, than you should change the behavior of all events. I can‘t understand, why this should only be for some events. Or is there a reason?

It is changed for all UI events (events triggered by user interaction via touch or mouse).
I'm not sure what u mean by the 'Callout problem'.....
The only 'problem' addresses with this PR wrt Callout is that the Demo tries to call new Callout on worker thread(s) on Windows.  Do you mean the use of Lazy<Callout> v. the original 'manual' lazy construction!

Ok, here's a Repro branch showing only modified demo app to show:

Adding Pin via MapView.Clicked event in existing Demo app throws x-thread exception on UWP
Adding Pin via Button.Clicked event works.

The mods to the demo app are to show the 2 call stacks and that the MapView.MapClicked event is not behaving as would be expected on Windows.
In order to completely avoid any user interaction with the MapView events, the demo gets the current location of the host system, and uses this location to geo-ref the Pin that is added when the "ctreatePinBtn" is clicked by the user.
Note: I tried to run the WPF demo, and the demo from master doesn't even load.  I get exception very early in the initialization.  Not sure what the issue is at this time.  I suggest this issue be handled separately.

@pauldendulk Xamarin Forms 4+ provides an easy fix with a synchronous Device.RunOnMainThread. Even with refactor, this issue will likely remain on UWP if the tap event source remains SKGLView (MapControl).  These events are raised on worker thread, which is a problem on windows since the event handlers will likely interact with UI controls/elements.

@jasells I searched and searched, but couldn‘t find any report, where someone talks about events and on which thread they are running. Do you have any information with link about this?

@jasells Mapsui's MapControl has a cross platform RunOnUIThread private method. Is there a reason we need the Xamarin Forms 4 version?
A while back I looked into the loop issue. The way is see it is that this code is complicated because it it is a workaround for a Mapsui limitation. The MapView adds layers on top of Mapsui's layers and keep it under its own control. But it has to use some tricks to get this done. The implementation could be a lot simpler if you could hook into Mapsui's layer rendering. So we need some changes at a lower level.

@pauldendulk That’s not correct. The problem with the layers was, that there isn’t the possibility to have layers, that stay always on top. We don’t have a order for layers. If someone than create their own layers, than this layers are on top of the pin and drawable layers. I solved this by always remove and read the layers, when something changes in Layers. Than work and there shouldn’t be a problem.
The loop problem you mentioned is because of the callouts. They contain objects from Forms, which must be created in the UI thread. I did this, because I thought it would be good, if you could put anything you want into a callout. This way, you could add layouts, grids, lists and so on in a callout. But it seems, that there are race conditions. And when creating the callout in the constructor of the pin (what I do), than we get problems.
While looking throw this discussions, I have to say, that the long comment from @jasells at August 15. describes the problem very good.
What we can do, if you want to make a stable 2.0, we could remove all callout functionality. It would be nice to have a callout, because users expect this from a map library, but it is better to have a working one 😉. And it seems, that no one uses callouts, because there isn’t any discussion about this feature.

@charlenni Not being able to keep the MapView's layers on top is the Mapsui limitation I mention. Adding and removing the layers in the MapView is the workaround.
The consequences is complicated code where an action somewhere else can cause a loop if it is on the wrong thread 🙄. The focus should not be on figuring out how we can fix things in this logic but on how we can avoid this kind of logic. So, we need to think about an option to order layers, or what I indicated, a way to render MapView's layers without adding them to Map.Layers (this way they will be completely hidden from the user).
How far removed are we from replacing the current callout with the skia callouts that you are developing in your branch?

@charlenni Not being able to keep the MapView's layers on top is the Mapsui limitation I mention. Adding and removing the layers in the MapView is the workaround.
The consequences is complicated code where an action somewhere else can cause a loop if it is on the wrong thread 🙄. The focus should not be on figuring out how we can fix things in this logic but on how we can avoid this kind of logic. So, we need to think about an option to order layers, or what I indicated, a way to render MapView's layers without adding them to Map.Layers (this way they will be completely hidden from the user).
How far removed are we from replacing the current callout with the skia callouts that you are developing in your branch?

skia-based call outs will help, to an extent.  However, calling this lib Mapsui.Forms is misleading, if your are required to use only skia UI elements...
It is not just creation that must be on main thread, it is also any update to the object.  I pushed examples that show the same issue outside of the Callouts soecifically, I believe above.
Any Xamarin.Forms UI element will show the same issue, if it is modified via a callback from the MapControl tap/touch events.

Mapsui's MapControl has a cross platform RunOnUIThread private method. Is there a reason we need the Xamarin Forms 4 version?

Unsure, it depends on what the MapControl thinks is the UI thread...  I'd assume it is the thread already firing events, but i'd have to look.  If it is the UWP UI thread, then yes, that should work.

@jasells I searched and searched, but couldn‘t find any report, where someone talks about events and on which thread they are running. Do you have any information with link about this?

Here's a link to a post about the same problem: updating UI from background thread.
ObservableCollection also behaves this way on windows

@jasells Mapsui's MapControl has a cross platform RunOnUIThread private method. Is there a reason we need the Xamarin Forms 4 version?
A while back I looked into the loop issue. The way is see it is that this code is complicated because it it is a workaround for a Mapsui limitation. The MapView adds layers on top of Mapsui's layers and keep it under its own control. But it has to use some tricks to get this done. The implementation could be a lot simpler if you could hook into Mapsui's layer rendering. So we need some changes at a lower level.

MapControl.RunOnUIThread just calls Device.BeginInvokeOnMainThread(action).
This does avoid the cross threading exception, BUT it is async, with no way to know when it completes.  This is the root of the problem.  lol
Xamarin.Forms has a synchronous Device.InvokeOnMainThread  whech blcoks the caller until the call completes on the other thread.  This sould be a one line fix in the current code.

@jasells I know, that UI elemts have to be updated from the UI thread. But I don't know, on which thread I'm when a Pin is created. If I'm on not the UI thread, than I try to create the Callout in the UI thread and wait, until it is created. That's working. But when this is called on the UI thread, than I start creating the Callout on the UI thread by BeginInvokeOnMainThread and than wait for the creation. But this wouldn't happen, because the creation takes place, after my function, that waits for the creation, ends. So we have a lock situation. If I now don't use any part for the Callout, that has to be created on a UI thread, it should work perfect, but I loose the ability to use Xamarin.Forms parts for the Callout. That's ok for me. I draw them now direct to the Skia canvas. Advantage is, that they always have the same distance to their anchor point. Disadvantage is, that click detecting is much more difficult than with a Xamarin.Forms view.
So there is only one decision, we have to make: should all events fired on the UI thread or should the user do this? We have some Xamarin.Forms elements in the MapView. This is the SkiaSharp view and the buttons on the side for zoom, center and northing. Only changes to the buttons had to take place in the UI thread. I should look into this, if this is done right. And we have to decide, if Xamarin.Forms events are called on the UI thread or any other thread, so that the user had to switch to the UI thread, if she/he wants to manipulate something on the UI. The link you send says only, that changes of the UI had to be done on UI thread. But it don't said, that events in Xamarin.Forms always have to be fired on UI thread. That was the question: is there a statement from Xamarin, that all events have to be fired on UI thread. Could the user rely on this. I found this link, where you could read in the second paragraph of background, that the user can't rely on this.
If you look into the code, I only propagate the events from SkiaSharp. And SkiaSharp propagate the events from the underlying os (in case of UWP see here). So all events should be called by their underlying event system.

We all know that we need to update the User Interface on the UI thread. This problem also arises with other packages when dealing with asynchronous events. I think Mapsui should not bother about that and the suser should manage it.
The way I do it is to call an init procedure (Initialize() ) of a ThreadHelper at the page contructor level (or appearing) and th check later when I update the UI.

namespace RoadBook.Threading
{
    public static class ThreadHelper
    {
        public static int MainThreadId { get; private set; }

        public static void Initialize(int mainThreadId)
        {
            MainThreadId = mainThreadId;
        }

        public static bool IsOnMainThread => Environment.CurrentManagedThreadId == MainThreadId;

        public static void RunOnMainThread(Action function)
        {
            ThreadLog("RunOnMainThread");

            if(ThreadHelper.IsOnMainThread)
            {
                function.Invoke();
            }
            else
            {
                Device.BeginInvokeOnMainThread(function);
            }
        }
    }
}

I then use the ThreadHelper to update the Ui from the UI thread
        // user decides to cancel his click, just go back to no info pin
        private void OnDeleteInfoPinRequest(RBEditPin pin)
        {
            ThreadLog("EditPage OnDeleteInfoPinRequest 1");

            ThreadHelper.RunOnMainThread(() =>
           {
               ThreadLog("EditPage OnDeleteInfoPinRequest 2", true);

               EditMapView.RemoveNativePinFromTheMap(infoPin);
               infoPin = null;
           });

        }

Note I also write a small TrhredLog procedure that logs the current thread and if required raises an exception if not on the UI thread (susage in  code above).
Implementing that in Mapsui would mean somehow Calling a Mapsui initialisation while being in the UI thread or to pass the UI thread to Mapsui …
Both solution doesn't prevent the user to care about UI thread in other asynchronous events, so in my humblre opinion it is OK like that ...

We all know that we need to update the User Interface on the UI thread. This problem also arises with other packages when dealing with asynchronous events. I think Mapsui should not bother about that and the suser should manage it.
The way I do it is to call an init procedure (Initialize() ) of a ThreadHelper at the page contructor level (or appearing) and then check later when I update the UI.
namespace RoadBook.Threading
{
public static class ThreadHelper
{
public static int MainThreadId { get; private set; }
    public static void Initialize(int mainThreadId)
    {
        MainThreadId = mainThreadId;
    }

    public static bool IsOnMainThread => Environment.CurrentManagedThreadId == MainThreadId;

    public static void RunOnMainThread(Action function)
    {
        ThreadLog("RunOnMainThread");

        if(ThreadHelper.IsOnMainThread)
        {
            function.Invoke();
        }
        else
        {
            Device.BeginInvokeOnMainThread(function);
        }
    }
}

}
I then use the ThreadHelper to update the Ui from the UI thread
// user decides to cancel his click, just go back to no info pin
private void OnDeleteInfoPinRequest(RBEditPin pin)
{
ThreadLog("EditPage OnDeleteInfoPinRequest 1");
        ThreadHelper.RunOnMainThread(() =>
       {
           ThreadLog("EditPage OnDeleteInfoPinRequest 2", true);

           EditMapView.RemoveNativePinFromTheMap(infoPin);
           infoPin = null;
       });

    }

Note I also wrote a small ThreadLog procedure that logs the current thread and if required raises an exception if not on the UI thread (ussage in  code above).
Implementing that in Mapsui would mean somehow Calling a Mapsui initialisation while being in the UI thread or to pass the UI thread to Mapsui …
Both solution doesn't prevent the user to care about UI thread in other asynchronous events, so in my humblre opinion it is OK like that ...

The issue here is that MapsuiView is a Xamarin.Forms View, and it is not an unreasonable expectation that it behaves like other Xamarin Views.  This the only time I've had to deal with this in Xamarin.
Adding this helper, with modification, could allow syncing properly (add a mutex so the calling thread knows when async action is complete), but Xamarin.TouchEffect provides this functionality, and is native to Xamarin, and cleaner.

@jasells I use only events from SkiaSharp.Views.Forms. So SkiaSharp call them in the wrong context. So we had to mention this for SkiaSharp and change this behavior there. Or is this wrong?

@jasells  If you use a simple Timer to get GPS position or other things you will be out of the UI thread anyway, this is not a special case ...
ThreadLog : RunOnMainThread  Current Thread / UI Thread 23 / 3
I agree with you for the mutex but I don't need in my app for now :-)

I now tested it with the Xamarin.Essential NuGet package and the MainThread.IsMainThread flag. The code than looks like
        if (MainThread.IsMainThread)
        {
            _callout = new Callout(_mapControl)
            {
                Anchor = position
            };
        }
        else
        {
            // In Forms.UWP, the Callout needs to be created on the UI Thread
            Device.BeginInvokeOnMainThread(() =>
            {
                _callout = new Callout(_mapControl)
                {
                    Anchor = position
                };
            });
        }

When I do it this way, than I see, that it crashes, when I call the BeginInvokeOnMainThread() function un UWP, because on UWP the event is called on main thread and so the callout creation is started with BeginInvokeOnMainThread and than is awaitet in the main thread. That's why the app stops. So all is ok. The only problem was, that I already was on the main thread.
And, if I test with this function, than on UWP the touch functions are called always on the main thread. No problem to call any UI updates.
But now I would say, that the new approach is the better way to create callouts.

Crashing … strange …
Micosoft doc …
You might wonder if you should check if code is running on a secondary thread before calling BeginInvokeOnMainThread, for example, like this:
C#

Copy

if (MainThread.IsMainThread)
{
    MyMainThreadCode();
}
else
{
    MainThread.BeginInvokeOnMainThread(MyMainThreadCode);
}
You might suspect that this check might improve performance if the block of code is already running on the main thread.
However, this check is not necessary. The platform implementations of BeginInvokeOnMainThread themselves check if the call is made on the main thread.
There is very little performance penalty if you call BeginInvokeOnMainThread when it's not really necessary.

Lets look into the code for BeginInvokeOnMainThread: UAP, Android, iOS, Tizen and GTK.
For me it looks like the most systems don't check, if they are on the main thread, but start the action (put it in a queue) and go back. So there could be problems like the above.
@jasells Did you see any Mapsui.Forms event on UWP, that aren't called on the main thread?

ThreadLog : EditPage OnEditMapClicked 1  Current Thread / UI Thread 26 / 3
ThreadLog : RunOnMainThread  Current Thread / UI Thread 26 / 3

ThreadLog : EditPage OnEditMapClicked 1  Current Thread / UI Thread 7 / 3
ThreadLog : RunOnMainThread  Current Thread / UI Thread 7 / 3
ThreadLog : EditPage OnEditMapClicked 2  Current Thread / UI Thread 3 / 3

From time to time MapClicked is on another thread, but it is not systematic. Being on another thread is not always synonym to crash ...

@sailman342 Did you use OnEditMapClicked as your normal MapView.MapClick event? I checked for main thread in the function, that call MapClicked and never get a IsMainThread = false. So it would be nice, if you could check this.

Here is the code :
    public class RBMapView : MapView

    {
        public RBMapView() : base()

            this.MapClicked += OnMapClicked;
            this.PinClicked += OnPinClicked;

        private void OnMapClicked(object sender, MapClickedEventArgs eArg)
        {
            ThreadLog("RBMapView OnMapClicked");

            RBMapClickedEventArgs eventArgs = new RBMapClickedEventArgs(
                eArg.Point.Latitude,
                eArg.Point.Longitude
                );

            onMapClickHdlr((RBMapView)sender, eventArgs);
        }

And the threadlog you see in the OnMapClicked handler is below
ThreadLog : RBMapView OnMapClicked  Current Thread / UI Thread 7 / 1
This one is done on Forms.WPF because I am fighting with WPF for the time being :-(

Oh, I would say, that Xamarin.Forms support for WPF isn't perfect. So I would say, that we couldn't rely on this results.

This one is on UWP (like the ones I showed yesterday)
ThreadLog : RBMapView OnPinClicked  Current Thread / UI Thread 26 / 3

I checked on UWP and don't get ever a call of OnPinClicked, that isn't on main thread.

I am Swiss Citizen, possibly slow but not dreaming :-) :-)
Windows 10 family, Visual studio 2019, mapsui from the master 4 days ago …
What do you think I should do next !
What are the procédures you use to test UI Thread ?
I am just willing to help, these thing are OK for me as they are :-)

@jasells
Microsoft
Applications sometimes need to use events that call the event handler on a secondary thread of execution. (The Xamarin.Essentials classes Accelerometer, Compass, Gyroscope, Magnetometer, and OrientationSensor all might return information on a secondary thread when used with faster speeds.) If the event handler needs to access user-interface elements, it must run that code on the main thread. The MainThread class allows the application to run this code on the main thread.

I don't want to say, that you aren't correct, but I couldn't see it. So, perhaps my changes to the callout creation changed something or we do different tests.
I added the lines as descibed above and added Xamarin.Essential for the check of the main thread. Than I use a line like
        System.Diagnostics.Debug.WriteLine("PinClick.IsMainThread " + Xamarin.Essentials.MainThread.IsMainThread.ToString());

in MainPageLarge.xaml.cs in function OnPinClicked. The result is, that I always get a true.
If you have time, please try to add your thread logic at PolylineSample. Than start the polyline sample of Forms and click on the map around. You will see a red polyline, extending with each click. On which thread this click is called? On the same thread should OnPinClicked be called. Both are called from the touch event without changing the thread.

I was just joking I didn't felt offensed but I am a newbie with Xamarin ...

And thank you for all your efforts.

ThreadLog :  Current Thread 14 (on UWP Ui Thread use to be 3
        public bool OnClick(object sender, EventArgs args)
        {
            System.Diagnostics.Debug.WriteLine($"ThreadLog :  Current Thread {Environment.CurrentManagedThreadId} (on UWP Ui Thread use to be 3 " );

I copied your line into my code, which I derived yesterday from master. The result:

As you see, each time I start the app, I get 3 as ID.
Perhaps another version of SkiaSharp? I use 1.68.2-preview.17. You have the same?

Changed the branch and used the master now. Added your line of code. Get always 3 as Id.

However, calling this lib Mapsui.Forms is misleading, if your are required to use only skia UI elements...

How the callout supplied by Mapsui is implemented does not matter. The features on the map are also rendered with skia, but it is still a Xamarin.Forms component because it can be used in Xamarin.Forms. Users do not even need to know how it is implemented. Then again, we could add features that make it possible to let users add their own callout logic, using Xamarin.Forms elements.

@charlenni
Added your code to mine
ThreadLog : OnClick Current Thread 13 (on UWP Ui Thread use to be 3 )
OnClick.IsMainThread False
ThreadLog : OnClick Current Thread 15 (on UWP Ui Thread use to be 3 )
OnClick.IsMainThread False
ThreadLog : OnClick Current Thread 9 (on UWP Ui Thread use to be 3 )
OnClick.IsMainThread False

        public bool OnClick(object sender, EventArgs args)
        {
            System.Diagnostics.Debug.WriteLine($"ThreadLog : OnClick Current Thread {Environment.CurrentManagedThreadId} (on UWP Ui Thread use to be 3 ) " );
            System.Diagnostics.Debug.WriteLine("OnClick.IsMainThread " + Xamarin.Essentials.MainThread.IsMainThread.ToString());

I don't know what to do. I started the sample app in x86 and x64 on a Win 10 system with VS 2019 16.4.3. In both version I go to the "Add polyline Sample" and start clicking. I get always a 3 as Id. Nothing else.

What about @jasells who started the thing ?
I downloaded a fresh version of Mapsui Master, flushed my Nuget stuff and will restart building mapsui this evening, I will keep you posted :-)

@jasells  If you use a simple Timer to get GPS position or other things you will be out of the UI thread anyway, this is not a special case ...
ThreadLog : RunOnMainThread  Current Thread / UI Thread 23 / 3
I agree with you for the mutex but I don't need in my app for now :-)

GPS, other things are nit UI events (touch, tap, click, etc)

Lets look into the code for BeginInvokeOnMainThread: UAP, Android, iOS, Tizen and GTK.
For me it looks like the most systems don't check, if they are on the main thread, but start the action (put it in a queue) and go back. So there could be problems like the above.
@jasells Did you see any Mapsui.Forms event on UWP, that aren't called on the main thread?

@jasells
Microsoft
Applications sometimes need to use events that call the event handler on a secondary thread of execution. (The Xamarin.Essentials classes Accelerometer, Compass, Gyroscope, Magnetometer, and OrientationSensor all might return information on a secondary thread when used with faster speeds.) If the event handler needs to access user-interface elements, it must run that code on the main thread. The MainThread class allows the application to run this code on the main thread.

This us what I've been saying..,.

However, calling this lib Mapsui.Forms is misleading, if your are required to use only skia UI elements...
How the callout supplied by Mapsui is implemented does not matter. The features on the map are also rendered with skia, but it is still a Xamarin.Forms component because it can be used in Xamarin.Forms. Users do not even need to know how it is implemented. Then again, we could add features that make it possible to let users add their own callout logic, using Xamarin.Forms elements.

Correct, and the problem currently is that any pin implemented as Xamairn.Forms.View or similar, will have same issue as Calllout.  Changing Callout impl to skia avoids the issue for Callout, but the general issue remains, waiting to spring on app debs using Mapsui.Forms.

What about @jasells who started the thing ?
I downloaded a fresh version of Mapsui Master, flushed my Nuget stuff and will restart building mapsui this evening, I will keep you posted :-)

Its been 6  months....  We forked  4 months ago, but I'll  look at the repro branch I pushed then, see if needs syncing with current master

        public bool OnClick(object sender, EventArgs args)
        {
            System.Diagnostics.Debug.WriteLine($"Pin Sample : OnClick Current Thread {Environment.CurrentManagedThreadId}");
            System.Diagnostics.Debug.WriteLine("Pin Sample : OnClick.IsMainThread " + Xamarin.Essentials.MainThread.IsMainThread.ToString());

Pin Sample : OnClick Current Thread 3
Pin Sample : OnClick.IsMainThread True

        public bool OnClick(object sender, EventArgs args)
        {
            System.Diagnostics.Debug.WriteLine($"Polygon Sample : OnClick Current Thread {Environment.CurrentManagedThreadId}");
            System.Diagnostics.Debug.WriteLine("Polygon Sample : OnClick.IsMainThread " + Xamarin.Essentials.MainThread.IsMainThread.ToString());

I don't know if it is normal but I had to make multiple regenerations to get down to 23 warnings from 185 first try (I enabled automatic download of missing nugets)
We are always on main (UI) thread but pin exemple crahes ... polygon is Ok
And ployline as well
Polyline Sample : OnClick Current Thread 3
Polyline Sample : OnClick.IsMainThread True
Polyline Sample : OnClick Current Thread 3
Polyline Sample : OnClick.IsMainThread True
Polyline Sample : OnClick Current Thread 3
Polyline Sample : OnClick.IsMainThread True
Polyline Sample : OnClick Current Thread 3
Polyline Sample : OnClick.IsMainThread True

Are these warning meaningfull to you ?

Pin sample crashes are ok, that's the callout bug. But now you don't see any call on other threads than the UI thread? Is that correct?

Yes, we are inline now …
Do you have an idea about the skia warnings ?
Had a quick look on my project and the master rebuild, I was on Forms 4.4 while the master is on 4.3 !
I will make further investigations but it can be interesting to remember in case you would move to version 4.4 of forms (@jasells  can you kindly advise us on that ?)
Kindest and thanks again

@jasells Ok, the callout bug should be solved with the new type of callouts.
But there is always the problem with that, that you say, each event should be called on main thread. All I found is, that each change to the UI should be called on the UI thread. The event, from which such changes are fired, should call the event handler from the UI thread, that I couldn't find.
But I think, I could do this. It is a little bit of more effort. And I should use the MainThread functions from the Xamarin.Essential package. But that package contains no WPF version. So we have no Mapsui.Forms.WPF vrsion.
The other way would be to move this to the users and say, that they have to be carefull, because events mustn't be called on main thread.

@jasells Ok, the callout bug should be solved with the new type of callouts.

I just pulled master, and it gets stuck in MapView.CreateCallout().
This appears to be because the tap events are now firing on the main thread from MapControl, which is a SKGLView (as expected), but MapView.CreateCallout() is trying to sync with creating the callout out on the main thread, but the sync is a crude spin wait, so the main thread is spinning waiting for itself to complete the Device.BeginInvokeOnMainThread....
This is due to the fact the SkiaSharp.Forms.Views has been updated from 1.6 to 1.68.2-preview.17.  The UI event on Main thread issue has been resolved there (the home lib of SKGLView which is the ultimate source of all UI events sources my the Map in MapsUI.Forms).
The current master branch requires SkiaSharp.Forms.Views v 1.68.2-preview.17, which in turn requires Xamarin.Forms v >= 4.2, which removes one of the self-imposed conditions on this original PR: DO NOT CHANGE the Xamarin.Forms version requirement.
SO....
Unless you guys want to fix this bug on the older release of MapsUI.Forms (1.?) Whatever it was when this thread started, it would probably be best to start a new thread about the current issue on UWP, and not mix up different bugs from different versions of Mapsui.UI.Forms.
The issue in current master (2.0.0-beta.25) is that MapView.CreateCallout() no longer needs to call Device.BeginInvokeOnMainThread and it should never try to sync without first knowing it is needed (the calling thread != UI thread, as sailman342 has shown with his examples).
It appears to me that MapView.CreateCallout()  isn't even needed, since the UI events are firing on the expected thread!
The fix can be made in Pin.Callout.get and remove MapView.CreateCallout()
See this branch.
JAS

@jasells Ok, the callout bug should be solved with the new type of callouts.
But there is always the problem with that, that you say, each event should be called on main thread. All I found is, that each change to the UI should be called on the UI thread. The event, from which such changes are fired, should call the event handler from the UI thread, that I couldn't find.
But I think, I could do this. It is a little bit of more effort. And I should use the MainThread functions from the Xamarin.Essential package. But that package contains no WPF version. So we have no Mapsui.Forms.WPF vrsion.
The other way would be to move this to the users and say, that they have to be carefull, because events mustn't be called on main thread.

ON WINDOWS  UI updates must be made by UI thread.  Because of this, Windows HAS ALWAYS FIRED UI EVENTS ON MAIN THREAD.   Full. Stop.
Not ALL events (like network data, GPS updates, etc. etc. etc).
There are 2 categories of events on Windows:

UI events (clicks, taps, mouse move, keypress, etc)
everything else

IF any decides to try to fix this issue in the older version of Maspui.UI.Forms, or needs it for their app:
Here is the BLUFF  that does not require upgrading Xamarin.Forms or SkiaSharp.

Yes, we are inline now …
Do you have an idea about the skia warnings ?
Had a quick look on my project and the master rebuild, I was on Forms 4.4 while the master is on 4.3 !
I will make further investigations but it can be interesting to remember in case you would move to version 4.4 of forms (@jasells  can you kindly advise us on that ?)
Kindest and thanks again

I pulled an built the master no problem...


The Wallet shut down when approving a multisig transaction.

I have a multisig account and 1 approval account on my PC wallet, and 2 approval accounts on my i phone X.
When I was about to approve the transaction on the accounts NEM wallet shut down suddenly. Is this a bug on the app？

Hi, @NEMorDIE
We need more information:
Device model and iOS version
Expected Behavior
Current Behavior
Steps to Reproduce
more Details;


Table rendering differs locally and deployed

On https://vchuravy.dev/notes/2021/08/bpftrace/ I am embedding a table source code https://github.com/vchuravy/vchuravy.github.io/blob/main/notes/2021/08/bpftrace.md
Locally this renders just fine.
Updating `~/work/vchuravy.github.io/vchuravy.github.io/Project.toml`
  [713c75ef] + Franklin v0.10.46
  [2bd173c7] + NodeJS v1.3.0

In your deploy script can you try using
optimize(minify=false)

? that should fix it

Hmm... That wasn't enough https://github.com/vchuravy/vchuravy.github.io/runs/3279617359

will have a look at your repo, give me 30 mins

there is something odd about all this, maybe to do with how quickly things propagate to your custom domains, not sure.
In any case this seems to work: https://tlienart.github.io/test_vchuravy/notes/2021/08/bpftrace/
I've not done much else than

removing the favicon (unrelated but the cause of an ugly red question mark)
simplified the deploy script (probably also unrelated)
some testing
there's a specific prepath bc it's a project site which you'd have to remove in config.md if you copy-pasted my stuff

Found the difference. My deploy scripty was using Julia 1.5 and yours is 1.6 https://github.com/vchuravy/vchuravy.github.io/commit/e23cfe9e47aca71f0b3526d745ca0b0ad241484f


beautify help

Build a responsive help

Coverage increased (+3.1%) to 89.628% when pulling 4c95dd8aa9e16159bb5912b8d0d3858c7f616a47 on beautiful-help into da9515902135a2c9071c7464a1ae660b8b24d994 on master.

[signal 0xb code=0x1 addr=0x0 pc=0x6c719e]

goroutine 1 [running]:
panic(0xf48fa0, 0xc820014170)
	/home/martin/.gvm/gos/go1.6/src/runtime/panic.go:464 +0x3e6
github.com/containous/traefik/vendor/github.com/containous/staert.(*Staert).LoadConfig(0xc82019beb8, 0x0, 0x0, 0x0, 0x0)
	/home/martin/go/src/github.com/containous/traefik/vendor/github.com/containous/staert/staert.go:71 +0x90e
main.main()
	/home/martin/go/src/github.com/containous/traefik/traefik.go:134 +0x1b61

Coverage increased (+3.1%) to 89.628% when pulling ec408dc4019a96c6ef00d51328e06006f1ce4ee7 on beautiful-help into da9515902135a2c9071c7464a1ae660b8b24d994 on master.

LGTM

Coverage increased (+3.1%) to 89.628% when pulling 271bd6f66a23fb4a4bf9a301a7dab0bf7b5a7746 on beautiful-help into da9515902135a2c9071c7464a1ae660b8b24d994 on master.

Still good for you @emilevauge ?

Yep, LGTM :)

Coverage increased (+3.1%) to 89.628% when pulling 299772451a582f9bd78ae699d06e4e6065444aee on beautiful-help into 308f7f8a685ee120daf95c1fef0f860805af6845 on master.


Dataclass has trouble parsing JSON string fields with newlines as well as double-quotes

I've been loving the dataclass parsing features, but I ran into this issue where the library errors when parsing string values that contain double-quotes or newline characters
This is my dataclass and the query that is used:
@dataclass
class CardData:
    name: str
    mana_cost: str
    supertypes: str
    types: str
    subtypes: str
    rules: str
    flavor: str
    # Creature
    attack: str
    defense: str
    # Planeswalker
    loyalty: str

@lmql.query
def convert_to_object(card_text):
    """lmql
    "{card_text}\n"
    "Structured: [CARD_DATA]\n" where type(CARD_DATA) is CardData

    CARD_DATA
    """

This is an example culprit value for card_text:
Yktlash, the Unseen Empress

{3}{B}{B}{G}{G}
Legendary Creature â€” Elf Shaman (5/5)
Trample

When Yktlash, the Unseen Empress enters the battlefield, create three 1/1 black and green Elf Warrior creature tokens.

{B}{G}, Sacrifice an Elf: Target creature gets -3/-3 until end of turn.

> 'In the depths of the forest, her reign remains elusive. Only the echoes of her whispers reveal the true power she wields.'

This was printed to the console. And you can see the newlines in the rules field that is likely the issue.
Failed to parse JSON from {"name":"Yktlash, the Unseen Empress","mana_cost":"{3}{B}{B}{G}{G}","supertypes":"Legendary","types":"Creature","subtypes":"Elf Shaman","rules":"Trample
When Yktlash, the Unseen Empress enters the battlefield, create three 1/1 black and green Elf Warrior creature tokens.
{B}{G}, Sacrifice an Elf: Target creature gets -3/-3 until end of turn.","flavor":"In the depths of the forest, her reign remains elusive. Only the echoes of her whispers reveal the true power she wields.","attack":"5","defense":"5","loyalty":"0"}

Then this exception was thrown:
Traceback (most recent call last):
  File "D:\REDACTED\src\main.py", line 98, in <module>
    result: LMQLResult = convert_to_object(card_rules)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\api\queries.py", line 148, in lmql_query_wrapper
    return module.query(*args, **kwargs)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\lmql_runtime.py", line 204, in __call__
    return call_sync(self, *args, **kwargs)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\loop.py", line 37, in call_sync
    res = loop.run_until_complete(task)
  File "C:\Users\krist\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete
    return future.result()
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\lmql_runtime.py", line 230, in __acall__
    results = await interpreter.run(self.fct, **query_kwargs)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\tracing\tracer.py", line 240, in wrapper
    return await fct(*args, **kwargs)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\interpreter.py", line 1070, in run
    async for _ in decoder_fct(prompt, **decoder_args):
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\dclib\decoders.py", line 22, in argmax
    h = await model.rewrite(h, noscore=True)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\dclib\dclib_rewrite.py", line 194, in rewrite
    result_items = await asyncio.gather(*[op_rewrite(path, seqs) for path, seqs in ar.sequences.items()])
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\dclib\dclib_rewrite.py", line 191, in op_rewrite
    return path, await self._rewrite_seq(seqs, noscore=noscore)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\dclib\dclib_rewrite.py", line 64, in _rewrite_seq
    rewritten_ids = await rewriter(seqs, mask)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\interpreter.py", line 905, in rewrite_processor
    results = await asyncio.gather(*[self.rewrite_for_sequence(s, needs_rewrite) for s,needs_rewrite in zip(seqs, mask_seq_to_rewrite)])
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\interpreter.py", line 719, in rewrite_for_sequence
    result: RewrittenInputIds = await si.rewrite_for_sequence(seq, needs_rewrite, assert_no_advance=assert_no_advance)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\interpreter.py", line 805, in rewrite_for_sequence
    state = await self.advance(state)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\interpreter.py", line 434, in advance
    await query_head.advance(None)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\multi_head_interpretation.py", line 89, in advance
    await self.handle_current_arg()
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\multi_head_interpretation.py", line 112, in handle_current_arg
    await self.advance(res)
  File "C:\Users\krist\AppData\Local\pypoetry\Cache\virtualenvs\REDACTED-cM1rhShM-py3.10\lib\site-packages\lmql\runtime\multi_head_interpretation.py", line 88, in advance
    self.current_args = await self.iterator_fct().asend(result)
  File "C:\Users\krist\AppData\Local\Temp\tmpd1cqlv38_compiled.py", line 101, in query
    yield ('result', await lmql.runtime_support.call(type_dict_to_type_instance, json_payload, ty))
UnboundLocalError: local variable 'json_payload' referenced before assignment

I just pushed a fix to main which resolves it. Thanks for reporting it, @dataclass support is still in Preview, so this kind of feedback is very helpful.

@lbeurerkellner Awesome, thanks for that! Do you have an official ingress for feedback?
Because I think that @dataclass support is super powerful, but could use some enhancements:

Supporting Optional[] fields (not forcing the llm to generate the field)
Supporting Literal[] fields (especially literal of different string values to be able to have more control over the values of the fields, e.g. Literal["red", "blue", "yellow"])
An alternative to supporting different data types like that would be to add a decorator to the fields. Maybe something like @lmql.field() that could specify restrictions for the field in LMQL syntax, which would be very powerful as it could support the above examples plus regex, etc

I could imagine something like this perhaps
@dataclass
class MyClass:
    @lmql.field("""where classification in ["positive", "negative", "neutral"]""")
    classification: str

All of these things can already be done if writing a prompt formatted in JSON for example, and then writing the python code to convert the JSON object to python. So using @dataclasses is really convenient compared to writing a JSON prompt and doing the conversion because it's already in the target format


Phonegap: cannot import the plugin

Can't seem to get this plugin to work by following the instructions.
Did phonegap plugin add cordova-plugin-admob-free
Tried: import {banner} from 'cordova-plugin-admob-free/admob'
But it complained it could not find the module.
Tried: import {banner} from 'cordova-plugin-admob-free/www/admob'
Then got the following error: Cannot find module 'cordova/exec' from 'cordova-plugin-admob-free/www/admob.js'

Hello, I'm having the same problem, were you able to solve it?

You have to wrap it in Ionic Native: npm install --save @ionic-native/admob-free
Check this tutorial: https://ionicframework.com/docs/native/admob-free/

@msolovyov Does @AlfredGjini suggestions work for you?
Or you may provide a repo demonstrate your problem, your error seems relate to project setup.

I'm using plain phonegap and I didn't want to bloat the app by including ionic-native. I ended up using a different admob plugin.

Closing this as nothing is actionable.


Fixed clippy issues

I fixed most of the "cargo clippy" issues. I even got a small performance increase because of the changes.
There are some left where the fix is not obvious.

Great! Do you know if we can integrate clippy to CI check or git pre-commit auto fix?
It would be good if we have them.

I have not yet tried this myself. https://doc.rust-lang.org/stable/clippy/continuous_integration/github_actions.html says it should work.


Single Quote Escaped with <

It looks like single quotes get escaped with < when ' would be more appropriate.  Am I missing something here?  Happy to submit a PR if the current behavior is unintended.

Thanks for reporting the bug. It's indeed a typo and I've fixed it already.


Robot: Reinit to define testbed_connection sooner

The ${testbed_connection} variable is used in teardown.
Its late initialization makes teardown fail if setup failed,
which prevents log gathering.
Signed-off-by: Vratko Polak<EMAIL_ADDRESS>

Coverage remained the same at 61.546% when pulling a973b22dd20b4331c8835391dfe364515bd11226 on vrpolakatcisco:testbed_connection0 into 75d90b60fda5fb6d1966f4b9ce996cbb95878918 on contiv:master.


Initializing Table Data

Hi Everyone,
I want to initialize table data with about 100 rows.  I see the examples which are essentially hard-coded data in code units.  Is there a more elegant way? Is there someway to embed an XML file in a manifest and read from it into the table?  What have other users done?
Thanks
-Rob

Hi @fibbert,
You can publish table data with your extension which allows you to seed your tables. Take a look at Exporting data for Extensions.


Fix created dataset naming convention

Feature or Bugfix

Bugfix

Detail

Before we were creating naming convention for s3 buckets, kms keys, etc. for newly create datasets without using the targetUri because we were referencing the Dataset object before it was added to the RDS Table and thus before the datasetUri is created

Relates

N/A

Security
Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)?

Is the input sanitized?
What precautions are you taking before deserializing the data you consume?
Is injection prevented by parametrizing queries?
Have you ensured no eval or similar functions are used?

Does this PR introduce any functionality or component that requires authorization?

How have you ensured it respects the existing AuthN/AuthZ mechanisms?
Are you logging failed auth attempts?

Are you using or adding any cryptographic features?

Do you use a standard proven implementations?
Are the used keys controlled by the customer? Where are they stored?

Are you introducing any new policies/roles/users?

Have you used the least-privilege principle? How?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Great catch, thanks for the quick PR. Overall changes would solve the issue but I think the create_dataset function in dataset_service can be improved. Right now in that function we:

check AWS account (check Quicksight subscription)
create the Dataset in RDS with the function build_dataset
check the imported resources
create Activity in RDS and update some dataset info with the function create_dataset
create bucket in RDS, create permissions, create stack in RDS, upsert in Catalog and deploy stack

The problem is that if the imported resources fail, we would have an orphan dataset in the UI and no permissions are attached at that point. Instead I would re-order the actions as:

check AWS account (check Quicksight subscription)
check the imported resources
create the Dataset in RDS + set_aws_resources + create Activity
create bucket in RDS, create permissions, create stack in RDS, upsert in Catalog and deploy stack

Great catch, thanks for the quick PR. Overall changes would solve the issue but I think the create_dataset function in dataset_service can be improved. Right now in that function we:
1. check AWS account (check Quicksight subscription)

2. create the Dataset in RDS with the function `build_dataset`

3. check the imported resources

4. create Activity in RDS and update some dataset info with the function `create_dataset`

5. create bucket in RDS, create permissions, create stack in RDS, upsert in Catalog and deploy stack

The problem is that if the imported resources fail, we would have an orphan dataset in the UI and no permissions are attached at that point. Instead I would re-order the actions as:
1. check AWS account (check Quicksight subscription)

2. check the imported resources

3. create the Dataset in RDS + set_aws_resources + create Activity

4. create bucket in RDS, create permissions, create stack in RDS, upsert in Catalog and deploy stack

I do not think the above is true - after this PR the way we implemented create_dataset is:

check AWS account (check Quicksight subscription)
create Dataset Model + _set_import_data (not inserted into RDS)
check the imported resources
create the Dataset in RDS + set_aws_resources + create Activity
create bucket in RDS, create permissions, create stack in RDS, upsert in Catalog and deploy stack

I also tested by importing a dataset with incorrect KMS Key alias and there was no orphaned dataset in RDS. Let me know if I am missing some edge case / testing scenario though

My bad! I did not see the commit(dataset) in create_dataset. I deployed the PR in AWS and I confirm:

[X] CICD pipeline succeeds
[X] Dataset creation/import succeeds with proper names


Can't run on headless machine (EGL and magnum errors)

Hi, I am trying to run Habitat RLEnv with this algorithm (also from facebookresearch) on a headless machine, but I am getting two errors.
First, I clone both habitat-lab and habitat-sim stable branches.
I follow the instructions and install habitat-lab with pip and habitat-sim with conda and the headless option. No errors at this point. I can also run python examples/examples.py successfully.
However, if I try to run the same environment with the RL code I linked I get
I0519 14:27:53.016191 2270664 Simulator.cpp:158] Loaded.
I0519 14:27:53.016216 2270664 SceneGraph.h:93] Created DrawableGroup:
Platform::WindowlessEglApplication::tryCreateContext(): unable to find EGL device for CUDA device 0
WindowlessContext: Unable to create windowless context

According to the common issues section, I have to build from source with DEGL_LIBRARY. I do it, but with one difference: I don't have /usr/lib/x86_64-linux-gnu/nvidia-opengl/libEGL.so but /usr/lib/x86_64-linux-gnu/nvidia-opengl/libEGL_nvidia.so.0
I build from source (no errors) and then run the same code with LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/nvidia-opengl:${LD_LIBRARY_PATH}, but this time I get another error. If I try to run the example I get
Traceback (most recent call last):
  File "examples/example.py", line 10, in <module>
    import demo_runner as dr
  File "/private/home/sparisi/habitat-baselines/habitat-sim/examples/demo_runner.py", line 15, in <module>
    from settings import default_sim_settings, make_cfg
  File "/private/home/sparisi/habitat-baselines/habitat-sim/examples/settings.py", line 5, in <module>
    import habitat_sim
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/__init__.py", line 13, in <module>
    from habitat_sim import (  # noqa: F401
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/agent/__init__.py", line 9, in <module>
    from .agent import *  # noqa: F401,F403
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/agent/agent.py", line 14, in <module>
    from habitat_sim import bindings as hsim
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/bindings/__init__.py", line 10, in <module>
    from habitat_sim._ext.habitat_sim_bindings import (
ImportError: generic_type: type "SceneNode" referenced unknown base type "Magnum::SceneGraph::Object<Magnum::SceneGraph::BasicTranslationRotationScalingTransformation3D<float> >"

If I try the other code I get
Traceback (most recent call last):
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/private/home/sparisi/habitat-baselines/minigrid/src/utils.py", line 345, in act
    raise e
  File "/private/home/sparisi/habitat-baselines/minigrid/src/utils.py", line 192, in act
    gym_envs = create_env(flags)
  File "/private/home/sparisi/habitat-baselines/minigrid/src/utils.py", line 99, in create_env
    envs.append(make_habitat())
  File "/private/home/sparisi/habitat-baselines/minigrid/src/utils.py", line 57, in make_habitat
    from habitat_baselines.common.environments import NavRLEnv
  File "/private/home/sparisi/habitat-baselines/habitat-lab/habitat_baselines/__init__.py", line 12, in <module>
    from habitat_baselines.il.trainers.pacman_trainer import PACMANTrainer
  File "/private/home/sparisi/habitat-baselines/habitat-lab/habitat_baselines/il/trainers/pacman_trainer.py", line 24, in <module>
    from habitat_baselines.il.data.nav_data import NavDataset
  File "/private/home/sparisi/habitat-baselines/habitat-lab/habitat_baselines/il/data/nav_data.py", line 15, in <module>
    from habitat.datasets.utils import VocabDict
  File "/private/home/sparisi/habitat-baselines/habitat-lab/habitat/datasets/utils.py", line 19, in <module>
    from habitat.sims.habitat_simulator.actions import HabitatSimActions
  File "/private/home/sparisi/habitat-baselines/habitat-lab/habitat/sims/habitat_simulator/actions.py", line 12, in <module>
    import habitat_sim
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/__init__.py", line 13, in <module>
    from habitat_sim import (  # noqa: F401
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/agent/__init__.py", line 9, in <module>
    from .agent import *  # noqa: F401,F403
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/agent/agent.py", line 14, in <module>
    from habitat_sim import bindings as hsim
  File "/private/home/sparisi/.conda/envs/hab/lib/python3.7/site-packages/habitat_sim/bindings/__init__.py", line 10, in <module>
    from habitat_sim._ext.habitat_sim_bindings import (
ImportError: generic_type: type "MapStringString" is already registered!

I have seen that this error have been reported before and it is due to magnum, but I cannot find a fix.
Any help is appreciated, thanks!

You do not build from source, that -DEGL_LIBRARY flag is only needed when cmake isn't finding libegl, which isn't a thing when grabbing conda binaries.  It's an issue when building from source, not when grabbing binaries.
Your current issue is likely that magnum and habitat-sim are out of sync.  Likely just forcing a reinstall of conda binaries will fix that.
As for your original issue, as you are able to run the example, my guess is that the code you are adapting initializes CUDA before the RLEnv.  It's a known issue with CUDA and EGL that EGL needs to be initialized first if you want OpenGL and CUDA in the same process.  Using habitat.VectorEnv is another way to get around that as that creates fresh processes.

Thanks, VectorEnv works!


Unable to use Filebeat service due to hard-coded default.

Describe the bug
The Filebeat service definition has a hardcoded default of an output to an elasticsearch instance on localhost. If I want to have a filebeat output to logstash, however, the Filebeat service only supports one output and with my configured output and the default, the service will not start.
Steps To Reproduce
Steps to reproduce the behavior:
A configuration of:
services.filebeat = {
    enable = true;
    settings = {
      output = {
        logstash = {
          hosts = ["logstash5044"];
        };
      };
    };
    inputs = {
      journald.id = "everything";
      log = {
        enabled = true;
        paths = [
          "/var/log/*.log"
        ];
      };
    };

Generates the following file:
{
  "filebeat": {
    "inputs": [
      {
        "id": "everything",
        "type": "journald"
      },
      {
        "enabled": true,
        "paths": [
          "/var/log/*.log"
        ],
        "type": "log"
      }
    ],
    "modules": []
  },
  "output": {
    "elasticsearch": {
      "hosts": [
        "<IP_ADDRESS>:9200"
      ]
    },
    "logstash": {
      "hosts": [
        "logstash:5044"
      ]
    }
  }
}

and a config of
services.filebeat = {
    enable = true;
    settings = {
      output = {
        logstash = {
          hosts = ["logstash:5044"];
        };
        elasticsearch = {
          hosts = [];
        }
      };
    };
    inputs = {
      journald.id = "everything";
      log = {
        enabled = true;
        paths = [
          "/var/log/*.log"
        ];
      };
    };
  };

generates a config of
{
  "filebeat": {
    "inputs": [
      {
        "id": "everything",
        "type": "journald"
      },
      {
        "enabled": true,
        "paths": [
          "/var/log/*.log"
        ],
        "type": "log"
      }
    ],
    "modules": []
  },
  "output": {
    "elasticsearch": {
      "hosts": []
    },
    "logstash": {
      "hosts": [
        "logstash:5044"
      ]
    }
  }
}

Neither is a config that Filebeats will accept with an error of
Exiting: error unpacking config data: more than one namespace configured accessing 'output' (source:'/var/lib/filebeat/filebeat.yml')
Expected behavior
There shouldn't be a default so that a valid config file can be generated with an output other than ElasticSearch
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.71, NixOS, 23.11 (Tapir), 23.11.2774.3dc440faeee9`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - channels(root): `"nixos-23.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Same issue here, I tried to get a work arround with lib.mkForce due it also didn't work

@roberth is that intended behavior? What do we miss?

@SuperSandro2000
How do you normally deal with problems like this (maintainer not responding for a few weeks)? Removing the option as I described above would be a breaking change, so that's probably not a good solution. Is it possible to customize the options in some way (e. g. output becomes submodule, which is set to this value by default, but can be overridden) to have the ability to disable this option?

Then it is a breaking change. Sometimes it is unavoidable. We just need to make sure to add a release notes entry.


Fix checklist item render issue

Summary
This PR fixes issue with RHS Checklist item showing the due date of an item with the same name from another channel. It happened because items with the same name had the same key attribute values, and switching between channels did not render the list component.
Updated key generation logic, now its value is unique.
This recording shows the problem:
https://user-images.githubusercontent.com/4368372/167859688-d401720a-c88f-40d1-8ff8-8df0809f2797.mp4
Ticket Link
https://mattermost.atlassian.net/browse/MM-43936
Checklist

- [ ] Telemetry updated
- [ ] Gated by experimental feature flag

[ ] Unit tests updated

@gbochora, is there an opportunity to test this via e2e?

@gbochora, is there an opportunity to test this via e2e?

I've added an e2e test.