Add -10 and -30 values to resin counter

Describe your changes
Added more options to the resin counter. Moved the buttons to a constant at the top to allow for more values later down the line easily.
Issue or discord link
#2148
Testing/validation

Checklist before requesting a review (leave this PR as draft if any part of this list is not done.)

[ ] I have commented my code in hard-to understand areas.
[ ] I have made corresponding changes to README or wiki.
[ ] For front-end changes, I have updated the corresponding English translations.
[ ] I have run yarn run mini-ci locally to validate format and lint.
[ ] If I have added a new library or app, I have updated the deployment scripts to ignore changes as needed

Will go through the checklist in a moment.

I've added a new commit which should also address :  #2150
Evidence here:

Woops didn't mean to close 😅 . What are next steps to getting this merged? @frzyc you just handle that I assume ha!


Add a script to check for content consistency with router-api

This PR adds a script to check for content consistency across the content-store and router-api.
See alphagov/publishing-api#742 for the equivalent script in the publishing-api.

Would you be able to post an output of results for this? Code looks cool 👍

Some example output from me running it locally on my own database (therefore errors might not reflect production).
❯ bundle exec rake check_content_consistency:all
/alpha-taxonomy/33dcaade-6ba2-4010-841c-8e929017e975-hazel-dormice 😱
Backend ID (collections) does not match item rendering app ().
/administrative-appeals-tribunal-decisions/cpip-756-2016-ms-jacqueline-baylisvs-secretary-of-state 😱
Backend ID (specialist-frontend) does not match item rendering app ().
/access-finance-hbv-loan-funds 😱
Handler (redirect) does not match expected item handler (backend).
/accessiblebritain 😱
Handler (redirect) does not match expected item handler (backend).
/accidents-at-work 😱
Path (/accidents-at-work) was not found!
/accredited-electrical-and-electronic-equipment-producer-compliance-scheme-scotland 😱
Handler (redirect) does not match expected item handler (backend).
/active-belfast-grants 😱
Handler (redirect) does not match expected item handler (backend).
/advanced-learning-loans 😱
Handler (redirect) does not match expected item handler (backend).
/advanced-manufacturing-supply-chain-initiative 😱
Handler (redirect) does not match expected item handler (backend).
/advisory-services-fermanagh 😱
Handler (redirect) does not match expected item handler (backend).
/agricultural-and-forestry-processing-and-marketing-grant-scheme 😱
Handler (gone) does not match expected item handler (backend).
/alcohol-and-tobacco-excise-duty 😱
Handler (redirect) does not match expected item handler (backend).
Backend ID () does not match item rendering app (frontend).


Add Hrp::{as_bytes, as_str}

Fixes https://github.com/rust-bitcoin/rust-bech32/issues/143

Done. Sorry I have noticed too late that my colleague Alexander also did a PR.


Use pwd.getpwuid - Fix #1

As mentioned in this issue, went with a slightly different approach. Thanks for bringing this to my attention!


Remove spurious dependencies to org.json and git-commit-id-maven plugin.

Fixes #142.
New dependencies:
[INFO] org.odftoolkit:odfdom-java:jar:0.11.0-SNAPSHOT
[INFO] +- xerces:xercesImpl:jar:2.12.1:compile
[INFO] +- xalan:serializer:jar:2.7.2:compile
[INFO] +- junit:junit:jar:4.13.2:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.apache.jena:jena-core:jar:4.3.0:compile
[INFO] |  +- org.apache.jena:jena-base:jar:4.3.0:compile
[INFO] |  |  +- org.apache.jena:jena-shaded-guava:jar:4.3.0:compile
[INFO] |  |  +- org.apache.commons:commons-csv:jar:1.9.0:compile
[INFO] |  |  +- commons-io:commons-io:jar:2.11.0:compile
[INFO] |  |  +- commons-codec:commons-codec:jar:1.15:compile
[INFO] |  |  \- com.github.andrewoma.dexx:collection:jar:0.7:compile
[INFO] |  +- org.apache.jena:jena-iri:jar:4.3.0:compile
[INFO] |  \- commons-cli:commons-cli:jar:1.5.0:compile
[INFO] +- org.apache.jena:jena-core:jar:tests:4.3.0:test
[INFO] +- net.rootdev:java-rdfa:jar:1.0.0-BETA1:compile
[INFO] +- commons-validator:commons-validator:jar:1.7:compile
[INFO] |  +- commons-beanutils:commons-beanutils:jar:1.9.4:compile
[INFO] |  +- commons-digester:commons-digester:jar:2.1:compile
[INFO] |  +- commons-logging:commons-logging:jar:1.2:compile
[INFO] |  \- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.32:compile
[INFO] +- org.slf4j:slf4j-simple:jar:1.7.32:compile
[INFO] \- org.apache.commons:commons-compress:jar:1.21:compile

Aargh, this library actually depends on org.json, that dependency was not spurious. This has a license that is not OSI-compliant. I'll open a separate issue about this.

Issue about the dependency to org.json: https://github.com/tdf/odftoolkit/issues/144

We might change/add
io.github.git-commit-id
to a test dependency.
The reason for this dependency:
During building the current GIT commit hash is being extracted and placed into the manifest to be read out whenever a "java -jar odfdom.jar is being called to be certain when similar behaviour can be expected from ODFDOM JARs.
But yes, this GIT extraction functionality is not required during run-time. Indeed a bug!

Well, if the tests pass without this dependency, I do not think we need to add this artifact as a test dependency. Maven can use it as part of the build process already, since it is declared as a plugin.


Typed Builder implementation Part 4: Actions

[ ] Tests added for any new code
[ ] Ran make fmt-fix (or had formatting run automatically on all files edited)
[ ] Documentation added for any new functionality
[ ] Entry added to UNRELEASED.md for any new functionality

Codecov Report

Merging #1745 (549baf0) into unstable (88b0643) will decrease coverage by 0.06%.
The diff coverage is 50.00%.

:exclamation: Current head 549baf0 differs from pull request most recent head 5b025cc. Consider uploading reports for the commit 5b025cc to get more accurate results

@@             Coverage Diff              @@
##           unstable    #1745      +/-   ##
============================================
- Coverage     76.67%   76.60%   -0.07%
============================================
  Files           376      380       +4
  Lines         11749    11774      +25
  Branches        568      559       -9
============================================
+ Hits           9008     9019      +11
- Misses         2741     2755      +14

Impacted Files
Coverage Δ

.../forsyte/apalache/tla/typecomp/ScopedBuilder.scala
0.00% <0.00%> (ø)

...alache/tla/typecomp/subbuilder/ActionBuilder.scala
0.00% <0.00%> (ø)

...ache/tla/typecomp/unsafe/UnsafeActionBuilder.scala
0.00% <0.00%> (ø)

...apalache/tla/typecomp/TypeComputationFactory.scala
90.00% <100.00%> (+1.11%)
:arrow_up:

...tla/typecomp/signatures/ActionOperSignatures.scala
100.00% <100.00%> (ø)

...he/io/annotations/parser/CommentPreprocessor.scala
95.60% <0.00%> (-1.10%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88b0643...5b025cc. Read the comment docs.


PR: gh#146 : Changes for linux message handler

Changes for linux message handler

close this for the time being, we can leave the branch alive if we decide todo something in the future.


add code contribution

@mjt320 , @petravanhoudt Should we keep the data file required to run the Jupyter Notebook or remove it from the develop branch but keep it in LCB_CodeContribution?

I would be inclined to keep it with the notebook so that anyone finding the notebook can run it easily(?)

The notebook is titled "Chapter 13" - what is the origin of this contribution, and are other chapters available?

@fedorov This was a notebook originally created as supplementary material for a book on neuroimaging techniques and Chapter 13  pertains to perfusion techniques.

@mjt320 sounds good. I can continue to add the code from the notebook as functions on my branch (just like I did with the leakage correction, as it is all the same code). It shouldn't take long.


Convert dropdown directive without using eval-rst

Is your feature request related to a problem? Please describe.
When trying to convert a rubric directive, e.g.:
.. rubric:: Footnotes

.. [#f1] We purposefully do not provide advanced commands for crystal structure manipulation in AiiDA, because python packages that accomplish such tasks already exist (such as ASE or pymatgen).

rst2myst parse -f produces:
:::{rubric} Footnotes

:::

[^f1]: We purposefully do not provide advanced commands for crystal structure manipulation in AiiDA, because python packages that accomplish such tasks already exist (such as ASE or pymatgen).

This causes the build to fail with the following warning:
WARNING: toctree contains reference to document 'sections/fundamentals/basics' that doesn't have a title: no link will be generated

Proposed solution
I tried using the extended markdown syntax for footnotes, but maybe mystnb doesn't support this? If no support can be added, maybe a warning would be useful during parsing, since the warning returned during the build does not point towards the footnotes issue.

Heyo, footnotes are indeed supported. But this does not seem to be the root of your issue. As the warning say, it cannot find a header in sections/fundamentals/basics.
What does the top of the file look like? Does it have a header, i.e.
# My Header

Indeed, it does. But it seems I passed the wrong warning, this one seems more critical:
/Users/mbercx/python/code/aiida-tutorials/docs/sections/fundamentals/basics.md:1: WARNING: MyST-NB: Conversion to notebook failed: list index out of range

Maybe this is related to the fact that ..dropdown directives seem to be wrapped in eval-rst:
```{eval-rst}
.. dropdown:: Defining a structure and storing it in the database
...

Indeed, moving the offending footnote out of the eval-rst fixes the issue as well. I also tried putting the footnote elsewhere before conversion and this worked, however, only if I leave 1 line empty after the footnote. Two lines or more leads in the index out of range error above... 🤔 I swear, I'm not going insane. 😅
@chrisjsewell maybe try the conversion with the basics.rst file and have a look. You can find it here:
https://filedn.com/lsOzB8TTUIDz2WkFj8o6qhp/basics.rst

Ok cheers will do, last question, what version of myst-nb, myst-parser and markdown-it-py (well if you just want to paste the output of pip freeze)

It's quite possibly because you are using a version before this bug fix: https://github.com/executablebooks/markdown-it-py/commit/a40f6a80f84f2c02e999c6b77f8112c050f8cbe6
I'm basically going to release a myst-no with this in imminently (you can even try myst-nb==0.13.0a1)

Here's the pip freeze:
pip freeze
alabaster==0.7.12
anyio==3.0.1
appdirs==1.4.4
appnope==0.1.2
argon2-cffi==20.1.0
async-generator==1.10
attrs==20.3.0
Babel==2.9.1
backcall==0.2.0
beautifulsoup4==4.9.3
black==21.5b1
bleach==3.3.0
certifi==2020.12.5
cffi==1.14.5
cfgv==3.2.0
chardet==4.0.0
click==7.1.2
colorama==0.4.4
decorator==5.0.7
defusedxml==0.7.1
distlib==0.3.1
docutils==0.15
entrypoints==0.3
filelock==3.0.12
gitdb==4.0.7
GitPython==3.1.17
identify==2.2.4
idna==2.10
imagesize==1.2.0
importlib-metadata==4.0.1
importlib-resources==3.3.1
ipykernel==5.5.5
ipython==7.23.1
ipython-genutils==0.2.0
ipywidgets==7.6.3
jedi==0.18.0
Jinja2==3.0.0
jsonschema==3.2.0
jupyter-cache==0.4.2
jupyter-client==6.1.12
jupyter-core==4.7.1
jupyter-server==1.7.0
jupyter-server-mathjax==0.2.2
jupyter-sphinx==0.3.2
jupyterlab-widgets==1.0.0
markdown-it-py==0.6.2
MarkupSafe==2.0.0
matplotlib-inline==0.1.2
mdit-py-plugins==0.2.6
mistune==0.8.4
mypy-extensions==0.4.3
myst-nb==0.12.3
myst-parser==0.13.7
nbclient==0.5.3
nbconvert==5.6.1
nbdime==3.0.0
nbformat==5.1.3
nest-asyncio==1.5.1
nodeenv==1.6.0
notebook==6.3.0
packaging==20.9
pandocfilters==1.4.3
parso==0.8.2
pathspec==0.8.1
pexpect==4.8.0
pickleshare==0.7.5
pre-commit==2.12.1
prometheus-client==0.10.1
prompt-toolkit==3.0.18
ptyprocess==0.7.0
pycparser==2.20
pydata-sphinx-theme==0.6.3
Pygments==2.9.0
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2021.1
PyYAML==5.4.1
pyzmq==22.0.3
regex==2021.4.4
requests==2.25.1
rst-to-myst==0.1.2
Send2Trash==1.5.0
six==1.16.0
smmap==4.0.0
sniffio==1.2.0
snowballstemmer==2.1.0
soupsieve==2.2.1
Sphinx==3.5.4
sphinx-book-theme==0.1.0
sphinx-copybutton==0.3.1
sphinx-panels==0.5.2
sphinx-togglebutton==0.2.3
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
SQLAlchemy==1.3.24
terminado==0.9.5
testpath==0.4.4
toml==0.10.2
tornado==6.1
traitlets==5.0.5
urllib3==1.26.4
virtualenv==20.4.6
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.59.0
widgetsnbextension==3.5.1
zipp==3.4.1

Will try the pre-release, thanks! 👍

I'm basically going to release a myst-nb with this in imminently (you can even try myst-nb==0.13.0a1)

With the prerelease I no longer run into the "index out of range" issue, and the document is built. 👍  The number of empty lines at the end of the file also no longer have any influence (phew 😅 ). So when the footnote is outside the dropdown, it works perfectly. 👌
When the footnote is inside the dropdown, it doesn't get converted (being in the rst-eval) and I get the following warning:
/Users/mbercx/python/code/aiida-tutorials/docs/sections/fundamentals/basics.md:1622: WARNING: Footnote [#] is not referenced.

So basically the issue boils down to the fact that:
.. dropdown:: Dropdown

    Hidden content

Gets converted into:
```{eval-rst}
.. dropdown:: Dropdown

    Hidden content
```#

And not
```{dropdown} Dropdown
Hidden content
```#

I suppose there is some reason for this, but this causes problems with the footnotes, since it seems the markdown footnotes can't be referenced from within the eval-rst directive.

Note, if you read the advanced instructions, you can change how dropdown is converted with the -c/--conversions option


os/linux_arm_64/decompile does not exist

Build Date: 2021-Dec-10 1523 EST
Ghidra Version: 10.1
Java Home: /usr/lib/jvm/java-11-openjdk-arm64
JVM Version: Debian 11.0.12
OS: Linux 4.19.0-17-arm64 aarch64
Workstation: debian-gnu-linux-10.localdomain

Currently running on an arm machine Linux debian-gnu-linux-10 4.19.0-17-arm64 #1 SMP Debian 4.19.194-3 (2021-07-18) aarch64 GNU/Linux. When trying to decompile the binaries (no matter is a x86 or arm) this error will show up and the decompilation fails. Would like to know is there a workaround to solve this issue? Thank you.
screenshot:

https://htmlpreview.github.io/?https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_10.1_build/GhidraDocs/InstallationGuide.html#Build

https://htmlpreview.github.io/?https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_10.1_build/GhidraDocs/InstallationGuide.html#Build

thanks so much man! sorry didn't check this

hello @ryanmkurtz
In the middle of the building process I encountered with this same issue then I tried to locate gradle/support/fetchDependencies.gradle this file but could not find on the 7.3 version of Gradle
Could you please give me a hint? Thank you!

URL: https://github.com/pxb1988/dex2jar/releases/download/2.0/dex-tools-2.0.zip
   Connecting...
   Downloading: 2362460 of 2362460 (100%)
URL: https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/android4me/AXMLPrinter2.jar
   Connecting...cripts
   Connection error! java.net.UnknownHostException: storage.googleapis.com
   Connecting (2/3)...
   Connection error! java.net.UnknownHostException: storage.googleapis.com
   Connecting (3/3)...
   Connection error! java.net.UnknownHostException: storage.googleapis.com

FAILURE: Build failed with an exception.

seems that it is because of the connection issue then I tried with myself, does not connect either.
wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/android4me/AXMLPrinter2.jar -P /tmp
--2021-12-11 23:10:56--  https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/android4me/AXMLPrinter2.jar
Resolving storage.googleapis.com (storage.googleapis.com)... failed: Name or service not known.
wget: unable to resolve host address ‘storage.googleapis.com’

I wonder do we have another work around? thanks!

Successfully built. Appreciate the hard work! yyds Ghidra team

@ryanmkurtz Is there any plans for eventually release official Linux ARM 64-bit binaries? Just wondering if I should go to the trouble of setting up automated builds for myself, or wait it out. =)

No, we have no plans to build and distribute those binaries.


AsyncData of child page is unable to access parent's page's props

Version
v2.10.2
Reproduction link
http://www.nourl.com
Steps to reproduce
Note that I am already aware of this:
https://nuxtjs.org/guide/async-data --> "You do NOT have access of the component instance through this inside asyncData because it is called before initiating the component."
However, I have a child page that I've passed a prop. I can't access the prop from asyncData of the child page.
  props: {
    searchQuery: {
      type: String,
      required: true
    },
  },
  computed: {
    getFeedUrl() {
      return '/' + process.env.backendPublicApiPrefix + '/search?q=' + this.searchQuery;
    },
  },

  async asyncData (context) {
    context.store.dispatch('resetPostFeed');
    let searchQuery = (context.route.query && context.route.query.q) ? context.route.query.q : '';
    await context.store.dispatch('retrievePosts', {'feedUrl': '/' + process.env.backendPublicApiPrefix + '/search?q=' + searchQuery, 'selection': 'alltime'} );
  },

As a more simplified example, I am trying to access 'searchQuery' prop in asyncData, but i'm not sure if I can get it from the context object.
The parent page does this:
              <nuxt-child
              :key="$route.params.slug"
              searchQuery="aaaaaaaaa"
              />

and in my child page (not component) I do this:
mounted() {
  console.log('mounted: searchQuery', this.searchQuery);
},
  async asyncData (context) {
console.log('asyncData: searchQuery', context.searchQuery);
}

The console.logs show this:
asyncData: searchQuery undefined
mounted: searchQuery aaaaaaaaa

Seems like this should be possible, but why is it not available from the context object?
What is expected ?
Seems like this should be possible, but why is it not available from the context object?
What is actually happening?
Props are inaccessible from AsyncData function of child pages

This bug report is available on Nuxt community (#c9947)

ping to keep this open

Still an issue


Add str.encode for utf-8

Codecov Report

Merging #901 into master will decrease coverage by 0.01%.
The diff coverage is 55.17%.

@@            Coverage Diff             @@
##           master     #901      +/-   ##
==========================================
- Coverage   64.02%   64.01%   -0.02%
==========================================
  Files          88       88
  Lines       14915    14938      +23
  Branches     3319     3326       +7
==========================================
+ Hits         9549     9562      +13
- Misses       3169     3172       +3
- Partials     2197     2204       +7

Impacted Files
Coverage Δ

vm/src/obj/objbyteinner.rs
69.32% <50%> (-0.34%)
:arrow_down:

vm/src/obj/objstr.rs
74% <57.14%> (-0.39%)
:arrow_down:

vm/src/function.rs
51.38% <60%> (+0.2%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08b716f...5acf3ed. Read the comment docs.

@youknowone I guess this PR looks pretty good. Could you have a look at the conflicted file? Then I could merge this.


Backfill backwards compatible auth_bypass_id

Trello: https://trello.com/c/IQ26eiaW/240-make-content-publisher-use-changing-auth-bypass-tokens
This is marked as draft until https://github.com/alphagov/content-publisher/pull/2105 has been merged and deployed to production.
This PR backfills the editions table to populate auth_bypass_id values for every row. Once this migration is complete all entries will have an auth_bypass_id so we can also set it with a NOT NULL db constraint.
In the next PR for this I'll make use of this column in code.

Thanks @benthorner 👍


Running multiple tasks on different instances using JDBC lock provider

Hi @lukas-krecan ,
What is [locked_by] [varchar](255) NOT NULL field in the table for the JDBC implementation? Is it possible to set this to an instance name or node name from the application, or make it a composite key (alongside [name] [varchar](64) NOT NULL field)?
I guess I am exploring to see the option where 2-3 tasks are using ShedLock (different lock name for each task) in a 4+ node environment and making sure that not all tasks are running on the same node (CPU/memory/load consideration)

Hi, this is not a feature that ShedLock supports out of the box.
If you are able to figure out naming scheme that would satisfy you requirements, you can generate task names dynamically using Spring Expression language in the annotation. In other words, you can add instance name to the name of the task in the runtime.
If you need something more complex, you can always implement your own lock provider.

Hi,
Thanks for your response. Just a follow up question around the same:-
Is it possible know if this node acquiring lock for task A does not already have a lock for tasks B or C, etc. is all the locks for tasks A, B, C, etc. are via ShedLock?
Appreciate your response :)

For that you would have to implement a custom LockProvider. But I think it would be better to use some job scheduler like Quartz


[marathon-lb] Fix defaults, permit image config.

This adds a new config item to change the Docker image for marathon-lb,
and also changes the default role back to slave_public. Due to the way
mustache templates work and how Marathon handles roles, it's very
difficult to specify 'no role' as the default without adding an addition
config parameter to wrap the role.
cc @discordianfish

Makes sense, thanks!
One question though: Should all json files be have keys ordered alphabetic? If so, we can also add that to the git hooks.

About the JSON: I don't know, but I used dcos package describe --config marathon-lb > repo/packages/M/marathon-lb/0/config.json to validate the config and update the file.


Configurable IP address and port for discovery

See issue #192.
How to test it:
Run the example/publisher with:
IGN_DISCOVERY_MULTICAST_IP=<IP_ADDRESS> IGN_DISCOVERY_MSG_PORT=11000 IGN_DISCOVERY_SRV_PORT=11001 IGN_VERBOSE=1 ./publisher

And the example/subscriber with:
IGN_DISCOVERY_MSG_PORT=11000 IGN_DISCOVERY_SRV_PORT=11001 ./subscriber

You should receive the messages as expected.


Incorrect reference to C procedure

Hi Philipp,
In sdl2_image.f90, line 27, you specify the c procedure binding label IMG_LoadTexture

function img_load(file_name) bind(c, name='IMG_LoadTexture')

If I'm not mistaken the correct c procedure binding should be IMG_Load?
Great work btw. I enjoy playing around with it.
Best wishes
Sebastian

Thank you for pointing this out. Has been fixed with commit 31d6884c700564989993282f8e404cb4f9d3c25f.


Add project incubation entry considerations

Signed-off-by: Tracy Kuhrt<EMAIL_ADDRESS>

Thanks @lehors for the feedback. Updated as requested.


Updated poms to docker-maven-plugin 0.12.0

Fixes #484

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/fabric8-quickstarts-master-pullreq/545/
Test PASSed.


Add support for GraphQL

Update constantly the location of the markers in the App requires real-time capabilities, the team have decided in Hasura's subscriptions so we will need GraphQL support. In the past Apollo Client has worked fine for us so we can continue using it, but there is a discussion to decide.
Checklist

[x] Request client and related packages are installed and don't break the app
[ ] Client can call graphQL queries
[ ] Client can call graphQL mutations
[ ] Client can call graphQL subscriptions
[ ] Client have cache configured per case using correct cache policy (optional at this point)

Depends on

#1
https://github.com/Azordev/dasher-user/discussions/9

In a meeting with @denisosuna we decided on Apollo

I watch the current code on a browser  i see some points on a map provided by google, are those points related with Hasuras Suscriptions?
i research about Hasuras and its the one that will allow the location to be live shared with the user, right?

I `ve watched the current code on a browser i see some points on a map provided by google, are those points related with Hasuras Suscriptions? Are Hasuras Suscriptions method the one that will allow the location to be live shared with the user, right?

Current location points come from a static file. In the future #20 should use Subscription to get those location points


logging bug: logged decision variable is scaled

as such, the evaluated function outputs are also done with the wrong inputs (scaled inputs)
see here

it's a little complicated, see here

recording decision variable: we want unscaled
call batch_forward: we want unscaled
call constraints: we want scaled

an additional issue: when the larger infeasibility is cyclic cons, the debugger gives wrong name and location of the maximum infeasibility (it gives the constraint that has the larger violation among the group excluding cyclic cons)
this is because cyclic cons are not recorded, to record, we can add the following snippet to the logging here:

closed by #79


iterm2 theme does not import

Unable to import the themes into iterm2

I ended up using https://github.com/enkia/tokyo-night-vscode-theme/blob/master/tokyo-night.itermcolors


docker connectivity problem

Describe the bug
Hi,
i am experiencing problems while trying to access a kafka broker from my kafka consumer when running in the same docker setup. Manually connecting to the kafka broker from inside the container works, so i believe the containers are running fine.
However, trying to access the kafka broker with my kafkajs consumer does not work. The problem only occurs when running inside docker.
Here is my docker file:
version: '3.3'
services:
  zookeeper:
    image: wurstmeister/zookeeper:3.4.6
    ports:
      - 2181:2181

  kafka:
    image: wurstmeister/kafka:2.11-2.0.0
    hostname: kafka
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    expose:
      - 9093
    environment:
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093,OUTSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_LISTENERS: INSIDE://<IP_ADDRESS>:9093,OUTSIDE://<IP_ADDRESS>:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_CREATE_TOPICS: "my-topic:1:1"

  my-kafkajs-consumer-app:
    build:
      context: .
      dockerfile: Dockerfile
    depends_on:
      - kafka
    ports:
      - 3149:3149
    environment:
      - KAFKA_BROKER_ADDRESS_LIST=kafka:9093

After running docker-compose up i got the following error message:
[ServerKafka] ERROR [Connection] Response Metadata(key: 3, version: 5) {"timestamp":"2020-07-02T05:02:02.145Z","logger":"kafkajs","broker":"kafka:9093","clientId":"  my-kafkajs-consumer-app-server","error":"This server does not host this topic-partition","correlationId":2,"size":147}

However executing the following from inside a running   my-kafkajs-consumer-app container succeeds:
root@8395504eae10:/usr/src/app# kafkacat -b kafka:9093 -L
Metadata for all topics (from broker 1001: kafka:9093/1001):
 1 brokers:
  broker 1001 at kafka:9093
 2 topics:
  topic "my-topic" with 1 partitions:
    partition 0, leader 1001, replicas: 1001, isrs: 1001

Any ideas what could be the problem here, i guess it is related to some networking issues. Using this docker setup for some java services seems to work.

It seems to work for me. I copied your docker-compose file verbatim and took the consumer from examples/consumer.js, modifying it just slightly so that it receives the brokers from the KAFKA_BROKER_ADDRESS_LIST and removed all the SSL/SASL stuff, and everything just worked. Of course, I wasn't waiting for the brokers to come online, so initially the connection failed, but starting the app container after the broker was up, it connected and consumed just fine.

@Nevon  Can you elaborate about waiting for brokers to come online? I'm facing this issue.

@Nevon Can you elaborate about waiting for brokers to come online? I'm facing this issue.

Docker Compose does not know when your container is ready to receive connections. It will just start the container, and as soon as the container is started it will start your application container. There's no guarantee that the Kafka brokers are ready to receive connections at that time.
For our test environment, we wait for the Kafka broker to be ready before proceeding with the tests:
https://github.com/tulios/kafkajs/blob/c6cc35deb21969a857ea2c819658c7c579bc5a1e/scripts/waitForKafka.js#L22-L28
This is an extremely common issue that there are many solutions for online, such as dockerize and wait-for-it.


Issue 567 - Add ability to sort Failure Rates list by failure type

Closes #567
Clicking on each of the error type headers in any of the columns will sort the data by that error type over the associated time period.  Clicking on "Failure %" will sort the data by failure percentage.  Default sort is desc, but clicking on each header multiple times will toggle between asc and desc. A directional caret icon is displayed next to the label currently being used for sorting.
Right now this sorting is not persisted through the URL. Please comment if that is necessary.

Updated to persist sort params through URL


ColBert Retrieval

Hello
Thank you again for your generous code release 😊
I tried to follow your instruction here, downloaded the cached data.
 I noticed that there are many json files in the unzipped cached file. Does each  json file correspond to the retrieval result to a question? Is the json file named with its question id or anything meaningful?
If I would like to run the ColBERT experiment, do I only need to change the "retriever" field in config file from serp into sth like colbert?
If I want to try out some new retrieval method and result, should I retrieve by myself somewhere else beforehand and probably generate a folder like the cached dir, store all retrieval results there, then change the decomposition.main_retriever_dir field in the config to be pointing to the new folder which contains my new retrieval result?

p.s. just a friendly reminder of a small typo at the bottom of the page:
At the end of the sentence should it be randomize_retrieval and retrieve_at_10 instead of randomize_retrieval and andomize_retrieval?
...
RetRobust experiments require configuring the following fields:
... To run a single setting, use the randomize_retrieval and retrieve_at_10 fields.
Tks again!

Hi, thanks for the questions!

Each cache file indeed corresponds to a question, it is simply a hash over the question (see the get_string_hash method). For @1 retrieval, simply set the randomize_retrieval and retrieve_at_10 flags as false. For @10 retrieval, set the retrieve_at_10 as false, this well cause the model to use the lower ranked retrieval result. For random retrieval, set randomize_retrieval as true and run_output_dir should be the dir with the retrieval result you want to randomly sample from. To run all three settings in one run, simply set settings field to ["reg", "random", "@10"], and this will update those field on the flight (see lines 554-571 in run.py). I hope this makes sense :)
Yes, simply change the retriever to colbert! This will cause the retriever to use the get_question_wiki_snippet_colbert method. I hope the ColBERT server I was using is still available, otherwise plz let me know if there are any problems (you can always use SerpAPI, the first calls are free).
Sure, that could work! You take a look at lines 181-189 in serpapi.py to see how we currently differentiate between colbert and serp, it simply requires updating the methods to call the retriever and cache the results.

Thx so much for catching the typo, will fix! And plz let me know if there are still issues :)


Implement the find command to support date

Format: find KEYWORD [MORE KEYWORDS]
Examples:

find 23
It will return tasks with 23 in its date.

Refer to issue #220

NA anymore. Planned not to do it.


Can't draw more than 140 texts

Here I minified the code as much as I could.
require "vips"
image = Vips::Image.new_from_file "bg_l10.jpg"

140.times do
  text = Vips::Image.text "M", width: image.width - 10, dpi: 1000
  text = text.new_from_image([192, 192, 192]).copy(interpretation: :srgb).bandjoin(text)
  image = image.composite2(text, :over, x: 0, y: 0).flatten
end

image.write_to_file "temp.png"

puts "OK"

If I change 140 to 150 I get:
Illegal instruction: 4

Here I attach the "bg_l10.jpg" but you might be able to reproduce it with blank image.

Oh, I've figured out how to use composite:
t = 600.times.map do
  text = Vips::Image.text "M", width: image.width - 10, dpi: 1000
  text = text.new_from_image([192, 192, 192]).copy(interpretation: :srgb).bandjoin(text)
  text
end

image = image.composite(t, :over, x: (1..600).to_a, y: (1..600).to_a).flatten

Yes, you're getting a stack overflow in the first one, I think.


Request ID displays in left navigation

As an IAO analyst or ministry coordinator
I want the Request ID to persist in the left nav bar
so that I can quickly see what request I am on

Assumptions & Scope
What are the assumptions for this story?
Users will have multiple request open in different browser tabs at one time. They need a quick reference on the page so that they know what request they are on.
This is the top priority item for the Flex Team.
What is IN scope?
ID should be visible on any view in the request (request view, CFR form, Attachments, Comments)
What is NOT in scope?
Browser tab display of request number is covered in #2318
Acceptance Criteria
Scenario 1: Request ID (MINISTRY-YEAR-12345) in left nav

GIVEN an IAO analyst or ministry coordinator is logged in
WHEN the user is on the following views: Request , Attachments, Comment or Fees
THEN the Request ID will appear in the left nav above the state dropdown menu
https://jacklyn808742.invisionapp.com/console/share/RSBDPXJGKAW/922730564

Dependencies? What is the impact of this dependency? (If so, link dependency in the ticket, make it visible in a team´s backlog)
Validation Rules? (If yes, list here)
Design
https://jacklyn808742.invisionapp.com/console/share/RSBDPXJGKAW/922730564
Definition of Ready

[ ] Is there a well articulated User Story?
[ ] Is there Acceptance Criteria that covers all scenarios (happy/sad paths)?
[ ] If there is a user interface, is there a design?
[ ] Does the user story need user research/validation?
[ ] Does this User Story needs stakeholder approval?
[ ] Design / Solution accepted by Product Owner
[ ] Is this user story small enough to be completed in a Sprint? Should it be split?
[ ] Are the dependencies known/ understood? (technical, business, regulatory/policy)
[ ] Has the story been estimated?

Definition of Done

[ ] Passes developer unit tests
[ ] Passes peer code review
[ ] If there's a user interface, passes UX assurance
[ ] Passes QA of Acceptance Criteria with verification in Dev and Test
[ ] Confirm Test cases built and succeeding
[ ] No regression test failures
[ ] Test coverage acceptable by Product Owner
[ ] Ticket ready to be merged to master or story branch
[ ] Developer to list Config changes/ Update documents and designs
[ ] Can be demoed in Sprint Review
[ ] Tagged as part of a Release
[ ] Feature flagged if required
[ ] Change Management activities done?

Can I suggest we not have the # preceding the request ID? I think that's probably the intention, but just wanted to make sure it doesn't get developed that way :)
cc: @JHarrietha-AOT

need screen resolution from ticket #1993 in order to make sure sidebar isn't too crowded in small screens


feat: Single metric sort

SUMMARY
Add sort_by_metric flag for charts with single metric
After
legacy-plugin-chart-heatmap

legacy-plugin-chart-sunburst

legacy-plugin-chart-force-directed

legacy-plugin-chart-chord

legacy-plugin-world-map

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[x] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #13058 (5e0aa34) into master (2ce7982) will increase coverage by 13.75%.
The diff coverage is 63.04%.

@@             Coverage Diff             @@
##           master   #13058       +/-   ##
===========================================
+ Coverage   53.06%   66.81%   +13.75%
===========================================
  Files         489      490        +1
  Lines       17314    28834    +11520
  Branches     4482        0     -4482
===========================================
+ Hits         9187    19265    +10078
- Misses       8127     9569     +1442

Flag
Coverage Δ

cypress
?

python
66.81% <63.04%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

superset/config.py
90.64% <ø> (ø)

...s/260bf0649a77_migrate_x_dateunit_in_time_range.py
0.00% <0.00%> (ø)

superset/viz.py
58.82% <20.00%> (ø)

superset/charts/commands/exceptions.py
92.85% <77.77%> (ø)

superset/db_engine_specs/elasticsearch.py
89.74% <93.75%> (ø)

superset/utils/date_parser.py
96.87% <100.00%> (ø)

superset-frontend/src/views/CRUD/data/hooks.ts

superset-frontend/src/components/EditableTitle.tsx

superset-frontend/src/setup/setupFormatters.ts

superset-frontend/src/utils/common.js

... and 966 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e0681b...5e0aa34. Read the comment docs.

@maloun96 thanks for the PR and dedicated work, Victor! when I pulled for testing, changes are not showing, did I miss something?

@maloun96 thanks for the PR and dedicated work, Victor! when I pulled for testing, changes are not showing, did I miss something?

@junlincc You have to link locally all the plugins with npm link

LGTM functionality wise. We should tweak UI when we implement drag and drop query, will also need special treatment for sort-by single metric.

https://www.figma.com/file/JWaGztdhZS0kS5ruG7x9tB/Control-Panel?node-id=195%3A36068

Merging to unblock #13100


Feature Improvement Request: Simplifying Documentation URL Format and Adding Latest Version Link

I hope this message finds you well. I'm writing to propose some enhancements to the documentation URL format for the kellnr projects. Currently, the documentation URL follows this format:
http://kellnr.rmds.local/docs/{my-crate-name}/{my-crate-version}/doc/{my-crate-name}/index.html

While functional, I believe there's room for improvement for better usability and maintenance.
Simplification of URL Format: The current URL format redundantly includes the package name {my-crate-name} twice. This duplication doesn't add value and unnecessarily complicates the URL structure. I suggest simplifying the URL to a more direct format to access the documentation:
http://kellnr.rmds.local/docs/{my-crate-name}/{my-crate-version}/index.html

This modification streamlines the URL, making it more intuitive and user-friendly.
Addition of Latest Version Link: Another observation is the lack of a direct reference to the latest version of the crate's documentation. Including a link to the latest version documentation is essential for users seeking the most up-to-date information without having to navigate through specific version URLs manually. I propose adding a symbolic link or a redirect to the latest version documentation to facilitate easy access.
I understand that these changes may introduce some breaking adjustments to the URL format. However, the benefits of improved usability and accessibility outweigh the inconvenience of transition.
I would appreciate your thoughts on this proposal, including any additional considerations or ideas for implementation. Thank you for considering these suggestions, and I look forward to your feedback.

regarding the "crate name duplication":
this is actually not a duplication. This second name is actually the target- or package-name and can differ from the crate name.
Example: https://docs.rs/futures-util/latest/futures_util/
The target name is also the name of the top level folder in the rustdoc output.
So I don't think it's a good idea to work around that separation.
/latest/ is one though, but some work.

regarding the "crate name duplication":
this is actually not a duplication. This second name is actually the target- or package-name and can differ from the crate name.
Example: https://docs.rs/futures-util/latest/futures_util/

Good point, then we still need to allow https://docs.rs/futures-util/latest to https://docs.rs/futures-util/latest/futures_util/

I'm not sure if non-canonical names are still a thing. There was this blog post some time ago: https://blog.rust-lang.org/2023/10/27/crates-io-non-canonical-downloads.html
But even, if the package name differs from the crate name, we could choose one as the identifier and skip the other in the URL or do I miss something?

API available under /api/v1/docs/<package>/latest.


[WIP] Adding a header

I meant to open this against the https://github.com/pdxgo/hugo-goa fork. Closing now, sorry!


Add resources page

This PR adds a pages with link to resources to read up on DLC.
I just put the blog posts in chronological orders, but I can reorder if someone has a better suggestion.
Also I think at some point we should add links to implementations, but it might be better to wait until we have compatibility.

I’d agree on waiting until there is compatibility personally.
On Tue, Mar 10, 2020 at 9:05 PM Thibaut Le Guilly<EMAIL_ADDRESS>wrote:

This PR adds a pages with link to resources to read up on DLC.
I just put the blog posts in chronological orders, but I can reorder if
someone has a better suggestion.
Also I think at some point we should add links to implementations, but it
might be better to wait until we have compatibility.
You can view, comment on, or merge this pull request online at:
https://github.com/discreetlogcontracts/dlcspecs/pull/31
Commit Summary

Add resources page

File Changes

A Resources.md
https://github.com/discreetlogcontracts/dlcspecs/pull/31/files#diff-4c14d9f87236b8e25a765c5b379737bf
(19)

Patch Links:

https://github.com/discreetlogcontracts/dlcspecs/pull/31.patch
https://github.com/discreetlogcontracts/dlcspecs/pull/31.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/discreetlogcontracts/dlcspecs/pull/31?email_source=notifications&email_token=AJS2WY5GPKNP4YOK23LKJYTRG3WV3A5CNFSM4LFLYYTKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IUCIG6Q,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJS2WY7J7AORC7PRHTEELPTRG3WV3ANCNFSM4LFLYYTA
.

--
Dan Smith
<EMAIL_ADDRESS>Director of Product and Operations
Suredbits.com - A Blockchain Company


Does ros2-for-unity work for UWP?

I am looking for a way to send and receive Ros2 messages via HoloLens2.

hello @ookayt , i have the same question, i had tested it by copying my standalone build ( made in ubuntu ) but it didn't work, maybe if i had build at windows it could.
However unity-robotics-hub (which requires a bridge) works ok at hololens2


Keymap descriptions

Hi! I've notice that the keymap descriptions were defined for the command config but not passed to the keymaps. When using the default config and leader menu, the key would appear, but not the description.
Here's an example in LazyVim :

A quickfix is to reuse the command description, which seems just fine. Could also be something shorter like Split/Join for the toggle? I just thought that would be nice to have something there, out of the box.
Great plugin by way! Seems very useful and enjoyable so far!

Thanks for contribution!


zstd compression-decompression sometimes corrupts the HTTP response

Title: zstd compression-decompression sometimes corrupts the HTTP response
Description:
We have a workflow that can be summarised as:
downstream ------> envoy A ------> envoy B ------> upstream
Here:

The communication between all entities is HTTP 1.1
Downstream to envoy A and envoy B to upstream is plain HTTP
envoy A to envoy B is over TLS
Envoy A sets the "Accept-Encoding: gzip, zstd" header (through the Zstd, gzip decompressor filters) and subsequently decompresses the body
Envoy B compresses the http response body (through the Zstd compressor filter)

Rarely (in the single digit percent), the body we get on the downstream is different to the one that was sent from the upstream. I say different because I am not sure if there are just bytes missing or it gets corrupted. This happens only with zstd - when I leave only the gzip filter, the problem disappears.
Since this happens rarely and for 1-2MB responses, I am yet to get my hands on one of the responses. Will upload one if I can.
Repro steps:
The two envoys above are configured with the "default" zstd decompressor and compressor config.
The decompressor:
- name: envoy.filters.http.decompressor
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.http.decompressor.v3.Decompressor
                response_direction_config:
                  common_config:
                    enabled:
                      default_value: true
                      runtime_key: zstd_response_decompressor
                decompressor_library:
                  name: zstd_decompressor
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.compression.zstd.decompressor.v3.Zstd

The compressor:
{
  "name": "envoy.filters.http.compressor.zstd",
  "typedConfig": {
    "@type": "type.googleapis.com/envoy.extensions.filters.http.compressor.v3.Compressor",
    "chooseFirst": true,
    "compressorLibrary": {
      "name": "zstd_compressor",
      "typedConfig": {
        "@type": "type.googleapis.com/envoy.extensions.compression.zstd.compressor.v3.Zstd",
        "compressionLevel": 1,
        "strategy": "FAST"
      }
    },
    "responseDirectionConfig": {
      "removeAcceptEncodingHeader": true
    }
  }
}

Logs:
Here is snippet from the logs on debug:
[Originator@6876 sub=http] [C1417][S12670642521859171609] request headers complete (end_stream=false):
[Originator@6876 sub=filter] [C1417][S12670642521859171609] DecompressorFilter::decodeHeaders advertise Accept-Encoding with value 'gzip'
[Originator@6876 sub=filter] [C1417][S12670642521859171609] DecompressorFilter::decodeHeaders: ':authority', 'localhost'
[Originator@6876 sub=filter] [C1417][S12670642521859171609] DecompressorFilter::decodeHeaders advertise Accept-Encoding with value 'gzip,zstd'
[Originator@6876 sub=filter] [C1417][S12670642521859171609] DecompressorFilter::decodeHeaders: ':authority', 'localhost'
[Originator@6876 sub=router] [C1417][S12670642521859171609] cluster REDACTED match for URL REDACTED
[Originator@6876 sub=router] [C1417][S12670642521859171609] router decoding headers:
[Originator@6876 sub=router] [C1417][S12670642521859171609] pool ready
[Originator@6876 sub=http] [C1417][S12670642521859171609] request end stream
[Originator@6876 sub=router] [C1417][S12670642521859171609] upstream headers complete: end_stream=false
[Originator@6876 sub=filter] [C1417][S12670642521859171609] DecompressorFilter::encodeHeaders: ':status', '200'
[Originator@6876 sub=filter] [C1417][S12670642521859171609] DecompressorFilter::encodeHeaders: ':status', '200'
[Originator@6876 sub=http] [C1417][S12670642521859171609] encoding headers via codec (end_stream=false):
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 6321 bytes to 131072 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 2523 bytes to 262144 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 331 bytes to 131072 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 6738 bytes to 393216 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 1751 bytes to 262144 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 1362 bytes to 131072 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 551 bytes to 131072 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 353 bytes to 131072 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 8713 bytes to 917504 bytes
[Originator@6876 sub=filter] [C1417][S12670642521859171609] response data decompressed from 0 bytes to 0 bytes
[Originator@6876 sub=http] [C1417][S12670642521859171609] encoding trailers via codec:
[Originator@6876 sub=http] [C1417][S12670642521859171609] Codec completed encoding stream.

In that particular case, the upstream sent 2'499'159 bytes, but the downstream got 2'490'368. One of the decompression messages claims some 396x compression ratio, but that is in line with other similar responses that were intact.

Adding @rainingmaster @mattklein123 as codeowners

Do these request use chunked transfer-encoding or content-length?

The responses from envoy B to envoy A to the downstream in the above are always with transfer-encoding. For the response from the upstream - I am almost sure it's always content-length, but I can check if it would help?

Upstream responds with transfer-encoding as well

Ugh, sorry folks, I think this will turn out to be:
- area: decompression
  change: |
    fixed CVE-2022-29225: Decompressors can be zip bombed. Previously decompressors were susceptible to memory inflation in
    takes in which specially crafted payloads could cause a large amount of memory usage by Envoy. The max inflation payload
    size is now limited. This change can be reverted via the
    ``envoy.reloadable_features.enable_compression_bomb_protection`` runtime flag.

The zstd errors for reaching the limit show only on trace. There is also no documentation in the decompressor filter about this flag and only gzip has a config option for setting the max inflation ratio.
I have re-enabled zstd compression in our test pipelines while disabling this runtime flag and will let you know how it goes.


bump version: 2.8.0

Coverage remained the same at 86.083% when pulling df91f4febed14cb8f196b316478bfdf6702c2af5 on release-280 into 1b9fefacf2c177dd6f5bc0e1bf174b36f373d321 on master.

blocked by https://github.com/katspaugh/wavesurfer.js/pull/1582


Implement Navbar numeric button badges

See the small red circled value at top left corner of settings icon:

Implement this as a method in class AWalletBottomNavigationView.
See Tinashe's answer here:
https://stackoverflow.com/questions/42682855/display-badge-on-top-of-bottom-navigation-bars-icon
This method is best as it avoids adding another library (creating another set of display issues). DO NOT use the library bottom navigation as suggested by the first answer. It should be straightforward to create a white number in a red circle and display it. Need to have a method that decrements and removes the red circle too.

@justindg please implement this, thanks :)

also a 'focus mode' where a popup dialog box focusses on the highlighted box, when the box is clicked.

@justindg perhaps you would like to take this on?

Sure I'll take this on. May I know more about the flow of 'focus mode'? Is that for onboarding only? I'm assuming this is like a 'tour' for new users

Yes I think so; I don't think it would show up more than once, although the little numeric circle will show up whenever there's a wallet that hasn't been backed up.
I'll ask the UX guy for clarification at the next meeting.

Here's another popup that I think will only happen once.
You can get the colour asset from the 'Implement-HD-Keys' branch, it's called 'nasty_green'

Completed.


DEV-1812 - Allow requesting multiple label rows.

NOTE: do not merge this until https://github.com/encord-team/cord-backend/pull/1093 is not rolled out to prod.

NOTE: this will be buggy if someone uses this function for a single label row. Better to pass some extra field.

@alexey-cord-tech @rad-cord please review.

@rad-cord FYI this is implementing reporting the label hash number limit already.

Denis TODO: integration tests and docs update.


MPI Support (new attempt)

TODO:

[x] mpi_getcpuids
[x] mpi_gethostnames
[x] mpi_getlocalrank
[x] mpi_pinthreads
[x] MPI testing (CI)
[x] example in docs

Changelog:

renamed pinthreads_mpi to mpi_pinthreads
new public API mpi_getcpuids, mpi_gethostnames, mpi_getlocalrank (based on explicit communication/exchange between MPI ranks)

(cc @jagot)
Closes #61

Basic multi-node test was successful.
NUMA node 1: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143]
NUMA node 2: [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159]
NUMA node 3: [32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175]
NUMA node 4: [48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191]
NUMA node 5: [64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207]
NUMA node 6: [80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223]
NUMA node 7: [96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239]
NUMA node 8: [112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255]

BEFORE: Where are the Julia threads of the MPI ranks running?
	rank 0 is running 2 Julia threads on the CPU-threads [127, 253] of node nid004406
	rank 1 is running 2 Julia threads on the CPU-threads [113, 182] of node nid004406
	rank 2 is running 2 Julia threads on the CPU-threads [109, 138] of node nid004406
	rank 3 is running 2 Julia threads on the CPU-threads [26, 115] of node nid004406
	rank 4 is running 2 Julia threads on the CPU-threads [4, 146] of node nid005218
	rank 5 is running 2 Julia threads on the CPU-threads [101, 236] of node nid005218
	rank 6 is running 2 Julia threads on the CPU-threads [42, 255] of node nid005218
	rank 7 is running 2 Julia threads on the CPU-threads [8, 90] of node nid005218
	rank 8 is running 2 Julia threads on the CPU-threads [80, 237] of node nid005908
	rank 9 is running 2 Julia threads on the CPU-threads [23, 198] of node nid005908
	rank 10 is running 2 Julia threads on the CPU-threads [5, 47] of node nid005908
	rank 11 is running 2 Julia threads on the CPU-threads [54, 26] of node nid005908
	rank 12 is running 2 Julia threads on the CPU-threads [42, 143] of node nid005915
	rank 13 is running 2 Julia threads on the CPU-threads [8, 120] of node nid005915
	rank 14 is running 2 Julia threads on the CPU-threads [238, 217] of node nid005915
	rank 15 is running 2 Julia threads on the CPU-threads [27, 159] of node nid005915

AFTER: Where are the Julia threads of the MPI ranks running?
	rank 0 is running 2 Julia threads on the CPU-threads [0, 1] of node nid004406
	rank 1 is running 2 Julia threads on the CPU-threads [16, 17] of node nid004406
	rank 2 is running 2 Julia threads on the CPU-threads [32, 33] of node nid004406
	rank 3 is running 2 Julia threads on the CPU-threads [48, 49] of node nid004406
	rank 4 is running 2 Julia threads on the CPU-threads [0, 1] of node nid005218
	rank 5 is running 2 Julia threads on the CPU-threads [16, 17] of node nid005218
	rank 6 is running 2 Julia threads on the CPU-threads [32, 33] of node nid005218
	rank 7 is running 2 Julia threads on the CPU-threads [48, 49] of node nid005218
	rank 8 is running 2 Julia threads on the CPU-threads [0, 1] of node nid005908
	rank 9 is running 2 Julia threads on the CPU-threads [16, 17] of node nid005908
	rank 10 is running 2 Julia threads on the CPU-threads [32, 33] of node nid005908
	rank 11 is running 2 Julia threads on the CPU-threads [48, 49] of node nid005908
	rank 12 is running 2 Julia threads on the CPU-threads [0, 1] of node nid005915
	rank 13 is running 2 Julia threads on the CPU-threads [16, 17] of node nid005915
	rank 14 is running 2 Julia threads on the CPU-threads [32, 33] of node nid005915
	rank 15 is running 2 Julia threads on the CPU-threads [48, 49] of node nid005915

cc @sloede (because you were interested in this at some point)

I plan to merge this tomorrow. @jagot are you happy with this PR or is something missing?

I think the API looks nice, but it does not work as I would naïvely expect/hope; I ran the example from the documentation, adding a call to threadinfo() to really see the thread distribution after pinning (I modified the output in case of no colour, see here https://github.com/jagot/ThreadPinning.jl/tree/improve-colorless-output):
Job official-example ID 183709.bossy on 2 nodes, 2 processes, and 48 threads per process
Host: janeway16
  Activating project at `~/projects/mpi-test`
  Activating project at `~/projects/mpi-test`
NUMA node 1: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71]
NUMA node 2: [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95]

BEFORE: Where are the Julia threads of the MPI ranks running?
	rank 0 is running 48 Julia threads on the CPU-threads [95, 37, 41, 43, 72, 35, 46, 78, 38, 39, 26, 28, 44, 82, 93, 30, 29, 81, 88, 90, 75, 47, 31, 84, 76, 25, 34, 40, 89, 45, 77, 32, 85, 92, 45, 24, 36, 79, 86, 33, 83, 87, 73, 85, 75, 89, 80, 0] of node janeway16
	rank 1 is running 48 Julia threads on the CPU-threads [48, 75, 2, 83, 88, 92, 46, 38, 73, 90, 28, 78, 33, 84, 74, 41, 91, 45, 77, 86, 25, 1, 82, 42, 36, 26, 87, 35, 29, 79, 43, 47, 34, 85, 76, 44, 31, 75, 73, 32, 0, 30, 39, 44, 93, 80, 94, 72] of node janeway17

AFTER: Where are the Julia threads of the MPI ranks running?
	rank 0 is running 48 Julia threads on the CPU-threads [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71] of node janeway16
Hostname: 	janeway16
CPU(s): 	2 x AMD EPYC 7352 24-Core Processor
CPU target: 	znver2
Cores: 		48 (96 CPU-threads due to 2-way SMT)
NUMA domains: 	2 (24 cores each)

Julia threads: 	48

CPU socket 1
  0,48h, 1,49h, 2,50h, 3,51h, 4,52h, 5,53h, 6,54h, 7,55h,
  8,56h, 9,57h, 10,58h, 11,59h, 12,60h, 13,61h, 14,62h, 15,63h,
  16,64h, 17,65h, 18,66h, 19,67h, 20,68h, 21,69h, 22,70h, 23,71h

CPU socket 2
  _,_, _,_, _,_, _,_, _,_, _,_, _,_, _,_,
  _,_, _,_, _,_, _,_, _,_, _,_, _,_, _,_,
  _,_, _,_, _,_, _,_, _,_, _,_, _,_, _,_

# = Julia thread, h = Julia thread on HT, ! = >1 Julia thread

(Mapping: 1 => 0, 2 => 1, 3 => 2, 4 => 3, 5 => 4, ...)
	rank 1 is running 48 Julia threads on the CPU-threads [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71] of node janeway17
Hostname: 	janeway16
CPU(s): 	2 x AMD EPYC 7352 24-Core Processor
CPU target: 	znver2
Cores: 		48 (96 CPU-threads due to 2-way SMT)
NUMA domains: 	2 (24 cores each)

Julia threads: 	48

CPU socket 1
  0,48h, 1,49h, 2,50h, 3,51h, 4,52h, 5,53h, 6,54h, 7,55h,
  8,56h, 9,57h, 10,58h, 11,59h, 12,60h, 13,61h, 14,62h, 15,63h,
  16,64h, 17,65h, 18,66h, 19,67h, 20,68h, 21,69h, 22,70h, 23,71h

CPU socket 2
  _,_, _,_, _,_, _,_, _,_, _,_, _,_, _,_,
  _,_, _,_, _,_, _,_, _,_, _,_, _,_, _,_,
  _,_, _,_, _,_, _,_, _,_, _,_, _,_, _,_

# = Julia thread, h = Julia thread on HT, ! = >1 Julia thread

(Mapping: 1 => 0, 2 => 1, 3 => 2, 4 => 3, 5 => 4, ...)

i.e., there are two MPI ranks, each running on a separate node. The problem is that all threads are pinned to the first socket only (on these machines, the socket and NUMA are equivalent), leaving the other socket un-utilized, and half of the threads are pinned to hyperthreads.
How do I achieve pinning across all sockets? Should there be a :node alternative?

You're running on two nodes each hosting one MPI rank with 48 Julia threads. What mpi_pinthreads(:numa) (or mpi_pinthreads(:sockets) because NUMA domains and sockets fall together) does is distributing MPI ranks (that is all of their Julia threads) in a round-robin fashion among NUMA domains on each node. Since their is only a single MPI rank per node, both MPI ranks (that is their 48 Julia threads) simply get assigned to the first NUMA domain of their respective nodes.
Why are threads pinned to hyperthreads?
Because 48 threads only fit into the NUMA domain if hyperthreads are included. If you don't want hyperthreads involved, choose 24 threads per MPI rank instead of 48.
How can I also occupy the second NUMA domain on each node?
The simple answer is: Use 2 MPI ranks per node. More generally speaking, choose as many MPI ranks per node as there are NUMA domains. This is what one typically does/is advised to do anyways for MPI applications, irrespective of the programming language.
What if I don't want to use 2 MPI ranks per node?
Well, you probably should 😉. More seriously, if you only have a single MPI rank per node, you don't need any of the mpi_pinthreads business anyways. Just call pinthreads(:numa) on each MPI rank and you're done.
(We could imagine a more complicated case though. Say we have 2 MPI ranks per node but a system with 4 NUMA domains per socket. We might want to distribute MPI ranks among sockets but, within each socket, distribute the Julia threads of the MPI rank among NUMA domains. This is currently not available out of the box but one could imagine a mpi_pinthreads(:sockets, :numa) variant.)

BTW, threadinfo() adds little gaps between CPU IDs that don't belong to the same core (look closely and compare to threadinfo(; coregaps=false)). This should also be the case for threadinfo(; color=false) and allows you to identify hyperthreads without adding "h"s.

Thanks for the detailed writeup! I am relearning MPI, having not used it for a long while.
I still think it is a valid use-case to have only one MPI rank per node, if nothing else to be able to compare. AFAIU, architectures vary, and for some it is/should be beneficial to utilize all cores within the same SMP program.
pinthreads indeed does the trick, but for that to be applicable, the code needs to be aware that it is alone on a particular node, which is what I implemented in https://github.com/carstenbauer/ThreadPinning.jl/issues/61#issuecomment-2267110936. That logic could easily be baked into mpi_getlocalrank
https://github.com/carstenbauer/ThreadPinning.jl/blob/345cd1be8760ea218cb056a03b48664620e3e912/ext/MPIExt/mpi_querying.jl#L55-L65
if we so wish, or it could be up to the user to do something like
using ThreadPinning
using MPI

function am_I_alone()
    ...
end

if am_I_alone()
    pinthreads(:numa)
else
    mpi_pinthreads(:numa)
end

I would prefer the former, I think.

I would prefer the former, I think.

I think I don't. Not because 1 MPI rank per node is unreasonable (it isn't) but because it would make the mpi_pinthreads API confusing. What you're proposing is to do one thing if you're not alone on a node and a different thing if you're not.
Currently, the semantics of mpi_pinthreads is simple: The :numa in mpi_pinthreads(:numa) means that MPI ranks (with all their threads as "one unit") will be distributed among NUMA domains. It does not mean that the threads of a MPI rank get distributed among different NUMA domains. But that's what you want it do in the case where the MPI rank is alone on the node. While I believe to see, where you're coming from, this seems pretty arbitrary and confusing to me.

Fair point. I will play around, having the above function in my user code for the time being. When/if something solidifies, and it is useful to anyone but me, we can revisit this issue.


Support for google.cloud.speech.v2

Is your feature request related to a problem? Please describe.
The current google_speech package in Flutter supports Google Speech v1, but I need support for Google Speech v2 due to its improved accuracy and features.
Describe the solution you'd like
'm requesting an upgrade to the google_speech package to support Google Speech v2, enabling developers to leverage its benefits while maintaining package integration.
Additional context
Supporting Google Speech v2 would enhance speech recognition accuracy and capabilities, benefiting a wide range of Flutter applications, including accessibility features and voice assistants.

Hey @bilalahmed24.
Thank you for your suggestion. I will check which adjustments are necessary for this. Unfortunately, I will not have time for this until next week.

@felixjunghans Thanks for your prompt response and for this great package, I can surely wait for another week :)

@felixjunghans Any update?

Hey @bilalahmed24, sorry for the Delay.
Yesterday I regenerated all proto APIs for version 2 and made the necessary adjustments. However, a few things still didn't work as expected.
I plan to update google_Speech on Pub with the Google Speech version 2 Support this weekend.

@felixjunghans Great to hear that, I'm looking forward to it.

Hey @bilalahmed24  google_speech 4.0.0 now Supports V2 Api. I also added a new example (audio_file_v2) because the api is a little bit different. Please test it and if you find any bugs or missing features, feel free to open a new issue or reopen this issue.
Hope that helps

@felixjunghans There seems to be some null check issue.
W/Parcel  ( 5350): Expecting binder but got null!
E/flutter ( 5350): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Null check operator used on a null value
E/flutter ( 5350): #0      new SpeechToTextV2.viaServiceAccount        speech_to_text_v2.dart:35

Do you have a project_id set in your ServiceAccount.json?

@felixjunghans Thanks, I got that working.
But now I am facing issue with streamingRecognize which immediately throws error as audio data is passed through the stream. The same serviceAccount.json is working fine with v1.
GrpcError (gRPC Error (code: 7, codeName: PERMISSION_DENIED, message: Permission 'speech.recognizers.recognize' denied on resource (or it may not exist)., details: [reason: IAM_PERMISSION_DENIED]
message: "Permission 'speech.recognizers.recognize' denied on resource (or it may not exist)."

@felixjunghans Let me get back to you with audio_example_v2 result.

@felixjunghans Same error with audio_example_v2 provided code, with both streamingRecognize and recognize.
That's intriguing, is recognize using gRPC as well instead of restAPI?

Ok. Yes everything in google_speech uses gRPC. It looks like you need some IAM permissions since Google Speech V2 according to this doc https://cloud.google.com/speech-to-text/v2/docs/reference/rpc/google.cloud.speech.v2#google.cloud.speech.v2.Speech.StreamingRecognize
Could you try to add these permissions to your service account?

@felixjunghans Thanks for the help you were right regarding the permissions.
Streaming request is abruptly being closed due to two errors, sometimes it's "stream timeout" or "the operation was cancelled".
I am using microphone to pass audio to stream req.
I wasn't facing this using v1. Could it be due to some v2 streaming request restriction?

@bilalahmed24 Unfortunately, I can't say much about it, since I've only used version 2 for testing purposes so far. The only thing I noticed is that since version 2 the chunkLength for StreamingRecognize is limited to 25600 bytes per request.
If you find out more or think a bug in google_speech is causing this, I'll be happy to fix it. But so far I don't know where the bug should be.

@felixjunghans Okay, I will let you know if I bump into any bug.
Thanks for the help. :)


Arbitrary types

I don't see whether serde-yaml supports user types or not (it seems not to me), but is this article relevant?
https://community.embarcadero.com/blogs/entry/yaml-and-remote-code-execution-38738

Serde in general is not vulnerable to arbitrary object instantiation / remote code execution. That applies to serde-yaml and all other Serde data formats. The code that deserializes untrusted input always specifies what type is expected i.e. which Deserialize impl is going to execute.

I filed https://github.com/chyh1990/yaml-rust/issues/87 to follow up with some documentation on the yaml-rust side.


Add support to infix operators in moonblade

Use the pratt parser crate.
We probably need to drop support for spaces in identifiers for this though.

@boogheta I am getting there. Should I consider adding "à la perl" operators such as =~ or . for string concat or eq for string equality and such. Adding operators does not alleviate the fact that all CSV values are strings by default, which means I must either distinguish between string-oriented and number-oriented functions and operator (the current choice) or force users to cast values around, which means arithmetics don't work outside of the box without casting which feels somewhat tedious.

Current plan for next steps:

[ ] remove clunky s_eq functions and make them the eq functions (we don't really need the numerical comparison functions now since we can have the equality operators)
[ ] deal with +, -, /, *, //, !, ==, >, <, <=, >=

Updated the checklist below with your notes.


New+ isn't coming in right-click Context menu

Microsoft PowerToys version
0.85.1
Installation method
Microsoft Store
Running as admin
No
Area(s) with issue?
New+
Steps to reproduce
New+ doesn't come up in any right-click context Menu. I'm currently using Windows 11 Enterprise OS Build: 22631.4249
✔ Expected Behavior
New+ should work as it is developed.
❌ Actual Behavior
New+ doesn't come up in any right-click context Menu.
Other Software
No response

it is not working for me when i am using the legacy context menu registry change
registry path: HKEY_CURRENT_USER\SOFTWARE\CLASSES\CLSID{86ca1aa0-34aa-4e8b-a509-50c905bae2a2}\InprocServer32

If New+ doesn't show up initially, what worked for me was first a reboot, then right click in a folder white area. From the context menu select 'Show More Options', Then select the 'New+' from there, selected the Example Folder. Once it does this and places the example folder the first right-click context menu now shows the 'New+' PowerToys option under Properties.
This is on Windows 11 Pro. Hope this helps others.

If New+ doesn't show up initially, what worked for me was first a reboot, then right click in a folder white area. From the context menu select 'Show More Options', Then select the 'New+' from there, selected the Example Folder. Once it does this and places the example folder the first right-click context menu now shows the 'New+' PowerToys option under Properties.
This is on Windows 11 Pro. Hope this helps others.

Did not work for me.

it is not working for me when i am using the legacy context menu registry change
OS Version: Windows 11 Pro Build 22631.4169 path of registry change: HKEY_CURRENT_USER\SOFTWARE\CLASSES\CLSID{86ca1aa0-34aa-4e8b-a509-50c905bae2a2}\InprocServer32

Same here.
Restarted many times after installing 85.0 last week and after 85.1, it's still missing. No entry for New+ appears in the full context menu, no matter where I right click.

it is not working for me when i am using the legacy context menu registry change
OS Version: Windows 11 Pro Build 22631.4169 path of registry change: HKEY_CURRENT_USER\SOFTWARE\CLASSES\CLSID{86ca1aa0-34aa-4e8b-a509-50c905bae2a2}\InprocServer32

for some reason it started working for me just now.

/dup https://github.com/microsoft/PowerToys/issues/35210

I've fully uninstalled Power Toys and installed it again from Microsoft store.
It works now.
😊
I'm using Windows 11

Since this issue is being cited by Similar-issues-ai, I'll bring the solution from #35210
Thank @stefansjfw for the solution.

Enable Developer Mode from Settings->System->For developers->Developer Mode.
disable New+ from PowerToys settings
run Remove-AppxPackage Microsoft.PowerToys.NewPlusContextMenu_<IP_ADDRESS>_neutral__8wekyb3d8bbwe
make sure it's removed - run Get-AppxPackage -Name powertoysnewplus*
enable New+ from Powertoys settings
open task manager and restart Windows Explorer process
right click several times and see if New+ is there

You might have to repeat these some of these steps after updating PowerToys. Mainly 1, 4, 5, 6, and 7.


How do I set the timeout for minio-cpp?

When I use the UploadObject(args) function provided by minio-cpp to upload files, if there is a network failure, the program will be blocked and there is no opportunity to exit and correct the error. How do I set the timeout for minio-cpp? It seems that there are no settings related to timeout and network issues in minio-cpp.

As minio-cpp uses libcurl which doesn't provide a proper timeout mechanism like individual read/write timeouts, it is recommended to use progressfunc to implement timeout as per your requirement.
You could also enable Debug to trace what's going on in the network for know exact issue.

@balamurugana
Would it make sense to incorporate this functionality?
https://curl.se/libcurl/c/CURLOPT_TIMEOUT.html
In libcurlpp this is implemented via Timeout trait:
typedef curlpp::OptionTrait<long, CURLOPT_TIMEOUT> Timeout;

I believe that CURLOPT_TIMEOUT would allow us to have the timeout supported. Any objections?

@kobalicek CURLOPT_TIMEOUT is overall timeout for the entire HTTP request. Practically it won't work.

As minio-cpp uses libcurl which doesn't provide a proper timeout mechanism like individual read/write timeouts, it is recommended to use progressfunc to implement timeout as per your requirement.

Could you give an example on cancelling the upload from that callback? I actually see two issues:

Client::UploadObject never transfers the UploadObjectArgs::processfunc into the PutObjectArgs : https://github.com/minio/minio-cpp/blob/0eb2a19df1e07a2b5c7f04e0c5c6049566fc2a12/src/client.cc#L782-L796

When processfunc would be actually called further down the chain, it has no return or mechanism to override the CURL_PROGRESSFUNC_CONTINUE that is returned to curlpp:
https://github.com/minio/minio-cpp/blob/0eb2a19df1e07a2b5c7f04e0c5c6049566fc2a12/src/http.cc#L405-L420

Assuming 1 is fixed, are we supposed to just throw from the callback? Or does 2 also need to be addressed to make the callback return/influence the constant?

@Demonslay335 (1) and (2) are bugs and will be fixed by https://github.com/minio/minio-cpp/pull/128
Once https://github.com/minio/minio-cpp/pull/128 is merged, you could return false to abort the upload/download operation.

`UploadObjectResponse Client::UploadObject(UploadObjectArgs args) {
if (error::Error err = args.Validate()) {
return UploadObjectResponse(err);
}
std::ifstream file;
file.exceptions(std::ifstream::failbit | std::ifstream::badbit);
try {
file.open(args.filename);
} catch (std::system_error& err) {
return error::make(
"unable to open file " + args.filename + "; " + err.code().message());
}
PutObjectArgs po_args(file, args.object_size, 0);
po_args.extra_headers = std::move(args.extra_headers);
po_args.extra_query_params = std::move(args.extra_query_params);
po_args.bucket = std::move(args.bucket);
po_args.region = std::move(args.region);
po_args.object = std::move(args.object);
po_args.headers = std::move(args.headers);
po_args.user_metadata = std::move(args.user_metadata);
po_args.sse = std::move(args.sse);
po_args.tags = std::move(args.tags);
po_args.retention = std::move(args.retention);
po_args.legal_hold = std::move(args.legal_hold);
po_args.content_type = std::move(args.content_type);
po_args.progressfunc = std::move(args.progressfunc);
PutObjectResponse resp = PutObject(std::move(po_args));
file.close();
return UploadObjectResponse(resp);
}`
Hi, it seems that this function is also missing the line po_args.progress_userdata = std::move(args.progress_userdata);. There is an error when calling it.

Based on my testing, the progressfunc is not called when it's unable to connect to the MinIO server.It's possible that I still don't understand how to use minio-cpp correctly. I tested with the commit 0C3F92E, but it doesn't seem to solve the problem. In the MinIO Java client, after a timeout, we can catch an exception, which allows us to cache the file to be uploaded and handle subsequent business and fault processing, such as delayed sending. However, in minio-cpp, if the network is unstable and we lose connection to the MinIO server, the program will be in a blocked state, and progressfunc will not be called. The client will stop working, and we don't have a convenient way to handle this kind of network exception. Under these circumstances, minio-cpp is not usable unless we can guarantee 100% that the network between the client and server is functioning normally. In other words, an anomaly in the network may cause all clients using minio-cpp to fail and stop working.

I think this needs changes of the implementation - using multiple-handles API instead of the easy API that is currently used. I would be happier if we used curl directly without the C++ layer that just adds a lot of overhead anyway.
I'm relatively new to this project so I cannot promise I can resolve this issue quickly, but I understand the problem and I think that blocking to infinity is just not a good design and that it should be fixed.

Based on my testing, the progressfunc is not called when it's unable to connect to the MinIO server.It's possible that I still don't understand how to use minio-cpp correctly. I tested with the commit 0C3F92E, but it doesn't seem to solve the problem. In the MinIO Java client, after a timeout, we can catch an exception, which allows us to cache the file to be uploaded and handle subsequent business and fault processing, such as delayed sending. However, in minio-cpp, if the network is unstable and we lose connection to the MinIO server, the program will be in a blocked state, and progressfunc will not be called. The client will stop working, and we don't have a convenient way to handle this kind of network exception. Under these circumstances, minio-cpp is not usable unless we can guarantee 100% that the network between the client and server is functioning normally. In other words, an anomaly in the network may cause all clients using minio-cpp to fail and stop working.

@adahiro I couldn't understand what kind of network issue you have between client and server.  Please enable Debug and share the output.

I understand the problem - progress func is for monitoring progress, but if there is no progress it won't be called. So if you are stuck, waiting for bytes that are not coming, the thread is blocked for a long time / infinity and your progress func is never actually called.

I understand the problem - progress func is for monitoring progress, but if there is no progress it won't be called. So if you are stuck, waiting for bytes that are not coming, the thread is blocked for a long time / infinity and your progress func is never actually called.

@kobalicek Have you checked this by enabling Debug and observed the behavior? As per https://curl.se/libcurl/c/CURLOPT_TIMEOUT.html, progress function is recommended approach to have custom timeout logic.

As per https://curl.se/libcurl/c/CURLOPT_TIMEOUT.html, progress function is recommended approach to have custom timeout logic.

The issue described does make sense, given this text for the option.

While data is being transferred it is invoked frequently, and during slow periods like when nothing is being transferred it can slow down to about one call per second.

In that same vein, it's actually been deprecated as of 7.32.0, but I'm not sure which version this project actually gets linked against. Although, we all know curl never really kills features, so it's maybe a moot point. 😅 The docs for the "new" CURLOPT_XFERINFOFUNCTION are just copy/pasted from CURLOPT_PROGRESSFUNCTION anyways, so it likely is subject to the same problem.
Perhaps also exposing CURLOPT_TIMEOUT and the like would still help in conjunction to at least avoid running to infinity, which I believe is the worst-case scenario a user would want to avoid.

@Demonslay335 Firstly I would need to know the exact problem by investigating output. The issue you are facing is mostly unrelated to the timeout.Debug

This issue is relatively easy to reproduce:
Write a client test case code, set the server address and related parameters. Here we only write something related to the problem, for example:
args.progressfunc = [](minio::http::ProgressFunctionArgs args) -> bool{// print something }
minio::s3::UploadObjectResponse resp = client.UploadObject(args);
Start the minio server and run the above code; the file uploads successfully.
Shut down the minio server or disable the network on the machine where the client code is running (for example, by unplugging the network cable to simulate an unstable network situation). The program will be perpetually blocked at the line “client.UploadObject(args)”, and args.progressfunc will not be called, resulting in no output.
In actual tests, “args.progressfunc” is only called when the client successfully connects to the minio server.
In real-world network environments, especially with multiple clients, I cannot guarantee that my network will always be healthy. If there is a network anomaly during file upload and it is not possible to connect to the minio server, client.UploadObject(args); and other code will block my thread and will not return any information, leaving me no way to correct the issue. The client's business will become inoperable unless I restart the process.

Additionally, I have observed that only a limited number of methods, such as "get_object", "put_object", "upload_object", and so on, support "args.progressfunc". Methods like "list_buckets" may still cause thread blocking. My understanding is that args.progressfunc is designed to track upload and download progress. Currently, minio-cpp does not offer a way to handle network exceptions, which means using minio-cpp assumes that the network and the minio server are always 100% operational. Unlike minio-cpp, minio-go and minio-java can detect when the network is down or the server is not running, and they do not block under such circumstances.

I tried to reproduce the issue but minio-cpp worked as expected
#include <miniocpp/client.h>

int main() {
  // Create S3 base URL.
  minio::s3::BaseUrl base_url("<IP_ADDRESS>:9000");
  base_url.https = false;

  // Create credential provider.
  minio::creds::StaticProvider provider("minioadmin", "minioadmin");

  // Create S3 client.
  minio::s3::Client client(base_url, &provider);
  client.Debug(true);

  // Create upload object arguments.
  minio::s3::UploadObjectArgs args;
  args.bucket = "my-bucket";
  args.object = "my-object";
  args.filename = "my-object.csv";

  // Call upload object.
  minio::s3::UploadObjectResponse resp = client.UploadObject(args);

  // Handle response.
  if (resp) {
    std::cout << "my-object.csv is successfully uploaded to my-object"
              << std::endl;
  } else {
    std::cout << "unable to upload object; " << resp.Error().String()
              << std::endl;
  }

  return 0;
}

When server and client network are in up state

[root@master minio-cpp]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:de:e4:bb brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever

[root@master minio-cpp]# ./build/UploadObject
* !!! WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x2b057d8; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x2b057d8; line 2001
*   Trying <IP_ADDRESS>:9000...
 WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x2b06f28; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x2b06f28; line 2001
*   Trying <IP_ADDRESS>:9000...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 9000
* STATE: CONNECTING => PROTOCONNECT handle 0x2b06f28; line 2109
* STATE: PROTOCONNECT => DO handle 0x2b06f28; line 2139
> PUT /my-bucket/my-object HTTP/1.1
Host: <IP_ADDRESS>:9000
Accept: */*
Authorization: AWS4-HMAC-SHA256 Credential=minioadmin/20240402/us-east-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=322ba18641029be53b11fc0443760330f99cf4f1274fc801ef135d08dc8062cc
Content-Length: 15
Content-Type: application/octet-stream
User-Agent: MinIO (linux; x86_64) minio-cpp/MINIO_CPP_MAJOR_VERSION.MINIO_CPP_MINOR_VERSION.MINIO_CPP_PATCH_VERSION
x-amz-content-sha256: db8ae25f84f1b0b384c08289997b778745b1dda0126d42d35aed9fc07a00fa5b
x-amz-date: 20240402T132940Z

* STATE: DO => DID handle 0x2b06f28; line 2233
* STATE: DID => PERFORMING handle 0x2b06f28; line 2351
* We are completely uploaded and fine
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 0
< ETag: "21a8156d2669225f97395dfc226cb10b"
< Server: MinIO
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< Vary: Accept-Encoding
< X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
< X-Amz-Request-Id: 17C279AA3EAE902F
< X-Content-Type-Options: nosniff
< X-Xss-Protection: 1; mode=block
< Date: Tue, 02 Apr 2024 13:29:40 GMT
<
* STATE: PERFORMING => DONE handle 0x2b06f28; line 2550
* multi_done[DONE]: status: 0 prem: 0 done: 0
* Connection #0 to host <IP_ADDRESS> left intact
* Expire cleared
my-object.csv is successfully uploaded to my-object

When client network is down

[root@master minio-cpp]# ip addr show eth0
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 52:54:00:de:e4:bb brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever

[root@master minio-cpp]# ./build/UploadObject
* !!! WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x158b7d8; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x158b7d8; line 2001
*   Trying <IP_ADDRESS>:9000...
* connect to <IP_ADDRESS> port 9000 failed: No route to host
* Failed to connect to <IP_ADDRESS> port 9000 after 3076 ms: Couldn't connect to server
* multi_done[CONNECTING]: status: 7 prem: 1 done: 0
* multi_done, not reusing connection=0, forbid=0, close=0, premature=1, conn_multiplex=0
* The cache now contains 0 members
* Curl_disconnect(conn #0, dead=1)
* Closing connection
* Expire cleared
unable to upload object; server failed with HTTP status code 0

When server network is down, but client network is up

[root@master minio-cpp]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:de:e4:bb brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever

[root@master minio-cpp]# ./build/UploadObject
* !!! WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x1f1c7d8; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x1f1c7d8; line 2001
*   Trying <IP_ADDRESS>:9000...
* connect to <IP_ADDRESS> port 9000 failed: No route to host
* Failed to connect to <IP_ADDRESS> port 9000 after 3096 ms: Couldn't connect to server
* multi_done[CONNECTING]: status: 7 prem: 1 done: 0
* multi_done, not reusing connection=0, forbid=0, close=0, premature=1, conn_multiplex=0
* The cache now contains 0 members
* Curl_disconnect(conn #0, dead=1)
* Closing connection
* Expire cleared
unable to upload object; server failed with HTTP status code 0
[root@master minio-cpp]#

I am closing this issue now. Please reopen with appropriate information along with Debug output.

The problem with your test is that you created scenarios where the failure is controlled (no route to the host). A better test would be to drop the packets when trying to establish the connection, which is what would happen if a firewall blocked the connection. Then you run into the aforementioned timeout issues @adahiro mentioned.

Just checked out the libcurl source code and unless you specify a CURLOPT_CONNECTTIMEOUT value, it will default to 300 seconds. That means, trying to connect to a server that has a firewall interfering with the connection will block your application for 5 minutes. That's pretty bad. I would suggest setting a much lower value by default (15 to 30 seconds) or exposing this setting to the consumer of the library. Keep in mind that CURLOPT_CONNECTTIMEOUT only applies to the protocol negotiation phase, which is usually the TCP/IP handshake as well as any transport encryption negotiation. It does not apply to the actual transfer of data once the connection is negotiated.

I tried to reproduce the issue but minio-cpp worked as expected
#include <miniocpp/client.h>

int main() {
  // Create S3 base URL.
  minio::s3::BaseUrl base_url("<IP_ADDRESS>:9000");
  base_url.https = false;

  // Create credential provider.
  minio::creds::StaticProvider provider("minioadmin", "minioadmin");

  // Create S3 client.
  minio::s3::Client client(base_url, &provider);
  client.Debug(true);

  // Create upload object arguments.
  minio::s3::UploadObjectArgs args;
  args.bucket = "my-bucket";
  args.object = "my-object";
  args.filename = "my-object.csv";

  // Call upload object.
  minio::s3::UploadObjectResponse resp = client.UploadObject(args);

  // Handle response.
  if (resp) {
    std::cout << "my-object.csv is successfully uploaded to my-object"
              << std::endl;
  } else {
    std::cout << "unable to upload object; " << resp.Error().String()
              << std::endl;
  }

  return 0;
}

When server and client network are in up state

[root@master minio-cpp]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:de:e4:bb brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever

[root@master minio-cpp]# ./build/UploadObject
* !!! WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x2b057d8; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x2b057d8; line 2001
*   Trying <IP_ADDRESS>:9000...
 WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x2b06f28; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x2b06f28; line 2001
*   Trying <IP_ADDRESS>:9000...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 9000
* STATE: CONNECTING => PROTOCONNECT handle 0x2b06f28; line 2109
* STATE: PROTOCONNECT => DO handle 0x2b06f28; line 2139
> PUT /my-bucket/my-object HTTP/1.1
Host: <IP_ADDRESS>:9000
Accept: */*
Authorization: AWS4-HMAC-SHA256 Credential=minioadmin/20240402/us-east-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=322ba18641029be53b11fc0443760330f99cf4f1274fc801ef135d08dc8062cc
Content-Length: 15
Content-Type: application/octet-stream
User-Agent: MinIO (linux; x86_64) minio-cpp/MINIO_CPP_MAJOR_VERSION.MINIO_CPP_MINOR_VERSION.MINIO_CPP_PATCH_VERSION
x-amz-content-sha256: db8ae25f84f1b0b384c08289997b778745b1dda0126d42d35aed9fc07a00fa5b
x-amz-date: 20240402T132940Z

* STATE: DO => DID handle 0x2b06f28; line 2233
* STATE: DID => PERFORMING handle 0x2b06f28; line 2351
* We are completely uploaded and fine
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 0
< ETag: "21a8156d2669225f97395dfc226cb10b"
< Server: MinIO
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< Vary: Accept-Encoding
< X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
< X-Amz-Request-Id: 17C279AA3EAE902F
< X-Content-Type-Options: nosniff
< X-Xss-Protection: 1; mode=block
< Date: Tue, 02 Apr 2024 13:29:40 GMT
<
* STATE: PERFORMING => DONE handle 0x2b06f28; line 2550
* multi_done[DONE]: status: 0 prem: 0 done: 0
* Connection #0 to host <IP_ADDRESS> left intact
* Expire cleared
my-object.csv is successfully uploaded to my-object

When client network is down

[root@master minio-cpp]# ip addr show eth0
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 52:54:00:de:e4:bb brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever

[root@master minio-cpp]# ./build/UploadObject
* !!! WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x158b7d8; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x158b7d8; line 2001
*   Trying <IP_ADDRESS>:9000...
* connect to <IP_ADDRESS> port 9000 failed: No route to host
* Failed to connect to <IP_ADDRESS> port 9000 after 3076 ms: Couldn't connect to server
* multi_done[CONNECTING]: status: 7 prem: 1 done: 0
* multi_done, not reusing connection=0, forbid=0, close=0, premature=1, conn_multiplex=0
* The cache now contains 0 members
* Curl_disconnect(conn #0, dead=1)
* Closing connection
* Expire cleared
unable to upload object; server failed with HTTP status code 0

When server network is down, but client network is up

[root@master minio-cpp]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:de:e4:bb brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global noprefixroute eth0
       valid_lft forever preferred_lft forever

[root@master minio-cpp]# ./build/UploadObject
* !!! WARNING !!!
* This is a debug build of libcurl, do not use in production.
* STATE: INIT => CONNECT handle 0x1f1c7d8; line 1948
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => CONNECTING handle 0x1f1c7d8; line 2001
*   Trying <IP_ADDRESS>:9000...
* connect to <IP_ADDRESS> port 9000 failed: No route to host
* Failed to connect to <IP_ADDRESS> port 9000 after 3096 ms: Couldn't connect to server
* multi_done[CONNECTING]: status: 7 prem: 1 done: 0
* multi_done, not reusing connection=0, forbid=0, close=0, premature=1, conn_multiplex=0
* The cache now contains 0 members
* Curl_disconnect(conn #0, dead=1)
* Closing connection
* Expire cleared
unable to upload object; server failed with HTTP status code 0
[root@master minio-cpp]#

I am closing this issue now. Please reopen with appropriate information along with Debug output.

Hello, I just copied your code and ran it on Windows, keeping the same URL of "<IP_ADDRESS>:9000" (of course, I didn't run the MinIo server and couldn't access it), and then the information and Debug output was only one line "* trying <IP_ADDRESS> VR9000..." and waiting forever for blocking. Could this be because of the difference between Windows and Linux?

@lxily Maybe. Please try curl tool or use libcurl library with demo code.


Merge beta into alpha

Description
Merge beta into alpha
Breaking Changes
NA
JIRA Link
NA
Checklist

[x] Updated the Readme.md (if required) ?

:tada: This PR is included in version 19.2.0-alpha.2 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 20.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add handling of empty query to load_query

Calling build on empty simple query builder resulted in segmentation fault due to access to null pointer. This PR adds handling of such case by throwing an exception if query payload is not set.

Codecov Report

Merging #813 into trunk/bindings-test will decrease coverage by 0.13%.
The diff coverage is 20%.

@@                   Coverage Diff                   @@
##           trunk/bindings-test     #813      +/-   ##
=======================================================
- Coverage                 31.7%   31.56%   -0.14%
=======================================================
  Files                      455      458       +3
  Lines                    13415    13483      +68
  Branches                  8574     8593      +19
=======================================================
+ Hits                      4253     4256       +3
- Misses                    1397     1457      +60
- Partials                  7765     7770       +5

Impacted Files
Coverage Δ

...shared_model/bindings/model_query_builder_test.cpp
0% <0%> (ø)

...red_model/backend/protobuf/queries/proto_query.hpp
54.05% <50%> (-0.24%)
:arrow_down:

shared_model/builders/protobuf/queries.hpp
38.02% <0%> (-39.75%)
:arrow_down:

shared_model/bindings/model_query_builder.cpp
8.33% <0%> (ø)

shared_model/bindings/model_query_builder.hpp
25% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b155be...9efbd16. Read the comment docs.