Manual curation for 2022.06 release

Refer to documentation for full description of steps.
Checklist:

[x] Step 1 — Process
[x] Step 2 — Curate

[x] Curation
[ ] Review 1
[ ] Review 2

[ ] Step 3 — Export
[ ] Step 4 — EFO feedback

Spreadsheet is ready here.

@tcezard @M-casado
Ready for review:
906 DONE
36 IMPORT
4 UNSURE

@BaronKoy
I reviewed those without previous nor exact mappings, of which I suggest to change 6 from DONE to IMPORT (I left them as DONE for now, but you can find my suggestion in the comments). These are exact mappings in MONDO while, for now, are mapped to their parent terms in EFO. These are good mappings in our rubric, but the following is also stated:
If a mapping can't be found in EFO, look for a mapping to a HP, ORDO, or MONDO trait name. ... These can be imported to EFO...

If we follow that order, in which a mapping is not found and we look for it in MONDO, perhaps these subtypes should be imported. In any case, I'm fine with both options, whether they are imported or mapped to their parental terms. It's just a suggestion.
I also added 7 new mappings (they have a comment as in added during manual curation review).
I would also like to check 2 other things:

While creating the 7 new mappings, I realized that the shortened format URL|LABEL|||EFO_STATUS is not present in the mapping to use when you curate them, @BaronKoy. It does get parsed correctly in the spreadsheet, but is there a reason why those two extra | are not in the mapping but are explicitly required in the rubric?
@tcezard @apriltuesday @BaronKoy - This time I have found 12 exact mappings in EFO that were not suggested by our scripts. Was it because there was a new release in EFO right after the mapping was done? Or should we review our mapping code? Perhaps there's an underlaying issue.

@M-casado
Thank you for the review
I have changed the import suggestions from IMPORT to DONE and used the mappings you had left in the comments
Although they could be mapped to their parent term it would be nice to have them mapped to an exact match which we can add to the import list
As for the shortened format, this was something I had originally raised with Kirill
I believe the consensus was to use a single | character. @tskir if this is the case I will change the wording within the DOC
Count:
951 DONE
38 IMPORT
4 UNSURE

@M-casado I think you're right, there was an EFO release between spreadsheet generation and manual curation which would explain the exact mappings not having been picked up.  It's irritating but unavoidable I think given our current process.
For the shortened format, I think the spreadsheet formula doesn't parse the final column (EFO status) unless those extra |s are there, but everything else is parsed correctly and you can always just fill in that value manually...  I don't mind either way and it doesn't matter for the export step, but that's my 2 cents.
Finally, @BaronKoy I saw your comment about the new NT expansion terms.  Previous terms can become NT expansion terms if new data gets submitted to ClinVar associating the phenotype with a repeat expansion for the first time.  I assume that's what happening here but I might try to double check.

@M-casado
Agreed. We will take a look at the correct syntax on our side to avoid any future issues (hopefully migrating over to OntoString also happens relatively soon!)

Thanks all, export is done, ready for EFO feedback @BaronKoy


Issue with hiding status bar on child view controllers

First off, thanks for the great library!
I have two child view controllers. The first is a camera view that takes up the entire screen which is where I would like to have my status bar hidden. The second VC is a TableViewController where I would like to have the status bar visible. (Much like Snapchat)
In each perspective VC class I have the status bar setting set accordingly however it seems to be overridden by the Parent VC which is the YZSwipe class.
Any idea how I would go about implementing this?

Never mind, I figured it out!
Thanks

That's good. How did you implement it?

In the scrollViewDidEndDecelerating method I just detect the scrollView contentOffset currently displayed on the screen and hide the status bar accordingly.


Snapcraft Build

This is not a issue.

I built a Snap version of Lorien and published it on https://snapcraft.io/lorien
The source-code for snap are in https://github.com/tonybolzan/snap-lorien
To use, install using terminal or in GUI Software Center on Ubuntu or another Snap powered distro
sudo snap install lorien --edge

Thanks! Closing this.


Go to Definition doesn't work in Diff View

VSCode Version: 1.0.0
OS Version: Mac OSX

Steps to Reproduce:

Open any git repository
Make changes to a file
Navigate to Git pane and pick the changed file
On the left side pane, right click on any function as below

Go to Definition doesn't work, nothing happens. While if you do the same thing on the right side pane, it works as charm.

We know that the code is already changed and Intellisense won't work for original version (the left side one), should we hide the context menu instead of confusing users with the not-working Got to Definition or Peek Definition?

IMO we should disable language feature for the left hand side of a diff viewer? @jrieken what is your take.

The actions should be disabled when there no providers for that file. To my knowledge we register the left hand side with some git-index-scheme, so TypeScript and no one else is offering a provider - @dbaeumer given TypeScript registers like so { scheme: file, language: typescript}. I'll investigate...

@dbaeumer The issue is the TypeScript extension registers for typescript models not only for those it can reason about - files on disk. Instead of the simple document selector it should use one that selects files that are associated to typescript AND that sit on disk. The context menu of the left hand side editor will then honour that.

This is how it looks for C#/Omnisharp which registers like this: https://github.com/OmniSharp/omnisharp-vscode/blob/master/src/omnisharpMain.ts#L39

@jrieken makes sense.

Disabled the language features for the left hand side of a diff.


Enable offline deduplication

This PR depends on the latest version of Thanos and kube-thanos.
Depends: https://github.com/observatorium/configuration/pull/222
xref: https://github.com/thanos-io/kube-thanos/pull/105
About flags:
--delete-delay=48h

Time before a block marked for deletion is deleted from bucket.

If delete-delay is non zero, blocks will be marked for deletion
and compactor component will delete blocks marked for deletion from the bucket.

If delete-delay is 0, blocks will be deleted straight away.
Use this if you want to get rid of or move the block immediately.

Note that deleting blocks immediately can cause query failures,
if store gateway still has the block loaded, or compactor is ignoring the deletion
because it's compacting the block at the same time.

--ignore-deletion-marks-delay=24h

Duration after which the blocks marked for deletion will be filtered out while fetching blocks.
The idea of ignore-deletion-marks-delay is to ignore blocks that are marked for deletion with some delay.

This ensures store can still serve blocks that are meant to be deleted but do not have a replacement yet.
If delete-delay duration is provided to compactor or bucket verify component,
it will upload deletion-mark.json file to mark after what duration the block should be deleted rather than deleting the block straight away.

If delete-delay is non-zero for compactor or bucket verify component,
ignore-deletion-marks-delay should be set to (delete-delay)/2 so that blocks marked for deletion are filtered out while fetching blocks before being deleted from bucket.
Default is 24h, half of the default value for --delete-delay on compactor.

Changes

Add deduplication replica label flags daca967a6e36765ab9d0628bbb3f283034f2e461
Add delete delay flags 57f99e9a2cbf2b6a27a07b8a2960fa0671e30f7b

cc @squat @metalmatze @krasi-georgiev

@bwplotka I've created another PR for the upgrade https://github.com/observatorium/configuration/pull/222.
Updated deduplication.replicate-labels and changed the delete-delay flag values with default ones without deleting them to make them visible.


mapnik.register_default_input_plugins results in a node abort

Version
node -v: v22.11.0
npm version: 10.9.0
Platform
Linux u24vm 6.8.0-48-generic #48-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 14:04:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

What steps will reproduce the bug?

Try to install the package: mapnik

a. npm init -y
{
  "name": "package-63c2d3cf9bb14c5be59cc1a05e27100d",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": ""
}

b. npm shrinkwrap

c. npm install --save mapnik
added 60 packages, and audited 61 packages in 11s

4 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

d. npm audit fix
added 60 packages, and audited 61 packages in 11s

4 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

the final package.json is as follows:
{
  "name": "package-63c2d3cf9bb14c5be59cc1a05e27100d",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": "",
  "dependencies": {
    "mapnik": "^4.5.9"
  }
}

Setup a node instance,

node

and run the following javascript code.

(async function() {
    var lib = await import('mapnik');
    var result = lib.register_default_input_plugins();;
})();

Then the node instance occurs an abort.
How often does it reproduce? Is there a required condition?
This abort can always be triggered following the steps above.
What is the expected behavior? Why is that the expected behavior?
If any error occurs, an exception or other similar error-reporting stuff should be thrown. There is no reason to abort the whole node process.
What do you see instead?
There are two types of crash log:
» node
Welcome to Node.js v22.11.0.
Type ".help" for more information.
> (async function() {
...     var lib = await import('mapnik');
...     var result = lib.register_default_input_plugins();;
... })();
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 28,
  [Symbol(trigger_async_id_symbol)]: 6
}
> free(): invalid pointer
[1]    746303 IOT instruction (core dumped)  node

» node
Welcome to Node.js v22.11.0.
Type ".help" for more information.
> (async function() {
...     var lib = await import('mapnik');
...     var result = lib.register_default_input_plugins();;
... })();
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 28,
  [Symbol(trigger_async_id_symbol)]: 6
}
> [1]    746429 segmentation fault (core dumped)  node

Thank you for your response. It's glad that the issue has already be fixed on v4.6.5.
FYI, I retried the npm installation steps without setting the version, and I found that the default installed version is still v4.5.9.
It seems because that the latest tag was not updated and npm view mapnik shows latest: 4.5.9.
» npm view mapnik

<EMAIL_ADDRESS>| BSD-3-Clause | deps: 3 | versions: 183
Tile rendering library for node
http://mapnik.org

keywords: map, graphics, canvas, tile, mapnik, carto

bin: mapnik-index.js, mapnik-inspect.js, mapnik-render.js, mapnik-shapeindex.js

dist
.tarball: https://registry.npmjs.org/mapnik/-/mapnik-4.5.9.tgz
.shasum: ef1b19233d8712cdcb9c23b2169b5a3ad4f696ca
.integrity: sha512-Vq5rnG76IVOSkGj9MIF+ZxbbORLOFmQK5jmBb9o5r9gpVfrNQsV7VYV5hKG7pjve0vRMbbZWm5rTNlZjZ3xpiw==
.unpackedSize: 1.5 MB

dependencies:
@mapbox/node-pre-gyp: ^1.x mapnik-vector-tile: 3.0.1  node-addon-api: ~3.1.x

maintainers:
- artem<EMAIL_ADDRESS>- springmeyer<EMAIL_ADDRESS>- gretacb<EMAIL_ADDRESS>- kkaefer<EMAIL_ADDRESS>- jrpruit1<EMAIL_ADDRESS>- flippmoke<EMAIL_ADDRESS>- bergwerkgis<EMAIL_ADDRESS>- yhahn<EMAIL_ADDRESS>- mapsam<EMAIL_ADDRESS>
dist-tags:
dev: 4.5.3-imagetest.1  latest: 4.5.9

published over a year ago by artem<EMAIL_ADDRESS>---------------------------------------------------------

@zyscoder - latest NPMs are published on Github, see the link I posted above ^ . In order to access it you'll need to create Github token as per https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-npm-registry
npm install<EMAIL_ADDRESS>

.npmrc
You can choose user config or project config.

per-project config file (/path/to/my/project/.npmrc)
per-user config file (~/.npmrc)

Create a new .npmrc file if one doesn't exist.
edit the .npmrc file for your project to include the following line, replacing TOKEN with your personal access token.
@mapnik:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=XXXXXXXXXXXXXXXXXXXXXXXXX


measurementCohort

conceptCohort + requirement around test result (value_as_number, value_as_concept_id) and unit_concept_id

Should we somehow add the information of valueAsConcept and valueAsNumber in the attribute cohort_codelist?

@nmercadeb I wouldn't worry about that now, but let's leave this open to tackle later


[PowerRename] Disabled SplitButton Foreground text is hard to read

Microsoft PowerToys version
0.49.1
Running as admin

[ ] Yes

Area(s) with issue?
PowerRename
Steps to reproduce
Run PowerRename
✔ Expected Behavior
The Disabled state of the Apply button should still show text that can be read (when using Dark Theme).
❌ Actual Behavior
It's hard to read, need to validate that the correct background or foreground brush is used.
We're using a custom style for the SplitButton, so my guess is that we're using a wrong brush. Needs validation against WinUI and then correct (if necessary).

Other Software
No response

This seems to be an inconsistency / bug in WinUI. Tracking it here: https://github.com/microsoft/microsoft-ui-xaml/issues/6500

Is this still an issue with the latest version? /needinfo

From the comment on issue 6500, looks like winappsdk 1.5 will have this fixed if 1.4 doesn't already


mix --production got stuck on Ubuntu 20.04

Laravel Mix Version: v6.0.5
Node Version (node -v):  v15.5.0
NPM Version (npm -v): v7.3.0
OS: Ubuntu 20.04

Description:
I tried to compile my assets on a fresh Ubuntu 20.04 droplet with npm run production and it got stuck showing this output:

My package.json :
{
    "private": true,
    "scripts": {
        "dev": "npm run development",
        "development": "mix",
        "watch": "mix watch",
        "watch-poll": "mix watch -- --watch-options-poll=1000",
        "hot": "mix watch --hot",
        "production": "mix --production"
    },
    "devDependencies": {
        "@vue/compiler-sfc": "^3.0.2",
        "axios": "^0.19.2",
        "bootstrap": "^4.0.0",
        "cross-env": "^7.0",
        "jquery": "^3.2",
        "lodash": "^4.17.19",
        "popper.js": "^1.12",
        "resolve-url-loader": "^2.3.1",
        "sass": "^1.20.1",
        "sass-loader": "^8.0.0",
        "vue": "^3.0.2",
        "vue-loader": "^16.1.2"
    },
    "dependencies": {
        "@fortawesome/fontawesome-svg-core": "^1.2.32",
        "@fortawesome/free-solid-svg-icons": "^5.15.1",
        "@fortawesome/vue-fontawesome": "^3.0.0-3",
        "autoprefixer": "^10.1.0",
        "laravel-mix": "^6.0.5",
        "mitt": "^2.1.0",
        "postcss": "^8.2.1",
        "primeflex": "^2.0.0",
        "primeicons": "^4.1.0",
        "primevue": "^3.1.0",
        "tailwindcss": "^2.0.2",
        "vue-router": "^4.0.1"
    }
}

My webpack.mix.js:
const mix = require('laravel-mix');

mix.js('resources/js/app.js', 'public/js')
    .vue()
    .sass('resources/sass/app.scss', 'public/css')
    .postCss("resources/css/app.css", "public/css", [
        require("tailwindcss"),
    ]);

Steps To Reproduce:

[optional] Create a new droplet on Digital Ocean using Ubuntu 20.04

install node from source
curl -sL https://deb.nodesource.com/setup_15.x | sudo -E bash -
sudo apt-get install -y nodejs

Clone repository
git clone https://github.com/davidhernandeze/mix-issue.git
cd mix-issue

install dependencies
npm install

compile assets
npm run production

after some time the production command ends and wrote this log:
0 verbose cli [ '/usr/bin/node', '/usr/bin/npm', 'run', 'production' ]
1 info using<EMAIL_ADDRESS>2 info using<EMAIL_ADDRESS>3 timing config:load:defaults Completed in 2ms
4 timing config:load:file:/usr/lib/node_modules/npm/npmrc Completed in 1ms
5 timing config:load:builtin Completed in 1ms
6 timing config:load:cli Completed in 2ms
7 timing config:load:env Completed in 0ms
8 timing config:load:file:/var/www/mix-issue/.npmrc Completed in 0ms
9 timing config:load:project Completed in 1ms
10 timing config:load:file:/root/.npmrc Completed in 0ms
11 timing config:load:user Completed in 0ms
12 timing config:load:file:/usr/etc/npmrc Completed in 0ms
13 timing config:load:global Completed in 0ms
14 timing config:load:cafile Completed in 0ms
15 timing config:load:validate Completed in 1ms
16 timing config:load:setUserAgent Completed in 0ms
17 timing config:load:setEnvs Completed in 1ms
18 timing config:load Completed in 8ms
19 verbose npm-session b5f7a8c8fa47726b
20 timing npm:load Completed in 21ms
21 timing command:run-script Completed in 463251ms
22 verbose stack Error: command failed
22 verbose stack     at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
22 verbose stack     at ChildProcess.emit (node:events:376:20)
22 verbose stack     at maybeClose (node:internal/child_process:1063:16)
22 verbose stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:295:5)
23 verbose cwd /var/www/mix-issue
24 verbose Linux 5.4.0-51-generic
25 verbose argv "/usr/bin/node" "/usr/bin/npm" "run" "production"
26 verbose node v15.5.0
27 verbose npm  v7.3.0
28 error code 137
29 error path /var/www/mix-issue
30 error command failed
31 error command sh -c mix --production
32 verbose exit 137

I had the same problem on a DigitalOcean 1GB instance, it didn't have enough RAM. See https://github.com/JeffreyWay/laravel-mix/issues/2479#issuecomment-751682456.

Do you have a swap space?
DigitalOcean recommends against it because it  can wear out their SSDs faster, but you could enable one temporarily while running the mix compile.
You can check if you have a swap by running this:
sudo swapon --show

Otherwise you can set one up pretty quickly using this guide:
https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-18-04/

Do you have a swap space?
DigitalOcean recommends against it because it  can wear out their SSDs faster, but you could enable one temporarily while running the mix compile.
You can check if you have a swap by running this:
sudo swapon --show

Otherwise you can set one up pretty quickly using this guide:
https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-18-04/


Faulty calculator

Heyy! manzoo I've added the description in the file I hope you like it. Thanks:)

heyy! Sorry for sending this to you again but I spotted a mistake in the first one which may affect the result. Sorry for the inconvenience caused if you like the programme please re-consider it.


Create namespaces required by shbose tenant

Without the namespaces, ArgoCD fails to sync the authentication app.
Signed-off-by: Luiz Carvalho<EMAIL_ADDRESS>

No no no :)


Stack-Based Approach to Detect 132 Pattern

Intuition
We are looking for a subsequence in the array that follows the 132 pattern: a sequence where there exists indices i < j < k such that nums[i] < nums[k] < nums[j]. The key is to track the largest '2' in the 132 pattern using a stack.
Approach

Traverse the array in reverse (starting from the rightmost element).
Use a stack to keep track of potential '3' values in the 132 pattern.
Maintain a variable prev to store the most recent valid '2' found in the sequence.
For each element nums[i], check if it is less than the current prev ('2'). If yes, we've found a valid 132 pattern.
Update prev by popping from the stack, ensuring we always have the largest possible '2'.
Push the current element into the stack as a potential '3'.

Complexity

Time complexity:
The time complexity is ( O(n) ), where ( n ) is the size of the input array. Each element is pushed and popped from the stack at most once.

Space complexity:
The space complexity is ( O(n) ) in the worst case, due to the space used by the stack.

Follow the rules make sure to star and follow for Hacktoberfest


Unexpected client ip behavior

I'm running an uws server on localhost. Whenever I retrieve the clients ip address, it throws <IP_ADDRESS>. I'm expecting it to throw my public ipv4. Why's that?
Note: I'm using the same device for both client and server.

<IP_ADDRESS> is localhost so not sure what you are expecting, this is basic stuff. it would only show the ip if you were coming in externally from that ip. but you are coming from localhost

If you want to send a note to someone who lives in your house, do you write your address down and ship it out in the mail? Nope you put localhost <IP_ADDRESS> and keep it in the house


Implement Async AzureDataFactoryRunPipelineOperator

Based on the research in https://github.com/astronomer/astronomer-providers/issues/185, add a Async version of AzureDataFactoryRunPipelineOperator.
Acceptance Criteria:

[ ] Unit Tests coverage in the PR (80% Code Coverage -- We will need to add CodeCov separately to measure code cov) with all of them passing
[ ] Example DAG using the async Operator that can be used to run Integration tests that are parametrized via Environment variables. Example - https://github.com/apache/airflow/blob/8a03a505e1df0f9de276038c5509135ac569a667/airflow/providers/google/cloud/example_dags/example_bigquery_to_gcs.py#L33-L35
[ ] Add proper docstrings for each of the methods and functions including Example DAG on how it should be used (populate
[ ] Exception Handling in case of errors
[ ] Improve the OSS Docs to make sure it covers the following:

[ ] Has an example DAG for the sync version
[ ] How to add a connection via Environment Variable & explain each of the fields. Example - https://airflow.apache.org/docs/apache-airflow-providers-postgres/stable/connections/postgres.html
[ ] How to use Guide for the Operator - example: https://airflow.apache.org/docs/apache-airflow-providers-postgres/stable/operators/postgres_operator_howto_guide.html

Based on initial analysis, I think this should be good to implement considering that the operator returns a job_id

I have started working on this

PR https://github.com/astronomer/astronomer-providers/pull/258


bibi v1.2.0 Mobile browsers are not compatible

bibi v1.2.0 Mobile browsers are not compatible ，status is loading loading 。。。。
bibi v1.1.0 Mobile browsers are  compatible

What do you mean by not compatible? Bibi doesn't load? I've noticed that it can take pretty long to load on mobile


weather_i

Thanks so much for sharing.
I have a really basic question. What are the "weather_i" variables are they dummy variables or prehaps some other weather observation type supplementing the temperature data?
Thanks
samebohon

Hi there!
Thanks for checking it out, hope its helpful... Sorry, the documentation on
the public repository is not great and I had to check my term paper
(attached) I wrote from back then:
"For each city, the temperature (◦F) and a qualitative weather
categorization (bad, neutral, nice) has been collected. In total, there are
32 features describing weather."
So its actually One-Hot-Encoding of a 3 category variable and the original
data was the "icon" from the Dark Sky API, with the mapping defined in the
config.json:
"iconmap": {
"clear-day": 1,
"clear-night": 2,
"rain": 3,
"snow": 3,
"sleet": 3,
"wind": 3,
"fog": 3,
"cloudy": 3,
"partly-cloudy-day": 1,
"partly-cloudy-night": 2
}
Looking back at this, it seems actually quite random so I wouldn't
interpret much into this variable ;)
Am Mo., 24. Juni 2019 um 14:50 Uhr schrieb samebohon <
<EMAIL_ADDRESS>
Thanks so much for sharing.
I have a really basic question. What are the "weather_i" variables are
they dummy variables or prehaps some other weather observation type
supplementing the temperature data?
Thanks
samebohon
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dafrie/lstm-load-forecasting/issues/2?email_source=notifications&email_token=ABM5HV3NCWV6LOXHQO3V52LP4C7KXA5CNFSM4H26JT62YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G3JDSMQ,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABM5HVZY3I6OLGBIBWSUYLLP4C7KXANCNFSM4H26JT6Q
.

Hi there!
Thanks for checking it out, hope its helpful... Sorry, the documentation on
the public repository is not great and I had to check my term paper I wrote from back then:
"For each city, the temperature (◦F) and a qualitative weather
categorization (bad, neutral, nice) has been collected. In total, there are
32 features describing weather."
So its actually One-Hot-Encoding of a 3 category variable and the original
data was the "icon" from the Dark Sky API, with the mapping defined in the
config.json:
{
   ...
  "iconmap": {
    "clear-day": 1,
    "clear-night": 2,
    "rain": 3,
    "snow": 3,
    "sleet": 3,
    "wind": 3,
    "fog": 3,
    "cloudy": 3,
    "partly-cloudy-day": 1,
    "partly-cloudy-night": 2
  }
}

Looking back at this, it seems actually quite random so I wouldn't
interpret much into this variable ;)

Hi there,
Thanks for this. After I sent the query I actually figured it out but it is great to have your confirmation.
I am intrigued, are you working in energy modelling at present, or did you leave energy behind after this?
Thanks


[Bug]: Charts should remain top-aligned when we compare locations and periods

What happened?
Charts should always be top-aligned. This works as expected if you don't compare indicators / time periods.
Which country / deployment are you running?
All
Add a screenshot (if relevant)
Working correctly when using two indicators for the same location / period:

Not top-aligned when comparing indicators:

Sorry @laurentS - I didn't catch this before

This works as expected for more than one chart, but I just noticed that we're now reducing the height when we have a single chart. Here's what it looks like in Mozambique now:

Whereas before it looked like this:

Sorry for not catching this before @laurentS !

Closing this issue and created a new issue for the single chart bug: https://github.com/WFP-VAM/prism-app/issues/989


Added support for ros-sitl on cmake-2

Ros sitl builds about 95% with this. We will have to fix a few minor issues left.

You certainly have the developer of the month award and you're closing in to the developer of the year! Thank you so much!


from pyfasta import Fasta

Hello, the work is very amazing, and i am very interested. However, when I test the Gen_data.py, the code can't run, because the pyfasta is absent. Can you share it?

You can get it from here. It is archived by the author but you should still be able to use it.


fix: rueidiscompat's ZPopMax and ZPopMin with count 1

Resolves #431

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (0249ec8) 95.97% compared to head (a81d7e4) 95.35%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #433      +/-   ##
==========================================
- Coverage   95.97%   95.35%   -0.63%
==========================================
  Files          77       59      -18
  Lines       32403    27647    -4756
==========================================
- Hits        31098    26362    -4736
+ Misses       1112     1100      -12
+ Partials      193      185       -8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


error in preimage example?

Hi,
While playing around with preimage example, I noticed that the preimage_bits is being right padded.  Shouldn't preimage_bits be reversed before padding, and then reversed again before hashing? (or am I misunderstanding something?)
E.g:
    let preimage_bits: [bool; FIELD_SIZE] = to_bits(preimage);
    let preimage_bits_reversed: [bool; FIELD_SIZE] = reverse(preimage_bits);
    let preimage_bits_padded: [bool; FIELD_SIZE_PADDED] = pad(preimage_bits_reversed, 256, false);
    let preimage_bits_padded_rereversed: [bool; FIELD_SIZE_PADDED] = reverse(preimage_bits_padded);
    let sha256_digest = sha256(preimage_bits_padded_rereversed);

I'd be happy to submit a PR.
Thanks, zinc is awesome !

@tshabs hi! Sorry for the late reply!
I think your remark makes perfect sense. The example will be fixed in the next version of Zinc!


Attempt to deploy Hive v1alpha on OpenShift 4.3 failed

What I did
# sourced my KUBECONFIG
git clone https://github.com/openshift/hive
git checkout v1
cd hive
DEPLOY_IMAGE=quay.io/rhibmcollab/hive:v1-alpha-latest make deploy

What happened
Deployment failed with error message:
The RoleBinding "hiveapi" is invalid: roleRef.kind: Unsupported value: "": supported values: "Role", "ClusterRole"
make: *** [deploy] Error 1

What I expected:
Successful deployment and operational Hive controllers.

I don't know if this will fix your problem, but v1 was merged to master, so you should use master now. We'll get v1 deleted shortly.

Yes, this should be resolved in the master branch.

Confirmed that master deployed cleanly with no changes. Thank you!


Optmizie Batcher and L0 infer shared memory tests

Improve runtime of tests by reused old shared memory regions.
PS: Depends on sharedMemoryVariable changes. Before merging.

@CoderHam can you rebase this to master


rollbar: only post on scale up, not down

lgtm

Yeah, definitely would be useful to have a concrete and first-class notion of deploys. Semantics here would be a bit difficult though - do we record the deploy the first time the new tag is stable? Or when the image starts building? When the image is built?


Solve the issue of interrupts not working within C++ files

After setting up interrupts on UART0 and creating the handler within the main.cpp file the startup file couldn't find the handler. It hits the end of the list of handlers and spins.

The issue was solved by using extern "C" void UART0_Handler(void);. C++ does something called name mangling that changes the name of functions to make them unique, which is done by adding information about the arguments. extern "C" makes the function in C++ usage C linkage so that the name doesn't change within the object code.


Impossible to use Sequlize defaultScope 'include' option after v5.6.1 release

Hi,
After the v5.6.1 release, I can no longer use Sequlize defaultScope 'include' option. When I save a record, it tries to update fields from an included model.
Here is an example.
My model:
    ModelA.init(
        {
            name: DataTypes.STRING,
            modelB: {
                type: DataTypes.INTEGER,
                references: {
                    model: 'ModelB',
                    key: 'id'
                }
            }
        },
        {
            defaultScope: {
                include: ['modelB']
            },
            sequelize,
            modelName: 'ModelA'
        }
    );

When I am saving the record, /src/utils/param-converter/prepare-params.ts returns data:
{
    id: 129,
    name: 'Some name',
    modelB: '5',
    createdAt: 2022-02-28T22:10:38.000Z,
    updatedAt: 2022-02-28T22:10:38.000Z,
    'modelB.id': '5',
    'modelB.name': 'Some model b name',
    ...
    here are orher modelB fields
    ...
    'modelB.createdAt': '2021-12-20T17:50:54.000Z',
    'modelB.updatedAt': '2022-03-14T20:47:03.000Z'
}

And I am getting an error:
Executing (default): UPDATE `ModelA` SET `name`=?,`modelB`=?,`updatedAt`=?,`ModelB`=? WHERE `id` = ?

Error
    at Query.run (.../node_modules/sequelize/lib/dialects/mysql/query.js:52:25)
    at .../node_modules/sequelize/lib/sequelize.js:313:28
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async MySQLQueryInterface.bulkUpdate (.../node_modules/sequelize/lib/dialects/abstract/query-interface.js:355:12)
    at async Function.update (.../node_modules/sequelize/lib/model.js:1943:28)
    at async Resource.update (.../node_modules/@adminjs/sequelize/lib/src/resource.js:136:13)
    at async BaseRecord.update (.../node_modules/adminjs/lib/backend/adapters/record/base-record.js:147:30)
    at async Object.handler (.../node_modules/adminjs/lib/backend/actions/edit/edit-action.js:66:23)
    at async ActionDecorator.handler (.../node_modules/adminjs/lib/backend/decorators/action/action-decorator.js:73:19)
    at async ApiController.recordAction (.../node_modules/adminjs/lib/backend/controllers/api-controller.js:168:28)
    at async handler (.../node_modules/@adminjs/express/lib/buildRouter.js:33:30)

And before v5.6.1 release /src/utils/param-converter/prepare-params.ts returned data without modelB fields:
{
    id: 129,
    name: 'Some name',
    modelB: '5',
    createdAt: 2022-02-28T22:10:38.000Z,
    updatedAt: 2022-02-28T22:10:38.000Z
}

And it worked fine. Could you help me to find a solution?

This is probably because of this line: https://github.com/SoftwareBrothers/adminjs/blob/master/src/utils/param-converter/prepare-params.ts#L43
I've just tried to figure out how to filter these default JOINs out but it's difficult if we want to support various ORMs. We also do not want to remove that line, the logic is to send whole payload including virtual fields but convert types of known properties.
You can, however, fix this on your end by using a generic before hook for your edit action:
import { flat } from 'adminjs';

const stripPayload = (...fields) => (request) => {
  if (request.method !== 'post') return request;

  const { payload = {} } = request;

  const filteredPayload = flat.filterOutParams(payload, fields ?? []);

  request.payload = filteredPayload;

  return request;
};

resource:
    actions: {
      edit: {
        before: [stripPayload('category')]
      }
    },

Hi @dziraf
Ok, I will fix it on my end, thank you for the response!


WIP: Replace bleach with nh3

Fixes #3693
Aims to fully replace bleach with nh3 due to bleach deprecation. Currently, django-nh3 is in it's infancy, but seems like it could be an almost drop in replacement for django-bleach, for I forked it and made some small additions that would allow it to work for our purposes and be smoothly migrated.
Initial smoke testing in Hypha seems to work exactly as bleach did but needs more extensive testing. Ideally I would smooth out some edges of my fork and put in a PR to django-nh3. Let me know any thoughts/questions!

Nice work getting a start on this. Ammonia say they are 20 times faster then Bleach, that should shave of a number of milli seconds for nearly every request.
Would be good to upstream the changes if they are generally useful.
I see you added code to pick up any BLEACH_* settings. Is this needed? We control the default settings in Hypha.
Other instances might have set their own values but the release notes and the docs can inform them of the need to update the settings.
I prefer to be explicit, remove every mention of BLEACH from the code so there is no confusion about what we are using. The settings are complex and confusing as it is :-).

Thanks for taking a look! I'm looking forward to seeing the stats once it's implemented, should be significant.

I see you added code to pick up any BLEACH_* settings. Is this needed? We control the default settings in Hypha.

I did this only really with the upstream in mind, I wasn't totally sure if other adopters would've wanted somewhat of a drop-in solution but realistically I think you're totally right in the fact that it only adds complexity and confusion so I'll strip that out.
Hoping to iron out some unit tests I have locally on my fork then going to be putting up a PR. Should be cool!

Upstream is all merged, should be good to go!

@wes-otf  Excellent work on the upstream PR! I noticed a contribution from Torchbox/Wagtail people as well so they are looking at using nh3 as well.
This functionality is core to Hypha security since we filter on output, not on input. (It is a long term goal to change this.) Therefor I do not want to rush in to deploying this, especially since the django-nh3 package is so fresh.
What do you think?

@frjo I totally agree. Especially with the long term deprecation plan for bleach I don't see any reason to jump into this without some exhaustive user testing. Totally fine waiting to merge this in the new year.

@frjo What are you thinking on the timeline for this? Once we get everything all set with the public stuff we could let the user testing group give this a go and ensure nothing weird changed.

@wes-otf There are some public pages that gets re-added by mistake in this PR.
My concern regarding timeline is that the  django-nh3 package is very new. On the other hand it is a simple wrapper and this PR touch a lot of files and will likely get a lot of merge conflicts if we have it hanging around.
I suggest we make v5.4.0 with all the public go out first. Then we work to get this in to the next release.

Good catch! That was the result of a sloppy merge on my part. That timeline sounds good though - the continuous merge conflicts were also my worry so I think that'll work nice.

@wes-otf Can you rebase this? Then we can put it on test and get it in before more rebases are needed.


App users SNBAT pulsing dots on home screen

Expected Behavior
Users experience no confusion due to blue pulsing dots on Account Overview
Current Behavior
The blue pulsing dots are shown to new users, but users do not understand them

Part of UI fixes for Berlin blockchain week here https://docs.google.com/document/d/1thh3gXycni84bc7GWNO6Pxkiy--x0JEM54gvL-UigzM/edit?usp=sharing


Library Serial CAN_BUS errors can't compile or install in arduino, workaround

Hi everyone, my first post here.
I've purchased a serial can_bus unit and want to start working with it, but can't compile due to errors.
Can i use the one made for SPI?, or do I have to write my own canbus script.
Thank you in advance.
Ps. can't directly install the library through arduino (which will be recognized), also includepath errors in vscode, so now simple insert both the .cpp & .h file.  (checked includepaths and it's there).
src\init_CANBUS.h:3:17: error: expected ',' or '...' before numeric constant  #define can_tx  2           // tx of serial can module connect to D2                  ^ src\Serial_CANBUS\Serial_CAN_Module.h:37:20: note: in expansion of macro 'can_tx'      void begin(int can_tx, int can_rx, unsigned long baud);                     ^ D:/Paul/Documents/PlatformIO/Projects/191113-221452-megaatmega2560/src/test: In function 'void setup()': D:/Paul/Documents/PlatformIO/Projects/191113-221452-megaatmega2560/src/test.ino:6:35: error: no matching function for call to 'Serial_CAN::begin(int, int, int)'      can.begin(can_tx, can_rx, 9600);      // tx, rx                                    ^ In file included from src\Includes.h:7:0,                  from D:/Paul/Documents/PlatformIO/Projects/191113-221452-megaatmega2560/src/test.ino:1: src\Serial_CANBUS\Serial_CAN_Module.h:37:10: note: candidate: void Serial_CAN::begin(int)      void begin(int can_tx, int can_rx, unsigned long baud);           ^ src\Serial_CANBUS\Serial_CAN_Module.h:37:10: note:   candidate expects 1 argument, 3 provided

`
D:\Paul\Documents\Arduino\libraries\Serial_CAN_Arduino-master\Serial_CAN_Module.cpp: In member function 'unsigned char Serial_CAN::exitSettingMode()':
D:\Paul\Documents\Arduino\libraries\Serial_CAN_Arduino-master\Serial_CAN_Module.cpp:172:31: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
 int ret = cmdOk("AT+Q\r\n");

                           ^

D:\Paul\Documents\Arduino\libraries\Serial_CAN_Arduino-master\Serial_CAN_Module.cpp: In member function 'unsigned char Serial_CAN::factorySetting()':
D:\Paul\Documents\Arduino\libraries\Serial_CAN_Arduino-master\Serial_CAN_Module.cpp:286:26: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
     if(cmdOk("AT\r\n"))

                      ^

`

Fixed it, appears that after correcting previous errors, visual studio code now accepts without errors.   Arduino ide on the other hand not.


Trident user generating security.invalid.login: Failed to authenticate login attempt to Vserver: message on NetApp filer

Describe the solution you'd like
Would like some clarification on the rationale of Trident generating http error messages on the NetApp filer  when all the trident user has, is api access. There seems to be no clear indication when this happens, we only get notified by the NetApp admins that our Trident user has generated below event.
Additional context

Trident version used: 24.06.1
Trident is soley used to create NFS PVs on a separate SVM reserved for a K8S cluster
The message that gets generated on the filer:
11/27/2024 14:46:01 filer01      ALERT         security.invalid.login: Failed to authenticate login attempt to Vserver: vserver123, username: tridentuser, application: http.

The Trident user is created with SVM privileges like described below:
User/Group                 Authentication                 Acct          Authentication
Name           Application Method        Role Name        Locked        Method
+++++++++ +++++++++ ++++++++ +++++++++     ++++++++  ++++++++++
tridentuser        ontapi      password      vsadmin          no         none

Documentation also points that ontapi access should be enough:
https://docs.netapp.com/us-en/trident/trident-use/ontap-nas.html#ontap-nas-driver-details
https://docs.netapp.com/us-en/trident/trident-use/ontap-nas-prep.html#update-authentication-methods-or-rotate-credentials
Any clarification what could cause this behaviour and how to mitigate it,  is very much appreciated!

Ontap is in a transition phase from the traditional API (ZAPI, ontapi) to a RESTful API. Trident supports both APIs. It will attempt RESTful API first, then fallback to the traditional API if needed. I guess that initial REST API attempt is what triggers the log messages for you.
With future Ontap versions only REST API will be available so I'd suggest to enable http (which really is https/TLS of course) for the user account already.
So for your setup:
security login create -username tridentuser -application http -authmethod password -role vsadmin –vserver vserver123

Ontap is in a transition phase from the traditional API (ZAPI, ontapi) to a RESTful API. Trident supports both APIs. It will attempt RESTful API first, then fallback to the traditional API if needed. I guess that initial REST API attempt is what triggers the log messages for you.
With future Ontap versions only REST API will be available so I'd suggest to enable http (which really is https/TLS of course) for the user account already.
So for your setup:
security login create -username tridentuser -application http -authmethod password -role vsadmin –vserver vserver123

Many thanks @wonderland that makes sense, will update the storage team with this info, thanks for the quick reply!


Filetree - deleting directories not working - file not found and may crash

I tried to delete a directory that had subdirs and then I tried deleting one of the subdirs - both failed.
Then I created an empty directory and tried to delete it. That failed and then the panic.
Aside: None of the files in the directories are "stored" but I notice the directories themselves always display as "stored".
fatal: pathspec 'sims/temp' did not match any files
2020/02/04 21:20:17 fatal: pathspec 'sims/temp' did not match any files
2020/02/04 21:22:29 fatal: pathspec 'sims/temp/axFiles' did not match any files
fatal: pathspec 'sims/temp/axFiles' did not match any files
2020/02/04 21:23:45 fatal: pathspec 'fooFolder' did not match any files
fatal: pathspec 'fooFolder' did not match any files
panic: runtime error: index out of range [-1]
goroutine 872 [running]:
github.com/goki/gi/gi.(*TextLink).Bounds(0xc01181bb58, 0xc02a5b5340, 0x4438bd334537c1dc, 0xd, 0x2b, 0xc01181bb80, 0xc01181bb40)
/Users/rohrlich/go/src/github.com/goki/gi/gi/text.go:754 +0x1a6
github.com/goki/gi/giv.(*TextView).MouseMoveEvent.func1(0x517d640, 0xc024804000, 0x516f940, 0xc01563e000, 0x1, 0x4ddb340, 0xc075a170e0)
/Users/rohrlich/go/src/github.com/goki/gi/giv/textview.go:4379 +0x298
github.com/goki/gi/gi.(*WinEventRecv).Call(0xc01181bd88, 0x516f940, 0xc01563e000, 0x1, 0x4ddb340, 0xc075a170e0)
/Users/rohrlich/go/src/github.com/goki/gi/gi/events.go:116 +0x8c
github.com/goki/gi/gi.(*EventMgr).SendEventSignal(0xc01563e190, 0x5106cc0, 0xc075a170e0, 0x1)
/Users/rohrlich/go/src/github.com/goki/gi/gi/events.go:237 +0x54f
github.com/goki/gi/gi.(*Window).ProcessEvent(0xc01563e000, 0x5106cc0, 0xc075a170e0)
/Users/rohrlich/go/src/github.com/goki/gi/gi/window.go:1508 +0x8ec
github.com/goki/gi/gi.(*Window).EventLoop(0xc01563e000)
/Users/rohrlich/go/src/github.com/goki/gi/gi/window.go:1445 +0x36
created by github.com/goki/gi/gi.(*Window).GoStartEventLoop
/Users/rohrlich/go/src/github.com/goki/gi/gi/window.go:860 +0xa6

I pushed pretty hard on the filetree with add/delete/duplicate/rename etc and also some pushing on git commands and so far only found this one bug.

great!  yep I probably didn't test deleting dirs..  I will take a look at this.
Btw, in git, directories themselves are never literally stored in the repository -- only implicitly if their children are.

Yes, knew that :)
Fixed - don't try to delete dirs from repo.


CVE-2023-4863

Dear developers,
I am finding the current webp vulnerability CVE-2023-4863 in the DefectDojo I am creating. I did not see an issue yet, so I am creating one to keep track on this.
Best regards,
Erhan

@erhan- Thanks for reporting the issue - for future reference, any potential security issue should be reported per https://github.com/DefectDojo/django-DefectDojo/blob/dev/readme-docs/SECURITY.md
We've already updated for the webp issue you mention in the dev branch by updating the library even though DefectDojo does not appear to every call  the vulnerable code in webp:
https://github.com/DefectDojo/django-DefectDojo/blob/dev/requirements.txt#L39
A new release (2.27.0) will go out on Monday which includes the newer version of the library that includes libwebp shared libraries (.so files).

Thanks for taking the time to write this message and also explaining that DD is not really affected.

The nginx image for 2.27.0 is also being flagged for this, I realize its mostly just a matter of time.


Client Allocation Request for: Zoland-TV

Client Allocation Questions
Core Information

Name: Zoland-TV
Website / Social Media: [http://www.zolandtv.com/ http://www.zolandanimation.com/](http://www.zolandtv.com/ http://www.zolandanimation.com/), (Social Media: https://twitter.com/ZolandGroup https://www.facebook.com/zolandanimation http://www.instagram.com/zolanddistribution http://www.youtube.com/c/zolandanimation https://www.linkedin.com/company/zolanddistribution/, web)
Region: Greater China
DataCap Requested: 5 TiB
Addresses to be Notarized: f1ka66piios6kglll2ace2amu26kxl4m4rgh5m56q
Notary Requested: Anne

Declaration：Get contact from Slack.
Hey @zolandtv  , thanks for the request. I have some questions regarding your request. Please describe answers as much specificity as you can.
1，Can you introduce yourself? Where can I find you online? (Github, Twitter, Linkedin handle, Wechat, etc.),
2，If you represent an organization, what is the name, nature, registration country, time since inception, and named officers of your organization?
3，In which region(s) do you intend to use the DataCap?,
4，What is your use case for the DataCap being requested?
5，Do you plan to specify advanced features when utilizing a DataCap in your storage deal? If so, please specify the feature.,
6，How many miners do you plan on allocating the DataCap? If you have a specific miner(s) you would like to allocate a DataCap, please specify.
7，Do you agree on providing transparency of DataCap allocation decisions & execution details in public to prevent excessive/unfair (20% of DataCap allocation) self-dealing?
Moreover, I encourage you to take a look of my allocation https://github.com/filecoin-project/notary-governance/issues/732 which shows how I utilize the assessment. Thanks.

@zolandtv  Any update?

@1ane-1 Sorry for the late reply.
You can contact me through github.
In 2003, ZolandTV Group realized cross domain industry upgrading and established ZhejiangZoland animation Co., Ltd. to build a pan entertainment ecosystem integrating animation creativity, animation distribution, authorized marketing, technology research and development, cultural tourism, animation performance and other industries. At present, it has become one of the largest original animation companies in China.
Our main name is: Zoland-TV. established in '03, and our main business is currently in China. The small 5t plan we are currently applying for is for use in China, and mainly stores public data from the work computers of corporate content employees.

I will send you more information via slack, please check it out.

I will send you more information via slack, please check it out.

Thank you! I will check it.

signed, pls check it.


Prepare for Fedora packaging

I tested compiling with and without turbojpeg-devel installed, both cases were building fine with the desired outcome.

Compiles on Kubuntu 20.04 with contrib/libjpeg-turbo-dev. Test applications run.
Thank you for your contribution!


Performance impact on Ruby on Rails ?

Description of the bug
We had a sampling rate of 0.1 enabled for quite some time.
As soon as we have enabled the sampling to 1.0 (100%) we had terrible performances on various spans like the ones related to PostgreSQL for example

We pushed for similar changes on other part of the our infrastructure (python and go code) and did not face similar issues.
We even saw some span (still related to database) usually taking less than 100ms taking up to 30s for no reasons, we had to rollback the configuration to the 0.1 sampling rate.
It's not clear what could cause this ? I would expect some performance impact but nothing near a x300 duration on the operations.
I did not do any kinf of optimization on the OTLP exporter settings or thing like that.
We are using Ruby on Rails with puma.

Share details about your runtime
Operating system details: Linux, Ubuntu 20.04 LTS
RUBY_ENGINE: "ruby"
RUBY_VERSION: "3.2.1"
RUBY_DESCRIPTION: "ruby 3.2.1 (2023-02-08 revision 31819e82c8) [x86_64-linux]"
Configuration
OTEL_EXPORTER_OTLP_ENDPOINT=http://xxx:4318
OTEL_LOG_LEVEL=warn
OTEL_RESOURCE_ATTRIBUTES=env=us,version=2.3.21
OTEL_SERVICE_NAME=back
OTEL_TRACES_SAMPLER_ARG=1.0
OTEL_TRACES_SAMPLER=parentbased_traceidratio

OpenTelemetry::SDK.configure do |config|
  is_otel_disabled = ENV.fetch('OTEL_SDK_DISABLED', 'false').to_b
  is_logging_disabled = Rails.const_defined?(:Console) || Rails.env.test? || is_otel_disabled

  config.service_name = service_name

  if is_logging_disabled
    config.logger = Logger.new(nil)
  else
    config.logger = Logger.new($stdout, level: ENV.fetch('OTEL_LOG_LEVEL', Logger::INFO), formatter: OpenTelemetryLogFormatter.new)
  end

  next if is_otel_disabled

  # Configure custom resource
  config.resource = OpenTelemetry::SDK::Resources::Resource.create(
    'git.commit.sha' => xxx::Git.commit,
    'git.repository_url' => xxx::Git::REPOSITORY
  )

  # Rails
  config.use 'OpenTelemetry::Instrumentation::Rack'
  config.use 'OpenTelemetry::Instrumentation::Rails'
  config.use 'OpenTelemetry::Instrumentation::Rake'
  config.use 'OpenTelemetry::Instrumentation::ActiveSupport'
  config.use 'OpenTelemetry::Instrumentation::ActionPack'
  config.use 'OpenTelemetry::Instrumentation::ActiveRecord'
  config.use 'OpenTelemetry::Instrumentation::ActionView'

  # Network libraries
  config.use 'OpenTelemetry::Instrumentation::ConcurrentRuby'
  config.use 'OpenTelemetry::Instrumentation::Faraday'
  config.use 'OpenTelemetry::Instrumentation::Net::HTTP'

  # Storage
  config.use 'OpenTelemetry::Instrumentation::AwsSdk'

  # Database and queues
  config.use 'OpenTelemetry::Instrumentation::Bunny'
  config.use 'OpenTelemetry::Instrumentation::PG'
  config.use 'OpenTelemetry::Instrumentation::Redis'

  # Worker
  config.use 'OpenTelemetry::Instrumentation::Sidekiq', {
    propagation_style: :child,
  }
end

What kind of optimization could be done ?
Could it be some kind of locking issue with the puma multiprocessing ? (I did see a warning at the startup about the opentelemetry thread started in puma), I found this https://github.com/open-telemetry/opentelemetry-ruby/pull/472 that could disable it but I am not sure this would have a performance impact ?
I see this post contains a few settings that can be tuned, would this help ?
https://uptrace.dev/get/opentelemetry-ruby.html#already-using-otlp-exporter

I checked I could find both spans related to PostgreSQL, ActiveRecord and Redis having this similar issue (which are most of the spans our apps generate, so it's possible that all spans are affected).
Could there be some kind of lock or semaphore that could impact the code globally outside of the opentelemetry thread ? It looks like something is blocking it.
Unfortunately I could not to any profile in production, what tool would you recommend to be able to do that ?
I will try to reproduce outside of the production environment, strange thing is that I did not face this issue during previous load tests
This is a proper trace with a low sampling rate (and same load)

This is an example of trace with 100% sampling rate

On the DB size we don't see any potential lock that could explain that (and the same query appears normal and fast on the pg_stat_statements so it's definitively not an issue on the database side)

It's not the first time I see strange behaviour under high load, maybe it's related: https://github.com/open-telemetry/opentelemetry-ruby/issues/1418

Thanks for all of the details.
All otel data structures are threadsafe data structures and use a mutex during creation, mutation, processing etc...
The BatchSpanProcessor has a worker thread that uses a mutex to ensure thread safe access to the finished span buffer.
It's possible to disable this thread, or disable the batch spam processor altogether and see if it suffers from the same issues. That may help us narrow down the problem a bit.
For continuous production profiling the only open source option I can recommend is pyroscope but we would not of course be able to provide support for that.

Thank you for your response.
Which settings would you recommend to set for that ?
Also what would be the behaviour of the span buffer is full and it cannot keep up sending the spans faster than they arrive ? Could this be blocking the application code if it's full ?
I was thinking to test with multiple span processor to see if it fixes the issues (I think I saw an env variable that allows this)

Also what would be the behaviour of the span buffer is full and it cannot keep up sending the spans faster than they arrive ?

They get dropped.
https://github.com/open-telemetry/opentelemetry-ruby/blob/557baa162ecb9356c27883323b1153ad58fc7191/sdk/lib/opentelemetry/sdk/trace/export/batch_span_processor.rb#L76-L90

Could this be blocking the application code if it's full ?

I unfortunately have a few applications that regularly overflow the buffer when they see a sudden burst in traffic, and we have not observed this behaviour. That said it doesn't mean it isn't possible.

I was thinking to test with multiple span processor to see if it fixes the issues (I think I saw an env variable that allows this)

The BatchSpanProcessor is the production ready one. There's a simple span processor in the repo that I would caution you from using in a production environment, I don't know of anyone using it in their prod env. If that's what you meant.

I wonder if you're encountering a similar situation to the one described here https://deploy.equinix.com/blog/opentelemetry-provisioning-observability-distributed-tracing/

I'll try to avoid casually suggesting things for you to "test" in your production environment, but I wonder if it is worth drastically dropping down the amount of instrumentation you have enabled. Starting simply with Rack, ActionPack, and Net::HTTP. The idea behind using that subset is that you would get the minimal amount of instrumentation so that it doesn't break the trace graph if this service is part of a larger constellation of apps.
From there you could slowly increase the sample rate until it just works, or it breaks. This approach will reduce the amount of touch points opentelemetry has on your service. Assume this works, I would individual add single instrumentation packages one at a time.

Another question, have you added a lot/any of your own custom instrumentation to the application in question?

The BatchSpanProcessor is the production ready one. There's a simple span processor in the repo that I would caution you from using in a production environment, I don't know of anyone using it in their prod env. If that's what you meant.

I was more talking about tuning up the OTEL_BSP_MAX_CONCURRENT_EXPORTS they are showing there that I think is related to the BatchSpanProcessor, also with all the others OTEL_BSP_... variables. But I this concurrent exports one is not supported in ruby ? I could not find any trace of it in the code of this repository.

Another question, have you added a lot/any of your own custom instrumentation to the application in question?

We however add various attributes in the existing automated spans (like user id, upload id, etc...).

I would retry to do some load tests with the various OTEL integration enabled and not (I will try to make them configurable via environment variables to avoid lot of code changes every time we want to toggle something)
Also so far we don't set anything related to this issue https://github.com/open-telemetry/opentelemetry-ruby/pull/472 , could this be the cause ? (We do get a warning when the puma master process start, but from what i understood it should take care automatically of starting a new thread on each worker ?)

I was more talking about tuning up the OTEL_BSP_MAX_CONCURRENT_EXPORTS they are showing there that I think is related to the BatchSpanProcessor, also with all the others OTEL_BSP_... variables. But I this concurrent exports one is not supported in ruby ? I could not find any trace of it in the code of this repository.

Haven't encountered that env var before. A quick search tells me this exists in rust and no results elsewhere in any of the official language implementations.

Another question, have you added a lot/any of your own custom instrumentation to the application in question?

We however add various attributes in the existing automated spans (like user id, upload id, etc...).

As you linked uptrace docs in your response (I'm not familiar with them) I would encourage you not to use the approach they suggest for adding attributes in their example.
  tracer.in_span('GET /posts/:id') do |child1|
    child1.set_attribute('http.method', 'GET')
    child1.set_attribute('http.route', '/posts/:id')
    child1.set_attribute('http.url', 'http://localhost:8080/posts/123')
    child1.set_attribute('http.status_code', 200)
    child1.record_exception(ArgumentError.new('error1'))
  end

Languages like Ruby or Go for example will make sure spans are thread safe, and they will hold a lock when setting attributes.
I would be surprised if this was the source of your problem, but regardless this is best practices if you want to minimize the impact instrumentation has on the code it is measuring.
Here's an explanation I have saved from a while back.
# Adding attributes on span creation is the most performant way
# to add attributes in general and should be the default choice
# when possible.
def a(arg)
  Tracer.in_span("a", attributes: { "arg" => arg }) {}
end

# Sometimes we want to add an attribute to a span but
# we don't have it when we're creating the span.
# The span method `set_attribute` allows you set an attribute
# on an existing span.
#
# Warning: As span mutation is thread safe, that means setting an attribute
# on an existing will hold a mutex lock.
def a
  Tracer.in_span("a") do |span|
    result = something_interesting
    span.set_attribute("result", result)
  end
end

# alternate form
def a
  Tracer.in_span("a") do |span|
    result = something_interesting
    span["result"] = result
  end
end

# Really bad
# Never do this as you're hitting a mutex lock for each call to set_attribute
def a
  Tracer.in_span("a") do |span|
    result = something_interesting
    span.set_attribute("a", result.a)
    span.set_attribute("b", result.b)
    span.set_attribute("c", result.c)
    span.set_attribute("d", result.d)
  end
end

# alternate form of really bad
def a
  Tracer.in_span("a") do |span|
    result = something_interesting
    span["a"] = result.a
    span["b"] = result.b
    span["c"] = result.c
    span["d"] = result.d
  end
end

# If we have a set of attributes only available after span creation.
# We're still hitting the mutex lock, but only once.
def a
  Tracer.in_span("a") do |span|
    result = something_interesting
    span.add_attributes({
      "a" => result.a,
      "b" => result.b,
      "c" => result.c,
      "d" => result.d,
    })
  end
end

I would retry to do some load tests with the various OTEL integration enabled and not (I will try to make them configurable via environment variables to avoid lot of code changes every time we want to toggle something)

Disabling by env var is supported by default. See -> https://github.com/open-telemetry/opentelemetry-ruby-contrib/blob/main/instrumentation/base/lib/opentelemetry/instrumentation/base.rb#L313-L327
Here's an example using Active Record.
ENV["OTEL_RUBY_INSTRUMENTATION_ACTIVERECORD_ENABLED"] = "false"

Also so far we don't set anything related to this issue https://github.com/open-telemetry/opentelemetry-ruby/pull/472 , could this be the cause ? (We do get a warning when the puma master process start, but from what i understood it should take care automatically of starting a new thread on each worker ?)

I'm using the default configuration here for all of our applications without issue. I think more investigation needs to be done before we can point at one particularly piece of code being responsibly for the issue.

Thanks for the info.
We usually only put 1 attribute on the existing spans (sometime 2, never more) so it's unlikely to be related to this.
I will continue to investigate, so far it looks like I get better performances with those settings than the default ones
OTEL_BSP_MAX_BATCH_SIZE=2048
OTEL_BSP_MAX_QUEUE_SIZE=16384
OTEL_BSP_SCHEDULE_DELAY=2000
OTEL_RUBY_BSP_START_THREAD_ON_BOOT=false

But not yet tested in production where the biggest impact was

With thread boot set to false you'll need to flush the span buffer yourself using tracer provider force_flush.
This is best done either using either Rack's after_response or the Rack Events API.
For background workers you'll want to configure the options to force active job to do the same.

I think the thread boot to false just prevents it from working on the initial puma master no ?
I could properly get spans after having set it to false


use vxlan set subnet failed!

Describe your issue here

Useful
Info

Versions
Rancher v1.6.0 Cattle: v0.179.7 UI: v1.6.1

Access
localauth admin

Orchestration
Cattle

Route
settings.projects.edit-template

in vxlan's config,I set the subnet <IP_ADDRESS>/24,but the container created by rancher is not work correctly。it's netmask is <IP_ADDRESS> why?

@694982827
Can you share the vxlan cniconfig ? You can view the Vxlan stack "view config" to find these contents.

now I have another question,the container created by vxlan failed to ping  <IP_ADDRESS> but it can success ping the host ip.and the host could also ping  <IP_ADDRESS> and <IP_ADDRESS>/16.
netwofk config:

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto br0
iface br0 inet static
address <IP_ADDRESS>
network <IP_ADDRESS>
netmask <IP_ADDRESS>
broadcast <IP_ADDRESS>
gateway <IP_ADDRESS>
dns-nameservers <IP_ADDRESS>
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0

root@expcloud13121843:~# ifconfig
br0       Link encap:Ethernet  HWaddr 1a:53:82:41:b8:3e
inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
inet6 addr: fe80::e051:a7ff:fe76:4eea/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:1169003 errors:0 dropped:0 overruns:0 frame:0
TX packets:237725 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:231944004 (231.9 MB)  TX bytes:277995390 (277.9 MB)
docker0   Link encap:Ethernet  HWaddr 02:42:8d:24:00:01
inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
inet6 addr: fe80::42:8dff:fe24:1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:2766545 errors:0 dropped:0 overruns:0 frame:0
TX packets:2862510 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4258594791 (4.2 GB)  TX bytes:737480141 (737.4 MB)
eth0      Link encap:Ethernet  HWaddr e2:51:a7:76:4e:ea
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:1460782 errors:0 dropped:0 overruns:0 frame:0
TX packets:530780 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:611847070 (611.8 MB)  TX bytes:297782618 (297.7 MB)

root@expcloud13121843:~# brctl show
bridge name	bridge id		STP enabled	interfaces
br0		8000.1a538241b83e	no		eth0
veth-cni
veth0c70dfa5
veth45e4464c
veth577a6acc
veth7b8dbac9
vethb1dcc392
vethb66f3cb5
vethecd04d67
vethee332d5e
docker0		8000.02428d240001	no		veth-doc
veth0ff0dcb
veth1644d22
the vxlan config is :
*************************** rancher-compose.yml**********************************
version: '2'
services:
vxlan:
network_driver: &id001
cniConfig:
10-rancher-vxlan.conf:
bridge: br0
bridgeSubnet: <IP_ADDRESS>/24
hairpinMode: true
hostNat: true
ipam:
isDebugLevel: 'false'
logToFile: /var/log/rancher-cni.log
routes:
- dst: <IP_ADDRESS>/32
type: rancher-cni-ipam
isDebugLevel: 'false'
isDefaultGateway: true
linkMTUOverhead: 50
logToFile: /var/log/rancher-cni.log
mtu: 1500
name: rancher-cni-network
type: rancher-bridge
defaultNetwork:
dns:
- <IP_ADDRESS>
dnsSearch:
- rancher.internal
hostPorts: true
name: vxlan
subnets:
- networkAddress: <IP_ADDRESS>/24
cidrSize: 24
defaultPolicyAction: allow
name: Rancher VXLAN
start_on_create: true
cni-driver:
network_driver: *id001
scale: 1
start_on_create: true
****end  rancher-compose.yml
*************************docker-compose.yml ******************
version: '2'
services:
vxlan:
cap_add:
- NET_ADMIN
image: rancher/net:v0.11.2
environment:
RANCHER_DEBUG: 'false'
network_mode: vxlan
logging:
driver: json-file
options:
max-file: '2'
max-size: 25m
volumes_from:
- cni-driver
ports:
- 4789:4789/udp
command:
- start-vxlan.sh
labels:
io.rancher.container.agent_service.vxlan: 'true'
io.rancher.sidekicks: cni-driver
io.rancher.cni.link_mtu_overhead: '0'
io.rancher.service.selector.link: io.rancher.internal.service.vxlan=true
io.rancher.network.arpsync: 'true'
io.rancher.container.create_agent: 'true'
io.rancher.network.macsync: 'true'
io.rancher.scheduler.global: 'true'
io.rancher.internal.service.vxlan: 'true'
cni-driver:
privileged: true
image: rancher/net:v0.11.2
network_mode: host
logging:
driver: json-file
options:
max-file: '2'
max-size: 25m
pid: host
command:
- sh
- -c
- touch /var/log/rancher-cni.log && exec tail ---disable-inotify -F /var/log/rancher-cni.log
labels:
io.rancher.network.cni.binary: rancher-bridge
io.rancher.container.dns: 'true'
io.rancher.service.selector.link: io.rancher.internal.service.vxlan=true

@694982827

You mentioned before using the subnet is <IP_ADDRESS>/24, but your cniconfig setting is <IP_ADDRESS>/24. I dont know why you set this?
what's your br0 support for? Rancher cannot run with other platform, because iptable rules may conflict.

@niusmallnan
1、Before the subnet is <IP_ADDRESS>/24  and the config setting is <IP_ADDRESS>/24 . Because of some reasons I changed the subnet <IP_ADDRESS>/24  to <IP_ADDRESS>/24 and now  the cniconfig setting is <IP_ADDRESS>/24。
2、I use br0 in vxlan's config，when creating vxlan there is a blank  "Docker Bridge" and I set br0 .  I just want to set  container on a same subnet as host.

If you set br0 for vxlan, you must set docker bridge to br0(not docker0).
If you  want to set container on a same subnet as host, you cannot do this. Because vxlan is an overlay networking, use rancher-flat network will meet your requirements.

Here some points:
Rancher flat: https://github.com/rancher/rancher/issues/8673
Rancher macvlan: https://github.com/rancher/rancher/issues/8686
Rancher Calico: https://github.com/rancher/rancher/issues/8603

Is it the same way as http://www.greatops.net/?id=106&utm_source=tuicool&utm_medium=referral?I use this to meet myrequirements.

@niusmallnan
I changed to rancher-flat network,I add one host on UI is ok,but when I add another host,there is something  wrong with the first host,the first host's container will restart and it's host ip also changed to <IP_ADDRESS>.And the new added host could't show on UI,  Is there anything wrong with my config？

@694982827
You can add host with CATTLE_AGENT_IP param, this will avoid your problem.

@niusmallnan
I have tried to set CATTLE_AGENT_IP like this :

host-one:

docker run -e CATTLE_AGENT_IP="<IP_ADDRESS>"  --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.2 http://<IP_ADDRESS>:8080/v1/scripts/631738D6E1F6D4450F85:1483142400000:8NIp8TxMw94rWA9iDD9fVUzc4EA

host-two::

docker run -e CATTLE_AGENT_IP="<IP_ADDRESS>"  --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.2 http://<IP_ADDRESS>:8080/v1/scripts/631738D6E1F6D4450F85:1483142400000:8NIp8TxMw94rWA9iDD9fVUzc4EA
but when I add host-two it seems does not work and  caused rancher/server exited

@niusmallnan
The second host is copyed by the first one,and I changed the ip、hostname、interface‘s mac address and restart it but is seems there is something conflict with the first when adding the second host。I changed a new host everything is ok。
and today I find another problem:using rancher-flat network  env,when add an user stack etcd on catalog,the etcd container could init successfully,the log output:
2017-5-27 11:34:32wget: server returned error: HTTP/1.1 404 Not Found
and I find the etcd container's run.sh:
http://rancher-metadata.rancher.internal/2015-12-19/self/stacks/name, it can't get is's data from here because there is no data.but when I changed the default env,it works well and the etcd contailer could get data from http://rancher-metadata.rancher.internal/2015-12-19/self/stacks/name.I can;t understand this，could you give me some advises?

@694982827
Are you in China？My wechat: niusmallnan.
See this: https://github.com/rancher/rancher/issues/8884#issuecomment-304425067

@niusmallnan
Yes , thank you very much!

Is this still on-going or can we close this? @694982827 @niusmallnan ?

@superseb
You can close it.

Same as:
https://github.com/rancher/rancher/issues/8938
https://github.com/rancher/rancher/issues/6511
Waiting for this PR to be merged: https://github.com/rancher/rancher-net/pull/60


update for mpw8

update to https://github.com/efabless/caravel_user_project/ main branch
update la and power pins

Sir how to update to updated commits on local system?

@radhe-2022 on a existing project? You could apply the patch locally:
https://github.com/proppy/tiny_user_project/pull/24.patch

@t-sasatani confirmed that the template works for his project:
https://github.com/t-sasatani/xls_workshop_tapeout_test_all_branch/actions/runs/3712018428

Merging so we can validate the flow at the next workshop: https://github.com/chipsalliance/silicon-notebooks/pull/37


switched expand and fnamescape so that files names with spaces are op…

…ened correctly

Essential!

Thanks, if you rewrite your commit message properly and force push the change, I will merge it. See http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html

Done (i think)!


NCL-3221 - Can't capture the removed repository file

Now it checks if the file exists, because when no repos are removed the
file doesn't get created.

Apologies for not seeing this PR earlier. LGTM! :)


BART: TransformerEncoder vs TransformerSentenceEncoder

Hi!
I’m working on porting BART to the transformers repo's API and was interested in whether the reasons for using TransformerEncoder instead of TransformerSentenceEncoder (which Roberta uses).  I read the code for both and there are some differences, but they seem mostly aesthetic.
Thanks in advance!

Do you know why attn_mask is not passed to self.self_attn after https://github.com/pytorch/fairseq/blob/4e48c4ae5da48a5f70c969c16793e55e12db3c81/fairseq/modules/transformer_layer.py#L84
Thats the only difference I see.

what about refactor TransformerSentenceEncoder as BertEncoder


ArangoDB doesn't start after a power failure!

my environment running ArangoDB
I'm using the latest ArangoDB of the respective release series:

[x] 3.1
On this operating system:
[x] Windows, version: 10

Last night the laptop that was running ArangoDB ran out of power, after connecting the power cord and starting the system up, we couldn't re-run arango.
this is the eoutput we get:
`
2017-03-15T12:45:54Z [2988] INFO ArangoDB 3.1.14 [win64] 64bit, using VPack 0.1.30, ICU 54.1, V8 <IP_ADDRESS>, OpenSSL 1.0.2a 19 Mar 2015
2017-03-15T12:45:54Z [2988] INFO using SSL options: SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2017-03-15T12:45:54Z [2988] INFO Starting up with role SINGLE
2017-03-15T12:45:54Z [2988] INFO Authentication is turned on
2017-03-15T12:45:54Z [2988] INFO Authentication system only
`
I also tried the arango-dfdb but I get the same output.

do you run it as a service? or from the commandline?

@dothebart from the command line

@dothebart this is what I get in windows logs when i try to run it as a service:
Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: arangod.exe
P2: <IP_ADDRESS>
P3: 58c67af2
P4: arangod.exe
P5: <IP_ADDRESS>
P6: 58c67af2
P7: c0000005
P8:<PHONE_NUMBER>159179
P9:
P10:
Attached files:
These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_arangod.exe_8b2705149ef69a4d17780af835f279a4757a337_ed432589_1b145204
Analysis symbol:
Rechecking for solution: 0
Report Id: af37cdab-8173-4669-b212-3df86432d5b3
Report Status: 4100
Hashed bucket:

@dothebart Report.wert Contains:
Version=1
EventType=APPCRASH
EventTime=131340575907196961
ReportType=2
Consent=1
ReportIdentifier=1130d7e5-0982-11e7-9bcb-28d2443d6463
IntegratorReportIdentifier=af37cdab-8173-4669-b212-3df86432d5b3
NsAppName=arangod.exe
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=arangod.exe
Sig[1].Name=Application Version
Sig[1].Value=<IP_ADDRESS>
Sig[2].Name=Application Timestamp
Sig[2].Value=58c67af2
Sig[3].Name=Fault Module Name
Sig[3].Value=arangod.exe
Sig[4].Name=Fault Module Version
Sig[4].Value=<IP_ADDRESS>
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=58c67af2
Sig[6].Name=Exception Code
Sig[6].Value=c0000005
Sig[7].Name=Exception Offset
Sig[7].Value=0000000000159179
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=10.0.10<IP_ADDRESS>.256.4
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=b75b
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=b75b84256649e3b31c439ba88c1240ba
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=d83b
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=d83b03b912d8017db86f710f7a8da748
UI[2]=D:\Program Files\ArangoDB3 3.1.14\usr\bin\arangod.exe
UI[5]=Check online for a solution (recommended)
UI[6]=Check for a solution later (recommended)
UI[7]=Close
UI[8]=ArangoDB - the multi-model database stopped working and was closed
UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available.
UI[10]=&Close
LoadedModule[0]=D:\Program Files\ArangoDB3 3.1.14\usr\bin\arangod.exe
LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\Windows\system32\KERNEL32.DLL
LoadedModule[3]=C:\Windows\system32\KERNELBASE.dll
LoadedModule[4]=C:\Windows\system32\ADVAPI32.dll
LoadedModule[5]=C:\Windows\system32\msvcrt.dll
LoadedModule[6]=C:\Windows\system32\sechost.dll
LoadedModule[7]=C:\Windows\system32\RPCRT4.dll
LoadedModule[8]=C:\Windows\system32\SHLWAPI.dll
LoadedModule[9]=C:\Windows\system32\combase.dll
LoadedModule[10]=C:\Windows\system32\GDI32.dll
LoadedModule[11]=C:\Windows\system32\USER32.dll
LoadedModule[12]=C:\Windows\system32\WS2_32.dll
LoadedModule[13]=C:\Windows\system32\NSI.dll
LoadedModule[14]=D:\Program Files\ArangoDB3 3.1.14\usr\bin\LIBEAY32.dll
LoadedModule[15]=D:\Program Files\ArangoDB3 3.1.14\usr\bin\SSLEAY32.dll
LoadedModule[16]=C:\Windows\SYSTEM32\WINMM.dll
LoadedModule[17]=C:\Windows\SYSTEM32\MSWSOCK.dll
LoadedModule[18]=D:\Program Files\ArangoDB3 3.1.14\usr\bin\VCRUNTIME140.dll
LoadedModule[19]=C:\Windows\SYSTEM32\WINMMBASE.dll
LoadedModule[20]=C:\Windows\system32\cfgmgr32.dll
LoadedModule[21]=C:\Windows\SYSTEM32\ucrtbase.dll
LoadedModule[22]=C:\Windows\SYSTEM32\DEVOBJ.dll
LoadedModule[23]=C:\Windows\system32\kernel.appcore.dll
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=ArangoDB - the multi-model database
AppPath=D:\Program Files\ArangoDB3 3.1.14\usr\bin\arangod.exe
NsPartner=windows
NsGroup=windows8
ApplicationIdentity=A6DF77E21014518916D4DA71293ABFF7

can you try whether it aborts on error as described here (using procdump) ?
https://github.com/arangodb/arangodb/blob/devel/README_maintainers.md#windows-debugging

@dothebart
[17:33:20] Exception: C0000005.ACCESS_VIOLATION
[17:33:20] The process has exited.
[17:33:20] Dump count not reached.

hm :-/ ACCESS_VIOLATION usually should exactly not make it tell Dump count not reached. but write a dump file

@dothebart
I changed some switches, -e 1, now I have a 400MB Dump File

ok, it should zip pretty well, can you provide it somewhere for download?

@dothebart
https://ufile.io/da27f1

OK, failing to construct the RocksDBIndexFeature - seems to be some initialization sequence issue @jsteemann
RocksDBIndex.cpp:211

@dothebart Thanks, I'll mention the WinDBG .ecxr here
0:000> .ecxr
rax=0000000000000000 rbx=0000009744b28060 rcx=0000000000000000
rdx=0000000000000000 rsi=0000000000089bc3 rdi=0000009727a7bbf8
rip=00007ff6f26f9179 rsp=0000009727a7b9f0 rbp=0000009727a7bb40
r8=0000009744b30670  r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000008
r14=0000009744b3ae70 r15=0000000000000001
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
arangod!arangodb::RocksDBFeature::db [inlined in arangod!arangodb::RocksDBIndex::RocksDBIndex+0x39]:
00007ff6f26f9179 488b88f0000000  mov     rcx,qword ptr [rax+0F0h] ds:00000000000000f0=????????????????

thanks, yes, its a NULL-pointer to an object, and the object member is at 0xF0h I guess.

@jsteemann How can I test the fix you've committed? should I compile the source code or you will release a new build?

i strongly recommend you to switch to Rust or Go as the underling programming language of Arangodb.
C++ is far too hard to manipulate in a big project.

we may also try to port it to an abacus, it will be most probably more resistant to power outages then.

Reopening this, as it is not officially resolved.

@jsteemann please give me a new release to test the fix, I tried to compile it myself, but its really time consuming on a 2011 Mac Book Pro!

I've uploaded a zip package containing the fix to: https://www.arangodb.com/repositories/ArangoDB3-3.1.14-1_win64.zip
Please test it, so we can see whether its fixed for 3.1.15

@dothebart The Fix Worked! Thanks ;)
D:\Program Files\ArangoDB3-3.1.14-1_win64>usr\bin\arangod.exe
2017-03-16T16:06:27Z [4476] INFO ArangoDB 3.1.14 [win64] 64bit, using VPack 0.1.30, ICU 54.1, V8 <IP_ADDRESS>, OpenSSL 1.0.2a 19 Mar 2015
2017-03-16T16:06:27Z [4476] INFO using SSL options: SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2017-03-16T16:06:27Z [4476] INFO Starting up with role SINGLE
2017-03-16T16:06:27Z [4476] INFO Authentication is turned on
2017-03-16T16:06:27Z [4476] INFO Authentication system only
2017-03-16T16:06:28Z [4476] INFO running WAL recovery (1 logfiles)
2017-03-16T16:06:28Z [4476] INFO replaying WAL logfile 'D:\Program Files\ArangoDB3-3.1.14-1_win64\var/lib/arangodb3\journals\logfile-489161.db' (1 of 1)
2017-03-16T16:06:28Z [4476] INFO WAL recovery finished successfully
2017-03-16T16:06:29Z [4476] INFO JavaScript using startup 'D:\Program Files\ArangoDB3-3.1.14-1_win64/usr/share/arangodb3/js', application 'D:\Program Files\ArangoDB3-3.1.14-1_win64\var/lib/arangodb3-apps'
2017-03-16T16:06:32Z [4476] INFO using endpoint 'http+tcp://<IP_ADDRESS>:8529' for non-encrypted requests
2017-03-16T16:06:35Z [4476] INFO ArangoDB (version 3.1.14 [win64]) is ready for business. Have fun!

ArangoDB 3.1.15 is available for download now.


fix(nodebuilder/share): Only provide necessary components to Bridge

Bridge does not need:

shrexeds.Client
shrexnd.Client
*shrex.Getter
peer.Manager

Bridge and Lights will also spawn EnsurePeers lifecycle from nodebuilder instead (as they don't use peer.Manager yet)

We discussed that we want to start EnsurePeers in discovery lifecycle methods instead, as we anyway want to call it everywhere. This means we should also stop calling it for fulls in peer manager


Enhanced types for Assoc

Updating types for Assoc for better return types when the type for value is the same for the type of the key on the object
Also added tests

Closing. Will re-apply changes in new types repo as described here: https://github.com/ramda/ramda/discussions/3329


Files & History spinner

When looking at a movie, sometimes the files don't load until you refresh.
Maybe a cancellation error? Enable Sentry reporting for a while?

I can't replicate this anymore.


proposal: log/slog: Record.Attrs should return a book for iterators

given the syntax in #61405, the only thing missing from making log/slog.Record.Attrs an iterator is the bool return, perhaps we should preemptively add it?
cc @jba @rsc

(typo in title: I'm sure you meant bool for iterators, not book)

Or perhaps #61405 will drop the bool. It is looking that way.


A - fundamentally unsupported - attempt to initialize a 2D (multidimensional) array with a literal results in a confusing error message

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
Related:

#20676

[int[,]]  ((1,2), (2,3))

Expected behavior
An error message indicating that initializing multidimensional arrays with (jagged) array literals isn't supported.
Actual behavior
InvalidArgument: Cannot convert the "System.Object[]" value of type "System.Object[]" to type "System.Int32".

This error message is confusing, and doesn't convey the root cause of the problem.
Curiously, (mistakenly) trying to assign a 1D array at least correctly reflects the multidimensionality of the target type, and the prefix changes from InvalidArgument to OperationStopped:
# -> "OperationStopped: Unable to cast object of type 'System.Int32[]' to type 'System.Int32[,]'."
[int[,]] @(1)

Error details
No response
Environment data
PowerShell 7.4.0-rc.1

Visuals
No response

Two dimensions (1024² = 1048576):
[Int128[][]] $1048576 = ((1MB, 2MB),  (3MB, 4MB))
$1048576
1048576
2097152
3145728
4194304
$1048576[0]
1048576
2097152
$1048576[0][1]
2097152
But three dimensions (1024³ =<PHONE_NUMBER>):
[Int128[][][]] $1073741824 = (((1MB, 2MB),  (3MB, 4MB)),   ((5MB, 6MB),  (7MB, 8MB)))
$1073741824
Length         : 2
LongLength     : 2
Rank           : 1
SyncRoot       : {1048576, 2097152}
IsReadOnly     : False
IsFixedSize    : True
IsSynchronized : False
Count          : 2
Length         : 2
LongLength     : 2
Rank           : 1
SyncRoot       : {3145728, 4194304}
IsReadOnly     : False
IsFixedSize    : True
IsSynchronized : False
Count          : 2
Length         : 2
LongLength     : 2
Rank           : 1
SyncRoot       : {5242880, 6291456}
IsReadOnly     : False
IsFixedSize    : True
IsSynchronized : False
Count          : 2
Length         : 2
LongLength     : 2
Rank           : 1
SyncRoot       : {7340032, 8388608}
IsReadOnly     : False
IsFixedSize    : True
IsSynchronized : False
Count          : 2
Write-Host $1073741824
1048576 2097152 3145728 4194304 5242880 6291456 7340032 8388608
P.S.  «6291456» is a site bug, not PowerShell bug.

@u20221022
Your array is single-dimensional, like $arr[0][1], multi-dimensional is like $arr[0,1]:
$arr = [int32[,]]::new(1,3)
$arr[0,2] = 3
$arr[0,0] = 1
$arr[0,1] = 2

$arr[0,0]  # 1

Indeed, @u20221022: What you're creating are jagged arrays (arrays whose elements are arrays), not multidimensional ones - but it is good to know that you can create strongly typed jagged arrays.
While jagged arrays are far more common in PowerShell and are usually sufficient, this issue is about true multidimensional arrays.

As for your specific examples:

[Int128[][]] $1048576 = ((1MB, 2MB),  (3MB, 4MB))

This creates a strongly typed two-element jagged array: each of its elements is also a 2-element array, so the total element counts is 4.

[Int128[][][]] $1073741824 = (((1MB, 2MB),  (3MB, 4MB)),   ((5MB, 6MB),  (7MB, 8MB)))

This creates a more deeply nested jagged array that has 8 elements in total.
Due to the nesting of arrays, the standard display formatting is somewhat unfriendly, but you can get a better sense of the structure with Format-Custom -InputObject $1073741824
(Write-Host seemingly recursively enumerates jagged arrays and produces a flat display of all elements.)
6291456 is what the 6MB number literal evaluates to, and I'm unclear on why you think there's a bug and what the relevance of the commit you link to is.

Now I understand that there is a difference between this two TYPES of arrays.  Sorry.  But I still do not understand what this difference is.  As to me, the jagged array is simply more convenient.
But you really do not see hyperlink «https: // github.com / PowerShell / PowerShell / commit / 6291456aa7aabb6f4285ee7eb3eaf3a5668c06d9» and pop-up window at ALL instances of number «6 291 456»?

@u20221022:
Ah, I see: Because you didn't use a code block around or inline code styling for 6291456 it turned into a hyperlink, because the number happened to match the ID (the first 7 (hex) digits of a commit-identifying SHA number) of a commit in this repo (you'll see the same number in the top-right corner of the linked page:
 and at the start of the linked page's last URL component)
I'd say it's a feature of the site (I wasn't aware of it), though it is unusual that no sigil is required to use it (for linking to issues and PRs by their issue number, you need # as the sigil, e.g. #20675 to link to this issue). My guess is that the assumption was that the likelihood of accidentally using such numbers, especially since they often contain hex. digits (e.g., 7c2cb52), which are usually prefixed with "0x" if meant to be mere numbers,  was low enough not require a sigil.
The short of it: to prevent such accidental links outside of fenced code blocks, enclose 7-digit hexadecimal numbers in `...`, which renders them as 6291456, for instance, without a hyperlink.

OK, thank you.

@u20221022

As to me, the jagged array is simply more convenient.

Yes, in PowerShell jagged arrays are easier to initialize and work with, and the performance advantage of true multidimensional arrays is usually not relevant.

But I still do not understand what this difference is.

In short:
A multidimensional array is a single array of fixed, preallocated size that can be accessed more efficiently than a jagged array. Elements must be accessed with a single index expression that specifies an index for every dimension (e.g., [1, 2] to access an element in a 2D array).
A jagged array is composed of multiple, nested arrays - an array whose elements happen to be arrays themselves.
Elements must be accessed with a separate index expression, one for each array involved (e.g., [1][2] to access an element in a singly nested jagged array).
While slower than multidimensional arrays (because multiple arrays must be accessed, in sequence), jagged arrays are more flexible:

They allow each nested array to have a different element count (hence the name "jagged": you can think of a 2D-like (singly nested) jagged array with elements of varying count as forming a table of values with a "jagged edge" rather than forming a perfect square of values as  a true 2D array would).
They only occupy as much space as needed, whereas a multidimensional array invariably preallocates space for all elements it can hold (e.g., a jagged array [int[][]] $arrJagged = (1..100), $null contains 102 elements in total (2 for the outer array, and 100 for the nested array in its [0] element, none in the element [1]), whereas multidimensional array $arr2D = [int[,]]::new(2, 100) invariably contains 200 elements (the result of multiplying its dimensions, 2 * 100).
They allow swapping out nested arrays later, by assigning a different array to an element of the enclosing array (e.g., $arrJagged[1] = 1..5)

Thank you again.
Until I plan to climb into such wilds. You can see it from my issues at this site.
I myself suspected about difference in speed and performance.  I remember the optimization tricks of Fortran on slow 360/370 series.
Thank you again and again.

Looks like the conversion path LanguagePrimitives.ConvertUnrelatedArrays gets hit here which apepars not to check array rank. Definitely should at least fail more gracefully

Engine-WG reviewed this issue today and agreed that the error message should be improved. Marked this issue as a bug.

So the bug was corrected or not?


Installation error on mac, xampp

I'm following your installation steps, but when I write "php composer.phar install" I'm getting this error :

What's wrong? What can I do?
Thank you :)

Awesome


Preventdefault touch ios fix2

possible fix for #127
supersedes #130
published as<EMAIL_ADDRESS>
After more digging and exploration this mimics v4 by using a ref callback to then set the touchstart and touchmove event listeners before any touch/swipe starts being tracked.
This solution is a work around for these issues:

the event delegation that react uses is a problem for ios and chrome by not having the { passive: false } option set, which is needed to call preventDefault
ios requires that the touchmove be attached before a touch event starts
passive option defaults to false when attaching the touch listeners to the swiping container via the ref, chrome/firefox and possibly ios

if you attach a touch listeners to document the passive option defaults to true
https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener#Improving_scrolling_performance_with_passive_listeners

TODO:

[ ] update tests

This solved my issue. @hartzis thank you very much for your help and especially for creating an alpha release of this feature 🎉
I'm going to close my PR

@JiiB Thank you for your help in testing and pushing to get this issue fixed! It is fun to go on deep dives like this to truly understand how this stuff works.
I got the tests updated tonight, should be able to get the docs updated and 5.1.0 published this week!


Support for configurable BPF map sizes

Description
This PR covers the phase 1 of configurable bpf map sizes.
The following maps' size can be configured

NAT frontend
NAT backend
NAT affinity
Conntrack map
IPsets map
Route map

GetMapInfo API is modified to return the current size of the pinned map. If the current size is different from the configured size,

Current map is repinned to map_path_old
New map is created with the new size
Old pinned map is deleted.

As part of the next phase, the data from the old pin will be copied to the newly created map. Conntrack map handling requires some special handling and will be taken up separately.
TC layer as well has the newly configured size and the max entries in the bpf maps will be set to the configured value during load time.
Related issues/PRs
Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

Didn't we agree to do some code clean up first to place map creations in a single place?

I am planning to do this as a separate task, as I need to look into enterprise as well.


Change frequency cutoff to use Hz instead of second

NiLearn frequency cutoff is in Hz.
Ni Stats uses inv frequency/TimePeriod in seconds.
Make it Hz.
@bthirion Anna-Louisa @alpinho is interested in working on this. Any info helpful to flesh this issue out shold be added here.

The default should be 0.01 Hz, as we do for resting-state (see in Nilearn).
I'm happy to help if you have specific questions.
Thx !

Where is this value stated in Nilearn?

https://github.com/nilearn/nilearn/blob/77dbcfee8fcf93929ad21c51bf607d798c40dd23/nilearn/input_data/nifti_masker.py#L101

My question was related to the value that is assigned to the cutoff frequency. You say that is set by default to 0.01Hz but I cannot find where that is declared.

Right, I can see it in the examples, yet not in the library itself. I still think that this is a good default.


:truck: update: Add 152nd CatchUp summary

Please review this PR! Let me know if there are any changes.

This looks good! Thank you so much, Mohit! We appreciate all your help!