Future plans for laz-perf

I been watching some of the recent pull requests and merges from @tmontaigu (really awesome BTW!) and also noticed that you've had a lot of recent work to add support to read from buffers in laspy. Do you think you'll try add laz-perf to laspy to get native LAZ io?
@hobu , also noticed you had a pull request from a few years ago to do just this, but it doesn't look like the laspy devs went anywhere with it...

@hobu , also noticed you had a pull request from a few years ago to do just this, but it doesn't look like the laspy devs went anywhere with it...

I am a lazpy developer 😄. I made a push to get laspy updated to support LAZ with lazperf, but I fizzled out of energy to do it, and I didn't want to integrate something that was incomplete. My branch at https://github.com/hobu/laspy/tree/lazperf-decomp was where I left off.
lazperf still needs to be caught up to LASzip 1.4, which it hasn't yet. It shouldn't be too difficult to do this, but it will take a chunk of time.

recent work to add support to read from buffers in laspy

Well I do have a branch that works, however as I replied in the laspy's issue it's not a clean solution, and
I'm not a fan of the design of the code, so I'm not sur what to do

Do you think you'll try add laz-perf to laspy to get native LAZ io?

I think I'll probably try something, I got lazperf working in my own las file reader so I know how to use it.
As for the laz-perf python bindings, I'll still need to make another PR, as there is bug at the moment
(the written chunk table at the end of the point data is not correct) and I want the api to feel less awkward  and more Pythonic

I'm not a fan of the design of the code, so I'm not sur what to do

Feel free to rip things apart and put them back together again in both laspy and lazperf-python. laspy wasn't designed with the expectation that we would also be doing LAZ, and I find the design itself a bit overwrought (there's 2 too many levels of indirection that aren't really needed). As long as the tests pass, 👌.

Well, I tried to refactor the code but I did not continue as it was getting hard, and I wasn't sure how the laspy maintainers would react to the massive changes that I would have done, because if I were to refactor laspy's code i would change ~65% of the code.
What I don't like in laspy's implementation is as you said the overthinking but also the fact that object-oriented techniques like inheritance are not used to handle cleanly the different file formats (mainly between [1.1, 1.2 ,1.3] files and 1.4 files).
But to me the worst things are the circular dependencies between the classess in laspy which makes the code very hard to follow and even harder to modify.
thats why I ended up starting something from scratch in the mean time

Well, I tried to refactor the code but I did not continue as it was getting hard, and I wasn't sure how the laspy maintainers would react to the massive changes that I would have done, because if I were to refactor laspy's code i would change ~65% of the code.

Speaking as a laspy maintainer, I would tell you not to feel bashful about doing this. I've felt that the laspy code is a bit too complex, and things like LAZ support are an opportunity to flush out things that aren't needed. My attempts to plumb things in were met with the same challenges you describe, but I didn't have three days to spend to go and fix it right while modernizing the code to today's Python idioms at the same time. If you have that time, go for it. We'll merge it.

@hobu I totally agree.

I also agree. Go ahead and improve laspy! None of us really spend much time on it so any contribution (that doesn't completely ruing things) is valuable. This sounds like an awesome contribution.

Then I will give the refactoring another shot.
I want laspy to work on streams/buffer and not just files present on the disk as I was able to quickly make it work in my pylas
As consequence, I will have to remove the mmap. Using a mmap by default does not make sense if we want a proper LAZ support. However I think i'll reintroduce the mmap as an optionnal way to read LAS data because it's still a cool way to handle large uncompressed files.
There is also  all the different file modes that I will remove because it will make less sense to still have them and the current handling of it in the code just make things more complex.
And I want to follow the python principle "We are all consenting adults", so If you don't want a file to be modified don't modify it.
In the end there are things that will inevitably have to change in the end user interface, and I don't know if I will achieve this refactoring

In the end there are things that will inevitably have to change in the end user interface, and I don't know if I will achieve this refactoring

If you find it necessary to change the API I'd say that is worth it to have proper support for laz and streams. No objections from me on that part. We can just call the next release Laspy 2.0 and be done with it.

I wholeheartedly agree with @hobu and @kbevers. Laspy started as a learning
exercise for both python and the LAS standard, so I have no objections to a
complete overhaul.
On Fri, Nov 16, 2018, 4:23 PM Bryce Frank<EMAIL_ADDRESS>wrote:

@tmontaigu https://github.com/tmontaigu
Supporting streams is excellent news to me. I am trying to implement a
Python native version of spatial indexing (i.e. Isenburgs .lax files). For
this to make any sense I need access to a stream. I will take a look at
your developments in pylas and see if that can work in the interim.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hobu/laz-perf/issues/43#issuecomment-439547526, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABolp4ytQKTXRv5Wd1ERh-TMaAhURkyKks5uvzrTgaJpZM4SMpZq
.

@brycefrank From what I've tested a few months ago, stream access (as in read the points one by one)
was extremely slow in python, depending on how LAX files allows to filter points (do we just need the index, or the x,y,z), it may be done in a faster way

@tmontaigu You are correct. I was able to check this on Friday with the same result. I don't think there is a straightforward way around this problem. My solution for now is to just call lasindex from the operating system and parse the .lax file after it is produced (similar to how laspy calls laszip).
Here is my progress so far:
https://github.com/brycefrank/laxpy

@brycefrank I don't know much about LAX files and what they allow us to do in terms of queries,
but if what we can get from the LAX file queries is a bunch of point indexes then we might be able to get something with acceptable performance in pure Python for LAS files.

@tmontaigu
Let's continue this over here if you don't mind!

@hobu @kbevers Could you please help me to read a .laz file in python? I am only able to read .las file using laspy.


Tabs error; everything work increasingly slowing down after error

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Current behavior 😯
When i get the following error, the application is working slowly, everything work increasingly slowing down.
I am using latest version of material ui.
MUI: The value provided to the Tabs component is invalid.
The Tab with this value ("0") is not part of the document layout.
Make sure the tab item is present in the document or that it's not display: none.
Expected behavior 🤔
No response
Steps to reproduce 🕹
No response
Context 🔦
No response
Your environment 🌎
No response

It doesn't look like this bug report has enough info for one of us to reproduce it.
Please provide a CodeSandbox (https://material-ui.com/r/issue-template-latest), a link to a repository on GitHub, or provide a minimal code example that reproduces the problem.
Here are some tips for providing a minimal example: https://stackoverflow.com/help/mcve


Is it possible to override the default config ?

Is it possible to override the default ajax provider ?
I need to set axios instead of the default one.
Or is it possible to add headers to the request ?
I need to add csrf token or if I could just override the defaults and use axios since I have headers set by default ?

This is related to FineUploader.
As stated in the documentation, I use fine-uploader-wrapper (documentation).
And to answer your question, you can set custom headers.


now.json seems to be deprecated

After the ZEIT rebrand to Vercel seems like the now.json file now should be named vercel.json. (see https://github.com/codesandbox/codesandbox-client/issues/5052).
I have also found the schema (hosted by Vercel): https://vercel.com/changelog/schema-autocomplete-and-validation-for-vercel-json.
I have tried to compare the two files but i couldn't tell if there are differences.
The docs for the vercel.json file: https://vercel.com/docs/cli#project-configuration
I'm willing to make a PR but i don't know if i should add vercel.json to the fileMatch, create a new schema and/or delete the older schema.
Any recommendations?

Hi! This project doesn't control the schema catalog, it just converts the catalog into a Neovim-friendly format. Please raise an issue upstream for issues related to adding/removing/changing schemas.
That being said, vercel.json is already supported: https://github.com/b0o/SchemaStore.nvim/blob/002444f8c9c69eaf1f755ef1bd91c0860ec56cb9/lua/schemastore/catalog.lua#L905-L910
Are you having trouble getting the Vercel schema working?

Oh! my bad 😅  nevermind me then.
Now that you mencioned, for files that have the *.json extension works fine, but I'm having trouble with files like .prettierrc and .eslintrc (the lsp is attached correctly, which is jsonls in my case).. But idk, i'm pretty much new
to nvim and I'm pretty sure that is just a misconfiguration on my side.
require('lspconfig').jsonls.setup {
  settings = {
    json = {
      schemas = require('schemastore').json.schemas(),
    },
  },
}

I'm using this setup. If you have any idea of what's possibly wrong you'll make my night haha.

Hi! thanks the quick reply.
My bad then 😅  sorry about that.

No worries! Everything is working for you then? Happy to help if not.

https://github.com/b0o/SchemaStore.nvim/issues/6#issue-1134653530


Jupyter Activity bar

Variables
Display notebook metadata? with ability to edit this.
Display cell metadata? (tags, attachments, etc) with ability to edit this
Display remote Jupyter files (users have requested this)?
Should we display the current kernel yet again here?

(Note: This is a place holder for the list of requirements and spec).

Is this still necessary? Aren't we getting a variable area?

What about:

Viewing/Editing cell metadata
Viewing/Editing notebook metadata
Remote File System (remote Jupyter or Jlab)

Not saying we need this right now, perhaps later.
The plan was to use this to do other stufff.

Okay I'll move it to the backlog then.


Remove "type to add your-quotes" post card

Remove "type to add your-quotes" post card after that Add plus icon on the bottom-right-corner and when user click on plus icon a popup should be open for posting qutoes..

Hey ! Can I work on this issue ?

@mritul You can work on this issue.. 😊😊😊

Hey @danishzayan  please assign this to me.

Hey @danishzayan please assign this to me.

@JatinSingh08 you can work on this issue 😊

@danishzayan I'm uploading the screenshots of the changes done. If it's fine I can raise a PR

@danishzayan I'm uploading the screenshots of the changes done. If it's fine I can raise a PR

@mritul ok I will do.

On opening popup

After adding the YourQuote

@danishzayan  I've implemented such that the user now has to click the plus button on the bottom right to open a popup that helps them add a YourQuote

@danishzayan I've implemented such that the user now has to click the plus button on the bottom right to open a popup that helps them add a YourQuote

@mritul good I like it your work ❤ create a PR i will merge you code...

@danishzayan That's great ! I've raised a PR but there seems to be a conflict to be resolved can you please look into it ? Thanks in advamce .

@danishzayan That's great ! I've raised a PR .

@mritul
fix all this issue.. sure after that i will merge your code...

@mritul Really you done great work 💗

@danishzayan That's great ! I've raised a PR .

 @mritul

when will open so popup has overlay transparent color...
when user click in app anywhere so popup should be close..
fixed design so that popup looks good..
fix all this issue.. sure after that i will merge your code...

Sure I'll work on it

@danishzayan I've implemented all the requested changes and I couldn't implement a black overlay but still implemented a white toned overlay. Hope it's fine with you. It would be great if you merged the PR with the hacktoberfest-accepted tag once you're satisfied with the performed changes. Thank you ❤

@danishzayan I've implemented all the requested changes and I couldn't implement a black overlay but still implemented a white toned overlay. Hope it's fine with you. It would be great if you merged the PR with the hacktoberfest-accepted tag once you're satisfied with the performed changes. Thank you ❤


✏ Change the name of the field esp_idf_svc::http::server::Configurat…

Change the name of the field esp_idf_svc::http::server::Configuration::max_resp_handlers to max_resp_headers, because it's called in this way in ESP-IDF's documentation)

It is a breaking change (API change) but given that we'll likely publish a breaking version of esp-idf-svc soon anyway, due to the upgrade to heapless 0.8, I think I'm fine with that.
Thanks!


New release, update eredis, add TCP option suggestions

Update the eredis dependency.
Document suggested TCP options as discussed in #59.
New release 0.8.0.

@e-kmisk

Hi,
Are you guys planning to add a tag or release entry for 0.8.0? Atm. I cannot see either.

Yes, done. I had some trouble because I forgot about the Mix files and rebar.lock which needs to be deleted, otherwise the deps aren't actually updated. :-( I published it on hex and then I had to do it again (on hex you have an hour to fix trouble with a new release) and after that I tagged it and pushed the tag.


dart2js removes method call with potential side effects in compiled javascript

I have come across an issue where the dart2js compiler completely removes an important method call from the output js.
This happens with dart2js on all optimization levels (even 0) but not with dartdevc.
dart --version
Dart SDK version: 2.16.0 (stable) (Mon Jan 31 15:28:59 2022 +0100) on "macos_x64"

This can be reproduced with the following code:
void main() {
  load();
  load(cond: true);
}

/// we have two optional booleans
/// cond can be both true and false
/// use will always be false
dynamic load({bool cond = false, bool use = false}) {
  print("start of load()");
  int? data;
  if (cond) {
    data = callWithSideEffects();
  }
  return [if (use && cond) data];
}

int callWithSideEffects() {
  print("SIDE EFFECT");
  return 2;
}

Running dart2js main.dart -O0 results in the following output (only relevant part shown)
main() {
  A.load(false);
  A.load(true);
},
load(cond) {
  var t1;
  A.print("start of load()");
  cond;
  t1 = A._setArrayType([], type$.JSArray_nullable_int);
  return t1;
}

We see that the full if statement with the method call is stripped and only a simple cond remains.
For the last line in load() the array will always be empty because use is always false. This is correctly compiled since use does not exist and an empty array is returned.
I think what is going on is that the compiler thinks that therefore data is never used and that it can be removed, together with all assignments to it - including the method call with side effects.
From my testing it appears that the crutial part is the use && cond with the list for some reason. It will correctly compile if:

the last line is return [if (use) data];
the last line is if (use && cond) return data;
the last line is return use ? [data] : [];

It will however wrongly compile if:

the last line is return use && cond ? [data] : [];

Thanks for the detailed report, it was helpful in understanding the root cause.
This fix will probably land in a few days: https://dart-review.googlesource.com/c/sdk/+/232781


make tre-deploy: 'ssl_cert_name' is a required property  Failed validating 'required' in schema:

When runnign make tre-deploy and the nexus shared service is installing it fails with:
'ssl_cert_name' is a required property

Failed validating 'required' in schema:
...

@jjgriff93 any ideas?

@jjgriff93 can you add any insight into this?

Should we move to deploying the shared services via the UI? Does it need to be in the make file?

I hit this as well:
Deploying shared service tre-shared-service-sonatype-nexus of version 2.1.3
Failed to deploy shared service: 400
HTTP/1.1 400 Bad Request
Date: Tue, 02 Aug 2022 18:09:27 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 2703
Connection: keep-alive
Server: uvicorn

'ssl_cert_name' is a required property

Failed validating 'required' in schema:
    {'actions': [],
     'additionalProperties': False,
     'current': True,
     'customActions': [],
     'description': 'Provides Nexus shared service',
     'id': '377dfa6a-1565-4251-8f94-96d49a2cfb8b',
     'name': 'tre-shared-service-sonatype-nexus',
     'properties': {'description': {'description': 'Description of the '
                                                   'shared service',
                                    'title': 'Description of the shared '
                                             'service',
                                    'type': 'string',
                                    'updateable': True},
                    'display_name': {'description': 'The name of the '
                                                    'shared service to be '
                                                    'displayed to users',
                                     'title': 'Name for the shared service',
                                     'type': 'string',
                                     'updateable': True},
                    'overview': {'description': 'Long form description of '
                                                'the shared service, in '
                                                'markdown syntax',
                                 'title': 'Shared Service Overview',
                                 'type': 'string',
                                 'updateable': True},
                    'ssl_cert_name': {'description': 'The name of the '
                                                     'certificate to use '
                                                     '(located in the core '
                                                     'KeyVault) for '
                                                     'configuring Nexus '
                                                     'SSL',
                                      'title': 'SSL certificate name',
                                      'type': 'string'}},
     'required': ['description', 'display_name', 'ssl_cert_name'],
     'resourceType': <ResourceType.SharedService: 'shared-service'>,
     'system_properties': {'azure_location': {'type': 'string'},
                           'shared_service_id': {'type': 'string'},
                           'tre_id': {'type': 'string'}},
     'title': 'Nexus Shared Service',
     'type': 'object',
     'uiSchema': {},
     'version': '2.1.3'}

On instance:
    {'description': 'Automatically deployed '
                    'tre-shared-service-sonatype-nexus',
     'display_name': 'Shared service tre-shared-service-sonatype-nexus'}

I hit this as well,
What's the current workaround, to not use nexus automatic deployment ?

Think in the env vars can set DEPLOY_NEXUS to false as per https://microsoft.github.io/AzureTRE/tre-admins/setup-instructions/configuring-shared-services/.  Then can create the cert and nexus install via the API/UI.
@tamirkamara was the plan to take out the auto nexus deployment?
Sorry you have hit a couple of bumps. Hopefully as we start doing releases that will be smoother than taking latest from main.


UpdateResource - Data mapping - Version not Updated

Version does not change

[ ] If NewResource.version (integer) is same as ExistingResource.version (integer):

[ ] Proceed with overlaying NewResource on ExistingResource
[ ] Add System Metadata:

[ ] On merged NewResource, assert Administrative.remediatedBy (Agent object) using Agent value.
[ ] Persist Resource Metadata:

[x] Write merged NewResource (map[string]interface{}) to DynamoDB.
[ ] Notify Processing Stream:

[ ] Send Message to Kinesis Router Stream with TACO Internal Identifier (NewResource.Id [string, UUID]), Resourcetype, & Action == Update (this may and probably will continue to evolve).

[ ] Return to client the SDR Identifier (NewResource.identification.identifier [string, DRUID], should not have changed).

Reference
https://github.com/sul-dlss-labs/taco/wiki/TACO-Internal-Processing-Steps#update-resource

Messaging is addressed in #261


fix: use correct = operator in bash

Our readiness checks incorrectly compare strings, resulting in an error:
root@gnosis-sts-0:/nethermind# cat test.sh
#!/bin/sh

ETH_SYNCING=$(curl -s http://localhost:5052/eth/v1/node/syncing)
SYNCING_RESULT=$(echo "$ETH_SYNCING" | jq -r .data.is_syncing)

if [ "$SYNCING_RESULT" == "false" ]; then
  echo "node is synced"
else
  echo "node is still syncing"
  exit 1
fi

root@gnosis-sts-0:/nethermind# ./test.sh
./test.sh: 6: [: true: unexpected operator
node is still syncing
root@gnosis-sts-0:/nethermind#

After the patch:
root@gnosis-sts-0:/nethermind# cat test.sh
#!/bin/sh

ETH_SYNCING=$(curl -s http://localhost:5052/eth/v1/node/syncing)
SYNCING_RESULT=$(echo "$ETH_SYNCING" | jq -r .data.is_syncing)

if [ "$SYNCING_RESULT" = "false" ]; then
  echo "node is synced"
else
  echo "node is still syncing"
  exit 1
fi

root@gnosis-sts-0:/nethermind# ./test.sh
node is still syncing

This doesn't alter the execution but gets rid of the error

Thanks for the context, will patch as suggested and then get back to it

@kaladinlight updated


Fix dependency graph link

The path to the dependency graph was relative, and wasn't updated in the move of the Getting Started documents. Also update the date it was last generated.

bors r+


add multi stage docker file for discover.

now that docker hub supports it, lets do it the way we did for k8guard-action

Wooooooo!

Solved by https://github.com/k8guard/k8guard-discover/pull/29


Add locking in cache.py

fixes: #822 #815

I've run the benchmark but getting inconsistent results on every run, so can't say for sure if this doesn't impact performance.

Codecov Report

Merging #825 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #825      +/-   ##
==========================================
+ Coverage    50.1%   50.13%   +0.02%
==========================================
  Files          37       37
  Lines        3439     3441       +2
  Branches      495      495
==========================================
+ Hits         1723     1725       +2
  Misses       1604     1604
  Partials      112      112

Impacted Files
Coverage Δ

lib/carbon/cache.py
80.92% <100%> (+0.25%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b469e99...a96726a. Read the comment docs.

It looks like what is needed to solve the problem, I'll try running the benchmarks and see what I can find

I'll try to create multicarbon configuration, maybe it will be better for benchmarking - comparing single carbon instance is hard.

@deniszh I was referring to https://github.com/graphite-project/carbon/blob/master/lib/carbon/tests/benchmark_cache.py I've run it a bunch of times with and without locking on Python 2.7 and 3.7 but did not get consistent results.

I manually picked up this commit, but still get the same result.
07/12/2018 17:24:59 :: [console] Unhandled Error
Traceback (most recent call last):
File "/opt/graphite/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/opt/graphite/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/opt/graphite/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/opt/graphite/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
---  ---
File "/opt/graphite/lib/carbon/writer.py", line 189, in writeForever
writeCachedDataPoints()
File "/opt/graphite/lib/carbon/writer.py", line 98, in writeCachedDataPoints
(metric, datapoints) = cache.drain_metric()
File "/opt/graphite/lib/carbon/cache.py", line 190, in drain_metric
metric = self.strategy.choose_item()
File "/opt/graphite/lib/carbon/cache.py", line 116, in choose_item
return next(self.queue)
builtins.StopIteration:

I manually picked up this commit, but stilil get the same error, did I miss something?
07/12/2018 15:17:56 :: [console] Unhandled Error
Traceback (most recent call last):
  File "/opt/graphite/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/opt/graphite/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/opt/graphite/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/opt/graphite/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
--- <exception caught here> ---
  File "/opt/graphite/lib/carbon/writer.py", line 189, in writeForever
    writeCachedDataPoints()
  File "/opt/graphite/lib/carbon/writer.py", line 98, in writeCachedDataPoints
    (metric, datapoints) = cache.drain_metric()
  File "/opt/graphite/lib/carbon/cache.py", line 190, in drain_metric
    metric = self.strategy.choose_item()
  File "/opt/graphite/lib/carbon/cache.py", line 116, in choose_item
    return next(self.queue)
  File "/opt/graphite/lib/carbon/cache.py", line 104, in _generate_queue
    metric_counts = sorted(self.cache.counts, key=lambda x: x[1])
  File "/opt/graphite/lib/carbon/cache.py", line 163, in counts
    in self.items()]
  File "/opt/graphite/lib/carbon/cache.py", line 162, in <listcomp>
    return [(metric, len(datapoints)) for (metric, datapoints)
builtins.RuntimeError: dictionary changed size during iteration

@tongyingrui Could you please test https://github.com/graphite-project/carbon/pull/829

@tongyingrui Could you please test #829

Yes, sure.
And I think I can't reproduce it anymore. It has been running for 12 hours.

@tongyingrui Thank you for testing. Did you notice any side-effect such as higher cpu usage, stalling?

Sorry, I didn't pay much attention on the CPU usage before.
But at least I didn't see an extremely high cpu usage with the new commit.


Missing meetings for Northeast Alabama Area

Describe the bug
BMLT listst meetings in Jackson AL 35771 but when we call our helpline is gives a different zipcode for Jackson AL than what is listed. In addition if you input 35771 it will not list Section Serenity group.
What version are you running?
2.5.x
Did this happen on prior versions to the one you are running?
Uknown
To Reproduce
Steps to reproduce the behavior:

Call XXX-XXX-XXXX
option 2, option one "Jackson AL" (returns a different zipcode that what is in bmlt)
option 2, option two "35771" meeting information for Wednesday Section Serenity group doesn't come up.

Expected behavior
Meeting should show up.
Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Section is the town. When I say "Section Alabama" it returns the correct Zipcode but again, no listing for the Section Serenity group.

What do you mean it returns the incorrect zip code?

I was saying "Jackson" thinking it was the city. It was the county. So, when I said Jackson, I was getting a zipcode for a Jackson Alabama?

Is it returning 36545?

@jbraswell for your awareness.

Google Maps API returns 35771 as "Section, AL"

Yep, and it looks correct in BMLT right? (screen shot)

The coordinates in that record seem off, a quick search in Google Maps returns a result west of Montgomery, AL.  https://www.google.com/maps/place/32°19'05.6"N+86°54'08.3"W/@32.2190354,-87.189033,9.26z/data=!4m5!3m4!1s0x0:0x0!8m2!3d32.3182314!4d-86.902298

Delete the coordinates?

Follow up the BMLT administrator to see what's going on.


Repo is getting huge (>1gb), maybe encourage use of arweave?

I'm submitting a ...
[ ] bug report
[ x] feature request
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Summary
Hi guys,
love this repo a lot!
however ti is getting huge, trying to clone it rn and takes forever at 1mb/s.
I have been using arweave (and I also built a web uploader for arweave which I am happy to link) in order to avoid adding to it.
But maybe you should consider finding a better storage solution?

Other information (e.g. detailed explanation, stack traces, related issues, suggestions how to fix, links for us to have context, eg. StackOverflow, personal fork, etc.)

possibly related: #11897


Missing/incorrect fee data

Symptoms

Fees and fee rates are missing or negative in the mouse-over info for transactions
Transactions colored dark red in color-by-fee-rate mode
API server logs show many Failed to inflate input and :timeout messages
Most likely to occur immediately after a system update or device reboot on Umbrel or similar home node deployments

Diagnosis
When the API server receives a very high rate of new transaction events, the task thread pool used to fetch and process transaction prevout data becomes saturated, leading to cascading timeouts.
In normal operation, the transaction rate never comes close to a fraction of this saturation threshold, but Bitcoin Core can send an overwhelming number of transaction events while 'catching up' after a restart, especially when there is mempool congestion.
On home node devices like Umbrel, this is exacerbated by low powered hardware and high CPU demand from other services following a device reboot or system update.
Short Term Mitigation
Can be resolved by restarting just the Bitfeed server after Bitcoin Core has fully resynchronized.
On Umbrel/Citadel nodes:

Manually:

Uninstall Bitfeed

Go to the 'Apps' page on your node dashboard
Click 'Edit'
Uninstall Bitfeed

Reinstall from the App Store page

Via SSH:

run docker restart bitfeed_api_1

The issue is likely to re-occur after rebooting the node or restarting Bitcoin Core.
Long Term Fix
Need to enforce sane limits on multi-threading & simultaneous RPC connections when processing transactions.

Pooled task queue?
Fail transaction tasks faster?

:kill_task instead of :exit on task timeout?

Fall back to skipping prevout inflation during new transaction floods?

Regarding your Citadel instructions: Unlike Umbrel, we use docker-compose v2 (Umbrel uses v1). v2 generates container names with - instead of _, so the correct command for Citadel would be docker restart bitfeed-api-1

thanks, I'll fix that now


Adds max/avg distribution plots

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

Complexity increasing per file
==============================
- varats/data/reports/blame_report.py  2
- varats/utils/git_util.py  3

Clones added
============
- varats/plots/blame_interaction_degree.py  2

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- varats/utils/git_util.py  3
- varats/plots/blame_interaction_degree.py  1
- varats/data/reports/blame_report.py  2

Clones added
============
- varats/plots/blame_interaction_degree.py  2

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- varats/data/reports/blame_report.py  2
- varats/utils/git_util.py  3
- varats/plots/blame_interaction_degree.py  1

Clones added
============
- varats/plots/blame_interaction_degree.py  2

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- varats/data/reports/blame_report.py  2
- varats/utils/git_util.py  3
- varats/plots/blame_interaction_degree.py  1

Clones added
============
- varats/plots/blame_interaction_degree.py  2

See the complete overview on Codacy


115: upload files in bill

Draft for #115
What I'm still working on are:

DHT logic (providing and requesting files between peers)
A few more tests for issue_bill

It would be great to get some feedback in the meantime regarding the structure, error handling, testing and the current implementation of adding file upload to /issue bill, since this will act as a bit of a template to implement further features (and implementing file upload for Company and Identity), so it would be good to get this right early.

LGTM

Wow this one is quite massive. I can not really follow how the file upload stuff works just from the diffs but impl. looks fine. The rest of the refactoring also looks very good (as already mentioned). A bit small chunks might be easier to chew 😁

Yes, you're absolutely right, this escalated a bit :see_no_evil: - sorry! And in retrospect it seems a lot clearer, where there might have been good lines to split this by and I'll take care to keep things smaller, also for my own sanity's sake. :sweat_smile:
This will become easier with more structure as well, as it's already easier to spot which parts of the system will have to be touched by a certain change with the changes we already did. :+1:


Dockerise portal

This PR creates a deployment of the Nomis data portal using docker.
The versions are fixed using R's remotes library and the Docker image from rocker.

Hi @sbancal, I have containerised the application and would appreciate your input to review this PR.
I have created an environment lockfile with the R library renv to fix library packages at the versions in the current running portal (nomis-data.epfl.ch), a Dockerfile to replicate the system setup as defined in https://github.com/ninojeannet/SBER-NOMIS-PORTAL/wiki/Deployment, a github workflow to auto build an image on PR/new version, and a docker-compose file for a local development environment.
If you wish to get this running locally, performing a docker compose up --build should be all that is needed on the application side, and pointing localhost to nomis.local in /etc/hosts.
You will notice there are some local mappings for some folders to manage assets:
      - ./data:/srv/shiny-server/data
      - ./protocols:/srv/shiny-server/www/protocols
      - ./db_backups:/srv/shiny-server/db_backups
      - ./log:/srv/shiny-server/log

I can supply some data to fill in the data and protocols folders if you run into any trouble, and also a recent database dump that will allow you to login.

LGTM, with tiny comment.
Do you want me to test it also ?

No need, the Dockerfile is being built in github actions in the case you would like to validate the build steps.


[Turbo] Web Debug Toolbar script is blocked by CSP when navigating

On initial page load the toolbar works fine, but is missing after visiting a link via Turbo Drive.
This is the console output from Google Chrome:
turbo.es2017-esm.js:4651 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src https://<IP_ADDRESS>:5173 'nonce-ZkGUwME8QO5iHCXt1kgvHw==' 'unsafe-inline' 'nonce-e276b18f073898209eb899b46ffa3f01'". Note that 'unsafe-inline' is ignored if either a hash or nonce value is present in the source list.

renderElement @ turbo.es2017-esm.js:4651
assignNewBody @ turbo.es2017-esm.js:4826
(anonymní) @ turbo.es2017-esm.js:4732
preservingPermanentElements @ turbo.es2017-esm.js:1717
preservingPermanentElements @ turbo.es2017-esm.js:1814
replaceBody @ turbo.es2017-esm.js:4730
render @ turbo.es2017-esm.js:4682
renderSnapshot @ turbo.es2017-esm.js:1523
render @ turbo.es2017-esm.js:1483

The CSP is handled by the NelmioSecurityBundle. Here is an excerpt from its config file (nelmio_security.yaml):
...
csp:
        hosts: []
        content_types: []
        enforce:
            level1_fallback: false
            browser_adaptive:
                enabled: false
            # report-uri: '%router.request_context.base_url%/nelmio/csp/report'
            default-src:
                - 'none'
            style-src:
                - 'self'
                - https://www.gstatic.com https://fonts.googleapis.com                                # Google Fonts
            font-src:
                - 'self'
                - https://fonts.googleapis.com https://fonts.gstatic.com                              # Google Fonts
            img-src:
                - 'self'
                - 'data:'
                - https://tile.openstreetmap.org/
            form-action:
                - 'self'
            frame-src:
                - 'self'
                - 'blob:'
                - https://www.google.com
            frame-ancestors:
                - 'self'
            object-src:
                - 'self'
            media-src:
                - 'self'
            connect-src:
                - 'self'
                - 'wss:'
                - https://tile.openstreetmap.org/
            block-all-mixed-content: true # defaults to false, blocks HTTP content over HTTPS transport
            # upgrade-insecure-requests: true # defaults to false, upgrades HTTP requests to HTTPS transport
...

Could you whitelist the WDT in dev ? As it wont be a problem/question in prod anyway ?
You may need to look at the web profiler bundle, and maybe update something here?
https://symfony.com/doc/current/profiler.html#updating-the-web-debug-toolbar-after-ajax-requests
Or do you have something precise in mind in the TurboBundle, as i'm not seeing right now what could be done here ?

I set the header according to the documentation. However, I still get the CSP error, and now I'm also seeing a new error:
Uncaught TypeError: ajaxToolbarPanel is null

How can I whitelist WDT in NelmioSecurityBundle? Using a hash will only work temporarily until a new version is released.

Ok, I created a new issue here: https://github.com/symfony/symfony/issues/58223


Segmentation fault error

What happened?
Kind regards.
While processing the dataset ds000158 with fmriprep-docker I got a segmentation fault error.  The happened around 48 hours after the process started.
What command did you use?
fmriprep-docker --fs-license-file /home/carlos/data_models/neuro/freesurfer_home/license.txt ../ds000158-download/ ../ds000158-download/fmriprep participant -w work_ds000158/ --nprocs 8

RUNNING: docker run --rm -e DOCKER_VERSION_8395080871=24.0.2 -it -v /home/carlos/data_models/neuro/freesurfer_home/license.txt:/opt/freesurfer/license.txt:ro -v /home/carlos/data_models/neuro/ds000158-download:/data:ro -v /home/carlos/data_models/neuro/ds000158-download/fmriprep:/out -v /home/carlos/data_models/neuro/process/work_ds000158:/scratch nipreps/fmriprep:23.1.4 /data /out participant --nprocs 8 -w /scratch

What version of fMRIPrep are you running?
23.1.4
How are you running fMRIPrep?
Docker
Is your data BIDS valid?
Yes
Are you reusing any previously computed results?
No
Please copy and paste any relevant log output.
230808-16:30:09,809 nipype.workflow INFO:
	 [Node] Finished "_threshold264", elapsed time 0.074402s.
230808-16:30:13,727 nipype.workflow INFO:
	 [Node] Setting-up "_threshold265" in "/scratch/fmriprep_23_1_wf/single_subject_053_wf/func_preproc_task_passivelistening_wf/bold_bold_trans_wf/threshold/mapflow/_threshold265".
230808-16:30:13,728 nipype.workflow INFO:
	 [Node] Executing "_threshold265" <fmriprep.interfaces.maths.Clip>
230808-16:30:13,797 nipype.workflow INFO:
	 [Node] Finished "_threshold265", elapsed time 0.069109s.
230808-16:30:17,720 nipype.workflow INFO:
	 [Node] Setting-up "_threshold266" in "/scratch/fmriprep_23_1_wf/single_subject_053_wf/func_preproc_task_passivelistening_wf/bold_bold_trans_wf/threshold/mapflow/_threshold266".
230808-16:30:17,721 nipype.workflow INFO:
	 [Node] Executing "_threshold266" <fmriprep.interfaces.maths.Clip>
230808-16:30:17,780 nipype.workflow INFO:
	 [Node] Finished "_threshold266", elapsed time 0.05883s.
 It could be a memory error related to having so many nodes, though I've never seen this before. Do you see this behavior if you run on one subject, or a few subjects, at a time?


Issue with changing notification position - bug?

Having trouble uploading gifs at the moment to demonstrate, but will update later when they are working

Taking what is in fade_in_slide_out.lua and changing the col value from vim.opt.columns:get() to just 1 moves the notifications from the top right to the top left as I would expect.
However, when changing the stages_util.DIRECTION.TOP_DOWN to stages_util.DIRECTION.BOTTOM_UP in an attempt to then move them to the bottom left results in issues where a secondary notification overlaps the first one vertically by ~2 rows rather than having them stay disjointed. Because this is an issue with the row (and I the initial position of the 2+ notification is correct), I believe this is an issue with slot_after_previous (which is used in [lua] index 2+ of the stages configuration).
Here is what I have after swapping things around:
local notify = { col_offset = 1 }

local function get_stages()
  local stages_util = require("notify.stages.util")

  return {
    function(state)
      local next_row = stages_util.available_slot(
        state.open_windows,
        state.message.height + 2,
        stages_util.DIRECTION.TOP_DOWN
      )

      if not next_row then
        return nil
      end

      return {
        relative = "editor",
        anchor = "NE",
        width = state.message.width,
        height = state.message.height,
        col = notify.col_offset,
        row = next_row,
        border = "rounded",
        style = "minimal",
        opacity = 0,
      }
    end,
    function(state, win)
      return {
        opacity = { 100 },
        col = { notify.col_offset },
        row = {
          stages_util.slot_after_previous(win, state.open_windows, stages_util.DIRECTION.TOP_DOWN),
          frequency = 3,
          complete = function()
            return true
          end,
        },
      }
    end,
    function(state, win)
      return {
        col = { notify.col_offset },
        time = true,
        row = {
          stages_util.slot_after_previous(win, state.open_windows, stages_util.DIRECTION.TOP_DOWN),
          frequency = 3,
          complete = function()
            return true
          end,
        },
      }
    end,
    function(state, win)
      return {
        width = {
          1,
          frequency = 2.5,
          damping = 0.9,
          complete = function(cur_width)
            return cur_width < 3
          end,
        },
        opacity = {
          0,
          frequency = 2,
          complete = function(cur_opacity)
            return cur_opacity <= 4
          end,
        },
        col = { notify.col_offset },
        row = {
          stages_util.slot_after_previous(win, state.open_windows, stages_util.DIRECTION.TOP_DOWN),
          frequency = 3,
          complete = function()
            return true
          end,
        },
      }
    end,
  }
end

It seems to be working fine for me, are you sure you didn't miss changing one of the direction arguments to BOTTOM_UP? Can you provide the config that isn't working for bottom up?

This is what I am currently using (notify.col_offset = 1):
  return {
    function(state)
      local next_row = stages_util.available_slot(
        state.open_windows,
        state.message.height + 2,
        stages_util.DIRECTION.TOP_DOWN
      )

      if not next_row then
        return nil
      end

      return {
        relative = "editor",
        anchor = "NE",
        width = state.message.width,
        height = state.message.height,
        col = notify.col_offset,
        row = next_row,
        border = "rounded",
        style = "minimal",
        opacity = 0,
      }
    end,
    function(state, win)
      return {
        opacity = { 100 },
        col = { notify.col_offset },
        row = {
          stages_util.slot_after_previous(win, state.open_windows, stages_util.DIRECTION.TOP_DOWN),
          frequency = 3,
          complete = function()
            return true
          end,
        },
      }
    end,
    function(state, win)
      return {
        col = { notify.col_offset },
        time = true,
        row = {
          stages_util.slot_after_previous(win, state.open_windows, stages_util.DIRECTION.TOP_DOWN),
          frequency = 3,
          complete = function()
            return true
          end,
        },
      }
    end,
    function(state, win)
      return {
        width = {
          1,
          frequency = 2.5,
          damping = 0.9,
          complete = function(cur_width)
            return cur_width < 3
          end,
        },
        opacity = {
          0,
          frequency = 2,
          complete = function(cur_opacity)
            return cur_opacity <= 4
          end,
        },
        col = { notify.col_offset },
        row = {
          stages_util.slot_after_previous(win, state.open_windows, stages_util.DIRECTION.TOP_DOWN),
          frequency = 3,
          complete = function()
            return true
          end,
        },
      }
    end,
  }

is what I'm using. I still cannot upload pictures or GIFs for some reason, otherwise I would show you what it looks like @rcarriga.

Still can't reproduce unfortunately. I'm thinking there's some setting you have enabled/disabled that the previous slot function doesn't account for.
Can you try with this init.lua
nvim --clean -u init.lua
-- ignore default config and plugins
vim.opt.runtimepath:remove(vim.fn.stdpath("config"))
vim.opt.packpath:remove(vim.fn.stdpath("data") .. "/site")
vim.opt.termguicolors = true

-- append test directory
local test_dir = "/tmp/nvim-config"
vim.opt.runtimepath:append(vim.fn.expand(test_dir))
vim.opt.packpath:append(vim.fn.expand(test_dir))

-- install packer
local install_path = test_dir .. "/pack/packer/start/packer.nvim"
local install_plugins = false

if vim.fn.empty(vim.fn.glob(install_path)) > 0 then
  vim.cmd("!git clone https://github.com/wbthomason/packer.nvim " .. install_path)
  vim.cmd("packadd packer.nvim")
  install_plugins = true
end

local packer = require("packer")

packer.init({
  package_root = test_dir .. "/pack",
  compile_path = test_dir .. "/plugin/packer_compiled.lua",
})

packer.startup(function(use)
  use("wbthomason/packer.nvim")

  use({
    "rcarriga/nvim-notify",
    config = function()
      local stages_util = require("notify.stages.util")
      require("notify").setup({
        background_colour = "#000000",
        stages = {
          function(state)
            local next_row = stages_util.available_slot(
              state.open_windows,
              state.message.height + 2,
              stages_util.DIRECTION.BOTTOM_UP
            )

            if not next_row then
              return nil
            end

            return {
              relative = "editor",
              anchor = "NE",
              width = state.message.width,
              height = state.message.height,
              col = 1,
              row = next_row,
              border = "rounded",
              style = "minimal",
              opacity = 0,
            }
          end,
          function(state, win)
            return {
              opacity = { 100 },
              col = { 1 },
              row = {
                stages_util.slot_after_previous(
                  win,
                  state.open_windows,
                  stages_util.DIRECTION.BOTTOM_UP
                ),
                frequency = 3,
                complete = function()
                  return true
                end,
              },
            }
          end,
          function(state, win)
            return {
              col = { 1 },
              time = true,
              row = {
                stages_util.slot_after_previous(
                  win,
                  state.open_windows,
                  stages_util.DIRECTION.BOTTOM_UP
                ),
                frequency = 3,
                complete = function()
                  return true
                end,
              },
            }
          end,
          function(state, win)
            return {
              width = {
                1,
                frequency = 2.5,
                damping = 0.9,
                complete = function(cur_width)
                  return cur_width < 3
                end,
              },
              opacity = {
                0,
                frequency = 2,
                complete = function(cur_opacity)
                  return cur_opacity <= 4
                end,
              },
              col = { 1 },
              row = {
                stages_util.slot_after_previous(
                  win,
                  state.open_windows,
                  stages_util.DIRECTION.BOTTOM_UP
                ),
                frequency = 3,
                complete = function()
                  return true
                end,
              },
            }
          end,
        },
      })
      vim.notify = require("notify")
    end,
  })

  if install_plugins then
    packer.sync()
  end
end)

Good point - I should have started with this as well. The above minimal configuration does not see the same issues, so I will start to bisect mine to see where things are going wrong and report back.

@rcarriga I was able to reproduce it with the init.lua file you provided above, and executing :luafile % on the following in a buffer:
vim.schedule(function()
  vim.notify("foo")

  vim.defer_fn(function()
    vim.notify({"bar", "baz"})

  end, 1500)
end)

https://user-images.githubusercontent.com/31262046/186188042-2def25b2-fedf-4742-b4a5-2cacd9d9c26a.mp4

Not sure if it makes a difference @rcarriga, but when using (the first vim.notify is called immediately, rather than scheduled for later):
vim.notify("foo")

vim.defer_fn(function()
  vim.notify({"bar", "baz"})
end, 1500)

note that the top-most notification sliding down over the bottom one no longer happens:
https://user-images.githubusercontent.com/31262046/186451764-d55373b8-45a8-4c99-960c-7599e856b213.mp4
however the top-most notification doesn't move itself all the way down to the bottom either.

Spamming a bit with debugging notes - apologies in advance.

The print statements below print out next_row for the first stages function, which shows us where the window starts. The subsequent prints are for the second stages function with all other stages functions removed.
For the first window (1007):

and for the second window (1008) with some co-mingling of the first window as well:

it starts in the right place with the return value from available_slot, but then the return value from slot_after_previous ends up pushing it down by one. Later when 1007 is gone and 1008 slides down, we can also see that it is off by one (too high):

Will keep looking as time permits and post here if I find any updates, but hopefully this helps narrow in on what the problem actually is.

I was able to reproduce the issue with the second notification not going to the bottom and have fixed it, pull the latest master to confirm. Unfortunately, I'm not seeing that issue with the second appearing over the first though

Interesting that it does not reproduce for you. When you are testing, what are the values of vim.opt.columns:get() and vim.opt.lines:get()? I wonder if it has something to do with the editor size where I am running it versus where you are @rcarriga.
The issue with notifications not going back down all the way is fixed per your last commit. Thanks!

OK figured out the issue. Please try out master now :smile:

Things are flip-flopped now. The issue with notifications overlapping is fixed, however now the second notification doesn't go all the way to the bottom of the screen.
Appreciate the effort either way though @rcarriga. Let me know if you want me to try to setup a minimal config to reproduce the "not going all the way to the bottom" issue.

I found an issue with cmdheight=0, did you have that set? Could you try lastest master?

I have cmdheight=1 - will try the latest master.

@rcarriga as far as I can tell it works:
https://user-images.githubusercontent.com/31262046/188733441-f501926f-36b7-4ce9-bdf6-847b8d84e43e.mp4
There's a bit of jitter at the end there, but it goes down to the correct spot for sure. Thanks for all the work on this!

Brilliant :smile: I've updated the rounding to help a little with the jitter slightly. You can also tweak the damping/frequency options to change the spring physics but unfortunately rendering in a terminal has its limitations


IRQL NOT LESS OR EQUAL

IRQL NOT LESS OR EQUAL Bluescreen
tested on Windows 11 22H2
WIndows 10 22H2
using kdmapper.

Fixed it.
--> https://github.com/thesecretclub/callout-poc/issues/1


Updating CFBundleVersion to match multiple targets

Troubleshooting

[X] I've searched discuss.bitrise.io for possible solutions.
Which version of the step is effected? 1.2.3
Is the issue reproducible with the latest version? YES
Does the issue happen sporadically, or every time? EVERY TIME
Is the issue reproducible locally by following our local debug guide? NOT APPLICABLE

Issue description
I have two targets in my application. For that I've added two separate 'Set Xcode Project Build Number' steps, with each Info.plist path specified. Goal is to set CFBundleVersion to be $GIT_CLONE_COMMIT_COUNT.
When I look at the logs, I can see the correct commit count number is displayed in both of the steps and the build overall is successful. However, during the next step, where it uploads to app store connect, I receive a ITMS-90473: CFBundleVersion Mismatch warning email saying that the bundle version of my targets is not matching (one is correct, another one is just left as 1). I am confident, that the Info.plist path for each step is correct.
I've spend a lot of hours already trying to figure it out, but no luck. Currently resolved it by just specifying the build numbers manually in the code, but would be great to get it working again in Bitrise.
Any ideas what might be causing this?
Bitrise info

Build URL: https://app.bitrise.io/build/6e1c35eb-066a-41ae-bc8d-eaa7e90cd9e9

@jgonfer Did you get it resolved by any chance?

I'm experiencing the same issue.

I ended up creating a custom script which I added as a step in my build pipeline:
cd $BITRISE_SOURCE_DIR
#Insert your project path below
PROJECT=$XCODE_PROJECT_PATH/project.pbxproj
sed -i '' -e 's/CURRENT_PROJECT_VERSION \= [^\;]*\;/CURRENT_PROJECT_VERSION = '$GIT_CLONE_COMMIT_COUNT';/' $PROJECT
build_number=`sed -n '/CURRENT_PROJECT_VERSION/{s/CURRENT_PROJECT_VERSION = //;s/;//;s/^[[:space:]]*//;p;q;}' $PROJECT`


Finnish translation

We're using this in a research study, which aims to teach civil servants about complexity :) Nov 9 is our first session.

Ay, lucky you, I actually managed to pull a Pull Request less than a whole month later. Best wishes on reaching & teaching those civil servants! Your translation is up at https://ncase.me/attractors/fi.html 👍


Explain the SIZE column in "docker ps -s" and what "virtual" keyword means

File: engine/reference/commandline/ps.md
Under the SIZE column of the outpout of "docker ps -s". Is it possible to explain what the "virtual" keyword means ?
Example : for a container I am runing I can see 2.045 MB (virtual 540.5 MB)
Does this mean that the container takes 2.045 MB on disk, or 540.5 MB ?

We may be able to explain it. This is not a Docker-specific concept. For now, there is a great explanation here: http://serverfault.com/questions/48582/how-is-memory-usage-reported-in-linux

I'm confused. I understand the concept of virtual memory but @mstanleyjones are you sure the -s switch is about virtual memory ? Why is it reporting reporting a virtual size for stopped containers then ?
I understand the -s switch is about file sizes, right ?

@thaJeztah can you help here? I may be missing something.

As I see it, the -s options outputs the size of additonal layers on top of the given image the container runs. The virtual size of all image layers plus the first value. Volumes are not counted.  I see this behaviour with aufs and docker on mac. Right ?

The "size" and "virtual size" describe the amount of disk space used by a container. Let me try and explain:
When starting a container, the image that the container is started from is mounted read-only. On top of that, a writable layer is mounted, in which any changes made to the container are written.
The read-only layers of an image can be shared between any container that is started from the same image, whereas the "writable" layer is unique per container (because: you don't want changes made in container "a" to appear in container "b" :smile:)
Back to the docker ps -s output;

The "size" information shows the amount of data (on disk) that is used for the writable layer of each container
The "virtual size" is the amount of disk-space used for the read-only image data used by the container.

The reason it's named "virtual size", is that (as described earlier), the disk space for the read-only layer(s) can be shared between containers, so only take up disk space once (perhaps a different name ("shared" size?) would have been better in hindsight, but naming is hard :smile:).
In the example below, I started 10 nginx containers;
CONTAINER ID        IMAGE                      COMMAND                  CREATED              STATUS              PORTS               NAMES                                         SIZE
6ca0cef8db8d        nginx                      "nginx -g 'daemon ..."   55 seconds ago       Up 54 seconds       80/tcp, 443/tcp     pedantic_mahavira                             2B (virtual 183MB)
0b75532c34d2        nginx                      "nginx -g 'daemon ..."   56 seconds ago       Up 55 seconds       80/tcp, 443/tcp     hardcore_haibt                                2B (virtual 183MB)
a86d8818795d        nginx                      "nginx -g 'daemon ..."   56 seconds ago       Up 55 seconds       80/tcp, 443/tcp     happy_pasteur                                 2B (virtual 183MB)
3e9151f0332d        nginx                      "nginx -g 'daemon ..."   57 seconds ago       Up 56 seconds       80/tcp, 443/tcp     pensive_khorana                               2B (virtual 183MB)
429c418114a1        nginx                      "nginx -g 'daemon ..."   57 seconds ago       Up 56 seconds       80/tcp, 443/tcp     nostalgic_bose                                2B (virtual 183MB)
01f6a0818b32        nginx                      "nginx -g 'daemon ..."   58 seconds ago       Up 57 seconds       80/tcp, 443/tcp     zen_davinci                                   2B (virtual 183MB)
9ed93ee43912        nginx                      "nginx -g 'daemon ..."   58 seconds ago       Up 58 seconds       80/tcp, 443/tcp     unruffled_ptolemy                             2B (virtual 183MB)
c7c17bf50744        nginx                      "nginx -g 'daemon ..."   59 seconds ago       Up 58 seconds       80/tcp, 443/tcp     zen_khorana                                   2B (virtual 183MB)
a7b0502d95ce        nginx                      "nginx -g 'daemon ..."   About a minute ago   Up About a minute   80/tcp, 443/tcp     gallant_leavitt                               2B (virtual 183MB)
8b9e9ee6c4e8        nginx                      "nginx -g 'daemon ..."   About a minute ago   Up About a minute   80/tcp, 443/tcp     clever_pasteur                                2B (virtual 183MB)

All these containers use the same image, so the "Virtual size" (183MB in the example) is used only once, irregardless of how many containers are started from the same image - I can start 1 container or a thousand; no extra disk space is used. The "Size" (2B in the example) is unique per container though, so the total space used on disk is:
183MB + 10 * 2B
Be aware that the size shown does not include all disk space used for a container. Things that are not included currently are;

disk space used for log-files (if you use the json-file logging driver) - which can be quite a bit if your container generates a lot of logs, and log-rotation (max-file / max-size logging options) is not configured
volumes used by the container
disk space used for the container's configuration files (hostconfig.json, config.v2.json, hosts, hostname, resolv.conf) - although these files are small
memory written to disk (if swapping is enabled)
checkpoints (if you're using the experimental checkpoint/restore feature)

Is there anyway to determine the total amount of disk space used by each container including log files, volumes, container config files, memory written to disk and checkpoints?  I mean, a COMPLETE analysis?
Can this be done via using docker restful API's?  See link here:
https://docs.docker.com/engine/api/v1.37/#operation/ContainerStats

@yonglam "same image", but also same version of the image? Were they started at different points in time? For example;
(using docker image tag just for illustration);
myimage:latest is created, and a container is started with it;
docker image tag busybox myimage:latest
docker run -dit --name container1 myimag

docker ps -s

CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES               SIZE
7767a6e5b649        myimage             "sh"                     About a minute ago   Up About a minute                       container1          0B (virtual 1.15MB)

myimage is updated with a new version, and a second container is started
docker image tag ubuntu myimage:latest
docker run -dit --name container2 myimage

docker ps -s

CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES               SIZE
74f0e474467c        myimage             "/bin/bash"              About a minute ago   Up About a minute                       container2          0B (virtual 79.6MB)

Both were stared from "the same image"
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES               SIZE
74f0e474467c        myimage             "/bin/bash"              About a minute ago   Up About a minute                       container2          0B (virtual 79.6MB)
7767a6e5b649        8c811b4aec35        "sh"                     About a minute ago   Up About a minute                       container1          0B (virtual 1.15MB)

@thaJeztah Thank you for your response. I don't think they are different image versions. Because the image tagged latest was created at 2017-12-04. It is far before I pull the image. I created the first container about 3 months ago, and the second one about 13 days ago. What I did was:
docker pull docker.xx.com:8080/my-jenkins-node-full:latest
docker run -d --net=host --name the_first docker.xx.com:8080/my-jenkins-node-full:latest
docker run -d --net=host --name the_second docker.xx.com:8080/my-jenkins-node-full:latest

If they are each Jenkins images, they are probably carrying different state in the container's writable layer, which is always unique per running container.

That wouldn't explain the difference in virtual size (which should be the read-only layers); I do see @yonglam is using a very old version of Docker, which could be related (could be a presentation bug), or they were actually started from different images;
@yonglam what does docker inspect show for each of those containers? Mainly the .Image and .Config.Image properties;
docker inspect --format '{{.Image}} {{.Config.Image}}'  the_first the_second

Should show something like;
sha256:c51a2cac7341fe55d38feb70157eb12d56b2039977d4175229432de09bd18896 docker.xx.com:8080/my-jenkins-node-full:latest
sha256:c51a2cac7341fe55d38feb70157eb12d56b2039977d4175229432de09bd18896 docker.xx.com:8080/my-jenkins-node-full:latest

They have the same image's sha. @thaJeztah
[root@Linux64 ~]# docker inspect --format '{{.Image}} {{.Config.Image}}' the_first the second
sha256:c9a1b52800257d545ad617981fd8ac204619859c824d05b1bb2604442a52cc43 docker.xx.com:8080/my-jenkins-node-full:latest
sha256:c9a1b52800257d545ad617981fd8ac204619859c824d05b1bb2604442a52cc43 docker.xx.com:8080/my-jenkins-node-full:latest

I will update my docker.

Erm, actually, had my wires cross; @mistyhacks is absolutely right; Virtual Size shows both the read-only and writable layer 😊;

If they are each Jenkins images, they are probably carrying different state in the container's writable layer, which is always unique per running container.

So yes, if those containers are writing lots of files to the container's filesystem itself (not to a volume), then the Virtual Size would grow as well (as it's the sum of both the image's size, and all files written in the container's filesystem);
docker run -dit --name foobar alpine
docker run -dit --name foobar2 alpine

docker exec foobar sh -c ' dd if=/dev/zero of=output.dat  bs=24M  count=1'

docker ps -s
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES                                     SIZE
60bab76f2ae8        alpine              "/bin/sh"           33 seconds ago      Up 32 seconds                           foobar2                                   0B (virtual 4.15MB)
9a6e24f0cdfe        alpine              "/bin/sh"           36 seconds ago      Up 35 seconds                           foobar                                    25.2MB (virtual 29.3MB)

although I acknowledge that google does a good job of indexing this github issue, and most developers and infrastructure engineers will be able to find it if they need it, this information should still be  easily locate-able in the docker  documentation and this issue should not be closed until then.

@forestjohnsonpeoplenet I see this ticket was closed by a pull request in the documentation repository: https://github.com/docker/docker.github.io/pull/3615
That PR added this section to the documentation; https://docs.docker.com/storage/storagedriver/#container-size-on-disk
If you think more information would be needed for the documentation; could you open a ticket in that issue tracker (with some description which information you were looking for, but could not find) so that it doesn't get lost?
You can open a new issue from that page in the documentation ("request docs changes" link on the right-hand side of that page), or using this link; https://github.com/docker/docker.github.io/issues/new

I guess I am talking about a web usability issue, not a "information is missing" issue.
I think it should be here, since thats where I looked. https://docs.docker.com/engine/reference/commandline/ps/
I would never have guessed to look inside a storage driver article for information on the docker ps command.
Thanks for the reply and information on how to create an issue !!! very helpful :)
Here is my PR: https://github.com/docker/docker.github.io/pull/8371
Hope this helps.


Enemy Module

Implement an enemy module to break away from ALiVE.

Beginning work on an enemy module to replace ALiVE

Tasks:

[ ] Headless client compatibility
[ ] Debug system

Moving this to backlog for now since using ALiVE for enemy forces seems to be working great and most people have ALiVE DL'd already since it's a popular mod. In the future, I will try and see what I can do for an enemy module to replace ALiVE to see if we can get the entire mission made for vanilla gameplay.


Using a lazy representation of the kernel matrix

Currently this package eagerly constructs and uses a dense kernel matrix. However, iterative GP methods are very useful when the data is large enough that the kernel matrix can't even fit into memory. It would be better to construct and use a lazy representation of the kernel matrix.
I've started work on adding this functionality to the KernelFunctions API: https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/pull/515

Very good comment, and I completely agree. Because of  https://github.com/SebastianCallh/IterGP.jl/issues/5 I have been hesitant to introduce additional moving parts since it would probably make it more difficult to solve.
Looks like this should be fairly straightforward after https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/pull/515 lands? So perhaps it makes sense to wait for the linked PRs to be resolved before tackling this?

Looks like this should be fairly straightforward after JuliaGaussianProcesses/KernelFunctions.jl#515 lands? So perhaps it makes sense to wait for the linked PRs to be resolved before tackling this?

Agreed! This issue is mostly for tracking the feature.


Buy DNS and Cloudflare

Buy DNS => https://www.hostinger.vn/ .
Buy CDN => https://www.cloudflare.com/.

Estimate: 16h
Deadline: 17 Dec 2022 - 18 Dec. 2022
please change the status In Progress


Segmentation fault during linclust

Current Behavior
While running mmseqs cluster I am getting a segmentation fault during the align step.
Thanks a lot, and I appreciate your help with this issue!
Steps to Reproduce (for bugs)
I created the mmseqs db then ran clustering:
mmseqs createdb sequences.fasta protein_db_dir
mmseqs cluster --min-seq-id 0.7 -c 0.7 --threads 100 --split-memory-limit 1000G protein_db_dir protein_clu_70 protein_db_dir/tmp
MMseqs Output (for bugs)
align /media/16TBNVME/datasets/hf/OMG/mmseqs_dir/tmp/16908102601775414309/linclust/2777415611686231349/input_step_redundancy /media/16TBNVME/datasets/hf/OMG/mmseqs_dir/tmp/16908102601775414309/linclust/2777415611686231349/input_step_redundancy /media/16TBNVME/datasets/hf/OMG/mmseqs_dir/tmp/16908102601775414309/linclust/2777415611686231349/pref_rescore2 /media/16TBNVME/datasets/hf/OMG/mmseqs_dir/tmp/16908102601775414309/linclust/2777415611686231349/aln --sub-mat 'aa:blosum62.out,nucl:nucleotide.out' -a 0 --alignment-mode 3 --alignment-output-mode 0 --wrapped-scoring 0 -e 0.001 --min-seq-id 0.7 --min-aln-len 0 --seq-id-mode 0 --alt-ali 0 -c 0.7 --cov-mode 0 --max-seq-len 65535 --comp-bias-corr 1 --comp-bias-corr-scale 1 --max-rejected<PHONE_NUMBER> --max-accept<PHONE_NUMBER> --add-self-matches 0 --db-load-mode 0 --pca substitution:1.100,context:1.400 --pcb substitution:4.100,context:5.800 --score-bias 0 --realign 0 --realign-score-bias -0.2 --realign-max-seqs<PHONE_NUMBER> --corr-score-weight 0 --gap-open aa:11,nucl:5 --gap-extend aa:1,nucl:2 --zdrop 40 --threads 100 --compressed 0 -v 3

Compute score, coverage and sequence identity
Query database size: 526411015 type: Aminoacid
Target database size: 526411015 type: Aminoacid
Calculation of alignments
Segmentation fault (core dumped)                                  ] 0.00% 1 eta -
Error: Alignment step died
Error: linclust died

Context
I am clustering a large protein dataset on a high ram machine (2Tb), from htop it does not look like ram is the problem at the time of segfault. Running the code on a small subset of the fasta did not reproduce the issue.
I also tried rerunning the align command with gdb and seeing:
Compute score, coverage and sequence identity
Query database size: 526411015 type: Aminoacid
Target database size: 526411015 type: Aminoacid
[New LWP 1278445]
[New LWP 1278446]
[New LWP 1278447]
...
Calculation of alignments
[>                                                                ] 0.00% 1 eta -
Thread 24 "mmseqs" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 1278464]
0x0000000000506f98 in SubstitutionMatrix::calcLocalAaBiasCorrection (m=0x12c1570, int_sequence=0x7f1b78000b60 "\r&+,$.)+&401$),(+1&+%2.)'+'9$%!$6$!&,$$6)+$9$49.+%,%2,.$%.%2,2!+6$%,.216%6'33-3.14!3210633!4.-$),+2,3.,%226&'31,$'.$..$3(,,\016\017\t\005\004\f\v\004\022\005\022\005\t\003\002\v\n\a\r\002\016\001\t\b\a\005\t\b\a\002\016\020\r\004\023\006\n\016\002", N=123, compositionBias=0x7f1b780d4a00, scale=1) at /home/vsts/work/1/s/src/commons/SubstitutionMatrix.cpp:112
112     /home/vsts/work/1/s/src/commons/SubstitutionMatrix.cpp: No such file or directory.
(gdb) bt
#0  0x0000000000506f98 in SubstitutionMatrix::calcLocalAaBiasCorrection (m=0x12c1570,
    int_sequence=0x7f1b78000b60 "\r&+,$.)+&401$),(+1&+%2.)'+'9$%!$6$!&,$$6)+$9$49.+%,%2,.$%.%2,2!+6$%,.216%6'33-3.14!3210633!4.-$),+2,3.,%226&'31,$'.$..$3(,,\016\017\t\005\004\f\v\004\022\005\022\005\t\003\002\v\n\a\r\002\016\001\t\b\a\005\t\b\a\002\016\020\r\004\023\006\n\016\002", N=123, compositionBias=0x7f1b780d4a00, scale=1) at /home/vsts/work/1/s/src/commons/SubstitutionMatrix.cpp:112
#1  0x00000000006e87f7 in SmithWaterman::ssw_init (this=0x7f1b78049100, q=0x7f1efdebaeb0, mat=0x7f1b78048f30 "\004", m=<optimized out>) at /home/vsts/work/1/s/src/alignment/StripedSmithWaterman.cpp:1247
#2  0x00000000006d4fdb in _ZN9Alignment3runERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_mmb._omp_fn.0(void) () at /home/vsts/work/1/s/src/alignment/Alignment.cpp:340
#3  0x00000000007d455e in gomp_thread_start ()
#4  0x00000000007e3299 in start_thread (arg=<optimized out>) at pthread_create.c:477
#5  0x0000000000982533 in clone ()

Your Environment
I installed the latest mmseqs release 16 using wget https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz; tar xvfz mmseqs-linux-avx2.tar.gz; export PATH=$(pwd)/mmseqs/bin/:$PATH
MMseqs Version:                         c2c3ad9c2956fac691d5a6041a9a4affa7fa27ad
Substitution matrix                     aa:blosum62.out,nucl:nucleotide.out
Seed substitution matrix                aa:VTML80.out,nucl:nucleotide.out
Sensitivity                             4
k-mer length                            0
Target search mode                      0
k-score                                 seq:2147483647,prof:2147483647
Alphabet size                           aa:21,nucl:5
Max sequence length                     65535
Max results per query                   20
Split database                          0
Split mode                              2
Split memory limit                      1000G
Coverage threshold                      0.7
Coverage mode                           0
Compositional bias                      1
Compositional bias                      1
Diagonal scoring                        true
Exact k-mer matching                    0
Mask residues                           1
Mask residues probability               0.9
Mask lower case residues                0
Minimum diagonal score                  15
Selected taxa
Include identical seq. id.              false
Spaced k-mers                           1
Preload mode                            0
Pseudo count a                          substitution:1.100,context:1.400
Pseudo count b                          substitution:4.100,context:5.800
Spaced k-mer pattern
Local temporary path
Threads                                 100
Compressed                              0
Verbosity                               3
Add backtrace                           false
Alignment mode                          3
Alignment mode                          0
Allow wrapped scoring                   false
E-value threshold                       0.001
Seq. id. threshold                      0.7
Min alignment length                    0
Seq. id. mode                           0
Alternative alignments                  0
Max reject<PHONE_NUMBER>
Max accept<PHONE_NUMBER>
Score bias                              0
Realign hits                            false
Realign score bias                      -0.2
Realign max seqs<PHONE_NUMBER>
Correlation score weight                0
Gap open cost                           aa:11,nucl:5
Gap extension cost                      aa:1,nucl:2
Zdrop                                   40
Rescore mode                            0
Remove hits by seq. id. and coverage    false
Sort results                            0
Cluster mode                            0
Max connected component depth           1000
Similarity type                         2
Weight file name
Cluster Weight threshold                0.9
Single step clustering                  false
Cascaded clustering steps               3
Cluster reassign                        false
Remove temporary files                  false
Force restart with latest tmp           false
MPI runner
k-mers per sequence                     21
Scale k-mers per sequence               aa:0.000,nucl:0.200
Adjust k-mer length                     false
Shift hash                              67
Include only extendable                 false
Skip repeating k-mers                   false

Thank you for the detailed bug report.
It seems like something has gone wrong with the input database:
"\r&+,$.)+&401$),(+1&+%2.)'+'9$%!$6$!&,$$6)+$9$49.+%,%2,.$%.%2,2!+6$%,.216%6'33-3.14!3210633!4.-$),+2,3.,%226&'31,$'.$..$3(,,\016\017\t\005\004\f\v\004\022\005\022\005\t\003\002\v\n\a\r\002\016\001\t\b\a\005\t\b\a\002\016\020\r\004\023\006\n\016\002"

This looks very broken. The first half should look like the second half.

Thank you for the quick reply! I am retrying the createdb and cluster from scratch to see if this was a one-off corruption.

Hi @milot-mirdita, I reran the createdb and cluster commands (after deleting previous mmseqs outputs), and ran into the same issue:
(gdb) bt
#0  0x0000000000506f98 in SubstitutionMatrix::calcLocalAaBiasCorrection (m=0x12c1570,
    int_sequence=0x7f1958000b60 "\r,3-0+'662(,0$$&7,3)$&,9(',,0+0$0,0#,.,\017\b\t\017\004\003\002\t\023\t\f\003\v\003\f\t\t\005\002\004\003\a\020\b\003\004\t\b\003\021\t\017\v\003\021\v\r\b\006\a\002\t\v\023\005", N=39, compositionBias=0x7f19580d4a00, scale=1) at /home/vsts/work/1/s/src/commons/SubstitutionMatrix.cpp:112
#1  0x00000000006e87f7 in SmithWaterman::ssw_init (this=0x7f1958049100, q=0x7f1bdf8efeb0, mat=0x7f1958048f30 "\004", m=<optimized out>) at /home/vsts/work/1/s/src/alignment/StripedSmithWaterman.cpp:1247
#2  0x00000000006d4fdb in _ZN9Alignment3runERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_mmb._omp_fn.0(void) () at /home/vsts/work/1/s/src/alignment/Alignment.cpp:340
#3  0x00000000007d455e in gomp_thread_start ()
#4  0x00000000007e3299 in start_thread (arg=<optimized out>) at pthread_create.c:477
#5  0x0000000000982533 in clone ()

Do you have suggestions on what else I can check? Thank you!

Can you try to check if there is something weird in the input fasta:
awk '/^>/ { next; } $0 ~ /[^A-Za-z]+/ { print; }' sequences.fasta

This should match any line that contains sequences that don't consist of letters A-Z in upper or lower case.

Hi @milot-mirdita thanks for the tip, I did find there are some rare occurrences of sequences starting with a "-" which I guess was meant to indicate a protein fragment, for example
-ERLANPIVAVVGRPNVGKSTLFNRIAGERISIVEDTPGVTRDRIYAHAEWLGKHFSMID
TGGIEISDQPLLTQIRQQAEVAIDEADVIIFVADVENGVTDADEQVARILYRSNKPVVLA
VNKVDNPERRSDIYDYYSLGLGEPYAVSSVHGIGMGDLLDAVIKEFPDNAANDEDDSIHF
SFIGRPNVGKSSLVNAILGENRVIVSNVAGTTRDAINTQFETADGQKFTMVDTAGIRKKG
KIYENTERYSLMRSMRAIDDSDVVLVVLNAEEGIRELDKHIAGYAHEAGCGVIIVVNKWD
TLKEKDHRTMTDFTNLIRQEFQYLSYAPIIFVSAKTKQRLNQLPGLIEEVYQHHRQRIQS
AVLNDVLMDAIAANPTPTQNGRRLRVYYGTQVATEPPTFVIFVNDPELMHFSYERYLENQ
IRKAFDFSGTPIHLIKRQRQ

Could this cause a segmentation fault? I am also trying to run the same code on the previous mmseqs release 15-6f452. So far it has not crashed but I will update here.

Very weird. I would really like to reproduce this issue locally.
The gap character should silently be turned into an unknown residue (X), it should not affect anything.


Data source not available

Good afternoon.
The data source cannot load, although it is added to the exceptions
Screenshots

alexanderzobnin-zabbix-datasource.log
Software versions

Grafana
Zabbix
Grafana-Zabbix Plugin

8.5.5
6.0.0
4.2.8

Oracle Linux Server release 8.6
5.4.17-2136.304.4.5.el8uek.x86_64 #2 SMP Wed Mar 9 15:08:23 PST 2022 x86_64 x86_64

@giovanny07
Hello.
The plugin is activated, zabbix does not play a role here at all, since the datasource does not appear in the list in grafana itself.

I found this in error log nginx
2022/06/16 07:48:22 [error] 37772#37772: *1 open() "/usr/share/grafana/public/plugins/alexanderzobnin-zabbix-app/module.js" failed (2: No such file or directory), client: <IP_ADDRESS>, server: grf-bor.xxxx.xxxx, request: "GET /public/plugins/alexanderzobnin-zabbix-app/module.js?_cache=4.2.8 HTTP/2.0", host: "grf-bor.xxxx.xxxx", referrer: "https://grf-bor.xxxx.xxxx/plugins/alexanderzobnin-zabbix-app"

Figured out, the problem was in nginx, he incorrectly redirected to the module.js, which is why the config page was not available

@dantte-lp,
could you elaborate how did you solved that problem?
My system upgraded to grafana 9.0.1 and now I lost zabbix graphics.

Grafana 10
Zabbix plugin installed and enabled but Data source ZABBIX is not available

logger=context userId=1 orgId=1 uname=admin t=2023-10-18T10:26:09.6947604+06:00 level=error msg="Request Completed" method=GET path=/api/gnet/plugins/alexanderzobnin-zabbix-app status=502 remote_addr=xx.xx.xx.xx time_ms=10 duration=10.0012ms size=0 referer=http://karvideoserv03:3000/plugins/alexanderzobnin-zabbix-app handler=/api/gnet/*

@baur can you please open a separate issue with what you think is the issue?


Docker file doesn't work

Hi,
Can you check the docker file?
The first error I got was this:
ERROR: failed to solve: nvidia/cuda:11.3.0-cudnn8-devel-ubuntu20.04: docker.io/nvidia/cuda:11.3.0-cudnn8-devel-ubuntu20.04: not found
I then edited the Dockerfile to use the following version:
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu20.04
But then I get the following error later:
**0.530 Using pip 23.3 from /opt/conda/envs/TradeMaster/lib/python3.9/site-packages/pip (python 3.9)
0.570 Processing /home/apex
0.573   Installing build dependencies: started
...
...
2.504     File "", line 5, in
2.505   ModuleNotFoundError: No module named 'packaging'
2.515   error: subprocess-exited-with-error
2.515
2.515   × Getting requirements to build wheel did not run successfully.
2.515   │ exit code: 1
2.515   ╰─> See above for output.
2.515
2.515   note: This error originates from a subprocess, and is likely not a problem with pip.
2.515   full command: /opt/conda/envs/TradeMaster/bin/python /opt/conda/envs/TradeMaster/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp7uip7l83
2.516   cwd: /home/apex
2.516   Getting requirements to build wheel: finished with status 'error'
2.518 error: subprocess-exited-with-error
2.518
2.518 × Getting requirements to build wheel did not run successfully.
2.518 │ exit code: 1
2.518 ╰─> See above for output.
2.518
2.518 note: This error originates from a subprocess, and is likely not a problem with pip.
Dockerfile:41
39 |     RUN git clone https://github.com/NVIDIA/apex
40 |     WORKDIR apex
41 | >>> RUN /opt/conda/envs/TradeMaster/bin/python -m pip install -v --no-cache-dir .
42 |
43 |     # Install requirements
ERROR: failed to solve: process "/bin/sh -c /opt/conda/envs/TradeMaster/bin/python -m pip install -v --no-cache-dir ." did not complete successfully: exit code: 1**

Found this worked per https://github.com/NVIDIA/apex/issues/1594
RUN /opt/conda/envs/TradeMaster/bin/python -m pip install -v --no-cache-dir --no-build-isolation .

Once I got through the apex install. I did run into another issues w/ the requirements.txt when installing gym 0.21. I fixed it with adding these lines before the install of requirements.txt. Might be able to just adjust the version of gym that is installed also.
RUN /opt/conda/envs/TradeMaster/bin/python -m pip install setuptools==65.5.1 pip==21
RUN /opt/conda/envs/TradeMaster/bin/python -m pip install wheel==0.38.0
RUN /opt/conda/envs/TradeMaster/bin/python -m pip install -r requirements.txt


Hotkeys being robbed by the browser instance instead of going to obsidian.

I'm having the following issue when an open gate has focus.
I have some hotkeys like F1 and F2 setup (without modifiers), and these keys do absolutely nothing, also ctrl+w is setup to close the open tab, it also only works after I click the sidebar.

I think it is normal behavior. Because when the frame has focus, it should handle the hotkey itself.

This could be an option, do you have any idea on how to implement the passthrough? I could do a PR.


Feature request: allow passing auxiliary data when defining custom gradients

Currently, jax allows defining custom gradients (where the gradient of a function a -> b has the form CT b -> CT a), but does not allow passing through auxiliary variables. Allowing auxiliary data would be particularly useful for reversible networks, where the backward pass has the form b -> CT b -> (a, CT a).

It's also useful to support jax.defvjp_all(f, rule) where f has auxiliary output (e.g. f is used as y, new_state = f(x, params, state)).


Connect to github repository assumes master branch and fails

Describe the bug
When I click on the Connect to git repository, import a repo, it assumes master branch, tries to get it, and fails. My default branch is main.
I think the default branch should be considered in the github integration
Link to the blitz that caused the error
https://stackblitz.com/edit/node-4iheam
Steps to reproduce
Go to the URL
Wait for it to try and load
Expected behavior
Stackblitz should not assume default branch as master, and should get the default branch
Screenshots
No response
Platform
Browser name  = Chrome
Full version  = 96.0.4664.45
Major version = 96
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36
performance.memory = {
  "totalJSHeapSize": 83505490,
  "usedJSHeapSize": 72264658,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Hash = 4a765956abf077f03ded84a27eb9cf5e3bfff5ad

Additional context
No response

Duplicate of #195

Thanks! I will stay on top of that issue


Fix copied doc in IReceiveEndpoint

Apparently doc for the Start method was copied from the Stop method, without any changes. Fixed the operation name in the doc.

Thanks


guide tour.

look up react library that gives user a guided tour through your application.

React Joyride

am I talking to myself?


backend saleor was unavailable after upgrading to 3.4.6 (reverted back to 3.4.3)

What I'm trying to achieve
Was trying to upgraded to latest saleor from 3.4.5
Steps to reproduce the problem
I ran the DB migrations just fine, but it seemed the graph QL API was totally unavailable, and something was causing many long running queries against the Database that would never finish on their own.
What I expected to happen
Even despite killing the long running queries, the graph QL playground UI was reachable but it would never get a response back from saleor API (unable to connect to server).
Screenshots and logs

I suppose though that even 3.4.5 has long running queries that never finish

I believe those long running queries must correlate back to a new penalty on my web.dev Best Practices score with the timeouts to /graphql

Last version that seems to run reliably for me is 3.4.3

Well this is quite an awful day for me after I upgraded saleor, now I cannot get back to a version that works reliably despite trying to downgrade.   I guess I didn't pay enough attention to what commits I was on that actually had a reliably working site.  The thing I notice is long running queries that get wedged for 5-10+ minutes at a time.  When I run the same query in data grip it is super fast.   I don't really understand either what these queries are about:  when its FETCH FORWARD 2000 FROM "_django_curs_139866817226496_sync_12" or CLOSE FORWARD 2000 FROM "_django_curs_139866817226496_sync_12"
I am not sure if the stack traces in the logs are from when I manually terminate the long running queries or what, but these are the only things in the logs:
  File "/app/saleor/core/permissions/__init__.py", line 124, in permission_required
    if isinstance(requestor, User):
  File "/usr/local/lib/python3.9/site-packages/django/utils/functional.py", line 246, in inner
    self._setup()
  File "/usr/local/lib/python3.9/site-packages/django/utils/functional.py", line 382, in _setup
    self._wrapped = self._setupfunc()
  File "/app/saleor/graphql/context.py", line 70, in user
    return get_user(request) or AnonymousUser()
  File "/app/saleor/graphql/context.py", line 60, in get_user
    request._cached_user = cast(UserType, authenticate(request=request))
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/debug.py", line 42, in sensitive_variables_wrapper
    return func(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/auth/__init__.py", line 76, in authenticate
    user = backend.authenticate(request, **credentials)
  File "/app/saleor/core/auth_backend.py", line 55, in authenticate
    return request.plugins.authenticate_user(request)
  File "/usr/local/lib/python3.9/site-packages/django/utils/functional.py", line 246, in inner
    self._setup()
  File "/usr/local/lib/python3.9/site-packages/django/utils/functional.py", line 382, in _setup
    self._wrapped = self._setupfunc()
  File "/app/saleor/core/middleware.py", line 101, in <lambda>
    lambda: _get_manager(_get_requestor_getter(request))
  File "/app/saleor/core/middleware.py", line 94, in _get_manager
    return get_plugins_manager(requestor_getter)
  File "/app/saleor/plugins/manager.py", line 1454, in get_plugins_manager
    return PluginsManager(settings.PLUGINS, requestor_getter)
  File "/app/saleor/plugins/manager.py", line 115, in __init__
    global_db_configs, channel_db_configs = self._get_db_plugin_configs()
  File "/app/saleor/plugins/manager.py", line 150, in _get_db_plugin_configs
    for db_plugin_config in qs:
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 280, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 98, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/app/saleor/graphql/views.py", line 53, in tracing_wrapper
    return execute(sql, params, many, context)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: terminating connection due to administrator command
FATAL:  server conn crashed?
SSL connection has been closed unexpectedly
ERROR saleor.graphql.errors.unhandled A query failed unexpectedly [PID:26:ThreadPoolExecutor-0_0]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/graphql/execution/executor.py", line 452, in resolve_or_error
    return executor.execute(resolve_fn, source, info, **args)
  File "/usr/local/lib/python3.9/site-packages/graphql/execution/executors/sync.py", line 16, in execute
    return fn(*args, **kwargs)
  File "/app/saleor/graphql/menu/schema.py", line 75, in resolve_menu
    return resolve_menu(
  File "/app/saleor/graphql/menu/resolvers.py", line 17, in resolve_menu
    menu = models.Menu.objects.filter(slug=slug).first()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 674, in first
    for obj in (self if self.ordered else self.order_by('pk'))[:1]:
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 280, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 98, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/app/saleor/graphql/views.py", line 53, in tracing_wrapper
    return execute(sql, params, many, context)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: SSL SYSCALL error: EOF detected

Its not a complete outage as the site sporadically  works making it even harder to figure out why its breaking.

Closing this as this is a duplicate of a discussion https://github.com/saleor/saleor/discussions/10021


Add support for registering decoration view classes rather than nibs

UICollectionViewLayout usually supports registering decoration views as both nibs and classes.
This change is to allow the SpreadsheetLayout subclass to also provide that functionality.

My idea at the moment is to create an empty protocol (such as DecorationViewType) and put extensions on UINib and AnyClass to make them conform to it.  Then the init can again take one parameter per corner.  The parameter names would need to change though, since they're not longer just nibs.  I'd prefer not to make a breaking change if possible.
I'm hesitant to just overload init with one that only takes AnyClass?, because the developer may want to mix nibs and classes.  This might be a temporary solution, though.

My concerns with that approach are:

The developer could pass something other than UINIb or AnyClass, so it would have to either ignore it, and/or output a runtime error.
Combining the initializer arguments would mean renaming them, which breaks backward compatibility.

If you're ok with both of these, I'll make the change.

To your concerns:

Yes thats true but I think a log warning in case of a wrong class and additional documentation for this method should be sufficient to address this issue. I think this is still better than additional parameters.

You are right, but it is just a minor change though so people will be able to handle that :)

It would be great if you could update the README and the demo application as well.
Thank you!

@swoolcock this is addressed in update 1.2.0:
I am using an enum as parameter which either holds a nib or class as associative value.


Changes to apply_corrections in studio.py

Addressing https://github.com/cleanlab/cleanlab-studio-integration/issues/1388

fixed current failed caused because "clean_label" column has been renamed to "corrected_label"
cleaned up null check

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@AnimatorJoe Great to see your first contribution.
Please do sign the CLA linked above (only need to do it once), thanks!

Can you also fix the formatter issues that are failling? You should be able to get the black vscode extension and set up autoformatting on save so you don't have to manually reformat