PID namespace handling

When running Perfetto in a containerized Linux environment the PID handling becomes rather tricky due to the different PID namespaces in play. Consider the following setup where we try to gather a timeline for the entire system using ftrace mixed with some of the processes using the SDK for instrumentation:
Host                traced
                    traced_probes

LXC Container A     application using SDK

LXC Container B     application using SDK

Directly on the host everything will be fine and PIDs seen by traced_probes match the root ns. Ftrace event parsing is also fine. However the SDK instances in applications A and B operate in their respective PID namespaces which means they will calculate different uuids for their ProcessTracks and ThreadTracks which in turn will cause these processes to show up twice in the recorded trace. Once with the Host PID and a second time with the respective Container PID.
I can think of several ways to solve this:

Make traced_probes aware of the namespaces a PID lives in and make it use the container PID. Downsides: Will require translation of PIDs in ftrace events and also increase the risk of collision between PIDs.
Add a way for traced to send the host PIDs to a client once it connects and receives the trace config so that it can use the host PIDs and report matching uuids accordingly. The major challenge here is that we need to keep this updated as the application spawns new threads.
Solve this in post processing using trace_processor by making it transparently merge uuids if they originate from the same process. This would require traced or traced_probes to record all the different PIDs a process or thread is known by so they can be merged later.

Has this scenario ever been considered? Do you have a preferred way on how to approach this?
Thanks!

Oh hey,
yes I know namespaces are very tricky.
We have been discussed this internally in the last month, the proposed solution involves a mixture of: (1) scraping the NS fields from /proc/pid/status (as part of traced_probes), (2) using the socket peer identity to resolve to the root pid identity, (3) doing massaging at import time in trace_processor.
It's very nuanced because:

ftrace events always emit events with the root namespace PIDs (The thread id really)
userspace events coming form the SDK instead will see only their own in-namespace pid
traced_probes will see the outer one but not necessarily know the namespace.

Let me try to clean up the document and share it publicly.
If you are interested in the discussion there are 15 pages of nuances :)

Awesome, that should come down to what I tried to (possibly oversimplified) sketch as option 3.
I would be very interested to learn about the nuances and what the proposed solution turned out to look like. So I look forward to and appreciate you preparing the document.

I just asked the author to share the doc (turns out I don't have ownership and I can't move it myself), ping me if you don't see the doc linked here in 24h.
3 alone won't work, because we need to correlate ftrace data (e.g. scheduling with root PIDs) with track-event data (With non-root pids). This involves changes in traced, traced_probes and trace_processor.

Thank you for that - an interesting read for sure.
From looking through the document I learn there is already a WIP at https://android-review.googlesource.com/c/platform/external/perfetto/+/1711848 adding initial support for this. Considering the WIP was not touched since May - do you happen to know if there is plans and/or resources allocated to drive this to production? Would you be open to accepting contributions on this very change from my side?

An update about this feature: we've recently landed the support of tracing processes running inside PID namespaces. To enable this:

Use a build that contains this fix.
Ensure that processes running in containers are visible to both traced and traced_probes. The easiest way to do this is running traced and traced_probes are in one of the ancestor of the PID namespace tree, or just running traced and traced_probes in the root PID namespace.
Use the canary channel of Perfetto UI (the trace_processor change may not hit the stable channel yet). This is unnecessary after the trace_processor change is on the stable channel.

Let us know if you see any bugs. Thanks for your patience!

Thanks a lot for the update. I will pull my local build and try if this is all we need for our use case.


Use Homebrew curl for fetching and audit when specified in the formula

Provide a detailed description of the proposed feature
When explicitly specified in a formula, using depends_on "curl" => :fetch or similar, Homebrew curl must be used instead of macOS curl. This will ensure that Homebrew curl is used when the formula's source is fetched and when auditing the formula URLs. This is not done by default to prevent unwanted linkage to Homebrew's curl. See https://github.com/Homebrew/brew/pull/11536 for some discussion.
What is the motivation for the feature?
macOS curl does not have TLS v1.3 support, so many servers which support only TLS v1.3 do not work with it. For example, sources for Xiph.org formulae are hosted on one such server, and thus fail the audit with Stable URL is not reachable. This prevents bottling for a specific OS using dispatch-build-bottle. Here is a list of formulae that would benefit from this feature (there may be others as well):

cdparanoia
ezstream
flac
icecast
libfishsound
libshout
libvorbis
libxspf
oggz
speex
theora
vorbis-tools

How will the feature be relevant to at least 90% of Homebrew users?
It may not be relevant to 90% of Homebrew users since we do not support building from source, but it would definitely help those on older, unsupported OSes which do not have bottles. It is also relevant to Homebrew maintainers and contributors working with affected formulae, and would help bottle them on other/newer OSes.
What alternatives to the feature have been considered?
Using Homebrew curl by default, but this is unnecessary (a very small portion of formulae require this feature so it isn't necessary to make this the default and encounter issues we do not want to, e.g. linkage with Homebrew-provided libcurl).

Another strategy would be to move the brew audit <formula> and brew livecheck steps out of the main test-bot call, and run these in the tap_syntax part of the workflow. (there should be a separate test bot command for that, really). And change from ubuntu 16.04 to 20.04 here https://github.com/Homebrew/homebrew-core/blob/master/.github/workflows/tests.yml#L21-L26, so that we use a newer version of curl. I tested the audit call with the curl provided by ubuntu 20.04 and it worked.
I think this is probably easier than having to manually define which curl should be used for specific formulae or to introduce a new DSL.

I think this also requires refactoring our audits so to run the ones that don't need a formula installed can be run separately from the ones that do.
It also doesn't solve the problem of fetching the tarball so we can actually build something, I think.

Another strategy would be to move the brew audit <formula> and brew livecheck steps out of the main test-bot call, and run these in the tap_syntax part of the workflow. (there should be a separate test bot command for that, really). And change from ubuntu 16.04 to 20.04 here https://github.com/Homebrew/homebrew-core/blob/master/.github/workflows/tests.yml#L21-L26, so that we use a newer version of curl. I tested the audit call with the curl provided by ubuntu 20.04 and it worked.

This doesn't solve the problem entirely but I do like the idea of moving some more logic like brew audit that's not platform-dependent onto Linux.

Test PR in homebrew-core: https://github.com/Homebrew/homebrew-core/pull/83470

This has been implemented and is now being used. You can add , using: :homebrew_curl to the url stanza in a formula and it will be used for downloads and audits. Test-bot will install brewed curl for you on CI.


Install Node.js without strict patch version in Docker build

Since we now require ~10.14 and ~6.4 versions for Node.js and NPM, we don’t need to install specific versions (with patch number) in the Docker build.

🦀 Requesting reviewers for this pull request:

@simonprev (contributor with 5 commits in the last 90 days and 15 commits overall)
@charlesdemers (reviewer for the elixir stack)


Get null or undefined to print as string 'null' or 'undefined' rather than as empty string.

Hi -
I have a dependency on your library in mine https://github.com/RobPethick/jest-theories - I was wondering what the recommended way would be to get null or undefined to print as 'null' or 'undefined' rather than printing as empty string? Is it possible with a format.extend?

@davidchambers looks like you fixed this in the commit you've referenced - has this been re-released to NPM since?

@RobPethick, a new version is now available from the npm registry:
$ npm view string-format version
2.0.0


onnx optimized for distilbert fails with batch > 1

To replicate , run make docker build on main branch
with batch 1 1 1

with batch 2 2 2

with batch 1 8 8

@pommedeterresautee
#84 - The same behaviour is observed for QA model

fix is merged. Closing


ACI - update partners filter section

Let's add all partners (partner sections) we have for now.

For all categories except ACI

I get - 404-not found


Harvester: Need a way to configure default replica of data / longhorn to 2 instead of 3

We do not want to have replica=3 on the harvester cluster in our disaster recovery data center. Due to that we want to have two replicas, only. How can we change the default of "replica=3" to "replica=2".

StorageClass is immutable in the k8s design, so in general, we would only suggest users create a new SC. If users have had to modify the replicas of the longhorn storageClass(i.e., before Harvester v1.1.2, the csi-driver didn't accept to pass custom storageClass name, so it will always default to use longhorn SC from the bare-metal Harvester cluster), the best approach is described as below:
TO TEST:

pre-requisite: the Harvester version should be v1.1.0 or v1.1.1.
from the UI, go to the Advanced -> Storage Classes page and check its listed storageClasses, and make sure the longhorn storageClass is not the default one.
change its replica count to 2 via:$kubectl edit managedchart -n fleet-local harvester

# add the persistence and defaultClassReplicaCount to the managedchart spec
spec:
  longhorn:
    defaultSettings:
      defaultDataPath: /var/lib/harvester/defaultdisk
      taintToleration: kubevirt.io/drain:NoSchedule
    enabled: true
    persistence: # add the persistence and defaultClassReplicaCount as below
      defaultClassReplicaCount: 2

upgrade the cluster to a newer version, e.g., v1.1.2-rc1, and verify the upgrade will work properly(not blocked by the above change), and the numberOfReplicas of longhorn storageClass is kept as 2.
create a new volume using the longhorn SC, and its newly generated volumes will only contain 2 replicas.

Verified failed on v1.1.2-rc1. Move the issue to implement
Result
After we change its replica count to 2 via $kubectl edit managedchart -n fleet-local harvester

And upgrade from v1.1.1 to v1.1.2-rc1, we failed to upgrade since the process stuck in Upgrading System Services

Test Information

Test Environment: 2 nodes Harvester on local kvm machine
Harvester version: v1.1.1 to v1.1.2-rc1

Verify Steps

Check the longhorn storageClass is not the default one and replica number is 3

Change its replica count to 2, under spec.value.longhorn

n1-111:~ # kubectl edit managedchart -n fleet-local harvester
managedchart.management.cattle.io/harvester edited

longhorn:
      defaultSettings:
        defaultDataPath: /var/lib/harvester/defaultdisk
        taintToleration: kubevirt.io/drain:NoSchedule
      enabled: true
      persistence: # add the persistence and defaultClassReplicaCount as below
        defaultClassReplicaCount: 2

Then upgrade to v1.1.2-rc1

Additional Context

Support bundle
supportbundle_dab56c69-ab45-46e3-8267-0bd42b008ce0_2023-02-22T09-15-27Z.zip

@TachunLin I have updated the testing cases, and please help to verify it again using the upcoming v1.1.2-rc2, thanks.

Verified fixed on v1.1.2-rc3. Close this issue.
Result

After upgrade, we can update the default replica of longhorn storage class to 2 without error message

longhorn:
      defaultClassReplicaCount: 2
      defaultSettings:
        defaultDataPath: /var/lib/harvester/defaultdisk
        taintToleration: kubevirt.io/drain:NoSchedule
      enabled: true
      persistence: # add the persistence and defaultClassReplicaCount as below
        defaultClassReplicaCount: 2

The longhorn storage class have two number of replicas

We create a new volume on longhorn storage class, it now have two replicas

Test Information

Test Environment: 3 nodes harvester on bare machines
Harvester version: v1.1.2-rc3

Verify Steps

Prepare a v1.1.1 three nodes Harvester cluster
Change the longhorn storage replica count to 2 via

kubectl edit managedchart -n fleet-local harvester

Run the same command to check the status
Upgrade to v1.1.2-rc3
Wait for upgrade complete
Change the longhorn storage replica count to 2 and wait the status into Ready

$ kubectl edit managedchart -n fleet-local harvester

longhorn:
      defaultClassReplicaCount: 2
      defaultSettings:
        defaultDataPath: /var/lib/harvester/defaultdisk
        taintToleration: kubevirt.io/drain:NoSchedule
      enabled: true
      persistence:
        defaultClassReplicaCount: 2
...
status:
  conditions:
  - lastUpdateTime: "2023-03-13T12:09:02Z"
    status: "True"
    type: Ready

Check the longhorn replia number on dashboard UI
Create a new volume on longhorn storage class
Access the longhorn UI  https://<IP_ADDRESS>/dashboard/c/local/longhorn
Check details setting of newly created volume


docs: 更新日志 --story=119092682

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.cheynechen seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


⚠ Bofrost.lu has degraded performance

In ca48e38, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 2021 ms

Resolved: Bofrost.lu performance has improved in facf303 after 18 minutes.


Feature request: Count the number of rows

I often want to count the number of rows in a CSV file, so that I can confirm with the person who produced the file that I've received the same number of rows that they exported from the database that originally holds the data.

On a Mac? Use the terminal: wc -l filename.csv
wc = word count
-l = lines (not words)

@Jmuccigr I had already tried that. It reported the correct line count but the wrong row count, because there were line breaks in some of the fields.

@stevevance Ah, yes, that can be a problem.

I'd suggest a few options:

Put it after the file title, so "Untitled (1,234 rows)". Could get tough with lots of rows.
Put it at the bottom with the other metadata.
Insert a dummy first column that's just a row number.

I like the column that has row numbers.

From the evil Excel 😈

First column idea! +1 🎉


Support stride for array_slice

Is your feature request related to a problem or challenge?
Currently, the array_slice function only accepts three arguments: the array, from, and to. However, it would be more convenient if it also supported a stride parameter.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response

I will work on this ticket.

I wonder if there is any other database system that has array_slice and that we can use as a prototype for our syntax and function support?

I wonder if there is any other database system that has array_slice and that we can use as a prototype for our syntax and function support?

In DuckDB, it has list_slice(list, begin, end, step) @alamb


hdparm: 9.48 -> 9.51

Motivation for this change
Update
Things done

[x] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Crap didn't notice will try to fix later today thanks
Sent from my iphone
On Mar 30, 2017, 12:00 PM -0500, Franz Pletz<EMAIL_ADDRESS>wrote:

@fpletz requested changes on this pull request.
In pkgs/os-specific/linux/hdparm/default.nix (https://github.com/NixOS/nixpkgs/pull/24369#discussion_r108980646):

}; - preBuild = '' - makeFlagsArray=(sbindir=$out/sbin manprefix=$out) - ''; + installPhase = '' + make DESTDIR=$out install

You omitted manprefix here. This causes the manpages to be installed to $out/usr/share/man.
—
You are receiving this because you were mentioned.


Windows taskbar progress bar and some miscellaneous changes

fixed build for -DWITH_CLIENT_INTERFACE=ON for windows
added -DWITH_PROGRESS_BAR=ON (by default ON shows window minimised with marquee progress bar until video is ready, OFF results in no window (not even taskbar) until video is ready)
raised the width/height limit to 8192 as wide screens go past 4096

@freerdp-bot test


Merge to live 2022-08-03

No detected changes to .json files in the root

OPS has passed. This can merge.


sentry/sentry not installable with openai-php/client

How do you use Sentry?
Sentry Saas (sentry.io)
Version
3.16.0
Steps to Reproduce
Run composer require openai-php/client and composer require sentry/sentry.
Expected Result
Both librarys should be installable.
Actual Result
Because of php-http/discovery: ^1.11, <1.15 it is not possible to install sentry. Maybe it is possible to upgrade php-http/discovery to a newer version?

See https://github.com/getsentry/sentry-php/pull/1504
We need to do some more testing, so expect a release next week.

Thanks :) This would be awesome

3.17 released


FontManager & multithread

Hi,
have an application for generating images with texts. As images are huge amount, I make new version with multiple threads. How can I make this ready, when FontManager should be only one instance, idealy under linux commandline application?
Could you help me?
Thanks

FontManager single instance object is constructed in the initialization section, so there should be no issue with thread safety there. (However, the constructor could be improved by raising an exception if the aFontManager var isn't nil, just in case a user tries to create a second instance.)
From what you've described, ISTM that you could reasonably load all your fonts into FontManagerin the primary thread before you start multithreading. And assuming the above TFontManager.GetBestMatchFont should be thread safe.

Thank you, will try it this way. I'm also using only one font.


Service or alias has been removed or inlined when the container was compiled

Hi,
I'm using this bundle in a Symfony 4 app and am getting the following warning when clearing cache etc:
The "App\Application\EventListener\User\PasswordResetRequestedListener" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead.

My event listener is wired up in services.yaml as follows:
App\Application\EventListener\User\PasswordResetRequestedListener:
    class: App\Application\EventListener\User\PasswordResetRequestedListener
    tags:
        - { name: tactician.event_listener, event: App\Domain\Events\PasswordResetRequested }

My Tactician config is as follows:
tactician:
    commandbus:
        default:
            middleware:
                - tactician_domain_events.middleware.release_recorded_events
                - tactician.middleware.locking
                - tactician.middleware.doctrine
                - tactician.middleware.command_handler

tactician_domain_event:
    collect_from_all_managed_entities: true

I've done some searching and it seems to be related to Symfony 4 making services private by default. I don't want to override this for every event listener. Am I missing something in the config?
Thanks

I'm in the process of upgrading a project towards sf4 and hit the same error.
My services were previously private by default but I made my subscribers implement an interface and made them public via this
    _instanceof:
        App\EventListener\DomainEventSubscriberInterface:
            public: true
            tags:
                - { name: tactician.event_subscriber }

But now, this does not seem to have the appropriate effect at the right moment. It could also very well be something else happening during this migration.

did you find a solutin @tmcnicholls ?

@borNfreee not yet, other than making the listeners explicitly public as a temporary measure like the below:
App\Application\EventListener\User\PasswordResetRequestedListener:
    public: true
    tags:
        - { name: tactician.event_listener, event: App\Domain\Events\PasswordResetRequested }

Do you have any other ideas?
Thanks


Materials should use stereo eye position for reflections in multiview

In this playground, for example: https://playground.babylonjs.com/#9K3MRA#754
We have a PBR material that is rendered in multiview, but the uniform vEyePosition used to calculate reflections is only derived from the current active camera (which in multiview rendering corresponds to the left eye). In order to get proper reflections when rendering in multiview we should have a second uniform vEyePositionR or a matrix that compensates for the stereo offset between the left & right eyes that we'd multiply by vEyePosition in order to create vEyePositionR in the shader.
https://github.com/BabylonJS/Babylon.js/blob/4bc7b8000a3a3a94e8701f96087020d71e1df7aa/src/scene.ts#L786-L791
https://github.com/BabylonJS/Babylon.js/blob/4bc7b8000a3a3a94e8701f96087020d71e1df7aa/src/Shaders/ShadersInclude/reflectionFunction.fx#L132

https://forum.babylonjs.com/t/materials-should-use-stereo-eye-position-for-reflections-in-multiview/52145


Enforce chunks ordering when writing index.

This PR enforces ordering of chunk references when adding series to the index. This is done to avoid direct users of chunks.Writer and index.Writer to write chunks unordered. Prometheus itself already writes chunks ordered.

Since https://github.com/cortexproject/cortex/pull/3371 is merged, I think that this is no longer needed, but that we could still have the documentation update. Is that correct? Thanks!

I think this PR would still be nice to get in, so that others don’t make the same mistake. I have forgotten about it, but will try to finish it (address Bartek’s suggestions) soon. Thanks for patience.

Awesome 😎

Nice, I think this is great but we talked with @codesome and we have 2 suggestions:

Can we update format.md for this invariant?

Added.

Can we also ensure that chunks are put together for the series. Wonder if it's efficient to check it, but maybe we can have at least the test AND docs 🤔

Check for increasing chunk ref number already guarantees this property.
I've updated the PR with checks for ordering of minT and maxT for chunks in the same series:

for chunk, c.minT must be less or equal to c.maxT
for two subsequent chunks, c_i.maxT < c_(i+1).minT must hold

These are already implicit in encoding of minT/maxT in the index.

Tests are failing.

I think this is still relevant, I will update the PR to resolve conflicts.

for two subsequent chunks, c_i.maxT < c_(i+1).minT must hold

Funnily enough, this check would help us to detect https://github.com/prometheus/prometheus/issues/12873 earlier :)

Any chance for getting this merged, so that I don't need to rebase again? :)

:wave: from the bug scrub again. Thank you very much @pstibrany. This is almost good to go. @bboreham will have a last look.


Support running against vite-builder projects

The assets generated by vite differ somewhat from webpack's, so a few tweaks were needed here in order to accommodate  @storybook/builder-vite projects.
This tool does not have knowledge of which builder is being used, so one side-effect is that it will no longer throw an error if preview files are missing from a webpack build.  I'm not sure whether that's a show stopper or not here, but I was trying to avoid making any architectural changes.
One other minor change that isn't directly related to vite, is that I used a Promise.all() rather than multiple awaits which result in a promise chain.  This should help performance of the tool itself slightly.

:rocket: PR was released in v0.7.0 :rocket:


Missing license

Hi Jean,
Great work, I have found your video on the Verlet magic quite educational! I was wondering what sort of license did you think of for this piece of code. Have you considered what license to use for this specific repo?
If you went with something permissive I'd like to use this as a starting point for a project.
Thanks

Hi,
I just updated the repo with the MIT license which should allow you to do whatever you want with the code :)

Super, thank you for that and for the blazing swift response. :)


Update OAI-PMH feed for New OpenVault

We need to make sure that the new Open Vault record pages that have digital objects can be harvested by Digital Commonwealth. This is important for the MVP.

@mccalluc: I am fine with this being post-launch as long as it is implemented within 2-3 weeks after launch.

@afred: there were more conflicts before, but even after a bundle update I get this error:
$ bundle update
...
Bundler could not find compatible versions for gem "rails":
  In Gemfile:
    rails (= 4.1.2)

    blacklight_oai_provider was resolved to 0.0.1, which depends on
      rails (~> 3.0)

Options:

Go with an older Rails version. (bad)
Invest time to updating blacklight_oai_provider. (good? but uncertain how big a job.)
Make our own. (bad)
Or take a step back: The DPLA seemed actually pretty flexible about how the data gets to them... ie, from some folks it sounds like they get CSVs. Could we just give them the same zip of pbcore that the site ingests? (Zero development!)

How do we broach this?

@mccalluc @afred Please see email I've cc'd you both on from Digital Commonwealth, and follow up with Eben.

While feasible to use a zip file, it would make doing future updates more difficult. The OAI feed is likely more sustainable on our end.

Would the zip file be just OpenVault items, or would there be other content mixed in? Would there be file-only records that we'd have to filter out?

What's the general timeline for getting an OAI feed up, with links that point to items on the new OpenVault site? (Don't need specifics, and not trying to add any pressure, just wondering for the sake of managing our own harvesting timelines here at the BPL.)
I should just try again tomorrow... there's the chance gem have been updated and it'll just work this time.

same error. alas. Going to make just ListRecords; shouldn't be hard.

:+1: thanks.
I'll bet that if you peek at the gem, it's probably not doing much more than that anyway - maybe with a little extra blacklight magic (black magic?)


Updating vulnerable dependencies according to assessment report : 665e10ba4b5c9c11599f7905

Summary Report

Scenario
Upgrades
Critical
High
Medium
Low

Full Upgrade
0
0
0
0
0

Full Upgrade
66
7
16
43
0


Enhance Web Interface

Yield proper output and make it possible to interact with the program on the web interface.

Output is done, interaction comes next. Currently we can't even interact on the console, but that's going to be fixed


Support resolving $refs that point to other local files on disk

Hello!
I have a big project that uses many, many JSON schemas split up in dozens of files. I have one folder called types which contains schemas that many other schemas reference via $ref statements.
I'd like to add support to json-schema-defaults for resolving these file $refs. I can see two routes that would both need to be supported:

If a filepath is provided as the first argument, automatically take care of parsing that file and resolving any $refs relative to that filepath.
If a JSON object is provided as the first argument, the caller would need to also supply some option that tells json-schema-defaults what directory to start in when resolving file $refs.

Does this sound like functionality that you want in this package? If so, I'll work on a pull request.
Thanks!
Alex

Oops, I spoke too soon.
Turns out that the incredible json-schema-ref-parser is purpose-built for this exact task, and its output can be fed directly into json-schema-defaults. Maybe adding a link to the README would be helpful?

Great that you were able to get it working. Using a separate package is indeed the preferred way to do it.


CUDA error

Bug description
I try the esmfold_inference.py script with the included data and got a CUDA error.
Reproduction steps
After installing the program following the instructions in the repo, I run this:
python scripts/esmfold_inference.py -i examples/data/few_proteins.fasta -o /home/ubuntu/

Expected behavior
Get no error.
Logs
(esm) ubuntu@ip-10-0-0-77:~/esm$ python scripts/esmfold_inference.py -i examples/data/few_proteins.fasta -o /home/ubuntu/
23/02/15 22:31:53 | INFO | root | Reading sequences from examples/data/few_proteins.fasta
23/02/15 22:31:53 | INFO | root | Loaded 3 sequences from examples/data/few_proteins.fasta
23/02/15 22:31:53 | INFO | root | Loading model
23/02/15 22:31:55 | INFO | torch.distributed.nn.jit.instantiator | Created a temporary directory at /tmp/tmpwt8fglax
23/02/15 22:31:55 | INFO | torch.distributed.nn.jit.instantiator | Writing /tmp/tmpwt8fglax/_remote_module_non_scriptable.py
23/02/15 22:33:43 | INFO | root | Starting Predictions
Traceback (most recent call last):
  File "/home/ubuntu/esm/scripts/esmfold_inference.py", line 157, in <module>
    output = model.infer(sequences, num_recycles=args.num_recycles)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/esm/esmfold/v1/esmfold.py", line 282, in infer
    output = self.forward(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/esm/esmfold/v1/esmfold.py", line 163, in forward
    esm_s = self._compute_language_model_representations(esmaa)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/esm/esmfold/v1/esmfold.py", line 108, in _compute_language_model_representations
    res = self.esm(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/esm/model/esm2.py", line 112, in forward
    x, attn = layer(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/esm/modules.py", line 125, in forward
    x, attn = self.self_attn(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/esm/multihead_attention.py", line 357, in forward
    attn_weights = torch.bmm(q, k.transpose(1, 2))
RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasGemmStridedBatchedExFix( handle, opa, opb, m, n, k, (void*)(&falpha), a, CUDA_R_16F, lda, stridea, b, CUDA_R_16F, ldb, strideb, (void*)(&fbeta), c, CUDA_R_16F, ldc, stridec, num_batches, CUDA_R_32F, CUBLAS_GEMM_DEFAULT_TENSOR_OP)`

Additional context
Ran in a g4dn.2xl EC2 instance that has 32Gb RAM and everything installed without any error.
Also:
/usr/local/cuda/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

Same when running one of the sample programs.
This is the code:
import torch
import esm

model = esm.pretrained.esmfold_v1()
model = model.eval().cuda()

# Optionally, uncomment to set a chunk size for axial attention. This can help reduce memory.
# Lower sizes will have lower memory requirements at the cost of increased speed.
# model.set_chunk_size(128)

sequence = "MKTVRQERLKSIVRILERSKEPVSGAQLAEELSVSRQVIVQDIAYLRSLGYNIVATPRGYVLAGG"
# Multimer prediction can be done with chains separated by ':'

with torch.no_grad():
    output = model.infer_pdb(sequence)

with open("result.pdb", "w") as f:
    f.write(output)

import biotite.structure.io as bsio
struct = bsio.load_structure("result.pdb", extra_fields=["b_factor"])
print(struct.b_factor.mean())  # this will be the pLDDT
# 88.3

And here is the run:
(esm) ubuntu@ip-10-0-0-77:~/esm$ python p2.py
Traceback (most recent call last):
  File "/home/ubuntu/esm/p2.py", line 15, in <module>
    output = model.infer_pdb(sequence)
  File "/home/ubuntu/esm/esm/esmfold/v1/esmfold.py", line 305, in infer_pdb
    return self.infer_pdbs([sequence], *args, **kwargs)[0]
  File "/home/ubuntu/esm/esm/esmfold/v1/esmfold.py", line 300, in infer_pdbs
    output = self.infer(seqs, *args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/ubuntu/esm/esm/esmfold/v1/esmfold.py", line 277, in infer
    output = self.forward(
  File "/home/ubuntu/esm/esm/esmfold/v1/esmfold.py", line 156, in forward
    esm_s = self._compute_language_model_representations(esmaa)
  File "/home/ubuntu/esm/esm/esmfold/v1/esmfold.py", line 103, in _compute_language_model_representations
    res = self.esm(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/esm/esm/model/esm2.py", line 112, in forward
    x, attn = layer(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/esm/esm/modules.py", line 125, in forward
    x, attn = self.self_attn(
  File "/home/ubuntu/miniconda3/envs/esm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ubuntu/esm/esm/multihead_attention.py", line 357, in forward
    attn_weights = torch.bmm(q, k.transpose(1, 2))
RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasGemmStridedBatchedExFix( handle, opa, opb, m, n, k, (void*)(&falpha), a, CUDA_R_16F, lda, stridea, b, CUDA_R_16F, ldb, strideb, (void*)(&fbeta), c, CUDA_R_16F, ldc, stridec, num_batches, CUDA_R_32F, CUBLAS_GEMM_DEFAULT_TENSOR_OP)`

Hi @sbassi , could you please provide your CUDA and pytorch version numbers?

Here is according to the AMI:
PyTorch 1.11.0 (Ubuntu 20.04)
CUDA version: 11.5
NVIDIA driver version: 510.47.03
And this is what I actually using:
CUDA:
(esm) ubuntu@ip-10-xx:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

Pytorch:
(esm) ubuntu@ip-10-0-0-77:~$ python
Python 3.9.16 | packaged by conda-forge | (main, Feb  1 2023, 21:39:03)
[GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.__version__)
1.13.1+cu117

From this listing, which one do you recommend?
https://docs.aws.amazon.com/dlami/latest/devguide/appendix-ami-release-notes.html
MOTD:

Looks like it was my error since I made my own conda env instead of using the pre built environment that is ran with "source activate pytorch".
Now I am using a new AMI (Deep Learning AMI GPU PyTorch 1.12.1 (Ubuntu 20.04) 20220926) and trying this conda env, and worked, so I will close this issue. Thank you very much for your help.


[BUG] exited with status 100 at line 118

I don't know how to better describe it than the screenshot already shows.
Using the OAK-D-POE for the first time, first time set up.
I am Working in Ubuntu 20.04 atm, have a virtual environment set up for it with python 3.10. I can provide additional information, not sure which would be useful to inspect this problem further.
Cheers

Can you describe your environment in more details - what HW are you running on?
How have you set-up your environment, e.g. creating the virtual-env?
Which exact Ubuntu OS do you use, is it a public distribution or your own? Is it a stock-ubuntu?
To circumvent local OS-config issues (e.g. like the multiple repo entries) - would it be possible to use a "clean" Ubuntu docker container, start it with host-network and proxy ENV variables - and call the install-script?
Are you behind a corporate network proxy? Can you check whether the failing repo-URLs can be resolved in your environment, e.g. via a web-browser or a wget?

Hello Markus,
Thanks for the immediate response. Attached you find my try to immediately install without going trough a virt env. Meanwhile my colleague was able to immediately get it running on his machine.
what HW are you running on?
I have the G513QR from Asus so basically https://rog.asus.com/de/laptops/rog-strix/2021-rog-strix-g15-series/spec/
Relevant (I guess) RTX™ 3070 Laptop GPU
AMD Ryzen 9 5000 Series
creating the virtual-env?
Yes, the first attempt was set up with a virtual environment with anaconda. At creation time I installed python3 and nothing else. Then I tried to follow the instructions.
Which exact Ubuntu OS do you use, is it a public distribution or your own? Is it a stock-ubuntu?
Using of the rack Ubuntu 20.04 focal (atm 20.04.3 LTS)
Are you behind a corporate network proxy?
No
Can you check whether the failing repo-URLs can be resolved in your environment, e.g. via a web-browser or a wget?
Yes and yes for http://ppa.launchpad.net/webupd8team/atom/ubuntu/ with regards to the missing IP I found https://stackoverflow.com/questions/72284245/apt-upgrade-failing-keys but that doesn’t tell me anything as I am still a light Linux user.
To circumvent local OS-config issues (e.g. like the multiple repo entries) - would it be possible to use a "clean" Ubuntu docker container, start it with host-network and proxy ENV variables - and call the install-script?
Having trouble with docker atm, I will look into a clean reinstall, how about a clean ubuntu in a VM with virtual box? Unless that creates known issues
Hope this helps with the search for a solution. If you need more/other information feel free to ask.
Cheers,
Malte Weber
From: Markus Broghammer @.>
Sent: Thursday, September 8, 2022 7:48 PM
To: luxonis/depthai @.>
Cc: Malte Weber, AI.Land @.>; Author @.>
Subject: Re: [luxonis/depthai] [BUG] exited with status 100 at line 118 (Issue #788)
Can you describe your environment in more details - what HW are you running on?
How have you set-up your environment, e.g. creating the virtual-env?
Which exact Ubuntu OS do you use, is it a public distribution or your own? Is it a stock-ubuntu?
To circumvent local OS-config issues (e.g. like the multiple repo entries) - would it be possible to use a "clean" Ubuntu docker container, start it with host-network and proxy ENV variables - and call the install-script?
Are you behind a corporate network proxy? Can you check whether the failing repo-URLs can be resolved in your environment, e.g. via a web-browser or a wget?
—
Reply to this email directly, view it on GitHubhttps://github.com/luxonis/depthai/issues/788#issuecomment-1241097038, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AXGHTTTI3NQEST2IM2R6I43V5IYELANCNFSM6AAAAAAQHZYNXM.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

The aforementioned screenshot

@Weber-AILand
Usually apt-get update does not error out on such cases, but it seems that the nvidia & atom repo sources are throwing it off - an solution to that is to remove/fix those two particular sources and retry.
Another option is to downloading the script (install_dependencies.sh), removing the 3rd line (set -x) and then launch it: bash install_dependencies.sh and check if it works as expected for you.


Implement operators with 128bits

Depends on #63
This is the second part to #40

Please rebase on the rebased+merged #63.

Please rebase on the rebased+merged #63.

@cuviper done

Thanks!
bors r+

bors r+


nm: Reporting should not fail if ovs plugin is missing

When NetworkManager-ovs package is missing, the report execution failed
by raising an exception.
nmstate should not fail in such cases. It is sufficient to ignore OVS
related interfaces.

Coverage decreased (-0.09%) to 72.96% when pulling 064ef3030cfcb1dbd2c0ed9cda99d94f76247fd7 on EdDev:show-with-no-ovs-support into 5065c4fd99e9d1797cb5fbd6c8d11718de5d9a68 on nmstate:master.


Error when running azurite with host.docker.internal

Which service(blob, file, queue, table) does this issue concern?
Queue service
Which version of the Azurite was used?
3.31.0, 3.14.0
Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
Dockerhub
What's the Node.js version?
What problem was encountered?
I am trying to run aspire, this is my code:
var storage = builder.AddAzureStorage("Storage");

if (builder.Environment.IsDevelopment())
{
    storage.RunAsEmulator(
        azuriteBuilder =>
        {
            azuriteBuilder
                .WithArgs(
                [
                    "docker-entrypoint.sh",
                    "azurite",
                    "-l",
                    "/data",
                    "--blobHost",
                    "<IP_ADDRESS>",
                    "--queueHost",
                    "<IP_ADDRESS>",
                    "--tableHost",
                    "<IP_ADDRESS>",
                    "--disableProductStyleUrl",
                    "--debug",
                    "/data/debug.log"
                ]);
        });
}

var blobs = storage.AddBlobs("blobs");
var queues = storage.AddQueues("queues");

builder.AddContainer("kernel-memory", "kernelmemory/service", "latest")
    .WithImageRegistry("docker.io")
    .WithBindMount("kernel.memory.appsettings.Development.json", "/app/appsettings.json")
    .WithEnvironment("KernelMemory__Services__AzureBlobs__ConnectionString", blobs)
    .WithEnvironment("KernelMemory__Services__AzureQueues__ConnectionString", queues)
    ;

The connection strings generated look like this:
DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;QueueEndpoint=http://host.docker.internal:61222/devstoreaccount1;

The issue is that every time that Kernel-Memory service tries to create a Queue It gets back a  400 bad request respose.
Kernel Memory connects to QueueClient using nuget package:
Azure.Storage.Queues 12.19.1
Steps to reproduce the issue?
-09-02T14:00:27.877Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 info: QueueStorageContextMiddleware: RequestMethod=PUT RequestURL=http://host.docker.internal/delete-generated-files RequestHeaders:{"host":"host.docker.internal:61572","x-ms-version":"2018-11-09","accept":"application/xml","x-ms-client-request-id":"3a95397e-18b5-4ce7-a40d-3ae9e763ded1","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Queues/12.18.0 (.NET 8.0.7; Alpine Linux v3.20)","x-ms-date":"Mon, 02 Sep 2024 14:00:22 GMT","authorization":"SharedKey devstoreaccount1:JOmKGBzEHz1pb5AQehQfdU7xpimHkEyXmu1GnVOrY8g=","content-length":"0"} ClientIP=<IP_ADDRESS> Protocol=http HTTPVersion=1.1
2024-09-02T14:00:27.878Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 info: QueueStorageContextMiddleware: Account=delete-generated-files Queue=undefined Message=undefined MessageId=undefined
2024-09-02T14:00:27.879Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 verbose: DispatchMiddleware: Dispatching request...
2024-09-02T14:00:27.893Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 error: DispatchMiddleware: Incoming URL doesn't match any of swagger defined request patterns.
2024-09-02T14:00:27.895Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-09-02T14:00:27.907Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 error: ErrorMiddleware: ErrorName=UnsupportedRequestError ErrorMessage=Incoming URL doesn't match any of swagger defined request patterns.  ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=undefined ErrorHTTPHeaders=undefined ErrorHTTPBody=undefined ErrorStack="UnsupportedRequestError: Incoming URL doesn't match any of swagger defined request patterns.\n    at dispatchMiddleware (/opt/azurite/dist/src/queue/generated/middleware/dispatch.middleware.js:41:30)\n    at /opt/azurite/dist/src/queue/generated/ExpressMiddlewareFactory.js:49:47\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:328:13)\n    at /opt/azurite/node_modules/express/lib/router/index.js:286:9\n    at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:346:12)\n    at next (/opt/azurite/node_modules/express/lib/router/index.js:280:10)\n    at queueStorageContextMiddleware (/opt/azurite/dist/src/queue/middlewares/queueStorageContext.middleware.js:97:5)\n    at /opt/azurite/dist/src/queue/middlewares/queueStorageContext.middleware.js:15:16\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)"
2024-09-02T14:00:27.907Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 error: ErrorMiddleware: Set HTTP code: 400
2024-09-02T14:00:27.907Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 error: ErrorMiddleware: Set HTTP body: undefined
2024-09-02T14:00:27.908Z 6419cfd6-b51c-4dd7-98c0-32262af4fca5 info: EndMiddleware: End response. TotalTimeInMS=32 StatusCode=400 StatusMessage=undefined Headers={"server":"Azurite-Queue/3.31.0"}

Have you found a mitigation/solution?
No

@EmmaZhu
Would you please help to look?
Besides that, it looks the Uri in the debug log is not right "http://host.docker.internal/delete-generated-files", the Uri not include storage account name. It should looks like "http://host.docker.internal/[AccountName]/[QueueName]"

Hello @blueww
I have figured out that this problem is not coming from the azurite emulator, but from QueueClient in the .net azure-sdk.
Currently I have not opened an issue there, but I will and link it afterwards.

@Valkozaur
I will close this issue as it's not an Azurite issue but a client issue.
Feel free to contact us if need any further assistance on Azurite.


How dynamically add ractive partials

Description:
I am trying to show my partials, They are set but it not showing
The following is possible:
{{> xxx-xxx-xxx {paths: path}}} -- this works
but I am tring to load it dynamically: I got all my partial templates in conceptTemplates and would like to do the following:
{{#each conceptTemplates : num }}
{{#if num === conceptType}}
    {{> {{ . }} {paths: path} }}
{{/if}}
{{/each}}

please help, need a solution

Could you share a link to ractive.js.org/playground or jsfiddle or something like that to give us something to poke at? I think I understand what you're doing, but I'm not positive. You definitely don't need nested mustaches, but the data in concept templates would have to be pre-parsed partial arrays. There are other ways to approach it that don't require parsing up front, but an example to work with would be helpful.

@evs-chris  Sorry body I got no time maybe later: just to give you little more description
my objective is to load partial template from external source:
using the following:
let partials = {}, promises;

promises = Object.values(codeSystemTemplates[0]).map( function ( name, type ) {
                        return fetch( '/' + name + '.html' ).then( function ( response ) {
                                return response.text()
                        }).then(function(body) {
                            if (type == conceptType) {
                                partials[name] = body;
                            }
                        });
});

do the following:
Promise.all( promises ).then( function () {
                    let ractive = Ractive({
                        el: '#' + path,
                        template: mainTemplate,
                        data: init_data,
                        partials: partials
                    });
 });

So the Partials come as:
partials: name: Array(1)
But my problem is when it comes to display in the view?? how??
{{> partials.name {paths: path}}} ????? is not working

This example might help:
https://dagnelies.github.io/ractive-examples/content/tabs.html

I solved by generating new ractive instance for each template and pass it on.


Simple text with stroke border example

Hello,
Im trying to render text with a solid color border, similar to like the Stroke example in the readme.
Ive been playing with the style / context and other config. Heres my current attempt with no output and no success. Please let me know what I'm missing from the config.
func main() {
	c := canvas.New(200, 200)
	ctx := canvas.NewContext(c)

	prohibition := canvas.NewFontFamily("prohibition")
	_ = prohibition.LoadFontFile("ProhibitionTest-Regular.ttf", canvas.FontRegular)
	face := prohibition.Face(51, canvas.Green)

	faceFrontPath, _, _ := face.ToPath("Lorem ipsum")

	ctx.SetStrokeWidth(10)
	ctx.SetStrokeColor(color.RGBA{0, 255, 0, 255})

	ctx.RenderPath(faceFrontPath, canvas.Style{
		Fill: canvas.Paint{
			Color:    color.RGBA{0, 0, 255, 255},
			Gradient: nil,
			Pattern:  nil,
		},
		Stroke:       canvas.Paint{Color: color.RGBA{255, 0, 0, 255}, Gradient: nil, Pattern: nil},
		StrokeWidth:  10,
		StrokeCapper: canvas.SquareCapper{},
		StrokeJoiner: canvas.MiterJoiner{},
		DashOffset:   0,
		Dashes:       nil,
		FillRule:     0,
	}, canvas.Matrix{})

	if err := renderers.Write("getting-started.png", c, canvas.DPMM(10.0)); err != nil {
		panic(err)
	}
}

Hi Mark, thanks for reaching out!
The problem is the use of canvas.Matrix{}. The zero value is a transformation matrix of all zeros, which multiplies the path by zero horizontally and vertically, so nothing gets draw. Use canvas.Identity instead.
You can improve your code, and I would recommend using the DrawPath command instead of the lower-level RenderPath command. I would rewrite your example as:
package main

import (
	"image/color"

	"github.com/tdewolff/canvas"
	"github.com/tdewolff/canvas/renderers"
)

func main() {
	c := canvas.New(200, 200)
	ctx := canvas.NewContext(c)

	prohibition := canvas.NewFontFamily("prohibition")
	if err := prohibition.LoadFontFile("ProhibitionTest-Regular.ttf", canvas.FontRegular); err != nil {
            panic(err) // check error!
}

        pt := 51.0
	face := prohibition.Face(pt)
	p, _, err := face.ToPath("Lorem ipsum")
        if err != nil {
            panic(err) // check error!
        }

        x, y := 100.0, 100.0
	ctx.SetStrokeWidth(10.0)
	ctx.SetStrokeColor(canvas.Green)
	ctx.DrawPath(x, y, p)

	if err := renderers.Write("out.png", c, canvas.DPMM(10.0)); err != nil {
		panic(err)
	}
}

Awesome! Thank you! Im not well versed in graphics, this is just a means to an end, so this might be an ignorant question. Playing with some of the configuration Im getting some artifacting/pixelation.

Is that a consequence of the ttf file or something else? Is there a way to Make the transition between the border and fill more binary and less... blurry?

Hey Mark, what happens is that the vector paths (the mathematical definition of the shapes from point to point) gets converted into a raster image (such as your picture) to be possible to show on a computer screen. This is called rasterization and causes diagonal lines to be pixelated.
This isn't a problem gives your pixels are small enough (i.e. high resolution). In your case it looks like your resolution is very low, and then you zoomed in. Try to increase the DPMM value at the end of the code to 40.0 for example. Let me know if that looks better.


Check if all tools are installed and configure linters, code formatters

Added flake8 and black for syntax checking and code formatting. Also added pytest dependencies for testing.

Closing the issue now since the pull request has been merged.


fix: patch class in FeatureProxy

Codecov Report

Merging #299 (0244aed) into main (5d8f673) will increase coverage by 0.08%.
The diff coverage is 39.75%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main     #299      +/-   ##
==========================================
+ Coverage   35.75%   35.84%   +0.08%
==========================================
  Files          59       59
  Lines        4984     5164     +180
==========================================
+ Hits         1782     1851      +69
- Misses       3202     3313     +111

Impacted Files
Coverage Δ

siibra/core/concept.py
77.21% <ø> (ø)

siibra/core/region.py
40.08% <0.00%> (-0.25%)
:arrow_down:

siibra/core/space.py
67.50% <ø> (ø)

siibra/features/connectivity/__init__.py
100.00% <ø> (+20.00%)
:arrow_up:

siibra/retrieval/cache.py
58.20% <0.00%> (-4.70%)
:arrow_down:

siibra/configuration/factory.py
22.64% <12.50%> (-3.43%)
:arrow_down:

siibra/features/__init__.py
37.50% <16.66%> (-51.39%)
:arrow_down:

siibra/volumes/nifti.py
18.91% <16.66%> (-0.53%)
:arrow_down:

siibra/volumes/sparsemap.py
15.45% <16.66%> (-0.26%)
:arrow_down:

siibra/volumes/parcellationmap.py
37.13% <25.00%> (+1.59%)
:arrow_up:

... and 38 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Bump reaction & Collect meta tags

Some bug fixes for collection
Collect meta tags

We already looked at it together during pairing so merging.


Scanner crashes (nil deref)

Description
Scanner crashes.
Expected Behavior
Scanner does not crash.
Actual Behavior
time="2024-10-14T16:52:18Z" level=info msg=Running... app=openclarity
time="2024-10-14T16:52:18Z" level=info msg="Initializing configuration..." app=openclarity
time="2024-10-14T16:52:18Z" level=info msg="Using config file (/etc/openclarity/config.yaml):\nexploits:\n  enabled: true\n  input_from_vuln: true\n  inputs:\n  - input: /mnt/snapshot/image.tar\n    input_type: oci-archive\n    strip_path_from_result: true\n  scanners_config:\n    exploit_db:\n      base_url: http://openclarity-exploit-db-server:1326\n  scanners_list:\n  - exploitdb\ninfofinder:\n  enabled: true\n  inputs:\n  - input: /mnt/snapshot/image.tar\n    input_type: oci-archive\n    strip_path_from_result: true\n  scanners_config:\n    ssh_topology: {}\n  scanners_list:\n  - sshTopology\n  strip_input_paths: false\nmalware:\n  enabled: false\n  inputs: ]\n  scanners_config:\n    clam:\n      alternative_freshclam_mirror_url: \"\"\n      clam_daemon_binary_path: \"\"\n      clam_daemon_client_binary_path: \"\"\n      clam_daemon_config_path: \"\"\n      clamscan_binary_path: \"\"\n      clamscan_exclude_dirs: ]\n      clamscan_exclude_files: ]\n      freshclam_binary_path: \"\"\n      freshclam_config_path: \"\"\n      use_native_clamscan: false\n    yara:\n      cache_dir: \"\"\n      compiled_rule_url: \"\"\n      directories_to_scan: ]\n      rule_sources: ]\n      yara_binary_path: \"\"\n      yarac_binary_path: \"\"\n  scanners_list: ]\n  strip_input_paths: false\nmisconfiguration:\n  enabled: true\n  inputs:\n  - input: /mnt/snapshot/image.tar\n    input_type: oci-archive\n    strip_path_from_result: true\n  scanners_config:\n    cisdocker:\n      registry: null\n      timeout: 0\n    lynis:\n      binary_path: \"\"\n  scanners_list:\n  - lynis\n  - cisdocker\n  strip_input_paths: false\nplugins:\n  binary_artifacts_clean: false\n  binary_artifacts_path: \"\"\n  binary_mode: false\n  enabled: false\n  inputs: ]\n  scanners_config: null\n  scanners_list: ]\nrootkits:\n  enabled: false\n  inputs: ]\n  scanners_config:\n    chkrootkit:\n      binary_path: \"\"\n  scanners_list: ]\n  strip_input_paths: false\nsbom:\n  analyzers_config:\n    syft:\n      exclude_paths: ]\n      local_image_scan: false\n      registry: null\n      scope: squashed\n    trivy:\n      cache_dir: \"\"\n      local_image_scan: false\n      registry: null\n      temp_dir: \"\"\n      timeout: 300\n  analyzers_list:\n  - syft\n  - trivy\n  - windows\n  - gomod\n  enabled: true\n  inputs:\n  - input: /mnt/snapshot/image.tar\n    input_type: oci-archive\n    strip_path_from_result: null\n  local_image_scan: false\n  merge_with: ]\n  output_format: cyclonedx-json\n  registry:\n    auths: ]\n    skip-verify-tls: false\n    use-http: false\nsecrets:\n  enabled: true\n  inputs:\n  - input: /mnt/snapshot/image.tar\n    input_type: oci-archive\n    strip_path_from_result: true\n  scanners_config:\n    gitleaks:\n      binary_path: \"\"\n  scanners_list:\n  - gitleaks\n  strip_input_paths: false\nvulnerabilities:\n  enabled: true\n  input_from_sbom: true\n  inputs: ]\n  local_image_scan: false\n  registry: null\n  scanners_config:\n    grype:\n      local_grype_config:\n        db_root_dir: \"\"\n        listing_file_timeout: 0\n        listing_url: \"\"\n        local_image_scan: false\n        max_allowed_built_age: 0\n        registry: null\n        scope: \"\"\n        update_db: false\n        update_timeout: 0\n      mode: REMOTE\n      remote_grype_config:\n        grype_server_address: openclarity-grype-server:9991\n        grype_server_schemes: ]\n        grype_server_timeout:<PHONE_NUMBER>00\n    trivy:\n      cache_dir: \"\"\n      registry: null\n      server_addr: http://openclarity-trivy-server:9992\n      server_token: \"\"\n      temp_dir: \"\"\n      timeout: 300\n  scanners_list:\n  - grype\n  - trivy\n" app=openclarity
time="2024-10-14T16:52:33Z" level=info msg="Running scanners..." app=openclarity
time="2024-10-14T16:52:33Z" level=info msg="Running family \"misconfiguration\" in progress..." app=openclarity family=misconfiguration
time="2024-10-14T16:52:33Z" level=info msg="Scanning inputs in progress..." inputs="[oci-archive:/mnt/snapshot/image.tar]" app=openclarity family=misconfiguration
time="2024-10-14T16:52:33Z" level=info msg="Scan job \"lynis\" for input oci-archive:/mnt/snapshot/image.tar in progress..." scanner=lynis input="oci-archive:/mnt/snapshot/image.tar" app=openclarity family=misconfiguration
time="2024-10-14T16:52:33Z" level=info msg="Scan job \"cisdocker\" for input oci-archive:/mnt/snapshot/image.tar in progress..." app=openclarity family=misconfiguration scanner=cisdocker input="oci-archive:/mnt/snapshot/image.tar"
time="2024-10-14T16:52:33Z" level=warning msg="Scan job \"cisdocker\" for input oci-archive:/mnt/snapshot/image.tar failed" app=openclarity family=misconfiguration scanner=cisdocker input="oci-archive:/mnt/snapshot/image.tar" error="unsupported source type=oci-archive"
time="2024-10-14T16:52:33Z" level=warning msg="Scan job \"lynis\" for input oci-archive:/mnt/snapshot/image.tar failed" error="failed to convert input to filesystem: failed to expand container to rootfs directory: unable to get image: unable to parse image from src oci-archive:/mnt/snapshot/image.tar: unable to detect input for '/mnt/snapshot/image.tar', errs: unable to parse OCI directory as an image: unexpected media type for sha256:beee3efa712609c45c5a5822bc633fbf36cb7917cd43108d4edcad8a9f7c1d97: application/vnd.oci.image.index.v1+json" family=misconfiguration scanner=lynis input="oci-archive:/mnt/snapshot/image.tar" app=openclarity
time="2024-10-14T16:52:33Z" level=error msg="Scanning inputs failed with 1 errors" error="scan job \"cisdocker\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: unsupported source type=oci-archive\nscan job \"lynis\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: failed to convert input to filesystem: failed to expand container to rootfs directory: unable to get image: unable to parse image from src oci-archive:/mnt/snapshot/image.tar: unable to detect input for '/mnt/snapshot/image.tar', errs: unable to parse OCI directory as an image: unexpected media type for sha256:beee3efa712609c45c5a5822bc633fbf36cb7917cd43108d4edcad8a9f7c1d97: application/vnd.oci.image.index.v1+json" app=openclarity family=misconfiguration
time="2024-10-14T16:52:33Z" level=error msg="Family \"misconfiguration\" finished with error" app=openclarity family=misconfiguration error="failed to process inputs for misconfigurations: scan job \"cisdocker\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: unsupported source type=oci-archive\nscan job \"lynis\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: failed to convert input to filesystem: failed to expand container to rootfs directory: unable to get image: unable to parse image from src oci-archive:/mnt/snapshot/image.tar: unable to detect input for '/mnt/snapshot/image.tar', errs: unable to parse OCI directory as an image: unexpected media type for sha256:beee3efa712609c45c5a5822bc633fbf36cb7917cd43108d4edcad8a9f7c1d97: application/vnd.oci.image.index.v1+json"
time="2024-10-14T16:52:33Z" level=info msg="Running family \"sbom\" in progress..." family=sbom app=openclarity
time="2024-10-14T16:52:34Z" level=info msg="Scanning inputs in progress..." family=sbom app=openclarity inputs="[oci-archive:/mnt/snapshot/image.tar]"
time="2024-10-14T16:52:34Z" level=info msg="Generating hash for input /mnt/snapshot/image.tar" family=sbom app=openclarity
time="2024-10-14T16:52:34Z" level=info msg="Skip generating hash in the case of image"
time="2024-10-14T16:52:34Z" level=info msg="Scan job \"syft\" for input oci-archive:/mnt/snapshot/image.tar in progress..." app=openclarity family=sbom scanner=syft input="oci-archive:/mnt/snapshot/image.tar"
time="2024-10-14T16:52:34Z" level=info msg="Scan job \"trivy\" for input oci-archive:/mnt/snapshot/image.tar in progress..." family=sbom input="oci-archive:/mnt/snapshot/image.tar" scanner=trivy app=openclarity
time="2024-10-14T16:52:34Z" level=info msg="Scan job \"windows\" for input oci-archive:/mnt/snapshot/image.tar in progress..." app=openclarity family=sbom input="oci-archive:/mnt/snapshot/image.tar" scanner=windows
time="2024-10-14T16:52:34Z" level=info msg="Scan job \"gomod\" for input oci-archive:/mnt/snapshot/image.tar in progress..." input="oci-archive:/mnt/snapshot/image.tar" app=openclarity family=sbom scanner=gomod
time="2024-10-14T16:52:34Z" level=warning msg="Scan job \"windows\" for input oci-archive:/mnt/snapshot/image.tar failed" family=sbom input="oci-archive:/mnt/snapshot/image.tar" error="skipping analyzing unsupported source type: oci-archive" scanner=windows app=openclarity
time="2024-10-14T16:52:34Z" level=warning msg="Scan job \"gomod\" for input oci-archive:/mnt/snapshot/image.tar failed" input="oci-archive:/mnt/snapshot/image.tar" app=openclarity error="unsupported input type=oci-archive" family=sbom scanner=gomod
time="2024-10-14T16:52:34Z" level=warning msg="Scan job \"syft\" for input oci-archive:/mnt/snapshot/image.tar failed" input="oci-archive:/mnt/snapshot/image.tar" error="failed to create source analyzer=syft: an error occurred attempting to resolve '/mnt/snapshot/image.tar': oci-archive: unable to parse OCI directory as an image: unexpected media type for sha256:beee3efa712609c45c5a5822bc633fbf36cb7917cd43108d4edcad8a9f7c1d97: application/vnd.oci.image.index.v1+json" app=openclarity family=sbom scanner=syft
2024/10/14 16:52:35 INFO Java DB Repository repository=ghcr.io/aquasecurity/trivy-db:2
2024/10/14 16:52:35 INFO Downloading the Java DB...
time="2024-10-14T16:52:35Z" level=warning msg="Scan job \"trivy\" for input oci-archive:/mnt/snapshot/image.tar failed" family=sbom input="oci-archive:/mnt/snapshot/image.tar" error="failed to generate SBOM: image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:bff6683ea42c9216ad9427b6487fd00feadce5219a3a26e2923d563f6c4c865e): post analysis error: post analysis error: Unable to initialize the Java DB: Java DB update failed: DB download error: unacceptable media type: application/vnd.aquasec.trivy.db.layer.v1.tar+gzip" scanner=trivy app=openclarity
time="2024-10-14T16:52:35Z" level=error msg="Scanning inputs failed with 1 errors" error="scan job \"windows\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: skipping analyzing unsupported source type: oci-archive\nscan job \"gomod\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: unsupported input type=oci-archive\nscan job \"syft\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: failed to create source analyzer=syft: an error occurred attempting to resolve '/mnt/snapshot/image.tar': oci-archive: unable to parse OCI directory as an image: unexpected media type for sha256:beee3efa712609c45c5a5822bc633fbf36cb7917cd43108d4edcad8a9f7c1d97: application/vnd.oci.image.index.v1+json\nscan job \"trivy\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: failed to generate SBOM: image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:bff6683ea42c9216ad9427b6487fd00feadce5219a3a26e2923d563f6c4c865e): post analysis error: post analysis error: Unable to initialize the Java DB: Java DB update failed: DB download error: unacceptable media type: application/vnd.aquasec.trivy.db.layer.v1.tar+gzip" family=sbom app=openclarity
time="2024-10-14T16:52:35Z" level=error msg="Family \"sbom\" finished with error" app=openclarity error="failed to process inputs for sbom: scan job \"windows\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: skipping analyzing unsupported source type: oci-archive\nscan job \"gomod\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: unsupported input type=oci-archive\nscan job \"syft\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: failed to create source analyzer=syft: an error occurred attempting to resolve '/mnt/snapshot/image.tar': oci-archive: unable to parse OCI directory as an image: unexpected media type for sha256:beee3efa712609c45c5a5822bc633fbf36cb7917cd43108d4edcad8a9f7c1d97: application/vnd.oci.image.index.v1+json\nscan job \"trivy\" failed for input oci-archive:/mnt/snapshot/image.tar, reason: failed to generate SBOM: image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:bff6683ea42c9216ad9427b6487fd00feadce5219a3a26e2923d563f6c4c865e): post analysis error: post analysis error: Unable to initialize the Java DB: Java DB update failed: DB download error: unacceptable media type: application/vnd.aquasec.trivy.db.layer.v1.tar+gzip" family=sbom
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xe7115d]

goroutine 103 [running]:
github.com/openclarity/openclarity/scanner/families/sbom/types.(*Result).EncodeToBytes(0x0, {0xc00469a980?, 0x0?})
    /build/scanner/families/sbom/types/result.go:45 +0x7d
github.com/openclarity/openclarity/cli/presenter.(*DefaultPresenter).ExportSbomResult(0xc0061a2618, {0xc004984410?, 0xc003ccb080?}, {{0x65c8c31, 0x4}, {0x5bc2a20, 0x0}, {0x7a94000, 0xc0001e9860}})
    /build/cli/presenter/default.go:76 +0x8a
github.com/openclarity/openclarity/cli/presenter.(*DefaultPresenter).ExportFamilyResult(0xc0061a2600?, {0x7aee930?, 0xc00650ff50?}, {{0x65c8c31, 0x4}, {0x5bc2a20, 0x0}, {0x7a94000, 0xc0001e9860}})
    /build/cli/presenter/default.go:47 +0x8f
github.com/openclarity/openclarity/cli/presenter.(*MultiPresenter).ExportFamilyResult(0xc0001e9880?, {0x7aee930, 0xc00650ff50}, {{0x65c8c31, 0x4}, {0x5bc2a20, 0x0}, {0x7a94000, 0xc0001e9860}})
    /build/cli/presenter/multi.go:31 +0xd2
github.com/openclarity/openclarity/cli.(*CLI).FamilyFinished(0x0?, {0x7aee930?, 0xc00650ff50?}, {{0x65c8c31, 0x4}, {0x5bc2a20, 0x0}, {0x7a94000, 0xc0001e9860}})
    /build/cli/cli.go:65 +0x72
github.com/openclarity/openclarity/scanner/internal/family_runner.(*Runner[...]).Run(0x7a92660, {0x7aee968, 0xc005183b30}, {0x7aac308, 0xc0064ee300}, {0x7ad8998, 0xc007413760})
    /build/scanner/internal/family_runner/runner.go:79 +0x79a
github.com/openclarity/openclarity/scanner.New.newFamilyTask[...].func1({{0x7aac308, 0xc0064ee300}, {0x7ad8998, 0xc007413760}, 0xc00506aa80})
    /build/scanner/scanner.go:211 +0x12a
github.com/openclarity/openclarity/workflow/types.Task[...].Run(...)
    /build/workflow/types/task.go:36
github.com/openclarity/openclarity/workflow/internal/dispatcher.(*Dispatcher[...]).Dispatch.func1()
    /build/workflow/internal/dispatcher/dispatcher.go:121 +0xe8
created by github.com/openclarity/openclarity/workflow/internal/dispatcher.(*Dispatcher[...]).Dispatch in goroutine 100
    /build/workflow/internal/dispatcher/dispatcher.go:120 +0x48d
Stream closed EOF for openclarity/openclarity-scan-8ba61151-a678-46c2-88c4-21345e319c4e-kp7tv (openclarity-scan-8ba61151-a678-46c2-88c4-21345e319c4e)

Affected Version
1.0.0
Steps to Reproduce

Initiate a scan

Checklist

[x] I have read the contributing guidelines
[x] I have verified this does not duplicate an existing issue

Hi @jfroy, thanks for reporting! How are you running the scan, as a full stack (which provider), as a CLI, or as a golang import?

Ah sorry I should have specified. I installed the helm chart on a Talos 1.8 baremetal k8s cluster with the k8s provider.


Open the O/R designer missing install instructions

Save people hitting this page a HUGE amount of frustration by adding information from the following links (e.g. "How to Install" or "How to Enable") to the "Open the O/R designer" section:
https://developercommunity.visualstudio.com/content/problem/4616/linq-to-sql-dbml-designer-in-not-working.html
https://stackoverflow.com/questions/42485881/missing-linq-to-sql-classes-dbml-designer-in-visual-studio-2017-rc

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d1d50880-0c44-6bce-f39e-ac4d3d9f7430
Version Independent ID: 41bcd05d-a7d1-28ec-75cc-208ec0b7f1a8
Content: O/R Designer overview - Visual Studio
Content Source: docs/data-tools/linq-to-sql-tools-in-visual-studio2.md
Product: visual-studio-windows
Technology: vs-data-tools
GitHub Login: @gewarren
Microsoft Alias: gewarren

@wittelw Thanks for letting us know! We especially appreciate the links to the Developer Community and Stack Overflow, too. We'll get this page updated.

@TerryGLee Thanks! I've also discovered the designer doesn't seem to work in .Net Core projects, but the generated code from a Framework app can be used in Core projects.
If this is the case and there isn't a better workflow perhaps you can add a note about this too (or the better workflow :).

We will be revamping this documentation and downplaying LINQ to SQL in favor of Entity Frameworks.


Accuracy is not working in the compile method for TensorFlow Keras Sequential

Click to expand!
Issue Type
Bug
Have you reproduced the bug with TF nightly?
No
Source
source
Tensorflow Version
tf v2.12.0-rc1-12-g0db597d0d75 2.12.0
Custom Code
Yes
OS Platform and Distribution
Arch Linux | BUILD_ID: rolling | Linux kernel 6.2.10
Mobile device
No response
Python version
Python 3.10.10
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
cuda 12.1.0-3 | cudnn <IP_ADDRESS>-1
GPU model and memory
GPU: NVIDIA GeForce GTX 1080 | Memory: 8388608 kBytes
Current Behaviour?
I expect the same result if I used the metric with quotes and if I used without quotes. For me, it means there is something wrong at code level, maybe I am wrong.
The word accuracy with quotes, I got these values:
self.__model.compile(
    optimizer=Adam(),
    loss=categorical_crossentropy,
    metrics=['accuracy', Precision()]
)

The word accuracy without quotes, I got these values:
from tensorflow.python.keras.metrics import accuracy, Precision
self.__model.compile(
    optimizer=Adam(),
    loss=categorical_crossentropy,
    metrics=[accuracy, Precision()]
)

Note:
I can't reproduce my example in TF Nightly (v1.12.1-92737-g0ae43a3fd1f 2.13.0-dev20230415) because it throws an error: AttributeError: module 'tensorflow.python.distribute.input_lib' has no attribute 'DistributedDatasetInterface'. Did you mean: 'DistributedDatasetSpec'?
Standalone code to reproduce the issue
As, I show in this Jupyter Notebook. With the same data, if I change the metrics in the compile method with quotes `'accuracy'` or without quotes `accuracy`, it shows different results.

I repeat three times these steps in the Jupyter notebook, to avoid any fluctuation.

Bug-TensorFlow-Keras-Sequential-compile-metrics.ipynb
https://colab.research.google.com/drive/1sle0JaEl-hdRCKeMQCUdY43nNWnYY4mB

Relevant log output
Without quotes:

{'loss': [1808.695068359375,<PHONE_NUMBER>7226562, 54.72672653198242], 'accuracy': [0.5349747538566589, 0.4256841242313385, 0.16613984107971191], 'precision_9': [0.2498059868812561, 0.24983149766921997, 0.2497267723083496], 'val_loss': [443.4054870605469,<PHONE_NUMBER>790039, 11.466686248779297], 'val_accuracy': [0.532325029373169, 0.38062500953674316, 0.001075000036507845], 'val_precision_9': [0.24899999797344208, 0.24692469835281372, 0.25175511837005615]}

{'loss': [1874.009521484375,<PHONE_NUMBER>0058594, 2.5201101303100586], 'accuracy': [0.5312250256538391, 0.35342004895210266, 0.0018018229166045785], 'precision_11': [0.25014790892601013, 0.249727264046669, 0.2505243122577667], 'val_loss': [175.01815795898438, 16.236042022705078, 1.3863223791122437], 'val_accuracy': [0.44843751192092896, 0.018687499687075615, 0.0], 'val_precision_11': [0.24866242706775665, 0.2497398555278778, 0.0]}

{'loss': [1835.9310302734375,<PHONE_NUMBER>7773438, 33.215553283691406], 'accuracy': [0.5334452986717224, 0.38880130648612976, 0.09400103986263275], 'precision_13': [0.25037264823913574, 0.2497129887342453, 0.25012683868408203], 'val_loss': [331.0498046875, 54.01085662841797, 2.3827459812164307], 'val_accuracy': [0.4865500032901764, 0.18593749403953552, 0.0], 'val_precision_13': [0.24895000457763672, 0.2504253089427948, 0.2584269642829895]}

With quotes:

{'loss': [2104.91552734375,<PHONE_NUMBER>064453, 41.532161712646484], 'accuracy': [0.24953334033489227, 0.25047290325164795, 0.2501385509967804], 'precision_10': [0.24953436851501465, 0.2504711151123047, 0.25022196769714355], 'val_loss': [299.38861083984375, 87.65056610107422, 1.386319637298584], 'val_accuracy': [0.24914999306201935, 0.2533999979496002, 0.24940000474452972], 'val_precision_10': [0.24914999306201935, 0.25342532992362976, 0.0]}

{'loss': [1932.6285400390625,<PHONE_NUMBER>9492188, 62.687828063964844], 'accuracy': [0.25043436884880066, 0.24967291951179504, 0.24986353516578674], 'precision_12': [0.25043463706970215, 0.2496640384197235, 0.24984458088874817], 'val_loss': [281.5032653808594, 66.43962860107422, 21.2304744720459], 'val_accuracy': [0.2502000033855438, 0.25029999017715454, 0.25209999084472656], 'val_precision_12': [0.2502000033855438, 0.2503626048564911, 0.25167566537857056]}

{'loss': [1990.3583984375,<PHONE_NUMBER>5253906, 56.19692611694336], 'accuracy': [0.2503645718097687, 0.24988020956516266, 0.24988438189029694], 'precision_14': [0.25036510825157166, 0.2498749941587448, 0.24987071752548218], 'val_loss': [414.74072265625, 72.6060562133789, 28.769975662231445], 'val_accuracy': [0.24705000221729279, 0.25224998593330383, 0.24639999866485596], 'val_precision_14': [0.24705000221729279, 0.2522878348827362, 0.24636809527873993]}

From the documentation of compile for metrics, it explains about the string "accuracy" usage as below.

List of metrics to be evaluated by the model during training and testing. Each of this can be a string (name of a built-in function), function or a tf.keras.metrics.Metric instance. See tf.keras.metrics. Typically you will use metrics=['accuracy']. A function is any callable with the signature result = fn(y_true,y_pred). To specify different metrics for different outputs of a multi-output model, you could also pass a dictionary, such as metrics={'output_a':'accuracy', 'output_b':['accuracy', 'mse']}. You can also pass a list to specify a metric or a list of metrics for each output, such as metrics=[['accuracy'], ['accuracy', 'mse']] or metrics=['accuracy', ['accuracy', 'mse']]. When you pass the strings 'accuracy' or 'acc', we convert this to one of tf.keras.metrics.BinaryAccuracy, tf.keras.metrics.CategoricalAccuracy, tf.keras.metrics.SparseCategoricalAccuracy based on the shapes of the targets and of the model output. We do a similar conversion for the strings 'crossentropy' and 'ce' as well. The metrics passed here are evaluated without sample weighting; if you would like sample weighting to apply, you can specify your metrics via the weighted_metrics argument instead.

Let me know if this explains your confusion about the metric. Thanks!

Yes, it explains the confusion. Furthermore, it makes sense. I'll close this issue. Thank you for all.


Not able to move with the mouse on Windows

I am not able to move around using the mouse on Windows 10. The camera keeps facing downwards.
It can be fixed by uncommenting this line: https://github.com/Game4all/vx_bevy/blob/master/src/voxel/world/player.rs#L23

Sounds like cursor isn't in locked mode. Try pressing escape to toogle cursor locking and enable moving the camera around.


Support RTL styling for Dialog, DropDownMenu, MenuItem

I am not sure if this is by design but setting muiTheme.isRtl to true does not automatically change default styling to render followings right-to-left:

Dialog
DropDownMenu
MenuItem

In this PR, I've modified code to get right-to-left styling by default when muiTheme.isRtl is set to true.

That was pretty insane before this PR:

I have opened https://github.com/callemall/material-ui/issues/6874 following my previous point on the next branch.


installing ccm maven returns Error: undefined method `rebuild'

Hello,
Not sure if this could be of help, but here you go. :)
$  brew install ccm maven
Error: undefined method `rebuild' for #<BottleSpecification:0x0055a3395d2528>
Please report this bug:
  https://github.com/Linuxbrew/linuxbrew/blob/master/share/doc/homebrew/Troubleshooting.md#troubleshooting
/home/user/.linuxbrew/Library/Taps/homebrew/homebrew-core/Formula/python@2.rb:10:in `block in <class:PythonAT2>'
/home/user/.linuxbrew/Library/Homebrew/software_spec.rb:91:in `instance_eval'
/home/user/.linuxbrew/Library/Homebrew/software_spec.rb:91:in `bottle'
/home/user/.linuxbrew/Library/Homebrew/formula.rb:1729:in `bottle'
/home/user/.linuxbrew/Library/Taps/homebrew/homebrew-core/Formula/python@2.rb:9:in `<class:PythonAT2>'
/home/user/.linuxbrew/Library/Taps/homebrew/homebrew-core/Formula/python@2.rb:1:in `load_formula'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:21:in `module_eval'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:21:in `load_formula'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:34:in `load_formula_from_path'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:83:in `load_file'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:74:in `klass'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:70:in `get_formula'
/home/user/.linuxbrew/Library/Homebrew/formulary.rb:207:in `factory'
/home/user/.linuxbrew/Library/Homebrew/dependency.rb:32:in `to_formula'
/home/user/.linuxbrew/Library/Homebrew/dependency.rb:93:in `block in expand'
/home/user/.linuxbrew/Library/Homebrew/dependencies.rb:9:in `each'
/home/user/.linuxbrew/Library/Homebrew/dependencies.rb:9:in `each'
/home/user/.linuxbrew/Library/Homebrew/dependency.rb:80:in `expand'
/home/user/.linuxbrew/Library/Homebrew/formula.rb:1223:in `recursive_dependencies'
/home/user/.linuxbrew/Library/Homebrew/requirement.rb:196:in `expand'
/home/user/.linuxbrew/Library/Homebrew/formula.rb:1229:in `recursive_requirements'
/home/user/.linuxbrew/Library/Homebrew/formula_installer.rb:313:in `expand_requirements'
/home/user/.linuxbrew/Library/Homebrew/formula_installer.rb:263:in `compute_dependencies'
/home/user/.linuxbrew/Library/Homebrew/formula_installer.rb:130:in `verify_deps_exist'
/home/user/.linuxbrew/Library/Homebrew/formula_installer.rb:123:in `prelude'
/home/user/.linuxbrew/Library/Homebrew/cmd/install.rb:231:in `install_formula'
/home/user/.linuxbrew/Library/Homebrew/cmd/install.rb:93:in `block in install'
/home/user/.linuxbrew/Library/Homebrew/cmd/install.rb:93:in `each'
/home/user/.linuxbrew/Library/Homebrew/cmd/install.rb:93:in `install'
/home/user/.linuxbrew/Library/brew.rb:84:in `<main>'  ```

Please enclose copy-and-paste blocks in triple back ticks.
See https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code

I got this error too, it went away after running brew update.


Task: mark external Harvester cluster provisioning support as experimental

Need it in the release note and the doc.
It’s due to lacks of automatically field population #1244

Note: RKE1 node driver is GA

Verified that Experimental label has been added to the Cloud Credentials for External Harvester

.

Verified in Rancher v2.6.3-rc5, although to get latest upstream UI changes I specified ui-offline-preferred to be Remote. Verified using previous steps. Closing as fixed.

Test with Rancher 2.6.3-rc5 with UI set to remote.
Test result: Failed
Reopen the issue.
Verified that Experimental label has been added to the Cloud Credentials for External Harvester
And cloud credential creation.
However, it's not in the RKE1 Node template.


Another pattern matching space leak

Hi there,
I am working on a project and getting the error elm-make: out of memory (requested 1048576 bytes). That is a lot of memory! Here is a stripped down example which causes the issue. I am using Linux and elm 0.18.

OK, so breaking the example up in to nested case expressions solves the issue. I'm sure you know about this issue already, so feel free to close.

I think this is a duplicate of #1362 which will be a lot nicer in 0.19. Sorry you are running into it for now!


Documentation of "Advanced Features" is partly out of date.

Describe the bug
Solving the problem described in https://gist.github.com/freddyalfonsoboulton/b3d7a17b913e928a349e04acb6b0a740
is no problem.
To Reproduce
Getting the standard form does not work with:
rawdata = problem.get_problem_data(cvxpy.GLPK_MI)
But rawdata is still an internal representation. Two further steps a necessary:
glpk = GLPK_MI(); data = glpk.get_problem_data(rawdata['objective'], rawdata['constraints'], {glpk.name(): ProblemData()})
One step later unpacking the solution isn't possible with
prob.unpack_results(cvx.GLPK_MI, solver_output)
Expected behavior
Things should work as described in the documentation.
Version

OS: Windows
CVXPY Version: 1.0.21

Thanks for reporting this. The documentation is indeed out-of-date compared to the current semantics of get_problem_data (the docstring is accurate). I'll look into fixing this.

I've updated the docs (the website should be updated within a couple days). The semantics of get_problem_data were changed in 1.0, and they're not ideal. We're currently working on a major update to CVXPY. We'll try to better semantics for get_problem_data in this update; perhaps we'll just match the pre 1.0 behavior.

Thanks for updating the documentation and I am looking forward to the new version :-)


UseItemStackEvent doesn't work on SpongeForge (but maybe on SpongeVanilla)

I am currently running
SpongeForge version: spongeforge-1.12.2-2611-7.1.0-BETA-2970.jar
Forge version: Forge Mod Loader version <IP_ADDRESS>35
SpongeVanilla version: NO
Java version: 8_186
Operating System: DEBIAN

Plugins/Mods: SpongeForge and DynmapForge, rest are private

Issue Description
    @Listener
    fun onEat2(event: UseItemStackEvent.Finish) {
        println("FINISHING USE ITEM STACK")
    }

    @Listener
    fun onEat3(event: UseItemStackEvent) {
        println("GENERAL USE ITEM STACK EVENT")
    }

registering this Listener and eating something or drinking a potion does nothing. I'd expect it to print something to the console which it doesnt. no reaction what so ever. I remember my code worked in the past. I was using spongevanilla back then. it might be related.

I checked SpongeForge and SPongeCommon and found that SpongeForge has this function
createPlayerUseItemFinishEvent
https://github.com/SpongePowered/SpongeForge/blob/5087d8342d0ed2f6ae5f7dd8941115f29d9b29a1/src/main/java/org/spongepowered/mod/event/SpongeForgeEventFactory.java#L720
but apparently it's never called? I might err though.

see https://github.com/SpongePowered/SpongeForge/issues/1928


[Query][Output Mapping]Output mapping wizard does not close automatically once a value get saved.

Description:
Output mapping wizard does not get closed even after value gets saved to the Output Mapping table. User may feel this operation is hung as this does not get closed.

This is fixed in the latest.


added text_to_speech python program

My hacktoberfest contribution

@srinidh-007 Please merge my pull request or if there is any issue please tell me

Changes done !! Please see into it ....


App crash on add wallet

Build 1.1.6 (248) Dev
https://github.com/bithyve/bitcoin-keeper/assets/107177634/2bf73dcd-227a-4786-b2d1-a266c42914b8

Verified this issue on keeper app v 1.1.6(250)


ins.add_selected_area() Exception: The selected area's data cannot be empty!

I don't know  why, I do it by the method

You must click the selected area before running ins.add_selected_area, did you do that?

Hello, you can refer to this https://github.com/STOmics/Stereopy/issues/220 to see if the steps you performed are correct.
I tried it here, if done correctly there will be no problem


arch-x86: Added softstrobe mode to intel8254 timer

This PR should fix the #1195
Change-Id: I9b7d1483079b5d81e0d37cc5f2533475baff8075

I am not exactly sure on how the timer code was implemented but I tried following what was already done with the differences between the modes (as mentioned in https://wiki.osdev.org/Programmable_Interval_Timer#Mode_4_.E2.80.93_Software_Triggered_Strobe)

I wonder how did you figure out the timer was the problem :D

It gave a panic that this mode was not implemented when kvm failed.


"Could not find a version that satisfies the requirement gdax_recurring (from versions: )"

I just tried installing your script and it seems pip can't find it. Am I doing something wrong?
The error:
Could not find a version that satisfies the requirement gdax_recurring (from versions: )
No matching distribution found for gdax_recurring

Hey @davidpgil! Thanks for the report. This was probably happening because you were using Python >= 3, and this package was only published for Python 2.x.x.
I just published a new version that's compatible with Python 3 as well as Python 2: https://pypi.python.org/pypi/gdax-recurring/
Enjoy!


RuntimeError: Numpy is not available

Describe the bug
RuntimeError: Numpy is not available
To Reproduce
Running on Windows 11 and python 3.10 numpy==1.22.3
Code file -> https://github.com/protectai/llm-guard/blob/main/examples/openai_api.py
Screenshots

Have you installed Numpy?

numpy is already installed version numpy==1.22.3, mentioned the same above earlier, please check
the issue is with internal python files where numpy is being used token_classification.py (used by anonymize.py & evaluate.py of llm-guard)part of transformers library
line 302 logits = model_outputs["logits"][0].numpy()
Please refer to the screenshot attached


Take into account cfs quota and affinity.

In addition to cpusets, cpu affinity can constrain cpus so this adds looking at sched_getaffinity in addition to effective cpus in the cgroup.
For the recommended threads, we additionally take into account cfs quota doing similar quota->cpu math as systemd and lxcfs.

there was a small bug with cpuset.cpus.effective parsin

Can you make this one a separate commit, and add a test that exercises the bug?

there was a small bug with cpuset.cpus.effective parsin

Can you make this one a separate commit, and add a test that exercises the bug?
edit: also, looks like these all need signed-off-by.

It is a separate commit, do you mean separate PR? The new test cases do exercise the bug

It is a separate commit, do you mean separate PR?

Ideally the bug fix would live in a commit on its own, separate from the tests for the new feature.

signed commits

Ok I re-worked the git commits and switched out effective cpus for sched_getaffinity. PTAL

Looks great, thank you!


HeroState._setChild exception in scheduler callback after popping route

Run Gallery and go to Pesto
Scroll the recipe list
Select Home in the menu
Tap the back button

This produces the exception below.
══╡ EXCEPTION CAUGHT BY SCHEDULER LIBRARY ╞═════════════════════════════════════════════════════════
The following assertion was thrown during a scheduler callback:
'package:flutter/src/widgets/heroes.dart': Failed assertion: line 193: 'mounted' is not true.
When the exception was thrown, this was the stack:
#2      HeroState._setChild (package:flutter/src/widgets/heroes.dart:193)
#3      HeroParty._handleUpdate (package:flutter/src/widgets/heroes.dart:393)
#4      Animation&AnimationLazyListenerMixin&AnimationLocalStatusListenersMixin.notifyStatusListeners (package:flutter/src/animation/listener_helpers.dart:118)
#5      ReverseAnimation._statusChangeHandler (package:flutter/src/animation/animations.dart:276)
#6      Animation&AnimationLazyListenerMixin&AnimationLocalListenersMixin&AnimationLocalStatusListenersMixin.notifyStatusListeners (package:flutter/src/animation/listener_helpers.dart:118)
#7      Animation&AnimationEagerListenerMixin&AnimationLocalListenersMixin&AnimationLocalStatusListenersMixin.notifyStatusListeners (package:flutter/src/animation/listener_helpers.dart:118)
#8      AnimationController._checkStatusChanged (package:flutter/src/animation/animation_controller.dart:240)
#9      AnimationController._tick (package:flutter/src/animation/animation_controller.dart:251)
#10     Ticker._tick (package:flutter/src/scheduler/ticker.dart:77)
#11     BindingBase&SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:488)
#12     BindingBase&SchedulerBinding._invokeTransientFrameCallbacks. (package:flutter/src/scheduler/binding.dart:471)
#13     _HashVMBase&MapMixin&&_LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:345)
#14     BindingBase&SchedulerBinding._invokeTransientFrameCallbacks (package:flutter/src/scheduler/binding.dart:469)
#15     BindingBase&SchedulerBinding.handleBeginFrame (package:flutter/src/scheduler/binding.dart:445)
#16     _beginFrame (file:///ssd2/sky/engine/src/out/android_debug_unopt/gen/sky/bindings/dart_ui/hooks.dart:48)
(elided 2 frames from class _AssertionError)
This exception was thrown in the context of a scheduler callback. When the scheduler callback was
registered (as opposed to when the exception was thrown), this was the stack:
#0      _FrameCallbackEntry._FrameCallbackEntry. (package:flutter/src/scheduler/binding.dart:76)
#2      _FrameCallbackEntry._FrameCallbackEntry (package:flutter/src/scheduler/binding.dart:57)
#3      BindingBase&SchedulerBinding.addFrameCallback (package:flutter/src/scheduler/binding.dart:250)
#4      BindingBase&SchedulerBinding.scheduleFrameCallback (package:flutter/src/scheduler/binding.dart:219)
#5      Ticker._scheduleTick (package:flutter/src/scheduler/ticker.dart:87)
#6      Ticker.start (package:flutter/src/scheduler/ticker.dart:42)
#7      AnimationController._startSimulation (package:flutter/src/animation/animation_controller.dart:217)
#8      AnimationController.animateTo (package:flutter/src/animation/animation_controller.dart:182)
#9      AnimationController.reverse (package:flutter/src/animation/animation_controller.dart:162)
#10     TransitionRoute.didPop (package:flutter/src/widgets/routes.dart:185)
#11     TransitionRoute&&LocalHistoryRoute.didPop (package:flutter/src/widgets/routes.dart:330)
#12     NavigatorState.pop (package:flutter/src/widgets/navigator.dart:442)
#13     _WidgetsAppState.didPopRoute (package:flutter/src/widgets/app.dart:134)
#14     BindingBase&SchedulerBinding&GestureBinding&ServicesBinding&RendererBinding&WidgetsBinding.handlePopRoute (package:flutter/src/widgets/binding.dart:163)
#15     _popRoute (file:///ssd2/sky/engine/src/out/android_debug_unopt/gen/sky/bindings/dart_ui/hooks.dart:38)
(elided one frame from class _AssertionError)

@Hixie
Is something supposed to coordinate ordering between the hero's animation status listener and the disposal of the HeroState?

Looks like @jason-simmons is on this.


Show method name in the failed 'return' check

Right now we use a generic text Detected an attempt to return null from a method marked by @NonNull. It's worth to indicate target method as well because we experienced situations when the exception is caught and stored for further processing. It's just more convenient to see the whole information immediately instead of analyzing stack-trace.

Done


provided link to longer discussion with Amos Yee

Link specifically starts at the moment that Amos asks "What about child pornography?" I don't know if the words I chose to act as the hyperlink are the best choice but I thought the link might need to be there for better context.

Sorry for the double pull request


BBMM fits seems not quite right

For the attached dataset, the createContour() fits variable gets set to FALSE currently but in the pre-refactor version (currently available in the gateway), fits is TRUE for that dataset.
This may be related to the previous fits issue you fixed?
condor.txt

This is likely due to the tolerance that was set up, it may need to be adjusted. I will look into if this would work or if it this functionality requires a different kind of check now

@matthewwangg, the problem shows up when cell size = 1000 and buffer = 10000 and for some other values but not for cell size = 100 and buffer = 1000 (actually this should be fits = FALSE but looks like it is TRUE?)

I fixed this issue, closing now because for every test case I have access to, the plot warning message matches up for the existing gateway version and the refactordev branch version


[Feature Request] Test driver test-common.rs needs to build dependences (stdlib files).

In order to use a module that is not textually part of a .move test file, we need the test driver to build those dependences to bytecode just like the test file itself.
The driver needs to invoke move-build with -d. Say we have copies of a stdlib files in a designated directory, then test-common needs to invoke below on the test file (assuming, in this case, it depdends on option which itself depends on vector).
move-build --flavor none -m my_test.move -d std-vector.move -d std-option.move
And then each of std-vector.move and std-option.move must be built separately be compiled by move-build.
So we were uninformed in #186-- we don't need a bespoke include mechanism. The use directive is that mechanism-- the test driver just needs to build those dependences. The same will be true when we have a pre-built move-stdlib.

I've changed this to a bug rather than enhancement. It is certainly a defect that the test driver doesn't build a module's dependences.


Translations

second commit only adds the strings to simplify updating the translations later

ack @ 28d1831
regarding the missing translations I would only add when we really have a translation - not duplicating the en version as this is used as fallback anyway

I added the English versions to make it easier for the people doing the
translations to identify the missing strings. I planned to @ them in an
issue after this is merged.
On Sep 11, 2016 4:58 PM, "ligi"<EMAIL_ADDRESS>wrote:

ack @ 28d1831
https://github.com/criticalmaps/criticalmaps-android/commit/28d1831c5a39b18cac41ae986787f7e5f8324825
regarding the missing translations I would only add when we really have a
translation - not duplicating the en version as this is used as fallback
anyway
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/criticalmaps/criticalmaps-android/pull/178#issuecomment-246184626,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AI1mmxJM5rzRgpGtsfz58a-X-coUnPUMks5qpBcigaJpZM4J5-3G
.

I would suggest we use a translation management service like transifex
for this.

I know and I think we already planned to set it up but no one ever did ;)
On Sep 11, 2016 6:09 PM, "ligi"<EMAIL_ADDRESS>wrote:

I would suggest we use a translation management service like transifex
for this.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/criticalmaps/criticalmaps-android/pull/178#issuecomment-246188267,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AI1mm-CXyn5Dq5YwgShaJBpG0emCWOmnks5qpCfFgaJpZM4J5-3G
.

I ( think *1 ) I can set it up if wanted @stephanlindauer - would really prefer the placeholders to be there than rather in the xml's
*1 - have to check if this is possible as contributor to a project or if you have to be owner

@cbalster i agree with ligi to not add english strings to languages where we don't have a translation for yet.
in case it's missing it will fall back to english anyway and in android studio its easier to spot missing translation if we don't add them.
however +1 on the first commit.
@ligi i tried transifex but i wasn't properly convinced due to the fact the there was still some manual labour involved to pull in the strings. also i think its nice to have everything in one place (github) without external dependencies. but i don't have such a strong opinion in this. if you want to set it up, juts go ahead. i will supply you with the necessary rights.

@stephanlindauer there is a tool called tx - so usually this is the workflow then:

tx pull
git commit -a -m "Update translations"

@ligi https://www.transifex.com/criticalmaps/critical-maps/

let's maybe take this discussion to https://github.com/criticalmaps/criticalmaps-android/issues/180
regarding this pr: i would merge if the second commit would be reverted.

second commit removed and rebased on top of develop


PPAify srcf-python

My understanding is that current SRCF policy encourages installing Python dependencies from Ubuntu packages. In which case, could this be distributed via PPA please?

See https://github.com/SRCF/control-panel/pull/4

This has now been done -- python3-srcf is available from the srcf/serveradmin PPA.