feat: add flatt-security/shisho

flatt-security/shisho: Lightweight static analyzer for several programming languages
$ aqua g -i flatt-security/shisho

How to confirm if this package works well
Reviewers aren't necessarily familiar with this package, so please describe how to confirm if this package works well.
Please confirm if this package works well yourself as much as possible.
Command and output
$ shisho --help
shisho 0.5.2

USAGE:
    shisho <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    check
    completion    Prints scripts for shell completion
    find          Checks files with a pattern given in command line arguments
    help          Prints this message or the help of the given subcommand(s)

Thank you!

v4.102.0 is out 🎉
https://github.com/aquaproj/aqua-registry/releases/tag/v4.102.0


Update bad-cards.ts

Added 'Enron scandal'

Can you please provide an explanation of why you think it should be removed?

Enron Corporation was founded in 1985; The Enron scandal happened in 2001.
Either change the card to 'Enron' or fix the year to 2001.
On Sun, Feb 6, 2022 at 12:10 PM Tom Watson @.***> wrote:

Can you please provide an explanation of why you think it should be
removed?
—
Reply to this email directly, view it on GitHub
https://github.com/tom-james-watson/wikitrivia/pull/51#issuecomment-1030800620,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC2M23TPBYIIZ6NVUEVTYODUZZCJ3ANCNFSM5NPGQ24Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>


⚠ ClamAV has degraded performance

In 60e5542, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 611 ms

Resolved: ClamAV performance has improved in b7c9eeb.


Add operator MergeGapsBetweenActivityOperator

Related to Interns notebook

I ran through the interns code, and tested some cases. I understood how the merging algorithm works in details, but in some cases, I'm not sure whether the sleep should be merged or not.
I am discussing this with Joao very soon

Ready to be reviewd


Upgrade flutter to 3.10

This was a much more boring upgrade than I thought :D
It needs testing on iOS though.
Here's a changelog:
https://docs.flutter.dev/release/whats-new#10-may-2023-google-io-2023-310-release

Tested on ios simulator (everything but QRCode)

@michael-markl did you test protobuf with this version?
I can not run "build config" and for me it looks like the protoc plugin does only support dart sdk until >3.0.0

Update: okay had to delete .dart_tool folder

Interesting, I didn't have to do that 🤔  CI seems happy as well... Not sure what the actual issue was for you 😕


fix:class 'ResultSetUtil.java' parse datetime type error #240

I had put this fix PR as hacktoberfest, check https://bit.ly/hack-nebula on how to participate :)

Codecov Report
Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (d6b73ea) 0.00% compared to head (d10eaa4) 0.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@          Coverage Diff           @@
##           master    #241   +/-   ##
======================================
  Coverage    0.00%   0.00%
======================================
  Files          74      74
  Lines        2532    2533    +1
  Branches      275     275
======================================
- Misses       2532    2533    +1

Files
Coverage Δ

...rg/nebula/contrib/ngbatis/utils/ResultSetUtil.java
0.00% <0.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Dropdown caret not displaying for dropdown menus in navbar in iOS Chrome

The dropdown caret in the responsive navbar is not displaying on iOS Chrome. I can even reproduce it in the docs. When you click where the caret is supposed to be, it magically shows up.
I have made a JSBin demonstrating the problem: http://jsbin.com/vemavamexa

Your example is missing a viewport meta tag; see https://github.com/twbs/bootlint/wiki/W003
What version of iOS and Chrome are you using?
Can you post a screenshot?
Does this happen on iOS Safari too? I can't repro on iOS 9 Safari:

iOS 9, Chrome 45.0.2454.89
Cant reproduce it on Safari either.
Seems that it doesn't happen anymore. which is weird

@wolfy1339 Sorry, just to clarify, are you still experiencing this bug in iOS Chrome? If yes, could you please post a screenshot?

I am experiencing a similar bug whilst using IOS 9.0.2. Well, based on not much of a description of the bug that is. The caret is there,  however when I am in portrait orientation it disappears until I interact with a dropdown, then it reappears, and when I lose focus it disappears.
If I change the border property from dashed to solid the issue is no more. Why is the border dashed in the first place?

Why is the border dashed in the first place?

Because Firefox sizes solid ones differently for some odd reason: #15697

Thanks for clearing that up. So now I think why not use a glyph instead?

I believe there was a desire to avoid a hard dependency on Glyphicons for something so simple.
And looking ahead, Bootstrap v4 drops Glyphicons completely.

In that case it was a good choice. I think @GlobalGISteam had a point. The change from solid to dashed was global, yet the issue only occurred in Firefox.

I encountered this problem too; it happens only when not using the viewport meta tag and the page is at full zoom. When you zoom in the caret appears again. It could be a bug in Mobile Safari too.

@media (max-width: 767px){
.caret{
border-top: 4px solid;
}
}

I was having pretty much the same problem with the carets not showing up only on Safari and Apple devices but after I used Raghuveersingh's solution and it corrected the issue.


colima: 0.6.10 -> 0.7.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/project/colima/versions.
meta.description for colima is: Container runtimes with minimal setup
meta.homepage for colima is: https://github.com/abiosoft/colima
Updates performed

Golang update

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.7.0 with grep in /nix/store/6zk9sfwmphxifdd6icnb4ca01ylg7bwl-colima-0.7.0
found 0.7.0 in filename of file in /nix/store/6zk9sfwmphxifdd6icnb4ca01ylg7bwl-colima-0.7.0

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

colima

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/6zk9sfwmphxifdd6icnb4ca01ylg7bwl-colima-0.7.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A colima https://github.com/r-ryantm/nixpkgs/archive/b8c7a12443e9d91beff0ba2fdd742065f35826fb.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/b8c7a12443e9d91beff0ba2fdd742065f35826fb#colima

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/6zk9sfwmphxifdd6icnb4ca01ylg7bwl-colima-0.7.0
ls -la /nix/store/6zk9sfwmphxifdd6icnb4ca01ylg7bwl-colima-0.7.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.


[SE-0320] Rename CodingKeyConvertible

The text contains two mentions of CodingKeyConvertible (probably the old name?) that should be CodingKeyRepresentable to be consistent.
cc @tomerd @mortenbekditlevsen

Thank you, Ole!


Add Generalized Additive Linear Model

The package needs an example of adding models and GAM would be a natural next step.  See here and here for implementation examples.

Moved existing work to https://github.com/WardBrian/scikit-stan/tree/feature/5-gamm


Refactor to align with updated style guide

https://github.com/softlayer/ember-style-guide#defining-functions-to-be-called-when-events-are-triggered

willClearRender() - https://github.com/softlayer/sl-ember-components/blob/v0.11.0/addon/components/sl-radio-group.js#L150

Taking this


⚠ Bofrost.fr has degraded performance

In 74554b8, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1352 ms

Resolved: Bofrost.fr performance has improved in c78d3c8.


Bug in match_dense:102 assign_keypoints()

In assign_keypoints, if there is no keypoints ie dark indoor scene ceiling the KDTree will crash. Adding the commented out lines fixes it
if not update:
        #if len(other_cpts) == 0 or len(kpts) == 0:
         #   return np.array([-1] * len(kpts))
        dist, kpt_ids = KDTree(np.array(other_cpts)).query(kpts)
        valid = (dist <= max_error)
        kpt_ids[~valid] = -1
        return kpt_ids

Good catch, can you please send a PR? Thanks!

#307


security: update axios and yargs-parser

❯ npm audit
# npm audit report

axios  <0.21.1
Severity: high
Server-Side Request Forgery - https://npmjs.com/advisories/1594
No fix available
node_modules/axios
  @mobileposse/auto-delete-bucket  *
  Depends on vulnerable versions of axios
  node_modules/@mobileposse/auto-delete-bucket

yargs-parser  <=13.1.1 || 14.0.0 - 15.0.0 || 16.0.0 - 18.1.1
Prototype Pollution - https://npmjs.com/advisories/1500
No fix available
node_modules/yargs-parser
  meow  5.0.0 - 6.0.1
  Depends on vulnerable versions of yargs-parser
  node_modules/meow
    ts-clean  *
    Depends on vulnerable versions of meow
    node_modules/ts-clean

5 vulnerabilities (3 low, 2 high)

Some issues need review, and may require choosing
a different dependency.

our team would also be grateful for this upgrade.

I think Axios has upgraded. Waiting for a new release.

Hey @rwpswami @danconner-vertex ,
The option to auto delete objects upon bucket removal was added to the core functionality of aws-cdk recently. aws-cdk#3297 🥳
You just have to enable the autoDeleteObjects option and set removal policy to RemovalPolicy.DESTROY when creating the S3 bucket.
const bucket = new Bucket(this, 'MyTempFileBucket', {
  removalPolicy: RemovalPolicy.DESTROY,
  autoDeleteObjects: true,
});

See the official AWS CDK documentation


Yara 4.2.3 Hangs and segfaults when used with --rescursive or --scan-list on Alpine 3.17.2

$ docker run --rm -it -name yaratest alpine:3.17.2

# apk add yara --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing/

# yara --version
4.2.3

# ldd `which yara`
        /lib/ld-musl-x86_64.so.1 (0x7f5c85436000)
        libyara.so.9 => /usr/lib/libyara.so.9 (0x7f5c853d8000)
        libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f5c85436000)
        libcrypto.so.3 => /lib/libcrypto.so.3 (0x7f5c85020000)
        libmagic.so.1 => /usr/lib/libmagic.so.1 (0x7f5c84ffe000)

# tree
.
├── files
│   ├── file1.txt
│   └── file2.txt
├── list.txt
└── rule.txt

1 directory, 4 files

# cat rule.txt
rule AsciiExample {
strings:
        $ascii_string = "test"
condition:
        $ascii_string
}

# cat files/file1.txt
this is a test

# cat files/file2.txt
This file
is also a
test

# cat list.txt
files/file1.txt
files/file2.txt

# for i in `ls files`; do yara rule.txt files/$i; done
AsciiExample files/file1.txt
AsciiExample files/file2.txt

# yara --scan-list rule.txt list.txt
AsciiExample files/file1.txt
AsciiExample files/file2.txt
    # Hangs ^C

# yara --recursive rule.txt `pwd`/files
AsciiExample /yaratest/files/file1.txt
AsciiExample /yaratest/files/file2.txt
Segmentation fault

# yara --timeout=5 --scan-list rule.txt list.txt
AsciiExample files/file2.txt
AsciiExample files/file1.txt
    # Hangs. Does not timeout. ^C

$ system_profiler SPSoftwareDataType SPHardwareDataType
Software:

    System Software Overview:

      System Version: macOS 13.2.1 (22D68)
      Kernel Version: Darwin 22.3.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 21 hours, 50 minutes

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro16,1
      Processor Name: 8-Core Intel Core i9
      Processor Speed: 2.4 GHz
      Number of Processors: 1
      Total Number of Cores: 8
      L2 Cache (per Core): 256 KB
      L3 Cache: 16 MB
      Hyper-Threading Technology: Enabled
      Memory: 32 GB
      System Firmware Version: 19<IP_ADDRESS>.0 (iBridge: 20.16.3045.0.0,0)
      OS Loader Version: 564.40.4~66
      Activation Lock Status: Disabled

Package link: https://git.alpinelinux.org/aports/tree/testing/yara/APKBUILD
Doesn’t seem to be consistent between hangs and segfaults, but does one of the two.
Moving from Alpine (musl) to Fedora (glibc) appears to resolve the issue.

I'm seeing this too. I have installed Yara 4.4.0 rc1 and it seems to still be there.


Ruby 2.5.1 image, please?

Ruby has released a security patch, it would be really great if it would be available on
https://hub.docker.com/r/circleci/ruby/tags/
Any chance we can speed this up a bit? (i'm a bit nervous about security, sorry)

Ruby 2.4.4, 2.3.7 and 2.2.10 has also been released with the same security fixes.

#need

Thanks to @felicianotech for a fix. This is a second time I've raised issue of images updated, first with support and now here. Really happy that this is finally is resolved once and for all.


Complete: Swaroop: Task 745-749: Opus Ted Talks & Health Fact

@swarooprm Hi, I have created a pull request, request you to check and provide suggestions for the same.
Regards,
Vansh Patel
vpatel44

@shailaja183, wondering if you can help in reviewing these tasks?

Looks good mostly, minor fixes;
task745/747:

No need of mentioning "can be used by you to translate the speech for a Croatian delegate." (745)
include hyphen inside special character list (745/747)
Remove "to their equivalent in your language." -- "to their equivalent in XYZ language" (745)
Instead of "Please do the same in your translations.", say "Input is case-sensitive .... and output is expected to case sensitive ..." (745/747)
In negative examples, you include 'punctuations' in your explanation but you have not mentioned any guideline related to punctuations in definition (745/747)

task746:

"we have a paragraph and a claim" -- change to "you will be given a paragraph and a claim"
no need of -- The claim starts with Claim: and paragraph starts with Paragraph.
"We have four categories of output," --  change to "the task is to classify the claim in x,y,z,w labels based on the given paragraph"
typos in definition - a multiple, evidences

task748:

"need to find a claim from within the given paragraph" -- change "we need to generate a claim that is supported by the given paragraph".
Rewrite "Nothing will be outside the paragraph." -- "The claim must contain information ..."
Rewrite "A sentence within the paragraph can be a fact." -- "A sentence within the paragraph can be used as a claim"
Use "claim" consistently (do not use fact)
If you use sentences from the passage in explanation, include them in '' (do not copy too many sentences, just one or two which are most important)

task749:

"we have a paragraph and a claim" -- change to "you will be given a paragraph and a claim"
no need of -- The claim starts with Claim: and paragraph starts with Paragraph.
following points while providing an output -- following points while "generating" an output
"We need to generate" -- "generate"
bring point 1) after "you will be given a paragraph and a claim" (that explains what task is to be done and then explain what things should be taken care of when generating explanation)
typo - "emphasised"

@shailaja183, @swarooprm  Hi, I have corrected all the issues.
Regards,
Vansh Patel

@swarooprm check the categories of 748 and 749. 749 seems like Long text generation to me.
Rest of the tasks look good to me!
@vansh-patel-asu based on swaroop's response, please commit changes one final time. Also couple more fixes;
745/747 - remove 'TED talks', 'hyphens or'
748/749 - add a line in the definition suggesting -- how long the claim/explanation should be

@shailaja183 yes I'll commit the changes after Swaroop mentioned the changes along with yours.

@swarooprm waiting for your opinion --
check the categories of 748 and 749. 749 seems like Long text generation to me.

@vansh-patel-asu, category of 748-sentence generation looks good. 749 should be long text generation.

@vansh-patel-asu, category of 748-sentence generation looks good. 749 should be long text generation.

@vansh-patel-asu, can you address comments so that I can evaluate your tasks?

Hi @swarooprm, all corrections addressed

Your HW2 is complete now.
Would you like to submit the same tasks to our big project repo https://github.com/allenai/natural-instructions-expansion/pulls? You just need to fork that repo, rename tasks starting from the latest number in the repo and create PR there. This way, you can be part of our research on "learning from instructions instead of learning from examples (conventional machine learning)". This is purely optional and your HW2 grade does not depend on this @vansh-patel-asu

@vansh-patel-asu  Just a reminder that, if you plan to submit these tasks to the main Github repo and join our research project, Monday (27th Sept) is the deadline. Again, this is optional and independent of your HW2 grade.


robot testing of games

@boomsbloom this isn't priority, but when we have some games established let's talk about if there is a way to automatically test them. I built an experiment robot to go through the jspsych experiments, and this is pretty easy to do because you can extract the acceptable responses (keyboard, click, etc) from the jspsych object. Success for the test means making it to the end of the experiment without any javascript errors, 404, etc. Games are likely different, but there might be a way.

To start, I'll just validate the config.json files for the continuous integration.

this would definitely be a great feature - would be really excellent for
our screener as well and i'd love to hear more about your robot
On Fri, Apr 15, 2016 at 10:59 AM, Vanessa Sochat<EMAIL_ADDRESS>wrote:

To start, I'll just validate the config.json files for the continuous
integration.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/expfactory/expfactory-games/issues/6#issuecomment-210565765

If you cd into an experiment folder, you can just type
  expfactory --test

and the robot will run through it, and you can watch. It's the same robot that runs on circle, for example, if you poke around circle:
https://circleci.com/gh/expfactory/expfactory-experiments
The continuous integration typically has a few tests. First to validate the config.json files, again can be done on command line in a folder with:
  expfactory --validate

then to check that the circle.yml includes all experiments/surveys etc that should be tested, and then it clones the current master, compares to the PR being done, and tests the experiments that were changed using selenium. It's pretty cool, definitely useful to find errors you don't anticipate.

This should be a fairly straightforward port from what you have written for jspsych since I think the only difference will be taking a click from each sprite rather than from the jspsych object, right?

Right - if you just tell me the object in the console to look at to get the current state (and calculate what to click / do next) I can probably get a robot up and running for it!


Conda Forge

Hello! I have successfully put this package on to Conda Forge, and I am extending the invitation for the owner/maintainers of this package to be maintainers on Conda Forge as well. Let me know if you are interested! Thanks.
https://github.com/conda-forge/nannyml-feedstock

Hey Matt, thanks for the reminder. I've created the appropriate issue on the feedstock repository following the guidelines. Happily awaiting your merge.
Not 100% sure when I'll have to intervene manually in the process, I'll give the docs another proper reading.

@nnansters Thanks! I basically whenever an update is made upstream in PyPI, the bots will make a PR to update in conda as well. If all tests pass when we can just merge the PR. Else, sometimes we do need to modify the YAML recipe, usually to update the dependencies etc., but should all be pretty straightforward.
Thanks again!


feat: add some gov websites

What kind of change does this PR introduce?

[ ] Bug Fix
[ ] Documentation change
[x] Added a new website
[ ] Other(s)

Why was this change needed?

Added website of Department of Food Technology and Quality Control
Added website of Health Insurance Board
Added website of Kanti Children's Hospital
Added website of Medical Education Commission
Added website of National Archives of Nepal
Added website of Provincial and Local Governance Support Program

Please resolve all the merge conflicts @susanhsrestha


The alpha angle only works with a magic multiplier

https://github.com/tomayac/joy-con-webhid/blob/6ebaccf4dfd58cf6b1aae9fb2a706fead665ef72/src/parse.js#L118-L128

Just curious, is this the reason why alpha gets registered  but also slowly counts back down to 0?
(also the reason why the blue JoyCon in the demo turns back up to stand straight'

This sounds more like a drift in your Joy-Cons. Maybe try recalibrating and a firmware update as per the other issue.


Feature request: Router restart, reconnect and firmware version

Thanks for great job! This was in my wishlist and I'm very happy to see Home Assistant integration (based on this library).
As you already did most of the work, please think to implement router restart (maybe reconnect and some basic information as firmware version).
Most of the users has dynamic IP, so restart/reconnect  very useful then user want to change external IP.
Firmware version useful because sometimes Vodafone update it and if user monitor it and start getting the problems at least he can know the possible source :-)

HI @myhomeiot,
I'm for sure going to add more sensors.
I started with device tracker because the old custom integration was providing such functionality and for new HA official integration you need to start with only 1 domain at a time.
Which model are you using ? I would like to keep track of all working in the readme (you may be so kind to provide a PR to update the file).
Simone

Excellent! I will make tests with HA 2023.9 and lets you know.

Features added to v0.0.9 of the library.
Please test and report back.
Simone

@chemelli74 Tested, working with Vodafone Sercomm H300S


Work around miscompilation of entry point function on Windows with Swift 5.10 (Take 2).

I saw our Windows
CI fail even after the changes in #381. Trying a different approach here by using consuming to transfer ownership of the troublesome value rather than using a pointer to trick the ownership checker.
Checklist:

[x] Code and documentation should follow the style of the Style Guide.
[x] If public symbols are renamed or modified, DocC references should be updated.

@swift-ci please test

@swift-ci please test Linux


Add scheme setup to devcontainer

This adds Scheme dependencies to #871.

I can't get this to run euclidalg.ss successfully.
@leios do you know if this ever ran? Did you use http://www.gnu.org/software/mit-scheme/ instead when testing #582

Turns out scheme is like 10+  different languages definitions all adhering to the same spec (more or less).
Read more here. http://community.schemewiki.org/?your-first-implementation
@leios I'm not sure which to pick between Gauche and Chicken. The implementation I chose previously may be incomplete. Not really sure as I can't get it to run the euclidean algorithm.

Racket can run Scheme code as well, but I can't get the euclidalg.ss to run.
See here
This is confusing :sweat_smile:
Maybe this needs several scheme interpreters?

Ok I can make this run using racket like this :  racket -f euclidalg.ss

sorry for the late follow-up on this, looking into it now. If it works, let's go ahead and update the branch to the latest and remove the scheme section. Note in the racket section the command you used to run scheme so @stormofice  doesn't have to figure it out for AAA-Verifer and @Amaras doesn't need to look it up for SCONs.

Can confirm racket -f <file.ss> will run. Make the quick fix to the docker file merging the scheme and racket lines and then it will be good for me

Solved the merge issue through the Github interface, I hope I didn't break anything :joy:


Assignment 9 strings

@winslowt It looks like you opened a pull request to Bloc's main repo instead of your fork of the repo.

Okay, thanks for letting me know. @falkobuttler--do you know how I did that?

@winslowt You can select on the screen when you open a pull request:

"Base fork" is where you want the changes to go. If you select your own repo, the option to select other forks will disappear (this is what you want). So if you're doing it correctly, it will look like this:

Thanks, Aaron!
Sent from my iPhone

On Oct 13, 2015, at 4:03 PM, Aaron Brager<EMAIL_ADDRESS>wrote:
@winslowt You can select on the screen when you open a pull request:
"Base fork" is where you want the changes to go. If you select your own repo, the option to select other forks will disappear (this is what you want). So if you're doing it correctly, it will look like this:
—
Reply to this email directly or view it on GitHub.


Rename asXXX methods in BaseRequest

From #168
Hi there,
Please could you rename asXXX methods in BaseRequest related to send request? For example, asString()
It's confusing, looking like returning request as string, instead of response as string as having a look method code.
Thanks

The BaseRequest asXXX is intended to be the termination method call of the API and it represents the response, not the request.


Error running the standalone version of songbird

Hi @mortonjt ,
I hope all is well!
I am having trouble running the standalone version of Songbird. Please see below my code and the error message I am getting.
Would love to hear your thoughts on what could be going wrong here.
Thank you so much!!
source /PATH/miniconda/bin/activate
conda activate tf

songbird multinomial\
    --input-biom feature-table.biom\
    --metadata-file POSTPARTUM_mapping_sample_key.txt\
    --formula "OBS+Class+TRT+Cow_ID+Site+Cycling+P4" \
    --epochs 10000 \
    --differential-prior 0.5 \
    --summary-interval 1 \
    --summary-dir results

And I am getting the following error:
Traceback (most recent call last):
  File "/PATH/miniconda/envs/tf/bin/songbird", line 225, in <module>
    songbird()
  File "PATH/miniconda/envs/tf/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "PATH/miniconda/envs/tf/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "PATH/miniconda/envs/tf/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "PATH/miniconda/envs/tf/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "PATH/miniconda/envs/tf/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/PATH/miniconda/envs/tf/bin/songbird", line 154, in multinomial
    dense_table = table.to_dataframe().to_dense().T
  File "/PATH/miniconda/envs/tf/lib/python3.6/site-packages/biom/table.py", line 3839, in to_dataframe
    mat = [pd.SparseSeries(v.toarray().ravel()) for v in self.matrix_data]
  File "PATH/miniconda/envs/tf/lib/python3.6/site-packages/biom/table.py", line 3839, in <listcomp>
    mat = [pd.SparseSeries(v.toarray().ravel()) for v in self.matrix_data]
TypeError: object() takes no parameters

Hi, thanks for raising this issue.  I think this was linked to the pandas 1.0 release that broke sparse matrix operations in biom (when Songbird was first developed in 2019).  You can confirm this by providing your environment (aka provide the contents of conda env export).
If this is the case, the fastest solution is to downgrade pandas < 1.0 via conda install pandas<1

Thanks @mortonjt!!
I tried downgrading pandas to version condo install pandas==0.25 but got tons of conflicts.
Please see below the contents for conda env export.
(tf) [jmoraes@lewis4-r630-login-node675 Songbird_standalone]$ conda env export
name: tf
channels:
  - conda-forge
  - bioconda
  - anaconda
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - _tflow_select=2.3.0=mkl
  - absl-py=0.15.0=pyhd3eb1b0_0
  - astor=0.8.1=py36h06a4308_0
  - attrs=21.4.0=pyhd3eb1b0_0
  - backcall=0.2.0=pyhd3eb1b0_0
  - biom-format=2.1.6=py36_1
  - blas=1.0=mkl
  - brotlipy=0.7.0=py36h27cfd23_1003
  - c-ares=1.19.0=h5eee18b_0
  - ca-certificates=2023.5.7=hbcca054_0
  - cachecontrol=0.12.6=pyhd3eb1b0_0
  - certifi=2020.6.20=pyhd3eb1b0_3
  - cffi=1.14.6=py36h400218f_0
  - charset-normalizer=2.0.4=pyhd3eb1b0_0
  - click=8.0.3=pyhd3eb1b0_0
  - colorama=0.4.4=pyh9f0ad1d_0
  - coverage=4.0.3=py36_1
  - coveralls=1.3.0=py36_0
  - cryptography=35.0.0=py36hd23ed53_0
  - cycler=0.11.0=pyhd3eb1b0_0
  - decorator=5.1.1=pyhd3eb1b0_0
  - docopt=0.6.2=py_1
  - flake8=4.0.1=pyhd3eb1b0_1
  - freetype=2.12.1=h4a9f257_0
  - future=0.18.2=py36_1
  - gast=0.2.2=py36_0
  - google-pasta=0.2.0=pyhd3eb1b0_0
  - grpcio=1.31.0=py36hf8bcb03_0
  - h5py=2.10.0=py36hd6299e0_1
  - hdf5=1.10.6=hb1b8bf9_0
  - hdmedians=0.13=py36hdd07704_1000
  - idna=3.3=pyhd3eb1b0_0
  - importlib-metadata=4.8.1=py36h06a4308_0
  - importlib_metadata=4.8.1=hd3eb1b0_0
  - iniconfig=1.1.1=pyhd3eb1b0_0
  - intel-openmp=2022.1.0=h9e868ea_3769
  - ipython=7.16.1=py36h5ca1d4c_0
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - jedi=0.18.0=py36h06a4308_1
  - joblib=1.0.1=pyhd3eb1b0_0
  - jpeg=9e=h7f8727e_0
  - keras-applications=1.0.8=py_1
  - keras-preprocessing=1.1.2=pyhd3eb1b0_0
  - kiwisolver=1.3.1=py36h2531618_0
  - lcms2=2.12=h3be6417_0
  - ld_impl_linux-64=2.38=h1181459_1
  - lerc=3.0=h295c915_0
  - libdeflate=1.8=h7f8727e_5
  - libffi=3.3=he6710b0_2
  - libgcc-ng=11.2.0=h1234567_1
  - libgfortran-ng=7.5.0=ha8ba4b0_17
  - libgfortran4=7.5.0=ha8ba4b0_17
  - libgomp=11.2.0=h1234567_1
  - libpng=1.6.37=hbc83047_0
  - libprotobuf=3.17.2=h4ff587b_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - libtiff=4.5.0=h6a678d5_1
  - libwebp-base=1.2.4=h5eee18b_0
  - lockfile=0.12.2=py36h06a4308_0
  - lz4-c=1.9.4=h6a678d5_0
  - markdown=3.1.1=py36_0
  - matplotlib-base=3.3.4=py36h62a2d02_0
  - mccabe=0.7.0=pyhd3eb1b0_0
  - mkl=2020.2=256
  - mkl-service=2.3.0=py36he8ac12f_0
  - mkl_fft=1.3.0=py36h54f3939_0
  - mkl_random=1.1.1=py36h0573a6f_0
  - more-itertools=8.12.0=pyhd3eb1b0_0
  - msgpack-python=1.0.2=py36hff7bd54_1
  - natsort=7.1.1=pyhd3eb1b0_0
  - ncurses=6.4=h6a678d5_0
  - numpy=1.19.2=py36h54aff64_0
  - numpy-base=1.19.2=py36hfa32c7d_0
  - olefile=0.46=py36_0
  - openjpeg=2.4.0=h3ad879b_0
  - openssl=1.1.1l=h7f98852_0
  - opt_einsum=3.3.0=pyhd3eb1b0_1
  - packaging=21.3=pyhd3eb1b0_0
  - pandas=1.1.5=py36ha9443f7_0
  - parso=0.8.3=pyhd3eb1b0_0
  - patsy=0.5.3=pyhd8ed1ab_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=8.3.1=py36h2c7a002_0
  - pip=21.2.2=py36h06a4308_0
  - pluggy=0.13.1=py36h06a4308_0
  - prompt-toolkit=3.0.20=pyhd3eb1b0_0
  - protobuf=3.17.2=py36h295c915_0
  - ptyprocess=0.7.0=pyhd3eb1b0_2
  - py=1.11.0=pyhd3eb1b0_0
  - pycodestyle=2.8.0=pyhd3eb1b0_0
  - pycparser=2.21=pyhd3eb1b0_0
  - pyflakes=2.4.0=pyhd3eb1b0_0
  - pygments=2.11.2=pyhd3eb1b0_0
  - pyopenssl=22.0.0=pyhd3eb1b0_0
  - pyparsing=3.0.4=pyhd3eb1b0_0
  - pysocks=1.7.1=py36h06a4308_0
  - pytest=6.2.4=py36h06a4308_2
  - python=3.6.13=h12debd9_1
  - python-dateutil=2.8.2=pyhd3eb1b0_0
  - python_abi=3.6=2_cp36m
  - pytz=2021.3=pyhd3eb1b0_0
  - pyyaml=5.1.2=py36h516909a_1
  - readline=8.2=h5eee18b_0
  - requests=2.27.1=pyhd3eb1b0_0
  - scikit-bio=0.5.6=py36h6323ea4_0
  - scikit-learn=0.24.2=py36ha9443f7_0
  - scipy=1.5.2=py36h0b6359f_0
  - setuptools=58.0.4=py36h06a4308_0
  - six=1.16.0=pyhd3eb1b0_1
  - songbird=1.0.3=py36h9f0ad1d_0
  - sqlite=3.41.2=h5eee18b_0
  - tensorboard=1.15.0=pyhb230dea_0
  - tensorflow=1.15.0=mkl_py36h4920b83_0
  - tensorflow-base=1.15.0=mkl_py36he1670d9_0
  - tensorflow-estimator=1.15.1=pyh2649769_0
  - termcolor=1.1.0=py36h06a4308_1
  - threadpoolctl=2.2.0=pyh0d69192_0
  - tk=8.6.12=h1ccaba5_0
  - toml=0.10.2=pyhd3eb1b0_0
  - tornado=6.1=py36h27cfd23_0
  - tqdm=4.65.0=pyhd8ed1ab_0
  - traitlets=4.3.3=py36h06a4308_0
  - typing_extensions=4.1.1=pyh06a4308_0
  - urllib3=1.26.8=pyhd3eb1b0_0
  - wcwidth=0.2.5=pyhd3eb1b0_0
  - webencodings=0.5.1=py36_1
  - werkzeug=0.16.1=py_0
  - wheel=0.37.1=pyhd3eb1b0_0
  - wrapt=1.12.1=py36h7b6447c_1
  - xz=5.4.2=h5eee18b_0
  - yaml=0.2.5=h7f98852_2
  - zipp=3.6.0=pyhd3eb1b0_0
  - zlib=1.2.13=h5eee18b_0
  - zstd=1.5.2=ha4553b6_0
prefix: /storage/htc/lucylab/Joao-files/miniconda/envs/tf

Any further thoughts? Thank you!!

Hi, I'm assuming that you followed the readme instructions?
conda create -n songbird_env songbird "pandas>=0.18.0,<1" -c conda-forge
source activate songbird_env

It would help if you included the error messages.

@mortonjt,
I initially tried following the readme instructions but that did not work. Conda was stuck in "Solving environment". I am trying the code you provided again right now and will report back soon. But Conda is currently stuck in "Solving environment". That's what happened last time.
Thank you!

Try using mamba instead of conda -- it is much faster. You can install it in conda via conda install mamba.  Then this will look like mamba create -n songbird_env songbird "pandas>=0.18.0,<1" -c conda-forge

Hi, it looks like the issue is with biom-format.  So try mamba install biom-format=2.1.7 -c conda-forge.
Basically we cannot support any packages updated after 2019 (due to tensorflow) -- so you may need to downgrade all packages (sorry).

@mortonjt,
I tried downgrading biom-format mamba install biom-format=2.1.7 -c conda-forge but that did not fix the issue. I'll try downgrading more packages. Hopefully I'll be able to use songbird.
Thanks for your help!

I'm not sure the biom-format package is the exact problem here -- pinning
the version of pandas, as is done in the environment creation command,
should force conda / mamba to choose versions of other packages that are
compatible with this pandas version (and there should be old versions of
biom-format still available online). Downgrading packages one-on-one in an
existing environment will be challenging, because dependency issues in
these environments can get really complicated to solve manually (e.g. an
older version of package X requires an older version of package Y, which
requires a version of at least package Z, ...)
So: the fact that you are still receiving this error implies that the
environment creation process isn't working properly. Could you try running
the "mamba create" command with a new environment name
(e.g. songbird_env2)? I think this should fix the problem, and allow you to
use songbird from this new environment. I suspect that reusing the old
environment name is causing conda / mamba to get confused and not start
over from a fresh environment, which messes up the dependencies.
If this doesn't fix the problem, then please run "conda env export" from
within the newly created environment and post the output here (without
first trying to downgrade anything in the environment). But fingers crossed
that this works!
On Tue, May 16, 2023, 8:49 AM pyjm @.***> wrote:

@mortonjt https://github.com/mortonjt,
I tried downgrading biom-format mamba install biom-format=2.1.7 -c
conda-forge but that did not fix the issue. I'll try downgrading more
packages. Hopefully I'll be able to use songbird.
Thanks for your help!
—
Reply to this email directly, view it on GitHub
https://github.com/biocore/songbird/issues/168#issuecomment-1549930549,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA736P2BQX3HDGFVT47VPKTXGOOYZANCNFSM6AAAAAAX5FJCTI
.
You are receiving this because you were mentioned.
Message ID: @.***>

Hi @fedarko,
Thanks for chiming in!!
I tried running mamba create -n songbird_env2 songbird "pandas>=0.18.0,<1" -c conda-forge but I am getting the same error. Could it be that something is wrong with my .biom file?
Please see below the output of conda env export. Thank you!!
(songbird_env2) []$ conda env export
name: songbird_env2
channels:
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - _tflow_select=2.3.0=mkl
  - absl-py=0.15.0=pyhd8ed1ab_0
  - alsa-lib=1.2.8=h166bdaf_0
  - astor=0.8.1=pyh9f0ad1d_0
  - attr=2.5.1=h166bdaf_1
  - attrs=23.1.0=pyh71513ae_1
  - backcall=0.2.0=pyh9f0ad1d_0
  - backports=1.0=pyhd8ed1ab_3
  - backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0
  - biom-format=2.1.12=py37hda87dfa_1
  - brotli=1.0.9=h166bdaf_8
  - brotli-bin=1.0.9=h166bdaf_8
  - brotlipy=0.7.0=py37h540881e_1004
  - bzip2=1.0.8=h7f98852_4
  - c-ares=1.18.1=h7f98852_0
  - ca-certificates=2023.5.7=hbcca054_0
  - cachecontrol=0.12.11=pyhd8ed1ab_1
  - cached-property=1.5.2=hd8ed1ab_1
  - cached_property=1.5.2=pyha770c72_1
  - cairo=1.16.0=ha61ee94_1014
  - certifi=2023.5.7=pyhd8ed1ab_0
  - cffi=1.15.1=py37h43b0acd_1
  - charset-normalizer=3.1.0=pyhd8ed1ab_0
  - click=8.1.3=py37h89c1867_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - cryptography=38.0.2=py37h5994e8b_1
  - cycler=0.11.0=pyhd8ed1ab_0
  - cython=0.29.32=py37hd23a5d3_0
  - dbus=1.13.6=h5008d03_3
  - decorator=5.1.1=pyhd8ed1ab_0
  - exceptiongroup=1.1.1=pyhd8ed1ab_0
  - expat=2.5.0=hcb278e6_1
  - fftw=3.3.10=nompi_hc118613_107
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - font-ttf-inconsolata=3.000=h77eed37_0
  - font-ttf-source-code-pro=2.038=h77eed37_0
  - font-ttf-ubuntu=0.83=hab24e00_0
  - fontconfig=2.14.2=h14ed4e7_0
  - fonts-conda-ecosystem=1=0
  - fonts-conda-forge=1=0
  - fonttools=4.38.0=py37h540881e_0
  - freetype=2.12.1=hca18f0e_1
  - future=0.18.2=py37h89c1867_5
  - gast=0.2.2=py_0
  - gettext=0.21.1=h27087fc_0
  - glib=2.76.2=hfc55251_0
  - glib-tools=2.76.2=hfc55251_0
  - google-pasta=0.2.0=pyh8c360ce_0
  - graphite2=1.3.13=h58526e2_1001
  - grpc-cpp=1.48.1=hc2bec63_1
  - grpcio=1.48.1=py37h42e856d_1
  - gst-plugins-base=1.22.0=h4243ec0_2
  - gstreamer=1.22.0=h25f0c4b_2
  - gstreamer-orc=0.4.33=h166bdaf_0
  - h5py=3.7.0=nompi_py37hf1ce037_101
  - harfbuzz=6.0.0=h8e241bc_0
  - hdf5=1.12.2=nompi_h4df4325_101
  - hdmedians=0.14.2=py37hda87dfa_2
  - icu=70.1=h27087fc_0
  - idna=3.4=pyhd8ed1ab_0
  - importlib-metadata=4.11.4=py37h89c1867_0
  - importlib_metadata=4.11.4=hd8ed1ab_0
  - iniconfig=2.0.0=pyhd8ed1ab_0
  - ipython=7.33.0=py37h89c1867_0
  - jack=1.9.22=h11f4161_0
  - jedi=0.18.2=pyhd8ed1ab_0
  - joblib=1.2.0=pyhd8ed1ab_0
  - jpeg=9e=h0b41bf4_3
  - keras-applications=1.0.8=py_1
  - keras-preprocessing=1.1.2=pyhd8ed1ab_0
  - keyutils=1.6.1=h166bdaf_0
  - kiwisolver=1.4.4=py37h7cecad7_0
  - krb5=1.20.1=h81ceb04_0
  - lame=3.100=h166bdaf_1003
  - lcms2=2.14=h6ed2654_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - lerc=4.0.0=h27087fc_0
  - libabseil=20220623.0=cxx17_h05df665_6
  - libaec=1.0.6=hcb278e6_1
  - libblas=3.9.0=16_linux64_openblas
  - libbrotlicommon=1.0.9=h166bdaf_8
  - libbrotlidec=1.0.9=h166bdaf_8
  - libbrotlienc=1.0.9=h166bdaf_8
  - libcap=2.67=he9d0100_0
  - libcblas=3.9.0=16_linux64_openblas
  - libclang=15.0.7=default_had23c3d_1
  - libclang13=15.0.7=default_h3e3d535_1
  - libcups=2.3.3=h36d4200_3
  - libcurl=8.0.1=h588be90_0
  - libdb=6.2.32=h9c3ff4c_0
  - libdeflate=1.14=h166bdaf_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=h516909a_1
  - libevent=2.1.10=h28343ad_4
  - libexpat=2.5.0=hcb278e6_1
  - libffi=3.4.2=h7f98852_5
  - libflac=1.4.2=h27087fc_0
  - libgcc-ng=12.2.0=h65d4601_19
  - libgcrypt=1.10.1=h166bdaf_0
  - libgfortran-ng=12.2.0=h69a702a_19
  - libgfortran5=12.2.0=h337968e_19
  - libglib=2.76.2=hebfc3b9_0
  - libgomp=12.2.0=h65d4601_19
  - libgpg-error=1.46=h620e276_0
  - libiconv=1.17=h166bdaf_0
  - liblapack=3.9.0=16_linux64_openblas
  - libllvm15=15.0.7=hadd5161_1
  - libnghttp2=1.52.0=h61bc06f_0
  - libnsl=2.0.0=h7f98852_0
  - libogg=1.3.4=h7f98852_1
  - libopenblas=0.3.21=pthreads_h78a6416_3
  - libopus=1.3.1=h7f98852_1
  - libpng=1.6.39=h753d276_0
  - libpq=15.3=hbcd7760_0
  - libprotobuf=3.20.1=h6239696_4
  - libsndfile=1.2.0=hb75c966_0
  - libsqlite=3.41.2=h2797004_1
  - libssh2=1.10.0=hf14f497_3
  - libstdcxx-ng=12.2.0=h46fd767_19
  - libsystemd0=253=h8c4010b_1
  - libtiff=4.4.0=h82bc61c_5
  - libtool=2.4.7=h27087fc_0
  - libudev1=253=h0b41bf4_1
  - libuuid=2.38.1=h0b41bf4_0
  - libvorbis=1.3.7=h9c3ff4c_0
  - libwebp-base=1.3.0=h0b41bf4_0
  - libxcb=1.13=h7f98852_1004
  - libxkbcommon=1.5.0=h79f4944_1
  - libxml2=2.10.3=hca2bb57_4
  - libzlib=1.2.13=h166bdaf_4
  - lockfile=0.12.2=py_1
  - lz4-c=1.9.4=hcb278e6_0
  - markdown=3.4.3=pyhd8ed1ab_0
  - matplotlib=3.5.3=py37h89c1867_2
  - matplotlib-base=3.5.3=py37hf395dca_2
  - matplotlib-inline=0.1.6=pyhd8ed1ab_0
  - mpg123=1.31.3=hcb278e6_0
  - msgpack-python=1.0.4=py37h7cecad7_0
  - munkres=1.1.4=pyh9f0ad1d_0
  - mysql-common=8.0.32=hf1915f5_2
  - mysql-libs=8.0.32=hca2cd23_2
  - natsort=8.3.1=pyhd8ed1ab_0
  - ncurses=6.3=h27087fc_1
  - nspr=4.35=h27087fc_0
  - nss=3.89=he45b914_0
  - numpy=1.21.6=py37h976b520_0
  - openjpeg=2.5.0=h7d73246_1
  - openssl=3.1.0=hd590300_3
  - opt_einsum=3.3.0=pyhd8ed1ab_1
  - packaging=23.1=pyhd8ed1ab_0
  - pandas=0.25.3=py37hb3f55d8_0
  - parso=0.8.3=pyhd8ed1ab_0
  - patsy=0.5.3=pyhd8ed1ab_0
  - pcre2=10.40=hc3806b6_0
  - pexpect=4.8.0=py37hc8dfbb8_1
  - pickleshare=0.7.5=py37hc8dfbb8_1002
  - pillow=9.2.0=py37h850a105_2
  - pip=23.1.2=pyhd8ed1ab_0
  - pixman=0.40.0=h36c2ea0_0
  - pluggy=1.0.0=py37h89c1867_3
  - ply=3.11=py_1
  - prompt-toolkit=3.0.38=pyha770c72_0
  - protobuf=3.20.1=py37hd23a5d3_0
  - pthread-stubs=0.4=h36c2ea0_1001
  - ptyprocess=0.7.0=pyhd3deb0d_0
  - pulseaudio=16.1=hcb278e6_3
  - pulseaudio-client=16.1=h5195f5e_3
  - pulseaudio-daemon=16.1=ha8d29e2_3
  - pycparser=2.21=pyhd8ed1ab_0
  - pygments=2.15.1=pyhd8ed1ab_0
  - pyopenssl=23.1.1=pyhd8ed1ab_0
  - pyparsing=3.0.9=pyhd8ed1ab_0
  - pyqt=5.15.7=py37hf30b843_1
  - pyqt5-sip=12.11.0=py37hd23a5d3_1
  - pysocks=1.7.1=py37h89c1867_5
  - pytest=7.2.0=py37h89c1867_0
  - python=3.7.12=hf930737_100_cpython
  - python-dateutil=2.8.2=pyhd8ed1ab_0
  - python_abi=3.7=3_cp37m
  - pytz=2023.3=pyhd8ed1ab_0
  - qt-main=5.15.8=h5d23da1_6
  - re2=2022.06.01=h27087fc_1
  - readline=8.2=h8228510_1
  - requests=2.29.0=pyhd8ed1ab_0
  - scikit-bio=0.5.5=py37h3010b51_1000
  - scikit-learn=1.0.2=py37hf9e9bfc_0
  - scipy=1.7.3=py37hf2a6cf1_0
  - setuptools=59.8.0=py37h89c1867_1
  - sip=6.7.2=py37hd23a5d3_0
  - six=1.16.0=pyh6c4a22f_0
  - songbird=1.0.3=py37hc8dfbb8_1
  - sqlite=3.41.2=h2c6b66d_1
  - tensorboard=1.15.0=py37_0
  - tensorflow=1.15.0=mkl_py37h28c19af_0
  - tensorflow-base=1.15.0=mkl_py37he1670d9_0
  - tensorflow-estimator=1.15.1=pyh2649769_0
  - termcolor=1.1.0=pyhd8ed1ab_3
  - threadpoolctl=3.1.0=pyh8a188c0_0
  - tk=8.6.12=h27826a3_0
  - toml=0.10.2=pyhd8ed1ab_0
  - tomli=2.0.1=pyhd8ed1ab_0
  - tornado=6.2=py37h540881e_0
  - tqdm=4.65.0=pyhd8ed1ab_1
  - traitlets=5.9.0=pyhd8ed1ab_0
  - typing-extensions=4.5.0=hd8ed1ab_0
  - typing_extensions=4.5.0=pyha770c72_0
  - unicodedata2=14.0.0=py37h540881e_1
  - urllib3=1.26.15=pyhd8ed1ab_0
  - wcwidth=0.2.6=pyhd8ed1ab_0
  - werkzeug=0.16.1=py_0
  - wheel=0.40.0=pyhd8ed1ab_0
  - wrapt=1.14.1=py37h540881e_0
  - xcb-util=0.4.0=h516909a_0
  - xcb-util-image=0.4.0=h166bdaf_0
  - xcb-util-keysyms=0.4.0=h516909a_0
  - xcb-util-renderutil=0.3.9=h166bdaf_0
  - xcb-util-wm=0.4.1=h516909a_0
  - xkeyboard-config=2.38=h0b41bf4_0
  - xorg-kbproto=1.0.7=h7f98852_1002
  - xorg-libice=1.0.10=h7f98852_0
  - xorg-libsm=1.2.3=hd9c2040_1000
  - xorg-libx11=1.8.4=h0b41bf4_0
  - xorg-libxau=1.0.9=h7f98852_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xorg-libxext=1.3.4=h0b41bf4_2
  - xorg-libxrender=0.9.10=h7f98852_1003
  - xorg-renderproto=0.11.1=h7f98852_1002
  - xorg-xextproto=7.3.0=h0b41bf4_1003
  - xorg-xproto=7.0.31=h7f98852_1007
  - xz=5.2.6=h166bdaf_0
  - zipp=3.15.0=pyhd8ed1ab_0
  - zlib=1.2.13=h166bdaf_4
  - zstd=1.5.2=h3eb15da_6
prefix: /PATH/miniconda/envs/songbird_env2

Instead of using our HPCC, I tried installing songbird in a new environment in my personal computer (macOS) using mamba create -n songbird_env2 songbird "pandas>=0.18.0,<1" -c conda-forge and I am getting a different error here.
After running:
songbird multinomial\
    --input-biom feature-table.biom\
    --metadata-file POSTPARTUM_mapping_sample_key.txt\
    --formula "OBS+Class+TRT+Cow_ID+Site+Cycling+P4" \
    --epochs 10000 \
    --differential-prior 0.5 \
    --summary-interval 1 \
    --summary-dir results

I get:
Traceback (most recent call last):
  File "/Users/conda/envs/songbird_env2/bin/songbird", line 8, in <module>
    from songbird.multinomial import MultRegression
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/songbird/multinomial.py", line 3, in <module>
    from tensorflow.contrib.distributions import Multinomial, Normal
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow/__init__.py", line 50, in __getattr__
    module = self._load()
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow/__init__.py", line 44, in _load
    module = _importlib.import_module(self.__name__)
  File "/Users/conda/envs/songbird_env2/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_core/contrib/__init__.py", line 39, in <module>
    from tensorflow.contrib import compiler
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_core/contrib/compiler/__init__.py", line 21, in <module>
    from tensorflow.contrib.compiler import jit
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_core/contrib/compiler/__init__.py", line 22, in <module>
    from tensorflow.contrib.compiler import xla
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_core/contrib/compiler/xla.py", line 22, in <module>
    from tensorflow.python.estimator import model_fn as model_fn_lib
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_core/python/estimator/model_fn.py", line 26, in <module>
    from tensorflow_estimator.python.estimator import model_fn
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_estimator/__init__.py", line 10, in <module>
    from tensorflow_estimator._api.v1 import estimator
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_estimator/_api/v1/estimator/__init__.py", line 10, in <module>
    from tensorflow_estimator._api.v1.estimator import experimental
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_estimator/_api/v1/estimator/experimental/__init__.py", line 10, in <module>
    from tensorflow_estimator.python.estimator.canned.dnn import dnn_logit_fn_builder
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/canned/dnn.py", line 27, in <module>
    from tensorflow_estimator.python.estimator import estimator
  File "/Users/conda/envs/songbird_env2/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 36, in <module>
    from tensorflow.python.profiler import trace
ImportError: cannot import name 'trace' from 'tensorflow.python.profiler' (/Users/songbird_env2/lib/python3.7/site-packages/tensorflow_core/python/profiler/__init__.py)

Hi @pyjm,
Here are a few steps you can try to fix this; I'm separating my comment into sections to make it more readable. The first four apply to the HPCC that I think you've been using for this issue so far; the last is an idea for your personal computer's installation.
1. [On your HPCC] Checking about "the same error"

I tried running mamba create -n songbird_env2 songbird "pandas>=0.18.0,<1" -c conda-forge but I am getting the same error. Could it be that something is wrong with my .biom file?

Just to make sure we're on the same page with regards to what "the same error" is -- to clarify, mamba is able to successfully create the songbird_env2 environment and you are able to activate this environment, but upon running Songbird you still get an error that ends with the following message? Let us know if the error changed.
    mat = [pd.SparseSeries(v.toarray().ravel()) for v in self.matrix_data]
TypeError: object() takes no parameters

2. [On your HPCC] Checking if the problem is due to your BIOM file

Could it be that something is wrong with my .biom file?

It's possible, but I think it is more likely that something is going wrong in your conda environments / with conda in general on your system. You can check this by downloading the redsea.biom and redsea_metadata.txt files from Songbird's GitHub repository, and then running the following command from the same directory to which you downloaded these files:
songbird multinomial --input-biom data/redsea/redsea.biom --metadata-file data/redsea/redsea_metadata.txt --formula "Oxygen"

If this command succeeds, then we know that your BIOM file is the problem; if this command gives you the same error, then we know that the problem isn't with your BIOM file.
3. [On your HPCC] Checking another "standalone" conda environment
I copied the dependencies in the output of your conda env export command from this comment, created a new conda environment from them (I had to remove a few unrelated dependencies that made mamba complain, e.g. pulseaudio, libsystemd, matplotlib), and verified that this environment was able to run Songbird without problems. I'm attaching a copy of this environment YML below.
working-env.txt
It's a bit of a long shot, but you may want to try:

download this file,
rename it from working-env.txt to working-env.yml (sorry, GitHub won't let me upload files that end in .yml...),
run mamba env create -f working-env.yml,
if creating this environment succeeds, activate the environment (the file names its environment pyjm, so you can just run conda activate pyjm) and then try running Songbird (either on your dataset or the redsea dataset from above)

If you get the same error when running Songbird as before, then I think this proves that something is going wrong with conda on your system.
4. [On your HPCC] Checking a QIIME 2 conda environment
This idea is more cumbersome than the previous ideas, but I suspect it might help if you still have problems:

Try installing a version of QIIME 2 that is compatible with Songbird (e.g. version 2020.6)
After installing QIIME 2 and activating its conda environment, try installing Songbird into this environment
Try running Songbird through QIIME 2 (see the Songbird + QIIME 2 tutorial -- you can just work up through the first command that says qiime songbird multinomial to see if it runs successfully or still gives you an error)

If installing even QIIME 2 doesn't work on your system, then this is probably a problem with your system's conda installation or with the system itself. As a "nuclear option," I'd bet that installing QIIME 2 on your macOS system should allow you to use Songbird through it -- Songbird and QIIME 2 have been used together extensively before.
5. [On your macOS system] Checking about the new error you got
This error message suggests that you may have an incorrect TensorFlow version (outside of the range [1.15, 2)) present in this environment, which is strange because installing Songbird should have installed a TensorFlow version within this range. You can check which TensorFlow version is installed by running conda list | grep tensorflow (or you can use pip list | grep tensorflow, but I think you should see mostly the same results).
If the TensorFlow version installed is indeed incorrect (< 1.15 or ≥ 2 -- probably it'll be ≥ 2), then you can try to downgrade it to e.g. 1.15 -- but this may leave other dependencies in your environment still broken. An alternative solution is creating a new conda environment without Songbird, then installing Songbird into this environment using pip:
mamba create -n songbird_env3 "pandas>=0.18.0,<1" pip
conda activate songbird_env3
pip install songbird

I think this should work?

Anyway, you don't have to try all of these steps, but I suspect that at least Step 4 or Step 5 should fix your problem. The other ones will mostly be helpful for debugging :) Thanks!

@fedarko and @mortonjt
Thank you both very much for taking the time to help me with this issue.
I downloaded the latest Miniconda version released on April 24, 2023 and reinstalled songbird with mamba create -n songbird_env2 songbird "pandas>=0.18.0,<1" -c conda-forge and everything seems to be working well now.
I am excited to process my data using Songbird.
Thanks again for your help!


Automated Package Upgrade

:tada: This PR is included in version 25.3.501 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Cannot run with CGI

First, want to thank you for this highly usefull software!
As mentioned in readme, I tried to run it with cgi,  both with slowcgi, and apache with cgid module.
Slowcgi just shows blank page, and apache shows

Internal Server Error

With this log:
End of script output before headers: dhcpd-leasesd
The script has executable bit enabled. I can run it with /bin/sh dhcpd-leasesd.
Maybe i can somehow debug it?
Apache has LogLevel debug of course, but there is no usefull information except that one line.

Thanks for the kind words!
I had never tried using the script with httpd/slowcgi before but i gave it a shot now.
A main issue with running the script as a cgi is the chroot, but the script too needs some updating which is planned for the next version.
I got the basic script running but not vendor lookup. Getting nc running in the chroot looks messy and I'm using that to download the oui data.
So unfortunately the cgi functionality is currently broken but it should work (less vendor lookup) in the next update.
Until then i would recommend using the build in server or tcpserver.

I'm looking forward to it.
Chroot will not be in issue for me, because I run it on home router, and there is no point for hardening security with chroot.

With new release 1.5 it works like a charm!
Thanks! This is "killer feature" :)


Versioning ASWF containers and release tags

As per the discussion on PR 22, it might make sense to leverage the GitHub version tagging mechanism so we could relate the version of the code in this repo with the Docker Hub container tags.
To answer a specific question that came up in the initial discussion:

Would you agree tags should be added directly on master at release time by the azure pipeline job?

It would seem to make sense to automate this process: if a container image is pushed to Docker Hub and tagged there, the source tree should also be tagged accordingly, and automating that process would help to avoid mistakes.
Eventually we might also want to support GitHub Packages as an alternative repository for ASWF build containers. Since you cannot (easily) delete public packages pushed to GitHub Packages, we would need to make sure that the containers work before they are exposed.

At the moment a single commit on master can potentially spawn around 83 individually versioned docker images (as seen here: https://github.com/AcademySoftwareFoundation/aswf-docker/blob/cf8c55740f87a3da0b0c1d356e4a9e46795870f3/python/aswfdocker/constants.py#L16 ).
I've just tested the manual creation of a release (https://github.com/AcademySoftwareFoundation/aswf-docker/releases/tag/ci-common-1.4) and I'm wondering if I could automate the creation of a release using the aswfdocker python library, then running a manual build on Azure would fetch the tags on master's current commit and build docker images that correspond.
The release process would then be:

run aswfdocker release -t PACKAGE clang,qt which would fetch the current version from the constants.py file (or get it from a new versions.yaml resource). This would create the GitHub releases, with release notes and that would generate the appropriate git tags (`ci-package-clang-1.1,ci-package-qt-2018.1,ci-package-qt-2019.1,ci-package-qt-2020.1). The creation of the release should be also doable manually via the GitHub interface, the script is just a convenience when authoring large number of packages.
run a manual build on Azure which would fetch the tags on the given commit, then build and push the corresponding images to dockerhub.

Thoughts?


log: zero cltv delta warnings removed

The client might encounters edges that have a policy specifying a zero CLTV delta (possibly because of no/outdated gossip?). Every time an edge like this is used in the path finding code, a warning is logged. This pr removes that warning message.
Fixes: #6069

The client might encounters edges that have a policy specifying a zero CLTV delta (possibly because of no/outdated gossip?)

No, rather this should never happen as the node that is setting this value to zero can be stolen from as any HTLC they forward is instantly at risk for an on-chain race to claim funds.


Support lexicographical screenshot ordering

First of all thank you for creating and maintaining this adapter!
Recently we encountered a use case where users like to browse the screenshots directly using an image viewer without using the generated report file.
Some operating systems (Microsoft Windows in this case) order files by lexicographical order causing the screenshots to be displayed in the incorrect order.
Adding leading zeros to the screenshot counter would cause the screenshots to be ordered correctly.
We'll add a pull request which changes the screenshot file naming convention.

Hi,
Does it work for you if the counter is the first thing in the file name - 1_embedded.jpg?

Hi, Does it work for you if the counter is the first thing in the file name - 1_embedded.jpg?

This solution will not work because Windows uses lexicographical ordering and therefore we will encounter the same issue once reaching more than 10 screenshots.
The filenames are no longer a issue for us because our users no longer directly use the screenshot files. Therefore we are closing this issue as it is no longer relevant for us.


Fix typo: "pengiun" => "penguin"

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

This typo is deliberate and part of the challenge, as you can see from reading the challenge instructions https://learn.freecodecamp.org/responsive-web-design/basic-css/attach-a-fallback-value-to-a-css-variable

Ah, didn't catch that!


Site with LOTS of variables is very slow to load

This is admittedly on Stroud's very slow internet, and far more variables than any normal site would have, but http://data.envirodiy.org/sites/28/ seems to take quite a while to load up for me.

OK - fair enough. Before we invest in optimization, can we settle on what our design use case is for the number of sensors that should load in a reasonable amount of time?  I think it would be something way less than 50.  It's slow because the sparkline plots are generated on the fly when the page loads.

@aufdenkampe ?  I imagine most of our users are going to be using no more than 10 variables and none of them with 12 significant figures.

As part of our efforts to better support larger numbers of sites and sites with large numbers of variables and data values we are testing a new, Linux based deployment environment to replace our current deployment on Windows. We are also considering a service to provide the most recent data values so that the generation of the sparkline plots can be separated from the page load for sites with many variables (i.e., so the page can load while the sparklines are being generated and then the  sparklines appear when they are ready - rather than waiting for up to 10 seconds for the page to load).

@SRGDamia1 - we've done a few things to address this issue.

We are now running in a Linux environment using PostgreSQL on the back end for our production environment. I think that has helped some.
We have changed the way we are getting the data to display the sparkline plots and to preload the tabular views of the data.

Pages for sites that were taking 20+ seconds to load for me earlier are now loading much more quickly.  For a site with 8 variables, it takes just a couple of seconds for the page to load on my home WIFI network (e.g., see http://data.envirodiy.org/sites/12/).
do you have any sites you can check quickly to see if we can close this issue?

@horsburgh I've been on this for the past few days now because there are still a lot of performance issues coming from different places. I solved the slowest loading time by not loading the data values in the back-end, but doing it in a csv file, which reduced the time exponentially for results with a lot of values. Right now the loading time increases the more results a site has, and the site starts lagging when adding all the data (in some cases about 17,000 data values per result) to the table.

@jcaraballo17 - performance seems to be much improved for this. I think we've done enough here to close this issue when all of your enhancements have been pushed to production.

Nice work. It's all working much faster for me too!


Cluster not getting breakdown on tap or on zoom.

Hi,
Few days ago I updated google map sdk . now I get issue in clustering.
cluster not getting breakdown on tap or on zoom on map.

i am also facing same issue. Please let me know if you have the fix for the same


Does SMx algorithmes require clmuls?

Zks enables Zbb, Zbc, Zbx. I am not sure that SMx do really require carry-less multiplication.
Probably, Zks should be _Zkb_Zkx_Zksed_Zksh?

SM3/4 do not need CLMUL, likewise for AES. But CLMUL is very useful for the GCM mode of operation, which wraps SM4 or AES. Hence they often appear together in the extensions.


go.mod: better go-ordered-json

Almost no functional changes, the most important one is that virtuald dependency is removed.

I though it shouldn't work with 1.19

Let's wait for Go update before the merge.


Bitbucket Server - Catalog Registration Action not Registering Entities in Catalog

Expected Behavior
When a catalog-info.yaml file is generated using the fetch-base action followed by a publish and register action, the entity should become available in the catalog.
Sample steps:
    - id: fetch-base
      name: Fetch Base Template
      if: '{{ parameters.generateTemplate }}'
      action: fetch:template
      input:
        url: ./template
        cookiecutterCompat: true
        values:
             name: '{{ parameters.name }}'
             componentOwner: '{{ parameters.componentOwner }}'

    - id: publish
      name: Publish to a New Bitbucket Repository
      action: publish:bitbucket
      input:
        repoUrl: 'my-bitbucket-server.com?owner={{ parameters.projectKey}}&repo={{ parameters.repoName}}'

    - id: register
      name: Register Component in Catalog
      action: catalog:register
      input:
        repoContentsUrl: 'https://my-bitbucket-server.com/projects/{{ parameters.projectKey }}/repos/{{ parameters.repoName }}'
        catalogInfoPath: '/browse/catalog-info.yaml'

  output:
     links:
      - title: Open in Catalog
        icon: 'catalog'
        entityRef: '{{ steps.register.output.entityRef }}'

Screenshot showing Success:

Current Behavior
This did work before the latest update but the entity is no longer showing up in the catalog even though the output shows it was and the catalog-info.yaml file is generated as expected, i also don't see the links as well when using output.links in the template.
Here is what i see in the logs:
2021-07-19T23:07:17.387Z catalog warn Unable to read url, NotFoundError: https://my-bitbucket-server.com/projects/MY-PROJECT/repos/MY-REPO/browse/catalog-info.yaml could not be read as https://my-bitbucket-server.com/rest/api/1.0/projects/MY-PROJECT/repos/MY-REPO/raw/catalog-info.yaml?at=, 404 Not Found type=plugin entity=location:default/generated-80a9272e9d90df1fc0abb152730fb9f5b2221490
When i try hitting https://my-bitbucket-server.com/rest/api/1.0/projects/MY-PROJECT/repos/MY-REPO/raw/catalog-info.yaml?at= or just https://my-bitbucket-server.com/rest/api/1.0/projects/MY-PROJECT/repos/MY-REPO/raw/catalog-info.yaml i get the following response:
{"errors":[{"context":null,"message":"The path \"catalog-info.yaml\" does not exist at revision \"refs/heads/master\"","exceptionName":"com.atlassian.bitbucket.content.NoSuchPathException"}]}
I also tried adding a defaultBranch in the publish step as follows, and get the same result:
    - id: publish
      name: Publish to a New Bitbucket Repository
      action: publish:bitbucket
      input:
        repoUrl: 'my-bitbucket-server.com?owner={{ parameters.projectKey}}&repo={{ parameters.repoName}}'
        defaultBranch: master

I also have the bitbucket discovery processor enabled both in my app-config.yaml and catalog.ts as follows:
// app-config.yaml

  locations:
      - type: bitbucket-discovery
        target: https://my-bitbucket-server.com/projects/*/repos/*/catalog-info.yaml
        rules:
          - allow: [Location, Component, API, Resource, Domain]

// packages/backend/plugins/catalog.ts

  builder.replaceProcessors([
    new FileReaderProcessor(),
    BitbucketDiscoveryProcessor.fromConfig(env.config, { logger: env.logger }),
    LdapOrgReaderProcessor.fromConfig(env.config, { logger: env.logger }),
    new UrlReaderProcessor({ reader: env.reader, logger: env.logger }),
    new LocationEntityProcessor({ integrations }),
    new AnnotateLocationEntityProcessor({ integrations }),
  ]);

A similar issue has been reported for GitHub as well:
https://discord.com/channels/687207715902193673/687235481154617364/866739920340910091
Your Environment

NodeJS Version (v12): Node14
Operating System and Version (e.g. Ubuntu 14.04): Windows 10
Browser Information: Chrome
Bitbucket Server 7.12.1

Seems to be connected to: https://github.com/backstage/backstage/issues/6301#issue-934798561

Think you might want to use the output from the publish step as the URL right @regicsolutions? and the catalogInfoPath should just be /catalog-info.yaml if it's in the root? I'm pretty sure that we tested this with a very similar setup to this: https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/sample-templates/react-ssr-template/template.yaml#L72 and it worked

Ya, that worked. Thanks!


[BUG]: Low level ZeRO may have memory leaking when using gradient accumulation

🐛 Describe the bug
When using Low-level ZeRO with gradient accumulation, we observed that the CUDA memory allocation increased during the accumulation steps.
I think that there are memories that should have been released.
Environment
No response

It seems there is a misunderstanding of the usage of gradient accumulation with zero, not a bug.


Request: rename (or add) Invocation naming requirement

Right now, Devil's sight looks for a feature with the name "Invocation: Devil's Sight".  Similar for Ghostly Gaze and Witch's Sight.
The issue is that DDB names these features somewhat differently, e.g. "Eldritch Invocation: Devil's Sight".  Here's a screenshot from DDB, which is not exactly apropos because it is not exactly clear how the two levels are combined.  But based on how DDBI imports it, I'm guessing the JSON for the feature names it including Eldritch.

I've asked Mr Primate about changing how DDBI imports it, but he has said doing so would like break a bunch of other things.  So I'm here to ask if you can either change the string that it's looking for to match DDBI, or perhaps add a second string that matches the above, or do some regex magic.  This would help automate the proper vision for those of us using DDBI to import PCs from DDB.
Thanks!

Here is how the feature imports from DDBI:

I didn't know that. I can add to theses name to matching too. Not a problem.
Does DDB offer German and/or French translations? If so and the names are different, I'll need to add them too.

I didn't know that. I can add to theses name to matching too. Not a problem.
Does DDB offer German and/or French translations? If so and the names are different, I'll need to add them too.

It does not

There is also the Eldridge Adept feat, that offers a Devil's Sight choice.  Ideally we would catch that too.  I'm not positive what the DDB feature for that is called.
For edification, is there a reason you moved away from looking for an effect name and started looking directly for feats?

For edification, is there a reason you moved away from looking for an effect name and started looking directly for feats?

Some feats don't have an active effect configured. I didn't want to make it a requirement to set that up first. And if there's a need for an active effect for devil's sight, that's possible with ATL.

But how would I know that an effect is supposed to give a certain detection mode if it could be named anything?

I meant that you would be the one to define it, very much like you're doing now, just with effects rather than feature names. Because the automation crowd can easily add effects, but we can't as easily modify feature names, given current tools, data sources, and workflows. For example, one of our automation guys was saying that his player has the Devil's Sight feat, so now he has to go in and rename it as though it were a warlock invocation in order to get Devil's Sight working right.
Instead, you might say "V5E will give Devil's Sight (PC version) to any token with an effect named "Devil's Sight", and Devil's Sight (monster version) to any token with an effect named "NPC Devil's Sight" (or whatever terminology you want to use).
Then the effect writers could write the features, feats, spells, etc. so that anything that should give the devil's sight effect adds an affect with that name, rather than having to rename features to match how you've built it.
But listen, I was just asking, to try to make things simpler for everyone.  No promises that my thoughts about are actually good technical ones.  If what we have now works well enough (with the changes proposed above), I'm fine with that too.
Thanks!

Maybe looking for either feature names or effect names would give the best of both worlds, at the cost of some additional complexity on your side.

@dineshm72: I misunderstood you. I thought you wanted that the effect name could be anything and it would work somehow.
I'll add alternative names for invocations. For example:

Devil's Sight (will have a different effect on NPC actors)
Invocation: Devil's Sight
Invocations: Devil's Sight
Eldritch Invocation: Devil's Sight
Eldritch Invocations: Devil's Sight
Eldritch Adept: Devil's Sight

I could also treat effects with names as if they are feats. But would it still be necessary if all of the above is picked up by the automation?

@Tallon159: Do you know the official French translation of Tasha's Eldritch Adept feat?

@mhilbrunner: Do you know the official German translation of Tasha's Eldritch Adept feat?

Sounds good to me.  Looks like you have the options covered for Devil's Sight.

@dev7355608 The official translation in tasha's book is "Adepte occulte"

@dev7355608: Just checked, it is "Schauerlicher Adept" (thanks for the ping!)

Thanks, @Tallon159 and @mhilbrunner!

Released https://github.com/dev7355608/vision-5e/releases/tag/v1.10.4.


Improve Composition vs Inheritance

Page about So What About Inheritance? is quiet undigestable.
What about containers, <Provider> , <Router>  , aren't they Inherited.
Our whole wrapped inside them <App />
sending data from parent component through State and map them in child as props,
Isn't it a inheritance.

No, what you’re describing isn’t inheritance, as the doc says.


Phoenix hot deployment static assets not using new version

When using exrm to build releases for a Phoenix application (https://excasts.com), after the upgrade is complete, it seems that the static assets being used are  not the new set.   I am using Phoenix digesting in my application.
Paul, just a reminder from our conversation at ElixirConf, you theorized this may be related to the way Phoenix accomplishes its static asset serving. possibly their responsible server does not implement the code_change function.
Thanks and I look forward to working on this issue with you.

In my application, I usually make an asset_cache_bust/0 function:
def asset_cache_bust do
  Phoenix.Config.clear_cache MyApp.Endpoint
  Phoenix.Endpoint.Adapter.warmup MyApp.Endpoint
end

Then, as part of the deploy script, I use a RPC to call it.
$ /mnt/app/bin/my_app rpc Elixir.MyApp asset_cache_bust

This is obviously a workaround that shouldn't be there, but it works for me. :smiley:

@midas I think I may have found the problem, in Phoenix's config/config.exs, the following configuration is present:
config :my_app, MyApp.Endpoint,
  ...
  root: Path.expand("..", __DIR__),
  ...

The problem here is that root ends up expanding to an absolute path to the project directory when the release is built, so even when you do an upgrade, it's still pointing to the assets via that absolute path.
I overrode it in my example chat app I used in my talk at ElixirConf, by putting the following in config/prod.exs:
config :chat, Chat.Endpoint,
  ...
  root: ".",
  ...

I then tested a style change with a rolling upgrade, and it worked exactly as expected. On the plane home I tried playing around with that setting a bit to try and find something that would work both with releases and without, but couldn't come up with a simple solution. I'm also not 100% sure that this was the problem you had (because the "bug" I found is dependent on an absolute path in the config), but depending on your deployment setup, it's entirely possible this was the culprit.
/cc @chrismccord

Would dropping a comment above the config that mentions root for releases be enough? We do similar comments for prod config around starting the web server.

@bitwalker That seems plausible to me.  Will test on my end.  @chrismccord Want me to send a pull request for this comment addition?  I would love to get my first contribution in place.
Thanks gentlemen!

@midas Please do!

@bitwalker Can you expand on the reason why this solution works so that I can explain a little more in depth?

@midas Sure! The reason it works is because when the configured path is read in the release, it will be done so relative to the application directory in the release (the currently installed version that is). So since the current working directory is already the desired directory for root, we can just set it to ..

@chrismccord What effect will a root path of . have on the development environment?  Should this setting be overridden only in environments that will be built into a release, ie. production?
Thus, do you think this comment should be in config/config.exs or config/prod.exs?

It should go in prod.exs, and the problem with just making the default . is that with mix it only works when running the project from the project root directory, if you are running an umbrella project or the current working directory is not the project root, it will be incorrect, which I assume is why the default is what it is today, but @chrismccord would have to chime in on that bit. I think by documenting that root needs to be configured when using releases in prod.exs, we can at least make sure nobody else gets bit by this. It would be nice if there was a more general solution so that root needn't be configured at all though.

@midas Is this resolved on the exrm side from your point of view? Can I close this for the time being?

I think so.  Thanks again!

@bitwalker, @chrismccord I have finally figured out what is causing this intermittent issue.  For the assets to update, there has to be a call to config_change of the Endpoint.  If there are no changes to the config, config_change appears to not be called.  Thus, the location of the new assets is not updated in the endpoint.  The way I am currently working around this is to add this to my lib/config.exs:
config :my_app, MyApp.Endpoint,
  version: Mix.Project.config[:version]

This ensures that every release will have an update to the configuration without creating multiple places to update the version number.
I would like to update Exrm and Phoenix documentation and/or source code comments to reflect this if:

It is an acceptable solution in your opinions
We can work together and agree on where/what updates should occur

Looking forward to your thoughts.

@midas I don't see anything wrong with this solution so 👍 from me on adding it to the phoenix deployment guide unless @bitwalker sees issues

@midas where did you find this configuration? I actually can't find in the code where we used it :)

@chrismccord I did not find the configuration anywhere.  I worked with it in one of my private projects until I figured out the issue.  I can extract it into a public project so you can see it there.  Let me know if you would like for me to proceed with this solution.  Thanks!

Seems fine to me. Is there a particular reason why config_change is only called when the config changes, beyond the general idea of "don't do work you don't have to"? If there is no particularly significant cost to revaluation of the config every time, that would solve the issue without requiring users to see this workaround in the docs.

@bitwalker, @chrismccord I am not sure how you need for me to proceed here.  Do you need an example project .  are we just discussing stuff?  Is anything needed form me?
Thanks!

@midas I'd say at the moment documenting the workaround you found is the first priority, but I still would like to discuss my last question as a more permanent solution, assuming there aren't issues with it.

Adding version to my releases did not seem to fix this issue for me
My config.exs
# Configures the endpoint
config :scotchy, Scotchy.Endpoint,
  url: [host: "localhost"],
  root: ".",
  version: Mix.Project.config[:version],
  ...

and my prod.exs
config :scotchy, Scotchy.Endpoint,
  http: [port: 4444],
  url: [host: "beta.myurl.com"],
  debug_errors: true,
  root: ".",
  cache_static_manifest: "priv/static/manifest.json",
  server: true

Nevermind.... I forgot to run MIX_ENV=prod mix phoenix.digest :/


I'm having issues setting up langflow through docker-compose

I'm trying to deploy langflow on ubuntu staging server through docker-compose
every other services starts successful except - backend, celeryworker and flower as shown in the below screenshot.
With error message - /usr/local/bin/python: No module named langflow

when i access the frontend through the browser, i only see the loading image and javascript error as shown on the below screenshot

@dosu, the docker-compose you got your suggestion from was last updated 9months ago, meaning that i doesn't reflect the current way of deploy langflow with docker.
I've tried your suggestions they didn't work.

@Dosu, for celeryworker and flower services am now getting the below errors:
/app/.venv/lib/python3.12/site-packages/langchain/agents/json_chat/base.py:22: SyntaxWarning: invalid escape sequence '\ '
"""Create an agent that uses JSON to format its logic, build for Chat Models.
Usage: python -m langflow run [OPTIONS]
Try 'python -m langflow run --help' for help.
No such option: --upgrade

Please can someone in the core team update the docker-compose.yml in deploy folder as it was last updated 2months? which definitely doesn't conform to current version 1 deployment requirements.

@chibexme this is going to be fixed in 1.0.6.
The current issue is "latest" image doesn't point to 1.x release. Plus, there's a problem with the endpoint that has been already fixed in the main branch but not released yet
cc @ogabrielluiz

@chibexme this is going to be fixed in 1.0.6. The current issue is "latest" image doesn't point to 1.x release. Plus, there's a problem with the endpoint that has been already fixed in the main branch but not released yet
cc @ogabrielluiz

Alright, will be waiting for the 1.0.6 release.
Thanks.

@ogabrielluiz I can see that version 1.0.6 has been release with many bug fixed, that's impressive.
Regard my concern about docker-compose.yml file in the deploy folder, it still has not been updated as can be seen in the screenshot below

I checked the docker image the frontend is point to, it was updated 9months ago

Kindly point me to the right working docker-compose file to deploy langflow from.
Expecting your speedy response.

Fixed by https://github.com/langflow-ai/langflow/pull/2654


[FEATURE_REQUEST]: break stages process code to threads

Hi @miguelaeh, wonderfull job. I readed  source code and found all stages processed in execute_path，so fps will be very low when stages take a lot time, Maybe one stage have one thread，then the fps will be restricted with the most time-consuming stage, not all stages.

Hi @zjd1988 ,
You are right, there is indeed a TODO comment a couple lines above that one to precisely improve it.
Ideally stages should be as small as possible, so the CPU can be captured by a different one.
That said, I considered while writting that part to use rayon for that loop. However, what I still need to figure out is the fact that CPUs are passed to other threads when the one having it is blocked, but in this case, there won't probably be a block, just a task that takes long. Also, the restriction on FPS will still be conditioned by the stage that takes longer, since there is a frame there that must go out.
So, correct me if I am wrong, but I think even if you make use of threads there, if stages take too long the only solution would be to improve those stages themselves or add more resources to the device.

@miguelaeh Yes, you are right. I wish pipeless could replace deepstream equivalently.

@zjd1988 I am curious about how did you find that out. Are you suffering any problem with the FPS with some code that you had in deepstream? It would be really helpful if you could elaborate in order to see if we have to review the current approach

@miguelaeh I‘'m afriad there are no actual examples. I am interested in video processing pipeline, heard about this repository from blog a few days ago. I am also a beginner to rust, so this repository is good start.

aaah ok. Then I hope you liked it! jejej
I am closing this for now. As mentioned, stages are intended to be as small as possible to fully leverage the threading as commented on the getting started guide.
If I manage to find out an alternative that allows us to release the CPU between hooks I will re-open this, however, it does not seem like a bottleneck when the provided stages are properly separated.

@zjd1988 I found more problems with the code you mentioned the other day, and the above is PR I am working on to fix the bottleneck on that part, just in case you are interested.

@miguelaeh

ok，i will update your latest codes and take a look

@miguelaeh Hi miguelaeh, I have checked out latest code, and found you had added aync to execute_path, but I also have a question, is possible the (n+1)th frame processed done ahead of the nth frame in certain scenarios.

Hi @zjd1988 , yes, it is. I have set a maximum of 2 * cpu cores events that can be processed at the same time. See this: https://github.com/pipeless-ai/pipeless/blob/523be41b5127ff21087ee4ed1e0e8d38fb8a856b/pipeless/src/pipeline.rs#L173

Exactly @zjd1988. The changes contemplate the case of stateless and stateful hooks. Now, I think we should implement a sequential vs parallel executor.
Stateless hooks are designed to not maintain any internal state, making them easy to parallelize. Several instances of the same hook process frames at the same time among different processes/threads.
Stateful hooks on the other hand always use the same instance of the hook, allowing it to maintain state. For example, an object counter requires to maintain as the "state" the number of objects. Or trackin models may require to maintain an internal state.
Now, the next step as you mention should be to differentiate between parallel and sequential executor.
BTW regarding stateful and stateless hooks, both types of hooks are fully implemented, however, it is pending to parse stateful hooks yet. I think we will opt for something like a comment at the top of the files when using code files and a special key in the json files when using json so the user can specify something like # make stateful at the top of the hook file.

Update: I just added the mentioned missing parse step and the respective docs section. See https://www.pipeless.ai/docs/docs/v1/getting-started#stateless-vs-stateful-hooks


[Extensibility enhancements] Purchase Credit Memo API page (table 124 "Purch. Cr. Memo Hdr." and table 125 "Purch. Cr. Memo Line")

Expose table 124 "Purch. Cr. Memo Hdr." and table 125 "Purch. Cr. Memo Line" via separate API pages.
Similar to SalesCreditMemLines.Page.al on the sales side and taking into account location code issue https://github.com/microsoft/ALAppExtensions/issues/11818

@JesperSchulz @onbuyuka Could this be added in a minor release also?  How often are minor releases?

Extensibility Enhancement request sent to our application integration experts. They will get back to you :-)

@JesperSchulz  @onbuyuka Guys, please address this.  This really doesn't make any sense.
You just added APIV2Locations.Page.al in 18.1
Also APIV2SalesInvoices.Page.al, APIV2SalesCreditMemos.Page.al, and APIV2PurchaseInvoices.Page.al exist for a long time.
Yet you refuse to add  APIV2PurchaseCreditMemos.Page.al.  What the logic behind this?

@kyrylkov Sales and purchase documents are different in design than locations API. You can check the code yourself, in contrast to locations, purchase document APIs require more work and cannot be handled through a simple bug fix. That's why we kindly ask you to create an idea through https://experience.dynamics.com/ideas/, and if we see that idea gets enough attraction we can add it to our scope and future plans.
In the meantime if you have an urgent need for Purchase Credit Memos APIs you can consider developing your custom API. Here is the documentation for that: https://docs.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/devenv-develop-custom-api

@onbuyuka Onat, thank you for your comments.  You didn't address my main point.   You do have purchase documents already.  Namely Purchase Invoice APIV2PurchaseInvoices.Page.al.   The only one which is missing is its reversal - Purchase Credit Memo.
I don't understand how it is signifincantly complex than adding a complete new locations page.  Both are non-breaking changes extending existing functionality.

@kyrylkov You can check how the Sales Credit Memo API is implemented and how it is different from Sales Invoice API. You'd notice that source table is a buffer table, and there is a logic behind it since it combines Sales Header and Sales Credit Memo Header. We'd need to use the same design for Purchase Credit Memo API too, which requires upgrade code to copy records from Purchase Header and Purchase Credit Memo Header.
This not a question of breaking, it is a question of time and investment.

@onbuyuka Onat, I really appreaciate the explanation.
Let me add where we stand here.  I hope we both agree Purchase Credit Memo is one of the most basic pieces of functionality.
We integrate with multiple accounting and ERP solutions and in ALL, but D365BC, creating a vendor credit memo, means calling a REST endpoint or a SOAP op.   In D365BC, unfortunately, it means becoming experts in proprietary language - AL, creating pages, tables, writing AL code, deploying it to customer, and maintaining all that over time.   In short, the barrier of entry is extremely high, which discourages developers from building D365BC solutions.
At the same time D365BC probably has tens of thousands of clients and hundreds of companies interested in integrating with D365BC.  A large share of these clients and integrators probably require such basic functionality as Purchase Credit Memo via API.
It would take Microsoft x days (less than a week I assume) to implement this feature into the API.  But refusing to do it, means forcing hundreds of these clients and integrators spending disproportianately more time in total to re-implement the same basic functionality over and over again.
When our clients plan upgrades from Xero or QuickBooks to an ERP, we cannot recommend D365BC to them because the barrier of entry to implement basic functionality using API is so high.
I understand that this is a business matter of allocating time and resources, but it looks to me that this will greatly benefit D365BC ecosystem.  I cannot speak for others, but we will have to stop development of our D365BC integration, until the API is improved.
I hope you can draw attention of your program manager to this issue.
Thank you for your support.


Activate miq_task when deliver from miq_queue

This PR:

ensure that any task will have corect started date
if arguments for queued method is Hash than :miq_task_id => <task id> will be added to parameters when delivering

blocked by:

[ ] https://github.com/ManageIQ/manageiq-schema/pull/167 - Add miq_task_id column to miq_queue table
[ ] https://github.com/ManageIQ/manageiq/pull/17006 - adds associations between  MiqTask and MiqQueue

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1543289

\cc  @mkanoor

\cc @Fryguy

@miq-bot assign @Fryguy

@Fryguy

for existing messages in the queue task's update_status will be skipped and :miq_task_id => nil will be added to the parameter Hash when delivering message.
this PR changing task's status before delivering and all task's specific logic happened after that.  If tasks already setting active status and dealing with task's id than this code will become redundant, but would not inflict harm.
Examples of dealing with tasks in reporting:
when queueing,
here: https://github.com/ManageIQ/manageiq/blob/fbdd937efefcbc407d32136183417b95de761f97/app/models/miq_report/generator.rb#L172
and here: https://github.com/ManageIQ/manageiq/blob/fbdd937efefcbc407d32136183417b95de761f97/app/models/miq_report/generator/async.rb#L13
when delivered: https://github.com/ManageIQ/manageiq/blob/fbdd937efefcbc407d32136183417b95de761f97/app/models/miq_report/generator/async.rb#L23

also, if there is no willingness to use generic_action_with_callback (as in above examples) than adding :miq_task_id to parameters hash for  MiqQueue.put will also trigger activating task.

@miq-bot add-label  gaprindashvili/no


[EFR32] Added support for MG24(BRD4186C and BRD4187C) on RS911x

Problem
Added board configs for MG24 (4186C and 4187C) in rs911x
Change overview
Boards (BRD4186C/4187C) have been added in rsi_board_configuration.h file to support on RS911x
Testing
Tested Manually using MG24+RS911x

fast tracking. It has been open for three days and posted in the tsg-tt-development channel for reviews.
It is only a submodule update for efr32.