Log web sandbox creation works only with proxy

When submitting jobs with token only using:
jobsub_submit --debug -G fermilab --memory 500MB --disk 500MB --auth-methods token file:///usr/bin/sleep 300

in the submission log we see
[...]
Transferring files to web sandbox...
gfal-mkdir error: 13 (Permission denied) - HTTP 401 : Authentication Error
warning: error creating sandbox, web logs will not be available for this submission: Error: Unable to make directory https://fndcadoor.fnal.gov:2880/fermigrid/jobsub/jobs/2024_04_04/4111a9ea-f1e1-432d-a4f3-31109f3fee22
[...]
Checking if fermilab_5c91474037 -S ",compute.read,compute.create,compute.cancel,compute.modify,storage.read:/fermilab/users/vito,storage.create:/fermilab/users/vito,storage.create:/fermigrid/jobsub/jobs," credentials exist
[...]

the token scope is supposed to have write access for the web sandbox storage.create:/fermigrid/jobsub/jobs
so the error seems to imply the code creating the web sandbox is using proxy authentication.
As result the job runs and is able to upload its log, I guess this time using tokens for auth,
but in the web sandbox we are missing:
simple.cmd
simple.sh
sleep

If the submission doesn't use --auth-methods token all is working as expected, the web sandbox is created at job submission time and all expected files are present in there.

So, I'm thinking maybe the token with storage.create doesn't let us make directories under the /fermigrid/jobsub/jobs area?   And even then, we probably need to exempt that from the token weakening... (i.e. act like we got "--need-storage-modify /fermigrid/jobsub/jobs")

The fermilab Analysis token does have the following scope:
storage.create:/fermigrid/jobsub/jobs
See https://landscape.fnal.gov/monitor/d/56WoQ2cVk/token-scopes-capability-sets?orgId=1&var-experiment=fermilab&var-role=Analysis
Does gfal-mkdir, or the version present on interactive nodes, default to proxy authentication or tokens?

Hm... it will use tokens, but I think it needs BEARER_TOKEN set, as opposed to just BEARER_TOKEN_FILE...   And that's our problem.   If I set BEARER_TOKEN to the contents of my token file, the submission works.  So we just need to do that in fake_ifdh.py...


Bug in graphql-tools 3.0.1 - update to latest 3.0.3

There is a bug in graphql-tools that has been fixed in 3.0.2+ renders fragments in the query fairly broken.
https://github.com/apollographql/graphql-tools/issues/777
There are also a number of additional bug fixes added in 3.0.3.
Please upgrade to latest (also consider changing package.json version to "graphql-tools": "^3.0.3" - so we can get latest updates easier.
Many thanks.

Thanks for the suggestion, @develomark - we will investigate and resolve the merge conflict for our renovate automation around this workflow soon 🙂

Thanks!

This was released in 2.1.1 🎉


Roll engine 995e3247fe50..d2eb6efebb9c (4 commits)

https://github.com/flutter/engine/compare/995e3247fe50...d2eb6efebb9c
git log 995e3247fe50..d2eb6efebb9c --no-merges --oneline
d2eb6efeb Roll src/third_party/skia a5d2a9486fed..3b0a332fd970 (1 commits) (flutter/engine#7331)
e2ca868c3 Roll src/third_party/skia 3dc21675b05a..a5d2a9486fed (1 commits) (flutter/engine#7330)
cdc7cc8d3 Roll src/third_party/skia 739b760deaa6..3dc21675b05a (1 commits) (flutter/engine#7329)
e548f12e4 Roll src/third_party/skia aec8c7e280cb..739b760deaa6 (1 commits) (flutter/engine#7328)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/5005556605517824 , https://cirrus-ci.com/task/6412931489071104 , https://cirrus-ci.com/task/5287031582228480

Commit queue failed; closing this roll.


Uml 1301 :  Content change and hint addition

Purpose
https://opgtransform.atlassian.net/browse/UML-1301
Fixes UML-1301
Approach
Explain how your code addresses the purpose of the change
Learning
Any tips and tricks, blog posts or tools which helped you. Plus anything notable you've discovered about the Use a Lasting Power of Attorney service
Checklist

[X] I have performed a self-review of my own code
[ ] I have added relevant logging with appropriate levels to my code
[ ] I have updated documentation (Confluence/GitHub wiki/tech debt doc) where relevant
[ ] I have added tests to prove my work
[ ] I have added welsh translation tags and updated translation files
[ ] I have run an accessibility tool on any pages I have made changes to and fixed any issues found
[ ] The product team have tested these changes

Closing this PR and creating new one for the same. This has some mistakes in the pot.


[Search v2] [App] Use new query syntax

Implement this section of the design doc.
Holding on:

[ ] https://github.com/Expensify/App/issues/45025

cc @adamgrzybowski @Kicu @WojtekBoman

still on hold here

Ready to be worked on!

@luacmartins Looks like something for me!

I'm actively working on this one

The draft is here https://github.com/Expensify/App/pull/45617. I hope to finish it tomorrow

@rayane-djouah sent you an offer on Upwork! https://www.upwork.com/nx/enterprise-homepage/?offerRid=103425705&nav_dir=replace&teamUid=all_teams&tab=hire


Binary releases distribution

I propose to include Go binary releases also at github releases. Some enterprise firewalls block everything from dl.google.com.
Thanks..

I think this is a duplicate of #15966.


misc: replace deprecated variable HUGGING_FACE_HUB_TOKEN with HF_TOKEN

Motivation
ref https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables
cc @merrymercy @Ying1123 @hnyls2002
Modification
as titled
Checklist

Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
Modify documentation as needed, such as docstrings or example tutorials.

Deprecated Variable
Replacement

HUGGINGFACE_HUB_CACHE
HF_HUB_CACHE

HUGGINGFACE_ASSETS_CACHE
HF_ASSETS_CACHE

HUGGING_FACE_HUB_TOKEN
HF_TOKEN

HUGGINGFACE_HUB_VERBOSITY
HF_HUB_VERBOSITY


export a list of the GO slim term IDs

put it somewhere where we could link to it from here
~/fission-yeast-go-slim-terms

So just a file with the IDs from this page?:
https://www.pombase.org/browse-curation/fission-yeast-go-slim-terms
Should it have a column with the term names?

Yes ID and term would be useful.

OK,
Can we add an FAQ "Can I access the term IDs used in the PomBAse GO slims"
and
a link from here:
https://www.pombase.org/datasets

Please open a website ticket for any FAQ change.
Closing because Kim has done the originally requested thing.


Add style merging

Addresses issue #14
It now only safely patches Axlsx::Styles.add_style to implement a style cache. Style index had to be moved from the Workbook to Styles because you cant access the workbook from the add_styles method. Tests are all passing but new ones added for this probably aren't super meaningful yet.

This is looking very good! I'll try to take a closer look this or next week.

I like how clean this change is looking and this seems to be very close to the end goal :smile:
Lets make sure the new tests that you've added produce the results we want. I'll add some comments in the test file.

I think the Hash method key might not be doing a good job comparing hashes I am going to change this and see what happens.

Okay so I fixed the style merging in the latest commit. I had forgotten to add the cell styles from the index to the raw_styles before applying it to the cell.
As for the borders issue, I noticed that for test_merge_styles_1 that the problem was fixed when I switched from:
sheet.add_styles 'B2:D4', borders: { style: :thin, color: '000000' }
to this:
sheet.add_border 'B2:D4', { style: :thin, color: '000000' }
I would appreciate your insight as to why the first statement is failing.

Actually I just checked with the current release v0.1.5 and the sheet.add_styles 'B2:D4', borders: { style: :thin, color: '000000' } is broken here as well. I suspect that you have incorrect documentation regarding this or add_styles need to be fixed to use the BorderCreator.

I'm really glad you've fixed it! I'll take a closer look some time soon.
I apologise that I was wrong about the border. If we use
sheet.add_border 'B2:D4', { style: :thin, color: '000000' }

this goes to border creator which only puts up an outer border. But when we use
sheet.add_styles 'B2:D4', borders: { style: :thin, color: '000000' }

this would apply the given style to every cell in the range, i.e. put border around each cell in B2:D4. I think when I was making comments last time I misread add_styles for add_border. Sorry about that.

I think this is good to go. Good job!
There's one small comment that I've left in the tests (re: testing merge of old and new styles). We can merge when that's cleared.

Alright I have added those changes. With these new features maybe you want to bump this gem to v1.0.0

Well done! Thank you for your contribution!


Preview version installed via WinGet doesn't allow setting as default terminal application.

Windows Terminal version
1.14.1432.0
Windows build number
19042.1706
Other Software
No response
Steps to reproduce
Open PowerShell
Run winget install --id=Microsoft.WindowsTerminal.Preview -e
Open Terminal Preview
Open Settings
Note lack of "Default terminal application" prompt
Expected Behavior
No response
Actual Behavior
The Settings does not have the "Default terminal application" prompt.

Yep, this is by design. The Default Terminal feature is only available on Windows 11.

Good catch, fixed in https://github.com/MicrosoftDocs/terminal/pull/551


NaN inverse raises unexpected ZeroDivisionError

Reporting a bug

[x] I have tried using the latest released version of Numba (most recent is
visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
[x] I have included a self contained code sample to reproduce the problem.
i.e. it's possible to run as 'python bug.py'.

from numba import njit
import numpy as np

@njit
def foo(x):
    return x**-1

foo(np.array([np.nan])) # array([nan])
foo(np.array([np.nan], np.float32)) # ZeroDivisionError: division by zero

Tested under python 3.10.9. This behavior appears in 0.57.1, but is not present in 0.56.4. Setting fastmath=False per #8889 doesn't change this behavior.

Thanks for the report. Unfortunately this doesn't seem to reproduce for me, I've tested it on a couple of systems with different configurations. There are some issues with pow as noted in #7012, it might be another issue in the implementation that moving to LLVM 14 has highlighted.


Close issues

like this:

smh... Using Github for < 24 hours and already telling someone how to use github, who has been using it since 2013...


Use document/description/definition consistently

This tries to be consistent about using "describe" rather than "define" and using "description" for the entire potentially-multi-document thing, and "document" (or sometimes "description document") to refer to individual documents in the description.

There are still a couple instances of "OpenAPI definition" in the file:

Line 13
Line 2278

Should those be changed to "OpenAPI description"?

@mikekistler

Should those be changed to "OpenAPI description"?

Probably yes!  Good catch- I'll take a look and update.


Fix a bug in build_visit building CGNS on Linux.

Description
Resolves #5846
I fixed build_visit so that it builds CGNS properly on Linux. Kathleen discovered that removing the setting of LDFLAGS and LIBS for hdf5 solved the problem. Since these were added specifically for OSX, I made the setting of them dependent on being on OSX. This doesn't impact building on OSX and solves building on Linux. The reviewer may want to specifically test on OSX, just to make sure.
Type of change

[X] Bug fix
[ ] New feature
[ ] Documentation update
[ ] Other

How Has This Been Tested?
I ran the following on quartz and it built CGNS properly.
./build_visit3_2_1 \
    --no-thirdparty --no-visit \
    --cgns --hdf5 --szip --zlib \
    --makeflags -j4

Checklist:

[X] I have followed the style guidelines of this project.~~
[X] I have performed a self-review of my own code.~~
[X] I have commented my code where applicable.~~
[X] I have updated the release notes.~~
[ ] I have made corresponding changes to the documentation.
[ ] I have added debugging support to my changes.
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] I have confirmed new and existing unit tests pass locally with my changes.
[ ] I have added new baselines for any new tests to the repo.
[X] I have NOT made any changes to protocol or public interfaces in an RC branch.~~
[X] I have assigned reviewers (see VisIt's PR procedures for more information).~~

Am testing on my macOS system. Took me a while to get to a clean state to test it.

Ok, so this is failing on macOS...
Invoking command to configure CGNS
    env CXX="clang++" CC="clang"         CFLAGS="-O2" CXXFLAGS="-O2"         LDFLAGS="-L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/hdf5/1.8.14/i386-apple-darwin18_clang/lib -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang/lib -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang/lib" LIBS="-lhdf5 -lsz -lz"         ./configure --enable-64bit --enable-cgnstools=no --enable-shared=all --with-hdf5=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/hdf5/1.8.14/i386-apple-darwin18_clang --with-szip=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang --with-zlib=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang --with-fortran=no --prefix="/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/cgns/4.1.0/i386-apple-darwin18_clang"
CGNS configure failed. Giving up
Unable to build or install CGNS. Bailing out.

% tail -f build_visit_log
Done with HDF5
Done building HDF5
Building CGNS (~2 minutes)
prepare_build_dir: CGNS-4.1.0/src CGNS-4.1.0.tar.gz
Unzipping/Untarring CGNS-4.1.0.tar.gz . . .
Patching CGNS 4.1.0
patching file CGNS-4.1.0/src/configure
patching file CGNS-4.1.0/src/Makefile.in
Configuring CGNS . . .
env: -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang/lib: No such file or directory

That's surprising. The logic should have been the same for OSX and from looking at the configure line, it's what I would expect it to be on OSX. Does the latest build_visit on develop work on OSX?

Does the latest build_visit on develop work on OSX?

It should but haven't tested it.
I think the configure command that was constructed should have worked. However, looking at what CGNS' configure logic actually did, it appears to have wound up stripping the -L<path-to-hdf5-lib-dir> when constructing its internal CLIBS variable as in make.defs...
CC      = clang
CFLAGS  = -O2
COOUT   = -o $(SPACE)
CEOUT   = -o $(SPACE)
LDFLAGS = -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/hdf5/1.8.14/i386-apple-darwin18_clang/lib -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang/lib -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang/lib
CLIBS   = -lm -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang/lib -lsz -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang/lib -lz -lhdf5 -lsz -lz

I just tested on develop and the same build_visit command works on my macOS system.

How relevant is using build_visit3_2_1 here? I am using the build_visit that is in the respective branch.

Diffing build_visit log output on develop and this branch, I see nothing relevant...

I don't think using build_visit3_2_1 should have any bearing. It's just the name of the script.

There is a difference in where you handle quoting for space-separated args to env. variables. In the older code (before your update), the quoting needed to surrounded space-separated items in LDFLAGS and LIBS was handled in the invocation of the ./configure command. In the new code, it is handled in the definition of the new variable, LDFLAGS_DARWIN. I wonder if this can lead to something that visually appears correct in the echo'd commands and logs but is nonetheless not getting parsed in the shell command to configure CGNS correct?

Yes, I agree, that is probably what is happening.

Ok, I am trying a few changes.

A problem with our build_visit log messaging is that we're not using the shell's built-in set -x echo feature in all cases. As a result, we rely upon build_visit's info function to echo the commands that are being executed (which is wholly duplicative too as literally every bv_xxx.sh file has the configure or cmake command codified there twice, once to echo to the log and the other to actually invoke the command) and the quoting is handled differently and can ultimately show a false statement of the command's actually executed by the shell. This should really be removed from build_visit and we should instead bracket sections of code where the shell is doing something we want logged with set +x and set -x.

I've been experimenting with the script and I believe we need another set of double quotes around all the list of "-L" and the list of libs.
In other words I think it is doing something like (shortened to relevant portion of the command):
env LDFLAGS=-L/usr/local/lib -L/usr/lib ./configure
when it should be doing
env LDFLAGS="-L/usr/local/lib -L/usr/lib" ./configure

Totally agree...after adding set -x ahead of the configure invokation I see...
env CXX=clang++ CC=clang 'CFLAGS=-fno-common -fexceptions -O2' 'CXXFLAGS=-fno-common -fexceptions -O2' 'LDFLAGS="-L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/hdf5/1.8.14/i386-apple-darwin18_clang/lib' -L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang/lib '-L/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang/lib"' 'LIBS="-lhdf5' -lsz '-lz"' ./configure --enable-64bit --enable-cgnstools=no --enable-shared=all --with-hdf5=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/hdf5/1.8.14/i386-apple-darwin18_clang --with-szip=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang --with-zlib=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang --with-fortran=no --prefix=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/cgns/4.1.0/i386-apple-darwin18_clang

@markcmiller86 I believe I have a fix, can you try it again.

Still failing. I am sure I got the most recent code. Let me fiddle too. Maybe my initial fix was just getting lucky somehow.

I am not sure we can rely upon shell expansion of a variable to achieve env VAR="val1 val2 val3". I think we need to have different invokation lines as in...
if [[ $OPSYS == "Darwin" ]] then
    env LDFLAGS=\"$LDFLAGS_ENV\" ./configure ...
else
    ./configure ...
fi

Using set -x ahead of configure invokation yields...
env CXX=clang++ CC=clang 'CFLAGS=-fno-common -fexceptions -O2' 'CXXFLAGS=-fno-common -fexceptions -O2' 'LDFLAGS=\"$LDFLAGS_ENV\"' 'LIBS=\"$LIBS_ENV\"' ./configure --enable-64bit --enable-cgnstools=no --enable-shared=all --with-hdf5=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/hdf5/1.8.14/i386-apple-darwin18_clang --with-szip=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/szip/2.1/i386-apple-darwin18_clang --with-zlib=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/zlib/1.2.11/i386-apple-darwin18_clang --with-fortran=no --prefix=/Users/miller86/visit/visit/src/tools/dev/scripts/third_party/cgns/4.1.0/i386-apple-darwin18_clang

I have a meeting now. Will fiddle with after.

I'll go with the 2 different shell invocation lines.

@markcmiller86 I went with 2 different shell invocations for the OSX and non OSX cases. No more reliance on shell variable expansion trickery. This should work!

It tested it on quartz and it worked, but that was the easier case.

Testing 🤞🏻

Thats a winner @brugger1


how to config flow-layout for global or in route group?

how to config flow-layout for global (all route) or in route group?

There is no way to set it globally. You need to run it on each route's action.
But, if you want you can run flow-layout inside a middleware as well.

Thanks, pl example.

I'm curious about this myself. How do I define a global middleware? There's a possibility to define global subscriptions, but what if I want to check for a user field on all routes? Is this done on a route by route basis? Eg. would I have to apply a middleware to all my routes?

So I've been looking into a way to do this, and I've come with a decent pattern. I'm brand new to Flow Router so I'm interested in @arunoda's opinion on the matter.
Using _.extend for shared defaults:
// Default route options
var defaults = {
  layout: 'mainLayout',
  action: app.routes.action
};

// Home
FlowRouter.route('/', _.extend({}, defaults, {
  name: 'home'
}));

This allows easy DRY route definition, but the _.extend bit feels a bit verbose to me.
I tried to find a way to do it in the new triggers system. The problem is that you can't assign arbitrary properties on a group:
var publicRoutes = FlowRouter.group({
  name: 'public',
  layout: 'mainLayout'
});

// Home
publicRoutes.route('/',  {
  name: 'home'
});

FlowRouter.triggers.enter([
  function(context){
    console.log(context.route.group.layout); // undefined
  }
]);

So I would have to say it's not possible in a trigger yet, but I could be wrong. @arunoda would you mind weighing in when you have some time?

I close this because this is a question. Feel free to discuss.


[WIP] Fix thrift-gen handler.Process()

Fixed issue #184 .

I passed unit tests in my local environment (go1.9 darwin/amd64), but somehow it failed on TravisCI, could someone help me to find the reason?

what is the fix, exactly? it looks like you just upgrading to thrift master, while we have  version: ">=0.9.3, <0.11.0"

@yurishkuro Okay, I see the problem, but why not use the latest version of thrift? Is there some specific reason of it?

because the latest version of thrift is not compatible and has no semver

to put it differently, most of our users are using proper, semver-based dependency management, not the go get style.

I see. I would be grateful if you could share a little experience about the dependency management that you are using.

we're using glide, it's explained in the readme

Understood. It seems that there is no need to make this change anymore. Thank you for your time.


CCM interpretation

@PengTao-HUST  Thanks for developing this excellent library. I have a few questions.

In the CCM , do you compute the bi-variate cause and effect ? for example, In the air-pollution example , how do we choose the cause and effect  variable ?  or it is computed bi-directionally for the timeseries. for example NO2 --> RSP and then RSP ---> NO2 in the airpollution example ?

How do we interpret the following causal scores from the heatmap chart? for instance, RSP to NO2 is .605 , which is the cause here and which is the effect ?

What is the min and max for the scores? 0 to1 or -1 to 1 ?
Thanks in advance.

Thanks for your comments. Here are my answers:

For all algorithms in this package, causal strength is computed bi-directionally, i.e., all possible causalities are considered for all variables of the time series.
The label of x(y)-axis of the heatmap stands for the cause (effect) variable. Thus, in the case of "RSP to NO2 is 0.605", NO2 is cause and RSP is effect.
In this package, the calculated causal scores of all algorithms are normalized between 0 and 1, except GC and TE, which are 0 to infinity.

I hope above answers are helpful to you!

@PengTao-HUST  thank you for clarifying.


Wrong document command to start UI

Start the UI has the following command but server name is not functioning it's fnserver
Wrong Command
docker run --rm -it --link functions:api -p 4000:4000 -e "API_URL=http://api:8080" fnproject/ui
Correct command
docker run --rm -it --link fnserver:api -p 4000:4000 -e "API_URL=http://api:8080" fnproject/ui

I'm confused, what is fn-project's relationship to iron functions? They look very similar.


stylix plymouth get 404 for nix-snowflake.svg

Recently I tried using a config flake that builds a NixOS image, and it fails thusly:
error: builder for '/nix/store/q8kb6c881sbgszfxbi73i4vfwp918glp-nix-snowflake.svg.drv' failed with exit code 1;
       last 7 log lines:
       >
       > trying https://raw.githubusercontent.com/NixOS/nixos-artwork/master/logo/nix-snowflake.svg
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0    14    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (22) The requested URL returned error: 404
       > error: cannot download nix-snowflake.svg from any mirror
       For full logs, run 'nix log /nix/store/q8kb6c881sbgszfxbi73i4vfwp918glp-nix-snowflake.svg.drv'.
error: 1 dependencies of derivation '/nix/store/i838vqy1xn96ll9kbp6df03jk2i0hq5f-stylix-plymouth.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ha8xhw5ajhgrr3dg93yi1xgw44gizvfc-plymouth-themes.drv' failed to build

Looking at the master branch  of  nixos-artwork, it seems that logo has become two variants, coloured and white.
And if my interpretation of git commits are correct, files were renamed 17 hours ago, not long before I started my build. :)

Looking at the master branch of nixos-artwork, it seems that logo has become two variants, coloured and white.
And if my interpretation of git commits are correct, files were renamed 17 hours ago, not long before I started my build. :)

Thanks for the digging. This made it a lot easier for me.
Does https://github.com/danth/stylix/pull/335 resolve the issue?

In the long term we may wish to use the logo from pkgs.nixos-icons rather than downloading a separate copy.

This issue still affects release-23.11. Is there any appetite to support that branch?

This issue still affects release-23.11. Is there any appetite to support that branch?

This has already been requested in https://github.com/danth/stylix/pull/335#issuecomment-2082812402.
I opened a tracking issue for this: https://github.com/danth/stylix/issues/356.


Code examples on website are currently broken in Chrome Canary

Probably nothing that can be done on  your side, but the https://react-table.js.org website examples recently (I think) broke in Chrome Canary

Chrome Canary version: 64.0.3242.0

What are the steps to reproduce the issue?

Use chrome canary
Try to view examples
Examples broken

Errors that are displayed on the page
×
TypeError
Super expression must either be null or a function, not undefined
_inherits
https://oyrl04z4y.codesandbox.io/index.js:25:113
eval
https://oyrl04z4y.codesandbox.io/index.js:31:3
eval
/index.js:9:18
   6 | import ReactTable from "react-table";
   7 | import "react-table/react-table.css";
   8 |
>  9 | class App extends React.Component {
     |                  ^
  10 |   constructor() {
  11 |     super();
  12 |     this.state = {
View compiled
▼ 10 stack frames were expanded.
__webpack_exports__.a
https://codesandbox.io/static/js/sandbox.a9951f73.js:1:392234
e.value
https://codesandbox.io/static/js/sandbox.a9951f73.js:1:416894
e.value
https://codesandbox.io/static/js/sandbox.a9951f73.js:1:394138
e.value
https://codesandbox.io/static/js/sandbox.a9951f73.js:1:393960
(anonymous function)
https://codesandbox.io/static/js/sandbox.a9951f73.js:1:382430
t
https://codesandbox.io/static/js/common.b6caa094.js:1:9246
Generator._invoke
https://codesandbox.io/static/js/common.b6caa094.js:1:10292
Generator.e.(anonymous function) [as next]
https://codesandbox.io/static/js/common.b6caa094.js:1:9425
t
https://codesandbox.io/static/js/common.b6caa094.js:1:3953
(anonymous function)
https://codesandbox.io/static/js/common.b6caa094.js:1:4049
▲ 10 stack frames were expanded.
This screen is visible only in development. It will not appear if the app crashes in production.
Open your browser’s developer console to further inspect this error.
This error overlay is powered by `react-error-overlay` used in `create-react-app`.

Please read this FAQ (it is a codesandbox problem and not react-table)....
https://github.com/react-tools/react-table/wiki/FAQ#some-samples-on-codesandboxio-are-breaking


Why multiply the -1 this line?

https://github.com/KatherLab/STAMP/blob/828d1496c673ad782e306a55e39a056d73ba1b08/stamp/heatmaps/__main__.py#L216

Inverse colormap RdBu


fix(cli/serve): Spread WebSocket connectionParams to upgrade request headers

Description
Spread the ConnectionInit.payload (aka connectionParams) values in the WebSocket HTTP Upgrade request headers for a non-breaking fix of lacking authentication requirements.
Even through tampering with original request's headers feels illegal, it should be considered safe in this case since browsers dont allow specifying custom upgrade request header values when initialising a WebSocket connection. Because of this, passing along necessary header-like information is recommended to be done through the first WebSocket message inside an established connection, making the connectionParams synonymous to the upgrade request headers.
Note: Only the connectionParams.headers field is spread. Two reasons behind it:

Explicitness of where the values would end up. Using the headers field as an indicator of where the values would end up.
@graphql-tools/url-loader adds the headers inside the connectionParams.headers

Fixes #2638
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] ~New feature (non-breaking change which adds functionality)~
[ ] ~Breaking change (fix or feature that would cause existing functionality to not work as expected)~
[ ] ~This change requires a documentation update~

How Has This Been Tested?
@jgoux monkey patched the node_modules following the relevant changes and achieved the desired results.
Checklist:

[x] I have followed the CONTRIBUTING doc and the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests and linter rules pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

Thank you @enisdenjo 🙏

Thanks both of you ! 🥰


Where do we find the version in the code

Where do we find out which version we are using?
I do not see any references from the version number tagged in git to define/consts in the code.
As an example we may have 2.5.xx but since we can't see the patch number, but can't know if an update would fix any issues.
It is possible to add the tagged version number to lfs.h? Even a simple comment would be very helpful :-)
Please?
BR
Fink

Hi @Rasmus-Fink
This is not currently committed into the codebase. The patch numbers are informative, but the concern is committing them into the code would create a large amount of commit noise.
With git, the commit hash provides much more info than a patch number ever could.
Or if you copy the littlefs source into another codebase, you can always add the patch number/version to the relevant commit to help track what the most recent patch version was. This is what we do in littlefs-fuse (example).

Ok. The trouble is that we use the version that ST included with their CubeMX stuff and currently we have NO way of telling from the source code what version we are using...
There is only a define:
#define LFS_VERSION       0x00020005
-which does not tell us much...
Should you ever consider using a version number that you update when to create a new tag, then that would be highly appreciated :-)
Thanks for the reply :-)

Hmm. I'll have to think about this. It's a good point many users don't control how they get a copy of littlefs. Maybe the onus is on CubeMX to provide this?
Some of the problem is littlefs patch releases can be as small as one or two commits. We don't really want to force patches to be bigger, since it's good to get bug fixes out quickly, even if small, but the more overhead involved in a patch release the more things slow down...
In the meantime this command should be able to find the relevant patch tag if you check out littlefs locally:
git describe --contains $(git log --find-object $(git hash-object ../my-copy/lfs.c) --reverse --format="%H" | head -1)

Or if you just want the commit hash:
git log --find-object $(git hash-object ../my-copy/lfs.c) --reverse

Hi again,
Thanks for at least considering it.
As I see it a patch release does not have to be big. The main thing is that a bug gets squashed when found.
I don't know what numbering scheme you are using, but if you use semantic versioning 2.0 (https://semver.org/), then it should be more clear that a patch number does not need to be big - as long as the version number gets updated.
BR

@Rasmus-Fink,

we use the version that ST included with their CubeMX

How do you get LittleFS from STM32CubeMX? I don't see it.


Head texture never updates

Going to https://crafthead.net/cube/0c88a1d2eb944175a90ff57db901937d shows my skin from Halloween when I haven't had that skin since 2021 https://namemc.com/profile/HexedHero.1 but my skin never gets updated on CraftHead.

I have already done several full cache purges, and the most recent one was not long ago.
 I'll try and find some time to figure out why it's not working but things are a bit turbulent in my life right now (in a good way!) so it might be a while until I can look at this.

Thanks, glad to hear things are going good and I hope it's an easy fix in the future.

Hm, interestingly I see a very similar issue with my own cape. It has displayed the blank image despite that a) I have a cape enabled on my skin, b) it has shown said cape previously, and c) it works fine on my dev deployment. Looking at the caching there's nothing I can see wrong with the expiration TLLs, but this does still sound like an issue of something cached not expiring correctly. I'll try to see if I can recreate this in some way on my dev deployment as a way to gather more information about it.


CR; Add country of origin to manufacturer info and manufacturer offer list

This field should be a dynamic text based on the customers location.
Example: If customer from US is getting prices from Shapeways, the customer should see "USA"
If a customer from the EU is getting prices from Shapeways, the customer should see "Netherlands"

There are a few open questions left:

What is the design for this.
Where is the list/info of which provider produces where?
Do we still support different production locations? Shapeways will be removed as far as I know.

Production Countries:
Shapeways - USA, EU
i.Materialise - USA, EU
Sculpteo - USA, EU
Treatstock - USA, EU
Jawstec - USA
Harp - USA
Zelta3D - Singapore
Autotiv - USA
Zverse - USA
Incremental Engineering - EU/UK
Hudson Creative Group - USA
Simply3D - EU/UK

@Dansken could you add which is the "default" production country? eg. if someone from china is ordering from shapeways, do you want to show usa or eu?
Or should we always show USA, EU?

I would say that there should always be only one country.
Outside of the EU, Always use USA as a default country.

Design suggestion

@alex-all3dp this would be the data structure. Do you have any feedback:
{
      vendorId: 'imaterialise',
      name: 'i.Materialise',
      description:
        'i.Materialise is one of the leading 3D ....',
      production: {
        default: {country: 'USA'},
        exception: {
          BE: {country: 'EU', hint: 'No import duties'},
          BG: {country: 'EU'},
          CZ: {country: 'EU'},
          DK: {country: 'EU'},
          DE: {country: 'EU'},
          EE: {country: 'EU'},
          IE: {country: 'EU'},
          EL: {country: 'EU'}
        }
      }
    },


Update meteor. Use client, server directories, eat gummy worms.

Done.  Will close with commit later when not on my phone.

I'm no longer on my phone.


app:lintVitalRelease Causing app build to faild

On trying to build apk, I am getting the below error

Task :app:lintVitalRelease FAILED

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:lintVitalRelease'.

Could not resolve all artifacts for configuration ':react-native-community_checkbox:releaseRuntimeClasspath'.
Could not resolve com.facebook.react:react-native:+.
Required by:
project :react-native-community_checkbox
Failed to list versions for com.facebook.react:react-native.
Unable to load Maven meta-data from https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml.
Could not HEAD 'https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml'.
Read timed out

"@react-native-community/checkbox": "^0.5.9",
"react-native": "0.66.1",

`> Task :react-native-community_masked-view:mergeReleaseResources FAILED
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':react-native-community_masked-view:mergeReleaseResources'.

Could not resolve all files for configuration ':react-native-community_masked-view:releaseRuntimeClasspath'.
Could not resolve com.facebook.react:react-native:+.
Required by:
project :react-native-community_masked-view
Failed to list versions for com.facebook.react:react-native.
Unable to load Maven meta-data from https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml.
Could not HEAD 'https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml'.
Read timed out

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9/userguide/command_line_interface.html#sec:command_line_warnings
`

I also had this problem this morning!

I am also facing same issue.

@nicholaslee119 can you please look into this one. I think Jcenter dependency is causing the issue and it has been shut down since early morning today.

I am also facing the same issue.
Can anyone please help ?

I ran into this problem a few weeks ago which sort of resovled itself. It started again this morning.
I believe this is because JCenter is planned to shut down.
If your app (or a dependancy) is hosted in JCenter, you're going to start running into issues.

UPDATE: JFrog has decided to keep JCenter as a read-only repository indefinitely. New package and versions are no longer accepted on JCenter. All Bintray services have been shutdown.

In my case, I started removing dependencies (many were added to the project that weren't even used 🙄) to identify which dependency I needed to migrate off of.  I've been lucky; only 3 deps to get rid of and none of them were used and my builds are succeeding.
TLDR; Read the articles on JCenter shutting down and plan to migrate to maven hosted dependancies.

This is a temporary workaround:
[https://github.com/facebook/react-native/issues/35136#issuecomment-1297071737](Replace all jcenter in node_modules)

@tragicmj
Thanks, everyone, I will check and release this ASAP


Add toggle_attribute Turbo Stream helper

This pull request adds the toggle_attribute Turbo Stream helper on the Ruby side.
Introduced in https://github.com/marcoroth/turbo_power/pull/193

Somehow the CI isn't working anymore, but tests are passing locally. I'm going to merge this so I can cut a release, but need to look into why the CI is not working as expected.


Corrected typo in toolbar builder

Corrected %() to %w()

Euwe Backport details:
$ git log -1
commit ba829cb6fab52c544859adb57bfa61c09ce5d6b5
Author: Oleg Barenboim<EMAIL_ADDRESS>Date:   Tue Oct 18 05:40:46 2016 -0500

    Merge pull request #12011 from romanblanco/typo_fix

    Corrected typo in toolbar builder
    (cherry picked from commit 217176e91655b1ab3ac4f96862071fe55c189d7b)


() in scalar context should be changed to undef

() in scalar context should be changed to undef.  For example:
sub empty { return (); }
my $s = empty();
assert(!defined $s);

Fixed in v1.023


Implement Auto-Retry Mechanism for Python Code Generation

Introduce auto-retry framework for python code generation. The framework can be used on:

df.ai.plot
df.ai.verify
UDF generation.

This PR starts with df.ai.plot. After changes, the plotting will be like:
[CODE GEN 1]
INFO: Getting the following error:
[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column or function parameter with name `start_date` cannot be resolved. Did you mean one of the following? [`start_lat`, `start_lon`, `end_lat`, `end_lon`, `airline`, `airport1`, `airport2`, `cnt`].
INFO: Retrying with 3 retries left
INFO: I apologize for the confusion. It seems there was a misunderstanding in the requirements. If the DataFrame doesn't have a 'start_date' column, we can't filter for flights in February 2011.
[CODE GEN 2]
...

Also, this PR:

improves the prompt of plotting
changes the return type of df.ai.plot from None to str, so that it is easier for users to get the generated code, and easier to test plotting.

"changes the return type of df.ai.plot from None to str, so that it is easier for users to get the generated code, and easier to test plotting."

I'm wondering if we should make it configurable to enhance the experience for our current users. If users are only interested in the plots, having generated code could be an unnecessary complication for them.

I'm wondering if we should make it configurable to enhance the experience for our current users. If users are only interested in the plots, having generated code could be an unnecessary complication for them.

Note: the plotting will be executed anyway. We are returning the string so that users/tests can get the precise result code.

I understand. I was wondering if making the generated code optional will enhance the user experience.


Add trusted certs configuration to TOC and nav

[#145479641]

Thank you, ematpl

Thanks, @animatedmax !


feat(validator): refine custom validator and add example

What this PR does / why we need it:
Update validator.
Which issue(s) this PR fixes (optional, close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
/cc @ddysher @kdada
Release note:

NONE

LGTM
@kdada has more context


Port 5000 is used by MacOSX Ventura 13.6.2

** PLEASE ONLY USE THIS ISSUE TRACKER TO SUBMIT ISSUES WITH THE EXAMPLE VOTING APP **

If you have a bug working with Docker itself, not related to these labs, please file the bug on the Docker repo *
If you would like general support figuring out how to do something with Docker, please use the Docker Slack channel. If you're not on that channel, sign up for the Docker Community and you'll get an invite. *
Or go to the Docker Forums *

Please provide the following information so we can assess the issue you're having
Description
Port 5000 is used by one of the Mac OSX operating system tasks.
salaboy> example-voting-app % lsof -Pn -i4 |grep 5000
ControlCe   912 salaboy    7u  IPv4 0xf95d2afbcbcca679      0t0  TCP *:5000 (LISTEN)

Steps to reproduce the issue, if relevant:

docker-compose up (on a Mac OS M1 machine)

Describe the results you received:
Failed because port 5000 is already used.
Describe the results you expected:
It should run, port 8000 works ok
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.21.1 (114176)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:50:59 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:
Client:
 Version:    24.0.2
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.0
    Path:     /Users/salaboy/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.19.1
    Path:     /Users/salaboy/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/salaboy/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/salaboy/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/salaboy/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/salaboy/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/salaboy/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.16.1
    Path:     /Users/salaboy/.docker/cli-plugins/docker-scout

Server:
 Containers: 18
  Running: 1
  Paused: 0
  Stopped: 17
 Images: 105
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 7
 Total Memory: 38.87GiB
 Name: docker-desktop
 ID: b75ba505-ede3-4708-aaba-013e43e7a0f8
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  <IP_ADDRESS>/8
 Live Restore Enabled: false

Additional environment details (AWS, Docker for Mac, Docker for Windows, VirtualBox, physical, etc.):

Hi @salaboy,
could https://forums.developer.apple.com/forums/thread/682332?answerId=678289022#678289022 be of help, in the mean time?
Matteo

Changing the port of "vote" to 8000 worked for me.


Redirect HTTP to HTTPS

Dunno if the best way to do it but it works :D

Just confirmed that it works as intended.  I'll add on to this after I merge it.


user12986714: Watch This is not italic</i>, and [this is not a link](https://example.com)(?#markdown formatting guide nonsense copy-pasting trolling)

user12986714 requests the watch of the watch_keyword <i>This is not italic<\/i>, and \[this is not a link\]\(https:\/\/example\.com\)(?#markdown formatting guide nonsense copy-pasting trolling). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

<i>This is not italic<\/i>, and \[this is not a link\]\(https:\/\/example\.com\)(?#markdown formatting guide nonsense copy-pasting trolling) has been seen in 4 true positives, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ


Using Multiple guards with streaming

I am using Gemini 1.5 model , here is the code:
import vertexai
from vertexai.generative_models import GenerativeModel
import vertexai.preview.generative_models as generative_models
from guardrails.hub import(
ProfanityFree,
ToxicLanguage
)
from guardrails import Guard
def generate_and_validate(prompt):
vertexai.init(project="",location="")
model = GenerativeModel("")
generation_config = {
"max_output_tokens":8192,
"temperature":0.2,
"top_p":0.95
}
guard = Guard().use_many(
ProfanityFree(on_fail="exception")
ToxicLanguage(validation_method="sentence",threshold=0.5, on_fail="exception")
)
try:
validated_prompt = guard.validate(prompt)
print(f"Validated prompt: {validated_prompt}")
except Exception as e:
print(f"Invalid prompt: {e}")
return
try:
response = model.generate_content(a
[prompt],
generation_config=generation_config,
stream = True,
)
for chunk in response:
  generated_text = chunk.text
  print("Generated text",generated_text)
  validated_chunk = guard.validate(generated_text)
  print(f"Validated text: {validated_chunk}")

except Exception as e :
print(f"Invalid text: {e}")
prompt = "Write an essay on happiness"
generate_and_validate(prompt)
The code is working fine and generating output chunks as well as validating them but for prompts (like - write an essay on happiness) where there is no profanity and toxic language involved in the input as well as in the output -- I am encountering last line of the output as Invalid text: API must be provided.
whereas if I change the prompt as "Give me synonyms of rude" then after validating the response I am not facing the error API must be provided it's working absolutely fine.
PS: I have used guardrails configure token key correctly.
Kindly help

@Us191101 I'm taking a look at this.  Could you please share what version of guardrails-ai you are using?

I was able to reproduce this issue on v0.4.5.  Will post details and workarounds shortly.

@Us191101, @nichwch, and @zsimjee  Here's what I used to reproduce this error; note that it uses openai so this bug is not Gemini specific:
## OpenAI

import openai

from guardrails.hub import(
    ProfanityFree,
    ToxicLanguage
)
from guardrails import Guard

def generate_and_validate(prompt):
    guard = Guard().use_many(
        ProfanityFree(on_fail="exception"),
        ToxicLanguage(validation_method="sentence",threshold=0.5, on_fail="exception")
    )

    try:
        validated_prompt = guard.validate(prompt)
        print(f"Validated prompt: {validated_prompt}")

    except Exception as e:
        print(f"Invalid prompt: {e}")
        return

    try:
        response = openai.chat.completions.create(
            messages=[
                {
                    "role": "system",
                    "content": "You are a helpful assistant."
                },
                {
                    "role": "user",
                    "content": validated_prompt.validated_output
                }
            ],
            model="gpt-3.5-turbo",
            stream=True,
        )

        for chunk in response:
            generated_text = chunk.choices[0].delta.content
            print("Generated text", generated_text)
            validated_chunk = guard.validate(generated_text)
            print(f"Validated text: {validated_chunk}")
    except Exception as e:
        import traceback
        traceback.print_exception(e)
        print(f"Invalid text: {e}")

prompt = "Write an essay on happiness"
generate_and_validate(prompt)

The cause of the original error is here: https://github.com/guardrails-ai/guardrails/blob/main/guardrails/run/runner.py#L241
When the first chunk from the LLM is an empty string, which is often the case in the example I posted above, the runner tries to use the calling flow instead of the static parsing flow hence the required error for the llm_api argument.  Obviously the fix here is simple: check output is not None instead of just any falsy value.  There are however downstream issues that result even after this fix.  Specifically ToxicLanguage raises ValueError("Value cannot be empty.") in this scenario.  Looking at the validator, this check seems unnecessary but we would need to determine what the correct handling of an empty value would be.
An existing way around this issue is to utilize the builtin streaming support guardrails offers rather than validating each chunk manually in the script.  @nichwch can you add an example of that pattern to this issue?

Hi Us191101, here's how you'd use Guardrail's built in streaming support to get around the existing bug. The gist of it is that instead of consuming from the LLM API and running validation on the chunks, you can call the Guard itself directly, set stream to true, and have both validation results and LLM output streamed out of that returned generator instead.
import openai

from guardrails.hub import(
    ProfanityFree,
    ToxicLanguage
)
from guardrails import Guard
guard = Guard().use_many(
   ProfanityFree(on_fail="exception"),
   ToxicLanguage(validation_method="sentence",threshold=0.5, on_fail="exception")
)

fragment_generator = guard(
    openai.chat.completions.create,
    max_tokens=50,
    temperature=0.1,
    # The key here is to set stream to True.
    # This makes the guard() return a generator, that you can consume from
    stream=True,
)

for op in fragment_generator:
    # This will be a ValidationOutcome object
    print(op)

import vertexai
from vertexai.generative_models import GenerativeModel
from guardrails.hub import ProfanityFree, ToxicLanguage
from guardrails import Guard
def generate_and_validate(prompt):
vertexai.init(project="", location="")
model = GenerativeModel("gemini")  # Replace with your actual model name
generation_config = {
    "max_output_tokens": 50,
    "temperature": 0.1,
}

guard = Guard().use_many(
    ProfanityFree(on_fail="exception"),
    ToxicLanguage(validation_method="sentence", threshold=0.5, on_fail="exception")
)

try:
    validated_prompt = guard.validate(prompt)
    print(f"Validated prompt: {validated_prompt}")
except Exception as e:
    print(f"Invalid prompt: {e}")
    return

try:
    # Use Guard's streaming support directly
    guard_response_generator = guard(
        model.generate_content,
        {"prompt_params": [prompt]},
        generation_config=generation_config,
        stream=True
    )

    for chunk in guard_response_generator:
        if chunk.validation_outcome:
        print(f"Validation outcome: {chunk.validation_outcome}")
    if chunk.generated_text:
        print(f"Generated text: {chunk.generated_text}")

except Exception as e:
    print(f"Error: {e}")

Example usage:
prompt = "Write an essay on happiness"
generate_and_validate(prompt)
@nichwch @CalebCourier , I got your point that if empty string gets passed it was causing the issue, made the changes as per your suggestion, kindly review this; I think I am still missing something here.

Also note that API must be provided error was not occuring when I was using just the toxic language guard. It was dealing fine with empty string as well.
The problem is with profanity guard only.


Versioning options

How should we version this repo? The most straightforward method would be retaining the same version number as the version of murmur installed with the package -- this would allow users to easily docker pull bddenhartog/docker-murmur:<version> and get an image with the appropriate version of murmur.
The other option is to version based on the issues and goals from within this repo directly. This would be more appropriate and would allow version numbers to support specific development targets, but may be more confusing to the end user (as we may quickly out-version the mumble project, and vX of this repo may only correspond to vY of the murmur project.

Thoughts @SISheogorath?

Mhm, it's difficult. As long as we don't break it the versioning like the murmur version would work.
If we break something it's a problem.
Lets take a realistic point of view: How many people actually using this container? I guess less than 50. So we want to take a road where more people can use it. This leads us to the problem that we need a working version. Right now a lot of stuff is working but not everything. What could break in future? As we already have the config in environment variables, the container itself, besides the database, is stateless and logging goes to stdout/stderr there is not much we can break for now. So lets take the original murmur version and add different builds, when we add a feature.
The docker compose stuff can be prefixed with compose- if we want to create a release so the docker build can be ignored. should be easy.

I was thinking along the same lines; latest should always correspond to the latest stable version of murmur, so it makes sense that any tag in this repository (and on the image) corresponds to the appropriate version of murmur.
It's possible (although maybe not ideal) to change the commit that the tag points to. On the docker repository side of things, which would enable something like:
$ docker pull bddenhartog/docker-murmur:1.2.17  # latest version of image based on murmur v1.2.17
> ... changes happen upstream, we add some new feature or bugfix ...
$ docker pull bddenhartog/docker-murmur:1.2.17 # latest version of image based on murmur v1.2.17

I think this is the best workflow from a user-centric perspective, but a horrible workflow from a development perspective, as now we have no way to convey that users should update their image -- e.g. featureX added in v1.2.17 when the user has already pulled v1.2.17 from the repo, without featureX.
How would you want to add the build number, and what should it be based on -- the travis build? the commit SHA? Something like:
image:1.2.17-0064
image:1.2.17-670c69c3
image:1.2.17-stable (first build of 1.2.17? latest?)

I think the issue with using murmur's version, which I do not like, is there is no way for us to differentiate our versioning from theirs. We may add a whole bunch of project specific features for 1.2.17, for example, and sticking with semver and murmur's version there's no way to easily identify those.

I'm more and more coming the conclusion that the "docker magic" stuff we do here "shouldn't matter". Yes, we can't release new versions like you would usually would do it, but in fact Docker handles the "build numbers" itself. So no need to add something like -SHA1. We can simply use the murmur. If we really want we can also add a bugfix branch but as neither me, nor you use this in a high scaled setup where you can't edit the configs if something breaks I guess we can use it as it is. Docker itself handles the container updates and we make sure that it just works ^^

Docker doesn't handle the container updates itself actually, unless the user specifically pulls the image again they won't get a new iteration of whatever tag they have pulled down.
Because the image is built automatically from this repository and I have specified that latest should always be built from the master branch, we don't have to worry about what latest correlates to as that's already decided for us through the defined development workflow.
The thing is, while there isn't currently much in the project besides the image, there are plans for future development, integration of ice and a webui, automatic proxy provisioning, etc. I think it's important that we snapshot the growth of the project somehow, so users can easily grab the image from a specific point in time, and tags (on both the git repo and the built image) are the traditional way to do that.
If this project was only focused on providing 1:1 parity with the options available to murmur, then I guess it would make sense to follow the mumble project's versioning, but because there are things we do/may do that are outside of the scope of simply running murmur, I think it makes sense to stick with our own versions. As long as releases have good visibility, and specify the versions of the dependencies used (e.g. v1.2.3 of this repo uses v.1.2.17 of murmur and v3.4.5 of nginx) I don't think people will be too confused.
There's an interesting discussion regarding updates to versions after dependency updates here, and I think this kind of solidified the decision for me -- even though this project is really just a "wrapper" around murmur at the moment, that's not what it will be in the future and it makes sense to consider murmur (and any other external projects) as just that -- external projects, or dependencies.
As always, thanks for the chat and ideas :)

The stance on versioning changed with #66. containerly/murmur tags now match release versions from the mumble-voip/mumble project.


Copyright holder and license

Hi,
Could you please add LICENSE file and put the copyright holder and license inside each file? This way there is no chance of someone stumbling across a piece of code without being exposed to the license.

Done! #2

Done! #2

Ping @fpaternot e @marcelometal


feat(cli): add selector and field-selector option to the stop command.

Checklist:

[ ] My organization is added to USERS.md.

I add the selector and field-selector option to the stop command, it can stop workflows that are matched by labels and field. it is related to #3962.

@hanjunlee this is failing builds on master, so I'm going to use admin privileges to revert this merge.


Add c# Hello World

Hello World program in C#.

The program should only print "Hello World"
Reference issue 1 please
Name the file csharp.cs


Error's from SendGrid when sending email should throw exception

For non-successful responses from SendGrid, it should throw an error message with the error message received. Interacting with the API should not require you to check the response struct for message: "success" or errors: ['foo'].

Fixed with ffafed627347aac191ee52c403ed04e90c07ea83


Provide a way to alias all services in a server easily

Currently, armeria doesn't provide a way to easily add aliases for all services.
For instance, assume that a user adds services for paths /bar1, /bar2, ... /barN
It might be difficult to also serve /foo/bar1, /foo/bar2 ... /foo/barN without manually adding all of the services.

Redirecting internally could be a solution, but it would add an extra network hop which wouldn't be ideal.
Adding all of the services again may not be ideal if the same functionality is provided just for a different path.

A simple solution may be to add a path transformer like done for ServerBuilder#absoluteUriTransformer
e.g.
sb.absoluteUriTransformer(path -> {
    if (path.startsWith("/foo")) {
        return path.substring("/foo".length());
    }
    return path;
});

Another solution could be to allow users to customize the Router API.
cc. @tobias-
ref: https://discord.com/channels/1087271586832318494/1087272728177942629/1126484753508479026

This has been resolved via https://github.com/line/armeria/pull/5037


Reduce use of QUICConfig::scoped_config

QUICConfig::scoped_config was used here and there, and somehow it brought a dependency for libinknet after the refactoring around QUICMultiCertConfigLoader.
This PR reduces direct use of QUICConfig::scoped_config by introducing QUICTPConfig, QUICLDConfig and QUICCCConfig. Only QUICPacketHandler, QUICNetVConnection and QUIC (global) use QUICConfig directly. In other words, no classes under QUICNetVConnection use it.
The original motivation was removing dependency for libinknet, but unfortunately it still remains because of functions for OpenSSL callbacks. However, I think this change goes in the right direction.

Now QUICConfig::scoped_config is only in QNetVC and QPacketHandlerIn.
Unfortunately the change made QUICGlobal.cc include QUICTLS.h. We should probably move global functions for OpenSSL callbacks to QUICTLS class.


Docs: Table of contents view issue

The documentation (head version) does not correctly display the table of contects for every point under the section "Working with user smart contracts"
https://hyperledger.github.io/firefly/head/tutorials/custom_contracts/ethereum.html

Thanks for catching this. It's like a spacing or indention issue. The TOC plugin in the Markdown -> HTML generator is pretty picky about spacing, and sometimes the auto code formatter in our IDEs actually messes it up.

@nguyer https://github.com/hyperledger/firefly/pull/1409


Install PhantomJS, poltergeist and Capybara

WHY
Currently we cannot make behavior test with Capybara resource due to lack of dependencies.
  1) server 'target' http 'http://target/' with {:params=>{}, :method=>:get, :headers=>{}} capybara 'http://app.example.com' shows something
     Failure/Error: Unable to find matching line from backtrace
     Cliver::Dependency::NotFound:
       Could not find an executable ["phantomjs"] on your path.
     # /usr/local/bundle/gems/cliver-0.3.2/lib/cliver/dependency.rb:143:in `raise_not_found!'
     # /usr/local/bundle/gems/cliver-0.3.2/lib/cliver/dependency.rb:116:in `detect!'
     # /usr/local/bundle/gems/cliver-0.3.2/lib/cliver.rb:24:in `detect!'
     # /usr/local/bundle/gems/poltergeist-1.5.1/lib/capybara/poltergeist/client.rb:36:in `initialize'
     # /usr/local/bundle/gems/poltergeist-1.5.1/lib/capybara/poltergeist/client.rb:14:in `new'
     # /usr/local/bundle/gems/poltergeist-1.5.1/lib/capybara/poltergeist/client.rb:14:in `start'
     # /usr/local/bundle/gems/poltergeist-1.5.1/lib/capybara/poltergeist/driver.rb:42:in `client'
     # /usr/local/bundle/gems/poltergeist-1.5.1/lib/capybara/poltergeist/driver.rb:25:in `browser'
     # /usr/local/bundle/gems/poltergeist-1.5.1/lib/capybara/poltergeist/driver.rb:190:in `headers='
     # /usr/local/bundle/gems/infrataster-0.2.2/lib/infrataster/contexts/capybara_context.rb:38:in `before_each'
     # /usr/local/bundle/gems/infrataster-0.2.2/lib/infrataster/rspec.rb:18:in `block (2 levels) in <top (required)>'

WHAT
Install

PhantomJS 1.9.8
poltergeist gem
to simulate Webkit browser
capybara gem

EXAMPLE
# spec_helper.rb
require "infrataster/rspec"

Capybara.javascript_driver = :poltergeist

Infrataster::Server.define(
  :target,
  ENV["TARGET_IP"],
  vagrant: false,
)

# target_spec.rb
require "spec_helper"

describe server(:target) do
  describe capybara('http://app.example.com') do
    it 'shows page' do
      visit '/'
      expect(page).to have_css("#main")
    end
  end
end

$ docker run --rm --name tester -v $PWD/spec:/test/spec -e TARGET_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' target) quay.io/wantedly/infrataster
.

2 deprecation warnings total

Finished in 1.45 seconds (files took 0.44394 seconds to load)
1 example, 0 failures

@spesnova Please review this :bow: !

It seems that we need to fix CI first.

LGTM :+1:


feat: Adding original query inputs to query results

Closes #112

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#113  👈
#111
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @tazarov and the rest of your teammates on  Graphite


[Version 1.1.0] Pytest exception in thread after tests are run

After all my tests run, I get this output:
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/home/ubuntu/environment/local/lib/python2.7/site-packages/pytest_server_fixtures/base.py", line 46, in run
    if not isinstance(l, string_types):
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types

I'm using a custom fixture class I wrote to bring up postgres;
# conftest.py

@pytest.fixture(scope='session')  # pylint: disable=no-member
def postgres_server_sess(request):
    """ Postgres server with session scope
    """
    server = _postgres_server(request)
    return server

def _postgres_server(request):
    """Take a pytest fixture request and set up a postgres server for it"""
    test_server = PostgresServer()

    _getconn = mock.patch('app.connection_pool.getconn,
                          side_effect=lambda: test_server.connect('db_name'))
    _putconn = mock.patch('app.connection_pool.putconn',
                          side_effect=lambda conn: conn.close())
    _getconn.start()
    _putconn.start()
    request.addfinalizer(_getconn.stop)
    request.addfinalizer(_putconn.stop)

    # For orm session
    _connection_config = mock.patch.object(AppConfiguration, 'db_config',
                                           new_callable=PropertyMock,
                                           side_effect=lambda: dict(test_server.connection_config,
                                                                    password='',
                                                                    database='db_name'))

    _connection_config.start()
    request.addfinalizer(_connection_config.stop)

    test_server.start()
    request.addfinalizer(test_server.teardown)

    return test_server

class PostgresServer(TestServer):
    random_port = True

    def kill(self, retries=5):
        if hasattr(self, 'pid'):
            os.kill(self.pid, self.kill_signal)

    def pre_setup(self):
        (self.workspace / 'db').mkdir()  # pylint: disable=no-value-for-parameter

        self.pg_bin = subprocess.check_output(["pg_config", "--bindir"]).rstrip()
        subprocess.check_call([
            self.pg_bin + '/initdb',
            str(self.workspace / 'db')
        ])
        logger.debug("pg_bin = %s", self.pg_bin)

        self.connection_config = {
            u'host': u'localhost',
            u'user': os.environ['USER'],
            u'port': self.port,
            u'connect_timeout': 3,  # In seconds
        }
        self.delete = True

    def post_setup(self):
        set_up_schema_and_migrations()

    @property
    def run_cmd(self):
        logger.debug("Starting postgres")
        return [
            self.pg_bin + '/postgres',
            '-F',
            '-k', str(self.workspace / 'db'),
            '-D', str(self.workspace / 'db'),
            '-p', str(self.port),
            # '-c', "log_destination=stderr"
        ]

    def check_server_up(self):
        try:
            print("Connecting to Postgres at localhost:{}".format(self.port))
            with self.connect('postgres') as conn:
                conn.set_session(autocommit=True)
                with conn.cursor() as cursor:
                    cursor.execute("CREATE DATABASE db_name")
            self.connection = self.connect('db_name')
            with open(self.workspace / 'db' / 'postmaster.pid', 'r') as f:
                self.pid = int(f.readline().rstrip())
            return True
        except OperationalError as e:
            logger.warn("Could not connect to test postgres: %s", str(e))
        return False

    def connect(self, database):
        return psycopg2.connect(database=database, **self.connection_config)

Hi there - sorry for the terribly long delay in responding :(
I really like your postgres fixture! Would you like me to add it to the repository as part of pytest-server-fixtures, or a separate package pytest-postgresql? I'll sort out making it work across all the pyversions, perhaps could you submit a PR which has some tests in it?

But once I do that, then yes I'll pr it in
On Thu, Oct 27, 2016, 7:36 AM Gavin Bisesi<EMAIL_ADDRESS>wrote:

I'd like to refactor it first
On Thu, Oct 27, 2016, 7:36 AM eeaston<EMAIL_ADDRESS>wrote:
Hi there - sorry for the terribly long delay in responding :(
I really like your postgres fixture! Would you like me to add it to the
repository as part of pytest-server-fixtures, or a separate package
pytest-postgresql? I'll sort out making it work across all the pyversions,
perhaps could you submit a PR which has some tests in it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/manahl/pytest-plugins/issues/21#issuecomment-256617310,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAA5NJP5uOvP8TMqdqhF6Tl0Q3eXe6x6ks5q4IyygaJpZM4I3g36
.

I'd like to refactor it first
On Thu, Oct 27, 2016, 7:36 AM eeaston<EMAIL_ADDRESS>wrote:

Hi there - sorry for the terribly long delay in responding :(
I really like your postgres fixture! Would you like me to add it to the
repository as part of pytest-server-fixtures, or a separate package
pytest-postgresql? I'll sort out making it work across all the pyversions,
perhaps could you submit a PR which has some tests in it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/manahl/pytest-plugins/issues/21#issuecomment-256617310,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAA5NJP5uOvP8TMqdqhF6Tl0Q3eXe6x6ks5q4IyygaJpZM4I3g36
.

Okiedokie, I'll wait for the PR.

Closing this issue as we've merged the postgres fixture into master.


Add paths-ignore filter to push events on main for build workflow

Reference: paths-ignore on GitHub Docs

I would like for #134 to be merged first, so that we can see the new build steps in action.


limit of training data for input.

Hi there,
May I ask if there is any limit for input training data?
for example I want to put 50000 simulated data in the following size: (50000,3000,45)
when I try to run
%%time
losses = trainer.train_offline(epochs=100, batch_size=64, params=true_params, sim_data=x)
I got error:
InternalError: Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run _EagerConst: Dst tensor is not initialized.
is that because of my GPU limit (20G)?
actually all (3000,45) are integer only, is it possible to specify integer array to save ram?

You would need more than 25GB of GPU memory to fit only the simulated data set in memory (excluding network parameters). I suggest you store your simulated data into much smaller chunks which you can then sequentially read from disk and avoid memory blowup.

thanks for answering! I changed the data format to aggregated data and it works better now


Enabled generation of type declarations (.d.ts)

This update affects only changes of TypeScript  compiler options and enables generation of typings.
This should improve autocomplete expirience in modern IDEs like PHPStorm, WebStorm etc, which recognizes typings.
It will be nice if you will also dististribute those typings with NPM as well as source maps.
Fixes #<issue_number_goes_here> (it's a good idea to open an issue first for discussion)

[x] npm test succeeds
[x] Pull request has been squashed into 1 commit
[x] I did NOT manually make changes to anything in apis/
[x] Code coverage does not decrease (if any source code was changed)
[ ] Appropriate JSDoc comments were updated in source code (if applicable)
[ ] Approprate changes to readme/docs are included in PR

Codecov Report

Merging #821 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #821   +/-   ##
=======================================
  Coverage   79.23%   79.23%
=======================================
  Files           4        4
  Lines         236      236
  Branches       53       53
=======================================
  Hits          187      187
  Misses         30       30
  Partials       19       19

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update be324d5...2b384d7. Read the comment docs.

Actually it would be better to add a types property to the package.json so people don't need to install types separately.

Apologies, already covered in #785

@JustinBeckwith No problem. Thanks anyway.


Hash literals in puts statements are parsed differently

When debugging I realized adding puts or pp in front of a nested hash breaks the parser.  There is an easy work around  to assign it to a variable and then print it or to prepend the statement with Hash.  But I think this is unexpected and probably a bug.
https://play.crystal-lang.org/#/r/a2bm
puts { "varibles" => { "input" => "10.00" } }

syntax error in eval:1
Error: unexpected token: =>

https://play.crystal-lang.org/#/r/a2bk
x = { "varibles" => { "input" => "10.00" } }
pp x

https://play.crystal-lang.org/#/r/a2bu
puts Hash{ "varibles" => { "input" => "10.00" } }

Add parenthesis
puts({ "varibles" => { "input" => "10.00" } })

https://play.crystal-lang.org/#/r/a2bv
This is not a bug, puts is a method call and block syntax simply takes precedence.

https://github.com/crystal-lang/crystal/issues/7923#issuecomment-505412984
Add another to the list ;P

Alright, duplicate of #1051 :P


Course Direction

How you are converting floating course values into directions?

Check AttributeFormatter.js#L31.


Can't get file when call Export task dataset API

My actions before raising this issue

[ ] Read/searched the docs
[ ] Searched past issues

Steps to Reproduce (for bugs)

call this url by http request
http://xxx:8080/api/tasks/140/annotations?org=iauto&use_default_location=true&filename=123.zip&format=CVAT+for+images+1.1
2, this is my code

` public static void main(String[] args) {
String fileUrl = "http:/xxx:8080/api/tasks/140/annotations?org=iauto&use_default_location=true&filename=123.zip&format=CVAT+for+images+1.1";
String savePath = "E:\zip/file.zip";
try {
URL url = new URL(fileUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("Authorization","Token xxx");
connection.setRequestMethod("GET");
int responseCode = connection.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_CREATED) {
InputStream inputStream = connection.getInputStream();
            FileOutputStream outputStream = new FileOutputStream(savePath);
            byte[] buffer = new byte[4096];
            int bytesRead;
            while ((bytesRead = inputStream.read(buffer)) != -1) {
                outputStream.write(buffer, 0, bytesRead);
                buffer = new byte[4096];
            }
            outputStream.close();
            inputStream.close();
            System.out.println("ZIP download succeed: " + savePath);
        } else {
            System.out.println("zip download failed");
        }
        connection.disconnect();
    } catch (IOException e) {
        e.printStackTrace();
    }
}`

on cvat page,when click 'Export task dataset', it call this url.I then a xx.zip  was download;
I call the same url by postman or send http request in code,I get a xx.zip,but File corruption displayed when unzipping the file

Expected Behaviour

get a xx .zip, unzipping the file,I will get a useful file
Current Behaviour

get a xx .zip, unzipping the file, it throws  File corruption
Possible Solution

How to receive file stream
Context

Your Environment

Git hash commit (git log -1):
Docker version docker version (e.g. Docker 17.0.05):
Are you using Docker Swarm or Kubernetes?
Operating System and version (e.g. Linux, Windows, MacOS):
Code example or link to GitHub repo or gist to reproduce problem:
Other diagnostic information / logs:
Logs from `cvat` container

First request initiates the dataset preparation on the server side and returns code 202.
Then cvat client pools the server until 201 code is received, after ?action=download is required in URL to get the file.
Please, get acquainted with REST API.


SyntaxError: invalid syntax when running the pynt command

I followed the steps as per https://github.com/aws-samples/aws-etl-orchestrator, but when I run pynt the following error is shown. I tried both python3 and python 2.7 while creating the virtualenv. And having print statement in the same line didn't work either.
(etl-orchestrator) 8c85902107ff:aws-etl-orchestrator saggars$ python -V
Python 3.6.5
(etl-orchestrator) 8c85902107ff:aws-etl-orchestrator saggars$ pynt -v
pynt 0.8.2
(etl-orchestrator) 8c85902107ff:aws-etl-orchestrator saggars$ pynt
Traceback (most recent call last):
  File "/Volumes/Unix/MAC-DEV-DSK/EconETLOrchestrator/etl-orchestrator/bin/pynt", line 11, in <module>
    sys.exit(main())
  File "/Volumes/Unix/MAC-DEV-DSK/EconETLOrchestrator/etl-orchestrator/lib/python3.6/site-packages/pynt/_pynt.py", line 298, in main
    build(sys.argv[1:])
  File "/Volumes/Unix/MAC-DEV-DSK/EconETLOrchestrator/etl-orchestrator/lib/python3.6/site-packages/pynt/_pynt.py", line 48, in build
    module = imp.load_source(path.splitext(path.basename(args.file))[0], args.file)
  File "/Volumes/Unix/MAC-DEV-DSK/EconETLOrchestrator/etl-orchestrator/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 674, in exec_module
  File "<frozen importlib._bootstrap_external>", line 781, in get_code
  File "<frozen importlib._bootstrap_external>", line 741, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "build.py", line 34
    print 'zipping %s as %s' % (os.path.join(dirname, filename),
                           ^
SyntaxError: invalid syntax

Convert all your print statements to use the function and it will fine.  So in your error line above change to:  print('zipping %s as %s' % (os.path.join(dirname, filename),??)

Let me just qualify this for people - if you've created a virtualEnv using python 3.5.6, then yes, you'll need to put all your print statements (there are 32) from:
print 'doing something'
to
print('doing something')

print statements were updated throughout the project to be Python 3 compatible.


Studentfinancecontent2016

Student finance form finder update:
https://www.pivotaltracker.com/story/show/110438992
https://govuk.zendesk.com/tickets/1213218
Six commits that add smart answer outcomes for full-time students in the 2016/17 academic year.
Adding and deleting outcome files, and updating links to new forms.

This PR is superseded by #2296


Issue when attempting to install the suggested version of OpeanAI Gym (gym==0.18.0)

Hello dear Christian and dear Thunfisch,
Another problem that also occurred while attempting to install the OpenAI Gym version 0.18.0 in the karolos conda environment was the following:
Collecting gym==0.18.0
Using cached gym-0.18.0.tar.gz (1.6 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error in gym setup command: 'extras_require' must be a dictionary whose values are strings or lists of strings containing valid project/version requirement specifiers.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Any suggestions on how I might be able to overcome this issue?
Thank you very much for your valuable help!
Kind regards,
Christos Peridis

Looks like gym=0.18.0 made some trouble.
Hopefully this is fixed by using the newest gym version. Closing this issue (duplicate to #5)


[UserAnimeList] season_year and season_name are null for some users

Test case:
GET https://api.jikan.moe/v3/user/nekomata1037/animelist/watching
Every anime has season_year and season_name filled in correctly as expected.
But:
GET https://api.jikan.moe/v3/user/Xinil/animelist/watching
or
GET https://api.jikan.moe/v3/user/Luna/animelist/watching
Every anime has season_year and season_name equal to null.

I've looked into how Jikan gets the animelist data and it looks like it is a bug on MAL's end.
So I'll close this issue.


Use the HF optimizer for Reinforcement Learning (TD3)

Hi,
I tried to use this optimizer firstly only for the actor network of a TD3 agent.
Here is some simplified Code: (actor and qf1 are my NNs)
actor_optimizer = HessianFree(params=self.actor.parameters())

actions = actor(observation)

# ---------------------------------------------
# Training logic for the Q-Values etc.
# ---------------------------------------------

actor_optimizer.zero_grad()
def forward_actor():
    loss = -self.qf1(torch.cat([observations, actor(observations)], 1)).mean()
    return loss, outputs
actor_optimizer.step(forward=forward_actor)

I get the following error message
File ~/anaconda3/envs/ma310/lib/python3.10/site-packages/backpack/hessianfree/hvp.py:45, in hessian_vector_product(f, params, v, grad_params, detach)
     43     df_dx = tuple(grad_params)
     44 else:
---> 45     df_dx = torch.autograd.grad(f, params, create_graph=True, retain_graph=True)
     47 Hv = R_op(df_dx, params, v)
     49 if detach:

RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

I guess its because the loss is the output of a NN. Is there a correct way to define the forward function?
Best,
Felix

Hi Felix, Thanks for opening an issue!

I guess its because the loss is the output of a NN.

In principle, the optimizer should work fine with a loss that depends on the output of a NN.
In your example, some things are not clear to me, e.g. how is outputs in forward_actor defined? Could you maybe provide a minimal stand-alone example such that I can replicate the error?
Best,
Lukas

Hi Lukas,
Thank you for your response.
Sorry my Code was misleading and the error may also be caused by something else.
Here is a minimal stand-alone example to replicate the error:
import torch
from hessianfree.optimizer import HessianFree

BATCH_SIZE = 1
OBSERVATION_SPACE = 32
ACTION_SPACE = 4

observation = torch.randn(BATCH_SIZE, OBSERVATION_SPACE)

QNet = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE + ACTION_SPACE, 1),
)

Actor = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE, ACTION_SPACE),
)

opt = HessianFree(
    params=Actor.parameters(),
    verbose=True
)

def forward():
    actor_losses = -QNet(torch.cat([observation, Actor(observation)], 1))
    loss = actor_losses.mean()
    outputs = Actor(observation)
    return loss, outputs

loss = opt.step(forward=forward)

Best,
Felix

Hi Felix,
The error originates from the implementation of the GGN/Hessian vector products in backpack.hessianfree. Re-installing backpack from the ggn-materialize-grad branch, i.e. pip install backpack-for-pytorch git+https://github.com/f-dangel/backpack.git@ggn-materialize-grad seems to solve the issue. At least, this MWE works fine:
import torch

from hessianfree.optimizer import HessianFree

BATCH_SIZE = 1
OBSERVATION_SPACE = 32
ACTION_SPACE = 4

observation = torch.randn(BATCH_SIZE, OBSERVATION_SPACE)

QNet = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE + ACTION_SPACE, 1),
)

Actor = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE, ACTION_SPACE),
)

opt = HessianFree(
    params=Actor.parameters(),
    verbose=True
)

def forward():
    actions = Actor(observation)
    actor_losses = -QNet(torch.cat([observation, actions], 1))
    actor_loss = actor_losses.mean()
    return actor_loss, actions

loss = opt.step(forward=forward)

Hello Lukas,
I really appreciate your help!
I've recently tried applying the optimizer to the critics, but unfortunately, I'm running into the same error I encountered earlier. It seems like the issue might be related to using torch.min, because the function works fine if I return e.g. qf1_a_values in the forward_df(). Do you have any suggestions on how to correctly implement the forward function e.g. computing the min of the two tensors and keep them in the computation graph? I'd be really grateful for any advice you can offer.
import torch
import torch.nn.functional as F
from hessianfree.optimizer import HessianFree

BATCH_SIZE = 4
OBSERVATION_SPACE = 32
ACTION_SPACE = 4

observation = torch.randn(BATCH_SIZE, OBSERVATION_SPACE)
next_q_value = torch.randn(BATCH_SIZE, 1)

QNet1 = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE + ACTION_SPACE, 1),
)

QNet2 = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE + ACTION_SPACE, 1),
)

Actor = torch.nn.Sequential(
    torch.nn.Linear(OBSERVATION_SPACE, ACTION_SPACE),
)

opt_qf = HessianFree(
    params=list(QNet1.parameters()) + list(QNet2.parameters()),
    verbose=False
)

def forward_qf():
    input = torch.cat([observation, Actor(observation)], 1)
    qf1_a_values = QNet1(input)
    qf2_a_values = QNet2(input)
    outputs = torch.min(qf1_a_values, qf2_a_values)

    qf1_loss = F.mse_loss(qf1_a_values, next_q_value)
    qf2_loss = F.mse_loss(qf2_a_values, next_q_value)
    qf_loss = qf1_loss + qf2_loss

    return qf_loss, outputs

loss_qf = opt_qf.step(forward=forward_qf)

Traceback (most recent call last):
  File "/home/felix/Masterthesis/code/second_order_methods_testing/hf_td3_mwe.py", line 74, in main
    loss_qf = opt_qf.step(forward=forward_qf)
  File "/home/felix/anaconda3/envs/ma310/lib/python3.10/site-packages/torch/optim/optimizer.py", line 385, in wrapper
    out = func(*args, **kwargs)
  File "/home/felix/Masterthesis/third_party/PyTorchHessianFree/hessianfree/optimizer.py", line 258, in step
    x_iters, m_iters, cg_reason = cg(
  File "/home/felix/Masterthesis/third_party/PyTorchHessianFree/hessianfree/cg.py", line 188, in cg
    r = A(x0) - b
  File "/home/felix/Masterthesis/third_party/PyTorchHessianFree/hessianfree/optimizer.py", line 259, in <lambda>
    A=lambda x: mvp(x) + damping * x,  # Add damping
  File "/home/felix/Masterthesis/third_party/PyTorchHessianFree/hessianfree/optimizer.py", line 240, in mvp
    return self._Gv(loss, outputs, self._params_list, x)
  File "/home/felix/Masterthesis/third_party/PyTorchHessianFree/hessianfree/optimizer.py", line 454, in _Gv
    Gv = ggn_vector_product_from_plist(loss, outputs, params_list, vec_list)
  File "/home/felix/anaconda3/envs/ma310/lib/python3.10/site-packages/backpack/hessianfree/ggnvp.py", line 55, in ggn_vector_product_from_plist
    HJv = hessian_vector_product(loss, output, Jv)
  File "/home/felix/anaconda3/envs/ma310/lib/python3.10/site-packages/backpack/hessianfree/hvp.py", line 45, in hessian_vector_product
    df_dx = torch.autograd.grad(f, params, create_graph=True, retain_graph=True)
  File "/home/felix/anaconda3/envs/ma310/lib/python3.10/site-packages/torch/autograd/__init__.py", line 411, in grad
    result = Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

Hi Felix,
Sure, no problem!
Maybe you can use the Hessian instead of the GGN (I am skeptical anyway whether the GGN makes sense in your case, as it requires a very specific structure of the loss function).
opt_qf = HessianFree(
    params=list(QNet1.parameters()) + list(QNet2.parameters()),
    verbose=False,
    curvature_opt="hessian",
)

This seems to work fine.
The problem with the GGN is that, internally, we compute HJv = hessian_vector_product(qf_loss, outputs, ...), i.e. we compute derivatives of qf_loss with respect to outputs. But since outputs is not used for the computation of qf_loss, this results in an error.

Hi Lukas,
Thank you very much. This really helped me a lot.
Best
Felix

You're very welcome 😉


Add npm package for this project

Instead of compiling the js file on the server we should have a package to install with out existing node.js projects.
This should make development easier when accessing exported classes and components such as <PyComponent name={...} props={...}) and the EZ class instance.

Also, change the name from EZ to PyJSX or something, you're mixing projects.


Weird Subject-Pellet-Delivery Visualizer Behavior

We've noticed that, when stopping a workflow, with an active subject, in maintenance mode, and relaunching the workflow, the pellet delivery visualizers stop working. @lorycalca

Can't always reliably reproduce however

@jkbhagatio which branch is this? Is it on main or in one of the issue branches?

I was online with Jai and we where unable to reproduce the same behavior.
We were working on the RFID visualizer branch
We tried many times and agreed that it could be something else non related that happened on that specific time that the issue happened, so this issue is not major or blocking for the current experiment

Ok, I am clearing it from the milestone then, feel free to move it back if it happens again.

Couldn't recreate, so closing this, can re-open if we notice it again


Support PWA and SPA

Wouldn't it be awesome that Hugo — by using the latest technologies — natively outputs a progressive web app / single page application enjoying all their benefits (offline, instant navigation preload/prefetch, etc.) and thus be ready for the future of web without relying on external dependencies?
Wouldn't it be too difficult to implement, and above all would it accommodate Hugo's philosophy?

The short answer is a loud "yes".
I created #5894 to track a Proof of Concept that would include at least the evaluation part of what you talk about. One of the goals of that PoC would be to this without doing "too many" modifications to Hugo itself, e.g. by using the concepts of Output Formats etc. #5894 is certainly fairly high up on my list (others may get there before me), but I get easily sidetracked ... Let us keep this discussion in #5894.


feat(esl-select): ESLSelect and ESLSelectList migrated to ESLEventListeners

:tada: This PR is included in version 4.0.0-beta.19 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 4.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket: