fix(nuxt): avoid vue-router warning with routeRule redirect

🔗 Linked issue
resolves https://github.com/nuxt/nuxt/issues/24112
❓ Type of change

[ ] 📖 Documentation (updates to the documentation, readme or JSdoc annotations)
[x] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
When using nitro route rules, we support client-side handling of redirects with vue-router (as well as nitro on server side). However, there's still an error message printed in the console.
This PR adds a stub component (a bare object) for these routes, so that the error is suppressed.
📝 Checklist

[x] I have linked an issue or discussion.
[ ] I have added tests (if possible).
[ ] I have updated the documentation accordingly.

A bit unrelated, but before I forget it: We should also look into fetching payloads (prerendering + payloadExtraction) when the page has a redirect.

A bit unrelated, but before I forget it: We should also look into fetching payloads (prerendering + payloadExtraction) when the page has a redirect.

We I think should already skip fetching payloads for redirected paths:
https://github.com/nuxt/nuxt/blob/ce8a2aaf37aa65d0f9292b26c32f37f68bcf07e6/packages/nuxt/src/app/composables/payload.ts#L89

@danielroe sorry, I meant a redirect via definePageMeta (making it totally unrelated 😂).


Run the fuzzers in the CI

just like the coreutils, we should run the fuzzers in the CI to make sure we don't regress
https://github.com/uutils/coreutils/blob/main/.github/workflows/fuzzing.yml as example

hello!
I submitted a fuzzing.yml ci file and made minor modifications to the original code in https://github.com/uutils/diffutils/pull/29, but I forgot to pick up the issues here in advance, I hope it doesn't disrupt your work.
Here are some other issues I've encountered:

Noticed that both the Windows build and macOS before I forked didn't work.
Although the CI is passing, it doesn't seem to be testing properly.
here is the fuzzing ci log built on my fork: https://github.com/hanbings/diffutils/actions/runs/8331273334/job/22797824794
in the Run fuzz_ed for XX secords step, cargo threw an error in the console, but did not interrupt the CI.

Do I need to create some ci action to get full logs from ci? or where should I start to check the fuzzed code?


Convert Config to Json Files

Convert config from the .txt format to use json instead.

Done. Need to test now.

Done.


DragonBones 5.6 没有适配 mac 10.15 Catalina

如题，请问什么时候会有64bit 版本。
官网的最新包还是5.6.1 ，推送更新的5.6.3 在10.14.6 上也无法打开。

This is a big problem for me currently — I can't run DragonBonesPro at all on my development machines

This is affecting me too.  Is it just the installer that is 32 bit, or also Dragonbones itself?

Seems like both to. I still had dragonbones installed after the upgrade,
and it does not run. The installer does not run either. This continues to
be an annoying problem for me, and my small game studio
I hope it is not a big issue to overcome for the dev team. The year is 2020
and 64 bit support should be near ubiquitous in libs and compliers
On Tue, Jan 7, 2020, 3:11 PM Michael Sunwoo (AvatarLabs) <
<EMAIL_ADDRESS>wrote:

This is affecting me too. Is it just the installer that is 32 bit, or also
Dragonbones itself?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/DragonBones/DragonBonesJS/issues/123?email_source=notifications&email_token=AABSK2GRBJXIWCXA2DAH7ADQ4UDYXA5CNFSM4JMTZVT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIKTS3I#issuecomment-571816301,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABSK2HL3KCDVXBTDWQ4QQLQ4UDYXANCNFSM4JMTZVTQ
.


Rework json.dumps to support subclasses

We can currently only construct instances of string subclasses, so
that's the only code path that is actually tested.
Fixes #91.

Nice job! This looks really good.


Patch 5.57 Offset

Sorry for the delay

No need to apologize, you do this on your own time. Thanks.


require() webpack-cli/webpack-command's bin module

Rather than require()-ing the "main" module in webpack-cli/
webpack-command, require() the "bin" module.
This avoids the issue described in
https://github.com/webpack-contrib/webpack-command/pull/30
where installing packages in this order results in no output from
./node_modules/.bin/webpack:
$ npm install webpack-command
$ npm install webpack
$ ./node_modules/.bin/webpack
# exit 0 with no output

Note: the read-package-json package is by npm, and helps to normalize the shape of the pkg.bin object (since the "bin" field in package.json can be either a string or an object).

What kind of change does this PR introduce?
Bug fix
Did you add tests for your changes?
No. I could not find an existing test suite for the CLI. Welcome to suggestions here.
Does this PR introduce a breaking change?
No.
What needs to be documented once your changes are merged?
Nothing.

Note: This is still just a bandaid. webpack is trying to be too fancy/clever here. Updating documentation and dropping bin/webpack.js for webpack v5 should happen.

(Friendly bump)
@ooflorent I've removed the dependency on the read-package-json package, now using hard-coded names for the executables.

Thanks


Add existing input files to Integration testing suite

They  live in data/input_files.

Fixed in #650


GSM8K features.

The two main additional features in this pull are the ability for the role prompting experiment to start at any point in the dataset that is loaded, and backoff that words reliably.

MMLU pr covers this


[cxx-interop] Workaround a deserialization error for CxxStdlib on Linux

In certain versions of libstdc++, std::hash<std::string> defines operator() in a base class. It looks like Swift is not correctly deserializing an inherited operator() for inlinable functions. This change sidesteps the issue by moving the call to callAsFunction/operator() to the C++ shim layer.
rdar://140358388

@swift-ci please test

@swift-ci please build toolchain Ubuntu 24.04

@swift-ci please build toolchain Ubuntu 22.04

@swift-ci please build toolchain Ubuntu 24.04

Merging to unblock nightly builds.


Grammar examples as part of the deliverable.

This will help avoid duplication between the master branch and some of the examples
used in the playground.

These grammar examples are pretty static so the duplication does not cause a greater overhead.
Not sure this is worth the effort at this time.


Closes 116

🎯 Title and Issue Number

Closes #116

🗒 Checklist:

[x] I have mentioned the issue number in my Pull Request.
[x] I have gone through rules of when PR will get merged.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have created a helpful and easy-to-understand README.md.
[x] I have followed the proper file structure.
[x] I have gone through the repository guidelines.

Thank you for contributing! ❤
Don't forget to show some support by ⭐ the repository!

Hello @shuvadeepmondal, thank you for your pull request! We will review it shortly. 😇❣
Your contributions help us grow and improve our project. Please make sure you have read and followed our CONTRIBUTING GUIDELINES.
Your efforts are appreciated, and we value your participation in our community. Don't forget to ⭐ our repository!
Happy coding!✨

@Puskar-Roy  Bro Review it Once

Hey @shuvadeepmondal! We're excited to let you know that your pull request has been merged into the APIverse🚀🎁
Your commitment, effort, and expertise have greatly influenced our project, and we're incredibly thankful for your contributions.😇
Your code is now part of our expanding ecosystem, assisting developers worldwide in creating exceptional APIs.❣🔥
Happy Coding! ✨ We look forward to seeing you again soon!❣


Show long commit messages wrapped in commit history

When a commit message can't be shown as one line in commit history it would be nice to be able to show wrapped commit message on clicking on three dots at the end of such message.
Or a simpler solution is to show long message in a tooltip when the user hovers mouse over that commit message.

I would suggest using Shave


fix: add "type: module" for Next.js

Next.js required you to have "type: module" in your package.json in order for it to recognize it as an ESM module, no need for transpilation this way.
Source:

https://github.com/vercel/next.js/issues/31518
https://nextjs.org/blog/next-11-1#es-modules-support

Not sure if this breaks anything else, but would save some hassle/questions in the future.

Thanks Thomas! I'll look into this. If it doesn't cause any issues that's a nice fix.

@ThomasFKJorna Okay this is added with v 1.2.6 — turns out there were conflicts with my Rullup and babel config that needed to be resolved before this could work. I've tested it and it should be working fine, but let me know if you have any issues with the new setup.

Amazing, now it "just works"! Thanks for the quick response :)

The loaders are beautiful as well btw, really great work ❤


Correct a typo

Incorrect call to enable Sass math function.

This looks to be working as expected. Thank you for merging the pull request into master. Would you mind publishing this update to npm so others can get the update?
Thanks again.

Yes it should be published

I make a new Github repo based on my pull request and published to npm.
https://github.com/luebster/scss-loading-animations
https://www.npmjs.com/package/scss-loading-animations


On Robinson College of Business : 2019 US News & World Report (Summary)

Robinson Undergraduate Programs Among Nation's Best in Latest US News & World Report Rankings
*See link above for Source credit : robinson.gsu.edu
Quantifiable Educational Background

No Rules; Just Right, right?

Best,
C Brennan Poole
with Wind
🤓

Source URL:
https://robinson.gsu.edu/2019/09/three-robinson-undergraduate-programs-among-nations-best-in-latest-u-s-news-rankings/
BrowserChrome 85.0.4183.59OSWindows 10 64-bitScreen Size2560x1080Viewport Size3840x1405Pixel<EMAIL_ADDRESS>Level75%

img optimization by sqoosh.app

Best,
c

optimized by sqoosh.app

Best,
x.C Brennan Poole__
Class of '06


Changelog for 0.5.1 is not included in 0.5.1 tagged release

The tag for 0.5.1 points at f42ff873a0499f4988838dc7935c321927ae3a94 but the changelog was updated afterwards in 4482a024c2d48f354f372c78ce689974fbeff220.

Thanks. Some day I need to get out another release and I'll make sure it's in this time.


User Story - Move different goods concurrently

User Story:
As a: warehouse manager
I want: to get goods of different orders ready at warehouse at the same time
So that: the time of loading can be reduced and additional waiting time can be avoided
Acceptance Criteria:
And I know I am done when: different orders arrive at the spot concurrently

It belongs to Upgraded Line Following, which is not done.


Virtual VTEP support

Issue Type

Template enhancement
Role enhancement

Is your feature request related to a problem? Please describe.
We have a requirement were some of the VTEPs in the fabric will be acting as L2 VTEPs where the gateway (IRB) for the hosts connected to the L2 VTEPs will be on L3 VTEPs. The reason for this is to avoid L2 VTEPs flooding the ARP traffic to every single VTEP in the network, and just send it to the Virtual VTEPs (vVTEPs) in the flood list.
EVPN will generate Type-2 for the virtual MAC which is configured on the L3 VTEPs with the next-hop as the vVTEP IP, so that the L2 VTEPs will forward the traffic only to the next-hop with the IP of vVTEP (so destination will be the correct L3 VTEP with the IRB/SVI).
Describe the solution you'd like
Support to configure the secondary IP address on Loopback1 that has to be advertised in the Underlay.
The IP address assigned should be configured on all leaves acting as L3 VTEPs that are part of the tenant.
This has to be unique per tenant and should be configured under the tenant.
There should be a pool of IP addresses where we can use an IP per tenant.
Having the same IP in all tenants will not work as different tenants will have different VLANs.
interface loopback1
   ip address <IP_ADDRESS>/32
   ip address <IP_ADDRESS>/32 secondary

Describe alternatives you've considered
Creating IRB on all VTEPs, so all VTEPs will be L3 VTEPs. This would work in small deployments, however if you have hardware limitations of VRFs, dynamic mappings etc. this will not work, as the limitation factor will be the scale.
Additional context
Add any other context or screenshots about the feature request here.

Fixed with PR #580 and PR #457

Fixed with PR #580 and PR #457


[WIP] Docker Desktop Dashboard updates

Proposed changes
Adding Docker Desktop Dashboard updates for Desktop Stable <IP_ADDRESS> release
Unreleased project version
<IP_ADDRESS>
Related issues
https://docker.atlassian.net/browse/DESKTOP-1878
https://docker.atlassian.net/browse/DESKTOP-1852
https://docker.atlassian.net/browse/DESKTOP-2037
https://docker.atlassian.net/browse/DESKTOP-2039

Deploy preview for docsdocker ready!
Built with commit 8c603fce8f35c1e285d993cec68db396480df621
https://deploy-preview-10051--docsdocker.netlify.com

Updated the table of contents as per https://github.com/docker/docker.github.io/pull/10068/files to resolve merge conflicts.
The toc now reflects changes in the master branch.


Matplotlib package error

Problem
The Matplotlib package must be installed apart of pyRothC, otherwise it fails.
E.g. we install pyRothC and try to use the RothC object ourselves:
pip install pyRothC
ipython
...
In [1]: from pyRothC import RothC
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[1], line 1
----> 1 from pyRothC import RothC

File ~/.virtualenvs/playground/lib/python3.11/site-packages/pyRothC/RothC.py:12
     10 from scipy.integrate import odeint
     11 from scipy.interpolate import interp1d
---> 12 import matplotlib.pyplot as plt
     15 class RothC:
     16     """Class for Python version of The Rothamsted carbon model (RothC) 26.3."""

ModuleNotFoundError: No module named 'matplotlib'

This happens because the test lives in the same file as the code, RothC.py.
Reasoning
We should not need Matplotlib to use the RothC class but Matplotlib gets imported at the beginning of the file, hence this error.
Specially, it's important to avoid the usage of libraries that are not critical for the base operation expected from the package, so it can be used in different environments than just a Jupyter notebook with a whole desktop/UI suite installed. In my personal case, I do need the computations done by the RothC class on a variety of random inputs, but I don't need at all any graphics or plots being drawn because the computations are run in batches in a server and the results will go to a database - they will be visualized later from there in another systems, not necessarily using Matplotlib or even Python libraries.

@jose-lpa thank you! Approved


On Windows 10, lldb gets error regarding module 'six'

This is due to a "bug" in the Swift installation procedure for Windows 10.  Below, after describing the issue, I have a solution.  I am new at Swift, and other thoughts are welcome!
Environment
build number: Swift 5.6.1
OS: Windows 10 desktop computer
Issue Description
On Windows 10, lldb gets error: ModuleNotFoundError: No module named 'six'
Test Case
After an install of Swift on Windows 10, run 'lldb Factorial' as shown on Swift - Getting Started > Using the LLDB Debugger.  That elicits the error above.
Cause
The Python module 'six' has not been installed (it is not part of the python distribution), and/or the env var PYTHONPATH is missing the Python directory: (your-Python-root)\Lib\site-packages.
Solution
See notes on PYTHONPATH and the 'six' module in #58438.

closing. I provided the solution right within the issue. I am accepting my own solution.
(Perhaps I should have kept the solution out of the initial issue, and then provided the solution immediately afterward?)

Resubmitted as part of https://github.com/apple/swift-org-website/issues/287


Burnin custom font filepath

Changes

added possibility to set path to font for extract burnin

path may contain environment variables "{MY_PATH}/path/to/my/font.tff"

added to settings additional filtering of output definitions

additional filtering is defined by families and tags

don't use shell=True for running subprocess which may cause issues on Mac and Linux workstations

for windows is added CREATE_NO_WINDOW flag which should avoid creating new terminal window

font is not handled in burnin script but is prepared before got into the script

default font path should be set to LiberationSans font which is in OpenPype resources
default font is also used when can't access font set in settings

start frame on processing of sequence in burnin script is also explicitly set to be start frame of an output

Note

add modifiable placeholders for text and path input (now shows placeholder Executable path)
should raise an exception if can't find font from settings?

closes https://github.com/pypeclub/client/issues/47


Integrate feed analytics

It would be great to programmatically generate feeds that will be tracked via something like Feed Press so we can see how popular feeds are.

https://feed.press/api/methods#feeds-create

Fixed in #15


[SageMaker] list_endpoints & describe_endpoint

Collect list_endpoints and describe_endpoint. Needed for OpenCSPM aws-185 (Enterprise Control Pack).

Added in 1835e930fa7a8dd797f4474a7ee53bc154d434a0


No scrollbar for the API list on the documentation page

This page, https://facebook.github.io/immutable-js/docs/#/Map, for example, there is no scrollbar to view the consolidated list of apis on the left.Makes it impossible to use without a scrollwheel (ie. laptop). I can fix it but I don't know where the html source is located.

Same problem here. Tested on both Chrome and Firefox.

Fixed


[MM-28595] Open team links within the app

Before submitting, please confirm you've

[x] executed npm run lint:js for proper code formatting

Please provide the following information:
Summary
Team links were not opening when setting the help link as it was trying to open in a new window and we don't allow new windows for server links unless they are popups
Issue link
MM-28595
Test Cases
Change the help link to a channel or some other page that it is within the server's teams. App should reload to it.
Change the help link to a page not belonging to a team (like https://google.com). A new browser tab with the page is loaded.

@Willyfrog
Error trying doing the automated Cherry picking. Please do this manually


Add option to toggle code folding

Add an option to disable code folding entirely as it is not always desired by all users. This PR is a strawman for suggestions to enable more detailed code folding options. It is likely not ideal and probably doesn't work correctly (in terms of re-enabling and toggling on the fly) -- it works for me but I'm new to IntelliJ extensions, so criticism is more than welcome and probably quite mandatory.

Is there any interest in this PR?


Feature 5108 - move to debugpy

Implements #5108 : Switch from PTVSD to DebugPy.
Description

PTVSD has been deprecated, debugpy is the new official python debugger for vscode/vs
Related Issue

#5108
Motivation and Context

PTVSD will not be supported anymore, and the github project has been archived already.
Requires Documentation Change

The readme and relevant mentions to PTVSD have been updated to reflect the switch to debugpy.
How Has This Been Tested?

The tests were run in the CI before merge.
The changes have been tested locally in a windows 10 environment with Python 3.8.6 by debugging a Python Algorithm on Visual Studio Code.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Refactor (non-breaking change which improves implementation)
[ ] Performance (non-breaking change which improves performance. Please add associated performance test and results)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Non-functional change (xml comments/documentation/etc)

Checklist:

[x] My code follows the code style of this project.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

Hey @marcograss!
Thanks for the contribution. Believe we hadn't pushed this before because at the time debugging a docker container wasn't working correctly. Before updating we should confirm the following supported cases are still operational:

Visual Studio Code, Visual Studio
Debugging locally and debugging a docker container

Does it sounds right @C-SELLERS? 😁

Hey @marcograss!
Thanks for the contribution. Believe we hadn't pushed this before because at the time debugging a docker container wasn't working correctly. Before updating we should confirm the following supported cases are still operational:

Visual Studio Code, Visual Studio
Debugging locally and debugging a docker container

Does it sounds right @C-SELLERS? 😁

Hey @marcograss!
Thanks for the contribution. Believe we hadn't pushed this before because at the time debugging a docker container wasn't working correctly. Before updating we should confirm the following supported cases are still operational:

Visual Studio Code, Visual Studio
Debugging locally and debugging a docker container

Does it sounds right @C-SELLERS? 😁

the issue might be related (just a guess) to the fact that with debugpy you need to specify the python interpreter (which I do here debugpy.configure(python=shutil.which('python')) )  . it seems to use the current executable by default, which in our case is the mono interpeter (python.net is embedded)
However I don't have a suitable environment to build the docker myself, it seems the windows build, when pushed in the quantconnect/lean docker container when building, has some problems to find the python library due to differences between windows (at least my env) and the one inside the docker with conda

Hey @marcograss!
Thanks for the contribution. Believe we hadn't pushed this before because at the time debugging a docker container wasn't working correctly. Before updating we should confirm the following supported cases are still operational:

Visual Studio Code, Visual Studio
Debugging locally and debugging a docker container

Does it sounds right @C-SELLERS? 😁

the issue might be related (just a guess) to the fact that with debugpy you need to specify the python interpreter (which I do here debugpy.configure(python=shutil.which('python')) )  . it seems to use the current executable by default, which in our case is the mono interpeter (python.net is embedded)
However I don't have a suitable environment to build the docker myself, it seems the windows build, when pushed in the quantconnect/lean docker container when building, has some problems to find the python library due to differences between windows (at least my env) and the one inside the docker with conda

the issue might be related (just a guess) to the fact that with debugpy you need to specify the python interpreter (which I do here debugpy.configure(python=shutil.which('python')) ) . it seems to use the current executable by default, which in our case is the mono interpeter (python.net is embedded)

Ah that might of been it! 👍

However I don't have a suitable environment to build the docker myself, it seems the windows build, when pushed in the quantconnect/lean docker container when building, has some problems to find the python library due to differences between windows (at least my env) and the one inside the docker with conda

When building cross-platform, for example windows for Linux, you have to specify the build parameter ForceLinuxBuild set to true so that it picks up the correct pythonNet binary

the issue might be related (just a guess) to the fact that with debugpy you need to specify the python interpreter (which I do here debugpy.configure(python=shutil.which('python')) ) . it seems to use the current executable by default, which in our case is the mono interpeter (python.net is embedded)

Ah that might of been it! 👍

However I don't have a suitable environment to build the docker myself, it seems the windows build, when pushed in the quantconnect/lean docker container when building, has some problems to find the python library due to differences between windows (at least my env) and the one inside the docker with conda

When building cross-platform, for example windows for Linux, you have to specify the build parameter ForceLinuxBuild set to true so that it picks up the correct pythonNet binary

Yeah after testing today with this branch on DebugPy it seems that docker is failing to connect at all as well as some issues with local debugging. Here are my findings on testing it out.

I also tested PTVSD to confirm current behavior for the record

Now I recall why we didn't include this debugging method in the ./vs/readme.md

Yeah after testing today with this branch on DebugPy it seems that docker is failing to connect at all as well as some issues with local debugging. Here are my findings on testing it out.

I also tested PTVSD to confirm current behavior for the record

Now I recall why we didn't include this debugging method in the ./vs/readme.md

Hey!
Will close PR for now until the cases posted by @C-SELLERS are working with debugpy

Hey!
Will close PR for now until the cases posted by @C-SELLERS are working with debugpy


Model evaluation and metrics code

I would like to thank and congratulate for the research work developed and advances achieved.
I have run and implemented my changes on the code, and ow i would like to have the same measures presented in the paper: AUC Dist MDist Ang MAng. In the original code folder there is a text_gazefollow.py file, to evaluate right? After processing the test dataset (GazeFollow dataset recasens 2015) the result was:
average loss : [tensor(0.0587, device='cuda:0') tensor(0.1137, device='cuda:0') tensor(0.1724, device='cuda:0')]
average error: [ 0.14532723 19.64980047 18.1135517 ]

How can I extract the complete metrics?
Thank you

Does is possible plot ROC curve?

Solution to plot ROC curve and AUC, just copy and paste at end of calc_auc.py file.
`# Compute micro-average ROC curve and ROC area
fpr, tpr, threshold = roc_curve(gt_list, pred_list)
roc_auc = auc(fpr, tpr)
plt.figure()
plt.title('Receiver Operating Characteristic')
plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)
plt.legend(loc = 'lower right')
plt.plot([0, 1], [0, 1],'r--')
plt.xlim([0, 1])
plt.ylim([0, 1])
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()`

@niujinshuchong how can i have Recases et al. data to plot ROC curve?

@emannuell
We didn't save the results. But maybe you can run Recases et al. model yourself.


fix 692: add more imports to try/except import check

fixes #692

The failing test is something with wrong or missing dimensions declared? Not an issue with these changes.

Hey @michaelaye thanks for this fix! hvplot's CI is currently broken due to a bug introduced in the latest version of holoviews. This bug has already been fixed but has not yet been released. I think that I'd like to make soon a dev release of holoviews, which would fix the test suite here.
Once this is done, I'll rerun the test workflow on your PR to see if it passes the tests.
As for your fix it looks good to me.

there were some release on holoviews in the mean time, could this be merged then? It creates issues for newbies as seen in more comments on #692 .

Looks good, thanks!!


Tensortrade for portfolio

Is it currently possible to train and trade a portfolio of instruments?
Does the exchange object supports that?
There is a multi-discrete action schema, however wouldn't it be better to also have multi-continuous action with cash bias, something like (box(-1,1) * num_of_instruments + 1)?

Hi @hn2, this is currently being worked on in #57


Make sure we still close every open file

The Open() signature from daaku/go.zipexe is pretty non-standard, as it returns a Reader instead of a closeable file, preventing anyone from closing the file, breaking the Open() semantics.  Since that project hasn't moved in 2 years, and is a simple 135 lines file, I added it here, and tweaked Open to return something we can close, as well as anything we need to read the zip content.
This fixes the issue I introduced (mismatched 2 == 3 thing) as well as file closing.
/cc @nwolff

We'll take another route. See https://github.com/GeertJohan/go.rice/pull/40


chore(cname): delete CNAME from master

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@7b1a2d2). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master    #263   +/-   ##
========================================
  Coverage          ?   79.9%
========================================
  Files             ?      17
  Lines             ?     214
  Branches          ?       9
========================================
  Hits              ?     171
  Misses            ?      41
  Partials          ?       2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b1a2d2...07016ea. Read the comment docs.


Feat/add interactive shell

Execute python code in your Django app context.

I played around with it, and I like it. I agree it's much more convenient than using python manage.py shell. Let's figure out how to include it!
I'm not a fan of the selector as the first screen, but I think a key action (ctrl+i is already used by trogon, but I think ctrl+j works well) to switch to the interactive shell screen and back works well.
We could also add a command line argument as a shortcut to go directly into the interactive shell, something like:
python manage.py tui shell # to open up the new interactive shell directly

What do you think?

I played around with it, and I like it. I agree it's much more convenient than using python manage.py shell. Let's figure out how to include it!
I'm not a fan of the selector as the first screen, but I think a key action (ctrl+i is already used by trogon, but I think ctrl+j works well) to switch to the interactive shell screen and back works well.
We could also add a command line argument as a shortcut to go directly into the interactive shell, something like:
python manage.py tui shell # to open up the new interactive shell directly

What do you think?

I agree 100%

@anze3db, I added your suggestions.
You can now open shell directly with python manage.py tui --shell and switch to commands and shell with ctrl+j.


IOS version?

Thanks for the great work - has anyone go the ios working?
[VERBOSE-2:dart_vm_initializer.cc(41)] Unhandled Exception: Instance of 'ConnectionException'
#0      MethodChannelWireguardVpn.changeStateParams (package:wireguard_vpn/wireguard_vpn_method_channel.dart:24:7)

#1      _MyAppState._activateVpn (package:my_flutter_app/main.dart:82:9)

Lol there is no IOS version! :) Well not yet anyway.  The android version runs just fine in emulator S23 API 34 but you must add a PSK to the peer config if you dont have it there already.  wg pskgen will create a complaint key which you save to the server peer config and the main.dart e.g.:   final String presharedKey = 'eiTPIpnRi8er333g275HxUG...4I5kwR52sUVF3zfuI=';

sad

Regarding the iOS version, the package does not currently have one, and I lack both the resources and the knowledge to develop it at this time. However, please remember that you are welcome to contribute to the package whenever you wish. Your help would be greatly appreciated.

How can i get presharedKey ? i'm new to wg and i just created an account from ProtonVPN for testing but I didn't see presharedKey after created wireguard configuration


Odd behavior when changing kind/OS of app service plan

Bicep version
Bicep CLI version 0.4.1124 (66c84c8ee5)
Describe the bug
Here's my resource definition:
resource plan 'Microsoft.Web/serverfarms@2021-02-01' = {
  name: planName
  location: resourceGroup().location
  tags: resourceGroup().tags
  kind: planOS
  sku: {
    name: planSKU
  }
  properties: {
    reserved: toLower(planOS) == 'linux' ? true : null
  }
}

Running this with planOS=Windows and planSKU=S1 succeeds just fine. If I change this to planOS=Linux and run again, the operation succeeds, but there is conflicting information about the plan's settings in the portal:

If I run this again with planOS=Linux and also adding planSKU=S2, this is the output from --what-if:
Note: The result may contain false positive predictions (noise).
You can help us improve the accuracy of the result by opening an issue here: https://aka.ms/WhatIfIssues

Resource and property changes are indicated with these symbols:
  ~ Modify
  * Ignore

The deployment will update the following scope:

Scope: /subscriptions/REDACTED/resourceGroups/REDACTED

  ~ Microsoft.Web/serverfarms/REDACTED [2021-02-01]
    ~ sku.name: "S1" => "S2"

  * Microsoft.Storage/storageAccounts/REDACTED
  * microsoft.alertsmanagement/smartDetectorAlertRules/Failure Anomalies - REDACTED
  * microsoft.alertsmanagement/smartDetectorAlertRules/Failure Anomalies - REDACTED

And this is the output when deploying for real, without --what-if:
{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"BadRequest","message":"{\r\n  \"Code\": \"BadRequest\",\r\n  \"Message\": \"You cannot change the OS hosting your app at this time. Please recreate your app with the desired OS.\",\r\n  \"Target\": null,\r\n  \"Details\": [\r\n    {\r\n      \"Message\": \"You cannot change the OS hosting your app at this time. Please recreate your app with the desired OS.\"\r\n    },\r\n    {\r\n      \"Code\": \"BadRequest\"\r\n    },\r\n    {\r\n      \"ErrorEntity\": {\r\n        \"ExtendedCode\": \"59315\",\r\n        \"MessageTemplate\": \"You cannot change the OS hosting your app at this time. Please recreate your app with the desired OS.\",\r\n        \"Parameters\": [],\r\n        \"Code\": \"BadRequest\",\r\n        \"Message\": \"You cannot change the OS hosting your app at this time. Please recreate your app with the desired OS.\"\r\n      }\r\n    }\r\n  ],\r\n  \"Innererror\": null\r\n}"}]}}

For your convenience, the error message here is You cannot change the OS hosting your app at this time. Please recreate your app with the desired OS.
To Reproduce
Repro steps are described above.
Additional context
I was initially trying to change the kind/OS of an already-deployed plan. I have experience with Terraform, and Terraform will offer to destroy and re-create the plan in that scenario, so I assumed it would work with Bicep too. After thinking about it, I understand now that since Bicep doesn't have state files, it's probably pretty difficult to draw a hard link between a resource in the Bicep file and the actual resource in Azure (imagine if I changed the name, for example). So it'd be okay with me if you don't plan to support that functionality by design.
However the other behavior here seems odd. I don't understand why the first run with planOS=Linux left the plan in an odd state in the portal (one place says it's a Linux plan, the other says Windows), or why that run didn't give the error encountered when changing the SKU in the next run. Also the --what-if output doesn't mention that the OS change can't and won't happen when it probably should.

Kind property of app service is just for portal display. The actual property that selects between windows and Linux plan is reserved.
Also, you cannot change os for an existing plan or have both Linux and Windows plans in one region in this same resource group.

Basically the kind property should be a read only and that’s a provider bug I say.

@naveedaz  / @seligj95 -- @miqm is bringing up a good point. Given the "kind" property only affects the portal UX, should it be a read-only property and not set by end users?

@miqm Thanks for the additional info - I wasn't aware that you can't have both Linux and Windows plans in one region in the same resource group. Should that be considered a bug with --what-if then? I can file an issue there if so.
Thanks also for the clarification on kind  vs reserved too.

I spent a day or two trying to figure out why my supposed to be linux app plan showed conflicting results in the portal UI. That's how I learned the hard way that kind doesn't really affect anything, but reserved does.
This is something that's more or less impossible to figure out by reading the documentation. Only thing that the resource documentation says about the reserved property is this;

true if reserved; otherwise, false.

Feels more like someone tries to be sarcastic than actually documenting something here 🙄

Just to highlight - in my repro case, the planOS variable controls both kind and reserved, but I only got the error when I also changed the planSKU. That still doesn't seem right to me. I understand that kind is just the display property, but both it and reserved were modified at the same time.

Realized now that I referenced the wrong resource type documentation in my comment above.
On the serverfarms' resource documentation it actually says the correct thing

If Linux app service plan true, false otherwise.

But as you say, as that property is used elsewhere too for a different meaning it's a badly chosen property to control this behavior.
Would be interesting to know the reasoning behind the decision that led to this design.

Got stuck all afternoon with this "kind" and all sorts of issues trying to get the understanding of Windows vs Linux - a really easy thing to do in the portal, but it is so poorly documented here https://learn.microsoft.com/en-us/azure/templates/microsoft.web/serverfarms?tabs=bicep&pivots=deployment-language-bicep
It's actually really awful


(v3) Draft PR: add Pyright static typing to hook_points.py #590

Description
This is a copy of PR https://github.com/TransformerLensOrg/TransformerLens/pull/590 and #464 , but just remade so that @bryce13950 can close it.

This draft PR adds Pyright static typing to hook_points.py. See Alan's message on the slack for motivation/background. Specifically, this PR:

modifies the HookPoint and HookedRootModule to support a baseline amount of pyright typing
adds a [tool.pyright] configuration to pyproject.toml to configure the typing for this project

It's not entirely complete, but is in a state to where it would benefit from some review/assistance. Before being PR ready, we should:

Address some TODO's I've marked on lines of code that I'm ucnertain about
Clean up remaining type warnings on hook.py
If possible, adjust the pyproject.toml file so the type warnings quit applying to all files

Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have not rewritten tests relating to key interfaces which would affect backward compatibility

Alright, there is one more issue left to resolve. I am done for the day, but I am happy to take care of it tomorrow. If you want to look at it feel free, but I don't mind taking care of it.

okay feel free to knock it out tomorrow if i don't get to it

All set! Thank you so much for your help and patience on this! If you are interested in contributing more, that would be much appreciated. If you aren't quite sure what you should or could do next, feel free to reach out to me on Slack as "Bryce Meyer" I would be happy to have a quick call to discuss what you are interested in, and to see if we can get you something fun to do.


[QA] Onboarding | Map UI briefly appears during the onboarding

Explorer Alpha build versions:
release21-10-24 - v38
Issue Description:
When entering the explorer with a new account and triggering the onboarding, the map UI briefly appears with a white circle, overlapping the screen showing keyboard instructions. The map UI should not be visible during the onboarding process.
STR:

Enter the explorer with a new account.
Trigger the FTUE onboarding.
Observe the map UI briefly appearing with a white circle during the screen that explains how to control the avatar with the keyboard.

❌ Actual Result
The map UI briefly appears, overlapping with the FTUE screen showing keyboard instructions.
✅ Expected Result
The map UI should not appear during the FTUE onboarding process.
Evidence:

Additional Notes:
Windows 11. 12th Gen Intel(R) Core(TM) i7-12650H 2.30 GHz. nvidia geforce RTX
Macbook M1 Pro Sonoma 14.5

This issue is no longer happening on 0.46.0


Support for ArgoCD config management plugins

Great project idea, I see in the README that CMP are not supported at the moment though.
Is this planned to be implemented, or not possible at all to implement?
Would be great to have support previewing changes of applications using argocd-lovely and other plugins.

Ok, this can be achieved by using kustomize
PoC available here: https://github.com/davidfrickert/argocd-diff-preview/

Hi @davidfrickert - Thank you for showing interest in the project :)

not supported

What this means is that CMPs haven’t been tested yet, so I can’t confirm if they work properly with the tool.
I haven’t had the chance to explore CMPs much myself, so my experience with them is somewhat limited. From what I understand, there are a variety of ways to install and configure CMPs, which makes it tricky to develop a solution that supports every possible case while still keeping the tool user-friendly and customizable.
That said, if you have specific suggestions or ideas on how to integrate CMPs, I’d be happy to discuss them further!
And thank you for showing me your PoC. Maybe I should support adding kustomize patches to the argocd installation ⬇  In this case you could put your patch file in a kustomize-patches folder and it would be applied when installing argocd ⬇ How does that sound?
jobs:
  build:
    ...
    steps:
      ...
    - name: Generate Diff
      run: |
        docker run \
          --network=host \
          -v /var/run/docker.sock:/var/run/docker.sock \
          -v $(pwd)/main:/base-branch \
          -v $(pwd)/pull-request:/target-branch \
          -v $(pwd)/output:/output \
          -v $(pwd)/kustomize-patches:/kustomize-patches \          ⬅ Place all patches here
          -e TARGET_BRANCH=${{ github.head_ref }} \
          -e REPO=${{ github.repository }} \
          dagandersen/argocd-diff-preview:v0.0.15

No problem!
Yeah this sounds good. From my understanding CMPs are always atleast a container in the Repo server pod, so kustomize patches passed to the container should be enough!

Cool - i'll write that on my todo-list! :)
Thank you for this feature suggestion!

Hi @davidfrickert,
I’ve just released version 0.0.18, which installs Argo CD via a Helm Chart. This allows users to customize any Argo CD parameter they wish (as long as it’s supported by the Helm chart, of course).
From a tool perspective, supporting custom Argo CD installations (and CMPs) with a Helm chart is easier compared to Kustomize, since users only need to provide a single values.yaml file with their configuration overrides.
I believe you can install argocd-lovely through the Helm Chart. The chart includes values like repoServer.extraContainers and repoServer.volumes, which allow you to add your argocd-lovely-plugin container and volume. If you encounter any issues, please let me know.
Info on how to override the Argo CD installation

that should work fine! i'll report back when i have tested it

@davidfrickert
Did you have time to test it? 🚀

I wanted to but then I remembered I had another patch on my fork which is to be able to parametrize the kind cluster name due to using this in CI 😅

We can just add an option to specify "cluster-name" @davidfrickert!  Just like you did in your PoC
You can create the PR in case you want a contribution on the repo :) otherwise, I'll just do it in the next release 🚀

We can just add an option to specify "cluster-name" @davidfrickert!  Just like you did in your PoC
You can create the PR in case you want a contribution on the repo :) otherwise, I'll just do it in the next release 🚀

Great, I will try to submit a PR this week!

Since the cluster name is customizable as part of v0.0.24, I look forward to hearing if you can get it to work with your plugin! 🚀

went ahead and tested it but doesn't seem to work @dag-andersen
for some reason, the tool is ignoring many of my applications:
✨ Running with:
✨ - local-cluster-tool: Kind
✨ - base-branch: master
✨ - target-branch: fix/use-official-argocd-diff-preview-image
✨ - secrets-folder: ./secrets
✨ - output-folder: ./output
✨ - repo: davidfrickert/gitops-apps
✨ - timeout: 120 seconds
✨ - file-regex: rendered-apps.yaml
🤖 Fetching all files in dir: base-branch
🤖 Found 42 applications
🤖 Patching applications for branch: master
🤖 Patching 42 Argo CD Application[Sets] for branch: master
🤖 Fetching all files in dir: target-branch
🤖 Found 42 applications
🤖 Patching applications for branch: fix/use-official-argocd-diff-preview-image
🤖 Patching 42 Argo CD Application[Sets] for branch: fix/use-official-argocd-diff-preview-image
🚀 Creating cluster...
🚀 Cluster created successfully
🤫 Applied 1 secrets
🦑 Installing Argo CD Helm Chart version: 'latest'
🦑 Waiting for Argo CD to start...
🦑 Argo CD is now available
🦑 Logging in to Argo CD through CLI...
🦑 Argo CD installed successfully
🌚 Getting resources from base-branch
⏳ Waiting for 15 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 120 seconds...
⏳ Waiting for 15 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 115 seconds...
⏳ Waiting for 15 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 110 seconds...
⏳ Waiting for 15 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 105 seconds...
⏳ Waiting for 2 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 99 seconds...
🌚 Got all resources from 15 applications for base
🧼 Removing applications
🧼 Removed applications successfully
🌚 Getting resources from target-branch
⏳ Waiting for 15 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 120 seconds...
⏳ Waiting for 1 out of 15 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 114 seconds...
🌚 Got all resources from 15 applications for target
💥 Deleting cluster...
🔮 Generating diff between master and fix/use-official-argocd-diff-preview-image
🙏 Please check the ./output/diff.md file for differences
🎉 Done in 140 seconds
Command completed after 1 attempt(s).

this doesn't happen on my patched image:
✨ Running with:
✨ - local-cluster-tool: Kind
✨ - base-branch: master
✨ - target-branch: renovate/velero-velero-plugin-for-aws-1.x
✨ - secrets-folder: ./secrets
✨ - output-folder: ./output
✨ - repo: davidfrickert/gitops-apps
✨ - timeout: 120 seconds
✨ - file-regex: rendered-apps.yaml
🚀 Creating cluster...
🚀 Cluster created successfully
🦑 Installing Argo CD version: 'stable'
🦑 Waiting for Argo CD to start...
🦑 Argo CD is now available
🦑 Logging in to Argo CD through CLI...
🦑 Argo CD installed successfully
🤫 Applied 1 secrets
🤖 Fetching all files in dir: base-branch
🤖 Patching applications for branch: master
🤖 Patching 42 Argo CD Application[Sets] for branch: master
🤖 Fetching all files in dir: target-branch
🤖 Patching applications for branch: renovate/velero-velero-plugin-for-aws-1.x
🤖 Patching 42 Argo CD Application[Sets] for branch: renovate/velero-velero-plugin-for-aws-1.x
🌚 Getting resources for base
⏳ Waiting for 42 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 120 seconds...
⏳ Waiting for 18 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 113 seconds...
⏳ Waiting for 17 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 107 seconds...
⏳ Waiting for 15 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 102 seconds...
⏳ Waiting for 15 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 97 seconds...
⏳ Waiting for 2 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 91 seconds...
🌚 Got all resources from 42 applications for base
🧼 Removing applications
🧼 Removed applications successfully
🌚 Getting resources for target
⏳ Waiting for 15 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 118 seconds...
⏳ Waiting for 4 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 111 seconds...
⏳ Waiting for 1 out of 42 applications to become 'OutOfSync'. Retrying in 5 seconds. Timeout in 106 seconds...
🌚 Got all resources from 42 applications for target
💥 Deleting cluster...
🔮 Generating diff between master and renovate/velero-velero-plugin-for-aws-1.x
🙏 Please check the ./output/diff.md file for differences
🎉 Done in 168 seconds
Command completed after 1 attempt(s).

So I just added the custom values and mounted the file to the container and changed to v0.0.24 image from my patched image
      - name: Set ArgoCD Custom Values
        run: |
          cat > values.yaml << "EOF"
          repoServer:
            extraContainers:
            # ArgoCD Lovely plugin - https://github.com/crumbhole/argocd-lovely-plugin
              - name: lovely-plugin
                image: ghcr.io/crumbhole/lovely:1.1.1
                securityContext:
                  runAsNonRoot: true
                  runAsUser: 999
                volumeMounts:
                    # Import the repo-server's plugin binary
                  - mountPath: /var/run/argocd
                    name: var-files
                  - mountPath: /home/argocd/cmp-server/plugins
                    name: plugins
                  - mountPath: /tmp
                    name: lovely-tmp
            volumes:
              - emptyDir: {}
                name: lovely-tmp
          EOF

      - name: Generate Diff
        id: argocd-diff-preview
        uses: nick-fields/retry@v3
        with:
          timeout_minutes: 6
          max_attempts: 5
          continue_on_error: true
          command: |
            docker run \
              --network=host \
              -v $(pwd)/master:/base-branch \
              -v $(pwd)/pull-request:/target-branch \
              -v $(pwd)/output:/output \
              -v $(pwd)/secrets:/secrets \
              -v $(pwd)/values.yaml:/argocd-config/values.yaml \
              -e BASE_BRANCH=${{ github.event.pull_request.base.ref }} \
              -e TARGET_BRANCH=${{ github.head_ref }} \
              -e REPO=${{ github.repository }} \
              -e TIMEOUT=120 \
              -e FILE_REGEX=rendered-apps.yaml \
              -e CLUSTER_NAME=argocd-diff-preview-${{steps.kind-id.outputs.id}} \
              -e DOCKER_HOST="tcp://localhost:2376" \
              -e DOCKER_TLS_CERTDIR=/certs \
              -e DOCKER_CERT_PATH=/certs/client \
              -e DOCKER_TLS_VERIFY=1 \
              -e DOCKER_BUILDKIT=1 \
              -v /certs:/certs \
              dagandersen/argocd-diff-preview:v0.0.24

Interesting! Since v0.0.22, the tool will ignore Applications, which file did not change AND does not have any references to the repository that it is running on. The reason was that there is no reason to render these apps, because we know they will produce the same result for both branches. So I believe what you are seeing is expected behavior.
You can run the tool with --debug to see what is happening in more detail :) It should write something like Skipping application XXX, because it has not changed for each of the skipped apps.

@davidfrickert - Since v0.0.22, the tool ignores Applications whose files haven't changed and that don't contain any references to the repository it’s running on. This is because rendering these apps would produce the same result for both branches, so there's no need to process them. I believe the behavior you're seeing is expected.
You can run the tool with --debug to see more details about what's happening. It should log messages like Skipping application XXX, because it has not changed for each skipped app.
But that's a good point! I need to add a few log messages describing how/why some of the apps are being ignored. If it is not running in debug, it seems like it randomly goes from 42 to 15 apps.

Aha okay, that explains it. Then all good I guess :)

Thanks for the great work @dag-andersen !

@davidfrickert Great! So, you got it working with the code you shared in this comment, and it worked as expected? If so, I'll add that to the documentation as an example 🚀

@davidfrickert Great! So, you got it working with the code you shared in this comment, and it worked as expected? If so, I'll add that to the documentation as an example 🚀

Yep I think so, although that is only a snippet, would you like the full workflow?
Anyways, I will be monitoring along the next days how this handles incoming updates in my repo!


[docs] Contrast built-in and external providers

Consistently defines built-in providers and explains the differences with external providers on the provider page. Resolves #458

Welp @ericgregory I see it was set to auto-merge! Feel free to take or leave any of those suggestions!


Added attributes config option

Added the attributes config options. It makes it possible to pass custom attributes for the generated img, link, style, picture, and the container element for the background images.
Interface
declare interface Attributes {
  container?: AllHTMLAttributes<HTMLElement>;
  picture?: HTMLAttributes<HTMLPictureElement>;
  style?: StyleHTMLAttributes<HTMLStyleElement>;
  link?: Omit<
    LinkHTMLAttributes<HTMLLinkElement>,
    "as" | "rel" | "imagesizes" | "imagesrcset"
  >;
  img?: Omit<
    ImgHTMLAttributes<HTMLImageElement>,
    | "src"
    | "alt"
    | "srcset"
    | "sizes"
    | "width"
    | "height"
    | "loading"
    | "decoding"
  >;
}

Note: Imgs doesn't support the picture and container keys, Pictures doesn't support the container key, and BackgroundImages doesn't support the img and picture keys.

@KyleFontenot you may be interested in this PR. Any opinion will be appreciated.

Merging this PR!


ceylon war tool should be able to package .js files from a Ceylon module

[@sgalles] As I'm not sure what this packaging feature should look like, I quote @gavinking :
I guess it would be nice if ceylon war packaged .js files into a scripts directory except we’re actually looking into some slightly more sophisticated stuff than this. For example, a servlet that proxies the CMR
[Migrated from ceylon/ceylon-compiler#2429]

Perhaps some kind of option where you can specify Ceylon modules compiled for the JS backend to include in the WAR would indeed be useful. But not really urgent, moving to 1.3.


Please enable Azure Active Directory authentication for Azure

Please enable the authentication of an API call using an Azure Active Directory token for users who are unable to use API keys.
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#authentication

Hi @DushanthaS
Sorry for the delayed response. What auth method are you using most common types are already supported.

ApiKey
OAuth2
OpenID Connect

NOTE: You may need to setup the configuration in "Options" if you're using AAD for example

There is a standard class for security which can be overridden or updated with you tenant and app reg information.

sorry for the late reply , I am actually trying to use your VS code plugin.


Add support for vuetify  component

Version
Latest magpie
Problem
Images defined in the following component are not caught by magpie
<v-img :src="urlToSource"></v-img>

Example HTML before generation
<v-img :src="generateLink(item.image)"></v-img>

Expected HTML after generation

Actual HTML after generation

Notes
Works fine if just using standard  tag but this method causes the image to lose the aspect ratio and other stylings that come with vuetify.

@emiliobondioli I have narrowed down the issue to the regex in module.js, specifically here:
  function getImageUrls (string) {
    const urls = []
    const test = new RegExp(
      '(https?)(://.*?.)(?:' + options.extensions.join('|') + ')',
      'g'
    )
    ...
  }

The current regex is generated as:
/(https?)(:\/\/.*?.)(?:jpg|jpeg|gif|png|webp)/g

However this pattern doesn't match correct in all cases as seen here: https://regexr.com/5f6ls. As you can see the pattern will match across multiple tags and even across JSON+LD structured data.
I have tried to use a different pattern but I am also issues as well as seen here: https://regexr.com/5f6m2.

Also one more thing, using a CMS like strapi can sometimes generate image URLs that have uppercase extensions like this
{
  "id":2,
  "name":"teset",
  "published_at":"2020-10-13T09:33:23.577Z",
  "created_at":"2020-10-13T09:32:51.814Z",
  "updated_at":"2020-10-27T12:25:49.027Z",
  "image":{
    "id":3,
    "name":"DSC00044.JPG",
    "width":5152,
    "height":3864,
    "formats":{
      "thumbnail":{
        "name":"thumbnail_DSC00044.JPG",
        "hash":"thumbnail_DSC_00044_167687f904",
        "ext":".JPG",
        "mime":"image/jpeg",
        "width":208,
        "height":156,
        "size":9.9,
        "path":null,
        "url":"/uploads/thumbnail_DSC_00044_167687f904.JPG"
      },
      "large":{
        "name":"large_DSC00044.JPG",
        "hash":"large_DSC_00044_167687f904",
        "ext":".JPG",
        "mime":"image/jpeg",
        "width":1000,
        "height":750,
        "size":144.3,
        "path":null,
        "url":"/uploads/large_DSC_00044_167687f904.JPG"
      },
      "medium":{
        "name":"medium_DSC00044.JPG",
        "hash":"medium_DSC_00044_167687f904",
        "ext":".JPG",
        "mime":"image/jpeg",
        "width":750,
        "height":563,
        "size":91.9,
        "path":null,
        "url":"/uploads/medium_DSC_00044_167687f904.JPG"
      },
      "small":{
        "name":"small_DSC00044.JPG",
        "hash":"small_DSC_00044_167687f904",
        "ext":".JPG",
        "mime":"image/jpeg",
        "width":500,
        "height":375,
        "size":45.62,
        "path":null,
        "url":"/uploads/small_DSC_00044_167687f904.JPG"
      }
    },
    "hash":"DSC_00044_167687f904",
    "ext":".JPG",
    "mime":"image/jpeg",
    "size":2210.57,
    "url":"/uploads/DSC_00044_167687f904.JPG",
    "previewUrl":null,
    "provider":"local",
    "provider_metadata":null,
    "created_at":"2020-10-27T12:24:41.478Z",
    "updated_at":"2020-10-28T01:43:19.838Z"
  }
}

The current regex will ignore these images because of the capitalized JPG.

Also, the current regex pattern ignores capitalized extensions. For example:
Will match
http://localhost:1337/uploads/img_123.jpg

Will not match
http://localhost:1337/uploads/img_123.JPG

I ran into this problem because my camera saves images in this format. Adding /gi to the regex pattern seems to fix it okay.

After looking further, this may be a nuxt issue and not related to magpie. The images are not generated in the html when static is true, regardless of magpie.
However, I also did notice the the current regex in magpie would not catch the url("http://localhost:1337/uploads/small_test_Image1_c263df17a1.jpg") styling. This should be in another issue though.

Hi! Sorry for the late reply!
I've been investigating this issue and I might find have a fix to make this work with Vuetify. It seems v-img lazy loads the images so the actual url won't be found in the generated html or in the payload, but it seems to end up inside the js bundle with the rest of the scripts (the files with the auto generated names like 1ab23a1.js).
I can try to can parse those files together with the pages' html and payload so that Magpie can also catch remote images from there.
As for the second issue, I'll try to reproduce it and let you know :)


Filesystem Write Fails with Some SD-Cards

Writing Data to a file on the SD-card fails with the version *** Booting Zephyr OS build v3.1.99-ncs1-9-g85edda1e989c *** of zephyr. The Failure depends on the card and on the data being written. It fails with the cards Lexar 300x 32GB microSD HC and Transcend 4GB microSD HC. It succeeds with Delkin Devices. Utility + 4GB microSD HC. If the data is all zeros instead of initialized via the function provided below, the test always succeeds.
See more details and failing unit-test on target below.
This is a regression and it worked fine with version 2.6.
The target board is a variant of nrf9160ns.
To Reproduce
Steps to reproduce the behavior:

add the following test and invoke it with twister:

#include <fs/fs.h>
#include <ztest.h>

#define BUFFER_SIZE (1<<15)
#define BUFFER_CNT (15)

static char buffer[BUFFER_SIZE] = {0};

static void init_buffer(void){
    for(int i=0;i<BUFFER_SIZE;i++){
        buffer[i]+=3*(1+i);
    }
}

static bool write_file(void){
    struct fs_file_t file;

    fs_file_t_init(&file);
    int rc = fs_open(&file, "/SD:/write_test", FS_O_CREATE | FS_O_RDWR);
    if (rc < 0)
    {
        printk("Failed to open file\n");
        return false;
    }

    ssize_t ret = 0;
    for(int i=0;i<BUFFER_CNT;i++){
        printk("chunk: %d size: %d data: %02x ... %02x\n",
                i,
                BUFFER_SIZE,
                buffer[0],
                buffer[BUFFER_SIZE -1]);
        ret = fs_write(&file, buffer, BUFFER_SIZE);
        if (ret < 0)
        {
            printk("Failed to write %d\n", ret);
            printk("WRITTEN %d PACKETS A %d = %d\n", i, BUFFER_SIZE,i*BUFFER_SIZE);
            return false;
        }
    }

    rc = fs_truncate(&file, BUFFER_CNT*BUFFER_SIZE);
    if(rc < 0)
    {
        printk("error truncating file\n");
        return false;
    }

    rc = fs_close(&file);
    if (rc < 0)
    {
        printk("error closing file\n");
        return false;
    }

    return true;
}

static void test_write(void)
{
    init_buffer();
    zassert_true(write_file(), "failed to write file");
}

void test_main(void)
{
    ztest_test_suite(test_write_file,
            ztest_unit_test(test_write)
            );

    ztest_run_test_suite(test_write_file);
}

Expected behavior
The test should pass independently of the data written to the file and of the SD card being used.
Impact
Show stopper. We are likely not able to update zephyr version without a workaround.
Logs and console output
Failing test result using v3.1.99 and Transcend 4GB microSD HC:
*** Booting Zephyr OS build v3.1.99-ncs1-9-g85edda1e989c  ***
I: Start initializing SDHC.
I: Detected SD card
I: Maximum SD clock is under 25MHz, using clock of 8000000Hz
I: Block count 7680000
I: Sector size 512
I: Memory Size(MB) 3750
I: Disk mounted under "/SD:"
CENSOR
I: SDHC initialization succeeded.
Running TESTSUITE test_write_file
===================================================================
START - test_write
chunk: 0 size: 32768 data: 03 ... 00
chunk: 1 size: 32768 data: 03 ... 00
chunk: 2 size: 32768 data: 03 ... 00
chunk: 3 size: 32768 data: 03 ... 00
chunk: 4 size: 32768 data: 03 ... 00
chunk: 5 size: 32768 data: 03 ... 00
E: Write failed
E: file write error (-5)
Failed to write -5
WRITTEN 5 PACKETS A 32768 = 163840
Assertion failed at CMAKE_SOURCE_DIR/src/main.c:62: test_write: (write_file() is false)
failed to write file
FAIL - test_write in 17.438 seconds
===================================================================
TESTSUITE test_write_file failed.
===================================================================
RunID: fba8dcb6ff2a643162217b8944b19ed0
PROJECT EXECUTION FAILED

Succeeding test result using v3.1.99 and Delkin Devices. Utility + 4GB microSD HC:
DEBUG   - DEVICE: *** Booting Zephyr OS build v3.1.99-ncs1-9-g85edda1e989c  ***
DEBUG   - DEVICE: I: Start initializing SDHC.
DEBUG   - DEVICE: I: Detected SD card
DEBUG   - DEVICE: I: Maximum SD clock is under 25MHz, using clock of 8000000Hz
DEBUG   - DEVICE: I: Block count 7576576
DEBUG   - DEVICE: I: Sector size 512
DEBUG   - DEVICE: I: Memory Size(MB) 3699
DEBUG   - DEVICE: I: Disk mounted under "/SD:"
CENSOR
DEBUG   - DEVICE: I: SDHC initialization succeeded.
DEBUG   - DEVICE: Running TESTSUITE test_write_file
DEBUG   - DEVICE: ===================================================================
DEBUG   - DEVICE: START - test_write
DEBUG   - DEVICE: chunk: 0 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 1 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 2 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 3 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 4 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 5 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 6 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 7 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 8 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 9 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 10 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 11 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 12 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 13 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: chunk: 14 size: 32768 data: 03 ... 00
DEBUG   - DEVICE: PASS - test_write in 1.313 seconds
DEBUG   - DEVICE: ===================================================================
DEBUG   - DEVICE: TESTSUITE test_write_file succeeded
DEBUG   - DEVICE: ===================================================================
DEBUG   - DEVICE: RunID: ff4fd55abfb0591aa484e9d6b58d34af
DEBUG   - DEVICE: PROJECT EXECUTION SUCCESSFUL

Environment:

Ubuntu 22.04.1 LTS running as VM
Nordic SDK

Furthermore

Changes to device tree:

-    compatible = "zephyr,mmc-spi-slot";
-    reg = <0>;
-    status = "okay";
-    label = "SDHC0";
-       spi-max-frequency = <8000000>; // Maximum of NRF91
+      compatible = "zephyr,sdhc-spi-slot";
+      reg = <0>;
+      status = "okay";
+      mmc {
+          compatible = "zephyr,sdmmc-disk";
+          status = "okay";
+      };
+      spi-max-frequency = <8000000>; // Maximum of NRF91
  };

Changes to kconfig:

+CONFIG_DISK_ACCESS=y
+CONFIG_DISK_DRIVERS=y
+CONFIG_SDMMC_SUBSYS=y
+CONFIG_SPM=y
+CONFIG_ZTEST_STACK_SIZE=4096

This is how we initialize the SD-card using SYS_INIT. We do it early in the boot process (high priority).:

    static const char *disk_pdrv = "SD";
    uint64_t memory_size;
    uint32_t block_count;
    uint32_t block_size;

    LOG_INF("Start initializing SDHC.");

    const struct device *gpio = DEVICE_DT_GET(DT_NODELABEL(gpio0));
    if (!gpio)
    {
        LOG_ERR("Failed to get GPIO device.");
        return false;
    }
    gpio_pin_configure(gpio, DT_GPIO_PIN(DT_ALIAS(sdhcsw), gpios), GPIO_OUTPUT);
    gpio_pin_set(gpio, DT_GPIO_PIN(DT_ALIAS(sdhcsw), gpios), 0);

    int err = disk_access_init(disk_pdrv);
    if (err)
    {
        LOG_ERR("Storage init ERROR: %d!", -err);
        return false;
    }

    if(disk_access_ioctl(disk_pdrv, DISK_IOCTL_GET_SECTOR_COUNT, &block_count)) {
        LOG_ERR("Unable to get sector count!");
        return false;
    }
    LOG_INF("Block count %u", block_count);

    if(disk_access_ioctl(disk_pdrv, DISK_IOCTL_GET_SECTOR_SIZE, &block_size)) {
        LOG_ERR("Unable to get sector size!");
        return false;
    }
    LOG_INF("Sector size %u", block_size);

    memory_size = (uint64_t)block_count * block_size;
    LOG_INF("Memory Size(MB) %u", (uint32_t)(memory_size >> 20));

    mp.mnt_point = disk_mount_pt;

    int res = fs_mount(&mp);

    if (res == FR_OK) {
        LOG_INF("Disk mounted under \"%s\"", mp.mnt_point);
    }
    else {
        LOG_ERR("Error mounting disk!");
        return false;
    }
...

Let me know if more information is needed to reproduce.

@clamattia Can you try running this test with CONFIG_SDHC_LOG_LEVEL_DBG=y, and CONFIG_SD_LOG_LEVEL_DBG=y? I've tried to reproduce this failure with a few cards I have, and haven't had any luck

Thanks for the quick response.
You might need to increase the BUFFER_CNT to reproduce.
And I made it repeat the test with different data to increase the chance of the bug:
for(int i=0;i<10;i++){
        init_buffer();
        zassert_true(write_file(), "failed to write file");
    }

and added some data variation:
buffer[i]+=3*(1+i)+buffer[i];

The bug must be timing sensitive. If I enable the log output as you suggested or if I put a k_msleep(100) in the loop, it changes the behavior of the bug.
Debug Log for the Lexar 300x 32GB microSD HC:
*** Booting Zephyr OS build v3.1.99-ncs1-9-g85edda1e989c  ***
I: Start initializing SDHC.
D: Resetting power to card
D: cmd0 arg 0x0
D: cmd8 arg 0x1aa
D: Found SDHC with CMD8 support
D: cmd59 arg 0x1
D: cmd5 arg 0x0
I: Detected SD card
D: cmd0 arg 0x0
D: cmd8 arg 0x1aa
D: Found SDHC with CMD8 support
D: cmd59 arg 0x1
D: cmd58 arg 0x0
D: cmd55 arg 0x0
D: cmd41 arg 0x40000000
D: cmd55 arg 0x0
D: cmd41 arg 0x40000000
D: cmd55 arg 0x0
D: cmd41 arg 0x40000000
D: SDMMC responded to ACMD41 after 2 attempts
D: cmd58 arg 0x40000000
D: cmd10 arg 0x0
D: Card MID: 0x28, OID: EB
D: cmd9 arg 0x0
D: Card block count 62705664, block size 512
I: Maximum SD clock is under 25MHz, using clock of 8000000Hz
D: cmd55 arg 0x0
D: cmd51 arg 0x0
D: SD reports specification version 8
I: Block count 62705664
I: Sector size 512
I: Memory Size(MB) 30618
D: READ: Sector = 0, Count = 1
D: cmd17 arg 0x0
D: READ: Sector = 8192, Count = 1
D: cmd17 arg 0x2000
D: READ: Sector = 8193, Count = 1
D: cmd17 arg 0x2001
I: Disk mounted under "/SD:"
D: READ: Sector = 40960, Count = 1
D: cmd17 arg 0xa000
D: READ: Sector = 44352, Count = 1
D: cmd17 arg 0xad40
CENSOR
I: SDHC initialization succeeded.
Running TESTSUITE test_write_file
===================================================================
START - test_write
D: READ: Sector = 40960, Count = 1
D: cmd17 arg 0xa000
...............................chunk: 0 size: 32768 data: 03 ... 00
D: WRITE: Sector = 63552, Count = 32
D: cmd25 arg 0xf840
D: cmd13 arg 0x0
D: READ: Sector = 10365, Count = 1
D: cmd17 arg 0x287d
D: WRITE: Sector = 63584, Count = 32
D: cmd25 arg 0xf860
D: cmd13 arg 0x0
...............................chunk: 1 size: 32768 data: 03 ... 00
D: WRITE: Sector = 63616, Count = 32
D: cmd25 arg 0xf880
D: cmd13 arg 0x0
D: WRITE: Sector = 63648, Count = 32
D: cmd25 arg 0xf8a0
D: cmd13 arg 0x0
...............................chunk: 2 size: 32768 data: 03 ... 00
D: WRITE: Sector = 63680, Count = 32
D: cmd25 arg 0xf8c0
D: cmd13 arg 0x0
D: WRITE: Sector = 63712, Count = 32
D: cmd25 arg 0xf8e0
D: cmd13 arg 0x0
E: Card did not return to ready state
E: Write failed
E: file write error (-5)
Failed to write -5
WRITTEN 2 PACKETS A 32768 = 65536
Assertion failed at CMAKE_SOURCE_DIR/src/main.c:64: test_write: (write_file() is false)
failed to write file
FAIL - test_write in 0.292 seconds
===================================================================
TESTSUITE test_write_file failed.
===================================================================
RunID: 99c4093aad855c1545f29078355efbd3
PROJECT EXECUTION FAILED

I was unable to reproduce it on the Transcend today unfortunately.

FYI: I have the same issue in production. Adding the debug log there made the bug go away.

Ok, so it seems the card fails to return to a ready state. Can you try this patch? https://github.com/zephyrproject-rtos/zephyr/pull/53017
This change should poll the SD card for the full duration of CONFIG_SD_DATA_TIMEOUT until it returns to a ready state or times out. Currently the SD card status is only checked once after write

Thanks @danieldegrasse for the quick patch. On a first glance it seems to improve but not completely fix it.
Now I am unable to reproduce the issue with the logs enabled. Will try some more later. But with the logs disable I am still able to get the error:
...
...............................chunk: 100 size: 32768 data: 03 ... 00
...............................chunk: 101 size: 32768 data: 03 ... 00
...............................chunk: 102 size: 32768 data: 03 ... 00
E: Write failed
E: file write error (-5)
Failed to write -5
WRITTEN 102 PACKETS A 32768 = 3342336
Assertion failed at CMAKE_SOURCE_DIR/src/main.c:64: test_write: (write_file() is false)
failed to write file
FAIL - test_write in 23.516 seconds
===================================================================
TESTSUITE test_write_file failed.

On a first glance it seems unrelated to CONFIG_SD_DATA_TIMEOUT. I set CONFIG_SD_DATA_TIMEOUT=60000 but it failed already after ~<10s.
I will try/investigate some more.

I was able to reproduce the problem with the both "bad"-cards, even with the patches.

On a first glance it seems unrelated to CONFIG_SD_DATA_TIMEOUT. I set CONFIG_SD_DATA_TIMEOUT=60000 but it failed already after ~<10s.

Do you see this behavior even with the patch? Without the patch, CONFIG_SD_DATA_TIMEOUT wouldn't have an effect, as the SD card's ready status is only checked once after a write, without delays

It is hard to tell now. To increase the likelihood of the bug I removed all debug out. So it is hard to tell, when the timeout is starting and ending.
I re-enabled some debug out to check the timing. But then unfortunately it does not happen anymore.
But I think I have found what was causing the ignoring of the timeout earlier (unrelated) and believe we can assume, that the timeout is now respected. If you have an idea, how I could check this (without debug logging) let me know.

One way you can check the location of the error return would be to change the return code at the SD subsystem level. For example, I believe that your error is occurring on this line: https://github.com/nxp-zephyr/zephyr/blob/065b7ac02661d580b9300d4c1c1048874671f118/subsys/sd/sdmmc.c#L1421 (Note I've linked to code on the patched branch, as that is what I assume you're working from now). If you changed that return code to something obvious (say 0xDEADBEEF), you could likely get a better idea of if that is the location the error occurs at.

@danieldegrasse I can say confidently now that:

The timeout is not respected. I put it to 600 seconds  CONFIG_SD_DATA_TIMEOUT=600000 the test still fails after 20.4 seconds.

START - test_write
E: Write failed
E: file write error (-5)
Failed to write -5
WRITTEN 48 PACKETS A 32768 = 1572864
Assertion failed at CMAKE_SOURCE_DIR/src/main.c:136: test_write: (write_file() is false)
failed to write file
FAIL - test_write in 20.463 seconds

I had to change the transforming of the error codes in a few places, such that that I could figure out where the error first occurs.
It is actually https://github.com/nxp-zephyr/zephyr/blob/065b7ac02661d580b9300d4c1c1048874671f118/subsys/sd/sdmmc.c#L1358 called from https://github.com/nxp-zephyr/zephyr/blob/065b7ac02661d580b9300d4c1c1048874671f118/subsys/sd/sdmmc.c#L1410 and not the timeout.

The timeout is not respected. I put it to 600 seconds CONFIG_SD_DATA_TIMEOUT=600000 the test still fails after 20.4 seconds.

Just to be clear here, from what you are describing it sounds like the timeout is being respected, just that the card responds to a query after some time. In sdmmc_wait_ready, CONFIG_SD_DATA_TIMEOUT is treated as the maximum time the subsystem will wait for the card to respond before returning a timeout error. Based off the error location, it seems that ACMD22 is failing on your card. I would expect your card to support this command, as it is mandatory per the specification.
A few things that might help figure out the root cause here:

Is this the first time the write fails? If so, could you log the error code?
If you ignore the error and retry the call to fs_write, does the card write data?
In the SD implementation for v2.6.0, there was logic to retry a failed write in SPI mode: https://github.com/zephyrproject-rtos/zephyr/blob/zephyr-v2.6.0/drivers/disk/sdmmc_spi.c#L471. If possible, could you check if this function is called while running this test on v2.6.0? Not sure if you've got a branch you can revert to.

Also, I've added emulation of the retry behavior in v2.6.0 on this sha: https://github.com/nxp-zephyr/zephyr/tree/fdcf37a65a033d59081d8807f72866185d15ef59. Could you try this, and see if the error recovery logic can fix your issue?

Thanks for the help. Yes I did not express myself well. At first I thought, that maybe the timeout was ignored. But in fact, as you point out, and as I later found out by figuring out the exact source of the problem, it is not the timeout causing the issue.
The error code is -5 returned here https://github.com/nxp-zephyr/zephyr/blob/065b7ac02661d580b9300d4c1c1048874671f118/subsys/sd/sdmmc.c#L1358 (I will double check just to be sure)
I will try the other suggestions today.
Thank you.

I added a retry logic (with additional sleep) to call fs_write again in case of error. This did not help with the bug. (It failed the second time also, always).
The suggested sha does avoid the error from happening, but the test gets stuck and cannot proceed. I could not yet figure out why. I tried to debug using printk but cannot reproduce problem then (timing issue). I also put the timeout back to CONFIG_SD_DATA_TIMEOUT=1000. Maybe the card never returns from busy state?

One more thing I observed, The "good" card supports UHS-I with speed class 3. One of the "bad" cards does only support UHS-I with speed class 1, while the other does not seem to support UHS at all (seems to only support regular speed with class 10). Not sure if relevant.

I put the printks before each call to fs_write which makes the bug disappear.

Does the test get stuck at the same location the error previously occurred at?
Do you have a suggestion how to find out? Or where I could add a printk to help us debug the issue?

Can adding a call to printk before each fs_write call resolve the issue without any other changes (ie on the latest revision of Zephyr main), or does the fix only work with the changes in https://github.com/nxp-zephyr/zephyr/tree/fdcf37a65a033d59081d8807f72866185d15ef59?

I went back to before the proposed fixes. Note, that this is not zephyr main but a fork based on 3.1.99 https://github.com/nrfconnect/sdk-zephyr/commits/v3.1.99-ncs1. Is something important missing from upstream? I saw, that the history of subsys/sd and /driver/sdhc/ have quite some additions since then.
Compare for example: https://github.com/nrfconnect/sdk-zephyr/commits/86893246053c50d34618f09ec6722cae8ba19472/drivers/sdhc and https://github.com/nrfconnect/sdk-zephyr/commits/86893246053c50d34618f09ec6722cae8ba19472/subsys/sd with https://github.com/nxp-zephyr/zephyr/commits/fdcf37a65a033d59081d8807f72866185d15ef59/drivers/sdhc and https://github.com/nxp-zephyr/zephyr/commits/065b7ac02661d580b9300d4c1c1048874671f118/subsys/sd/sdmmc.c
Doing so, I can get the write error still. If I then add the printk before the fs_write, I do not get the write error but instead I get a -5 error when attempting to close the file. (Sorry this might not be helpful).
Note on how I tested the your proposed fixes from: I cherry picked them (and previous commits in that history) on our fork based on 3.1.99.

One more note. Before the proposed fix (and the other cherry-picks from that history), I can reproduce the bug with the DEBUG_LOG enabled:
D: cmd13 arg 0x0
D: WRITE: Sector = 473728, Count = 64
D: cmd25 arg 0x73a80
D: cmd13 arg 0x0
WRITTEN 452 PACKETS A 32768 = 14811136
attempting to closeD: READ: Sector = 8192, Count = 1
D: cmd17 arg 0x2000
D: WRITE: Sector = 8192, Count = 1
D: cmd24 arg 0x2000
D: cmd13 arg 0x0
E: file close error (-5)
error closing file

With the proposed fix (and the other cherry-picks from that history) and the DEBUG_LOG enabled, the bug can not be reproduced. Only with debug log disabled.
This suggests, that there might be two separate issues. One fixed by the proposal and a separate, timing-sensitive issue.

I went back to before the proposed fixes. Note, that this is not zephyr main but a fork based on 3.1.99<EMAIL_ADDRESS>(commits). Is something important missing from upstream? I saw, that the history of subsys/sd and /driver/sdhc/ have quite some additions since then.

Thanks for the clarification- there are two commits that might be useful since the branch you linked:

https://github.com/nxp-zephyr/zephyr/commit/2c2b8c342247ff767ee369169a74dbcf09b5edc6: This adds support for f_sync to the SPI mode SD driver. I am unsure if f_sync is being called internally by the filesystem, but this could cause issues if you don't have an implementation of the function present on your branch.

https://github.com/nxp-zephyr/zephyr/commit/ba9220c5bdf951983bcee3f6c49b195a066d98e8: This fixed a timing bug seen when the SD subsystem was compiled with CONFIG_SPEED_OPTIMIZATIONS=y. However, this was only observed on cards running in native SD mode, and the failure seen in that issue doesn't seem similar to the one you experienced. Either way, that fix is currently only applied to cards running in native mode. You could try adding a similar k_busy_wait call here: https://github.com/nxp-zephyr/zephyr/blob/ba9220c5bdf951983bcee3f6c49b195a066d98e8/subsys/sd/sdmmc.c#L975 (before the call to sdmmc_send_ocr) if you want to try this fix.

Doing so, I can get the write error still. If I then add the printk before the fs_write, I do not get the write error but instead I get a -5 error when attempting to close the file. (Sorry this might not be helpful).

The -5 error may be due to the support for f_sync being absent (see https://github.com/nxp-zephyr/zephyr/commit/2c2b8c342247ff767ee369169a74dbcf09b5edc6)

Could you try applying these commits?

https://github.com/nxp-zephyr/zephyr/commit/c6122991ba51cf4128064e7ac4cb352ab0e07073
https://github.com/nxp-zephyr/zephyr/commit/2c2b8c342247ff767ee369169a74dbcf09b5edc6

The implementation of sdmmc_wait_ready in the fix I initially proposed did not check if the card was busy prior to reading its status. This commit does, which may help resolve the issue. You'll need the second linked commit to add support for checking the busy status of a card in SPI mode.

Could you try applying these commits?
* [nxp-zephyr@c612299](https://github.com/nxp-zephyr/zephyr/commit/c6122991ba51cf4128064e7ac4cb352ab0e07073)

* [nxp-zephyr@2c2b8c3](https://github.com/nxp-zephyr/zephyr/commit/2c2b8c342247ff767ee369169a74dbcf09b5edc6)

I went back to our fork of version 3.1.99. I applied the two suggested commits (cherry-pick).
With those I can not reproduce the bug under conditions it consistently happens without them.
Great job and congratulations on finding this :+1:
Thanks for the help.
Feel free to close this issue, when appropriate.
[ Out of curiosity, do you know why only some of the SD-cards are affected? Might it be related to the speed-class, such that some of them are always ready again, before zephyr can contact them, while the slower might not be ready? ]

Thanks for the help.

Absolutely, appreciate the prompt responses.

[ Out of curiosity, do you know why only some of the SD-cards are affected? Might it be related to the speed-class, such that some of them are always ready again, before zephyr can contact them, while the slower might not be ready? ]

I believe that is the reason, yes. The changes present in the two commits you applied wait for the card to finish programming before checking its status, while the upstream SD implementation simply checks SD status directly after the write has completed.


[Feature] Replace mock supporting organizations on the challenge profile page

What projects is this feature for?
OpenChallenges
Description
As a prospective participant, I want to know who are supporting the organization of a challenge so that I have more information to decide on whether I want to participate to a given challenge.
As a prospective challenge organizer, I want to know what organization supported Challenge X so that I can contact them for the challenge I organize.
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@vpchung pointed out that we don't currently don't have a way to filter challenges by organization (work in progress, #1403 ) and to get the orgs that contributes to a challenges. I'm marking this ticket as Blocked for now.

Added to Sprint 23.06

@rrchai This task is no longer blocked since we can now filter challenges by org.

@rrchai @tschaffter jumping into the convo here :-)
We already have a table in both the challenge and organization services that has this information available:

I believe the APIs just have to expose this now.

@rrchai @vpchung That's right, we have the data and the challenge service partially supports this feature with the existing Java class ChallengeContributionEntity that links supporting organization to challenges.
The next steps are to define how we want to expose these data in the API description. One option is to add a property to the Challenge object that includes an array of contributors (org id, role). Another option is to provide this information via a dedicated endpoint, e.g. challenges/{challengeId}/contributors.
Option 1 comes the cost of extra backend processing to get challenges (even when contributors data is not needed, e.g. on the search page) and more data to transfer. For the sake of simplicity and performance, I propose to go with the second option. The API client will provide a dedicated method that we can call when needed.
@rrchai @vpchung Does one of you want to give a try adding this new endpoint to the API description?

A dedicated endpoint makes much more sense, imo - thanks!
I most likely won't have time to work on this until near the end of the sprint.  @rrchai , what about you?

@tschaffter @vpchung I can give a try to add the endpoint this sprint.

This ticket depends on #1666, which is planned for next sprint. So I'm also moving this ticket to next sprint. The priority goes to implementing the page state in the URL this sprint.

@tschaffter While I have been already working to update the front-end, I realize the orgs(contributions) can not be retrieved by current ChallengeContributionService and OrganizationService - see "Blocked" in #1668.
Moving to next sprint sounds good to me. Thanks.

@rrchai I will review the API description and backend as I may have forgotten to rename something. I will also update the app so that it will work with the latest client, then merge to main. That way the backend will be ready when we start working on the frontend next sprint.

This ticket is no longer blocked since the completion of #1669.

Blocked by #1666, which will define the UI/UX for displaying the orgs that contributed to a challenge.

I will work on this today.

@tschaffter I think I confused you about the backend support I needed for this ticket.
In order to get the challenge contributors (orgs) in the Challenge Profile page, we could use either of way listed below:

Make organizations (orgId) property accessible in the Challenge object -> Get all contributors of the challenge using all orgIds
Add challenges to OrganizationSearchQuery ->  Get all contributors of the challenge by query using challengeId

Adding "query Org object by login` will support the first approach, but still need to get "organizations" property in Challenge object.
I think the second approach is more straightforward to me, thus I created the associated ticket:

#1874

Do you think we could move this ticket to next sprint and I am more than happy to support ^ backend feature.

@rrchai We can get the list of orgs that contributed to a challenge with the endpoint /challenges/{id}/contributions, which we added recently.
https://dev.openchallenges.io/api/v1/challenges/1/contributions

{
  "number": 0,
  "size": 2,
  "totalElements": 2,
  "totalPages": 1,
  "hasNext": false,
  "hasPrevious": false,
  "challengeContributions": [
    {
      "challengeId": 1,
      "organizationId": 75,
      "role": "sponsor"
    },
    {
      "challengeId": 1,
      "organizationId": 1,
      "role": "sponsor"
    }
  ]
}

Do you think we could move this ticket to next sprint

Absolutely!

Moving to Sprint 23.08

We can get the list of orgs that contributed to a challenge with the endpoint /challenges/{id}/contributions

Oh right. I totally missed the endpoint. Thanks for reminding.


Profile v3: Mute colors for services

Part of https://github.com/studentinsights/studentinsights/issues/1990.
Who is this PR for?
educators
What problem does this PR fix?
Services had an old color scheme that I don't think ever meant much to anyone, and it was only partially implemented for some services.  With the profile v3 changes, the semantic meaning of the colors don't match the meaning elsewhere on profile v3, and these aren't in the same palette either.
What does this PR do?
Mutes the colors for services to gray.
Screenshot (if adding a client-side feature)

Checklists

[x] Author checked latest in IE - Student Profile v3
[x] Author included specs for new code

selfie


About the prediction accuracy after training

Hello author! I reproduced your AV1 training code, adjusted the epoch to 50, and adjusted the batch_size to 4, with no other adjustments made. What is the reason for the significant difference in the validation results obtained after training compared to your validation model?

@XinGP Generally, if you reduce the batch size, you should also reduce the learning rate (ref: https://arxiv.org/pdf/1812.01187.pdf). The relatively large LR will lead to unstable training.

 Hello author! I reproduced your AV1 training code, adjusted the epoch to 50, and adjusted the batch_size to 4, with no other adjustments made. What is the reason for the significant difference in the validation results obtained after training compared to your validation model?
Hello, I've also encountered similar issues while tuning parameters, and I can't pinpoint the specific learning rate. Could you share what learning rate you ended up using? I'd like to discuss this issue further. My email is<EMAIL_ADDRESS>

Have you found a suitable learning rate for batch_size 4 to close to the result which the author gives?

 Hello author! I reproduced your AV1 training code, adjusted the epoch to 50, and adjusted the batch_size to 4, with no other adjustments made. What is the reason for the significant difference in the validation results obtained after training compared to your validation model?
Have you found a suitable learning rate for batch_size 4 to close to the result which the author gives?

@XinGP一般来说，如果减小batch size，也应该减小学习率（ref: https://arxiv.org/pdf/1812.01187.pdf）。LR过大会导致训练不稳定。

Yes, after reducing all learning rates by half, I can obtain validation set results similar to those in the paper


Add rhel dependencies and install instructions

This adds the relevant dependencies needed for installing on RHEL(7) or Centos.

@Shelnutt2: Should I re-review this?

@josephschorr Yes this is ready for re-review.

LGTM


added initFromDate

fixes #52

disregard. invalid PR

Actually the PR is valid. I got triggered by the out-of-date documentation strings for init

👍 Thanks! Looks simple and straightforward and doesn't even cause a major bump.


Revert "tools: Fix CONFIG_BASE_DEFCONFIG generation"

Reverts apache/nuttx#9572 to fix regression described in https://github.com/apache/nuttx/issues/9670

@pkarashchenko maybe instead of reverting it, we could find a solution that works on Linux, BSDs, MacOS and Windows. Without this commit everytime we modify something on menuconfig and save it a new "-dirty" will be added to the name.
So it end up "board:profilename-dirty-dirty-dirty..."

@acassis yes, the goal is to have a universal solution, but I'm blocked on working with master from MAC because of this issue, so I created a revert. I'm fine to wait a few days if someone can provide a universal solution, but I think the correct strategy is:

Revert to working state
Provide universal solution

Ok, agree!

Fixed in #9754


add generic return value for .filter and .sort

This preserves the typing information when calling filter.sort and filter.filter on the SQLAlchemy Filters.
This works with pylance but seems like it's failing with mypy on CI.

I am curious if the maintainer have have any appetite to get an updated version of this PR merged? As it stands the type hints provide aren't overly useful with mypy.

I like when things work properly, so yes, if we get something that works with both pylance and mypy I would be very happy to merge and ship it! Does it help that support for python 3.8 has dropped?


Edit and Resend Issues

I've seen two problems so far

Sometimes it simply does not work, the Network Console  window opens up but attempting to resend from it fails and from then on if you close the window and try again it shows no data.
If it is working resending a message picks up a lot of new headers (presumably from the page context). This is not helpful if you're just trying to resend the original message.

Edit and Resend via the tunnel inspection page is not a feature we specifically supported so you mileage may vary with it.
I've marked this as a feature request but given we don't have plans to look at this in the short or medium term, I'm going to close this issue. If we decide to revisit this, we'll re-open this issue.

OK, thanks @derekbekoe, I can get around the resend limitation by using ngrok for development or testing that needs resend.
The non-working issue is different, and I assume it is likely to be resolved in due course but at present it's a showstopper - do you want me to submit it as a separate issue?

The non-working issue is different, and I assume it is likely to be resolved in due course but at present it's a showstopper - do you want me to submit it as a separate issue?

The "Edit and Resend" UI option you see is not something we specifically integrated into dev tunnels (instead, it came with browser devtools UI and we didn't remove that menu option). As such, we wouldn't expect it to work. So, it'd be a feature request for us to support "Edit and Resend" and unfortunately we don't have plans to add support for it at this time.
Hope that helps clarify.

Thanks @derekbekoe, that's what I understood from your first comment. I realize that resend is not one of the requirements for the dev tunnels feature so I'll need to use something else to satisfy that requirement when it arises, Please just take this as a vote for a new feature.


ARC-103 change ACTIVE to IN PROGRES

We would like to change the term 'ACTIVE' to 'IN PROGRESS' because:

the term 'active' can be misleading, particularly for a first time user. 'Active' may indicate to a user that their installation is now active and ready to use when, in fact, they may have a long sync ahead of them.
to be consistent with naming conventions at Atlassian.

I have tested and confirmed adding an org shows 'IN PROGRESS' and still completes. Additionally, I noted the updated status was present in the db for an org that had a sync in progress.

Updated so now we only make the change on the frontend.

DB remains the same.


[Question] Creating a own .csv

I want to use my own Dataset of .wav's to train spleeter and as far as i have noticed i need a .csv where the Dataset is described.
My Question: How do i create a .csv like given? https://github.com/deezer/spleeter/blob/master/configs/musdb_train.csv

As mentioned in this wiki section, the csv must contains path to the audio files of the instrument you want to separate and to the mixture of all instruments (that can be the sum of all the others) and the duration of the file (that has to be the same for all these files). For instance, if you want to separate instrumental part from vocal, you'll need to have a csv with a instrumental_path column, a "vocal_path" column, a "mix_path" column and a "duration" column.