Must submit a better answer?

Hello,
i have a question, does each submitted method have to be less gas cost than the previous ansuwer?

well. i know the answers,,
ahaha, just missed a few lines of code, hhh


PendingStateManager: More efficient validation of local op roundtrip

Description
Related to AB#5317, but doesn't fix it.
We have 2 ways to check data coherency for local ops as they round-trip through the ordering service:

compare the contents of each incoming message v. pending message
compare the batch info for the incoming batch and the next pending message

This PR adds an additional check to (2) to make it more robust - it checks the batch length in addition to batch start CSN.  This makes it pretty equivalent to the old validation (year+ ago) where we only looked at each message's CSN (Before GroupedBatching changed CSN semantics).
Ideally we can eventually remove check (1) because it's expensive (does a JSON stringify on every message).

@tylerbutler -- FYI I'm intending to merge this as soon as CI passes, wanting to make it in 2.2


Fixtests

Overview
Compiling the datapackage-java JAR under Windows gives me some of the same exceptions mentioned in my pull request for tableschema-java.
This is due to the incorrect construction of the path in question; a similar question and explanation can be found here: https://bugs.openjdk.java.net/browse/JDK-8197918?focusedCommentId=14156698&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14156698
The underlying problem doesn't manifest on Unix-like systems, but due to Windows peculiar file paths, it triggers an exception.
After applying the proposed solution, the tests all pass in my environment.

Please preserve this line to notify @georgeslabreche (maintainer of this repository)

Coverage decreased (-0.3%) to 79.393% when pulling 515c3fa7a04626742e437fe96fd698915d21c294 on iSnow:fixtests into 03aced05b1ad147753f080e86676eeb53d419da3 on frictionlessdata:master.


icinga2: 2.12.3 -> 2.12.4

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/icinga/icinga2/releases.
meta.description for icinga2 is: "Open source monitoring system"
meta.homepage for icinga2 is: "https://www.icinga.com"
meta.changelog for icinga2 is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4/bin/check_nscp_api passed the binary check.

/nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4/bin/icinga2 passed the binary check.

2 of 2 passed binary check by having a zero exit code.

2 of 2 passed binary check by having the new version present in output.

found 2.12.4 with grep in /nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4

found 2.12.4 in filename of file in /nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4

directory tree listing: https://gist.github.com/88727113b14974ff7e42d05683b0cc02

du listing: https://gist.github.com/f9c71746e31fa5301aea265c91258c7a

Rebuild report (if merged into master) (click to expand)

6 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
2 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
icinga2
icinga2-agent

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A icinga2 https://github.com/r-ryantm/nixpkgs/archive/5be9a1d161bd5ef37da9dd71e3545731247c5c35.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4
ls -la /nix/store/k3f5jgr4s6mizwjh4p3rd3whckr076lg-icinga2-2.12.4/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

icinga2
icinga2-agent

Maintainer pings
cc @dasJ for testing.

Backporting because it's just a maintenance release with fixes for crashes and other bugs


Produce metadata along with RelNode

add group id metadata so DAG can be identified
remove physical collector from optd-core

I though physical collector is in DF bridge?

I thought physical collector is in DF bridge?

Yes, but right now it relies on optd-core to take in an on_produce callback to inject the collector. As a result of producing the metadata in #83, we will remove the dependency.


feat(divider): create divider component

A thin horizontal line that separates content areas built on top of <hr>.
Also apply the Hr component in VercelInviteUser preview

This is awesome, thanks @pepeladeira!


fix(otlp-exporter-base): decrease default concurrency limit

Which problem is this PR solving?
See #4204, reduces the export limit to 30 from infinity. With default settings in metrics with a periodic exporting metric reader this would equate to 30 minutes of collector unavailability until metrics are dropped; in traces and logs, this would equate to 2.5min of unvailability (or shorter if reaching the BatchSpanProcessor limit).
The limit of 30 may be a bit low; I'm happy to change the limit based on suggestions on this PR :slightly_smiling_face:
Fixes #4204
Type of change

[x] Bug fix (non-breaking change which fixes an issue)

@pichlermarc Basis your calculations, 30 seems a very reasonable default value to me. In my opinion, holding large amount of data in RAM is not advisable for the agent as it can cause memory spikes and that can have adverse impact on application performance/stability.
For a service getting 10k requests/min, 25k traces will get generated in 2.5 min. Assuming each trace takes a modest 5kb memory (10 spans per trace * 500 bytes per span), the queue can consume upto 125mb.


high is out of bounds for int64 in cluster permutation test

I've been getting an error in cluster permutation tests when applied on single-trial data, when the number of trials exceeds ~60
FWICT this is due to max_perms being too large in cluster_level.py :
https://github.com/mne-tools/mne-python/blob/cd24596e14383dc978d754a51f8fe9fabbd76b21/mne/stats/cluster_level.py#L761-L768
I can reproduce the error when running this MNE tutorial, when rejecting no epochs (Reject=None). In that case epochs_power has shape (72, 16, 49), which isn't extreme, but I'm still getting the following error:
Log output
Traceback (most recent call last):
  File "/Users/nicolas/Toolboxes/mne-python-nbara/mne/stats/cluster_level.py", line 1163, in permutation_cluster_1samp_test
    buffer_size=buffer_size)
  File "<string>", line 2, in _permutation_cluster_test
  File "/Users/nicolas/Toolboxes/mne-python-nbara/mne/utils.py", line 726, in verbose
    return function(*args, **kwargs)
  File "/Users/nicolas/Toolboxes/mne-python-nbara/mne/stats/cluster_level.py", line 768, in _permutation_cluster_test
    orders = rng.choice(max_perms, n_permutations - 1)
  File "mtrand.pyx", line 1147, in mtrand.RandomState.choice (numpy/random/mtrand/mtrand.c:18069)
  File "mtrand.pyx", line 975, in mtrand.RandomState.randint (numpy/random/mtrand/mtrand.c:16083)
ValueError: high is out of bounds for int64

don't do 2**60 permutations. limit n_permutations to lets say 10000 for a
p-value threshold at 0.01

Mmh but doesn't n_permutations default to 1024 in permutation_cluster_1samp_test &co?
In the code snippet above, max_perms is only a function of the number of samples (in my case, trials)

@agramfort - the problem seems to be different, the realized premutations are chosen from all possible permutations (as an int that is later transformed to its binary representation) and it seems that if there are too many possible permutations rng.choice gives errors.

I see my "clever trick" is really a "bug" sometimes :)
We can go back to using resampling with replacement in the non-exact case. (My code changed it to be sampling without replacement.) Sound good? If so I can make a PR

you should not allow replacement in theory AFAIK

Okay I think there is another way to do it that should be safe enough


would anyone provide a start-up-demo-from-zero

as the title

as the title

Just create a project as well. The new project automatically includes a demonstration module test-party, which contains a large number of test cases and Kichen-sink, and continues to add new content.


transfer Gcode via USB

Bdurbrow
Is it possible to transfer Gcode via USB from Programs on a PC that supports GRBL?
For me, all GRBLPanel functions work ...... not only the G-code transmission over the cable does not work: (((((

That’s odd... I didn’t think I messed with that. It was working from the Arduino serial monitor... I will investigate.

bdurbrow!
 I had some errors in the NC code. Today, I once again connected the driver via USB cable. I was sending the code from the programs to the Machine:
GRBLPanel - connects without problems works OK!
Lightburn - connects without problems works OK!
LaserGRB - harder to connect to USB, but eventually connects and transmits OK!


Error "The keyword 'interface' is reserved" trying to use chat component in another project

I created a new project with:
npx create-next-app@latest

Installed this repo as a dependency:
npm install mckaywrigley/chatbot-ui-lite#main

In the file pages/index.tsx I added this line on top:
import { Chat } from "ai-chatbot-starter/components/Chat/Chat";

Then I inserted the component in the HTML of the home function:
        <Chat
        />

When I run the code (npm run dev) I get this error:
./node_modules/ai-chatbot-starter/components/Chat/Chat.tsx
Module parse failed: The keyword 'interface' is reserved (8:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| import { ResetChat } from "./ResetChat";
|
> interface Props {
|   messages: Message[];
|   loading: boolean;

Sometimes I get a similar error in ResetChat.tsx.
I suppose there is a configuration issue in my project or in this one, but I the few solutions I found on Google didn't work for me.
Also, I'm not sure this repo is ready to be imported into other projects.

Solved.
I had to add a line to my next.config.js:
/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  transpilePackages: ['ai-chatbot-starter'], // This line
}

module.exports = nextConfig


Remove unused Optimize Mesh import option from OBJ mesh import

This option was enabled by default, but it did nothing regardless of whether it was enabled or not.

See https://github.com/godotengine/godot/pull/90722.

Would it not make sense to fix the option by calling generate_lods? I assume that was the intent behind this checkbox.

I expect that the fix would be to call generate_lods.

I expect that the fix would be to call generate_lods.

That would be a different option which would be called generate_lods :slightly_smiling_face:

Thanks!


Proposed a link for Git branches in the table

Resolves #1032

@ClydeDz Thank you for for your contribution. I am merging this PR and it should be live soon.


Reading Docker timeout value from environment variable

Fixing the following issue by adding DOCKER_REGISTRY_TIMEOUT environment variable

Hi @mael-le-gal I mistakenly cancelled your CI build and restarted it again! Sorry!
I noted that one job of all the jobs had failed relating to not having run the autoformatter black.

My understanding is that this PR makes it so that repo2docker configures docker to explicitly use its own default value, unless a environment variable is set, called DOCKER_REGISTRY_TIMEOUT.
Considerations:

I wonder, would docker adjust itself if we simply provided an env var called DEFAULT_TIMEOUT_SECONDS ?
I wonder, if we add a new env that has meaning to repo2docker, should we perhaps prefix it with REPO2DOCKER or R2D? I've come to appreciate such things a lot as it can be quite a hurdle to figure out why and who considers what env vars if you have something nesting something that potentially even nests something else etc.

I've learned that no, Docker does not care about an environment variable named DOCKER_TIMEOUT_SECONDS.
Hardcoded constant...
https://github.com/docker/docker-py/blob/20b5491363a9124dbf53b00fb6562f8f257a6228/docker/constants.py#L1-L6
... used as fallback if no timeout argument was provided.
https://github.com/docker/docker-py/blob/20b5491363a9124dbf53b00fb6562f8f257a6228/docker/client.py#L82

I noted that one job of all the jobs had failed relating to not having run the autoformatter black.

Indeed, it should be fixed now

I wonder, would docker adjust itself if we simply provided an env var called DEFAULT_TIMEOUT_SECONDS ?

No there is only a hardcoded default timeout but it is never override by environment variable

Regarding my second consideration, I'm leaning towards naming the variable REPO2DOCKER_DOCKER_TIMEOUT or something making it obviously ours.

Fixed, I changed to REPO2DOCKER_DOCKER_TIMEOUT

Should we set the timeout explicitly, by using the docker default value imported from the docker package at? It will entwine us with the docker package slightly. If we need that, because we need to know the timeout someplace for some reason, then I think we should. If we don't need to know the implicitly set timeout, then I think it could be beneficial to avoid explicitly passing a timeout argument when we don't detect a REPO2DOCKER_DOCKER_TIMEOUT variable.

Two general comments:

we use https://github.com/ipython/traitlets for the configuration of repo2docker, lets use it also for this timeout. I need to dig a bit to find an example/docs of how traitlets and env variables work together. We should use that machinery as much as we can
not sure how I feel about fetching the default value and using it. I'd have used None as a value for the "default" value in r2d which signals use what ever the default in docker-py is. However docker-py sets the default to a particular value (not None) so I think in this case getting the value and passing it in is the thing to do.

I'd suggest that we close this PR. It hasn't been a problem in a long time and we can open a new PR using traitlets to implement this if we need to.
(We are trying to reduce the number of outstanding/stale PRs)


Added support for sonar branch parameter

Added support for Sonarqube maven plugin's sonar.branch parameter because current (1.2.2) implementation supports only default (no) branch which results in incorrect request for Sonarqube API polling when -Dsonar.branch parameter is used. This leads to polling timeout error.

This change is

Merged this in and pushed version 1.2.3 up to maven central.  It should be available within a day or two.


EDU-1579: Port to Temporal

Oh dear. Please ignore. Went to wrong repo by mistake due to forking. Thanks!


surival mode god spawning crashes

^survival mode i can't seem to fix the god spawn issues and the random crashing. use at your own risk.

I think i solved that bug. Would be cool if you check it on GD.

ok closing issue for now then
thanks kreischer


404 error Envelopes API

When calling the CreateEnvelope method, a 404 error is returned.  The issue seems to be that path is set to "/v2.1/accounts/{accountId}/envelopes" but the {accountId} is never replaced with the actual value. Instead, you end up with this https://demo.docusign.net/v2.1/accounts/{accountId}/envelopes.

can you share the C# code you're running?

CreateEnvelope() is one of the most popular methods, so if it has a bug - we should know ASAP

I'm using the SendEnvelopeForEmbeddedSigning method of the EmbeddedSigningCeremony.cs file I found in the repo.  I've attached it here.
EmbeddedSigningCeremony.zip

Any guess as to what might be going on?  Any help is appreciated.

Hi @rudyp321, I tested the code you provided and successfully used the CreateEnvelope method. Could you please share the basePath you are using? I suspect there may be an issue with it. Additionally, could you attach a screenshot of the error?
Here are some helpful links/steps that can help you in resolving the issue:

Please use Docusign eSign API Explorer and try sending similar inputs to see if you are still facing the same issue.
Please use the Quickstart wizard which lets you download code samples which are preconfigured with your account data.

Hello @rudyp321,
I wanted to follow up on my previous message. Did you have a chance to review it? Were the links helpful, or are you still experiencing the same issue?
If the problem persists, could you please provide the following information:

The basePath you are using, as I suspect there may be an issue with it.
Could you attach a screenshot of the error?
Which version of the package are you trying out?
Which .Net version are you using?
Do you have a Docker setup for this application?

Thank You.

Hello, Mudit,
Thank you very much for your help.  The issue was the basePath.  I was
missing the /api.  Added that and it worked.
On Tue, May 7, 2024 at 2:28 AM Mudit Garg @.***> wrote:

Hello @rudyp321 https://github.com/rudyp321,
I wanted to follow up on my previous message. Did you have a chance to
review it? Were the links helpful, or are you still experiencing the same
issue?
If the problem persists, could you please provide the following
information:

The basePath you are using, as I suspect there may be an issue with
it.
Could you attach a screenshot of the error?
Which version of the package are you trying out?
Which .Net version are you using?
Do you have a Docker setup for this application?

Thank You.
—
Reply to this email directly, view it on GitHub
https://github.com/docusign/docusign-esign-csharp-client/issues/442#issuecomment-2097542404,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AATNZAG3Q7XOVNYPW5ALRELZBBX7NAVCNFSM6AAAAABGQ7YIQSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJXGU2DENBQGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Rudy Padrón


2 Warnings during linux build with 5.0.x swift

I am not sure if this is a linux, swift, or core issue.
Core/Process+Execute.swift:163:17: warning: 'launchPath' is deprecated: renamed to 'executableURL'
Core/Process+Execute.swift:167:17: warning: 'launch()' is deprecated: renamed to 'run'
Do not get these warnings in Xcode.
Version 3.9.0 of Core

Hmm....maybe due to 5.0 having a better compatibility layer with Swift 4.1 mode on macOS? We should be able to silence the warnings with a combination of #if os and #if compiler, but this seems like it should be fixed in Swift / core libs foundation itself.

Should I put in a ticket with swift?

Yeah that would be great. :)

I created an issue at bugs.swift.org.


CORS policy error trying to open build report during userbuild

I run DBB user build and then, click on show build report when it completes , it just opens an empty browser with just a header "build report"
It looks like the report it is trying to open is from a local cached location in the workspace, so its ending up with a CORS policy error trying to open the BuildReport.json in the local folder.
Access to XMLHttpRequest at 'file:///C:/Users/xxxx/IBM/rationalsdp/workspace/RemoteSystemsTempFiles/xxxx/u/xxxx/DBB-Sandbox/work/BuildReport.json' from origin 'null' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: http, data, chrome-extension, edge, https, chrome-untrusted.
Is there any way to resolve this ?

There is currently no solution to fix it. Please stay tuned for the upcoming IDZ release.


New tutorial for: configurable elasticsearchdocumentstore

Describe the tutorial you would like to see here
All the current tutorials assume the document store will not change over time.
For example, assume we have a single elastic database engine and need to build a single application where each user should ask a question and retrieve an answer. However, each user should have access to a different index "database" on the elastic search.
The current implementation assumes a fixed document store information during application run-time, like :
https://github.com/deepset-ai/haystack-tutorials/blob/main/tutorials/03_Scalable_QA_System.ipynb
It would be beneficial to have a tutorial where we can change the document store username, password, and index during application runtime. Otherwise, we must build a duplicate application for each user and each index.
Additional context
None
[X] I've checked the existing tutorials

Hi @agemagician 👋  thank you for opening the issue. You have a nice problem here. Unfortunately, you cannot change username and password during the runtime. You can change the index but it will require re-connecting every time you change it.
I would recommend creating different DocumentStores rather than changing the credentials altogether. Another idea could be using metadata filtering and let Elastic handle the user management.


chore: add benchmark

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (b78e156) to head (4b82ebc).

Additional details and impacted files
@@            Coverage Diff            @@
##            master        #5   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines          161       161
  Branches        26        26
=========================================
  Hits           161       161

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Support Python 3.7

This issue is a meta issue to track related issues and PRs.

[x] https://github.com/chainer/chainer/issues/5097
[x]  #5223
[x]  CuPy Python 3.7 support: https://github.com/cupy/cupy/issues/1464

After all fixes has been done, we should fix chainer-test against py3.7 and update docs.

Related issue: conda will try and install the Python 3.7 version of Chainer along with Python 3.7 itself even if you don't have it already. You want to avoid this until a new release like so:
conda install chainer=4.2.0=py36_0

matplotlib/matplotlib#11733 hasn't been backported to matplotlib 2.x.
It needs a fix to the test https://github.com/chainer/chainer/compare/master...toslunar:test-py37-matplotlib

[x] chainer/chainer-test#442
[x] docs: #5573
[ ] conda

@toslunar  I'd like to close this issue as we now support Python 3.7, and Conda support is not task of ourself (we're asking Anaconda team to build CuPy package). How do you think?

And CuPy can be installed manually.  It seems OK to close the issue, "Support Python 3.7".


customize sidebar css

When adding a new article/page, you need to add an escaped route to the css
.menu__link[href*=introduction/how-to-use-this-handbook]:after, .menu__link[href*=introduction/about-offerzen]:after

Thanks so much @felzer121 ! 🙏


Game Creator - Upload Game - Github Auth

Background
Currently, any GameBox user can access /upload to upload a game. Only authenticated Github users should be allowed to upload games so that we can eventually associate their Github user to the games they have created and have the ability to ban users from uploading games. The purpose of this ticket is to implement Github Auth for game creators.
Scenario: Implement Github Auth
Given I am a Game Creator

[x] When I want to upload a game
[x] Then I must authenticate with Github Auth

completed in #24, moving to QA

works as expected, moving to UAT


Is it possible to go "back" to the "main" file?

Hi there, the plugin is very useful to navigate to the Python test file (in my use case).
I have created the following mapping for my Python files:
{
  pattern = '(.*)/src/(.*)/(.*).py',
  target = '%1/test/%2/test_%3.py',
  context = 'test',
}

I was wondering if there's a way to go "back" to my main file (basically to alternate between them).
I know I can use [Neo]vim's built-in C-^ mapping for this, but it could really be useful if I can use this plugin to do that.

@farzadmf Yes, that would be done be adding a second pattern to go back.
So in your case that would roughly be..
{
  pattern = '(.*)/test/(.*)/test_(.*).py',
  target = '%1/src/%2//%3.py',
  context = 'src',
}

Note: that's not tested but I think you get the idea.. You can also have a look at the built-in-mappings. There is also a lot of mapping back.

@farzadmf Yes it's the default lua.. Best documentation about it as far as i know is here https://www.lua.org/manual/5.1/manual.html and there you should look at chapter "5.4.1 – Patterns"

Thank you very much @rgroli , I appreciate it


Seam Connect Fake Integration + Seed

Introduces Seam Connect Fake jest tests and storybook integration
The jest testing still needs to be thought out a bit more, but the storybook runs against the fake smoothly which is a good place to begin.

@seveibar Looks like you might have a global .npmrc that is overriding the @seamapi namespace to resolve to GitHub Packages instead of npm, which would explain why the entry in the lockfile points there now. If you remove that override and re-install the package it should fix the build.

@razor-x chromatic needs some env vars and in general a completely different, local configuration- this should not be holding us up at this point. Let's just disable this.

chromatic needs some env vars and in general a completely different, local configuration- this should not be holding us up at this point. Let's just disable this.

Just taking a quick look, it giving this error. Did something change on seamapi SDK? It is trying to PUT now.

Just taking a quick look, it giving this error. Did something change on seamapi SDK? It is trying to PUT now.

I was wrong, I thought it had the wrong client session, but this is because the fake has PUT support (in the alpha seamapi sdk https://github.com/seamapi/javascript/pull/193) but the production API doesn't support it yet. That's actually a lot better than Chromatic having a configuration issue- I had incorrectly assumed it was the hard-coded PK i changed- nice catch

All good! I patched the minor issues on my new PR here: https://github.com/seamapi/react/pull/83
For the PUT issue, let's update the internal endpoint to match :)


added business search

Signed-off-by: Kial Jinnah<EMAIL_ADDRESS>

/gcbrun

/gcbrun

Temporary Url for review: https://yfthig-dev--pr-79-cou8gp5f.web.app

/gcbrun

Temporary Url for review: https://yfthig-dev--pr-79-cou8gp5f.web.app

Temporary Url for review: https://yfthig-dev--pr-79-cou8gp5f.web.app

/gcbrun

Temporary Url for review: https://yfthig-dev--pr-79-cou8gp5f.web.app

@tlebedovich ld is working on the temp link now so its ready to be looked at again!

@kialj876 - For the business search BCROS tile, i noticed the tool tip dotted underlines styling look a little weird in this latest version -

@janisrogers - did you need to make any content updates in this ticket based on Kaine's feedback from today or will you make a new ticket?

Temporary Url for review: https://yfthig-dev--pr-79-cou8gp5f.web.app


flutter_rust_bridge_codegen create says program not found

Describe the bug

I run flutter_rust_bridge_codegen create my_app and i got this error
it says that flutter create my_app my have sone problem, but i run flutter create and it'ok on my powershell, and i don't know why i got this error
Steps to reproduce
Hint: A simple way to reproduce is to clone and modify the https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_example/dart_minimal example package according to your needs.

...
...
...

Logs
PS C:\Users\wang2> flutter_rust_bridge_codegen create my_app
Error: The target folder "C:\\Users\\wang2\\my_app" already exists. Please use the `integrate` command in this case
PS C:\Users\wang2> rm C:\\Users\\wang2\\my_app

Confirm
The item at C:\Users\wang2\my_app has children and the Recurse parameter was not specified. If you continue, all
children will be removed with the item. Are you sure you want to continue?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): Y
PS C:\Users\wang2> flutter_rust_bridge_codegen create my_app
[2024-01-13T02:58:16.650Z INFO C:\Users\wang2\.cargo\registry\src\index.crates.io-6f17d22bba15001f\flutter_rust_bridge_codegen-2.0.0-dev.18\src\library\commands\flutter.rs:13] Execute `flutter create ["my_app"]`
Error: "powershell" "-noprofile -command & "flutter" "create" "my_app"" failed

Caused by:
    program not found
PS C:\Users\wang2>

Expected behavior
No response
Generated binding code
No response
OS
Windows11
Version of flutter_rust_bridge_codegen
No response
Flutter info
No response
Version of clang++
No response
Additional context
No response

Hmm, how do you install your flutter command? Or, does directly run powershell -noprofile -command & flutter create my_app finds the command or not

first of all I didn't expect you this quick 😄，and flutter command is working and

It looks okay for me

first of all I didn't expect you this quick 😄

I am PRing so is looking at github 😄

and flutter command is working and

Hmm, could you please provide more info about where your flutter is installed, and how is it installed (following official guide? fvm? etc)

I do install fvm but i didn't use it for now, i just download flutter installer and unzip it to my D disk and did some setup on my env, i think it's as offcial guide, did i miss somthing?

I do not use windows often, so your configuration looks reasonable for me. For this command, I tested both in a windows10 virtual machine, and on the CI, both works well.
 From my old memory it seems that env variables may not fully work before a reboot
If it still fails, one way may be create a simple reproducible sample like: Create a empty rust project, call rust's https://doc.rust-lang.org/std/process/struct.Command.html directly, and observe the bug. Then it may be easier to debug.

I did as the rust document said, and i got the same error, but i change "powershell" to "cmd" and this code runs fine, i don't know why is that

Interesting, seems your powershell does not recognize flutter...

I tried to add support to use cmd in addition to powershell in #1662, and realize it seems that powershell is the suggested and preferred way (correct me if I am wrong!), and there are some pitfalls e.g. https://stackoverflow.com/questions/44757893/cmd-c-doesnt-work-in-rust-when-command-includes-spaces.

Since this issue is unrelated to flutter_rust_bridge core, but instead "we cannot use rust's Command to execute Flutter", I guess it may be great to ask a general question for this in e.g. stackoverflow, rust community, etc.
Btw, does replacing flutter word with flutter.bat works for you?

Na , flutter.bat isn't work, i'll close it

More brainstorms (in addition to the suggestion of asking this general Rust question) here:

Maybe try WSL?
What about Command::new("powershell").arg("-noprofile").arg("-command").arg("& flutter create my_app") (i.e. the last one is a long arg)
What about Command::new("powershell").arg("-command").arg("& flutter create my_app"), i.e. remove "noprofile", since I suspect profile has something related.

okay I’ll try it. Thanks.
发自我的iPhone
------------------ Original ------------------
From: fzyzcjy @.>
Date: Sun,Jan 14,2024 11:56 AM
To: fzyzcjy/flutter_rust_bridge @.>
Cc: watson wang @.>, State change @.>
Subject: Re: [fzyzcjy/flutter_rust_bridge] flutter_rust_bridge_codegencreate says program not found (Issue #1654)
More brainstorms (in addition to the suggestion of asking this general Rust question) here:
Maybe try WSL?
What about Command::new("powershell").arg("-noprofile").arg("-command").arg("& flutter create my_app") (i.e. the last one is a long arg)
What about Command::new("powershell").arg("-command").arg("& flutter create my_app"), i.e. remove "noprofile", since I suspect profile has something related.
—
Reply to this email directly, view it on GitHub, or unsubscribe.


check_order_callback and after payment redirect gives signature mismatch

When mycelium gear makes the call back and you have an 'after payment redirect url' set, it will give a signature mismatch as mycelium urlencodes the enter you put in as the after payment redirect.

I tried decoding/encoding and signature still mismatches. I think it has something to do with the recent bug in Mycelium Gear.

were you encoding the whole $request_uri or just the part of the string after the after_payment_redirect_to?
I added this after the $request_uri in the check-order_calledback function and it seems to work just fine:
if (preg_match('/after_payment_redirect_to\=(?P<url>.*?)\&/i',$request_uri, $matches)){ 			$rewrite = urlencode($matches['url']); 			$request_uri = str_replace($matches['url'], $rewrite, $request_uri); 		}
It does require you to generate a new $gateway_secret if you add/remove an after payment url in mycelium gear

Your regex gives me the following error:
Warning: preg_match(): Compilation failed: unrecognized character after (?P at offset 29 in
Anyway, I solved it by directly encoding the $_GET['after_payment_redirect_to'] before I process it in $request_uri 35cb518
Thank you!

lol forgot to put name identifier in with the regex match when I typed it in here..


Fix for #73344

Fixes #73344

Agree with the change, but you will need to update your commit message to match the contribution guidelines, or just copy
drivers: rtc: rtc_mc146818: Patch incorrect field comparisons

Minutes and hour values are incorrectly compared with MAX_SEC.

Compare minutes and hour to MAX_MIN and MAX_HOUR
respectively.

[DanielKampert<EMAIL_ADDRESS>
Signed-off-by: Daniel Kampert<EMAIL_ADDRESS>
Do validate the signed-of-by field though :)

nearly there, the header of the commit message needs to be "drivers: rtc: rtc_mc146818: Patch incorrect field comparisons", right now it's "Update rtc_mc14618.c"

@bjarki-trackunit I hope everything is right now 👍


Update requirements.txt to include sphinx-autobuild

:shipit:  this looks better. Somehow I messed up the other PR.


Cramped PR title in pull request review mode

Issue Type: Bug
Possibly #2459

Check out a PR for review

Bug
The title on the pull request page ends up being cramped over on the left side of the editor, making it difficult to read

Extension version: 2021.2.34443
VS Code version: Code - Insiders 1.54.0-insider (c47da720f351455681978a48edc4d9a0d095ea0e, 2021-02-24T12:22:35.736Z)
OS version: Darwin x64 20.3.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
3, 3, 3

Memory (System)
32.00GB (0.43GB free)

Process Argv
--crash-reporter-id cc7f0c9b-aa2b-4a45-8baa-81ffda32679e

Screen Reader
no

VM
0%

This has since been fixed.


zfs-import-* attempts to load all keys, not just its own

Describe the bug
When multiple ZFS filesystems are present and configured to import, each individual zfs-import-pool attempts to load the keys for ALL pool, instead of just its own.
A clear and concise description of what the bug is.
Steps To Reproduce
Steps to reproduce the behavior:

Have encrypted zfs filesystems in your NixOS configuration.
Rebuild & Reboot

Expected behavior
All zfs-import-* targets complete successfully
Additional context
collection1 zfs pool failing on loading data2/privates key, completely unrelated
$ systemctl status zfs-import-collection1.service
× zfs-import-collection1.service - Import ZFS pool "collection1"
     Loaded: loaded (/etc/systemd/system/zfs-import-collection1.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Sun 2022-11-27 00:20:24 AEST; 1h 2min ago
    Process: 2208 ExecStart=/nix/store/9v7v8sbhvv667plrdmaaiqnz9bjyg295-unit-script-zfs-import-collection1-start/bin/zfs-import-collection1-start (code=exited, status=255/EXCEPTION)
   Main PID: 2208 (code=exited, status=255/EXCEPTION)
         IP: 0B in, 0B out
        CPU: 112ms

Nov 27 00:20:19 CAPRICA systemd[1]: Starting Import ZFS pool "collection1"...
Nov 27 00:20:24 CAPRICA zfs-import-collection1-start[2208]: importing ZFS pool "collection1"...
Nov 27 00:20:24 CAPRICA zfs-import-collection1-start[4139]: Key load error: Key already loaded for 'data2/private'.
Nov 27 00:20:24 CAPRICA systemd[1]: zfs-import-collection1.service: Main process exited, code=exited, status=255/EXCEPTION
Nov 27 00:20:24 CAPRICA systemd[1]: zfs-import-collection1.service: Failed with result 'exit-code'.
Nov 27 00:20:25 CAPRICA systemd[1]: Failed to start Import ZFS pool "collection1".

Cause
The below snippet from the services's ExecStart script. It's zfs list -Ho name,keylocation,keystatus  | while IFS=$'\t' read'ing, and just looping over all pools/datasets. Instead, it should just specify the name directly in the zfs list command.
(Generated in nixos/modules/tasks/filesystems/zfs.nix)
if poolImported "collection1"; then
  /nix/store/7l4v5aq85kc0sv8kzbr17x0ca4lrff0v-zfs-user-2.1.6/sbin/zfs list -Ho name,keylocation,keystatus  | while IFS=$'\t' read ds kl ks; do
  {
  if [[ "$ks" != unavailable ]]; then
    continue
  fi
  case "$kl" in
    none )
      ;;
    prompt )
      tries=3
      success=false
      while [[ $success != true ]] && [[ $tries -gt 0 ]]; do
        /nix/store/q11j05nqg829dkvgc8851mscbzi2iq8n-systemd-251.7/bin/systemd-ask-password "Enter key for $ds:" | /nix/store/7l4v5aq85kc0sv8kzbr17x0ca4lrff0v-zfs-user-2.1.6/sbin/zfs load-key "$ds" \
          && success=true \
          || tries=$((tries - 1))
      done
      [[ $success = true ]]
      ;;
    * )
      /nix/store/7l4v5aq85kc0sv8kzbr17x0ca4lrff0v-zfs-user-2.1.6/sbin/zfs load-key "$ds"
      ;;
  esac
  } < /dev/null # To protect while read ds kl in case anything reads stdin
done

  echo "Successfully imported collection1"
else
  exit 1
fi

Notify maintainers
@ElvishJerricco
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

system: "x86_64-linux"
host os: Linux 5.15.79, NixOS, 22.11 (Raccoon), 22.11.20221121.20fc948
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.11.0
nixpkgs: /etc/nixpkgs/channels/nixpkgs

@vs49688 Can you post the relevant Nix config you used? Do you have requestEncryptionCredentials set to anything?

Ok, I think this is fundamentally the same issue as #202164. Basically, when there are multiple pools and boot.zfs.requestEncryptionCredentials is a list, we accidentally run zfs list with no dataset argument in the import service for pools that have no datasets in the list. So those services end up iterating over all datasets on all pools instead of none.


Validate marker access by type

Description
Adds missing validation of access permissions against marker type.  Includes a migration function to remove any existing invalid permission entries from marker access grants
closes: #428

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[x] Targeted PR against correct branch (see CONTRIBUTING.md)
[x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[x] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[x] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

Good point. That should state that it's for restricted coins only. Worth a scan for any other places that mention restricted markers or transfer right as well...


Modify CPU CA to use the Frequency Governor

estimate 1
Parent [feature/story]: #2363

The implementation for the CPU CA feature was modified because it did not use the frequency governor to write frequency controls.
Done criteria:

[x] Integration tests that cover the feature run and pass
[x] Dependencies mocked in unit tests are still valid
[x] Documentation changes are not required since this change does
not modify interfaces or features.

Duplicate of #2645


Sort order for Custom Actions flyout is off

Describe the bug
I'm not sure how the custom actions flyout is sorting its entries but it's not what I would have expected:

Ah... It's sorting by the underlying action name versus title. I think title would be a better sort.

Those actions are not sorted yet. So the order seems random ( depends on Object.keys() order).
We can consider

Sort by displayed title
Sort by $kind
Sort by native order

If there are no objections to sorting by $kind while we look for a more scalable UX control, let's go ahead with that.


Hidden H2s still appear in Menu component

Hiding H2s via comments will prevent them from being shown in main content area, however they still appear in the menu section. This can reproduced by taking the boilerplate index.html (described in the "Quick Start" instructions), and creating a README.md with the following content.
# Title

lorem ipsum ...

<!-- hide-on-docup-start -->

## Table of Contents

- [Installation](#installation)
- [Getting Started](#getting-started)

<!-- hide-on-docup-stop -->

## Installation

`npm i some-package`

## Getting Started

blah blah blah

The generated website will show three sections in the menu: "Table of Contents", "Installation", and "Getting Started". My expectation is that it would only show the latter two.

thanks for reporting, will look into it asap


RFC: Extend the behavior of load/store ops to allow addressing all elements of a word

Background
I'm currently finishing up portions of the code generator in the compiler for Miden IR, and something came up as I was working through how to translate memory operations in the IR (which is byte-addressable) to Miden Assembly (which as we all know is word-addressable).
In particular, I noticed that the mem_load and mem_store instructions, which load/store an individual field element, rather than a word, only work for the first element of the word at the given address. Up until now, I had been operating under the assumption that those instructions could load/store any element of the word, but had missed that the addresses are word-aligned, thus making it (at least at first glance) impossible to represent a load for, say, the 3rd element of a word. In short, this significantly reduces the usefulness and generality of these instructions.
Proposal
I would like to suggest a change to these instructions that not only addresses this gap in functionality, but is backwards-compatible to boot.
The following are true of the current semantics:

The address operand for these instructions is a field element
The maximum address allowed is 2^32-1, which is in the u32 range. The ops in question trap if the address is out of this range.
If we were to u32.split the address operand, the high bits would be all zero, and the low bits would hold the address

I'm proposing the following changes:

The number and type of the operands remains the same
The address operand is treated as a pair of u32 values, and is split using the same rules as u32.split.
The previously unused (but zeroed) high bits, will now be used to represent an offset from the base of the word at the address represented in the low bits. The instructions will trap if this offset is anywhere outside of the range 0-3.

Without this change, to perform the equivalent of mem_load on the 3rd element of a word, you'd need to issue a sequence of instructions like padw, push.ADDR, mem_loadw, drop, drop, swap, drop. With this change, you would use push.ADDR, add.0x100000003, mem_load. That's, by my count (if we assume that mem_load goes from 1 cycle to 2 cycles to account for the u32 split), an improvement of 6 cycles (10 vs 4) for a basic operation that is going to occur very frequently in programs with any local or global memory allocations.
The syntax in MASM could be extended for the version which accepts an immediate, such that the current syntax (e.g. mem_load.0x0) is equivalent to specifying a constant offset of 0. This would be extended to accept a constant offset as well (e.g. mem_load.0x0.3).
Potential Issue(s)
The primary issue I can think of, if I'm trying to find ways to break this, is that distinguishing between an out-of-bounds address and a valid address + offset pair, becomes impossible. The case where an out-of-bounds address appears to be an in-bounds address with an invalid offset, isn't really important to distinguish, as both will trap as invalid. However, it is possible, albeit unlikely, for an out-of-bounds address to appear as a valid in-bounds address + offset pair, in which case a load or store that would have previously trapped, will now read/write some arbitrary location in memory.
We could defend against this by bit-packing a 24-bit CRC code in the high-bits after the offset, which must be checked when the offset is non-zero, but whether it is worth imposing such a cost on every load/store really depends on how important it is that we catch such cases. Personally, I think it would be best to specify that the behavior is undefined in such cases, just like there is little that protects you on a more typical system from writing past the end of an array - it might trap, or it might not.
Alternatives
The proposed changes are essential in my opinion, because without them, it is exceedingly awkward to treat memory as word-addressable. The primary alternative is to instead treat memory as felt-addressable. There are a couple of problems with this:

The amount of memory allocated for a program which discards 75% of each word allocated will in the worst case be 3x larger than an equivalent program that can make full use of each word.
It becomes much more expensive to load values larger than a field element, requiring 2x the cycles just to perform the loads, not counting the additional ops required to compute each element's address.
You lose the symmetry of being able to load, operate on, and then store word-sized values using the native instructions. You only really remain unaffected with regard to the word-oriented instructions where the word is already on the operand stack, but getting words on and off the stack are much more awkward.

--
I should note that this largely an issue which affects compilers, where fewer (or even no) assumptions can be made about where a given value is located in memory, so a consistent memory model is important. That said, even with hand-written assembly, you'd find yourself in a situation where you are either forced to waste most of a word, or required to emit expensive instruction sequences to read out a word and then extract what you need from it.

Thank you for such a detailed proposal! Overall, I think this would add very nice additional capability to the VM (i.e., having fully element-addressable memory could be useful in many circumstances). A few preliminary thoughts:
Implementation within the VM
I think implementing this within the VM will be a quite tricky - but I also haven't spent a lot of time on this - so, maybe there are good solutions which I haven't considered yet.
The main issue is that we actually always have to read a full word from a memory address. It is just for cases of mem_load we put the other 3 elements into helper registers (see here). So, to select a specific element from the word, we'd need to do the following:

Put all 4 elements into helper registers.
Split the address into 2 parts - this would require putting lower limb into another helper register.
Represent the upper limb with 2 binary registers (e.g., [0, 0] means 0, [0, 1] means 1 etc.). Then we'd be able to use these registers as selectors for selecting a specific value.

Summarizing the above means that we need 7 helper registers, but currently we have only 6. I have some thoughts on how we can increase the number of helper registers to 8, but this would require a fairly significant refactoring of the decoder.
Another potential complication is that selectors would be of degree 2, and thus, the constraint for populating the top stack slot would be of degree 3. This means that MLOAD and MSTORE operation flags cannot have degree greater than 6. This should't be a problem as we currently have some empty slots for degree 5 operations. But this is still something to consider for the future.
Instruction semantics
If I understood everything correctly, the semantics you propose would be useful when addresses are static (or known at compile time). But for dynamic addresses, we'd still run into problems. For example, if we wanted to iterate over a sequence of elements in a loop, we wouldn't be able to easily increment memory address. So, it seems to me the usefulness is relatively limited.
One other idea is to treat memory as if it was element-addressable, and then impose alignment requirements on mem_loadw and mem_storew operations. So, for example:

mem_load.0 would load the first element of the first word; mem_load.1 would load the second element of the first word, mem_load.4 would load the first element of the second word etc.
mem_loadw.0 would load the first word, mem_loadw.4 would load the second word, but mem_loadw.1 would be illegal.

I think this would be much cleaner, but there are two main downsides to this approach:

We still need to solve the implementation problem I described above.
This would probably break most programs which use memory operations.

If I understood everything correctly, the semantics you propose would be useful when addresses are static (or known at compile time). But for dynamic addresses, we'd still run into problems. For example, if we wanted to iterate over a sequence of elements in a loop, we wouldn't be able to easily increment memory address. So, it seems to me the usefulness is relatively limited.

I don't think that's necessarily true, since the element offsets for all 32-bit addresses are constant, i.e. adding 0 << 32, 1 << 32, 2 << 32, or 3 << 32 to any address gives you the "paired" address for the given element of that word.
If you wanted to traverse every element of n sequential words, completely dynamically, you'd do something like the following (using pseudocode):
let base_addr: *const Word = <the address in memory of the first word, doesn't have to be constant>;
for i in 0..n {
    let word_offset: u64 = i / 4;
    let element_offset: u64 = (i % 4) << 32;
    let word_addr: u64 = base_addr as u64 + word_offset;
    let element_addr: u64 = word_addr | offset;
    let element = *(element_addr as *const Element);
    ...do stuff with element..
}

That's useful in quite a few scenarios, certainly for those in which I was intending to make use of the proposed extension.

One other idea is to treat memory as if it was element-addressable, and then impose alignment requirements on mem_loadw and mem_storew operation

I'm definitely a fan of this, as it is much cleaner right from the get go, and is similar to how on, say x86-64, certain instructions impose additional alignment requirements on their operands (e.g. SSE2 instructions), though as you mentioned it breaks backwards-compatibility. That said, I'm not sure to what extent the memory ops are used currently, so perhaps it isn't too catastrophic. I would think memory gets used pretty frequently, for precomputed constant tables and things like that, but I have no idea.

We still need to solve the implementation problem I described above.

Is that true if addresses now refer to elements not words? I would think in that case, there is no longer a need for an offset, leaving things unchanged at the instruction level. Is it because the alignment requirements that would be imposed on mem_loadw and mem_storew require additional helper registers, much in the same way my proposal would?

Having native element addressable memory support at the VM level seems the superior option to me.  There are a number of instances in the transaction kernel which would benefit for this functionality.

If you wanted to traverse every element of n sequential words, completely dynamically, you'd do something like the following (using pseudocode):

I didn't write out the actually assembly code for this, but just doing a rough estimation I think we'd incur extra 6 - 7 cycles per element read in this code. This is better than 9 - 10 cycles per read using padw, push.ADDR, mem_loadw, drop, drop, swap, drop - but it is not dramatically better. So, there is still a question in my mind whether the extra effort is worth ~30% improvement.

I'm not sure to what extent the memory ops are used currently, so perhaps it isn't too catastrophic.

Quite a few modules currently use memory operations, but almost all of them are written (and maintained) by us, and most of them a pretty small. There are two exceptions:

Transaction kernel which is fairly complex and relies heavily on memory. Memory access there are fairly well encapsulated and @frisitano believes that it shouldn't be too difficult to move to the new memory model.
Recursive verifier which is also fairly complex and also relies on memory quite a bit. @Al-Kindi-0 also believes that we could migrate it to the new memory model (though effort may be more significant than with transaction kernel).

Is that true if addresses now refer to elements not words?

No, addresses right now refer to words and we can read/write from/to memory only full words. The way we get around this for mem_load and mem_store instructions is by putting the things which don't end up on the stack into helper registers. So, we still read (or write) a full word, but part of the word goes into (or comes from) the helper registers.
As a side note, now that I look at the constraints of the mem_store operation it seems to me that there might be a soundness bug there (i.e., a malicious prover could inject incorrect values into the temporary registers, and overwrite the other 3 elements at a given address with arbitrary values). @Al-Kindi-0 - what do you think? If so, we might have to move back to the approach we used previously when each memory row in memory chiplet contained both old and new values.
Combing back to the design for new mem_load and mem_store operations. Their transition diagrams could look like so:

and

In the above:

$addr$ is the element address.
$waddr$ is the word address computed as $waddr = addr / 4$.
$s_0$ and $s_1$ is a binary encoding of the element index within a word. Thus, we can impose a constraint for valid address decomposition as follows: $addr \cdot 4 + 2 \cdot s_1 + s_0 = waddr$. In addition to this, we'll need constraints to enforce that $s_0$ and $s_1$ are binary, and that $add$ is a u32 value (not sure about the best way to do this yet).
$val_0, ..., val_3$ are the values currently stored at $waddr$.

In the case of MLOAD, the constraint for $val$ would be:
$$
val = (1 - s_0) \cdot (1 - s_1) \cdot val_0 + s_0 \cdot (1 - s_1) \cdot v_1 + (1 - s_0) \cdot s_1 \cdot v_2 + s_0 \cdot s_1 \cdot v_3
$$
The constraints for MSTORE seem to be somewhat more complicated - so, I'll need to think more about them. But in either case, we need to have at least 1 extra helper register.

...I think we'd incur extra 6 - 7 cycles per element read in this code. This is better than 9 - 10 cycles per read...but it is not dramatically better. So, there is still a question in my mind whether the extra effort is worth ~30% improvement.

I agree that the savings are relatively minimal, though I do think it's a matter of frequency - if such instructions occur frequently, the cycles saved start to add up fast for a program with any significant runtime. That said, the design here is a compromise for the sake of backwards-compatibility, and is really more about ergonomics/uniformity (whether the assembly is hand-written or compiler-generated). Overall though, I much prefer the approach of making memory element-addressable in general, since that seems like an option we can actually consider. The difference in cycle count for equivalent memory accesses between the two models would be night and day I think, since it removes a lot of complexity around accessing individual elements. It does impose a bit more overhead when accessing words, but only in situations where the alignment of the address is not known, and must be checked at runtime. In practice, accessing values smaller than a word in memory is going to be far more common than >= word-sized values (at least for code compiled through Wasm), so favoring those operations is going to have a bigger performance impact.

No, addresses right now refer to words and we can read/write from/to memory only full words. The way we get around this for mem_load and mem_store instructions is by putting the things which don't end up on the stack into helper registers. So, we still read (or write) a full word, but part of the word goes into (or comes from) the helper registers.

That makes sense, though I guess what I was wondering, is if that changes at all with element-addressable memory vs word-addressable memory. In other words, would that change just make memory look element-addressable, but preserve how memory loads/stores work in the VM today (i.e. word-oriented, with some changes to the constraints as you described)? Or would it also involve reworking the implementation of memory in the VM so that it is element-oriented rather than word-oriented?
Obviously I'm looking at this through a very narrow lens, so the importance of doing things like memory accesses in word-sized units might have far reaching implications I'm not thinking about, but in an element-addressable world, it seems out of place to implement memory accesses in terms of words, particularly if mem_load/mem_store would require multiple  helper registers that go essentially unused (beyond their role in the constraint system). It would obviously still be important to enforce word-alignment for mem_loadw/mem_storew, but I'm surprised it would be necessary to have the memory itself be word-oriented, if the word-alignment restriction ensures that there is always a full word at a given address.
Anyway, I want to avoid derailing the conversation here just because I don't have a solid grasp of how some of these pieces fit together in the VM internals, so if I'm missing obvious stuff here (or, obvious if you know the internals well), feel free to ignore the above. I'm certainly curious about the relationship between the memory instructions and their implementation in the VM (and how that affects the constraints), but I imagine it's probably been discussed/documented somewhere already, and I just need to go do some reading.

That makes sense, though I guess what I was wondering, is if that changes at all with element-addressable memory vs word-addressable memory. In other words, would that change just make memory look element-addressable, but preserve how memory loads/stores work in the VM today (i.e. word-oriented, with some changes to the constraints as you described)? Or would it also involve reworking the implementation of memory in the VM so that it is element-oriented rather than word-oriented?

The approach I described above would still keep the internal mechanics of memory word-oriented. So, basically, it would just look element-addressable.

Obviously I'm looking at this through a very narrow lens, so the importance of doing things like memory accesses in word-sized units might have far reaching implications I'm not thinking about, but in an element-addressable world, it seems out of place to implement memory accesses in terms of words, particularly if mem_load/mem_store would require multiple helper registers that go essentially unused (beyond their role in the constraint system). It would obviously still be important to enforce word-alignment for mem_loadw/mem_storew, but I'm surprised it would be necessary to have the memory itself be word-oriented, if the word-alignment restriction ensures that there is always a full word at a given address.

The word-oriented design is done primarily for efficiency reasons. For the use cases we are targeting, it will be very common to read/write memory as full words rather than individual elements. For example, nodes in a Merkle tree are words, output of the RPO hash function is a word (and inputs are 3 words), assets in Miden rollup are words etc. etc. So, we frequently have a need to read or write a full word (or even two consecutive words), and I wanted to make this as efficient as possible.
Consider mem_stream for example. This instruction reads two consecutive words and overwrites top 8 elements of the stack with the result in a single cycle. Implementing this instruction with element-addressable memory would not be possible in our current design (or we'd need to add extra columns which would slow down the rest of the VM). And emulating this instruction with individual loads would probably require ~30 cycles.

I think it's possible to switch to element-addressable memory and retain the efficiency gains of the current word-addressable memory. The memory chiplet would remain similar in that it would store words; when a single address is read/written, the whole word is read in/out (similar to how our mload/store instructions work). The only difference is that each element in the word gets its own address. Words are also always aligned (i.e. the first address in a word is always addr % 4 == 0).
Column layout sketch
Basically, I see the columns looking like (with a few left out to make the argument simpler):
r/w | e/w | addr | batch | idx0 | idx1 | clk | OLD_VALUES | NEW_VALUES

where the columns are as follows:

r/w: this is the current s0 flag; 0 for writes, 1 for reads
e/w: a flag whether a full word or just one element was accessed

say = 0 when element, = 1 when word

addr: the address of the word/element accessed

if unit/word == 1, then addr is aligned

batch/idx0/idx1: these are the decomposition of addr, where addr = 4*batch + idx

and idx = 2*idx1 + idx0
Since idx < 4 due to the bit decomposition, then every addr has a single batch/idx decomposition
Hence, we can think of batch as the index of the word

OLD_VALUES and NEW_VALUES: this follows from bobbin's proposition in the previous comment to address the current bug in how the current mstore works

except in this design, addressof OLD_VALUES[0] == NEW_VALUES[0] == batch, addressof OLD_VALUES[1] == NEW_VALUES[1] == batch + 1, and so on

Constraints sketch
To ensure that a READ is correct, we check
rw * (OLD_VALUES[i] - NEW_VALUES[i]) = 0 ; i = 0,1,2,3

To ensure the correctness of writes, we define flags f_i such that f_i == 1 when addr % 4 == i:

f_0 = (1-idx1) * (1-idx0)
f_1 = (1-idx1) * idx0
f_2 = idx1 * (1-idx0)
f_3 = idx1 * idx0

Then, when a single element is written (mstore), we constrain the other 3 values to be unchanged:
(1 - rw) * f0 * (OLD_VALUES[i] - NEW_VALUES[i]) = 0; i = 1,2,3
(1 - rw) * f1 * (OLD_VALUES[i] - NEW_VALUES[i]) = 0; i = 0,2,3
(1 - rw) * f2 * (OLD_VALUES[i] - NEW_VALUES[i]) = 0; i = 0,1,3
(1 - rw) * f3 * (OLD_VALUES[i] - NEW_VALUES[i]) = 0; i = 0,1,2

We also ensure that when a word is read/written (mstorew/mloadw), that addr is aligned. This is equivalent to checking that idx = 0 (i.e. the remainder when dividing by 4 is 0):
ew * (1 - idx1) * (1 - idx0) = 0

Conclusion
With element-addressable memory, we'd have 4GB memory instead of 16GB, but I don't think that's an issue. Unless I missed something, I think we should move to element-addressable memory as it solves the compiler's problems, and still retains the efficiency of the current design.
Note: we'd need a few changes to some instructions, where e.g. MSTREAM would increment the address by 8 instead of 2.

Thank you! I'm not sure I fully understand the design yet - so, a coupe of questions:

To implement memory, we usually need to sort the trace by address and then by clock cycle. What are we sorting by in this design? I'm assuming it is by batch, right?
What actually goes onto the bus for read/writes? Specifically, how do full word read/writes look different from single element read/writes?
How does memory access look like on the stack side (i.e., the other side of the memory bus)?

Right, I should have addressed those in the initial post.

To implement memory, we usually need to sort the trace by address and then by clock cycle. What are we sorting by in this design? I'm assuming it is by batch, right?

That's right. To give a better intuition, I'll give an example of a few load/stores and how to chiplet columns would be built:
(clk)
  1   mstore(addr=80, value=2)
  2   mstore(addr=42, value=100)
  3   mstore(addr=45, value=200)
  4   mload(addr=42)
  5   mstorew(addr=40, value=[1,2,3,4])  # This writes to addresses 40,41,42,43
  6   mloadw(addr=44)

 rw  |  ew  | addr | batch | idx0 | idx1 | clk | OLD_VALUES | NEW_VALUES
========================================================================
 w      e      42     10      0      1      2    [0,0,0,0]    [0,0,100,0]  # mstore(addr=42, value=100)
 r      e      42     10      0      1      4    [0,0,100,0]  [0,0,100,0]  # mload(addr=42)
 w      w      40     10      0      0      5    [0,0,100,0]  [1,2,3,4]    # mstorew(addr=40, value=[1,2,3,4])
 w      e      45     11      1      0      3    [0,0,0,0]    [0,200,0,0]  # mstore(addr=45, value=200)
 r      e      44     11      0      0      6    [0,200,0,0]  [0,200,0,0]  # mloadw(addr=44)
 w      e      80     20      0      0      1    [0,0,0,0]    [2,0,0,0]    # mstore(addr=80, value=2)

What actually goes onto the bus for read/writes? Specifically, how do full word read/writes look different from single element read/writes?
How does memory access look like on the stack side (i.e., the other side of the memory bus)?

The bus for element-wise read/writes only needs to include the element being read/written; the bus message for word read/writes is unchanged from the current version. Note that the instructions' API do not change (except that the address for mstorew and mloadw now needs to be aligned).
Bus message for word read/writes
The message looks like
$$
x = \alpha_0 + \alpha_1 \cdot op_{mem} + \alpha_2 \cdot \mathrm{ctx} + \alpha_3 \cdot \mathrm{addr} + \alpha_4 \cdot \mathrm{clk} + \sum_{i=5}^8 \alpha_{i} v_i
$$
On the chiplet side, $v_i$ corresponds to the NEW_VALUES[0..3]; the other variables come from the corresponding columns. Note that the bus cares about the address, not the batch.
On the stack side, the $v_i$'s are all on the stack.
Bus message for element read/writes
The message looks like
$$
y = \alpha_0 + \alpha_1 \cdot op_{mem} + \alpha_2 \cdot \mathrm{ctx} + \alpha_3 \cdot \mathrm{addr} + \alpha_4 \cdot \mathrm{clk} + \alpha_5 \cdot v
$$
Here, $v$ is the element being written/read.
On the chiplet side, $v = \sum_{i=0}^3 f_i \cdot \mathrm{NEW\_VALUES}[i]$. That is, the $f_i$'s select the appropriate column in NEW_VALUES.
On the stack side, $v$ is simply on the stack. Note that in this design, we don't populate the helper registers with unconstrained values.
Bus (putting it all together)
Reusing the notation from the docs,
$$
v_{mem} = ew \cdot x + (1 - eq) \cdot y
$$
$v_{mem}$ corresponds to the chiplet bus response; the chiplet bus request $u_{mem}$ (from the stack) is analogous (where $x$ and $y$ are constructed as described above for the stack side). As usual, the final bus equation is
$$
b_{chip}' \cdot u_{mem} = b_{chip} \cdot v_{mem}
$$

I figured out how to make it work with 15 columns total - which wouldn't require adding any new columns if we make the memory chiplet the second one right after the hasher chiplet (instead of the bitwise).
Memory chiplet columns
The columns would look like:
f_scb | rw | ew | ctx | batch | idx0 | idx1 | clk | VALUES | d0 | d1 | t

Notable changes are:

I included all the remaining columns that I omitted in previous posts (i.e. ctx, d0, d1, and t)
f_scb is 1 when the context and batch don't change as well as whether the current and next rows are in the memory chiplet, and 0 otherwise

i.e. $f_{scb} = [1 - (n_0 + (1-n_0)n_1)] \cdot f_{mem} \cdot f_{mem}' | degree=8$,

where $n_0$ and $n_1$ are reused from our docs, with the slight modification that $n_1 = \Delta batch \cdot t'$.

scb is a mnemonic for "same context and batch"
(we need this to reduce the degree of some constraints)

We no longer commit to addr

It is now a degree 1 "virtual" column: addr = 4*batch + 2*idx1 + idx0

We got rid of the 4 OLD_VALUES columns

Memory chiplet constraints
The ctx, batch, clk, d0, d1 and t columns work exactly the same as they currently do (where the former addr column is replaced with batch in this version). We already described how f_scb works. So it remains to show how we constrain the VALUES columns.
We will make use of the following variables:
$$
\begin{align}
f_{mem} &= s_0 \cdot (1 - s_1) \quad | degree = 2 \
&\rightarrow \textrm{is 1 when the current row is the memory chiplet, 0 otherwise} \
f_{firstrow} &= s_0 \cdot f_{mem}' \quad | degree = 3 \
&\rightarrow \textrm{is 1 when the next row is the first row of the memory chiplet, and 0 otherwise} \
f_0 &= (1-idx1) \cdot (1 - idx0) \quad | degree = 2 \
f_1 &= (1-idx1) \cdot idx0 \quad | degree = 2 \
f_2 &= idx1 \cdot (1 - idx0) \quad | degree = 2 \
f_3 &= idx1 \cdot idx0 \quad | degree = 2 \
&\rightarrow \textrm{these are the same flags as described in the previous posts} \
\end{align}
$$
We will also use v[i] = VALUES[i] for conciseness.
First row constraints
This subsection describes the constraints for the first row of the memory chiplet.

If first row is a read (element or word), then VALUES = [0,0,0,0].

$$
f_{firstrow} \cdot rw' \cdot v'[0] \cdot v'[1] \cdot v'[2] \cdot v'[3] = 0 \quad | degree = 8
$$

If an element is written, then all VALUES are 0 except for the element written. For $0 \leq j < 4$, and for all $i \neq j$,

$$
f_{firstrow} \cdot (1 - rw') \cdot (1 - ew') \cdot f_j' \cdot v'[i] = 0 \quad | degree = 8
$$

Note: We can combine some of the above 12 constraints (where some will be degree 9), but I omit this optimization here for notational convenience.

When a word is written, VALUES is unconstrained (well, only constrained by the bus).
Non-first row constraints

When a read occurs in the same ctx and batch, VALUES don't change. For $0 \leq i < 4$,

$$
f_{scb} \cdot rw' \cdot ( v'[i] - v[i]) = 0 \quad | degree = 3
$$

When an element write occurs in the same ctx and batch, then VALUES remains unchanged except for the index that was written. For $0 \leq i < 4$,

$$
\begin{align}
f_{scb} \cdot (1 - rw') \cdot (1 - ew')  \cdot f_0' \cdot (v'[i] - v[i]) = 0 \quad | degree = 6
\end{align}
$$

When a read occurs in a different ctx or batch, VALUES = [0,0,0,0].

$$
f_{mem}' \cdot (1 - f_{scb}) \cdot rw' \cdot \prod_{i=0}^3 v'[i] = 0 \quad | degree = 8
$$

When an element write occurs in a different ctx or batch, all VALUES are 0 except for the element written. For $0 \leq j < 4$, and for all $i \neq j$, (12 constraints)

$$
f_{scb} \cdot (1 - rw') \cdot (1 - ew') \cdot f_j' \cdot v'[i] = 0 \quad | degree = 6
$$
Bus
The bus works just as described in the previous post, with the slight caveat that addr is no longer a committed column (as previously explained). However, since addr is degree 1, the degree of the constraints for the bus don't change.
Conclusion
Please double check everything, but assuming that this is right, I think we should definitely move to element-addressable memory, as we can do it without adding any column. There might also be a simpler set of constraints (e.g. that merge some of the "first row" and "not-first row" constraints), but at least we know of one set of constraints that works.

Thank you for this proposal.
A few questions to clarify my understanding:

The s0 and s1 selectors are probably the (global) chiplet selectors, right?
This is probably related to the first point, but I am not sure I follow the interpretation of the $f_{\text{firstrow}}$. If we want to select the first row of the memory chiplet, assuming it is right after the hasher, then I believe that expression should be changed.
I believe the constraints for first row read should be separated for each element. Also, and I think this is related to the previous two points, I am not following the use of the next modifier on some of the terms in that constraint.
In the non-first-row constraints, since $f_0$ is used, I believe the inequality should be strict $0 < i < 4$. For the other flags $f_i$, a variation on this idea should apply too, basically point 4 in your comment.
Same comment for point 3 in non-first-row constraints about separating the constraints.

Overall, I think this looks promising and we should go for it once we are all on-board.

Not a super detailed review of the proposal yet from my end. Overall, I think this approach should work - but a couple of questions comments:
First, shouldn't $f_{mem}$ be defined as $s_0 \cdot s_1 \cdot (1 - s_2)$? That is, memory chiplet uses 3 external selector columns. If so, then the degree of some constraints would increase by 1 or 2.
Second, I wonder if we could simplify constraint description by defining a flag per value which indicates whether the value should be copied over from the previous row. We could call this flag something like $c_i$ - one for each of the $v$ values. For example, $c_0$ could look something like this:
$$
c_0 = (f_0 + f_{ew} - f_0 \cdot f_{ew}) \cdot (1 - f_{rw}) | degree = 4
$$
Basically, we are saying that $c_0 = 1$ if (($f_0 == 1$ OR $f_{ew} == 1$) AND $f_{rw} == 0$).
Using $c_i$ flags, we should be able to express many of the read/write and word/element constraints more uniformly. Specifically:
$$
c_i \cdot (v'_i - v_i) = 0 | degree = 5
$$
Should be sufficient to define all non-bus constraints for the $v_i$ columns.
We can also extend this to handle "new batch" situations. Assuming we have $f_{scb}$ column which indicates that we are creating the first row for a given address, we may be able to write the constraint as:
$$
c_i \cdot (v_i' - (1 - f_{scb}) \cdot v_i ) = 0 | degree = 6
$$

Response to Al

The s0 and s1 selectors are probably the (global) chiplet selectors, right?

Correct.

This is probably related to the first point, but I am not sure I follow the interpretation of the $f_{firstrow}$ . If we want to select the first row of the memory chiplet, assuming it is right after the hasher, then I believe that expression should be changed.

Ah yes, I changed it to $f_{firstrow} = (1 - s_0) \cdot f_{mem}'$. So this evaluates to 1 if the current row is the hasher chiplet ($1 - s_0$), and the next row is the memory chiplet ($f_{mem}'$).

I believe the constraints for first row read should be separated for each element.

Fixed.

Also, and I think this is related to the previous two points, I am not following the use of the next modifier on some of the terms in that constraint.

And based on answer (2), the way to think about $f_{firstrow}$ is really "next row is first memory chiplet row", which explains the next modifiers. Or in plain terms, if we don't use them, then we're looking at the hasher chiplet, which is obviously wrong.

In the non-first-row constraints, since $f_0$ is used, I believe the inequality should be strict $0 < i < 4$ . For the other flags $f_i$ , a variation on this idea should apply too, basically point 4 in your comment.
Same comment for point 3 in non-first-row constraints about separating the constraints.

Fixed
Response to Bobbin

First, shouldn't $f_mem$ be defined as s 0 ⋅ s 1 ⋅ ( 1 − s 2 ) ? That is, memory chiplet uses 3 external selector columns. If so, then the degree of some constraints would increase by 1 or 2.

My proposal assumes that we put the memory chiplet right after the hasher chiplet (in place of the current bitwise chiplet).

Second, I wonder if we could simplify constraint description by defining a flag per value which indicates whether the value should be copied over from the previous row.

I'm not sure I follow how $c_0$ is defined. First, values will only ever need to be copied in the same context & batch, so I'd expect $f_{scb}$ in there. Also, when we write a word, $f_0 = 1$ (since index is always 0 for words), $f_{ew} = 1$, and $f_{rw} = 0$, so $c_0 = (1 + 1 - 1)(1 - 0) = 1$. But writing a word is a case where $c_0$ should be $0$ (i.e. we don't expect the value to be copied over from the previous row).

We can also extend this to handle "new batch" situations. Assuming we have $f_{scb}$ column which indicates that we are creating the first row for a given address, we may be able to write the constraint as:

$$
c_i \cdot (v_i' - (1 - f_{scb}) \cdot v_i ) = 0 | degree = 6
$$
Not sure I understand how this one is built either. If we are in a "new batch" situation, then $f_{scb} = 0$. So the above is simplified to
$$
c_i \cdot (v_i' - v_i ) = 0
$$
But to my previous comment, I'd always expect $c_i = 0$, since we are in a new batch and no element should be copied across a new batch. This doesn't enforce that all $v_i$ are 0 either.

I'm not sure I follow how $c_0$ is defined. First, values will only ever need to be copied in the same context & batch, so I'd expect $f_{scb}$ in there. Also, when we write a word, $f_0 = 1$ (since index is always 0 for words), $f_{ew} = 1$, and $f_{rw} = 0$, so $c_0 = (1 + 1 - 1)(1 - 0) = 1$. But writing a word is a case where $c_0$ should be $0$ (i.e. we don't expect the value to be copied over from the previous row).

Yes - I made quite a few mistakes in my logic here and I'll describe a (hopefully) more correct approach which illustrates what I was thinking. But first I think there may be an issue with $f_{scb}$ flag.
Specifically, I'm not sure we can rely solely on this flag to identify memory chiplet constraints. That is, every memory chiplet constraint still has to be multiplied by $f_{mem}$. The reasoning is that since $f_{scb}$ is a committed value, it can so happen that this value will be set to $1$ in other chiplets as well (e.g., in a hasher chiplet or in bitwise chiplet etc.). If this happens, some "irrelevant" constraints may be turned on in these places which could cause completness issues (maybe soundness issues are possible as well, but I haven't thought this through).
So, I'd propose to redefine this flag as $f_{scb} = [1 - (n_0 + (1-n_0) \cdot n_1)] \cdot f_{mem}'$. Basically, it would be $1$ if the context and batch don't change and the next row is in the memory chiplet. Or, conversely, this flag is set to $0$ for the first row of a trace for a given context+batch, and for the very last row of the memory chiplet.
Now, using this definition of $f_{scb}$, I'll try to define a $c_i$ value which is $1$ when the value of $v_i$ should be copied over from the previous row and $0$ otherwise. But let's build up to it step by step.
First, at the high-level, the value of $v_i$ needs to be copied over to the next row if: (1) we are doing a read operation, OR (2) we are doing a write operation and the $v_i$ is note selected for the update. So, let's first define a flag $z$ which will be $1$ when a value is not selected for the update. It would look something like this:
$$
z_i = (1 - f_{ew}) \cdot (1 - f_i)
$$
Basically, this flag is $1$ only when both $f_{ew} = 0$ (we are not operating on the entire word) and $f_i = 0$ (we are not operating on this specific value).
Using this definition of $z_i$ and assuming $f_{rw} = 1$ for read operations, we can define $c_i$ as:
$$
c_i = f_{rw} + z_i - f_{rw} \cdot z_i | degree = 4
$$
But as you mentioned in your comment, we want this flag to be set to $1$ only if the context/batch combination doesn't change - and so, we can update the definition of $c_i$ to be:
$$
c_i = f_{scb} \cdot (f_{rw} + z_i - f_{rw} \cdot z_i) | degree = 5
$$
Then, the constraint that we want to express becomes: when we are in the memory chiplet and $c_i = 1$, the value of $v_i$ should be copied over:
$$
f_{mem} \cdot c_i \cdot (v'_i - v_i) = 0  | degree = 9
$$
The degree $9$ here assumes that we keep the order of chiplets the same, but if we re-arrange them as you've suggested, it would drop to 8.
The above constraint should cover all $v_i$ transitions except for initializing the first row of the trace for each batch. I haven't though through these constraints - so, not 100% sure this structure will work for them, but may be worth exploring.

Ah yes, those constraints work!

Do we also need a constraint to ensure that e n value can be flipped from 0 to 1 only on rows which are multiples of 8 ?

Ah yes we need that, I will fix it in the previous comment.

The above constraint should cover all $v_i$ transitions except for initializing the first row of the trace for each batch. I haven't though through these constraints - so, not 100% sure this structure will work for them, but may be worth exploring.

I think I found a way to extend your idea to the "first row for each batch" case. I will assume that we make the memory chiplet come right after the hasher chiplet again in order to not have to add a new column to the VM for $f_{scb}$. I will also be reusing your definition of $f_{scb}$.
I will adjust slightly your definition of $c_i$ to be:
$$
c_i = (1 - f_{rw}) + f_{rw} (1 - f_{ew}) (1 - f_i) | degree=4
$$
which is $1$ when $v'[i]$ is not written to, and $0$ otherwise. Hence, the way to think about $c_i$ is "everytime $v'[i]$ needs to be constrained". Depending on the context, this constraint will either be "the same as $v[i]$" or 0.
first row
For $0 < i < 4$,
$$
f_{firstrow} \cdot c_i \cdot v'[i] = 0 | degree=8
$$
That is, if $row'$ is the first first row of the memory chiplet, and $v'[i]$ is not written to, then $v'[i]$ must be 0.
non first row, new batch/context
For $0 < i < 4$,
$$
f_{mem} \cdot (1 - f_{scb}) \cdot c_i \cdot v'[i] = 0 | degree=8
$$
That is, $row'$ is a new batch/ctx in the memory chiplet, and $v'[i]$ is not written to, then $v'[i]$ must be 0.
non first row, same batch/context
For $0 < i < 4$,
$$
f_{mem} \cdot f_{scb} \cdot c_i \cdot (v'[i] - v[i]) = 0 | degree=8
$$
That is, $row'$ is in the same batch/ctx in the memory chiplet, and $v'[i]$ is not written to, then $v'[i]$ must be 0.

Nice! I didn't go through all the constraints in detail, but on a quick look this should work. One question: if we keep the order of chiplets the same as now, would the degrees of these constraints go up to $9$? If so, maybe it is worth keeping the order the same as it will much less work.

Double checked the constraints and all looks good.
Two points:

There is a mismatch between the different comments on the definition of the flag $f_{rw}$.
Given the above, the definition of the $c_i$ flag is the same in all of the last three comments, modulo the factor $f_{scb}$.
Given the above, I would define $c_i$ in the penultimate comment using the intermediate value $z_i$ as this is cleaner and more principled.

I just noticed a flaw in the design: decomposing addr = 4*batch + idx, idx < 4 will force the prover to write e.g. for addr=41, batch=10 and idx=1 only if we were in the integers. In a field, we have a division operation (defined over all pairs of elements), so for addr=41, the prover could just write batch=41/4 and idx=0.
The fix for this is to do a 32-bit range check on batch and 4*batch + idx - hence, ensuring the "integer behavior" that we want.

Note: technically we want a 32-bit range check on 4*batch, but we check 4*batch + idx since it both checks that 4*batch < 2^32 (due to 0<=idx<4), and also constrains addr = 4*batch + idx < 2^32 (a check we don't have today).

Repercussions
This means we'll need an extra column in the chiplet (since batch will need to be split in its 16-bit parts), and we'll need an extra logup bus column for the new range checks (since the current b_chip is full). From offline discussions, we were going to need more columns for the hasher chiplet anyways, so the actual extra cost here would be the extra auxiliary trace column. But arguably our current memory constraints lack a range check on addr, so we'd need that auxiliary column anyways for correctness.
Appendix
This time I double-checked my intuition by proving (experimentally) that

x % 4 = 0 if and only if x / 4 < 2^32, and (equivalently)
x % 4 != 0 if and only if x / 4 >= 2^32

In other words, x/4 will only be "small" if x is "divisible by 4" in the integer sense. For example, 40 / 4 = 10 (mod MODULUS), which is indeed smaller than 2^32. But 41/4 =<PHONE_NUMBER>2060938251 (mod MODULUS) is greater than 2^32.

MODULUS is the modulus for the Goldilocks field

Below is the code to check these claims:
fn main() {
    const FOUR: Felt = vm_core::Felt::new(4);

    let max = u32::MAX / 4;
    let two_power_32 = 2_u64.pow(32);

    for batch_idx in 0..max {
        let x = Felt::from(batch_idx * 4);

        assert!((x / FOUR).as_int() < two_power_32, "x: {x}");
        assert!(((x + 1_u32.into()) / FOUR).as_int() >= two_power_32, "x: {x}");
        assert!(((x + 2_u32.into()) / FOUR).as_int() >= two_power_32, "x: {x}");
        assert!(((x + 3_u32.into()) / FOUR).as_int() >= two_power_32, "x: {x}");
    }
}


feat: update osm editors urls

https://kontur.fibery.io/Tasks/Task/BE-Default-OSM-editor-usage-causes-Unknown-editor-error-18784

Summary by CodeRabbit

New Features

Simplified URLs for JOSM and iD mapping editors to enhance readability and user experience.

Bug Fixes

Removed unnecessary query parameters to streamline access to mapping editors.

Chores

Minor formatting adjustments for cleaner configuration files.

A comment from previous PR that was lost: https://github.com/konturio/disaster-ninja-fe/pull/799#discussion_r1682744350


Refactor Configuration DSN into Endpoint and Provider Id

The DSN wasn’t thought through enough to work for WebSocket paths that were rooted under versions. I’ve moved away from it for now, though it may come back.
The DSN is replaced with a wss:// URI and Provider Id.
Additionally, authentication credentials are now specified separately, which invalidates and closes #7.

I just bumped the version to 0.2 as well, since this is a breaking change.

Code looks fine, still need to test functionality though.

Worked well!

Awesome! Merging and tagging, v0.2.0.


Web UI Status shows old device name when applying new config via UI

Current Behavior
When a new device config is applied via Web UI, the status info refreshes but the older Device Name is still shown.

This suggests that agent.currentSettings.env.FF_DEVICE_NAME was not updated (sinces that is where the value comes from.
Confusingly, the device Id appears to be correct and that comes from  agent.currentSettings.env.FF_DEVICE_ID
Expected Behavior
correct values to be displayed on status page.
correct values in env var
Steps To Reproduce
with a running device agent (with an instance & snapshot assigned):

create a new device & copy its config
apply the config to the running device via the devices Web UI

Environment

FlowForge version: 1.8-git
Agent Version - 0.9-git

Further findings:

/project/settings.json holds old values

/opt/flowforge-device/project$ cat settings.json | jq .

{
  "credentialSecret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "port": 5001,
  "codeEditor": "monaco",
  "forge-light": {
    "launcherVersion": "0.9.0",
    "forgeURL": "https://forge.xxxxxxxxxx.xxx",
    "projectURL": "https://forge.xxxxxxxxxxxxxxx/device/EpkyL0OyqX/overview"  // << OLD Device ID
  },
  "editorTheme": {
    "theme": "forge-light",
    "codeEditor": {
      "lib": "monaco"
    },
    "tours": false
  },
  "flowforge": {
    "forgeURL": "https://forge.xxxxxxxxxxx.xxx",
    "teamID": "WO2NoXgaex",
    "projectLink": {
      "token": "ffd_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "broker": {
        "url": "wss://mqtt.xxxxxxxxxxxxxxx.xxx",
        "username": "device:WO2NoXgaex:EpkyL0OyqX",    // << OLD Device ID
        "password": "ffbd_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      }
    }
  }
}

For reference, this is the device yml
deviceId: nqJykYLb98  // << Correct
token: ffd__xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
credentialSecret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
forgeURL: https://forge.xxxxxxxxxx.xxx
brokerURL: wss://mqtt.xxxxxxxxxx.xxx
brokerUsername: device:WO2NoXgaex:nqJykYLb98 // << Correct
brokerPassword: ffbd_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


openai[minor]: implement langchain-openai package

Todo

[x] copy over integration tests
[x] update docs with new instructions in #15513
[x] add linear ticket to bump core -> community, community->langchain, and core->openai deps
[ ] (optional): add pip install langchain-openai command to each notebook using it
[ ] Update docstrings to not need openai install

Contributor steps:

[x] Add secret names to manual integrations workflow in .github/workflows/_integration_test.yml
[x] Add secrets to release workflow (for pre-release testing) in .github/workflows/_release.yml

Maintainer steps (Contributors should not do these):

[ ] set up pypi and test pypi projects
[ ] add credential secrets to Github Actions
[ ] add package to conda-forge

Functional changes to existing classes:

now relies on openai client v1 (1.6.1) via concrete dep in langchain-openai package

Codebase organization

some function calling stuff moved to langchain_core.utils.function_calling in order to be used in both community and langchain-openai

Added to PR description


Prevent cross-origin information leakage via range requests

(this is a follow-up to https://phabricator.services.mozilla.com/D91746#3198225)
PDF.js does currently not validate the origin of a PDF response before using it. In web pages, this is not a problem, because unauthorized access is blocked by the same-origin policy. In privileged contexts, such as a browser feature or a browser extension, this can become problematic as soon as it is possible to read content from a PDF file into a web page. Even a unidirectional channel with a binary value (e.g. a boolean yes or no) is enough to read information. With the ongoing work for scripting support (relevant components linked from https://github.com/mozilla/pdf.js/pull/12689#pullrequestreview-549350284), the condition for exploitation is about to be met.
In pages 12 and 13 of https://robwu.nl/s/bugswat2019rw.pdf#page=12, I sketched an attack that abuses Range requests to stitch together responses from different origins to read data across origins, with a proof-of-concept exploit that targeted Chrome / PDFium (CVE-2016-5206).
PDF.js does currently not validate the origins/URLs of responses either - we should do that to prevent a similar attack from happening, at least in the following places:

https://searchfox.org/mozilla-central/rev/de782976bf97669f1e8edee59e7a2398154efe06/toolkit/components/pdfjs/content/PdfJsNetwork.jsm#84-116
src/display/network.js

We should determine the origin of the PDF file (e.g. from the first response) and then verify that the origin is consistent (and otherwise reject the response). xhr.responseURL can be used to obtain the final response URL, after all redirections. If fetch is used, response.url has the same information.
If we are concerned about potentially breaking functionality for users, we can start by adding telemetry to see how often the origin of the initial response differs from the origin of subsequent requests, before we start to enforce the new restriction.

PDF.js does currently not validate the origins/URLs of responses either - we should do that to prevent a similar attack from happening, at least in the following places:
* https://searchfox.org/mozilla-central/rev/de782976bf97669f1e8edee59e7a2398154efe06/toolkit/components/pdfjs/content/PdfJsNetwork.jsm#84-116

* [`src/display/network.js`](https://github.com/mozilla/pdf.js/blob/48e27a1a22a83f1f1be481f992f75f9cee7e5804/src/display/network.js#L84-L107)

Just asking, but shouldn't this include https://github.com/mozilla/pdf.js/blob/master/src/display/fetch_stream.js and https://github.com/mozilla/pdf.js/blob/master/src/display/node_stream.js as well?

Just asking, but shouldn't this include https://github.com/mozilla/pdf.js/blob/master/src/display/fetch_stream.js and https://github.com/mozilla/pdf.js/blob/master/src/display/node_stream.js as well?

Yes. I didn't enumerate everything, but just some of the examples (hence "at least in the following places").

For MC, I think we can disable redirects e.g. https://searchfox.org/mozilla-central/rev/8698fade12984b9a6a43a85a287a5f17e8fd4ddf/toolkit/components/captivedetect/CaptiveDetect.jsm#46

Disabling redirects is a way to resolve this issue, but only if we are absolutely sure that it is safe to do so.
I like the simplicity, but it is more restrictive than permitting same-origin redirects.

If we can disable it and prevent this potential security issue, please disable it and land the patch.

Disabling redirects is a way to resolve this issue, but only if we are absolutely sure that it is safe to do so.

Unless I'm missing something, I believe this is how chrome fixed it.

Disabling redirects is a way to resolve this issue, but only if we are absolutely sure that it is safe to do so.

Unless I'm missing something, I believe this is how chrome fixed it.

That is safe (i.e. not breaking legitimate use cases) IF the PDF resource URL is the final URL. If that is the case, simply blocking redirects can fix this issue.

FYI I've also reported this with more details at https://bugzilla.mozilla.org/show_bug.cgi?id=1683940 which was associated with CVE-2021-23953.
This issue was fixed in Firefox, without any code changes in PDF.js because the privileged viewer integration code is fully contained in Firefox's code base.
The Chrome extension (and any other environment without same origin policy) is still vulnerable to this issue.


Analytics View Helper

Once we have the DataCollector\Generic we could set it as default.
Renderer is very user specific, instead.


crypto, x/crypto: use math.bits rotate functions instead of ad-hoc implementations

@FiloSottile

The following query finds a number of additional cases in /x/crypto when run with GOARCH=arm64 to scan the non-amd64-assembly versions.
gogrep -x       '$a >> $k | $a << (64 - $k)' ./...
gogrep -x       '($a >> $k) | ($a << (64 - $k))' ./...
gogrep -x       '$a << $k | $a >> (64 - $k)' ./...
gogrep -x       '($a << $k) | ($a >> (64 - $k))' ./...
gogrep -x       '$a >> (64 - $k) | $a << $k' ./...
gogrep -x       '($a >> (64 - $k)) | ($a<< $k)' ./...
gogrep -x       '$a << (64 - $k) | $a >> $k' ./...
gogrep -x       '($a << (64 - $k)) | ($a >> $k)' ./...
gogrep -x       '$a >> $k | $a << (32 - $k)' ./...
gogrep -x       '($a >> $k) | ($a << (32 - $k))' ./...
gogrep -x       '$a << $k | $a >> (32 - $k)' ./...
gogrep -x       '($a << $k) | ($a >> (32 - $k))' ./...
gogrep -x       '$a >> (32 - $k) | $a << $k' ./...
gogrep -x       '($a >> (32 - $k)) | ($a<< $k)' ./...
gogrep -x       '$a << (32 - $k) | $a >> $k' ./...
gogrep -x       '($a << (32 - $k)) | ($a >> $k)' ./...

Probably we can close this one as soon as the #171731 would be merged.
➜  src git:(master) find crypto -type f -name "*.go" -exec egrep ".+>>(\d)+ \| .+<<\((\d)+-(\d)+\)" {} \+
crypto/sha256/sha256block.go: t1 := (v1>>17 | v1<<(32-17)) ^ (v1>>19 | v1<<(32-19)) ^ (v1 >> 10)
crypto/sha256/sha256block.go: t2 := (v2>>7 | v2<<(32-7)) ^ (v2>>18 | v2<<(32-18)) ^ (v2 >> 3)
crypto/sha256/sha256block.go: t1 := h + ((e>>6 | e<<(32-6)) ^ (e>>11 | e<<(32-11)) ^ (e>>25 | e<<(32-25))) + ((e & f) ^ (^e & g)) + _K[i] + w[i]
crypto/sha256/sha256block.go: t2 := ((a>>2 | a<<(32-2)) ^ (a>>13 | a<<(32-13)) ^ (a>>22 | a<<(32-22))) + ((a & b) ^ (a & c) ^ (b & c))

@FiloSottile

Probably we can close this one as soon as the #171731 would be merged.
➜  src git:(master) find crypto -type f -name "*.go" -exec egrep ".+>>(\d)+ \| .+<<\((\d)+-(\d)+\)" {} \+
crypto/sha256/sha256block.go: t1 := (v1>>17 | v1<<(32-17)) ^ (v1>>19 | v1<<(32-19)) ^ (v1 >> 10)
crypto/sha256/sha256block.go: t2 := (v2>>7 | v2<<(32-7)) ^ (v2>>18 | v2<<(32-18)) ^ (v2 >> 3)
crypto/sha256/sha256block.go: t1 := h + ((e>>6 | e<<(32-6)) ^ (e>>11 | e<<(32-11)) ^ (e>>25 | e<<(32-25))) + ((e & f) ^ (^e & g)) + _K[i] + w[i]
crypto/sha256/sha256block.go: t2 := ((a>>2 | a<<(32-2)) ^ (a>>13 | a<<(32-13)) ^ (a>>22 | a<<(32-22))) + ((a & b) ^ (a & c) ^ (b & c))

@FiloSottile

There is code in x/crypto that contains bit rotation operations. Should it be changed in case of this particular issue?

I would be happy to take two global CLs, one for crypto/... and one for x/crypto/..., fixing all cases identified by gogrep according to https://github.com/golang/go/issues/31456#issuecomment-483104913.

@FiloSottile Is there any chance to land changes mentioned in https://github.com/golang/go/issues/31456#issuecomment-946300450?

what's left here?

My semgrep rule (above) doesn't have any hits in crypto/... or x/crypto/....  Seems like we're probably good to close this.