[CELEBORN-885][SPARK] Shade RoaringBitmap to avoid dependency conflicts

What changes were proposed in this pull request?
Shade roaring bitmap to void dependency conflicts.
Why are the changes needed?
Some user reports that celeborn client will introduce roaring bitmap conflicts.
Does this PR introduce any user-facing change?
NO.
How was this patch tested?
GA and cluster.

sbt should be updated too, cc @cfmcgrady

Would you mind updating the file project/CelebornBuild.scala, following the same rule as we do in the Flink assembly project?
          ShadeRule.rename("org.roaringbitmap.**" -> "org.apache.celeborn.shaded.org.roaringbitmap.@1").inAll

BTW, NOTICE for the shaded module should be updated too

Would you mind updating the file project/CelebornBuild.scala, following the same rule as we do in the Flink assembly project?
           val cp = (assembly / fullClasspath).value
           cp filter { v =>
             val name = v.data.getName
-            !(name.startsWith("celeborn-") || name.startsWith("protobuf-java-") ||
-              name.startsWith("guava-") || name.startsWith("netty-") || name.startsWith("commons-lang3-"))
+            !(name.startsWith("celeborn-") ||
+                name.startsWith("protobuf-java-") ||
+                name.startsWith("guava-") ||
+                name.startsWith("netty-") ||
+                name.startsWith("commons-lang3-") ||
+                name.startsWith("RoaringBitmap-"))
           }
         },

           ShadeRule.rename("com.google.protobuf.**" -> "org.apache.celeborn.shaded.com.google.protobuf.@1").inAll,
           ShadeRule.rename("com.google.common.**" -> "org.apache.celeborn.shaded.com.google.common.@1").inAll,
           ShadeRule.rename("io.netty.**" -> "org.apache.celeborn.shaded.io.netty.@1").inAll,
-          ShadeRule.rename("org.apache.commons.**" -> "org.apache.celeborn.shaded.org.apache.commons.@1").inAll
+          ShadeRule.rename("org.apache.commons.**" -> "org.apache.celeborn.shaded.org.apache.commons.@1").inAll,
+          ShadeRule.rename("org.roaringbitmap.**" -> "org.apache.celeborn.shaded.org.roaringbitmap.@1").inAll
         ),

         (assembly / assemblyMergeStrategy) := {

OK，I‘ll update it.

BTW, NOTICE for the shaded module should be updated too

RoaringBitmap has no notice and its using Apache Licencese V2. We already contains its license info in LICENSE-binary file.

Would you mind updating the file project/CelebornBuild.scala, following the same rule as we do in the Flink assembly project?
           val cp = (assembly / fullClasspath).value
           cp filter { v =>
             val name = v.data.getName
-            !(name.startsWith("celeborn-") || name.startsWith("protobuf-java-") ||
-              name.startsWith("guava-") || name.startsWith("netty-") || name.startsWith("commons-lang3-"))
+            !(name.startsWith("celeborn-") ||
+                name.startsWith("protobuf-java-") ||
+                name.startsWith("guava-") ||
+                name.startsWith("netty-") ||
+                name.startsWith("commons-lang3-") ||
+                name.startsWith("RoaringBitmap-"))
           }
         },

           ShadeRule.rename("com.google.protobuf.**" -> "org.apache.celeborn.shaded.com.google.protobuf.@1").inAll,
           ShadeRule.rename("com.google.common.**" -> "org.apache.celeborn.shaded.com.google.common.@1").inAll,
           ShadeRule.rename("io.netty.**" -> "org.apache.celeborn.shaded.io.netty.@1").inAll,
-          ShadeRule.rename("org.apache.commons.**" -> "org.apache.celeborn.shaded.org.apache.commons.@1").inAll
+          ShadeRule.rename("org.apache.commons.**" -> "org.apache.celeborn.shaded.org.apache.commons.@1").inAll,
+          ShadeRule.rename("org.roaringbitmap.**" -> "org.apache.celeborn.shaded.org.roaringbitmap.@1").inAll
         ),

         (assembly / assemblyMergeStrategy) := {

I'll update it.

BTW, NOTICE for the shaded module should be updated too

RoarbingBitmap has no notice and its using apache license. We already contains its license info in LICENSE-binary. Is there anything else need to put in NOTICE?

Not the NOTICE-binary, technically, each jar we published to maven central should contain its own NOTICE file, which reflects the real content itself. For shaded jars, we need to maintain the NOTICE file manually.

Not the NOTICE-binary, technically, each jar we published to maven central should contain its own NOTICE file, which reflects the real content itself. For shaded jars, we need to maintain the NOTICE file manually.

Any reference for bundle notice into jar file？

https://github.com/apache/kyuubi/tree/master/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF


Do not delete the downloaded xip after a download has completed

Tell us how we can improve Xcodes
Hello, what about keeping the downloaded xip files on disk after a download? This could help sharing it with other people on the same LAN.
Is your feature request related to a problem? Please describe.
The "problem" is that after a long download has completed, I can't share my downloaded xip with other people in the room.
What would you like to see? How would you like it to work?
A solution could be to move the xip in the Trash when a download has completed, instead of deleting it.
This would:

Prevent the xip from remaining in the "Cache Local" folder (~/Library/Application\ Support/com.robotsandpencils.XcodesApp) forever, hidden from user's sight, consuming disk space like a big blob of useless bytes.
Allow users to easily delete the xip (empty the Trash)
Allow user to share the xip (move the xip out of the Trash and do whatever they want).

What do you think?

Hey @groue
Thanks for the suggestion. Xcodes does just move the xip to the trash bin. It doesn't delete it fully. So you could just move that xip out of the trash to whereever you need to so others can use it.
Please let me know if that answers your question

Oops, it just looks like I requested a feature that was already implemented 😅
(Just confirmed: a downloaded archive is sitting right in my trash bin)
Thank you for the great app!

Can we just provide an option to not move it to the trash? I checked out my trash, but unfortunately I've cleaned my trash..


KeyError: 'pi'

System enviroment:
Ubuntu20.04
torch                        2.0.0+cu118
torchvision                  0.15.1+cu118
Commandline:
#run_vqav2_ft.py  --train_config_file=vqa_train_config.json
Error Description as below:
/home/steven/anaconda3/envs/nlp/bin/python /home/steven/workstore/nlp/VLE-main/run_vqav2_ft.py --train_config_file=vqa_train_config.json
/home/steven/workstore/nlp/VLE-main/run_vqav2_ft.py:76: SyntaxWarning: "is" with a literal. Did you mean "=="?
max_epochs=_config["max_epoch"] if max_steps is -1 else 1000,
2023-09-20 13:52:41.071596: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Global seed set to 0
Some weights of VLEForVQA were not initialized from the model checkpoint at hfl/vle-base and are newly initialized: ['vqa_classifier.1.bias', 'vqa_classifier.3.bias', 'vqa_classifier.3.weight', 'vqa_classifier.0.bias', 'vqa_classifier.1.weight', 'vqa_classifier.0.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
File "/home/steven/workstore/nlp/VLE-main/run_vqav2_ft.py", line 107, in
main(train_config)
File "/home/steven/workstore/nlp/VLE-main/run_vqav2_ft.py", line 25, in main
model = VLEForVQA_PL(_config)
File "/home/steven/workstore/nlp/VLE-main/vqav2_train_module.py", line 73, in init
new_state_dict = extend_position_embedding(self.model.state_dict(), patch_size, config["image_size"])
File "/home/steven/workstore/nlp/VLE-main/models/VLE/modeling_vle.py", line 124, in extend_position_embedding
state_dict[keys['pi'][0]] = torch.arange(grid_after*grid_after + 1).unsqueeze(0)
KeyError: 'pi'
Process finished with exit code 1
when i debug step by step, i found that cannot found key value "vision_model.embeddings.position_ids"  in parameters list of model 'vle-base'.
Any body encounter same question as me?
please kindly help to solve this problem!
Tks~

@ymcui @GoGoJoestar @airaria waiting for your response

The pi was added to keys if there is a parameter ends with vision_model.embeddings.position_ids. The related parameter model.vle.vision_model.vision_model.embeddings.position_ids is a buffer registered in the class transformers.models.clip.modeling_clip.CLIPVisionEmbeddings, you can check if it in the model vle-base as following:
for n,b in model.named_buffers():
    print(n, b.shape)

If it not in the model, you could check the related code of CLIP model in transformers, or try reinstalling transformers, re-pull VLE repo.


Adding description for identity param

https://github.com/MicrosoftDocs/office-docs-powershell/issues/5131

Docs Build status updates of commit 9ba68cc:
:clock10: Full build: the average full build time is 12 min(s) 27 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 9ba68cc:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

skype/skype-ps/skype/Set-CsOnlineDirectoryUser.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


config/index.js hard-codes localhost as config.dev.host

If you override the host from localhost to <IP_ADDRESS> in your webpack config when starting webpack-dev-server, the script will still output "Your application is running here: http://localhost:8080" instead of "Your application is running here: http://<IP_ADDRESS>:8080".
I've fixed this locally by making the following config/index.js diff:
-    host: 'localhost', // can be overwritten by process.env.HOST
+    host: process.env.HOST || 'localhost', // can be overwritten by process.env.HOST

I'd issue a pull request but I'm not sure yet if this has any negative affect on things like tests.

Thanks for reporting this. We already fixed this in the develop branch / a PR.
It will be published shortly.


Use "app" label in selectors

Fixes issue #16 - Deployment's selector should be immutable
Scenario:

Deployment is created with kube.libsonnet, CI process adds additional labels
e.g. "ci-job-id"
Deployment is updated with the same process but the CI process was updated to
add new label e.g. "ci-cluster"
kubectl apply fails as the selector field is immutable

This adds an additional label "app" to Pod, Deployment, StatefulSet, DaemonSet,
Job and CronJob objects and use it when those objects needs to be referred by
selectors.

Thanks for the suggestion!  Using "app" is probably a common situation for people migrating an existing config to kube.libsonnet, so it's good to see what we can do to make this easier.
There are a couple of conceptually-separate changes here, which I'd like to discuss separately:
1. Rename name=foo label to app=foo
I don't think we can do (1) as it is in this PR :disappointed:.  I know a lot of the rest of the community uses "app=name", but this convention is purely cosmetic, and would introduce a change of labels to all existing kube.libsonnet users (triggering exactly the issue this PR is attempting to avoid!).
If I had a time machine, I would probably choose to use app=foo instead, although even then I would question it tbh.  In the low-level automated context we're talking about in kube.libsonnet we don't know what the "app" in question is - all we know is the name given to the k8s object (eg: we know "elasticsearch-master-node", not "elasticsearch").  The introduction of Application objects confuses this even further if the app labels don't match the same grouping hierarchy.
See also below.
2. Introduce a new _AppObject() "constructor"
(2) is unnecessary, if we drop (1).
As a general pattern however, creating new constructors doesn't compose well (what if we had several special tweaks we wanted to apply to this Object?), and "mixins" tend to work better.
ie:  Consider the following.  You can chain multiple "mixins" to add additional modifiers:
{
  applabels:: {
    local name = self.metadata.name,
    metadata+: {labels: {app: name}},
  },

  Deployment(name):: $._Object(name) + $.applabels + {
    // as normal ...
  },
}

3. Change selector to a subset of the labels (just {app: value})
Separating the top-level Deployment labels from the low-level Pod label/selector is probably a good idea, for the reasons you raise.  I liked the way the selector was "exported" from the object in question in the previous version (by just assuming that it was all metadata.labels), rather than having everything else "know" that the appropriate choice is {app=target.labels.app}.  It would be good to get some of that back - probably by adding a hidden selector field to all objects, and then using that everywhere we need a cross-object selector...
Let me think about that for a bit :stuck_out_tongue:
Note for (1)
If a local site was using an "app=foo" convention, and would like to continue using this convention with kube.libsonnet then they can easily do this with a local customisation of kube.libsonnet that has something like this PR in a separate "overlay" file:
// Call this "myorganisation.libsonnet" or similar
local kube = import "kube.libsonnet";
local applabels = {
  metadata+: {labels: {app: $.metadata.name}},
};

{
  Deployment(name):: kube.Deployment(name) + applabels {
    // .. and any other local policies
  },
  StatefulSet(name):: kube.Deployment(name) + applabels,
  // and so on, for other modified types.
}

.. and then of course use it as usual:
// Example usage
local myorg = import "myorganisation.libsonnet";
{
  example: myorg.Deployment(name) {
    spec+: {
      // ... as usual
    },
  },
}

Thank you for the review and ideas how to implement our fixes in a better way. I'll close the pull request as the issue needs to be fixed in a different way.


Fix install.sh sudo failure handling

What this PR does / why we need it

Error handling was added to exit if we are unable to sudo when trying to
install the tanzu binary into a system path. This had an error though
where the "exit 1" would be executed outside of the "or", so whether
sudo worked or not it would still exit with a failure.
Details for the Release Notes (PLEASE PROVIDE)

NONE

Describe testing done for PR

Ran make release. Then cd release/tce-linux-amd64-v0.11.0-dev.1/ to get the full release bundle.
From there, ran install.sh and the first time through entered the wrong sudo password three times
to get it to fail. On sudo failure, it correctly echoed the error message and exited with a 1 return code.
Then ran install.sh again, this time giving the correct password. Install proceeded and no errors.
Special notes for your reviewer

@dvonthenen @jpmcb - much more thorough testing this time. Should work as expected.


Feature/forgot password

Description
Implements forgot password feature.
Trello card: https://trello.com/c/70FssAp8/113-endpoints-for-forgot-password
Type of change

[x] New feature (non-breaking change which adds functionality)

Change Status

[x] Complete, tested, ready to review and merge

How Has This Been Tested?

[x] Manually tested with simulator (iPhone 11)

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[x] There are no merge conflicts

 Endpoints for Forgot Password

@alvare52  Changes made in last commit. Please review.


Update chart index file with action

Address the comment https://github.com/open-cluster-management-io/helm-charts/pull/11#discussion_r1623888512

/hold


Unify radiation tendency for EDMF and EDMFX

Currently, we can not run edmfx with --rad DYCOMS_RF01 as the radiation tendency is not unified.

@trontrytel @dennisYatunin Could any of you take this on?

Closed by #1936


No JSON object could be decoded

I used http://jsonlint.com to correct format of config.json and it still not work.
Traceback (most recent call last):
File "pokecli.py", line 220, in
main()
File "pokecli.py", line 197, in main
config = init_config()
File "pokecli.py", line 64, in init_config
load.update(json.load(data))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 291, in load
**kw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

And here is my json
{
"auth_service": "google",
"username"<EMAIL_ADDRESS>",
"password": “pw”,
"location": "59.744329, 10.204808",
"gmapkey": "key",
"max_steps": 5,
"mode": "all",
"walk": 4.16,
"debug": false,
"test": false,
"initial_transfer": 0,
"location_cache": true,
"distance_unit": "km",
"item_filter": "101,102,103,104",
"evolve_all": "NONE"
}

There is a space behind your user-name and the ". furthermore it looks like there some false quotes.
like “ instead of "
check the file with http://jsonlint.com/ like @Nihisil mentioned there:
https://github.com/PokemonGoF/PokemonGo-Bot/issues/738

like @FlorianKuehne mentioned, it's definitly the wrong type of quotes used. Check your configured keyboard layout, or just copy paste them from the other quotes.

I corrected all quote and the problem fixed now!
Thank you bro!


@loadable/component failed to asynchronously load component

🐛 Bug Report
@loadable/component raise the following exception when loading chunk.
The chunkName argument appears buggy as [undefined] while the url to the resource js is accessible.
loadable-components: failed to asynchronously load component [object Object]
logger: console
arguments: ["loadable-components: failed to asynchronously load component",{"chunkName":"[undefined]","error":"Loading chunk 31 failed.\n(error: https://static.hipacloud.com/web/0.2.610-8decb5a5/_next/static/chunks/XXXXHIDDENXXXXXX.js)","fileName":"[undefined]"}]

To Reproduce
GIVEN a user stays on page A of package version 0.0.1
AND the new package version 0.0.2 is deployed
AND the old package version 0.0.1 still has all static resources accessible
WHEN he triggers a component to be mounted, which was imported via loadable()
AND the chunk for this component is new in package version 0.0.2
THEN the page displays error info due to a ChunkLoadError is raised
AND the sentry log points out the exception comes from loadable
Expected behavior
No matter when the dynamically component load happens, it should load a chunk correctly if its url is accessible.
Link to repl or repo (highly encouraged)
Using nextjs 10.0.3 with @loadable/component.
Run npx envinfo --system --binaries --npmPackages @loadable/component,@loadable/server,@loadable/webpack-plugin,@loadable/babel-plugin --markdown --clipboard
Paste the results here:
## System:
 - OS: macOS 10.15.6
 - CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
 - Memory: 1.91 GB / 16.00 GB
 - Shell: 3.2.57 - /bin/bash
## Binaries:
 - Node: 14.15.1 - ~/.nvm/versions/node/v14.15.1/bin/node
 - Yarn: 1.21.1 - /usr/local/bin/yarn
 - npm: 6.14.9 - ~/Works/hipa/repositories/hipa-web/node_modules/.bin/npm

Loadable is not loading anything itself, only command webpack to do it.
So the question I need you to answer is about https://static.hipacloud.com/web/0.2.610-8decb5a5/_next/static/chunks/XXXXHIDDENXXXXXX.js - does that url exist and correct, or its "buggy"?
"undefined" is not an indication of "buggy", but an indication of missing babel plugin, which is needed only for SSR to work. In case of nextjs this is something you "dont want" - you don't want to use SSR variant of loadable-components for nextjs as it provides its own version - please consider using "nextjs" way of code splitting.

Loadable is not loading anything itself, only command webpack to do it.
So the question I need you to answer is about https://static.hipacloud.com/web/0.2.610-8decb5a5/_next/static/chunks/XXXXHIDDENXXXXXX.js - does that url exist and correct, or its "buggy"?
"undefined" is not an indication of "buggy", but an indication of missing babel plugin, which is needed only for SSR to work. In case of nextjs this is something you "dont want" - you don't want to use SSR variant of loadable-components for nextjs as it provides its own version - please consider using "nextjs" way of code splitting.

@theKashey thank you for getting back to me.  I confirmed the original url was accessible while I had hidden part of it with string "HIDDEN" in this thread.
Just as you suggested, I replaced loadable with next/dynamic 4 days ago, which is recommended by the nextjs. I will get back if the ChunkLoadError occurs again.

@theKashey thank you for getting back to me.  I confirmed the original url was accessible while I had hidden part of it with string "HIDDEN" in this thread.
Just as you suggested, I replaced loadable with next/dynamic 4 days ago, which is recommended by the nextjs. I will get back if the ChunkLoadError occurs again.

For the past 12 days, crossing several releases, ChunkLoadError does not happen so far.

For the past 12 days, crossing several releases, ChunkLoadError does not happen so far.

My chunk is not getting loaded, what is the reason, while other chunk got loaded on same component on same page, can you please help me on that ?
all are loaded success fully except AppPortals
My code as follows
const ToastContainer = loadable(() =>
import("@bbb-app/core-ui/toast-message/ToastMessage")
);
const AppPortals = loadable(() => import("@bbb-app/app-portals/AppPortals")); --->  this is not getting loaded.
const Tealium = loadable(() => import("@bbb-app/tealium/containers/Tealium"));
const GlobalNotification = loadable(() =>
import("@bbb-app/global-notification/container")
);
const StickyButtons = loadable(() =>
import("../common/containers/sticky-buttons/StickyButtons")
);


feat(rbac): display administration to authorized users

Resolves:
https://github.com/janus-idp/backstage-plugins/issues/842
Test setup:
app-config.yaml
auth:
  providers:
    github:
      development:
        clientId: $GITHUB_CLIENT_ID
        clientSecret: $GITHUB_CLIENT_SECRET

permission:
  enabled: true
  rbac:
    admin:
      users:
        - name: user:default/<your-github-id>
        - name: group:default/admins

DynamicRoot.tsx
Add SignInPage component

    if (!app.current) {
      app.current = createApp({
        apis,
        components: {
          SignInPage: props => (
            <SignInPage
              {...props}
              auto
              provider={{
                id: 'github-auth-provider',
                title: 'GitHub',
                message: 'Sign in using GitHub',
                apiRef: githubAuthApiRef,
              }}
            />
          ),
        },

auth.ts
      github: providers.github.create({
        signIn: {
          resolver({ result }, ctx) {
            const userRef = `user:default/${result.fullProfile.username}`;
            return ctx.issueToken({
              claims: {
                sub: userRef, // The user's own identity
                ent: [userRef], // A list of identities that the user claims ownership through
              },
            });
          },
          // resolver: providers.github.resolvers.usernameMatchingUserEntityName(),
        },
      }),

To load the Administration sidebar item dynamically
Test Setup
Add the @janus-idp/backstage-plugin-rbac package in the packages/app/package.json
Keep the app-config changes from the above set-up
app-config.yaml
dynamicPlugins:
  frontend:
    janus.dynamic-frontend-plugin:
      dynamicRoutes:
        …
        - path: '/rbac'
          importName: 'RbacPage'
          module: 'Administration'

Root.tsx
	export const Root = ({ children }: PropsWithChildren<{}>) => (
	…
        <ScalprumComponent
        scope="janus.dynamic-frontend-plugin"
        module="Administration"
        importName="Administration"
      />
	…

dynamic-modules-registry.json
      {
   	"module": "Administration",
   	 "scope": "janus.dynamic-frontend-plugin",
    	"manifestLocation": "http://localhost:8004/plugin-manifest.json"
  	}

dynamic-frontend-plugin/package.json
"scalprum": {
    …
    "exposedModules": {
       …
      "Administration": "./src/remotes/Administration"
    }
  }

Create Administration/index.ts under dynamic-frontend-plugin/src/remotes
export * from '@janus-idp/backstage-plugin-rbac';
GIF:
https://github.com/janus-idp/backstage-plugins/assets/22490998/69b8c23a-2f74-4db0-8839-222748f40f66
Updated the Administration Nav icon

@debsmita1 and frontend dynamic plugin support is there now so pls ensure we test that flow as well and capture it as part of the PR description

@debsmita1 looks good to me, thank you!

@divyanshiGupta @invincibleJai I have removed the changes from the app folder. Added ReadMe. And documented the steps to load the Administration sidebar item dynamically in the ReadMe. PTAL

Some existing tests are failing, working on them

Build failed. Corresponding fix is here https://github.com/janus-idp/backstage-plugins/pull/929


Assertion Error on git clone

I've been getting this stack trace after I push and then try to clone it to a different directory.
Traceback (most recent call last):
  File "/home/fazal/.bin/git-remote-dropbox", line 684, in <module>
    main()
  File "/home/fazal/.bin/git-remote-dropbox", line 672, in main
    helper.run()
  File "/home/fazal/.bin/git-remote-dropbox", line 344, in run
    self._do_list(line)
  File "/home/fazal/.bin/git-remote-dropbox", line 369, in _do_list
    refs = self._get_refs(for_push=for_push)
  File "/home/fazal/.bin/git-remote-dropbox", line 621, in _get_refs
    name = self._ref_name_from_path(path)
  File "/home/fazal/.bin/git-remote-dropbox", line 454, in _ref_name_from_path
    assert path.startswith(prefix)
AssertionError

Did a little digging myself. It seems helper users path_lower in _get_refs and I was using the path dropbox::/Repos/test and it's expecting dropbox::/repos/test.
git clone "dropbox::/repos/test works.

Oh, interesting! Yeah, Dropbox is case-insensitive, so I've just been using lower-case URLs.
Okay, opened #18 to take care of this.


docs: "WebSocket Part 1" blog post to refer to Studio

Description
Just a small update to refer to what nowadays we have as AsyncAPI Studio, instead of the old Playground.

@alequetzalli Thanks Alejandra for the quick feedback! But I had to update it (see the description of the 2nd change Fix link).


Some course titles not displaying correctly

This has to do with some courses having title=nil. Should we add code to handle this, or should we not care and assume that we'll fix up the database?

So for the search results page, I think this is soon going to turn into more of a course instance search page, so will only show courses offered in a specific semester. If you're looking for just a general course, that's what the autcompletion course select field is for. So this will solve the problems of course titles in the search results.
However, these results will still show up in the autocompletion, just with a blank title, which is fine, but it leaves us with the problem that if people select the course, it gives them errors. I'll submit a PR soon that fixes this and displays some generic message like "This course appears to not have any actual course offerings, sorry about that". However, once we populate old courses via the SIS scraper, this will probably be a very unlikely occurrence.

Temporarily patches #84

I think this is the same bug: clicking a course titled "Title not found" results in a 500 error.  Ex: http://advise.case.edu/courses/EECS304

This is actually related to the pull request we just merged in. I'll write a fix right now.
Where are we on getting the historical data in the database so that we don't need to care about this anymore?

I'm not seeing this happening anymore. Feel free to re-open this issue if it happens again.

Is this the issue for courses displaying "Title not found"? Or is there a separate issue for that? Cause that's still happening.

This is the one. Can you attach a screenshot or at least the offending course ID?

If you want to try to tackle this, feel free.

I know what this is and it is dependent on getting the historical data in which I need to fix the SIS Downloader first. We still need code to upload those files after we have them but that should be the easiest part of all

Your screenshot:

Let me know if I can help with this

This appears to be more of a one-time infrastructure-y thing we need to do on our end, so I don't think there's much you would be able to help with, but I will definitely let you know.


Not working at all in 3114

Not sure if I'm missing a component, but this is not working at all for me. Could it be a problem with the latest version of Sublime? There seem to also be some problems rendering jade blocks in Markdown Extended too; though scss still works fine there

Did you set the file type correctly? It's working properly for me in OS X build 3114.

I created a new user, and a minimal setup of sublime with this plugin, the tomorrow color schemes, and the jade and pug plugins from davidrios; I get the following in 3114

P.S. also tried some of the default color schemes; still no highlighting inside the <template> blocks

@lunelson try using " instead of ' around jade
<template lang="jade"> seems working for me.
not familiar with sublime plugin but I think single quote mark ' is common in html-like files
@yyx990803 how about adding support for single quote mark '

@arbipher thanks that seems to do the trick. I should have known to try that 😏


Unused lastmod.html

Template generation left lastmod.html partial, which is unused in the tailwind theme. It can be misleading because LastMod is covered extensively in Hugo's documentation. I suggest removing this partial and optionally adding LastModDisplay option to the configuration to toggle showing the last modification date in posts' headers, similar to "Reading Time" info.
I can prepare PR with the LastMod widget if you find it a beneficial change.

I must have forgotten to delete this file. I'll take a look.


Secret Values special characters are interpreted

Hi,
Running v0.7.0, and noticed that secret values are getting interpreted rather than passed as is.
Example:
"value": "password<"

is JSON rendered as:
"value": "password\u003c"

When using base as format, if newline (\n) is in value, this is also interpreted resulting in the below incorrect syntax
          │   ├── example-keystore-jks_base64=/u3+7QQAAAYtIxCqRAAAFATCCBP0wDgYKKwAAAAIAAAABAAAAAQABMYBBAEqAhEBAQUABIIE6Z9q
            │   │   w2lvG+76o2JhexbzP2qA8EnUwzRorbDkbgRvTrsp3dMFBEnYqFurMXJMVnY8qffoD4OC19OvMkFg
            │   │   /am4W0GdZV05AQh+OTmEgDPfyVNKZCosmGl5Zr6zYuxSR1a9jVOjIPh+KIgDFy/Q021S7R+1fHrR
            │   │   6eKX5JDtl3TWDSphgInCsNN+h8LVk+nC6DtJoAQSqC6+t+b1SRzvVOWk+EmnRHTLULmjG7zTCNx0

Oh interesting! Will have a look, thanks for reporting.

Special characters like < and > are now preserved.
I cant reproduce the \n breaking the base format output. Can you provide me an example secret value that leads to the output you have provided?

I'm out right now so can't provide the actual value but it was something on
the lines of:
"value":
"dhaisjckakqhdjckwk2940wjz\nkskfifoak48skfkalqorjc\nsksiwi3ifockqk1ofocowksfi\nfjskwi3if92oskvkzlap1p5010"
The secret itself is stored in a nested sub directory structure.
I expect that the value is rendered as a single line.
I will provide the actual secret later.
Thanks,
Nic
On Sat, 11 May 2024, 12:28 FalcoSuessgott, @.***> wrote:

Special characters like < and > are now preserved.
I cant reproduce the \n breaking the base format output. Can you provide
me an example secret value that leads to the output you have provided?
—
Reply to this email directly, view it on GitHub
https://github.com/FalcoSuessgott/vkv/issues/255#issuecomment-2105667841,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGCKD56DF6J2AFC5DAACEXLZBXXEXAVCNFSM6AAAAABHRUG5USVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGY3DOOBUGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

So value is:
"example-keystore-jks_base64": "/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B"

This is rendered like so:
            │   ├── example-keystore-jks_base64=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q
            │   │   w2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg
            │   │   /am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD
            │   │   6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0
            │   │   ukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/
            │   │   zn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B

Expected to be rendered as:
            │   ├── example-keystore-jks_base64=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B

Note: the above example is not a valid base64. Cant provide the actual base64 value for security reasons.

Thanks for the example. I dont really think I can avoid that .. I think the terminal will simply break the line which will lead to the underlaying library (gotree) to add another | in the front. But I will play around and see what I can do and get back to you

Bonjour @nichcuta!
I think I found a nice solution:
with vkv v0.7.0 secrets across multiple lines breakes the base output (Githubs Markdown Codeblocks do actually not break the lines like a terminal does ..)
> vkv export -p secret --show-values
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=password
├── demo [v=1]
│   └── foo=bar
└── sub
    ├── demo [v=1]
    │   ├── demo=hello world
    │   ├── password=s3cre5
    │   └── user=admin
    └── sub2
        └── demo [v=3] [admin=false key=value]
            ├── admin=key
            ├── foo=bar
            ├── key=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B
            ├── password=password
            └── user=user

with the changes from #257 secret values are now correctly indented by interpreting any \n:
> go run main.go export -p secret --show-values
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=password
│
├── demo [v=1]
│   └── foo=bar
│
└── sub
    ├── demo [v=1]
    │   ├── demo=hello world
    │   ├── password=s3cre5
    │   └── user=admin
    │
    └── sub2
        └── demo [v=4] [admin=false key=value]
            ├── admin=key
            ├── foo=bar
            ├── key=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q
            │   w2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg
            │   /am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD
            │   6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0
            │   ukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/
            │   zn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B
            ├── password=password
            └── user=user

would that work for you?

Good morning @FalcoSuessgott ,
> vkv export -p secret --show-values
secret/ [desc=key/value secret storage] [type=kv2]
├── admin [v=1] [key=value]
│   └── sub=password
├── demo [v=1]
│   └── foo=bar
└── sub
    ├── demo [v=1]
    │   ├── demo=hello world
    │   ├── password=s3cre5
    │   └── user=admin
    └── sub2
        └── demo [v=3] [admin=false key=value]
            ├── admin=key
            ├── foo=bar
            ├── key=/u3+EqAhEBAQUABII7QAAAAIAAAABAAAAAQABMQAAAYtIxCqRAAAFATCCBP0wDgYKKwYBBAE6Z9q\nw2lvbDkbgRvTrsp3dMFBEnYqFurMXJMVnG+76o2JhexbzP2qA8EnUwzRorY8qffoD4OC19OvMkFg\n/am4W0GdZVPfyVNKZCosmGl5Zr6zYuxSR1a9j1fHrR05AQh+OTVOjIPh+KIgDFy/Q021S7R+mEgD\n6eKX5JDtl3TWDSphgInCsN1SRzvVOWk+EmnRHTLULmjG7zTCN+h8LVk+nC6DtJoAQSqC6+t+bNx0\nukVSh0CPv7tjX5z6XeMJW+PNQUR3pe/w4QbH3MAzEQhkCg1l93CWxoooRrgXmIdSf0qq4joA2kt/\nzn65p359dOEx+owXCMBh/JRZ3Gf253m+jyLTYp0kPHzJGwXwD+6GVs6dum3B+lZuc8v45VvXTE4B
            ├── password=password
            └── user=user

The key above is the current output. Perhaps zsh (the terminal i use) is causing this issue for me?
I think vkv should never interpret the '\n' so the secret value is preserved
Thanks,
Nic

Hi,
you're right, interpreting \n is probably not a good idea. I decided to stick to the behavior of the official vault CLI:

Which does not interpret special character. This equals now with vkv export:

As you can see in vault and vkv output, the terminal & shell simply break long lines at whatever width the current terminal session has ( I use zsh + alacritty). I think this is fine and actually inevitable.

Thats the behaviour i expected. Looks good to me :D
Thanks for fixing


Running spotlessApply removes comments from XML

Steps to repro

Open any XML file and add a comment to it. Example https://github.com/android/nowinandroid/pull/1103/commits/28bb3f1b2fe56e0842eed15aa1f43243afdc634f
Run ./gradlew spotlessApply --init-script gradle/init.gradle.kts --no-configuration-cache

Expected result
Comment in XML file is not removed
Actual result
Comment in XML file is removed

Root cause here is this config from [init.gradle.kts](https://github.com/android/nowinandroid/blob/main/gradle/init.gradle.kts#L51)
format("xml") {
                target("**/*.xml")
                targetExclude("**/build/**/*.xml")
                // Look for the first XML tag that isn't a comment (<!--) or the xml declaration (<?xml)
                licenseHeaderFile(rootProject.file("spotless/copyright.xml"), "(<[^!?])")
            }

The regular expression (<[^!?]) matches the first element then replaces everything before it with the license header, deleting everything that was previously there.
Placing comments after the first XML element is the workaround. Affected PR: #1103


set rpc port in IWearRemapper optional

In order to be able to use Wearable data that are streamed trough yarpdataplayer, this PR introduces an optional parameter useRPC that allows to the need of an rpc port from wearable data.

Same comment of the separator here:
https://github.com/robotology-playground/wearables/blob/3dd2c7bb152864843c062541bc61b500ef0a20e2/devices/IWearRemapper/src/IWearRemapper.cpp#L851

Same comment of the separator here:
wearables/devices/IWearRemapper/src/IWearRemapper.cpp
Line 851 in 3dd2c7b
return WrapperName + "::";

fixed https://github.com/robotology-playground/wearables/pull/24/commits/ef5c8460c53bfb452d36ac4ac9f38030cff6721c

@diegoferigo did you have time for the last review?

@DanielePucci I already approved this PR. Ready to merge.


Question:where do theses fresh phishing domains come from?

Thank you for the tool!
Could you please specify where does "fresh Phishing DB" come from? How is it updated? How is it decided whether the domain is phishing?

Maybe with something like passivedns https://github.com/PolishFiltersTeam/KAD/issues/1241#issuecomment-645287499

Hey there, I no a little about your Q, so let me reply with what I know.
@mitchellkrogza have a number of servers with traps, these traps are then run true a several 1000's line long regex based bash scripts, so it a couple of years ago..... trust me, it's complexes and takes several hours to decode all the regex.
Hope it give you some answers.

Hey there, I no a little about your Q, so let me reply with what I know.
@mitchellkrogza have a number of servers with traps, these traps are then run true a several 1000's line long regex based bash scripts, so it a couple of years ago..... trust me, it's complexes and takes several hours to decode all the regex.
Hope it give you some answers.

Correct. Closing.


Error thrown in Js Sdk:  e.data.indexOf is not a function

Category

[ ] Question
[ ] Documentation issue
[x] Bug

Expected or Desired Behavior
No error should be thrown
Observed Behavior
As of v7.2 of the OneDrive file picker Sdk for JS, an error is thrown when invoking the open method:
OneDrive.js:4 Uncaught TypeError: e.data.indexOf is not a function
    at OneDrive.js:4

Steps to Reproduce
Simply invoke the open method
This was mentioned already in #863

There was also another issue of e.data.indexOf is not a function. That was fixed via adding an additional e.data.indexOf null check.

Any updates on this issue?
I have the same bug

I have the same issue. Although it looks like it doesn't break anything, just an annoying error message in a console.

+1

Having the same error trying to use Filepicker in React

+1

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.


More documentation for sweep bits

There currently isn't much explaining what the sweep bit arguments are. From the docstrings that do exist it seems that it's a way to use classical data as the control for two qubit gates but I'm not sure if this is true.
Could there be more documentation about what sweep bits are and where they are valid and invalid please? For example it seems it's possible for the target of a two qubit gate to be a sweep bit.
I'm happy to contribute if I can but I think my assumptions need checking first!

I was originally reading through the Python API where I first came across them and found:

sweep_bits_filepath: Defaults to None. A file containing sweep data, or None. When specified, sweep data (used for sweep[k] controls in the circuit, which can vary from shot to shot) will be read from the given file. When not specified, all sweep bits default to False and no sweep-controlled operations occur

Which I didn't understand until finding the file format doc. I also just stumbled upon the command line usage document which also adds some information.
I'm happy to add links to these other documents in the API if that helps others. Apologies for prematurely raising a ticket.


Tests need to be changed to adapte backend modifications

Bug Report Format

1.Bug URL:
Build server application error
2.Date Created:
Jan 30, 2022 with the PR #25
3.Defect Description:
The basis of service CRUD operations is changed to be service_code because the services with the same basic info can have different durations and prices, for example, when the user create a service, it can enter multiple durations and prices combinations and the backend should create different services based on these durations and prices with the same values for all other attributes, the same condition applies to UPDATE and DELETE operation. the backend has been modified, but the JEST test should also be modified to adapte these changes
4.Resolution:
Description of the defect’s resolution.
5.Resolution Date:
Date on which the defect is to be resolved (or is resolved depending upon its status).
6.Screenshot Bug:

Can we justify the removal of finding items by id? I understand that service codes can be used to retrieve services, but do we just not allow retrieving services by id? Maybe we could use service/code/<service code>/ for that

Can we justify the removal of finding items by id? I understand that service codes can be used to retrieve services, but do we just not allow retrieving services by id? Maybe we could use service/code/<service code>/ for that

yes, based on the mockup design, when creating services, we will create a group of services with the samebasic info such as service_code, name, etc., but different durations with corresponding prices, this condition also applies to other CRUD operations such as update, and deletion.


Audit code for U256 overflows

This would be done once the code is in good shape in terms with VM operation and chain sync.

Is the goal be to find places that U256 could overflow and either replace with, as appropriate, one of:

U256.wrapping_add; or
Conversion to Uint?

I opened it mostly focussing on conversion to Uint, but it would also be good to cross check that the wrapping operations are in place wherever appropriate.


[SPARK-6307][Core] Coalesce RDDs before RDD.cartesian

JIRA: https://issues.apache.org/jira/browse/SPARK-6307 and https://issues.apache.org/jira/browse/SPARK-6922
Alternative approach to this issue. Ref. #5572.
This PR tries to coalesce RDDs before RDD.cartesian. It also implements the idea 1 discussed in #5572. So in cartesian we will fetch each partition in RDD2 once for each partition in RDD1.
For performance test, using the same example codes in #5572, it costs about 4s.
cc @tbertelsen, @squito, @cloud-fan.

Merged build triggered.

Merged build started.

Test build #33656 has started for   PR 6454 at commit 72e199e.

Test build #33656 has finished for   PR 6454 at commit 72e199e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33656/
Test PASSed.

jenkins, test this please

Merged build triggered.

jenkins, test this please

Merged build triggered.

Merged build started.

Test build #33657 has started for   PR 6454 at commit 72e199e.

Test build #33657 has finished for   PR 6454 at commit 72e199e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

@viirya Sadly I don't have access to our compute cluster for the next couple of weeks so I can try it out right now.

Guys, any update on this one?

Merged build triggered.

Close this PR first as I don't know if it is needed now.


OPA filter smoke tests

This is a tracking issue for discussing OPA filter testing. The idea is that in agent we want to be able to test OPA filter https://github.com/hypertrace/javaagent/tree/main/filter-custom-opa against running https://github.com/open-policy-agent/opa. The filter gets data from /v1/data and runs evaluation on every request.
I would like to package https://github.com/open-policy-agent/opa into a docker container and make data.json and /v1/policies/remote-bundle/traceable/http/request/policy.rego configurable. This container could be used in smoke tests in all Hypertrace agents. There is already a docker container https://hub.docker.com/r/openpolicyagent/opa - we should have a look if it could be used.
In the meantime the OPA filter can be tested by:

port-forwarding OPA agent from traceableai namespace k port-forward service/opa 8181:8181 -n traceableai
get the data from OPA agent curl localhost:8181/v1/data and pick some IP address from denylist
run agent on a test/demo app and do curl localhost:8080 -H "X-Forwarded-For: <IP from deny list> - the request should be blocked with 403

cc) @mohit-a21 @jcchavezs @davexroth

Removing OPA is moved to a private repo


DJI Air 2S

New Firmware has added RID support through WiFi Beacon.
I have not been able to receive one but the drone says it's working.

Confirmed receipt using OpenDroneID-OSM.


Add fillCircle and strokeCircle taking a Circle for Context

In paths.nim there seems to be also no circle helpers like for Rect, I guess the ellipse helpers are enough but want me to also add paths.nim helpers?

If you don't mind, yeah sure lets get the Circle version in paths.nim added too. That'd be nice and consistent.

Thank you for the PR!


Be able to update or retrieve a single record including non-unique fields in the "where" conditions.

Problem
I am unable to utilize non-unique fields in single update queries. To be clear, I want to still use the unique fields, I just want to filter it down even more.
Suggested solution
Here are my models
model Person {
  id     String   @id @db.Uuid
  client Client[]
}

model Client {
  id       String @id @db.Uuid
  name     String
  personId String @db.Uuid
  person   Person @relation(fields: [personId], references: [id])
}

I have "people" who have "clients" attached to them. Only the person who owns the client can update the name. So when an API call comes in, I know who the current person is and what client they're trying to update. So the update query I would like to use is this:
const client = await prisma.client.update({
  data: { name: input.name },
  where: {
    id: input.clientId,
    personId: input.personId
  }
})

but it only allows fields which are marked as @unique or @id
Alternatives
One alternative is to do a .findUnique({ where: { id: input.clientId } }) and then check if the personId of the client is the same as the one passed in. This however creates two database calls where only one is needed.
Another alternative is to do a .updateMany({ where: { id: input.clientId, personId: input.personId } }) but I don't get any of the clients back. If I got the clients back in the query and if there was a limit I could pass in to limit it to one, I would feel better about this so it wouldn't have to do any unneeded scans of the rows, but it still feels less idiomatic than updating the .update() command to allow for non-unique fields.

The same goes for .delete({ where: {} }). I can only add unique fields to the where conditions.

You should use updateMany and deleteMany in this case - as these queries can potentially affect multiple entries.

But if I'm using a unique field along with the other non-unique fields, it won't potentially affect multiple entries, right? Like I mentioned, I would use those, except they don't return the updated records, just an updated count.

To be clear, my intention is to only update the singular record with the primary key field, I just want to add an additional condition to the field to make sure someone who shouldn't have permission to update the "client" won't be able to.

Couldn't you then not just add a unique index across these two fields and use that as the identifier?

I guess that would work, it just seems superfluous to add a unique index that includes a field that is already considered unique, especially if it's a workaround for a missing feature. Adding indexes to a database isn't "free" in that adding an indexes increases disk usage. It's probably not significant enough for my current use case currently, but I could see a future use case where adding unique indexes to every combination of fields I need to conditionally update/delete would be prohibitive.

I've stumbled across this issue a lot as well.
I see no reason why you shouldn't be able to optionally include additional conditions in the where clause.
So long as you provide a unique identifier that is.
There's some workarounds but none of them really make me too happy tbh.

What is the use case for adding another field, if you already have a unique identifier that you know uniquely identifies 1 specific row?

@janpio I use the additional filter as a way of further qualifying the operation.
For example, an API request comes in to update a Todo item. I have the unique id of the todo from the parsed URL, and I have the current user from the auth token. I want to send a single update command to the database where the id is the given todo id and the owner of the todo is the current user’s id. This way, if someone tries to update a todo item they are not owner of, the update returns null.

@janpio Because you might not want the query to return that row if it doesn't fit the additional criteria.

I have the same problem. Two use cases that come to mind: maybe there is some kind of "Soft-Delete-Strategy" in place or an update is only "allowed" as long as a version field hasn't changed.
The problem with the many methods is, they do not return the affected "entities". So you would have to do additional queries.

I have the same problem. Two use cases that come to mind: maybe there is some kind of "Soft-Delete-Strategy" in place or an update is only "allowed" as long as a version field hasn't changed.

Exactly! Especially since that is the recommended alternative to long running transactions.

The problem with the many methods is, they do not return the affected "entities". So you would have to do additional queries.
From a pragmatical standpoint that might be "the only" problem, but conceptually if you're trying to update a single record then you shouldn't use the updateMany method.

It would be useful to use this feature for Optimistic Concurrency Control https://github.com/prisma/prisma/issues/4988
const id = 'user_id'
const user = prisma.user.findUnique({ where: { id } })   // { id: 'user_id', version: 1, ... }

try {
  // Throws if not found
  prisma.user.update({
    where: {
      id: user.id,
      version: user.version,
    }
    data: {
      version: {
        increment: 1
      },
      ...
    },
  )
} catch (e) {
  // Consider redoing, or error out.
}

Another downside with this workaround: updateMany doesn't support nested upsert.

Did we ever find a solution to this?
      const todo = await prisma.todo.update({
          where: {
            id: query.id,
            userId: session.userId
          },
          data
        })

Would be nice to leverage this to confirm user owns the TODO object.

I've been looking for a way to upsert with multiple where conditions, I guess it is not supported yet ?
await db.user.upsert({
  where: { uuid: params_uuid, email: params_email },
  create: {
    email: email,
    uuid: uuid,
    products: {
      create: productsMap,
    },
  },
  update: {
    products: {
      create: productsMap,
    },
  },
})

I too require this function. My use case is a user can request a relationship with another user (e.g. trying to "friend" another user). When the second user tries to accept the request, I want to make sure there's a pre-existing record in the pivot table with the status currently at "REQUESTED" instead of "BLOCKED" or some other possible status.

+1

Definitely required. For the workaround we are first querying the record and see if that meet the conditions and then calling update function with the matched id. So, why not do that in single update query ?

I have a unique constraint on several columns
@@unique([userId, portfolioId, privilege])
I also tested
@@id([userId, portfolioId, privilege])
However, the Prisma client's types will not allow me to run a .delete() query using these three fields, so I have to resort to raw queries to actually delete specific rows from this table..

Any updates on this?

Agree with everyone else here - is very odd that this behaviour isn't supported.
It also appears updateMany actually just does an update on ID, but first runs a select statement to get the ids.
    await prisma.assignment.updateMany({
      where: {
        project_id: project_id,
      },
      data: {
        minutes_per_day: minutes,
      },
    })

Generates
prisma:query BEGIN
prisma:query SELECT "public"."assignments"."id" FROM "public"."assignments" WHERE "public"."assignments"."project_id" = $1
prisma:query UPDATE "public"."assignments" SET "minutes_per_day" = $1 WHERE "public"."assignments"."id" IN ($2,$3,$4,$5,$6,$7)
prisma:query COMMIT

Where handwritten sql would be
UPDATE "public"."assignments" SET "minutes_per_day" = $1 WHERE "public"."assignments"."project_id" = ($2)

Is there a reason for this? Is it to normalise the queries across different database types? Or is it related to the fact Prisma will only update based on unique's?

This is an extremely limiting and annoying issue that doesn't let me check "lastUpdated" field while updating a record.
I can use "updateMany" but then, in that query, I can't do nested creates/updates.
Please, please, change this.

it's limiting but also it's powerful. If updates are done by unique ids only, it opens doors to many automation around it. E.g. DynamoDB has limitation at database layer to be able to update/delete rows only if you give exact unique identifier to the row, no mass updates/removes and it's the reason why they are able to do something like seamless cache layer w/o any api modifications. You can also easily pipe updates/deletes to some queue/stream and have exact flow of events and ability to re-create state of data. etc. etc. update/delete should update delete only and only single item by it's unique identifier. in sql you have option to mass delete/update, that's why there are updateMany and deleteMany. If you don't like result of *Many functions, you can talk about having options for that call, but i would argue to keep update/delete limited to single item update/delete only no matter what.

@liesislukas You're right, it is super powerful, but the argument that I'm making isn't that the update call should be able to update many documents. I absolutely agree it should only be able to update single documents and that a unique/id field should be required to be passed in. The argument is that additional fields should be able to be passed in along-side a unique field.
Here's a real example that has severe security implications:
In the OAuth2.0 spec in the authorization flow where a user delegates permission to a 3rd party application, when the user grants access, the authorization server generates a one-time use auth code to give to the 3rd party application. The 3rd party application will exchange that auth code in addition to its credentials to get an access token. The prisma model for the auth codes might look something like this:
model AuthorizationCode {
  code                String            @id
  user                User              @relation(fields: [userId], references: [id])
  userId              String
  client              ClientApplication @relation(fields: [clientApplicationId], references: [id])
  clientApplicationId String
  createdAt           DateTime
  expiresAt           DateTime
  scope               String
  consumedAt          DateTime?
}

When the 3rd party application makes the call with the auth code, we want to in the same database call fetch the auth code and set the consumedAt field to the current date. That query ideally would look something like this:
const now = new Date()
const authCode = await prisma.authorizationCode.update({
  where: {
    authCode: req.body.code, // This is the unique field and should be required
    clientApplicationId: req.body.client_id, // not unique, but helps make sure clients don't consume other auth codes
    expiresAt: { gt: now }, // Make sure we don't touch an expired auth code
    consumedAt: null // Make sure we don't touch an auth code that has already been consumed
  },
  data: {
    consumedAt: now // Update consumedAt so it can't be consumed again
  }
})
if (!authCode) return res.status(400).json({ error: 'invalid_code' })

Because this kind of update query is not possible, it turns into something like this:
const now = new Date()
const authCode = await prisma.authorizationCode.findFirst({
  where: {
    code: req.body.code,
    clientApplicationId: client.id,
    expiresAt: { gt: now },
    consumedAt: null
  }
})
if (!authCode) return res.status(400).json({ error: 'invalid_code' })
await prisma.authorizationCode.update({
  where: { code: req.body.code },
  data: { consumedAt: now }
})

This change not only makes two database calls where one would be fine, but now there is a gap of time between when we fetch the auth code (and check whether or not it has been consumed or not) and when we update the consumed. This would allow for a Reply attack. The security consideration for this is briefly described here: https://www.oauth.com/oauth2-servers/access-tokens/authorization-code-request/#security-considerations

This is the workaround we currently use, based on

https://www.prisma.io/docs/concepts/components/prisma-client/relation-queries#update-a-specific-related-record
https://www.prisma.io/docs/concepts/components/prisma-client/relation-queries#delete-specific-related-records

prisma.person.update({
  data: { id: input.personId },
  where: {
    clients: {
      update: { where: { input.clientId }, data: { name: input.name } }
    }
  },
  select: { clients: { where: { id: input.clientId } }  }
}).then((data) => data.clients[0])

...and there you go. A little convoluted but does the job. Then select/then part is actually really importing, b/c  will get back a person object instead of a client.
Of course, there is no solution available for the  delete-mutation (other than a transaction as mentioned in this discussion), the only thing we can do there is .then(() => { return { id:data.clientId } } ), which is better than nothing.

@ksmithut Totally agree.
It should be allowed to provide an extra non-unique field along with an unique one in the .where clause. This simple change would make our life so much easier and the code would become much more stable.
Let me provide an example from my app.
I need to mark a transaction as "finished" and then apply the transaction amount to user's balance.
  const transactionUpdated = await db.transaction.update({
    where: {
      id,
    },
    data: {
      status: TransactionStatus.finished,
      user: {
        update: {
          balance: {
            increment: transaction.amountNet
          }
        }
      }
    }
  })

But when I do this, in order to avoid double balance change I must check first that transaction is not finished by another process. I do this as suggested on Prisma docs by checking lastUpdated field that is auto-incremented on every update. To make this kind of where possible, I'm changing .update to .updateMany:
  const transactionUpdated = await db.transaction.updateMany({
    where: {
      id,
      lastUpdated: transaction.lastUpdated
    },
    data: {
      status: TransactionStatus.finished,
      user: {
        update: {
          balance: {
            increment: transaction.amountNet
          }
        }
      }
    }
  })

But now the problem is that since I've changed .update to .updateMany - I can't do the nested update of user's balance!
I have all kinds of similar issues because of this limitation. I'm developing an app that works with money and I must pay close attention to race conditions.
Please change this.

+1

+1

I think this thread has uncovered a very real use case that is currently not well supported by Prisma.
We want the where clause to be limited to unique fields for reasons pointed out already.
The where clause is used to identify the singular row to be updated. Additionally, it should be possible to submit a filter against any columns on the row to decide wether to actually perform the update. Multiple use cases for this are given already. Another use case I have heard is to only perform the update if the data has actually changed. This is useful to avoid an updatedAt column being updated when all other data stays the same.
In the Prisma API, it could look like this:
const client = await prisma.client.update({
  data: { name: input.name },
  where: {
    id: input.clientId,
  },
  if: {
    personId: input.personId
  }
})

Conceptually, the where clause selects the row, and the if clause determines if it should be updated.
Eventually, this could be expanded to support more complex use cases than just evaluating columns on the row to be updated. This idea is explored in some detail in a long conversation last year: https://github.com/prisma/prisma/issues/1844#issuecomment-830860673

We have encountered this problem with soft delete that we worked around using two queries. But we are wondering how do people handle authorization checks with Prisma in the end? If you have a multi-tenant application and the concept of organization and a Comment model for example. And you want to ensure regardless of the request you want to achieve that this one will be scoped to the organization you are part of. With Rails, you can add a .where(organization_id: id) to your query and chain that to further filtering. Having where with more than unique fields would allow this.
Should all our endpoints (read, update, delete) have this kind of code then?
    const comment =
      await dataSources.prisma.comment.findFirst({
        where: {
          id,
          organizationId,
        },
      });
    return await dataSources.prisma.comment.findUnique({
      where: {
        id: comment?.id,
      }
    });

Any suggestions @janpio?

+1

+1

+1

+1

Dear Prisma developers, let me know please if investing some money in your project will help to speed up adding this feature. We need it very much for the project that is already in production. Thank you.

@ksmithut   https://github.com/prisma/prisma/issues/7290#issuecomment-1166276507
dynamodb has same issue and resolve it with ConditionExpression in update call. Adding sample query. Maybe something to think about. Edited query here inline, hope no types, but idea should be clear, check docs for more details.
related dynamo docs:
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html
  await db
      .update({
        TableName: "some_table",
        Key: {
          authCode: req.body.code,
        },
        ConditionExpression:
          "attribute_not_exists(consumedAt) and #clientApplicationId = :clientApplicationId and #expiresAt > :now ",
        UpdateExpression:
          "set #consumedAt = :now",
        ExpressionAttributeNames: {
          "#clientApplicationId": "clientApplicationId",
          "#expiresAt": "expiresAt",
        },
        ExpressionAttributeValues: {
          ":now": Date.now(),
          ":clientApplicationId": req.body.client_id,
        },
      })
      .promise();

I just realized what if we give prisma what it needs for the .update method - a unique field?
What if we use a uniqueuuid() value in the lastUpdated field, instead of a non-unique auto-incremented value, and update it every time the object updates? Then we will be able to use prisma .update and get what we need?
await db.user.update({
  where: {
    lastUpdated: uuidValueFromThePreviousQuery
  },
  data: {
    something: "foo",
    lastUpdated: generateNewUuid()
  }
})

@oyatek That's a good way to handle conflicts, similar to a version field like you mentioned. But there are a lot of other cases, such as making sure only an owner can update a post. The owner field cannot be unique because the owner can own multiple posts.

@oyatek That's a good way to handle conflicts, similar to a version field like you mentioned. But there are a lot of other cases, such as making sure only an owner can update a post. The owner field cannot be unique because the owner can own multiple posts.

My exact problem here. It just seems like such a base level feature although I am sure it probably is not to implement without breaking other things in their codebase. While I respect the heck out of all the work the team is doing, I feel like the community is being ignored here. This was opened on May of last year and since I posted, I keep getting notifications showing so many others pointing out this flaw. And that doesn't even account for the several comment threads elsewhere.

@oyatek That's a good way to handle conflicts, similar to a version field like you mentioned. But there are a lot of other cases, such as making sure only an owner can update a post. The owner field cannot be unique because the owner can own multiple posts.

Then you can make a unique index based on 2 fields:
@@unique([userId, lastUpdated])

Currently it isn't possible to do something like:
prisma.project.delete({
  where: {
    id: 'foo',
    userId: 'bar'
  }
}

To make sure that the user can only see the project that itself owns.
The only workaround I can find involves adding an additional unique index, and query on that:
prisma.project.delete({
  where: {
    id_userId: {
      id: 'foo',
      userId: 'bar'
    }
  }
}

Surely we shouldn't be required to add additional unique constraints on combinations with id? 🤔
And it also doesn't work at all for ensuring things only get deleted once, since @@unique exclude null values:
prisma.project.update({
  where: {
    id_userId_deletedAt: {
      id: 'foo',
      userId: 'bar',
      deletedAt: null
    }
  },
  data: {
    deletedAt: new Date()
  }
}

(property) deletedAt: string | Date
Type 'null' is not assignable to type 'string | Date'.ts(2322)

Very needed feature for everyone

@janpio another use case here is is for data security and compliance - we need to ensure that all queries where relevant are bound by the parent organization.
   prisma.someDao.findUnique({
      where: {
          id: 'someUniqueId', // <--- yes this will find one
          parentOrg: 'parentOrgThatOwnsSomeUniqueId' //<--- but for compliance we need to ensure that the id belongs to the requesting organization
      }
   })

this obvious easy thing should be allowed. How come prisma does not support this? Please bring this in...

+1 to supporting a updateOne() similar to findOne() based on id and attribute/where clause

Hey folks,
A solution was implemented by https://github.com/prisma/prisma-engines/pull/3281. It will be available in the next release under a new preview flag called extendedWhereUnique.
If you have any feedback you'd like to give about this feature, please post it here https://github.com/prisma/prisma/issues/15837#event-7600630285
Thanks for your patience, closing now 🙏.


chore(); deployment.json; Set dexes to prod that are prod ready in deployment.json

Context:
Set:

Pangolin
Solarbeam
mm finance
VVS Finance

To prod in deployment.json.

@steegecs is pangolin prod-ready? I see there is still an issue open for it: https://github.com/messari/subgraphs/issues/1230

also, what's the status on Uniswap v3? I thought some of them are prod ready

Spiritswap could most likely be added to this

@pietydespises I think so too, since we've just decided.
@this-username-is-taken All Uniswap V3 have been validated so they should all be prod. Pangolin should not.
Addressing here -- > https://github.com/messari/subgraphs/pull/1286


Removing Bitly credentials from JS, setting them using application config

No more credentials set within url_shortener class, they will be set from the cdb.config model and those values will be provided through the frontend_config from the backend.
@luisbosque @javisantana

Sounds good?

Yep, bitly should be used only if the login/key exists and it's not empty. Otherwise the real CartoDB map url should be returned instead of the bitly url.

Until this https://github.com/CartoDB/cartodb-platform/issues/1055 issue is not ready, we can't deploy this pull request.

Apart from that, everything ok @luisbosque @javisantana ?

:+1:


Make alertGC interval configurable

@brancz and I were discussing this earlier, and it seemed to make sense to make this configurable.
I'm happy to change the flag name, so any suggestions are welcome.

lgtm :+1:


QueryBuilder execute Method Doesn't Properly Map Embeddables

Describe the bug
The QueryBuilder class in MikroORM has an execute method with the mapResults option. When mapResults is set to false, the method returns the raw table representation, which is the expected behavior. However, when mapResults is set to true, the method does not convert embeddables back into objects. Instead, it returns them as separate properties named according to the table columns, which doesn't seem to make much sense.
Example Schemas
export const PersonNameSchema = new EntitySchema<PersonName>({
  class: PersonName,
  embeddable: true,
  properties: {
    givenName: {
      type: 'string',
    },
    surname: {
      type: 'string',
    },
  },
});

export const EmergencyContactSchema = new EntitySchema<EmergencyContact>({
  class: EmergencyContact,
  embeddable: true,
  properties: {
    name: {
      kind: 'embedded',
      entity: () => PersonName,
      prefix: 'emergency_contact_', // have to override the parent prefix
    },
    relationship: { type: 'string' },
  },
});

export const PatientSchema = new EntitySchema<Patient>({
  class: Patient,
  properties: {
    id: {
      type: 'text',
      primary: true,
    },
    name: {
      kind: 'embedded',
      entity: () => PersonName,
      prefix: false,
    },
    emergencyContact: {
      kind: 'embedded',
      entity: () => EmergencyContact,
      prefix: 'emergency_contact_',
      nullable: true,
    },
  },
});

Issue Reproduction
When executing the following command:
qb.select(['*']).where({ id: '1' });
  const patient = await qb.execute('get', true);

Expected Behavior
I would expect the mapped result to look like:
{
  "id": "1",
  "name": {
    "givenName": "John",
    "surname": "Doe"
  },
  "emergencyContact": {
    "name": {
      "givenName": "Jane",
      "surname": "Doe"
    },
    "relationship": "wife"
  }
}

Actual Behavior
Instead, I receive:
{
  "id": "1",
  "name_givenName": "John",
  "name_surname": "Doe",
  "emergency_contact_name_givenName": "Jane",
  "emergency_contact_name_surname": "Doe",
  "emergency_contact_name_relationship": "wife"
}

Importance
This feature is crucial for making queries without IDs and for correctly mapping complex structures.
Could you please address this issue and ensure that embeddables are properly mapped back to objects when mapResults is set to true? I have updated my repository from another bug to include test scenarios for this issue.
Thank you!
Reproduction
https://github.com/mikro-orm/reproduction/pull/5
What driver are you using?
@mikro-orm/sqlite
MikroORM version
6.2.8
Node.js version
v20.11.1
Operating system
MacOS
Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Slack.
[X] The provided reproduction is a minimal reproducible example of the bug.

FYI this will be quite a big task to achieve, and I am honestly not sure if we shouldn't even consider this breaking. I spent a better part of yesterday trying to make this work, and I can already see it will take a few more days to be able to finalize this.

IMO, it’s a bug, as we’re exposing MikroORM internal property names, instead of the entity properties, which is not the expected behavior.
But I understand if you prefer to consider it a breaking change. It’s impossible to say if someone currently depends on the current behavior.
Is there anything that I help with to facilitate this feature? I’d be happy to contribute.

Hmm, that's a good point, I thought it maps to the column names, but given your prefixing setup, it's not the case.

The problem here is that to get what you want from just result mapping, it means changing everything internally - result mapping, snapshotting, comparator, driver implementation, that all expect properties that map 1:1 to database columns in the entity snapshot. Even if I do all of that, I would now have to deal with a new problem of detecting partial changes on the inline embedded properties. I managed to implement the hard part, which is result mapping, but it opened a can of worms.
Maybe it would be better to deal with this locally, only in case you explicitly call qb.execute(), and keep the internally mapped state as is.

I don’t think the effort will be worthwhile if you need to change all the ORM internal functioning. I just really care about what we’re exposing to the users in the qb.execute since it seems to currently be the only way of querying the database without loading entire entity instances.
My use case is using CQRS, which MikroORM does a great job for commands, but it seems to be more difficult to make custom queries that return results straight from the database for views, without managing state.
So I want to generate reports with a few specific columns, using joins, pagination, and etc. It seems that QueryBuilder is the way to go. It just returns objects with weird property names. I could use mapResults = false and manually map data back to an entity-like format. But since the ORM already have tools to do so, and it seemed that it was the intended purpose for mapResults = true, I opened this ticked.
Let me change my question then. If changing the return of qb.execute will break everything, is there a way to expose some helper mapping functions that we could manually use to parse raw data? That would facilitate my use case a lot.

from the database for views

I would just use a raw query for that, via virtual entity mapping.

Okay, great! That will work. I have some questions about how to properly use the virtual entity, but I can ask on Discord.
Thanks @B4nan


Comparison with other computations

[x] Compare accuracy with Mandel & Agol computation.
[x] Compare accuracy and speed with Gimenez' code.
[x] Compare accuracy and speed with Kreidberg's quadratic limb-darkening (both unbinned, and integrated with adaptive binning).
[x] Compare with non-linear limb-darkening law (to what order do we need to go to achieve same accuracy of non-linear limb-darkening?).
[x] Compare with Pàl’s quadratic limb darkening with analytic derivatives.  (Right now test_transit_poly_struct.jl matches ntiq-fortran which was compiled with -O3;  both take 0.5 seconds to compute 10^6 models, with the same impact parameters for both.  We may be able to save some time by reusing elliptic integral computations.)
[ ] Compare with Maxted's power 2 limb-darkening law.
[x] Comparison with EXOFAST.

@ericagol Here's how your code compares to batman: you're killing it!

I'll code up the comparison to the nonlinear model next.

@rodluger I already did one version of the comparison to the nonlinear model - that's what the occult_nonlinear.jl does.  But, it would probably be good to explore a range of parameters.

Going to skip comparison with Maxted's power-2 limb-darkening.


Fix panic error when downloading podcast with existing ID3v2.2 tags.

Description

This pull request fixes #658 by checking if a podcast episode's tags are ID3v2.2 when downloading. If they are, a completely new tag is generated in the default format (ID3v2.4).
Linked Issues
#658 and #690
Additional context

The panic error was caused by the podcast episodes having ID3v2.2 tags which use three character frames. Since the existing tags are used as a baseline and the tag being written to new downloads is ID3v2.4 which use four character frames, there was a panic error due to the mismatch.
Thank you for your work on this project and the guidance via commit #690 so I could find the right area to explore!

Thanks for the pull request and for the fix. Sorry for closing the issue :)


Disconnect immediately on window close

When the window for your client is closed, it should disconnect immediately.

I had a look into fixing this the other day and it seems a little non-trivial to do it correctly. You can force-close the connection when the window is closed (by listening for the window close event), but that's not a graceful exit and it doesn't take the same code path on the server (i.e. it's reported as an error, not as the user leaving.)
Ideally, we would remove all of the handles to the connection so that the refcount drops to 0 and quinn naturally terminates the connection, but it gets stored in a multitude of places, including the UI tree. Not sure how to best deal with that.


Support for Laravel 11 and php 8.2

Currently the last version of Laravel 11 with php 8.2 throws installation error.
Thanks

wil fix soon


@conda-forge-admin, please add bot automerge

@conda-forge-admin, please add bot automerge

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added bot automerge in conda-forge/typing-feedstock#27.


Fix real app test 02

Fix real app test

[x] fix all warnings again
[x] fix some abstract classes
[x] fix real app test and tested real_app.dart 3 times on Android Nexus 6 API 30
[x] fix real app test and tested real_app.dart 3 times on iPhone 14 pro IOS 16

Please before run yarn run build because added Js code from this PR

Nice, cleanup! But actually, I don't understand what the bug was that you fixed. :D
That's my question too. But really we had 7 mistakes before. We fixed 6 errors by correcting our classes, such as BlockData, but I could not find 1 error, just for one error. But I didn't know where that mistake was 🧐


QC should treat fixed variables differently.

The QC currently works through each of the variables and works out the file paths to check for the existence of files. Fixed variables will invariably have a different file path so they get listed as a missing file based on the dates.
Fix is to create a testable "create_filepath" function that is used with the drs processing but automatically obscures the logic to rewrite the file path for fixed variables.

Also done in another issue


Changing RI region

Is there an ability to change the region of an RI I paid for?
I read this "Exchange Azure RIs across any region and any series as your workload or application needs change." at this page: https://azure.microsoft.com/en-us/pricing/reserved-vm-instances/, but didnt see instructions on how to perfom the action

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9be8cdc6-1a0e-25d6-8131-a56c5c0029a5
Version Independent ID: d27499e5-c8d4-2de0-273d-09f64aec3121
Content: Manage Azure Reservations
Content Source: articles/billing/billing-manage-reserved-vm-instance.md
Service: cost-management-billing
GitHub Login: @bandersmsft
Microsoft Alias: banders

@assaf-grth Hi there. Check out https://docs.microsoft.com/en-us/azure/billing/billing-azure-reservations-self-service-exchange-and-refund it should have all the details you need.
#please-close

@bandersmsft Thanks for the quick response. :+1:


Lift code restrictions and allow other subjects to be explored

I know I am asking a lot but could someone smarter than me create a PR to not have code as requirement and hence allow other subjects to be explored as well?
Related:

https://github.com/SakanaAI/AI-Scientist/issues/6
https://github.com/SakanaAI/AI-Scientist?tab=readme-ov-file#how-do-i-run-the-ai-scientist-for-different-subject-fields

Please see the community templates for lots of new examples, including new fields!


Allow more log levels

While reading the source code I found that you can pass --solo5:quiet to the guest to set the log level to ERROR. This change adds --solo5:error (alias of --solo5:quiet), --solo5:warn and --solo5:info.
Calling log_set_level(-1) should silence all log levels if it compiles. I can change the semantics of --solo5:quiet to that if anyone can confirm whether it is safe to pass -1.

Let's merge this PR (#528 already adds a new argument, so we can take this opportunity to improve the CLI). Thanks!


docs: Revise placement of colonequals functions in the Relations table.

Sixteen functions from the colonequals package were not in the docs until recently. PR #2651 added them in alphabetical order to the Relations table in supported.md. This PR rearranges their location, placing most of them in the same cell as the equivalent mathtools command.

Codecov Report

Merging #2704 (3ce6c5f) into master (8c08a3c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2704   +/-   ##
=======================================
  Coverage   93.85%   93.85%
=======================================
  Files          86       86
  Lines        6168     6168
  Branches     1262     1262
=======================================
  Hits         5789     5789
  Misses        348      348
  Partials       31       31

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c08a3c...3ce6c5f. Read the comment docs.


[WIP] New WAF Fingerprint Plugins

NOTE: This is a work in progress [WIP] pull request.

Pull request for new plugins for wafw00f. You can keep reviewing as I keep adding more commits to this PR. If you want to push any changes, please push it to my fork so that it gets reflected on the pull request itself.
I'll ping you when this work is complete.

Okay so now we have a hundred WAFs in our collection. : )

So the main coding part + testing phase are over. Now we need to discuss about the WAF corrections and WAF priority to be set in wafprio.py

Now waiting for the Travis CI tests to pass. Once they pass, this will be ready for merge.

Alrighty build passed, everything seems okay. Ready for merge.


Add week picker component

What this PR does / why we need it:
This component will be used on the insight page.

Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

Nice.
/approve


E2E Nightly is failing after Mar 5. Please update the token.

Describe the bug
The CI E2E Nightly is failing after Mar 5.

The reason seems expired token.
https://dev.azure.com/azure/application-gateway-kubernetes-ingress/_build/results?buildId=37969&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=5caf77c8-9b10-50ef-b5c7-ca89c63e1c86&l=1597

To Reproduce
Steps to reproduce the behavior:

Open https://dev.azure.com/azure/application-gateway-kubernetes-ingress/_build?definitionId=85&_a=summary on your browser.

@akshaysngupta I found your amazing activities in recent pull requests.
Would you happen to know how to update the token?
Or could you update it?

fixed!


TensorFlow 1.x models no longer working properly in SDK >= 7.2

Our team is trying to migrate to the newest SDK version, because it is required for the competition. The thing is, that earlier we had a fully working, custom tensorflow 1 object detection model and as soon as we updated the version, the model didn't work anymore (not detecting anything at all). Did the pre-trained model change? The same code works properly in SDK versions <7.2.(7.2 not included). We didn't find any breaking changes in the documentations of these newer versions :( Has anyone had this issue? If so, please let me know how to fix it.

This issue is a duplicate of #469.

@texasdiaz This doesn't sound like a duplicate to me.

@NoahAndrews What's different? Both are claiming that the TensorFlow 1.x Everyday Objects models not "working" any more - in other words, TensorFlow's stock Model Zoo models from TF 1.x are not properly classifying detections any more. Both claim that something broke the 1.x models from "working."
Did we break the 1.x model backwards compatibility when we separated Vuforia from TensorFlow? I don't know. Did a new version of TensorFlow libraries break backwards compatibility? I don't know. I'm not closing the issue because I don't know what the ultimate problem is. I just know both of these are effectively saying the exact same thing. The other issue came first, which is why I closed this one. This one has non-red-herring information in it. Maybe it makes sense to combine them.
The other issue confuses the topic by talking about the labels, if you're using the Model Zoo label map from 1.x there's an extra line in the label map (the first element is "????" in the 1.x map, but that extra line is not there in the 2.x map). I firmly believe that's a complete red herring. The "simple" and "easy" answer to both of these - until we figure out what went wrong - is to just use the TF 2.x Model Zoo models that Uday and I converted and provided for people to use. The more complex answer lies in determining what happened in SDK 7.2 that seems to have broken the TF 1.x compatibility.
-Danny

I reopened this topic because it now has a better title indicative of the actual problem, and is more succinct.

As a workaround for the time being, you should try using the TensorFlow 2.x COCO models that we have available for you to use. You can find the link to the Model Zoo models we converted here:
https://ftc-community.firstinspires.org/t/tensorflow-model-zoo-models/159/2
This only helps if the only TensorFlow 1.x model you want to use is the COCO Model Zoo (Everyday Objects) model.

What is the value of the inputSize parameter when you initialize TensorFlow?

300 or 320, this depends on the model.

I just want to know which pre-trained model to use from the Tensorflow model zoo

@texasdiaz I understand that TensorFlow Model Zoo models are full pre-trained tensorflow models and not .tflite. And the .tflite's you sent me are everyday object detection models, how are they useful for the competition? We are trying to take the ssd-mobilenet-v2 models from the model zoo(the .tflite's you sent me have the name ssd-mobilenet-v2 in them), train them, and exports as .tflite's. We did this successfully with TF1 models, but since TF1 is for some reason no longer supported, we are trying to use TF2. I need to know for sure which pre-trained models you used and how did you convert them to .tflite. Our .tflite models don't work, we get an error: "Unfortunatly, the robot controller has stopped", which gives us no information at all.

@Tarkkett I think I need to step back for a moment and ask what you're trying to do. How are you trying to train models? And what tool are you trying to train models with?
When you take a TensorFlow Model Zoo model and train on top of them, you get the benefit of a fully formed neural network (meaning your training performance using the new data will be much better than trying to train a network from scratch) but you completely lose all of the labels from the original model because you were unable to train and evaluate using the dataset used to create the original Zoo model (and no, we do not have access to that dataset).
So if you just use the "stock" zoo model you can use all of the trained labels and categories of the Zoo model as-is - meaning if you see an image of a dog the model will recognize it, if you see a keyboard the model will recognize it, and so on. HOWEVER, if you try to train on top of that model, you completely LOSE all of the Model Zoo labels, and you only get what you trained - and if you trained poorly, the model will be poor (garbage in, garbage out).
Does this make sense? Maybe you should tell me how you're TRYING to use the model zoo models, and we can go from there (rather than me assuming why you wanted the TF 1.x model zoo models to begin with).

Our .tflite models don't work, we get an error: "Unfortunatly, the robot controller has stopped", which gives us no information at all.

Can you send your .tflite file? I'd like to see if I can reproduce and debug the issue with it making the robot controller stop.

Our .tflite models don't work, we get an error: "Unfortunatly, the robot controller has stopped", which gives us no information at all.

Can you send your .tflite file? I'd like to see if I can reproduce and debug the issue with it making the robot controller stop.

You can download it Here , we made lots of models, but all of them have the same issue.

How did you make the model?

@texasdiaz We downloaded the pre-trained model from the zoo, then labeled many images and generated tfrecords from these files, we used our tfrecords and the fine_tune_checkpoint from the zoo to train the model, after training we got the saved_model. Converted that model to tflite_graph and after that, converted to tflite.

I see. Are you using TensorBoard to look at model metrics during training? Without using TensorBoard, you have no idea how well (or not) the model Training is going. Unfortunately one cannot simply "train" a model and then run it to see how well it does - that's just setting yourself up for frustration.

Do you have crash call stack?  I had crash issue using my customized model, but was able to work around by using the right tflite version in generating model and android gradle file:
implementation 'org.tensorflow:tensorflow-lite-task-vision:0.3.1
But your crash might be different though.

@texasdiaz we are using tensorboard, the loss seems normal, but it should detect something even if the loss would be very high.

Not necessarily. I've had many models with high loss that didn't detect anything at all, because the network couldn't properly classify any object features. It came down to training. Are you doing like few-shot learning or anything to limit the number of training steps required, or how much training are you doing?

But it wasn't detecting anything at all, o it crashed? It would make sence if it wouldn't detect anything, because, there would be a very high detection count. I am not sure what few-shot learning is. We aren't training much, because now we don't care if the detections are messed up and garbage, we just want to load the model, without the hub crashing.

Oh, that's literally the first time you said "crash" - that's pretty important. Just not detecting anything vs crashing is a very different issue.
I would very much appreciate it if you could reproduce the crash, and then download your Robot Controller log. Copy-paste it into pastebin.com, and create a new paste. Then provide me with the url to your pastebin. That way I can look at your crash log to see what happened.

Did you remember to do this:
tfodParameters.isModelTensorFlow2 = false;

That model was made with tensorflow2, if you set tfodParameters.isModelTensorFlow2 = false; it will load, but won't detect anything, i don't know if the model even load propelly. You need to set it to true;

Did you remember to do this?
tfodParameters.isModelTensorFlow2 = false;

I am also giving you a wetransfer link, because it wouldn't open for me here

I see two different types of crashes in the logs.

It was using tensorflow 2 code to load the model. If the model is tensorflow 1, you need to do tfodParameters.isModelTensorFlow2 = false;
The SoundPlayer was throwing an exception.

@texasdiaz Looking at the crashlog, i see that the VuforiaFrameGenerator is failing and need Nomalization metadata. I generated the metadata, but it still crashes. In the new SDK documentation i saw that something was changed for the FrameGenerator. Do you have a clue what could be wrong?

I have a clue what could be wrong!
It's because you have tfodParameters.isModelTensorFlow2 set to true!
Your log shows stack traces with TfodFrameManager2.java, which is only used for TensorFlow2.

I have a clue what could be wrong!
It's because you have tfodParameters.isModelTensorFlow2 set to true! Your log shows stack traces with TfodFrameManager2.java, which is only used for TensorFlow2.

Im telling you again, that the model was made with tensorflow 2. It should be like that.

Oh, sorry. My mistake.

@wangxdflight @lizlooney @texasdiaz @NoahAndrews Phefffff, after many failed attempts, i finally solved this problem. I had to add the metadate to the model, but also change the org.tensorflow:tensorflow-lite-task-vision:0.2.0 to org.tensorflow:tensorflow-lite-task-vision:0.4.3. Please look into this. That line of code caused so much pain! Can the robot-controller ropository developers change this?

Glad it worked for you know. Anything forced you go use 0.4.3 version? (it is almost the latest.) What TF version did you use to train your model? For me, I used TF 2.9.2, and vision task 0.3.1 worked with it.
But I am having another issue, maybe people here can help me. It may related with compatibility caused by different versions too.
I am trying to do int8 quantiziation, and was able to generate tflite model for it, but having weird crashes on android (I am experimenting with a spearte android application, where I can easily change libraries to use.)
I tried both vision task API and interpreter API. float16 quantization works, but int8 doesn't.

org.tensorflow:tensorflow-lite-task-vision:0.2.0

This worked. I have been making models with TFLITE Model Maker (which easy really really easy with Google Colab) and the robot controller app has been crashing all over the place.
It was crashing because the robot controller app is using an old version of org.tensorflow:tensorflow-lite-task-vision. In this old version it is looking for the output tensors in a very specific order. With TensorFlow 2.6, Google changed the order of those output tensors in the models.
If you build your own Robot Controller App (FtcRobotController), then you need to go to the build.dependencies.gradle file.
Change the line implementation 'org.tensorflow:tensorflow-lite-task-vision:0.2.0' into implementation 'org.tensorflow:tensorflow-lite-task-vision:0.4.3', build the app, load it onto the control hub and everything works perfectly.

TFOD has been removed from the SDK