Missing SSL for psql connection when using image

I'm trying to run asciinema-server in Kubernetes, and having it connect to a psql database using ssl.
This used to work when using the image asciinema/asciinema-server:latest from Dockerhub, but using the new images on ghcr, the container goes into Error state right after startup and logs this:
Running db migrations...
** (EXIT from #PID<0.96.0>) an exception was raised:
    ** (RuntimeError) SSL connection can not be established because `:ssl` application is not started,
you can add it to `extra_applications` in your `mix.exs`:

  def application do
    [extra_applications: [:ssl]]
  end

        (postgrex 0.16.5) lib/postgrex.ex:680: Postgrex.ensure_deps_started!/1
        (postgrex 0.16.5) lib/postgrex.ex:622: Postgrex.child_spec/1
        (ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:743: Ecto.Adapters.SQL.init/3
        (ecto 3.9.4) lib/ecto/repo/supervisor.ex:185: Ecto.Repo.Supervisor.init/1
        (stdlib 4.2) supervisor.erl:330: :supervisor.init/1
        (stdlib 4.2) gen_server.erl:851: :gen_server.init_it/2
        (stdlib 4.2) gen_server.erl:814: :gen_server.init_it/6
        (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

This happens with both ghcr.io/asciinema/asciinema-server:latest as well as ghcr.io/asciinema/asciinema-server:develop
I noticed that https://github.com/asciinema/asciinema-server/issues/378 is essentially the same issue, so I tried the fix suggested there.
We already mount /opt/app/etc/custom.exs , so I appended
config :asciinema, Asciinema.Repo,
   ssl: true,
   pool_size: 2

This did not change the behavior though.

Can you try pulling ghcr.io/asciinema/asciinema-server:develop now ? It includes this latest commit 0bfb919396a9dee17e8b5cf53f7ebe485c7a20ce which may help.

Try ghcr.io/asciinema/asciinema-server:develop again. I reproduced the problem and fixed it  :crossed_fingers:

Can confirm the issue is now resolved. Thanks!

@SNB-hz FYI I've published the fix to ghcr.io/asciinema/asciinema-server:latest, so you may want to switch from :develop to :latest tag as it's more stable in the long term.


Rails 4.1.9

Hello,
We're trying to upgrade our application to latest Rails version of the 4.1 series and it appears running data migrations is now storing in the wrong table. (i.e. in 'schema_migrations' and not 'data_migrations'). Is there a fix our there or is this a known problem?
Any help would be great.
Thanks!

George Diaz writes:

Hello,
We're trying to upgrade our application to latest Rails version of the 4.1 series and it appears running data migrations is now storing in the wrong table. (i.e. in 'schema_migrations' and not 'data_migrations'). Is there a fix our there or is this a known problem?
Any help would be great.
Thanks!

Reply to this email directly or view it on GitHub:
https://github.com/ajvargo/data-migrate/issues/22

--
Sent with my mu4e

any update on this?

@georgediaz88 looks like rails 4.0 made updates to migration process and introduced http://apidock.com/rails/ActiveRecord/SchemaMigration which conflicts with data-migrate code. What do you think if I create a new version data-migrate that has a dependency on rails 4.0 and up. This version will not be backward compatible though, so data-migrate "1.2.0" will not be maintained.

@ilyakatz, this sounds like a reasonable compromise for those of us who'd like to continue using this gem. Thanks.

Ok, cool. I have a working solution in #27, just need to clean it up and create separate branches for the versions, so will try to do this in the coming week

@ilyakatz, that sounds great to me. I'd like to continue using this gem as well. Let me know if you need a hand as well. Thanks.

@georgediaz88 thanks. if you could try it out with that new branch, that would be great. just put this in your gemfile
gem "data_migrate", github: "execonline-inc/data-migrate",
  branch: "fix_data_migration_table"

@ilyakatz, just tried your branch and it worked! I was able to add a new data migration, run the data migrate command, and see it in the data_migrations table. So, this worked for me. I'm glad to see this being worked on.

ok, thanks for checking. I want to do a bit more testing to make sure that this doesn't rerun all those data migrations that are recorded in schema_migrations as this could be pretty disastrous. Feel free to poke me again if i don't get it out next week.

yea, totally. Sounds good!

PR https://github.com/ilyakatz/data-migrate/pull/27 has been merged and new version published. Please keep an eye on anything suspicious and open new issues if anything comes up.


Add classpath only option

Once a project has bee created, then the most likely change over time is updates to the classpath. It would be good to have an option that would only update the classpath(s) of the project(s) rather than updating everything (settings file, source directories, .project & .classpath).
This has come about as I have updated my projects' classpaths and now have empty directories src/main/java etc. that are not required as this is a pure scala project.
I'm using 2.0.0-M3

It is by intention that the .classpath file created by sbteclipse exactly matches the sbt build definition. This will not change.
If you, for example, don't need the src/main/java source directory, why don't you remove it in sbt using these settings:
unmanagedSourceDirectories in Compile <<= (scalaSource in Compile)(Seq(_))

unmanagedSourceDirectories in Test <<= (scalaSource in Test)(Seq(_))

I don't mean that the .classpath file should not match the sbt build settings, I took this as one of the main purposes of this plugin.
I was wanting an option that would ONLY update the .classpath file and not make any additional changes.
Such as:

The other files that are modified when I have made other eclipse specific changes to them e.g. the settings file, result in the local changes being removed. If this was not updated then my eclipse specific settings would be left unaffected.
The creation of unnecessary source directories. It would be nice if the inclusion of source directories was limited to those that are explicitly set (if you are using non-default locations) or those that exist in the case of default configuration. In my case, I do not have a src/main/java directory and so it should not be created for me. This, to me, is part of the point of Convention over Configuration. If the convention is to place Scala code into src/main/scala and Java code into src/main/java then by not creating a src/main/java directory I am following the convention and stating that I do not want src/main/java on my classpath. It seems a bit unnecessary to add lines to my project configuration to explicitly remove directories that do not exist and, if following the convention, do not need to exist.

I think you are misunderstanding the purpose of sbteclipse. It is meant to be used in a setting where sbt is the master, i.e. the sbt build definition will be "translated" into an Eclipse project definition. The sbt build definition not only contains settings which will end up in .classpath, but also such, that will end up in the .settings directory. Therefore it is important to always write all these settings to the Eclipse project definition. Having an option to only write parts is against the main purpose of this plugin.
Let me again explain you how to avoid "unnecessary source directories": If you consider, for example, src/main/java and src/test/java, as unnecessary, then it certainly is unnecessary for sbt, too. Therefore please change your sbt build definition like shown in my previous answer, then sbteclipse will not create src/main/java and scr/test/java. If you keep the default sbt settings here, it is very important for sbteclipse to create these directories, because else users would not be able to start with fresh projects, because there would not be source directories in Eclipse. Also, please take a look at the documentation how to control the creation of Eclipse source entries for managed sources and resources.
By the way, I accepted your other issue, please see #81.

I thought the chain went SBT -> sbteclipse -> Eclipse.
If this is the case then for a scala project,  SBT will require me to manually setup my src/{main,test}/scala directories (as is implied here https://github.com/harrah/xsbt/wiki/Getting-Started-Directories). SBT will then discover the source directories (by convention) and add them to the classpath. The same is true for java projects or mixed projects.
The point being that sbt works by convention: If there is a source directory src/main/scala compile it with the scala compiler, if there is a source directory src/main/java compile it with the java compiler. It is therefore part of the SBT setup that you create your source directories and not the responsibility of Eclipse or sbteclipse


⚠ Bofrost.lu has degraded performance

In dcf64f7, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1086 ms

Resolved: Bofrost.lu performance has improved in 01c62ed after 6 minutes.


Order of spline

In earlier versions of pyGAM I was able to set the spline_order parameter, which signifies the order of the spline I want to fit. That way I could specify if I want a linear spline, a cubic spline or higher.

Is there still a provision to specify the order of the spline?

Does the model output (summary or otherwise) provide the information on the order of the spline which is used to fit the model?

I guess you could use spline_order argument anyway, through the s() function.
I have tested it and it works perfectly for me.
from pygam import LinearGAM, s, f
from pygam.datasets import wage

X, y = wage()
gam = LinearGAM(s(0, n_splines=10) + s(1, n_splines=20) + f(2)).fit(X, y)

Exactly what I was looking for. This helps. Thanks.

@dibyoghosh how might we document the spline_order api  so that it is easier for users to discover?
also, @daventero, thanks for your great answer.
thanks
-dani

For my usage, I would only need an exposition as a part of the summary. Maybe a column next to rank, which provides the order of the spline corresponding to each term. I observed that you the spline_order of each term can be different, just as the n_splines. So in the final output, it would help if each term in gam.summary is listed with the both the Rank (n_splines) and Order (spline_order).


Move command logic into commands.

At the moment command logic is stored within the Server or Client classes (depending on what it does). This works currently, but will be unsustainable as more and more commands are added.
I'm adding this here to remind my self later.

Add two virtual methods to the base command, one for server side logic and the other for client logic.
Move current logic into these virtual methods.
Change server and client classes to auto call these methods.

Are you already working on this or can I have a try?


PLATFORM-8994 add max length password policy

Related issue(s)

Checklist

[ ] I have read the contributing guidelines.
[ ] I have referenced an issue containing the design document if my change
introduces a new feature.
[ ] I am following the
contributing code guidelines.
[ ] I have read the security policy.
[ ] I confirm that this pull request does not address a security
vulnerability. If this pull request addresses a security vulnerability, I
confirm that I got the approval (please contact
<EMAIL_ADDRESS>from the maintainers to push
the changes.
[ ] I have added tests that prove my fix is effective or that my feature
works.
[ ] I have added or changed the documentation.

Further Comments

Codecov Report
Attention: Patch coverage is 58.33333% with 5 lines in your changes are missing coverage. Please review.

:exclamation: No coverage uploaded for pull request base (fandom_master_1_0_0@004fa74). Click here to learn what that means.

:exclamation: Current head 1edfaf3 differs from pull request most recent head a2472b1. Consider uploading reports for the commit a2472b1 to get more accurate results

Files
Patch %
Lines

selfservice/strategy/password/login.go
25.00%
2 Missing and 1 partial :warning:

selfservice/strategy/password/validator.go
0.00%
1 Missing and 1 partial :warning:

Additional details and impacted files
@@                  Coverage Diff                   @@
##             fandom_master_1_0_0     #106   +/-   ##
======================================================
  Coverage                       ?   76.74%
======================================================
  Files                          ?      330
  Lines                          ?    21983
  Branches                       ?        0
======================================================
  Hits                           ?    16871
  Misses                         ?     3816
  Partials                       ?     1296

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


VSCode staging changes to files that didn't change

Issue Type: Bug
I'm not really sure how it happens, or how to reproduce. It only happens in one project.
When I'm working sometimes I'm not even doing anything on the project, out of nowhere all my files turn green on file explorer (untracked marker),  when I check Source Control, I see that all my 81 tracked files are marked as deleted and staged, and then I have another 81 changes creating new 81 files.  It appears something is staging the deletion of those files. I doubt that it is Code doing this, though. Maybe an extension or an external tool. Any thoughts?

It happened even with all extensions disabled.
I am not aware of any tool I might have on my computer other than VSCode that uses git commands, so thats why I imagined it was here
I had taken a quick look at these logs, and thought it was VSCode staging it, but there doesn't seem to be any git add so I guess that's my mistake

Does it repro with another repositories?


Port protobuf structures to updated specs

This also migrates the protobuf-related files
into the ipc package
Fixes #647

Codecov Report

Merging #650 into rusk-ipc will increase coverage by 0.86%.
The diff coverage is 20.25%.

@@             Coverage Diff              @@
##           rusk-ipc     #650      +/-   ##
============================================
+ Coverage     55.53%   56.39%   +0.86%
============================================
  Files           160      150      -10
  Lines         10565     7389    -3176
============================================
- Hits           5867     4167    -1700
+ Misses         3978     2636    -1342
+ Partials        720      586     -134

Impacted Files
Coverage Δ

pkg/config/consts.go
0.00% <ø> (ø)

pkg/core/candidate/broker.go
72.00% <ø> (-3.00%)
:arrow_down:

pkg/core/chain/legacy.go
0.00% <0.00%> (ø)

pkg/core/chain/mock.go
50.00% <ø> (+6.41%)
:arrow_up:

pkg/core/consensus/candidate/factory.go
100.00% <ø> (ø)

pkg/core/consensus/candidate/genesis.go
71.42% <ø> (-1.91%)
:arrow_down:

pkg/core/consensus/candidate/testutil.go
83.05% <ø> (+0.19%)
:arrow_up:

pkg/core/consensus/selection/factory.go
100.00% <ø> (ø)

pkg/core/consensus/selection/handler.go
18.18% <0.00%> (-5.82%)
:arrow_down:

pkg/core/consensus/selection/selector.go
57.54% <ø> (-0.58%)
:arrow_down:

... and 169 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c9c680...8be3f3f. Read the comment docs.

Codecov Report

Merging #650 into rusk-ipc will increase coverage by 0.98%.
The diff coverage is 54.59%.

@@             Coverage Diff              @@
##           rusk-ipc     #650      +/-   ##
============================================
+ Coverage     54.96%   55.95%   +0.98%
============================================
  Files           156      150       -6
  Lines          9689     7454    -2235
============================================
- Hits           5326     4171    -1155
+ Misses         3723     2696    -1027
+ Partials        640      587      -53

Impacted Files
Coverage Δ

pkg/config/consts.go
0.00% <ø> (ø)

pkg/core/chain/legacy.go
0.00% <0.00%> (ø)

pkg/core/chain/mock.go
50.00% <ø> (+6.41%)
:arrow_up:

pkg/core/consensus/candidate/factory.go
100.00% <ø> (ø)

pkg/core/consensus/candidate/genesis.go
71.42% <ø> (-1.91%)
:arrow_down:

pkg/core/consensus/collector.go
0.00% <0.00%> (ø)

pkg/core/consensus/committee/handler.go
77.41% <0.00%> (-11.47%)
:arrow_down:

pkg/core/consensus/header/header.go
22.09% <0.00%> (-2.66%)
:arrow_down:

pkg/core/consensus/queue.go
100.00% <ø> (ø)

pkg/core/consensus/reduction/testutil.go
51.11% <0.00%> (+2.96%)
:arrow_up:

... and 221 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a46d12...6f89d08. Read the comment docs.


Are capital letters preserved?

I have string Α108 (Greek Capital Letter Alpha). Online version translates it as a108 (small letter).

@Ginden
SBL's handbook was a little ambiguous in regards to capitals. Note in the snapshot below that both alphas are transliterated with a lowercase 'a' and this is the case for every character.

But the capital pi below is transliterated with an uppercase 'P'

I went with the former since the later was implicit.

An option like:
transliterate("Αλφα", { capitals: true })
// Alpha

would not be difficult to implement


Improved gmres

Adds an improved version of Jax gmres. The new version builds the QR factorization of the Arnoldi overlap matrix by updating it at each Arnoldi iteration. This allows the solution residual to also be estimated at each Arnoldi iteration, so that the Arnoldi process can be terminated early if convergence is reached. Also adds improved docstrings and tests.

Codecov Report

Merging #767 into master will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #767      +/-   ##
==========================================
+ Coverage   97.89%   97.92%   +0.02%
==========================================
  Files         113      113
  Lines       19081    19291     +210
==========================================
+ Hits        18679    18890     +211
+ Misses        402      401       -1

Impacted Files
Coverage Δ

tensornetwork/backends/jax/jax_backend.py
97.59% <100.00%> (+0.40%)
:arrow_up:

tensornetwork/backends/jax/jax_backend_test.py
98.53% <100.00%> (+0.04%)
:arrow_up:

tensornetwork/backends/jax/jitted_functions.py
96.54% <100.00%> (+1.56%)
:arrow_up:

...ensornetwork/backends/jax/jitted_functions_test.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed96800...836bfd2. Read the comment docs.

I'm unable to reproduce the current build error on my machine, and the error message from Travis is pretty cryptic. I do get a typing error from block_sparse/tensordot_test.py and a linter error from tn_kera/test_layer, but I didn't change those things.

@Thenerdstation @mganahl Does anyone understand what is happening here?

Wut

Looks like you've hit a pytype bug. Ouch.
I think it's about time we've upgraded our pytype anyway. Let me do that tomorrow and then try this again.

@alewis is this the pylint bug you hit earlier?

@googlebot I consent.


pnpm install not running prisma generate

Bug description
When installing within a PNPM monorepo the generator isn't being run. I suspect it might have something to do with PNPM symlinking dependencies and therefore messing with the client postinstall script. Just a guess though.
How to reproduce
Reproduction: https://github.com/scriptcoded/prisma-issue-6603

Make sure PNPM is installed. (https://pnpm.io/installation)
Clone the reproduction repo: git clone git@github.com:scriptcoded/prisma-issue-6603.git
Enter the app directory: cd prisma-issue-6603/app
Install dependencies using PNPM: pnpm i
Run the application: pnpx ts-node index.ts
Observe that the Prisma client has not been generated

Expected behavior
The Prisma client should be generated and the application should run without issues.
Prisma information
Project set up accoring to the Start from scratch guide.
Environment & setup

OS: Manjaro Linux x86_64
Database: PostgreSQL 12
Node.js version: v14.15.2
Prisma version:

prisma               : 2.21.2
@prisma/client       : 2.21.2
Current platform     : debian-openssl-1.1.x
Query Engine         : query-engine e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ../node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Migration Engine     : migration-engine-cli e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ../node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Introspection Engine : introspection-core e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ../node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Format Binary        : prisma-fmt e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ../node_modules/.pnpm/@prisma+engines<EMAIL_ADDRESS>Default Engines Hash : e421996c87d5f3c8f7eeadd502d4ad402c89464d
Studio               : 0.371.0

I can confirm this. For now you can workaround it by adding a manual postinstall script to your package running prisma generate like so:
{
  "name": "app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "postinstall": "prisma generate"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@types/node": "^14.14.41",
    "prisma": "^2.21.2",
    "ts-node": "^9.1.1",
    "typescript": "^4.2.4"
  },
  "dependencies": {
    "@prisma/client": "^2.21.2"
  }
}

At the moment, it gets generated in your workspace's node_modules and not in the package's node_modules, this would be fine for only one package but not many. With https://github.com/prisma/prisma/pull/7111, the problem has been partially fixed, it still does not generate on install; but in turn it does generate on pnpx prisma generate in the right folder.
Now, there's another problem, you want to import @prisma/client. That's not possible because of pnpm and the way we resolve the generated .prisma/client. In short, when you install, pnpm will put @prisma/client in your workspace's node_modules. This has three effects:

We can't resolve the .prisma/client via @prisma/client because they don't live in the same node_modules folder. The one is in the workspace, and the other is in the package.
When you import @prisma/client it then tries to import the neighbor .prisma/client. For the reason explained above, this is not possible. Even though pnpm claims to hardlink, this does not happen on the folders (not possible).
For the same reasons, we would not be able to provide the right types for each package (assuming that you have multiple packages using multiple clients, each with with a different schema).
That's how things are resolved by import / require.

For the reasons above, I suggest that on such a pnpm monorepo setup, you would not import @prisma/client, but you would import the generated .prisma/client directly instead.
If you would like to give it a try, please use<EMAIL_ADDRESS>and<EMAIL_ADDRESS>It still won't generate automatically on install, but at least you can do pnpx prisma generate in your package's folder. Not closing the issue until that works.

I just ran into this issue.  It does work with the import {PrismaClient} from '../node_modules/.prisma/client just not sure if it'll pass our code review process.  Really its a prisma issue where it should allow generation in any folder

@millsp Hey, thanks for the very thorough reply! Totally missed your comment, sorry for that.
We got our project working by explicitly generating the client at build time, but your explanation makes me curious to try out those branches. I'll see if I get time to check it out, and in that case I'll probably have some thoughts :)
Thanks!

Thaaar be dragons in postinstall. The kind that loop forever...
Proceed with caution.

We have moved on to NPM workspaces since this issue was created and I can confirm that they suffer from the same issue. The following root package.json file works like a charm for anyone interested.
{
  "name": "my-project",
  "workspaces": [
    "apps/api"
  ],
  "scripts": {
    "postinstall": "npm exec -w apps/api prisma generate"
  }
}

I love Prisma, but it doesn't seem to work for me on pnpm, and I assume it also doesn't work on Yarn 2 for more or less the same reason.
Was it such a good idea to make the Prisma client to be generated on node_modules, assuming it can change depending on which package management you use? Maybe it wasn't obvious in the beginning that this would be a problem...

You can define your own output location with defining output in the generator block of Prisma Client if that helps - then Prisma Client is generated at that path @andrerpena.

After dealing with issues around GeneratorError: TypeError: collection is not iterable,  Ensure that there is only one instance of "graphql" in the node_modules directory., Error: Cannot find module 'strip-ansi and trying all sorts of option combinations with pnpm and keystonejs, @millsp ’s solution is the only one that worked.
@millsp is there a version released for prisma@3?

Hey @jlarmstrongiv this has not been released as it would cause many side-effects and breaking changes. Issues with automatic pnpm generation have not been fixed. However, if you just want to have multiple generated clients in your monorepo, you can have per-package generation folders (in their node_modules folder) instead of the root one. You can simply add this to your schema:
generator client {
  provider = "prisma-client-js"
  output = "../node_modules/.prisma/client" // <-- generate in your package node_modules, not the root
}

import { PrismaClient } from '.prisma/client' // <-- import the dot generated folder, not @prisma/client

Had to create a bit more complicated workaround because of a few issues.

import { PrismaClient } from '.prisma/client' not an option because I'm re-exporting prisma from a dedicated db package. Since there's no package.json in .prisma/client, browser packages using various derived types would pick up and try to bundle index.js, not index-browser.js.
output = "../node_modules/.prisma/client" sort of worked, but for some reason dist/*.d.ts files could resolve the paths and types, but not src/*.ts. Possible related to my personal TS and repo config, but who knows. I've spent over a week retooling my monorepo and trying different configs by now.

Anyway, here's my solution:

Generate and store client in the ./prisma folder of the package root. It's necessary because I'm wiping node_modules in CI after building all the dependencies. Prisma CLI won't be installed in production mode to generate it again.

generator client {
  // ...
  output = "./generated"
}

Create a client generation script and add it to package's package.json -> "postinstall": "./generate.sh"

#!/bin/sh

for arg in "$@"; do
  shift
  case "$arg" in
    "--rm") REMOVE_SOURCE="true" ;;
  esac
done

# always generate a fresh client if we have access to prisma cli
if [ -x "./node_modules/.bin/prisma" ]
then
  echo "Prisma: generating a fresh client"
  pnpm prisma generate
else
  echo "Prisma: skipping generation: no prisma CLI"
fi

# add the generated client to required locations.
if [ -d "./prisma/generated" ]
then
  echo "Prisma: installing .prisma/client..."
  # get the path of virtual store
  STORE=$(realpath<EMAIL_ADDRESS>  # remove possible autogenerated/old client and replace it with the properly generated one
  rm -rf ${STORE}/.prisma/client && cp -r ./prisma/generated ${STORE}/.prisma/client
  # make sure client folder exists in the project node_modules
  [ -d "./node_modules/.prisma" ] || mkdir ./node_modules/.prisma
  # remove existing link if exists and create a symlink to the store
  rm -rf ./node_modules/.prisma/client && ln -s ${STORE}/.prisma/client ./node_modules/.prisma/client
  # use env or --rm flag for cleanup in the final build stage to reduce image size
  [ "$REMOVE_SOURCE" = "true" ] && echo "Prisma: removing client source" && rm -rf ./prisma/generated
  echo "Prisma: done"
else
  echo "Prisma: skipping client copy: no client"
fi

Still happening

We are trying to run prisma with rushjs and having absolutely no luck.
The workspaces is failing the prisma generate.
Any news on this ?

Issue is still relevant.
I've solved it by installing different versions of prisma to each app that uses prisma:
apps/app-1:
- "@prisma/client": "4.13.0"
- "prisma": "4.13.0",

apps/app-2:
- "@prisma/client": "4.12.0"
- "prisma": "4.12.0",

It works, but obviously this solutions smells. Additionally, the worseness of it will grow along with apps that uses prisma...

Just ran into this issue today 😞

Hey everyone. pnpm maintains a cache of all installed dependencies so that it does not need to install them twice. With that, it also prevents a postinstall hook to run twice. This unfortunately breaks a core assumption. For now we suggest that you use this workaround, until we design a proper fix.

Since there's no package.json in .prisma/client, browser packages using various derived types would pick up and try to bundle index.js, not index-browser.js.

@vktrl I don't if that changed since but prisma generate does generate a package for me (wiht its own package.json)

same problem, This's my methods.

And PrismaClient imported use '@prisma/client', like this
import { PrismaClient } from "@prisma/client";

const prisma: PrismaClient = new PrismaClient();

Just ran into this issue today 😞 pnpm + turborepo

Encountered same issue. Any updates yet?

No updates yet, we will give you one when we have an update. This problem is not going to move forward in the short-term as it has wider implications. Please use the postinstall workaround for now.

Had to create a bit more complicated workaround because of a few issues.

import { PrismaClient } from '.prisma/client' not an option because I'm re-exporting prisma from a dedicated db package. Since there's no package.json in .prisma/client, browser packages using various derived types would pick up and try to bundle index.js, not index-browser.js.
output = "../node_modules/.prisma/client" sort of worked, but for some reason dist/*.d.ts files could resolve the paths and types, but not src/*.ts. Possible related to my personal TS and repo config, but who knows. I've spent over a week retooling my monorepo and trying different configs by now.

Edit: simplified solution - create a client generation script and add it to package postinstall:
#!/bin/sh

if [ -x "node_modules/prisma" ]
then
  pnpm prisma generate
else
  # for multistage docker build, dev deps won't be installed in production mode
  pnpm dlx $(pnpm list --depth 0 --dev --parseable | grep -o -m 1<EMAIL_ADDRESS>) generate
fi

# link to store from package node_modules because weird TS path resolution quirks
rm -rf node_modules/.prisma && mkdir node_modules/.prisma
ln -s $(realpath node_modules/@prisma/client/../../)/.prisma/client node_modules/.prisma/client
exit 0

hey, have you figured it out?


Added the first draft of the Jaxpr documentation.

This replaces the previous Google Doc version, and is now
updated with the latest changes in Jaxpr.

Thanks for doing this! I left some comments. I wonder, if you just want to get the jaxpr documentation in, if it's worth punting on the unrelated docstrings for now.

I did punt on most of the docstrings in this PR. But I do believe in docstrings and I think JAX code would be better with them. It is daunting to even start the effort, but we should do it.


Default systemd to fd:// as well as use upstream MountFlags=slave and LimitCORE=infinity

More information about systemd changes below, which will default docker host to fd:// among other unit file changes.
ExecStart=... -H fd://:

https://github.com/docker/docker/commit/076ac1d7d212c3a81a401a2fba8b9965bc26df16

LimitCORE=infinity:

https://github.com/docker/docker/commit/ae9cdbbde7824951b124a648e897ac4c291108e3
http://www.freedesktop.org/software/systemd/man/systemd.exec.html#LimitCPU=

MountFlags=slave:

https://github.com/docker/docker/commit/eb76cb2301fc883941bc4ca2d9ebc3a486ab8e0a
http://www.freedesktop.org/software/systemd/man/systemd.exec.html#MountFlags=

Regression passed locally, merging in preparation for 0.37.0 release.


Folder names prefixed with "."

Describe the bug
Folder names in this repo are prefixed with a period (".").
This is generally used for hidden folders, and as such causes issues in applications such as File Explorer when "Show Hidden Files and Folders" is disabled.
Is there a reason why the folder names are prefixed with a period as this seems to go against a generally accepted standard for naming conventions?
FYI - this is in reference to the DevOps Pipelines wiki page here:
https://github.com/azure/azops/wiki/azure-pipelines

Thanks for reaching out with your question about the .github, .vscode, and .devcontainer directories. It's great to see your engagement with the project!
To clarify, these folders are standard in many GitHub repositories and serve specific purposes:

.github: This folder typically contains GitHub-specific files like workflow templates for GitHub Actions, issue templates, and contribution guidelines. It helps in automating and streamlining various GitHub processes. For more details, you can check out GitHub's documentation here.

.vscode: This directory is used when the project is opened in Visual Studio Code (VSCode). It contains settings specific to the editor, such as code formatting rules, recommended extensions, and debug configurations. It ensures a consistent development environment for all contributors. More info can be found on the VSCode documentation page here.

.devcontainer: This folder holds configuration files for the Visual Studio Code Remote - Containers extension. It allows developers to work in a Docker container environment, ensuring that everyone has the same tools and dependencies, regardless of their local setup. You can learn more about it here.

These folders are not directly related to the core functionality of the project but are crucial for maintaining consistency and efficiency in development practices, especially in collaborative settings. If they are not relevant to your use case, you can typically ignore them without impacting the main functionality of the project.
Hope this helps! Let us know if you have any more questions.

This is actually related to the wiki page listed in the issue, and not the folder names you have listed, specifically the .pipelines & .scripts folders.


Error message displayed twice

For example, try to use a function that isn't defined. The error will be displayed twice momentarily, and then the duplicate will disappear after a moment.

It's interesting that the font size on each is different. I don't know why that would be.

This is because the some HTML and text is being created in the comment function of Comments object of the IDE and in the DocumentCommentSpan in Spans.ts. The Spans seems to be the more modern and is the first to appear. When the render fundtion gets called on the IDE object it drills through and calls the render function of the Comments object and injects the new DOM objects as children into the DocumentCommentSpan.
Relevant lines are around IDE and
Span
Regards,
Cormac.


In-game overlay should start with the size of previous session?

When starting Arena, the in-game overlay is tiny small and needs to be enlarged first. It takes a lot of clicks to enlarge the overlay. I would be great if it would remember the size (and transparency settings) of the last session.

Thank you for your message. We will investigate your suggestion

I'm also having this issue.  I run the game fullscreen at 1440p and the overlay starts out so tiny I almost can't even click the resize buttons.  Having to do this every time is a pain!


Adds skeleton for fungible asset queries

Description
see title
Test Plan
pnpm test

Moving the content to new PR - https://github.com/aptos-labs/aptos-ts-sdk/pull/73
Closing this repo now


Build tensorflow 2.2

I will like to build a tensorflow 2.2 image on a Mac. However, it seems that the docker build does not seems to end. Is it possible to find a whl pre-compiled?

I'm afraid that v2.2 is too old to find a prebuilt wheel. You might need to compile it yourself.
Note that you are probably using an Apple M1 Mac, so I would strongly advise you to try the official wheels for M1 first.


[Suggestion]: Height configuration

Suggestion (Required)
hey, I was playing with a setting which allowed higher than normal world Y levels and found myself wanting to change the height settings of thin air to allow high level to be breathable but leave low level unbreathable tho realized its not something I could modify or tweak.
Love the idea of the mod btw!

Check the dimensions entry in the config, specifically the entry for minecraft:overworld, it demonstrates you you can define specific air quality ranges.


EdjCase.Cryptography.BLS not found on nuget

https://www.nuget.org/packages/EdjCase.Cryptography.BLS/

Ya, currently it's sitting in limbo waiting on a crypto expert to help me figure out what isn't working.

Is this causing an error right now? I guess I didn't consider that. Maybe i can upload a dummy one

@kentsday
Alright. Try 2.0.7
https://www.nuget.org/packages/EdjCase.Cryptography.BLS/2.0.7

thx, it works!


Samsung Dex Support missing

App works well, but only in very small window. When trying to make window bigger (or turn on fullscreen) - got error "No connected servers" (not sure about english translation, I have russian interface, on russian sentence is "На данный момент нет подключённых серверов")
UPD: fixed with turn on DexLabs feature "Force apps that not support dex to resize" or somthing like that...

Having the same issue. Some minor corrections:
1.) The error message is: "No hosts currently connected".
2.) DeX Labs feature "Force apps to resize" doesn't solve this, i.e. behavior is consistent and unrelated to this setting.
3.) Only way to use Connectbot on DeX is to start at (small) windowed size, and then manually resize it to larger. Manually = by grabbing its corner with the mouse. This way I could resize it to whole screen.
BUT! If I used the "maximise" button on the window's top bar, it ruined the display.

I do not see the same problem, that other posters have. Particularly no error messages. However I have noticed that in some cases, if I attach to an already running GNU Screen session, terminal sizes and/or cursor navigation gets confused. Mainly I'm noticing vi does not refresh lines properly (if started inside the GNU screen session on Connectbot over DeX). GS20 and DeX v<IP_ADDRESS>, if that matters.
Also: Another thing that does not seem to work well under DeX: If I highlight text with mouse, then attempt right-click so I can copy to clipboard, the selection is lost, and thus the "copy" menu option unavailable. So I don't seem to have a way to copy text.
There is a "Paste" button on the toolbar, perhaps that could be given a long-press alternate action of "copy selected text".


[Discussion]: DESIGN PRINCIPLE: Maintaining public trust

Summary
Propose a principle around public trust to run across the architecture
Source
No response
Detail
This aligns with the DARE federation paper and the SATRE work on public engagement. That openness are key to public trust and this should be considered across as part of the design across the entire TRE architecture.
This should include a statement, rationale and implications.
Intended Output
No response
Who can help
@harisood - can you tag Katie? I don't know her username

@katie-rds! This info is also available on the Teams in Files -> Resources -> Team details

How much does this overlap with https://github.com/sa-tre/satre-specification/issues/101 ?

First attempt from @Katie-RDS:
TREs should build and maintain public trust by protecting privacy, keeping data secure and being transparent about their work.
Rationale
Public engagement work has indicated there is public support for TRE’s and the use of data to improve services and advance research as long as it is regulated, ethical and in the public benefit.  Maintaining public trust in TRE’s to hold and use data is essential to prevent public backlash or resistance to the use of data. Involving members of the public in the decision making of the TRE help ensure the TRE is acting in public good. Being transparent about the data held and the projects or organisations who access the data helps maintain that trust.
Implications

TREs should follow the Five Safes framework and make their adherence to the framework publicly accessible information.
TREs should allow access to data for projects that are in the public benefit.
TREs should publish information on the projects or organisations which access data held within the TRE.
Public Involvement and Engagement activities should be carried out and inform the work of the TREs.

Could we change the first line to
TREs holding public data should build and maintain public trust by protecting privacy, keeping data secure and being transparent about their work.
and make the implications conditional on holding public data?

Could we change the first line to TREs holding public data should build and maintain public trust by protecting privacy, keeping data secure and being transparent about their work. and make the implications conditional on holding public data?

I think the implications are definitely true for data held about members of the public so I'm happy with this edit. I think it would be interesting to discuss around how much needs to apply to other types of data?

Thoughts and updates from the Collaboration Café:
Statement
TREs should build and maintain the trust of data subjects and any other impacted individuals, groups and organisations by protecting privacy, keeping data secure and being transparent about their work.

This should be broadened out to communities and society

## Rationale
Maintaining trust in TREs to hold and use data is essential to prevent backlash or resistance to the use of data. This can include maintaining the trust of members of the public who are impacted by research, as well as the trust of commercial data providers.
In the case of public sector data, public engagement work has indicated there is widespread support for the use of regulated and ethical TREs working for the public benefit. Consulting impacted parties, including the public, can help show that a TRE is acting in their benefit. Being transparent about the data held and the projects or organisations who access the data can also help maintain trust.

can help show that a TRE is acting in their benefit

This phrasing makes it sound performative rather than emphasising they should have real impact. Could it be updated to 'can help ensure TREs are being used for positive, impactful and agreed-upon purposes'?

Implications (additional ones)

TREs should make information on the projects or organisations which access their data available to impacted parties
Access to public sector data should be reviewed by a panel and follow governance to ensure projects using this data are in the public benefit and provide clarity around commercial access
TREs should should be clear on infrastructure including ingress/accreditation
TREs should have processes in place for rigorously investigating and reporting incidents such as data leaks including processes required by the Information Commissioner’s Office

impacted parties
and publicly


Refactor tabs to fix duplicate ID bug (FP 453934)

Summary of Changes
This PR is to resolve FP 453934 opened by Student Experience. No backend changes were necessary.

Add drupal_id to GraphQL tabs widget query in basic-page.js
Use drupal_id for tabID in pageTabs.js to ensure each tab ID is unique
Clean up and refactor pageTabs.js code
Remove unneeded variables and stylesheet

Test Plan
Compare https://ugconthub-tabfix.gtsb.io/studentexperience/intlstudent/quarantine with https://www.uoguelph.ca/studentexperience/intlstudent/quarantine - on the live site, tab content fails to change because of duplicate tab IDs. On the PR build site, each tab should work properly

@mmafe Is this the sort of thing Backstop could test? Not sure since it involves interacting with the tabs


When a user hides an activity bar item, should it re-show based on conditional view contributions?

VSCode Version: 1.25.1
OS Version: Windows 10 x64

Originally opened here: https://github.com/eamodio/vscode-gitlens/issues/447
Steps to Reproduce:

Right-click on the GitLens activity bar icon and choose Hide
Restart vscode
The GitLens activity bar icon will re-appear

Does this issue occur when all extensions are disabled?: No

@sandy081 Related with this -- each time I restart vscode, the GitLens activity bar icon always gets its position reset to the bottom of the bar. I'm not exactly sure why this is the case, other than the fact that my views are conditionally attached to the view container.

Could be related to conditional enablement.


this.elem.summernote is not a function

After Copy-Paste code into my code got following error inside console.
app.js:77229 Uncaught TypeError: this.elem.summernote is not a function
   at Proxy.mounted (app.js:77229)
   at callWithErrorHandling (app.js:16877)
   at callWithAsyncErrorHandling (app.js:16886)
   at Array.hook.__weh.hook.__weh (app.js:12146)
   at flushPostFlushCbs (app.js:17075)
   at render (app.js:15101)
   at mount (app.js:13350)
   at Object.app.mount (app.js:19595)
   at Module../resources/js/app.js (app.js:26127)
   at __webpack_require__ (app.js:84423)

Nothing to say more, the most misticial thing in summernote is library import method. How to do it in proper way?
import $ from "jquery";
window.$ = window.jQuery = require('jquery');
import 'summernote';

OR

import $ from 'jquery';
import 'popper.js';
import 'bootstrap';
import 'bootstrap/dist/css/bootstrap.min.css';

import CodeMirror from 'codemirror/lib/codemirror';
import 'codemirror/lib/codemirror.css';
import 'codemirror/mode/xml/xml';
import 'codemirror/theme/monokai.css'

import 'summernote/dist/summernote-bs4';
import 'summernote/dist/summernote-bs4.css';

and webpack
module.exports = {
    plugins: [
        new webpack.ProvidePlugin({
            $: "jquery",
            jQuery: "jquery",
            jquery: "jquery",
        }),
    ]
}

any suggestions?
laravel-mix / Vue 3.2.26

Hi @musznik, here I import using require. require('summernote'), I think import 'summernote'; do the same thing. Look for another errors on console, jquery must be present and summernote too.

Hi, i see you’re using laravel-mix and vue3, i had the same problem with symfony and vue3. I solved the problem by adding an alias for jquery (jQuery -> jquery).
Exemple alias for encore webpack | Symfony :
.addAliases({         jQuery: path.resolve(__dirname, "node_modules/jquery")     })
ref: https://stackoverflow.com/a/69619307/11027923

Sorry to open this thread up again, exacaly the same issue as above.

vendor.js?id=778d0042e1eae3746081c1a26dc68486:106439 Uncaught (in promise) TypeError: this.elem.summernote is not a function
at Proxy.mounted (vendor.js?id=778d0042e1eae3746081c1a26dc68486:106439:15)
at callWithErrorHandling (vendor.js?id=778d0042e1eae3746081c1a26dc68486:10258:22)
at callWithAsyncErrorHandling (vendor.js?id=778d0042e1eae3746081c1a26dc68486:10267:21)
at hook.__weh.hook.__weh (vendor.js?id=778d0042e1eae3746081c1a26dc68486:12787:29)
at flushPostFlushCbs (vendor.js?id=778d0042e1eae3746081c1a26dc68486:10458:47)
at render (vendor.js?id=778d0042e1eae3746081c1a26dc68486:16291:9)
at mount (vendor.js?id=778d0042e1eae3746081c1a26dc68486:14497:25)
at app.mount (vendor.js?id=778d0042e1eae3746081c1a26dc68486:19685:23)
at setup (app.js?id=3a2739457665ce0eb701f151b3ed0005:957:174)
at vendor.js?id=778d0042e1eae3746081c1a26dc68486:9:9711

jQuery is installed and seems to be aliased fine for my other plugins that use it,

import $ from 'jquery';
window.$ = window.jQuery = $;

When attempting to use the SummernoteEditor component however I get the above. I note the statement about importing summernote for yourself but the inclusion of

import "summernote"

throws a compile error due to multiple references.
Trying to work this through with vue but could do with additional requirements that this requires.

Not resolve yet


app navigation component

New component that binds to the model and generates a tree that can be passed to the nave widget described in #6

it is now part of master.


Adding coverage target. Current coverage is 30%-36%

Codecov Report

Merging #82 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #82   +/-   ##
=======================================
  Coverage   59.76%   59.76%
=======================================
  Files           9        9
  Lines         778      778
  Branches      135      135
=======================================
  Hits          465      465
  Misses        206      206
  Partials      107      107

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9ec16f...7db60c0. Read the comment docs.


Cannot use moneta as Rails Session Store

Hi,
I've tried to configure moneta as session store for my rails application.
First I used the (3)way the documentation suggests:
# Only by adapter name
config.cache_store :moneta_store, store: :Memory

# Use Moneta.new
config.cache_store :moneta_store, store: Moneta.new(:Memory)

# Use the Moneta builder
config.cache_store :moneta_store, store: Moneta.build do
  use :Expires
  adapter :Memory
end

None of it worked and failed with this error:
gems/moneta-0.8.0/lib/moneta/builder.rb:7:in `initialize': No block given (ArgumentError)
Then I tried:
config.session_store :moneta_store, store: :MongoOfficial

This time it failed with:
ruby-2.2.2/gems/railties-4.2.2/lib/rails/application/configuration.rb:176:in `const_get': uninitialized constant ActionDispatch::Session::MonetaStore (NameError)
I made anything wrong?
My system:
Ruby 2.2.2
Rails 4.2.2
Mongo 1.12.3
regards,
Giuliano

Yeah, I could reproduce this issue, with this exact configuration.

Same here!

I think you have to use
require 'action_dispatch/middleware/session/moneta_store'


Refs 4478: Revert 'support cancel pending tasks'

This reverts commit 73360fc7193296b8c8d5100d8fb69ebfa7f6f72e.
Summary
Testing steps

⚠ This task currently requires qe-approval, but this PR does not fully resolve the task.  Please contact QE to determine appropriate testing required.


#299: allowAnyUpdates should be ignored

This might be seen as a breaking change, but I think it is in line with what most users expect to happen anyway.
So, this change sanctions these expectations -- allowAnyUpdates shall effectively assume the value of false if any of the other modifies are set to false.
Actually, this means that it's redundant, but the fact is that it's getting removed in version 3.0 anyway.
@slawekjaranowski what is your opinion?

Looks like this is also in line with what users expect – see the latest comments in #299.

While working on the refactoring job, I've discovered a bug around allowAnyUpdates: if allowAnyUpdates is false and all other allows are true, the result differs from when allowAnyUpdates is true.
In the first case, the updateScope is Optional.of(MAJOR), in the second case (with allowAnyUpdates), it's Optional.empty().
Strange that we haven't found it with our tests. Well, I'm fixing this now (we'll get rid of allowAnyUpdates), but this will take some time.

Looks more complicated ... so I have proposition to postpone this and related issues to next release.

Actually, I'd close this PR and I would postpone #961.
On the other hand, #960 is important and almost ready.

On the other hand, https://github.com/mojohaus/versions/issues/960 is important and almost ready.

Not ready, I see that integration tests are failing, so there's more work needed. Also, I need to see if this is restricted to display-dependency-updates. So, let's postponse that one too.


(Fix)Deprecate PrettyState in python API

Issue: https://github.com/nmstate/nmstate/issues/2588
Description;
The PrettyState is only used for test codes.

[x] Mark PrettyState as deprecated in Python API, providing message indicate libnmstate.show() output is already sorted.
[x] Remove test cases regarding PrettyState

Hi! Instead of creating multiple commits could you unify all the changes into a single one? In addition, instead of doing merges to update the branch, please do rebases. The git history will be more clear

cleaning this up, sorry!

It looks much better but the commit message it reflects messages from other commits. I guess this is because you squashed the commits. Please, adjust the commit message to reflect only the relevant information of this task :)

updated, thanks

LGTM! Let's wait for CI results before merging it

Whoops, the linter is failing. Could you follow the formatting suggestion? https://github.com/nmstate/nmstate/actions/runs/8202211120/job/22432495995?pr=2591#step:3:436
 class PrettyState:
     def __init__(self, state):
         yaml.add_representer(dict, represent_dict)
-        warnings.warn("The PrettyState class is deprecated and will be removed in a future release. "
-                      "The output of libnmstate.show() is already sorted.", DeprecationWarning)
+        warnings.warn(
+            "The PrettyState class is deprecated and will be removed in a future release. "
+            "The output of libnmstate.show() is already sorted.",
+            DeprecationWarning,
+        )
         self.state = _sort_with_priority(state)

the linter is quite annoying!

the linter is quite annoying!

It might be but it is important to take care of the style used when coding. An unified and clean style leads to a codebase easier to read and understand.
There is one more lint error, you are almost there!
 +
 class PrettyState:
     def __init__(self, state):
         yaml.add_representer(dict, represent_dict)
         warnings.warn(
             "The PrettyState class is deprecated and will be removed in a future release. "

All good, merging!

According to https://peps.python.org/pep-0702/  we should use @warnings.deprecated().
@jona42-ui Can you investigate and fix it in next PR?

thanks makes sense

can i make a followup PR with only the decorator fix?


Check how things are looking on diff browsers

Georgia mentioned there might be an issue, but she isn't 100% sure what it is. We should check if things are rendering ok on Firefox

IE11
Home Page

[ ] not showing tangram map

Location Page

[ ] lines get real scribbly on hover / initial render in line chart and small mults
[ ] small mults don't update on change metric
[ ] changing time fails to load data for hour chart, time series and small mults

Compare Page

lines get less scribbly here... weird.

There was a bug in d3-interpolate-path with IE that has now been fixed in v1.0.2 which fixes the scribbly lines.


Update README's Repo structure diagram (e.g., incorrectly lists "openwhisk-swift")

Environment details:

local deployment, vagrant, native ubuntu, Mac OS, Bluemix, ...
version of docker, vagrant, ubuntu, ...

Steps to reproduce the issue:

View README

Provide the expected results and outputs:
Fix diagram to remove "openwhisk-swift" as a project repo.

Provide the actual results and outputs:
N/A

Additional information you deem important:

issue happens only occasionally or under certain circumstances
changes you did or observed in the environment

If we are trying to reflect the Swift Client SDK repo "https://github.com/apache/incubator-openwhisk-client-swift" then we should clarify.

If we are going to update the "green box" component / repo. diagram on the main readme, we should have it reflect the most current set of repos. and include new ones like "wskdebug".

I'll take this issue

Thanks @modlanglais 🙏

Thanks @modlanglais


[LOCAL] Set kotlin.jvm.target.validation.mode=warning on user projects

Summary:
This is a fix for https://github.com/reactwg/react-native-releases/discussions/54#discussioncomment-5984629
It sets kotlin.jvm.target.validation.mode=warning so that projects won't break if they're using Kotlin 1.8+ and have JDK versions misaligned due to external libraries.
This doens't need to be backported to main as on main we set jvmToolchain(11) for all the libraries so this flag is unnecessary.
Also this is not a .0 blocker and can land in .1
Also this requires a bump of RNGP
Changelog:
[INTERNAL] - Set kotlin.jvm.target.validation.mode=warning on user projects
Test Plan:
Tested locally with ./gradlew build.

@cortinico yes - we had to clean up the branch ahead of 0.72.0. I'll take care of manually reapplying in the branch

@cortinico yes - we had to clean up the branch ahead of 0.72.0. I'll take care of manually reapplying in the branch once we start working on 0.72.1

In the future, I would suggest we avoid to force push on release branches (we can add a branch protection against force push, for security reasons also) and instead just add a revert commit.


[BUG]  Nightly Test: test_deleting_backup_volume  sometimes failed

Describe the bug
sometimes test_deleting_backup_volume test failed
To Reproduce
Install Longhorn master
Run test_deleting_backup_volume
Expected behavior
test should pass
Log
client = <longhorn.Client object at 0x7f624a44e550>

    def test_deleting_backup_volume(client):  # NOQA
        """
        Test deleting backup volumes

        1. Create volume and create backup
        2.

[ ] Is the backend code merged (Manager, Engine, Instance Manager, BackupStore etc)?
The PR is at longhorn-tests#372

[x] Is the reproduce steps/test steps documented?

[x] Which areas/issues this PR might have potential impacts on?
Area backup tests
Issues #710 #1604 #1603

[x] If the fix introduces the code for backward compatibility Has an separate issue been filed with the label release/obsolete-compatibility?
The compatibility issue is filed at

[x] If labeled: area/ui Has the UI issue filed or ready to be merged?
The UI issue/PR is at

[x] if labeled: require/doc Has the necessary document PR submitted or merged?
The Doc issue/PR is at

[x] If labeled: require/automation-e2e Has the end-to-end test plan been merged? Have QAs agreed on the automation test case?
The automation skeleton PR is at
The automation test case PR is at

[x] if labeled: require/automation-engine Has the engine integration test been merged?
The engine automation PR is at

[x] if labeled: require/manual-test-plan Has the manual test plan been documented?
The updated manual test plan is at

Validation - Passed
The test test_deleting_backup_volume is passing in nightly integration test.


Application isn't notarized

In modern versions of macOS, applications that aren't signed and notarised with a recognised Apple developer ID are deemed untrusted and require manual interaction from "Privacy and Security" in order to launch.
This application seems to be signed, but not notarised which means it won't run on off-the shelf installations.
Since this is an operation that costs money, I understand that it isn't reasonable for what seems to be an abandoned project to implement in releases, but I thought I'd mention it anyways.

I don't want to pay Apple a subscription to get rid of that for this app. If there were a free way like Fulcio, maybe, but I don't think Apple will ever allow something like that.


New Digital examples have "_examples" in model name

Reported by otter on 10 Oct 2010 10:34 UTC
Several new examples have been added to Electrical.Digital.Examples. These examples have names such as "DFFREG_example".
This is not the style in MSL, to repeat the package name in the class name. "_example" should be removed

Migrated-From: https://trac.modelica.org/Modelica/ticket/426

Comment by otter on 10 Oct 2010 10:36 UTC
Fixed in f1d58fa264944c359c237213eaab8437dbcf041d

Modified by dietmarw on 19 Oct 2010 06:17 UTC


Introduce --read_checked_file option

This PR introduces an option to make F* read a checked file and print it out (the tc_module in it, actually). It does not involve any include path, or finding the corresponding file, or parsing: it just reads whatever file the user passes in and prints it to standard output.
$ ./bin/fstar.exe --read_checked_file ulib/.cache/prims.fst.checked
module Prims
Declarations: [
assume new
val attribute:Type0
assume
val cps:Prims.attribute
assume
val hasEq (_: Type) : Prims.GTot Type0
let eqtype = a: Type0{Prims.hasEq a}
assume new
val bool:Prims.eqtype
type empty =
type trivial = | T : Prims.trivial

assume new
val unit:Prims.eqtype
let squash p = x: Prims.unit{p}
let auto_squash p = Prims.squash p
private
let logical = Type0
[...]

It would be nicer if we improve the format of modul_to_string, since it currently displays everything twice. See this comment:
(*
 * AR: 07/19: exports is redundant, keeping it here until vale is fixed to not parse it
 *)
let modul_to_string (m:modul) =
  U.format3 "module %s\nDeclarations: [\n%s\n]\nExports: [\n%s\n]\n" (sli m.name)
                                                                     (List.map sigelt_to_string m.declarations |> String.concat "\n")
                                                                     (List.map sigelt_to_string m.declarations |> String.concat "\n")

I'm merging this since it could be useful. We can also consider printing other pieces of the checked file (deps, hashes, etc).


Auto toggle on blur background when joining in Google Meet

@anshulahuja98 I think by default the blur background option is on

@DarshitJain04 it isn't

@DarshitJain04 it isn't

Oh is it? Because whenever a link opens for me, the blur background is on by default

@DarshitJain04 Does the blurred background issue still persist?

@DarshitJain04 Does the blurred background issue still persist?

Yes, the issue is still open. But as I mentioned earlier, whenever I open any meeting, the blur option is already on for me, which is not the case with Anshul. So I'm not sure if this is some issue with Google Meet itself.

@DarshitJain04 can you try the same in another browser or incognito window?

@DarshitJain04 Did you try clearing the cookies and the cache in the browser that you are using?

@DarshitJain04 Did you try clearing the cookies and the cache in the browser that you are using?

I confirmed it with some more people. They said that if we select the blur background option once, the same setting persists for other links also.
So I think it's a one time issue. There's no need to add another few lines of code for this. So I'm closing this issue.
Anyways, thanks for showing interest to work on this issue✌


[feature-request]: Make it very clear why we are extracting metadata at this stage

What feature would you like to request?
This could be a good point to include a reference to (internal/online) docs that list the explanation, rather than cluttering the UI with text on reasons. But maybe a small text somewhere explaining the steps taken here.
Optional: Suggest a solution
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

We decided to start the web app from scratch after the major design/architecture decisions and changes we made. No longer relevant.


Add VBDec

VBDec is a amazing tool for Visual Basic P-Code disassembling and debugging, it's free and very powerful!
http://sandsprite.com/vbdec/

no native?

Any ideas on how to automate VBDec's installation? I could not a find way do to so... :(

Any ideas on how to automate VBDec's installation? I could not a find way do to so... :(

repack it i guess?

Yeah it should be possible, but I don't have the time to do it at this point. If someone sends a PR, I'll be more than happy to add it. 🙂

I found this program on the web https://www.silentinstall.org/
seems quite effective for programs that don't support /s or other parameters for silent installs

I didn't have time to work on this. Will try to do something for the next release. I'll keep the issue open. Thanks!

Exeinfo Pe  - detect :  Silent Install Builder v.6.2.5/6  used ( NSIS Installer 3.0.x )


feat: Filter challenges by organization

Closes #1035
Closes #1402
Closes #1421

TODO

Challenge Search

Filter by Organization

[x] Get the org avatar URL using the new ImageService.getImage() (@rrchai )
[x] Fix the bug shown in the GIF below (@rrchai )

Changelog

#1401
#1420
#1423

Preview
Filter challenges by org

@rrchai I merged main into this feature branch, which adds DREAM as a contributor of more challenges.

Target deadline: June 12

@rrchai Something probably broken when I merged the conflict. Also, the logo of the organizations does not show in the widget because I commented out the avatar property. This property needs to be restored and the value should come from ImageService.getImage() as discussed on Slack.

CONFIRMED: Challenges can be filtered by one or more organizations.
GET {{basePath}}/challenges?organizations=1

HTTP/1.1 200
{
  "number": 0,
  "size": 100,
  "totalElements": 47,
  "totalPages": 1,
  "hasNext": false,
  "hasPrevious": false,
  "challenges": [
    {
      "id": 11,
      "slug": "broad-dream-gene-essentiality-prediction",
      "name": "Broad-DREAM Gene Essentiality Prediction",
      "headline": "",
      "description": "The goal of this project is to use a crowd-based competition to develop predictive models that can infer gene dependency scores in cancer cells (genes that are essential to cancer cell viability when suppressed) using features of those cell lines. An additional goal is to find a small set of biomarkers (gene expression, copy number, and mutation features) that can best predict a single gene or set of genes.",
      "status": "completed",
      "difficulty": "intermediate",
      "platform": {
        "id": 1,
        "slug": "synapse",
        "name": "Synapse"
      },
      "websiteUrl": "https://www.synapse.org/#!Synapse:syn2384331",
      "avatarUrl": null,
      "incentives": [
        "publication",
        "speaking_engagement"
      ],
      "submissionTypes": [
        "prediction_file"
      ],
      "inputDataTypes": [],
      "startDate": "2014-06-02",
      "endDate": "2014-09-29",
      "starredCount": 0,
      "createdAt": "2023-06-06T01:54:52Z",
      "updatedAt": "2023-06-06T01:54:52Z"
    },
    {
      "id": 55,
      "slug": "cough-diagnostic-algorithm-for-tuberculosis",
      "name": "COugh Diagnostic Algorithm for Tuberculosis",
...

The avatars of orgs in dropdown are retrieved using imageService. I borrow @tschaffter code used in org-search page. I notice there are a little bit delay (2-3s) to get all orgs. For example, when the challenge-search page is loaded, users will not see options in the organization dropdown until the orgs including avatars are all queried in 2-3s.
It's not a big deal now, but may cause longer delays with more orgs added into the database in the future.

More descriptions on the unexpected behavior reported by @tschaffter (shown in gif above):
When select the any option in organization dropdown, it will also select the rest of options. After troubleshooting with @tschaffter , we found the reason is because the angular client for organization service is not update-to-date. It's actually missing some chunks of code for orgId, which results in that the orgId of all organizations are returned as null. Since the dropdown is using orgId as filter value, selecting any of options (null) will select all options.
To resolve this issue, simply updating angular client for organization service with below code:
nx openapi-generate openchallenges-organization-service

Some steps we did while trying to find the reason:

checked the type of orgId in data-models, Organization.yaml
found the mismatched types of orgId in .sql file -> change the string orgId to integer
checked the orgId type in OrganizationDto.java, OrganizationEntity.java (whether it's Long type
checked the orgId type in schemas, V1.0.0__create_tables.sql
debugged organization service in dev mode:

launch the service in dev mode by nx serve openchallenges-organization-service
print out entity in OrganizationService.java by adding LOG.info("{}", org}
see the log by sending a request in Swagger

Steps to update organization data and organization service, if changes:

stop and remove organization service container (click "remove" in docker extension)
rebuild the image by nx build-image openchallenges-organization-service
restart the container: nx serve-detach openchallenges-app

@tschaffter The issues in this pr should be fixed.

For example, when the challenge-search page is loaded, users will not see options in the organization dropdown until the orgs including avatars are all queried in 2-3s.

What is the size in pixel of the org logo displayed in the dropdown? Is it smaller than the 100x100 px images requested from the image service? Downloading the image at the right size - after modified the image service to allow this size - should lead to an improvement but won't solve the overall performance issue discussed previously.

To resolve this issue, simply updating angular client for organization service with below code:
nx openapi-generate openchallenges-organization-service

@rrchai This command update the organization (micro)service, not the API client for Angular.

What is the size in pixel of the org logo displayed in the dropdown? Is it smaller than the 100x100 px images requested from the image service?

@tschaffter I used 100px, which is the smallest option from ImageHeight.

This command update the organization (micro)service, not the API client for Angular.

Ah, right.  Thanks for clarification.

@tschaffter I used 100px, which is the smallest option from ImageHeight.

But what is the size of the displayed images in the dropdown?

What is the size of the displayed images in the dropdown?

32px

@rrchai The PR shown above this comment will enable the app to request 32x32 logos.

I can't technically approve the PR because I created this feature branch, which we used to group multiple features branches together before merging to main. I've reviewed this feature that includes the latest contributions from @rrchai and approve it.


introduce importer command for setting up the initial config data.

See https://github.com/jmuk/istio/commit/fe0073b9b34c452ea25550d2d26659d0f99a223a for an example of using this tool.

This change is

Jenkins job mixer/presubmit passed

Can we expand on the documentation here (and in the code)? I'd like some write-up of the use case and explanation of the why to persist -- perhaps even in an issue against this repo.

Jenkins job mixer/manager-regression passed

Jenkins job mixer/e2e-smoketest passed

I'd prefer to see this written using Cobra like the other tools in the cmd dir.


[WIP] Handle unwanted props with React 16

Add test with unwanted prop. Related to #167

Fixed in #169


"Start review" with an empty comment breaks GUI

Gitea version (or commit ref): Verified on 1.14.2 and 1.15.0+dev-377-g894742278
Git version: 2.24.1
Operating system: Centos

Database (use [x]):

[ ] PostgreSQL
[X] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[X] Yes (provide example URL): https://try.gitea.io/cvennel/Test_gui_bug/pulls/1/files
[ ] No

Log gist:

Description

Clicking "Start review" with an empty comment seems like it should refresh the screen and display an error message, however the new screen is being shown in a smaller window where the new review comment should be. See screen capture below:
Screenshots
https://user-images.githubusercontent.com/35314989/120482896-94d23880-c37f-11eb-958a-fcfaa4efa38f.mp4

Hm, should we allow empty comment when click start review?

Hm, should we allow empty comment when click start review?

Not sure if you're asking for an outside opinion or not but personally I don't see the value of being able to start a review with an empty comment. I would think ideally the start review button and likely also the single comment button would be disabled until there is content in the comment box. This would be similar to how the new pull request button is grayed out when not signed in.
Alternatively, I think the existing refresh and show the error message at the top would be more acceptable if it actually refreshed the entire page. The down-side to this is if you are reviewing a large PR and accidentally click the start button it's annoying to have to scroll back to where you were last reviewing.

I would like to work on it. Actually, I would implement it as @cvennel suggested.

My first idea was to use a vue component and wrap it around the editor and the button. Unfortunately, the vue component is not rendered, because the comment_form.tmpl will be loaded after vue was initialised.
Does anyone has a different idea?
Otherwise there is a lot more to do like a refactoring of how the html is loaded.

@jannaahs I haven't looked too far into this, but it seems like right around here the javascript handles graying out the commit button on file edits if the commit message is empty or there are no file changes. Could that sort of thing be expanded for this situation or does that fall under where you were saying "there's a lot more to do."

@cvennel Yeah, I think it is not that easy.
While doing a code review you might have open more then one form. Committing files is a bit different, because there is only one form open at the same time.
In case you have two forms open, the buttons should only be disabled if the corresponding input is empty. Currently there is no mechanism in place to have the editor and the buttons in their own scope.
Implementing a own scope for each form is what I meant by "there's a lot more to do". At the moment I am not sure what would be a good solution to add a scope. Like I said a vue component would provide a scope, but is not possible due to the template being loaded after vue was initilized.
Maybe I didn't understood the initEditor you mentioned, but I don't see a quick and easy solution.

Same issue as #18151, fixed by #18156


Add Symfony 3 support

+1 feel free to merge.

@dantleech shall we still merge this? It contains a typehint change, so a BC break if anyone wrote a custom TokenProvider. I'm not sure if they do and if they do, I think it's a very easy fix to make things work with the new version.

I know some people have made custom token providers. I would rather not introduce BC breaks, it still means that if people have done ~1.0 expecting not to get BC breaks, then we potentially break their installation. Do we promise BC?
Saying that, if you think introducing Symfony 3.0 here is worth it then lets do it. It is a trivial change and if people wrote their own TokenProvider's then they should now how to quickly fix them :/ Otherwise we can wait until 2.0.

Hmm, let's skip this for 2.0. The bundle won't support Symfony 3 until 2.0 and Sulu does not support Symfony 3 either. Those are the only 2 projects referencing this package and I don't think many people use it in their custom build apps as a standalone component.

What about the slugifier? Shall we put that in Wednesdays release?
On Tue, Feb 09, 2016 at 04:49:46AM -0800, Wouter J wrote:

Hmm, let's skip this for 2.0. The bundle won't support Symfony 3 until 2.0
and Sulu does not support Symfony 3 either. Those are the only 2 projects
referencing this package and I don't think many people use it in their
custom build apps as a standalone component.
—
Reply to this email directly or [1]view it on GitHub.
Reverse link: [2]unknown
References
Visible links

https://github.com/symfony-cmf/RoutingAuto/pull/65#issuecomment-181850824
https://github.com/symfony-cmf/RoutingAuto/pull/65#issuecomment-181850824

Dropped Symfony <2.8 support. This PR is now ready to be merged, @dantleech

Cheers @WouterJ


Multi-line basic string trims empty lines

From spec:

Multi-line basic strings are surrounded by three quotation marks on each side and allow newlines. A newline immediately following the opening delimiter will be trimmed. All other whitespace and newline characters remain intact.

Test toml:
str1 = """
Roses are red

Violets are blue"""

Result:
>>> toml.load('test.toml')['str1']
'Roses are red\nViolets are blue'

Expected result:
>>> toml.load('test.toml')['str1']
'Roses are red\n\nViolets are blue'

Hi,
Thank you for the bug report.
Can you provide enough information so that I can ascertain whether this is a duplicate of #113, #123,  #119, #116, #80, or #68 ?
Closing as duplicate.


Add histogram validation

This is to the sparsehistogram branch
This PR implements the following validation checks:

Do any of the histogram's spans have a negative offset
Are there the appropriate number of negative and positive buckets based on their respective spans

Performance impact of the new checks looks pretty negligible I think (varies a little though between runs):
name                    old time/op    new time/op    delta
HistogramValidation-10    1.10µs ± 0%    1.54µs ± 0%   ~     (p=1.000 n=1+1)

Don't know why TestSparseHistogram_Sum_AddOperator is complaining about not getting the expected result now, the math checks out I think:
diff --git a/promql/engine_test.go b/promql/engine_test.go
index 472a43b16..1c6533527 100644
--- a/promql/engine_test.go
+++ b/promql/engine_test.go
@@ -3914,7 +3914,7 @@ func TestSparseHistogram_Sum_AddOperator(t *testing.T) {
                                },
                                {
                                        Schema:        0,
-                                       Count:         15,
+                                       Count:         36,
                                        Sum:           2345.6,
                                        ZeroThreshold: 0.001,
                                        ZeroCount:     5,
@@ -3933,7 +3933,7 @@ func TestSparseHistogram_Sum_AddOperator(t *testing.T) {
                                },
                                {
                                        Schema:        0,
-                                       Count:         15,
+                                       Count:         36,
                                        Sum:           1111.1,
                                        ZeroThreshold: 0.001,
                                        ZeroCount:     5,
@@ -3955,7 +3955,7 @@ func TestSparseHistogram_Sum_AddOperator(t *testing.T) {
                                Schema:        0,
                                ZeroThreshold: 0.001,
                                ZeroCount:     14,
-                               Count:         39,
+                               Count:         81,
                                Sum:           4691.2,
                                PositiveSpans: []histogram.Span{
                                        {Offset: 0, Length: 3},

We are doing the error check wrongly here. If you check for err inside the loop, you may find out what is the problem.

Yup, another validation error. Fixed 👍🏻


"relevant release notes" tool

It would be nice to have a tool that can determine which parts of the release notes are relevant by inspecting the nixos configuration. Perhaps this could be done based on metadata in the release notes themselves (making good use of our docbook, cc @grahamc :wink: ), and culminate in a nixos-upgrade tool (or nixos upgrade, #54188).

this is still a good idea


Update models.py object_from_dictionary

added in check for data in entry['comments']

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla - and if you have received this in error or have any questions, please drop us a line at<EMAIL_ADDRESS>Thanks!

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!


Color

hi how to change color (Blue And Pink)

The best way is to adapt my example drawables: https://github.com/SamThompson/BubbleActions/tree/master/bubbleactions-sample/src/main/res/drawable


Documentation using non existing method

The documentation bloc under User-defined functions - Python SDK relies on the method CreateUserDefinedFunction of the client object. This method does not exist anymore, please see the CosmosClient documentation page

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 239e3b2d-4a09-c6a1-ab36-70b60a18f8b5
Version Independent ID: 10a323f6-5a10-8b38-6339-48ee7a4c54b3
Content: Register and use stored procedures, triggers, and user-defined functions in Azure Cosmos DB SDKs
Content Source: articles/cosmos-db/how-to-use-stored-procedures-triggers-udfs.md
Service: cosmos-db
GitHub Login: @timsander1
Microsoft Alias: tisande

@arthur1993 Thank you for the feedback. We are actively investigating and will get back to you soon.

@arthur1993 This is being assigned to the content author to evaluate and update as appropriate.

Hi @arthur1993 , thanks for pointing this out! These samples are due to be updated for the latest Python SDK version and we've created a work item for this. The doc should be updated shortly.

Thanks @timsander1 for the updates.
@arthur1993 As the changes will be available with the latest Python sdk in a while, we will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.

@arthur1993 : This content was updated and will be publish in the next week. Tks


Reviewer Roulette

I've stumbled upon this project today and was wondering: wouldn't it be possible to implement mechanism for drawing reviewers? In larger teams it would be great if every member was involved at similar level, so this should draw reviewers considering review history and current activity. But MVP would be just ability to run command that would draw expected amount of reviewers.
List of users available for review should be configurable per project, as I don't see any reliable way based on available data (Slack channel's participant, Gitlab project's members would most probably be much wider than people interested in the MR), maybe CODEOWNERS would be closest.
What do you think?

Hello and thanks for the idea, loving it ❤
I could work on the MVP to allow drawing reviewers as an action (once you used the command /homer review <search>) like this

For expected number of reviewers, it can be asked in a following Slack message OR deduced from Gitlab Approval rules.
For users, it seems complex, yes.

CODEOWNERS seems a good idea, but Gitlab doesn't expose an easy way to match MR changes against CODEOWNERS.
Users of the Slack Channel could work
Gitlab Users with a minimum role could work

What do you think ?
cc @josselinbuils


Fields in BlockActionPayload

I am trying to use com.github.seratch.jslack.app_backend.interactive_messages.payload.BlockActionPayload
to respond to a button press in Slack.  I'm using Jackson to create an instance of this class from the payload sent to me from Slack.
However, many field names in this class and embedded objects had different names in the class, from those provided in the payload.  These vary systematically, E.g. apiappid in the class is api_app_id in the payload, similarly for trigger_id and response_url.
Within the Team object, the payload contains enterprise_id and enterprise_name.
The User object contains user_id.
and so on.

The only way to correctly parse/write JSON data is to use gson with some additional configurations enabled here: https://github.com/seratch/jslack/blob/baddda32ce254993f29647a85dd2b709ce2ff88b/jslack-api-client/src/main/java/com/github/seratch/jslack/common/json/GsonFactory.java
Consider using gson for the classes in this library. I don’t have any plans to support jackson in the short term as it’s tough for this library to guarantee it works regardless of the differences among jackson’s versions.

Thanks.  わかりました。


[DO NOT MERGE] Verify IT scala2.12

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2729 (e8805b8) into release-0.8.0 (3996044) will decrease coverage by 42.33%.
The diff coverage is n/a.

@@                 Coverage Diff                 @@
##             release-0.8.0   #2729       +/-   ##
===================================================
- Coverage            51.73%   9.40%   -42.34%
+ Complexity            3601      48     -3553
===================================================
  Files                  476      54      -422
  Lines                22595    1989    -20606
  Branches              2409     236     -2173
===================================================
- Hits                 11689     187    -11502
+ Misses                9888    1789     -8099
+ Partials              1018      13     -1005

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
9.40% <ø> (-60.34%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


hstack and vstack produce very inefficient jaxpr and jit slowly; possible fix with reshape?

hstack is very inefficient for tensors as it produces jaxpr code with length proportional to size of the traced array.
Compare:
{ lambda  ; a b c.
  let d = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] a
      e = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] b
      f = concatenate[ dimension=0 ] d e
      g = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] b
      h = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] c
      i = concatenate[ dimension=0 ] g h
      j = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4, 5)
                            shape=(1, 2, 2, 2, 3, 3) ] f
      k = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4, 5)
                            shape=(1, 2, 2, 2, 3, 3) ] i
      l = concatenate[ dimension=0 ] j k
      m = slice[ limit_indices=(1, 2, 2, 2, 3, 3)
                 start_indices=(0, 0, 0, 0, 0, 0)
                 strides=(1, 1, 1, 1, 1, 1) ] l
      n = squeeze[ dimensions=(0,) ] m
      o = slice[ limit_indices=(2, 2, 2, 2, 3, 3)
                 start_indices=(1, 0, 0, 0, 0, 0)
                 strides=(1, 1, 1, 1, 1, 1) ] l
      p = squeeze[ dimensions=(0,) ] o
      q = concatenate[ dimension=1 ] n p
      r = slice[ limit_indices=(1, 4, 2, 3, 3)
                 start_indices=(0, 0, 0, 0, 0)
                 strides=(1, 1, 1, 1, 1) ] q
      s = squeeze[ dimensions=(0,) ] r
      t = slice[ limit_indices=(2, 4, 2, 3, 3)
                 start_indices=(1, 0, 0, 0, 0)
                 strides=(1, 1, 1, 1, 1) ] q
      u = squeeze[ dimensions=(0,) ] t
      v = concatenate[ dimension=1 ] s u
      w = slice[ limit_indices=(1, 4, 3, 3)
                 start_indices=(0, 0, 0, 0)
                 strides=(1, 1, 1, 1) ] v
      x = squeeze[ dimensions=(0,) ] w
      y = slice[ limit_indices=(2, 4, 3, 3)
                 start_indices=(1, 0, 0, 0)
                 strides=(1, 1, 1, 1) ] v
      z = squeeze[ dimensions=(0,) ] y
      ba = slice[ limit_indices=(3, 4, 3, 3)
                  start_indices=(2, 0, 0, 0)
                  strides=(1, 1, 1, 1) ] v
      bb = squeeze[ dimensions=(0,) ] ba
      bc = slice[ limit_indices=(4, 4, 3, 3)
                  start_indices=(3, 0, 0, 0)
                  strides=(1, 1, 1, 1) ] v
      bd = squeeze[ dimensions=(0,) ] bc
      be = concatenate[ dimension=1 ] x z bb bd
      bf = slice[ limit_indices=(1, 12, 3)
                  start_indices=(0, 0, 0)
                  strides=(1, 1, 1) ] be
      bg = squeeze[ dimensions=(0,) ] bf
      bh = slice[ limit_indices=(2, 12, 3)
                  start_indices=(1, 0, 0)
                  strides=(1, 1, 1) ] be
      bi = squeeze[ dimensions=(0,) ] bh
      bj = slice[ limit_indices=(3, 12, 3)
                  start_indices=(2, 0, 0)
                  strides=(1, 1, 1) ] be
      bk = squeeze[ dimensions=(0,) ] bj
      bl = slice[ limit_indices=(4, 12, 3)
                  start_indices=(3, 0, 0)
                  strides=(1, 1, 1) ] be
      bm = squeeze[ dimensions=(0,) ] bl
      bn = concatenate[ dimension=1 ] bg bi bk bm
  in (bn,) }

to a better, equivalent code that can be achieved using jnp.reshape
{ lambda  ; a b c.
  let d = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] a
      e = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] b
      f = concatenate[ dimension=0 ] d e
      g = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] b
      h = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4)
                            shape=(1, 2, 2, 3, 3) ] c
      i = concatenate[ dimension=0 ] g h
      j = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4, 5)
                            shape=(1, 2, 2, 2, 3, 3) ] f
      k = broadcast_in_dim[ broadcast_dimensions=(1, 2, 3, 4, 5)
                            shape=(1, 2, 2, 2, 3, 3) ] i
      l = concatenate[ dimension=0 ] j k
      m = reshape[ dimensions=(0, 2, 4, 1, 3, 5)
                   new_sizes=(12, 12) ] l
  in (m,) }

Probably hstack can be re-expressed in terms of reshape in general. I'm new to jax so maybe there are some negative side effects to such approach?

Code to reproduce issue:
import jax
import jax.numpy as jnp

n = 2

mAA = 1.0*jnp.arange(3*n*3*n).reshape((n,n,3,3))
mBB = 10.0*jnp.arange(3*n*3*n).reshape((n,n,3,3))
mAB = 2.0*jnp.arange(3*n*3*n).reshape((n,n,3,3))

def stack_hard(AA,AB,BB):
    return jnp.hstack(
        jnp.hstack(
            jnp.hstack(
                jnp.hstack(
                    jnp.array(
                        [[AA,AB],[AB,BB]]
                    )
                )
            )
        )
    )

def stack_easy(AA,AB,BB):
    return  jax.lax.reshape(
                jnp.array([[AA,AB],[AB,BB]]),
                (6*n,6*n),
                dimensions = (0,2,4,1,3,5)
            )

# JIT is very slow in case of larger n
# fast_stack = jax.jit(stack_hard)
# fast_stack(mAA,mBB,mAB)

print('===========================')
print(
    jax.make_jaxpr(stack_hard)(mAA,mAB,mBB)
    )

print('===========================')
print(
    jax.make_jaxpr(stack_easy)(mAA,mAB,mBB)
    )

print(stack_easy(mAA,mAB,mBB))
print(stack_hard(mAA,mAB,mBB))

Hi - the issue here is that the call signature of hstack is that it accepts a single argument, which is a tuple of arrays.
A tuple is a Python concept, not an XLA concept, so when you pass an array to something that expects a tuple, it must be converted into N array objects that are then passed back to XLA.
I'm not sure what we could do to "fix" this – maybe we could raise an error in the case that a single array is passed to hstack, to prevent this sort of silent conversion back to a numpy tuple. It would be less convenient, but it would more explicitly surface the computational cost implicit in the function's signature.
What do you think?

Also, I don't think it's generally true that hstack of a single array can be expressed in terms of a reshape. Here's a counter-example:
>>> import jax.numpy as jnp
>>> x = jnp.arange(12).reshape(3, 2, 2)
>>> jnp.hstack(x)
DeviceArray([[ 0,  1,  4,  5,  8,  9],
             [ 2,  3,  6,  7, 10, 11]], dtype=int32)

Oh, I understand better why this happened. Perhaps we can improve just the case where only one jnp array is passed as argument.
I'm pretty sure in all cases jnp.hstack can be expressed with jax.lax.reshape (note: not jnp.reshape) due to it's cool feature of optional arg dimensions.
In case of your example it would be:
>>> import jax numpy as jnp
>>> import jax
>>> x = jnp.arange(12).reshape(3, 2, 2)
>>> jax.lax.reshape(x,(2,6),dimensions=(1,0,2)) - jnp.hstack(x)
DeviceArray([[0, 0, 0, 0, 0, 0],
             [0, 0, 0, 0, 0, 0]], dtype=int32)

Nice, I didn't know about that! If it's not too complicated, I think special-casing the single array case in terms of these kinds of reshapes would be worthwhile - maybe it could all be handled at the level of jnp.concatenate

Hi - the issue here is that the call signature of hstack is that it accepts a single argument, which is a tuple of arrays.
A tuple is a Python concept, not an XLA concept, so when you pass an array to something that expects a tuple, it must be converted into N array objects that are then passed back to XLA.
I'm not sure what we could do to "fix" this – maybe we could raise an error in the case that a single array is passed to hstack, to prevent this sort of silent conversion back to a numpy tuple, and require users to pass tuple(arr) explicitly. It would be less convenient, but it would make more apparent the computational cost implicit in the function's signature.
What do you think?

Is there a way to see which operations in a program might be undergoing such a forced conversion from XLA to numpy back to XLA? I am currently suffering a significant slow down in a program I have written and I am wondering if it due to similar issues going on in the background.

The main source of this kind of thing is calling __iter__ on a DeviceArray.

Alternatively, I think it might make sense to just raise an error when a single array is passed into hstack/vstack/stack/concatenate. We would force the user to write something more explicit like jnp.vstack(list(array)), which hopefully has more obvious performance implications.

That’s two votes for erroring on mplicit tulle conversion - maybe that’s a cleaner route

That’s two votes for erroring on implicit tuple conversion - maybe that’s a cleaner route

Oh, I see you did suggest that already 👍

Erroring implicit tuple conversion :+1:
As for hstack and vstack I'd still prefer if there was a special case with reshape, especially since the workaround is not immediately obvious.
I think I have working code for this (up to edge cases such as passing a scalar etc.)
import jax.numpy as jnp
import jax
import numpy as np

def hstack_alternative(array):
    shp = array.shape
    return jax.lax.reshape(
        array,
        (shp[1], shp[0] * shp[2]) + shp[3:],
        dimensions=((1, 0) + tuple(range(2, len(shp)))),
    )

def vstack_alternative(array):
    shp = array.shape
    return jax.lax.reshape(array, (shp[0] * shp[1],) + shp[2:])

x = jnp.arange(12).reshape(3, 2, 2)
y = jnp.arange(2 * 3 * 5).reshape(2, 3, 5)
z = jnp.arange(2 * 3 * 5 * 7).reshape(2, 3, 5, 7)

def test_hstack_alternative():
    assert np.allclose(jnp.hstack(x), hstack_alternative(x))
    assert np.allclose(jnp.hstack(y), hstack_alternative(y))
    assert np.allclose(jnp.hstack(y), hstack_alternative(y))

def test_vstack_alternative():
    assert np.allclose(jnp.vstack(x), vstack_alternative(x))
    assert np.allclose(jnp.vstack(y), vstack_alternative(y))
    assert np.allclose(jnp.vstack(y), vstack_alternative(y))

That's very cool – it would be worth adding this code-path to vstack and hstack in my opinion.
Are you interested in putting together a PR?

I was playing with this a bit - here's the implementation of jnp.concatenate for array inputs in terms of lax.reshape:
import operator
import jax.numpy as jnp
from jax import lax
from jax._src.util import canonicalize_axis

def _concatenate(x, axis=0):
  assert isinstance(x, jnp.ndarray)
  if x.ndim == 0:
    raise ValueError("Need at least one array to concatenate.")
  if x.ndim == 1:
    raise ValueError("Zero-dimensional arrays cannot be concatenated.")
  axis = canonicalize_axis(axis, x.ndim - 1)
  shape = x.shape[1:axis + 1] + (x.shape[0] * x.shape[axis + 1],) + x.shape[axis + 2:]
  dimensions = [*range(1, axis + 1), 0, *range(axis + 1, x.ndim)]
  return lax.reshape(x, shape, dimensions)

Quickly tested with:
import numpy as np

x = jnp.arange(2*3*4*5*4*3*2).reshape(2, 3, 4, 5, 4, 3, 2)
for axis in range(1 - x.ndim, x.ndim - 1):
  c1 = jnp.concatenate(x, axis=axis)
  c2 = _concatenate(x, axis=axis)
  np.testing.assert_array_equal(c1, c2)

That's very cool – it would be worth adding this code-path to vstack and hstack in my opinion.
Are you interested in putting together a PR?

Sure :)
I'm new here so if you could help with the github side of things I'll be thankful.

Great! We have a bit of contribution information here: https://jax.readthedocs.io/en/latest/contributing.html#contributing-code-using-pull-requests
Feel free to open a work-in-progress PR if that would be helpful, and let me know if you have any questions

@jakevpd I've had a busy week, sry. It seems it worked out in the end, thanks :)