[MB-1585] SIT Schedule lookup for dest and origin

Description
This PR creates a lookup function that is used for both SITScheduleOrigin and SITScheduleDest
Reviewer Notes

I believe the sources for the pickup and destination addresses will need to be updated once those are no longer found on the MTO Service item (tickets to update this: https://dp3.atlassian.net/browse/MB-6279 and https://dp3.atlassian.net/browse/MB-6257))
I broke out the query to find the domestic service area into a new file called lookup_query_helper.go. I based this off of the existing pricer_query_helper.go file.
There isn't any code set up (that I'm aware of for this ticket) but please double check that the query and unit test make sense

Acceptance

As noted in sprint planning, there's no acceptance for this ticket. Instead the lookup work can be accepted when the pricer is built

References

Jira story for this change

Messages

:book:
:link: MB-1585

  Generated by :no_entry_sign: dangerJS against 4f96ef15452f43c3f077bdc1969b7b643c45a08c

Messages

:book:
:link: MB-1585

  Generated by :no_entry_sign: dangerJS against 4f96ef15452f43c3f077bdc1969b7b643c45a08c


Not auto clear previous highlight when click other tag

Hi you! Thanks you for make this geany plugin. Very usefull.
When i click html tag 1, it highlight is nice. But i click other tag, highlight of tag 1 not clear highlight.
You can see below image of me.
Thanks for read!

Hello!
I am very sorry for such delay with reply. I haven't seen email notification about this issue because it went to spam. If the issue is still reproducible, could you please share problematic code?
I received several reports same with this one but I was unable to reproduce on my PC.
Thank you!

Hi!, I have the same problem :(
You can see the problem code here: http://pastebin.com/z9D3tgm7
If you have news, please notice me. Thanks a lot for this plugin !  :)

Hi Guys,
Thank you both for reporting!
I spent some time with debugging and found the issue. You may track it at Geany-Gevel mailing list.
While discussion in mailing list is not yet finished and fix will be proposed only in next Geany release, I propose you to downgrade to Geany 1.23.1 and install plugin as described here. PairTagHighlighter is working correctly on Geany 1.23.1.

Guys,
Root cause is found, fix is provided and merged. Was my mistake.
Will be available with the next release of Geany plugins.
Thanks for participation!

In order not to wait next official Geany plugins release, I've prepared 64bit version of plugin for Linux.
You may download it here - http://kononenko.ws/en/blog/2015-03-09/updated_pairtaghighlighter_plugin_for_geany_1.24

Really thank you, work perfectly, but I had to copy the file in /usr/local/lib/geany/. Maybe is because I compiled geany from source and I don't installed from repositories.
Thanks also for the recognition :D
see you later. from Chile


prepared statement "s0" already exists when connecting to pgbouncer with ssl

Bug description
I'm connecting to postgres on digital ocean with pgbouncer as guided here: https://www.prisma.io/docs/concepts/database-connectors/postgresql#configuring-an-ssl-connection. And also appended pgbouncer=true&statement_cache_size=0 as suggested by @ryands17
ConnectorError: prepared statement \"s0\" already exists when connecting to pgbouncer with ssl is thrown for every query.
Full error:
PrismaClientUnknownRequestError3 [PrismaClientUnknownRequestError]:
Invalid `prisma.siteMember.findMany()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42P05"), message: "prepared statement \"s0\" already exists", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("prepare.c"), line: Some(463), routine: Some("StorePreparedStatement") }) }) })
    at PrismaClientFetcher.request (/XXXXXXXXXXX/node_modules/@prisma/client/runtime/index.js:78125:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  clientVersion: '2.13.0'
}

How to reproduce

Setup a managed database on digital ocean.
Create pgbouncer pool.
Connect with SSL config the above.
Run any query

Expected behavior
Prisma queries should be executed normally when connected to a pgbouncer pool'd db.

Prisma information
Connection string: postgres://<XXXXXXXXXXXX>/default?sslmode=require&&sslcert=db-ssl.crt&sslpassword=<XXXXX>&sslidentity=db-ssl.p12&pgbouncer=true&statement_cache_size=0

Environment & setup

OS:  Mac OS 10.15.4
Database: PostgreSQL 12
Node.js version: 14.4.0
Prisma version: 2.13.0

@prisma/cli          : 2.13.0
@prisma/client       : 2.13.0
Current platform     : darwin
Query Engine         : query-engine 833ab05d2a20e822f6736a39a27de4fc8f6b3e49 (at node_modules/@prisma/engines/query-engine-darwin)
Migration Engine     : migration-engine-cli 833ab05d2a20e822f6736a39a27de4fc8f6b3e49 (at node_modules/@prisma/engines/migration-engine-darwin)
Introspection Engine : introspection-core 833ab05d2a20e822f6736a39a27de4fc8f6b3e49 (at node_modules/@prisma/engines/introspection-engine-darwin)
Format Binary        : prisma-fmt 833ab05d2a20e822f6736a39a27de4fc8f6b3e49 (at node_modules/@prisma/engines/prisma-fmt-darwin)
Studio               : 0.329.0

Complete Debug Log
  tryLoadEnv Environment variables not found at null +0ms
  tryLoadEnv Environment variables not found at undefined +0ms
  tryLoadEnv No Environment variables loaded +1ms
  tryLoadEnv Environment variables not found at null +10ms
  tryLoadEnv Environment variables not found at undefined +0ms
  tryLoadEnv No Environment variables loaded +0ms
  prisma-client { clientVersion: '2.13.0' } +0ms
  prisma-client Prisma Client call: +77ms
  prisma-client prisma.siteMember.findMany({
  prisma-client   where: {
  prisma-client     userId: 1
  prisma-client   },
  prisma-client   include: {
  prisma-client     site: {
  prisma-client       include: {
  prisma-client         posts: {
  prisma-client           select: {
  prisma-client             title: true
  prisma-client           }
  prisma-client         }
  prisma-client       }
  prisma-client     }
  prisma-client   },
  prisma-client   orderBy: {
  prisma-client     createdAt: 'desc'
  prisma-client   }
  prisma-client }) +2ms
  prisma-client Generated request: +0ms
  prisma-client query {
  prisma-client   findManySiteMember(
  prisma-client     where: {
  prisma-client       userId: 1
  prisma-client     }
  prisma-client     orderBy: [
  prisma-client       {
  prisma-client         createdAt: desc
  prisma-client       }
  prisma-client     ]
  prisma-client   ) {
  prisma-client     id
  prisma-client     siteId
  prisma-client     userId
  prisma-client     role
  prisma-client     createdAt
  prisma-client     updatedAt
  prisma-client     addedBy
  prisma-client     site {
  prisma-client       id
  prisma-client       name
  prisma-client       createdAt
  prisma-client       updatedAt
  prisma-client       posts {
  prisma-client         title
  prisma-client       }
  prisma-client     }
  prisma-client   }
  prisma-client }
  prisma-client  +0ms
  engine {
  engine   cwd: '/Users/XXXXXXX/prisma'
  engine } +0ms
  engine Search for Query Engine in /Users/XXXXXXX/node_modules/.prisma/client +1ms
  plusX Execution permissions of /Users/XXXXXXX/node_modules/.prisma/client/query-engine-darwin are fine +0ms
  engine {
  engine   flags: [
  engine     '--enable-raw-queries',
  engine     '--unix-path',
  engine     '/tmp/prisma-ccda131f44c9c5e83960f619.sock'
  engine   ]
  engine } +1ms
  engine stdout {
  timestamp: 'Dec 21 16:17:46.726',
  level: 'INFO',
  fields: { message: 'Starting a postgresql pool with 1 connections.' },
  target: 'quaint::pooled'
} +48ms
  engine stdout {
  timestamp: 'Dec 21 16:17:48.659',
  level: 'INFO',
  fields: {
    message: 'Started http server on http+unix:///private/tmp/prisma-ccda131f44c9c5e83960f619.sock'
  },
  target: 'query_engine::server'
} +2s
  engine Search for Query Engine in Users/XXXXXXX/node_modules/.prisma/client +5ms
  plusX Execution permissions of /Users/XXXXXXX/node_modules/.prisma/client/query-engine-darwin are fine +2s
  engine Client Version 2.13.0 +25ms
  engine Engine Version query-engine 833ab05d2a20e822f6736a39a27de4fc8f6b3e49 +0ms
  engine {
  engine   error: PrismaClientUnknownRequestError3 [PrismaClientUnknownRequestError]: Error occurred during query execution:
  engine   ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42P05"), message: "prepared statement \"s0\" already exists", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("prepare.c"), line: Some(463), routine: Some("StorePreparedStatement") }) }) })
  engine       at NodeEngine.graphQLToJSError (/Users/XXXXXXX/node_modules/@prisma/client/runtime/index.js:27363:14)
  engine       at /Users/XXXXXXX/node_modules/@prisma/client/runtime/index.js:27285:24
  engine       at /Users/XXXXXXX/node_modules/@risingstack/react-easy-state/dist/cjs.es6.js:243:63
  engine       at Object.batchedUpdates$1 (/Users/XXXXXXX/node_modules/react-dom/cjs/react-dom.development.js:21856:12)
  engine       at batch (/Users/XXXXXXX/node_modules/@risingstack/react-easy-state/dist/cjs.es6.js:243:30)
  engine       at Object.apply (/Users/XXXXXXX/node_modules/@risingstack/react-easy-state/dist/cjs.es6.js:264:16)
  engine       at runMicrotasks (<anonymous>)
  engine       at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  engine     clientVersion: '2.13.0'
  engine   }
  engine } +280ms
  prisma-client Error: Error occurred during query execution:
  prisma-client ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42P05"), message: "prepared statement \"s0\" already exists", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("prepare.c"), line: Some(463), routine: Some("StorePreparedStatement") }) }) })
  prisma-client     at NodeEngine.graphQLToJSError (/Users/XXXXXXXnode_modules/@prisma/client/runtime/index.js:27363:14)
  prisma-client     at /Users/XXXXXXX/node_modules/@prisma/client/runtime/index.js:27285:24
  prisma-client     at /Users/Users/XXXXXXX/node_modules/@risingstack/react-easy-state/dist/cjs.es6.js:243:63
  prisma-client     at Object.batchedUpdates$1 (/Users/XXXXXXX/node_modules/react-dom/cjs/react-dom.development.js:21856:12)
  prisma-client     at batch (/Users/XXXXXXX/node_modules/@risingstack/react-easy-state/dist/cjs.es6.js:243:30)
  prisma-client     at Object.apply (/Users/XXXXXXX/node_modules/@risingstack/react-easy-state/dist/cjs.es6.js:264:16)
  prisma-client     at runMicrotasks (<anonymous>)
  prisma-client     at processTicksAndRejections (internal/process/task_queues.js:97:5) +2s
PrismaClientUnknownRequestError3 [PrismaClientUnknownRequestError]:
Invalid `prisma.siteMember.findMany()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42P05"), message: "prepared statement \"s0\" already exists", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("prepare.c"), line: Some(463), routine: Some("StorePreparedStatement") }) }) })
    at PrismaClientFetcher.request (/Users/XXXXXXX/node_modules/@prisma/client/runtime/index.js:78125:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  clientVersion: '2.13.0'
}

Hey @s-kris
I was unable to reproduce this. I think you are using the database credentials here instead of pgbouncer credentials. Digitalocean provides different credentials for pgbouncer and you should use them when querying your database. The direct database credentials will not work and only use them for migrating the database structure.
Get the connection details for the pool from here:

Also, please absolutely make sure that you selected transaction mode here.

Other modes are not supported.

@pantharshit00 I'm using pgbouncer credentials in transaction mode only. I just reverified now.

I have a similar issue. Exact same setup, using DO with bouncer and Transaction mode. Prisma is using the credentials from DO bouncer too.

It's working for me. I use prisma/client 2.12.1 and the following DATABASE_URL:
postgresql://username:password@db_host:5432/dbname?schema=public&pgbouncer=true&sslmode=require

(note the pgbouncer and sslmode options)

@AlessandroA are you using a DO hosted postgres DB? and using their connection pool credentials?

Yes! It's a DigitalOcean PostgreSQL database and I connect through a connection pool. I configured it in transaction mode.
I don't know if this changed recently, but the credentials for pooled and non-pooled connections are exactly the same. The only difference is the db port.

OK awesome. Just added that extra flag and it works for me too.

@s-kris Can you please double check as well?

pgbouncer=true is the flag?

pgbouncer=true is the flag?

Yes, sample uri postgresql://user:pass@host:5432/mydb?schema=public&pgbouncer=true

Yes, sample uri postgresql://user:pass@host:5432/mydb?schema=public&pgbouncer=true

I have had this exact same issue. Adding &pgbouncer=true to the end of the connection string fixed the issue.
Is this something that should be fixed on the DO side, or on the Prisma side?

I have had this exact same issue. Adding &pgbouncer=true to the end of the connection string fixed the issue.
Is this something that should be fixed on the DO side, or on the Prisma side?

None of both - this just needs to be documented better on our side and possibly be caught as an error message. If you want to use pgBouncer, you need to supply that parameter in Prisma. No way around it right now.

None of both - this just needs to be documented better on our side and possibly be caught as an error message. If you want to use pgBouncer, you need to supply that parameter in Prisma. No way around it right now.

@pantharshit00 sorry for delayed response.
Yes, I added the flag. I double checked. Triple checked.

@pantharshit00 sorry for delayed response.
Yes, I added the flag. I double checked. Triple checked.

I guess we should wait for next release here. Unfortunately without a reproduction it is not feasible for us to investigate this further.

I guess we should wait for next release here. Unfortunately without a reproduction it is not feasible for us to investigate this further.

@pantharshit00 sure. I'll try to create a demo shortly.

@pantharshit00 sure. I'll try to create a demo shortly.

I'm starting to get this error as well, anything I can do to help?
 Omit when using with migrate. Also, we don't recommend running migrate over pgbouncer. Run it directly over your database connection as it is one off operation everytime

Any particular reason why migrations should not be run over the pooled connection? Does this apply to the new preview migrate command too?
It's worth noting that most applications have a single DATBASE_URL - so just ommiting it is not so easy. Larger applications are also likely to run migrations as part of the deployment too.

I was able to connect to pool with SSL. The change I did was to remove sslidentity and sslpassword from connection string. Was able connect to DO cluster using .crt file.
DATABASE_URL=postgres://XXXXXXXXXXXXXX?connection_limit=3&sslmode=require&&sslcert=ssl.crt&pgbouncer=true&connect_timeout=10&pool_timeout=30

Had similar issues, solved it by naming the connection pool the same as the database. No need to use ca_certificate. Found this here https://www.digitalocean.com/community/questions/how-to-set-up-client-certificates-for-managed-postgres

Having the same issue on Digitalocean using connection pool with NextJS and Postgres. To fix this I did the following:

added DATABASE_URL_MIGRATE environment variable with a regular database connection
added DATABASE_URL environment variable with database connection pool

Created a migrate.js script in the root:
#!/usr/bin/env node

'use strict';

const { execSync } = require('child_process');
const { DATABASE_URL_MIGRATE } = process.env;

try {
  // Run the migrate with database connection not using pooling
  // See: https://github.com/prisma/prisma/issues/4752
  execSync(`DATABASE_URL=${DATABASE_URL_MIGRATE} yarn prisma migrate deploy`);
} catch (error) {
  console.error(error);
  process.exit(1);
}

Invoke the script before every deployment from the package.json:
"scripts": {
  "start": "yarn migrate && next start",
  "migrate": "node ./migrate.js",
  }

This should solve it.

@lassegit You can probably skip the redirection to a js file, and just do "migrate": "DATABASE_URL=\"$DATABASE_URL_MIGRATE\" yarn prisma migrate deploy", instead (The yarn in the script command might even be not necessary - not sure).

Having the same issue on Digitalocean using connection pool with NextJS and Postgres. To fix this I did the following:

added DATABASE_URL_MIGRATE environment variable with a regular database connection
added DATABASE_URL environment variable with database connection pool

Created a migrate.js script in the root:
#!/usr/bin/env node

'use strict';

const { execSync } = require('child_process');
const { DATABASE_URL_MIGRATE } = process.env;

try {
  // Run the migrate with database connection not using pooling
  // See: https://github.com/prisma/prisma/issues/4752
  execSync(`DATABASE_URL=${DATABASE_URL_MIGRATE} yarn prisma migrate deploy`);
} catch (error) {
  console.error(error);
  process.exit(1);
}

Invoke the script before every deployment from the package.json:
"scripts": {
  "start": "yarn migrate && next start",
  "migrate": "node ./migrate.js",
  }

This should solve it.

On Windows I had to use:
require('dotenv').config();
const { execSync } = require('child_process');
const { DATABASE_URL_MIGRATE } = process.env;

try {
  // Run the migrate with database connection not using pooling
  // See: https://github.com/prisma/prisma/issues/4752
  execSync(`cross-env DATABASE_URL=${DATABASE_URL_MIGRATE} npx prisma db pull`, { stdio: 'inherit' });
} catch (error) {
  console.error(error);
  process.exit(1);
}

🤷♂

@lassegit You can probably skip the redirection to a js file, and just do "migrate": "DATABASE_URL=\"$DATABASE_URL_MIGRATE\" yarn prisma migrate deploy", instead (The yarn in the script command might even be not necessary - not sure).

Specifically for me with NextJS/vercel & a managed digitalocean postgres database, in the package.json:
"vercel-build": "prisma generate && DATABASE_URL=\"$DATABASE_URL_MIGRATE\" prisma migrate deploy && next build",
Where DATABASE_URL_MIGRATE is in the form: postgresql://user:pass@host:25060/db?sslmode=require&prepareThreshold=0&connection_timeout=60&connection_limit=1

make sure your user is set to inbound user

My current problem is that I've started using the Vercel integration and the generated connection string environment variable doesn't have ?pgbouncer=true. When I manually add it, prisma migrate starts failing with ERROR: prepared statement "s0" already exists error.

prisma migrate does not work with PgBouncer, you should use a direct connection string for that or you will hit this error.

I had to come up with a way to set the environment variable, such that during build I migrate can run, while at runtime it connects with pgbouncer. I provide a datasources parameter to the PrismaClient with the database url environment variable and concatenate ?pgbouncer=true to it.
Note that this may be highly specific to my Vercel+Neon setup with the official integration.

We have directUrl that overrides the url for the CLI commands: https://www.prisma.io/docs/data-platform/data-proxy/prisma-cli-with-data-proxy (This also applies for Pgbouncer, not only Data Proxy - we will have to update that docs page)

That sounds better, overriding prod values didn’t sit right. I’ll change my approach, thanks for the heads up.

Prisma Studio UI seems to not use the directUrl, so it has these issues.


Frictionfull Space.

About the PR
Gives ATS and the station a minor amount of friction, meaning cargo techs bumping into them no longer sends them into the void. Mirror of https://github.com/Simple-Station/Einstein-Engines/pull/514
Why / Balance
See above
Technical details
See original PR.
Changelog
:cl:

tweak: The station and ATS now have a minor amount of friction.

PEAK

we’re so back

peak, merge now

I can't believe y'all have somehow went an entire month with your cargo techs basically not being able to play the game LMAO.


Robust Keepalive Monitoring Accounting for 'Index Doesn't Exist'

Sentinl and Kibana version 5.6.2

Issue description
I have a 'keepalive' watcher which monitors an index and reports if keepalive messages don't come through, indicating the data pipeline is broken. This would work great except that if no data is getting published to the expected index, the index doesn't exist and Sentinl doesn't execute the watcher, thus I never receive a keepalive alarm.

Reproducing the issue step-by-step
Create a watcher that reports if data doesn't get ingested into a certain index. Turn off the data ingest into that index, the following day (if you're using daily indices), instead of a getting an alert that data hasn't been ingested, if you look through the Kibana logs, you'll see that alerts fire for the index not existing.

Expected results vs. actual results
I'd like to get some sort of alert that Sentinl is broken, perhaps it's a configuration setup that only soft-errors if an index doesn't exist, perhaps it's an administrative email informing me that Sentinl executed a watcher which caused an error.

Sample of data

{
  "search": {
    "request": {
      "index": [
        "<keepalive-{now/d}>",
        "<keepalive-{now/d-1d}>"
      ],
      "body": {
        "query": {
          "exists": {
            "field": "keepalive.keyword"
          }
        }
      }
    }
  }
}

Additional details
I'm asking for advice on how to best-architect a keepalive alert such as this. In the 'Expected vs Actual Results' section, I proposed a couple potential solutions. However I'm sure you'll likely have additional solutions which fit better with the established architecture.

@sethyes trying to understand the scenario. My first attempt would be to detect a missing (daily) index in parallel to this detection not to overload its role, is this something you already attempted?
Could you provide the logs (or better query) resulting from one of those borked executions when the index is not found? It's just a matter of catching the condition and probably adding "error" reporting in the body of a watcher.
Thanks!

I did attempt to detect a missing index on the input, however I received an error as exists doesn't support index field.
My input:
{
  "search": {
    "request": {
      "body": {
        "query": {
          "bool": {
            "must_not": {
              "exists": {
                "index": "keepalive-{now/d}"
              }
            }
          }
        }
      }
    }
  }
}

and the response:
Sentinl Watchers: Error 406 Not Acceptable: [parsing_exception] [exists] query does not support [index], with { line=1 & col=49 }

I'm not seeing much in the Kibana logs, but I do get a console alert:
Sentinl Watchers: Error 406 Not Acceptable: [index_not_found_exception] no such index, with { resource.type="index_or_alias" & resource.id="keepalive-2018.03.20" & index_uuid="na" & index="keepalive-2018.03.20" }

My only issue with just looking for a missing index, is that means I might not be alerted to a down pipeline until the following day. If I could make indices optional somehow, that would allow the watcher to still alert on no keepalive being received.
Let me know if I can clarify further. Thanks @lmangani

I see - What if we had an alert on index_not_found option in general for watchers? I guess its not useless in general....

That'd be great! Either that, or allow for the search to return no results due to the index not existing.

I think the long term solution would be to offer two parsers, for results AND errors so both can be handled. We'll do some brainstorming around this and find a viable path to start with.

Thanks! Your commitment to maturing this standout product is what has my whole company committed to using your products. Let me know if I can provide any assistance.


Keytar fails to load in Apple SI

I see this unit test failure:
1) Keytar
       loads and is functional:
     Error: dlopen(/Users/bpasero/Development/Microsoft/monaco/node_modules/keytar/build/Release/keytar.node, 1): no suitable image found.  Did find:
	/Users/bpasero/Development/Microsoft/monaco/node_modules/keytar/build/Release/keytar.node: mach-o, but wrong architecture
	/Users/bpasero/Development/Microsoft/monaco/node_modules/keytar/build/Release/keytar.node: mach-o, but wrong architecture
      at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1812)
      at Object.Module._extensions..node (internal/modules/cjs/loader.js:1203:18)
      at Object.func [as .node] (electron/js2c/asar_bundle.js:5:1812)
      at Module.load (internal/modules/cjs/loader.js:992:32)
      at Module._load (internal/modules/cjs/loader.js:885:14)
      at Function.f._load (electron/js2c/asar_bundle.js:5:12710)
      at Module.require (internal/modules/cjs/loader.js:1032:19)
      at Module.patchedRequire [as require] (/Users/bpasero/Development/Microsoft/monaco/node_modules/diagnostic-channel/dist/src/patchRequire.js:14:46)
      at require (internal/modules/cjs/helpers.js:72:18)

I wonder if this is a problem local to me or are we not running unit tests against Apple SI yet?

I wonder if this is a problem local to me or are we not running unit tests against Apple SI yet?

Its just a problem in local build because keytar uses prebuilt modules, and arm version is not available. For the product build I manually cross-compile the package https://github.com/microsoft/vscode/blob/master/build/azure-pipelines/darwin/product-build-darwin.yml#L116.

are we not running unit tests against Apple SI yet?

Yes we are not running tests, will need to add it back.

Its just a problem in local build because keytar uses prebuilt modules, and arm version is not available. For the product build I manually cross-compile the package https://github.com/microsoft/vscode/blob/master/build/azure-pipelines/darwin/product-build-darwin.yml#L116.

are we not running unit tests against Apple SI yet?

Yes we are not running tests, will need to add it back.

I just confirmed that a node-gyp rebuild fixes the unit test for me.

I just confirmed that a node-gyp rebuild fixes the unit test for me.

@deepak1556 I think this can be closed now with Electron 12?

Yup this is fixed with https://github.com/microsoft/vscode/pull/120673, thanks for tracking.


[#149558193] Upgrade compose-broker to v0.9.0 and update service catalog

What
Upgrades compose-broker to v0.9.0
The catalog config has changed slightly in compose-broker v0.9.0 to
allow setting kind (databaseType) and scale (units) per service plan.
The public facing service name for Elasticsearch should be
'elasticsearch' not 'elastic_search'
How to review

[x] Ensure paas-compose-broker PR is merged
[ ] Ensure v0.9.0 is correct version
[x] Ensure deploys ok
[x] Ensure Elasticsearch service is available under the correct name "elasticsearch"

Who can review
Not @chrisfarms no @henrytk

LGTM.

For future note, this merged v0.10.0 instead of the expected v0.9.0. During the merge of paas-compose-broker#13 I unintentionally advanced by two releases instead of one.


Resteasy 1662

@ronsigal Another pull request about String marshalling for String based @*Param.
This pull request is about enabling String marshalling for String based @*Param (@QueryParam, @MatrixParam...) into char primitive type or Character type as already done  for other primitive/wrapper types.

Hi @NicoNes ,
I've reviewed your PR and it looks fine, thanks. I'm only going to update a bunch of "@tpSince RESTEasy 3.0.16" with proper versions, after having merged the PR.
Btw, I think we forgot to deal with this previously, sorry, but we require that contributions are made under the terms of the ASL2.0 License: http://www.apache.org/licenses/LICENSE-2.0. Do you agree to these terms? (please explicitly comment below). Thanks.

Hi @asoldano,
Yes I agree to the  terms of the licence 👍.


Suggestion: Add feature-complete table showing flow syntax => tcomb syntax to README

I believe this would aid people in understanding the inner guts of how this works as well as the interop between flow and tcomb functionality.
Eg.
type MyString = string;
=> const MyString = t.String;

type MyRecord = {myString: string};
=> const MyRecord = t.interface({myString: t.String});

Also sharing an up-to-date ASTExplorer session for people to play around would be awesome.
Eg. https://astexplorer.net/#/dKZROdJGUy

I would actually take the time to contribute this table but I'm on holiday for a few weeks from tomorrow so just wanted to share the idea! :)

Nice! Thanks for the suggestion


macOS can't run this project [ bitpack , win32api ]

My test computer details:
macOS 12.5 (21G72)
intel Core i9
After I followed the ' Manual Installation ( For Developers ) ' to install this project, I found it can't run.
At first, it shows ' can't import name bitpack '
File "/Users/chocovanilazuki/Downloads/ultimatevocalremovergui-5.3.0/diffq/base.py", line 21, in <module>     from . import bitpack ImportError: cannot import name 'bitpack' from partially initialized module 'diffq' (most likely due to a circular import) (/Users/chocovanilazuki/Downloads/ultimatevocalremovergui-5.3.0/diffq/__init__.py)
Then I found this project is relied on the ' win32api '. Only runs on Windows.

After fiddling with it for an entire night, I got it running on my M1 Mac. I have to say, this project has more issues than assuming Win32 API is present.

It's referencing GUI-icon.png which isn't the correct case - (case sensitivity, Windows developers! Get it right!) and assumes a backslash path delimiter (Windows developers - Can you feel my hatred on backslashes?) so the program don't find the ICON and crashes (funny thing that it crashes for something ever so slightly wrong)
Doesn't ship a uvr_patch_version.txt that is read at startup (for checking updates) and crashes (same thing as above, duh)
Ships an x86_64 libtkdnd which isn't processed by Rosetta, and compiling tkdnd on M1 is really difficult since nobody did support on that. In my knowledge Tkdnd just adds drag and drop to Tcl/tk and so the developer of UVR5 SHOULD DROP THIS DEPENDENCY for compatibility
Ships a Windows AMD64 binpack in diffq so it's not gonna load at all, you'll have to replace that with a proper version
Assumes you're on Windows and calls sox**.exe** so if you put a macOS SoX into the respective folder it still fails to use it

And when I was configuring all this in a venv, Python dependencies are very messy. If you just do the pip install -r requirements.txt as README instructed you'll face tons of errors, mostly because the dependency is messy and something magically depends on an older SciPy & NumPy which isn't delivered in wheels. And if you try to compile Scipy/Numpy on M1, good luck fiddling with that. I ended up ignoring these weird versioning issues and just went with the latest wheels. No issues so far.
Many many small issues adds up. It's painful to debug through all this. It couldn't utilize M1 hardware (pure CPU) but ANE doesn't support a very large neural network either so I think this is the best case.

same issue on Arch Linux, I am trying to package this project on aur but no work.

After fiddling with it for an entire night, I got it running on my M1 Mac. I have to say, this project has more issues than assuming Win32 API is present.

It's referencing GUI-icon.png which isn't the correct case - (case sensitivity, Windows developers! Get it right!) and assumes a backslash path delimiter (Windows developers - Can you feel my hatred on backslashes?) so the program don't find the ICON and crashes (funny thing that it crashes for something ever so slightly wrong)
Doesn't ship a uvr_patch_version.txt that is read at startup (for checking updates) and crashes (same thing as above, duh)
Ships an x86_64 libtkdnd which isn't processed by Rosetta, and compiling tkdnd on M1 is really difficult since nobody did support on that. In my knowledge Tkdnd just adds drag and drop to Tcl/tk and so the developer of UVR5 SHOULD DROP THIS DEPENDENCY for compatibility
Ships a Windows AMD64 binpack in diffq so it's not gonna load at all, you'll have to replace that with a proper version
Assumes you're on Windows and calls sox**.exe** so if you put a macOS SoX into the respective folder it still fails to use it

And when I was configuring all this in a venv, Python dependencies are very messy. If you just do the pip install -r requirements.txt as README instructed you'll face tons of errors, mostly because the dependency is messy and something magically depends on an older SciPy & NumPy which isn't delivered in wheels. And if you try to compile Scipy/Numpy on M1, good luck fiddling with that. I ended up ignoring these weird versioning issues and just went with the latest wheels. No issues so far.
Many many small issues adds up. It's painful to debug through all this. It couldn't utilize M1 hardware (pure CPU) but ANE doesn't support a very large neural network either so I think this is the best case.

Thank you for this! I didn't have a MacBook to test this with so this is incredibly helpful for when I start making this Mac compatible.

After fiddling with it for an entire night, I got it running on my M1 Mac. I have to say, this project has more issues than assuming Win32 API is present.

It's referencing GUI-icon.png which isn't the correct case - (case sensitivity, Windows developers! Get it right!) and assumes a backslash path delimiter (Windows developers - Can you feel my hatred on backslashes?) so the program don't find the ICON and crashes (funny thing that it crashes for something ever so slightly wrong)
Doesn't ship a uvr_patch_version.txt that is read at startup (for checking updates) and crashes (same thing as above, duh)
Ships an x86_64 libtkdnd which isn't processed by Rosetta, and compiling tkdnd on M1 is really difficult since nobody did support on that. In my knowledge Tkdnd just adds drag and drop to Tcl/tk and so the developer of UVR5 SHOULD DROP THIS DEPENDENCY for compatibility
Ships a Windows AMD64 binpack in diffq so it's not gonna load at all, you'll have to replace that with a proper version
Assumes you're on Windows and calls sox**.exe** so if you put a macOS SoX into the respective folder it still fails to use it

And when I was configuring all this in a venv, Python dependencies are very messy. If you just do the pip install -r requirements.txt as README instructed you'll face tons of errors, mostly because the dependency is messy and something magically depends on an older SciPy & NumPy which isn't delivered in wheels. And if you try to compile Scipy/Numpy on M1, good luck fiddling with that. I ended up ignoring these weird versioning issues and just went with the latest wheels. No issues so far.
Many many small issues adds up. It's painful to debug through all this. It couldn't utilize M1 hardware (pure CPU) but ANE doesn't support a very large neural network either so I think this is the best case.

Thank you for this! I didn't have a MacBook to test this with so this is incredibly helpful for when I start making this Mac compatible.

If you manage to debug through most of this create a pull request for a Mac specific install and I'll merge it.

Thank you very much to the developers for developing such excellent software. Unfortunately, there is an Apple computer with only an m1 chip. I don't know if you can also develop a Mac version.

After fiddling with it for an entire night, I got it running on my M1 Mac. I have to say, this project has more issues than assuming Win32 API is present.

It's referencing GUI-icon.png which isn't the correct case - (case sensitivity, Windows developers! Get it right!) and assumes a backslash path delimiter (Windows developers - Can you feel my hatred on backslashes?) so the program don't find the ICON and crashes (funny thing that it crashes for something ever so slightly wrong)
Doesn't ship a uvr_patch_version.txt that is read at startup (for checking updates) and crashes (same thing as above, duh)
Ships an x86_64 libtkdnd which isn't processed by Rosetta, and compiling tkdnd on M1 is really difficult since nobody did support on that. In my knowledge Tkdnd just adds drag and drop to Tcl/tk and so the developer of UVR5 SHOULD DROP THIS DEPENDENCY for compatibility
Ships a Windows AMD64 binpack in diffq so it's not gonna load at all, you'll have to replace that with a proper version
Assumes you're on Windows and calls sox**.exe** so if you put a macOS SoX into the respective folder it still fails to use it

And when I was configuring all this in a venv, Python dependencies are very messy. If you just do the pip install -r requirements.txt as README instructed you'll face tons of errors, mostly because the dependency is messy and something magically depends on an older SciPy & NumPy which isn't delivered in wheels. And if you try to compile Scipy/Numpy on M1, good luck fiddling with that. I ended up ignoring these weird versioning issues and just went with the latest wheels. No issues so far.
Many many small issues adds up. It's painful to debug through all this. It couldn't utilize M1 hardware (pure CPU) but ANE doesn't support a very large neural network either so I think this is the best case.

I am using M1 mac too, but got many errors while building. Would you please share steps of how to build this project on M1 mac. Thanks a lot!

After fiddling with it for an entire night, I got it running on my M1 Mac. I have to say, this project has more issues than assuming Win32 API is present.

It's referencing GUI-icon.png which isn't the correct case - (case sensitivity, Windows developers! Get it right!) and assumes a backslash path delimiter (Windows developers - Can you feel my hatred on backslashes?) so the program don't find the ICON and crashes (funny thing that it crashes for something ever so slightly wrong)
Doesn't ship a uvr_patch_version.txt that is read at startup (for checking updates) and crashes (same thing as above, duh)
Ships an x86_64 libtkdnd which isn't processed by Rosetta, and compiling tkdnd on M1 is really difficult since nobody did support on that. In my knowledge Tkdnd just adds drag and drop to Tcl/tk and so the developer of UVR5 SHOULD DROP THIS DEPENDENCY for compatibility
Ships a Windows AMD64 binpack in diffq so it's not gonna load at all, you'll have to replace that with a proper version
Assumes you're on Windows and calls sox**.exe** so if you put a macOS SoX into the respective folder it still fails to use it

And when I was configuring all this in a venv, Python dependencies are very messy. If you just do the pip install -r requirements.txt as README instructed you'll face tons of errors, mostly because the dependency is messy and something magically depends on an older SciPy & NumPy which isn't delivered in wheels. And if you try to compile Scipy/Numpy on M1, good luck fiddling with that. I ended up ignoring these weird versioning issues and just went with the latest wheels. No issues so far.
Many many small issues adds up. It's painful to debug through all this. It couldn't utilize M1 hardware (pure CPU) but ANE doesn't support a very large neural network either so I think this is the best case.

I am using M1 mac too, but got many errors while building. Would you please share steps of how to build this project on M1 mac. Thanks a lot!

I　installed parallets desktop 17 and windows11

After fiddling with it for an entire night, I got it running on my M1 Mac. I have to say, this project has more issues than assuming Win32 API is present.

It's referencing GUI-icon.png which isn't the correct case - (case sensitivity, Windows developers! Get it right!) and assumes a backslash path delimiter (Windows developers - Can you feel my hatred on backslashes?) so the program don't find the ICON and crashes (funny thing that it crashes for something ever so slightly wrong)
Doesn't ship a uvr_patch_version.txt that is read at startup (for checking updates) and crashes (same thing as above, duh)
Ships an x86_64 libtkdnd which isn't processed by Rosetta, and compiling tkdnd on M1 is really difficult since nobody did support on that. In my knowledge Tkdnd just adds drag and drop to Tcl/tk and so the developer of UVR5 SHOULD DROP THIS DEPENDENCY for compatibility
Ships a Windows AMD64 binpack in diffq so it's not gonna load at all, you'll have to replace that with a proper version
Assumes you're on Windows and calls sox**.exe** so if you put a macOS SoX into the respective folder it still fails to use it

And when I was configuring all this in a venv, Python dependencies are very messy. If you just do the pip install -r requirements.txt as README instructed you'll face tons of errors, mostly because the dependency is messy and something magically depends on an older SciPy & NumPy which isn't delivered in wheels. And if you try to compile Scipy/Numpy on M1, good luck fiddling with that. I ended up ignoring these weird versioning issues and just went with the latest wheels. No issues so far.
Many many small issues adds up. It's painful to debug through all this. It couldn't utilize M1 hardware (pure CPU) but ANE doesn't support a very large neural network either so I think this is the best case.

All of this has been addressed.
UVR is now fully compatible with MacOS!
Please download the new version here (make sure to choose the installation for your architecture) -
https://github.com/Anjok07/ultimatevocalremovergui#macos-installation


Re-enable macOS CI jobs

It is still failing with:
Could NOT find Protobuf (missing: Protobuf_LIBRARIES Protobuf_INCLUDE_DIR)

Do github runners still get homebrew updates delayed?

Still not working. I hope this will be resolved once the next image will be released and rolled out: https://github.com/actions/runner-images/pull/7904.


CI: Move ios_tests to use Engflow's remote execution

Description:
CI: Move ios_tests to use Engflow's remote execution. Also extend the pattern to run the ios_tests workflow when the flow itself is modified.
Before (~1h20min):

After (~20min):

Risk Level: Low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A

/cc @ulfjack @goaway

/cc @ulfjack @goaway can you please take a look?

This is working now after the latest config changes in the cluster


Update codemirror and lezer

Looks like you added the build output to the repository.

It's already in the repo: https://github.com/lezer-parser/json/tree/master/docs
It will break the demo page when it is removed or am I missing something?

Ah, sorry, github's diff showed this with only green dots, leading me to assume it was newly added.
I do think keeping build output in the repository is a bad idea, but I'll let @satchmorun decide how to merge this.

I agree, I will remove the build output. When @satchmorun decides to keep the build in the repo, he can add it later again.

I do think keeping build output in the repository is a bad idea, but I'll let @satchmorun decide how to merge this.

I agree, I will remove the build output. When @satchmorun decides to keep the build in the repo, he can add it later again.

My thinking at the time was that the docs directory output was a useful thing to be able to click around to see how the thing actually worked.
I'm assuming that eventually something like that will be possible on the CodeMirror docs site, which would render this (and the associated github pages site) redundant.
And the real use of it is for development anyway, so I'm happy to remove the build output and the github pages site. (NOTE: It seems to be possible to actually use a build step to build a pages site, but feels like more effort than it's worth at the moment).
All that's to say: this is good with me 👍. Thanks for the PR!


Exercises-w1

Exercises
Name
Roy Kuijper
Class
211
GitHub username
https://github.com/RooyyDoe
Bootcamp exercises
https://github.com/RooyyDoe/js-bootcamp
Bootcamp notes
https://github.com/RooyyDoe/js-bootcamp

Seems like you've put a lot of effort into understand the exercises and comming up with solutions, good code comments (in dutch?) and torough explanation.
We haven't put your exercises in the admin sheet for this week since it was empty when Rijk checked. Make sure you hand them in after week 2


Memory leak or misconfigured

We are having an issue using Redisson with Hibernate 5.3.
It appears the items are getting removed correctly due to the TTL, but their is still a Sorted Set that grows without ever getting pruned.
Any of the "redisson__map_cache__last_access__set:{**}" grow without ever having elements removed.
Current properties:
hibernate.cache.redisson.timestamps.expiration.time_to_live: 10
hibernate.cache.redisson.naturalid.expiration.time_to_live: 10
hibernate.cache.redisson.collection.expiration.time_to_live: 10
hibernate.cache.redisson.entity.expiration.time_to_live: 10
hibernate.cache.redisson.query.expiration.time_to_live: 10
Add sensible max. Otherwise they are unbounded.
hibernate.cache.redisson.timestamps.eviction.max_entries: 10000
hibernate.cache.redisson.naturalid.eviction.max_entries: 10000
hibernate.cache.redisson.collection.eviction.max_entries: 10000
hibernate.cache.redisson.entity.eviction.max_entries: 10000
hibernate.cache.redisson.query.eviction.max_entries: 10000
Also tried adding:
hibernate.cache.redisson.timestamps.expiration.max_idle_time: 0
hibernate.cache.redisson.naturalid.expiration.max_idle_time: 0
hibernate.cache.redisson.collection.expiration.max_idle_time: 0
hibernate.cache.redisson.entity.expiration.max_idle_time: 0
hibernate.cache.redisson.query.expiration.max_idle_time: 0
I've traced the code and I don't see any locations where the redisson__map_cache__last_access__set has any elements removed.
In the MapCacheEvictionTask, it does pass in the key, but it is never used in the LUA.
Let me know if I'm missing something in the config. Thanks!

Is this a regression bug?

Thanks for report! It's not a regression bug.

Fixed

Thank you! Sorry to comment on a closed issue, any change to get a release so soon after the 3.12.2 was released 2 days ago?

Probably I'll make another one in 10 days.


Stream metrics 1.5

Rebased #981 and #986 against master

(copied from comment in #981)
This refactoring addresses #950.  It is a large change, so I want to give the community a chance to look it over/ask questions before I start in on the 1.5.0 RC series (this will be the first commit there, if satisfactory).
The main changes in this PR:

Instead of command executions writing directly to HystrixRollingNumber/HystrixRollingPercentile data structures, they now write to a Subject that models a stream of command execution events.
HystrixCommandMetrics now consumes this stream in 4 ways:

Cumulative counter of event types
Rolling counter of event types
Rolling distribution of command latencies
Health counts of a circuit

Sample-based stream of thread pool / queue utilization

This change has not hurt performance when used in Netflix production for ~24 hours.  I will publish jmh results here as well.  I'll also start linking more Github issues to this ticket as I build on this work.
There should be no visible change in application behavior as a result of this change (see caveats below).  However, it enables future work to add new functionality. (like #333)
Caveats (known differences in behavior)

Maximum concurrency of command is always 0 (HystrixCommandMetrics#getRollingMaxConcurrentExecutions)
HdrHistogram is used for latency distributions.  Percentile calculation is different from HystrixRollingPercentile.
On application startup, an object pool of Histograms is allocated.  This is currently hardcoded to 1000 objects.  It's an area I want to explore more deeply

Comments/questions are welcomed and encouraged.


Customer breakthrough should not be full width on this page

https://franklin.moleculardevices.com/technology/time-resolved-fluorescence-trf-tr-fret-htrf#overview

@duynguyen  To main consistency, wherever this styling is coming for customer breakthroughs the image should be on the left and the text on the right. Please update the same on this page. https://franklin.moleculardevices.com/technology/time-resolved-fluorescence-trf-tr-fret-htrf#overview
Also, the color for the "Customer Breakthroughs" title looks a bit off, please check that too.
styling should be like this example on Franklin -

https://franklin.moleculardevices.com/products/microplate-readers/multi-mode-readers

@duynguyen To main consistency, wherever this styling is coming for customer breakthroughs the image should be on the left and the text on the right. Please update the same on this page. https://franklin.moleculardevices.com/technology/time-resolved-fluorescence-trf-tr-fret-htrf#overview Also, the color for the "Customer Breakthroughs" title looks a bit off, please check that too.
styling should be like this example on Franklin -  https://franklin.moleculardevices.com/products/microplate-readers/multi-mode-readers

Done, this is the fix: https://issue-1004--moleculardevices--hlxsites.hlx.page/technology/time-resolved-fluorescence-trf-tr-fret-htrf


Make reinterpret cast conversions (fromBits / asUInt / etc) consistent

Split of the discussion from #424. I think we want to do the temporary fix first while we discuss how to improve the API.
@sdtwigg 's approach was:

Our approach was the following:

asUInt or from[blah] on a Clock is a runtime-exception (iirc FIRRTL doesn't support this at all anyway)
Instead of fromBits, Data has an "asTypeOf[T <: Data](model: T): T" since most commonly people were using fromBits to perform a cast and this made all casts look like as[blah]. This also made asTypeOf invert itself. (There are potentially some odd cases when unknown-width SInts are involved; however, it was generally advised to use only known width things under threat of emitting errors later.)
asTypeOf uses connectFromUInt(target: UInt) for each element, which basically acts like your proposed fromBits (but without having to worry about which Bits it is). connectFromUInt also doesn't create any wires.

So a concrete (API change) proposal would be:

Implement asTypeOf(model) for the fromBits conversions.
Deprecate fromBits. It didn't really have an intuitive API.

We will deprecate fromBits and replace with Data.asTypeOf(model) as per Stephen's approach.

So the new asTypeOf API has been around for a while, though fromBits hasn't been deprecated yet.
Since fromBits is a Data method, deprecation in chisel3._ only (and not in Chisel._) will likely be difficult. Options include:

Deprecate fromBits across the board. Legacy users will receive deprecation warnings.
Remove fromBits from Data., and use implicit casts (PML pattern) to support fromBits with different implementations in chisel3. and Chisel.. This will also be the only option to support fromBits in Chisel. if we want to remove it completely from chisel3._.

We will try #2 (PML to like FromBits-able) and manually fix up any instances where it breaks existing code.


Stripe charges

Hello,
I deployed stripe, registered new user which created in "stripe_customer".
How can I use this - /users/{userId}/charges/{id}.
userId is like "cus_AIUNyLkyv8mUUQ"
what is "id"?
How can I add amount to this request?
Thanks

@iThinkersTeam  I am having the same issue too. There is no  /users/{userId}/charges/{id} directory.

From your server you would need to push a node to the /users/{userId}/charges path.
For instance:
admin.database().ref('/users/' + userId + '/charges').push({amount: 100}); // This will charge 100 USD

I try configure for ios app.
I wrote
class func pay(amount: Int) {
let amount: [String: Any] = ["amount": amount]
if let curentUser = FIRAuth.auth()?.currentUser {
userRef.child(curentUser.uid).child("charges").setValue(amount)
}
}
I check logo in stripe and found error

I added test card manually for this user.
Also in the morning I have success request

But I checked my panel and I don't see this payment.
Have you any idea?
Thanks a lot for your help

I've repurposed #55 to create a front end to see the example in action + the expected DB schema. I'll close in favor of that.
It looks like in your code you're not pushing also from that example, the user needs a payment method. I'll make this more clear in the front end I'm putting together.


Licencing possible Issue.

Hey, I might be wrong, can find exactly where I was reading it. But remember you mentioned somewhere you used code that was without any licences. I just find out such code is all under copyright of owner him self (as I understood). Thought it is good idea to mention it.

@d0pa you are right about the copyright, but i think i checked all external code snippets i used for the license.
Do you mean the code used in distributed.py? The code i used there was based on code from a website, but the code snippets on the website are all licensed under an open source license. I wrote a note about this in distributed.py.

ok, I was thinking you took code from github that was without licences.


Add support for Discord slash commands, channel change announcements, and opt-in updater role

This PR adds supports for the following features:

Discord slash commands (Closes #5)
Sending announcements for opt-in creations, renames, or redescribes (Closes #21, Closes #23)

This supersedes #25

An opt-in updater role for rename/redescribe commands.

This supersedes #26

Slash Commands
The following slash commands have been added to mirror the existing text commands:

/create [channel name] [channel description] - Creates a new opt-in channel, and associated role.
/join [channel name] - Assigns that member the role to view the channel given.
/leave [channel name] - Removes the role to view the channel given from that user.
/list - Lists all of the opt-in channels.
/rename [current channel name] [new channel name] - Changes the name of an existing opt-in channel.
/redescribe [channel name] [channel description] - Changes the description of an existing opt-in channel.

A note on the /leave command
A noteworthy change from the existing text commands is that the /leave command is implemented as a guild command instead of a DM command. Thanks to ephemeral messages, the /leave command can still allow users to leave an opt-in channel without making the event noticeable to everyone in the server. Ephemeral messages are private to the user who issued the command (the fact that the command was issued is also private in this case) and will disappear after some time or when the Discord client is restarted.
Channel name auto-completion
For any parameter that accepts a channel name, auto-complete has been implemented based on the existing opt-in channel names. To provide better performance for auto-complete, caching of opt-in channel names in memory has been implemented. The cache life time is configurable via a new OptinChannelCacheLifeTimeInMinutes setting, but the cache is also cleared any time a bot command is issued that modified the set of opt-ins.
Channel Change Announcements
The bot can now send announcement messages when changes are made to opt-in channels.

Add an announcement channel value to the guild repository configuration (AnnouncementChannel)
Add an Announce behavior used to send announcement message
If an announcement channel is configured for a guild, send announcement messages when a opt-in channel is created, renamed, or redescribed.

Opt-in Updater Role
A new role OptinUpdatersRoles in the guild repository configuration allows guild owners more control over who can rename or redescribe opt-in channels using ChillBot commands.

Thank you @WardenGnaw for the thorough review.


Failed to start virtuoso server

Hi, I have followed the instructions in REAMDE, but still have problems when starting the virtuoso server.
I downloaded and unzipped virtuoso_db.zip, and confirmed the structure of my directory as follows:
.
├── LICENSE
├── README.md
├── fix_freebase_literal_format.py
├── numeric_properties.txt
├── virtuoso.py
└── virtuoso_db
    ├── virtuoso.db
    ├── virtuoso.ini
    ├── virtuoso.lck
    └── virtuoso.log

1 directory, 9 files

Then I ran the command: python3 virtuoso.py start 3001 -d virtuoso_db, the process exits after ~15 seconds, showing the following output:
RUNNING: cat /proc/meminfo | grep MemTotal | awk '{print $2}'
527485136 KB free, using 9890346.299999999 buffers, 4945173.149999999 dirty buffers
[Database]
DatabaseFile = virtuoso_db/virtuoso.db
ErrorLogFile = virtuoso_db/virtuoso.log
LockFile = virtuoso_db/virtuoso.lck
TransactionFile = virtuoso_db/virtuoso.trx
xa_persistent_file = virtuoso_db/virtuoso.pxa
ErrorLogLevel = 7
FileExtend = 200
MaxCheckpointRemap = 2000
Striping = 0
TempStorage = TempDatabase

[TempDatabase]
DatabaseFile = virtuoso_db/virtuoso-temp.db
TransactionFile = virtuoso_db/virtuoso-temp.trx
MaxCheckpointRemap = 2000
Striping = 0

[Parameters]
ServerPort = 13001
LiteMode = 0
DisableUnixSocket = 1
DisableTcpSocket = 0
ServerThreads = 100 ; increased from 20
CheckpointInterval = 60
O_DIRECT = 1 ; increased from 0
CaseMode = 2
MaxStaticCursorRows = 100000
CheckpointAuditTrail = 0
AllowOSCalls = 0
SchedulerInterval = 10
DirsAllowed = .
ThreadCleanupInterval = 0
ThreadThreshold = 10
ResourcesCleanupInterval = 0
FreeTextBatchSize = 100000
PrefixResultNames = 0
RdfFreeTextRulesSize = 100
IndexTreeMaps = 256
MaxMemPoolSize = 200000000
PrefixResultNames = 0
MacSpotlight = 0
IndexTreeMaps = 64
NumberOfBuffers = 9890346.299999999
MaxDirtyBuffers = 4945173.149999999

[SPARQL]
ResultSetMaxRows = 50000
MaxQueryCostEstimationTime = 600 ; in seconds (increased)
MaxQueryExecutionTime = 180; in seconds (increased)

[HTTPServer]
ServerPort = 3001
Charset = UTF-8
ServerThreads = 15 ; increased from unknown

virtuoso_db/virtuoso.ini
==== Starting Virtuoso server for virtuoso_db on port 3001...
RUNNING: /home/ubuntu/data/virtuoso-opensource/bin/virtuoso-t +configfile virtuoso_db/virtuoso.ini +wait
The VDBMS server process terminated prematurely
after opening the database.

Then I checked virtuoso.log, which gave the following message:
                Fri Jul 14 2023
22:46:14 OpenLink Virtuoso Universal Server
22:46:14 Version 07.20.3229-pthreads for Linux as of Aug 15 2018
22:46:14 uses parts of OpenSSL, PCRE, Html Tidy

Unfortunately, the server exits almost immediately and there isn't much informative log output.
Any help in troubleshooting this problem would be greatly appreciated.
System Information

Ubuntu 20.04.6 LTS
Python 3.9.16

I met the same problem. When I check the log, it says:
18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB.
18:05:26 The process must have last terminated while growing the file.
18:05:26 Please contact OpenLink Customer Support

Just found the issue. The NumberOfBuffers and MaxDirtyBuffers in virtuoso.ini were set too high by virtuoso.py. Lowered the buffer sizes and the problem is resolved.

I met the same problem. When I check the log, it says:
18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

It seems like your issue is different from my case. Based on the error message, your database might be corrupted because the file size is not a multiple of 2MB. Can you verify the file size by running ls -l virtuoso.db? For reference, my virtuoso.db file size is<PHONE_NUMBER>92 bytes.

I met the same problem. When I check the log, it says:
18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

It seems like your issue is different from my case. Based on the error message, your database might be corrupted because the file size is not a multiple of 2MB. Can you verify the file size by running ls -l virtuoso.db? For reference, my virtuoso.db file size is<PHONE_NUMBER>92 bytes.

It says the size is<PHONE_NUMBER>6. I just download it from another link in issues (from OneDrive).

I met the same problem. When I check the log, it says:
18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support

It seems like your issue is different from my case. Based on the error message, your database might be corrupted because the file size is not a multiple of 2MB. Can you verify the file size by running ls -l virtuoso.db? For reference, my virtuoso.db file size is<PHONE_NUMBER>92 bytes.

It says the size is<PHONE_NUMBER>6. I just download it from another link in issues (from OneDrive).

I think we downloaded it from the same link. In that case, you might want to verify the database integrity, since it's even smaller than the zip file.

ls -l virtuoso.db

hi, I've lowered the size, but the problem still exists.
Now I lower the size like this:
NumberOfBuffers =<PHONE_NUMBER>21875
MaxDirtyBuffers = 7.602841186523437

Just found the issue. The NumberOfBuffers and MaxDirtyBuffers in virtuoso.ini were set too high by virtuoso.py. That's why the process was killed by the os without any output. Lowered the buffer sizes and the problem is resolved.

hi, I've lowered the size, but the problem still exists.
Now I lower the size like this:
NumberOfBuffers =<PHONE_NUMBER>21875
MaxDirtyBuffers = 7.602841186523437

@Lanskttee for your issue, which is different from @AiRyunn as they rightfully pointed out, is most likely due to a corrupted DB file. I'd suggest to download again and make sure the download is 100% complete.

@Lanskttee for your issue, which is different from @AiRyunn as they rightfully pointed out, is most likely due to a corrupted DB file. I'd suggest to download again and make sure the download is 100%

hi! I've downloaded it again and it didn't report this:
18:05:26 It is impossible to have a database file virtuoso_db/virtuoso.db with a length not multiple of 2MB. 18:05:26 The process must have last terminated while growing the file. 18:05:26 Please contact OpenLink Customer Support
But it still exits almost immediately. And it even didn't have a log. There is only a .ini file.

@Lanskttee What is the full output of virtuoso.py and the files in the virtuoso_db directory?

@Lanskttee What is the full output of virtuoso.py and the files in the virtuoso_db directory?

there is only a virtuoso.ini and virtuoso.db in the directory.
the output is:
$ python virtuoso.py start 3001 -d virtuoso_db
'D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/virtuoso-t' is not recognized as an internal or external command,
operable program or batch file.
RUNNING: cat /proc/meminfo | grep MemTotal | awk '{print $2}'
16608660 KB free, using<PHONE_NUMBER>21875 buffers, 7.602841186523437 dirty buffers
[Database]
DatabaseFile = virtuoso_db/virtuoso.db
ErrorLogFile = virtuoso_db/virtuoso.log
LockFile = virtuoso_db/virtuoso.lck
TransactionFile = virtuoso_db/virtuoso.trx
xa_persistent_file = virtuoso_db/virtuoso.pxa
ErrorLogLevel = 7
FileExtend = 200
MaxCheckpointRemap = 2000
Striping = 0
TempStorage = TempDatabase
[TempDatabase]
DatabaseFile = virtuoso_db/virtuoso-temp.db
TransactionFile = virtuoso_db/virtuoso-temp.trx
MaxCheckpointRemap = 2000
Striping = 0
[Parameters]
ServerPort = 13001
LiteMode = 0
DisableUnixSocket = 1
DisableTcpSocket = 0
ServerThreads = 100 ; increased from 20
CheckpointInterval = 60
O_DIRECT = 1 ; increased from 0
CaseMode = 2
MaxStaticCursorRows = 100000
CheckpointAuditTrail = 0
AllowOSCalls = 0
SchedulerInterval = 10
DirsAllowed = .
ThreadCleanupInterval = 0
ThreadThreshold = 10
ResourcesCleanupInterval = 0
FreeTextBatchSize = 100000
PrefixResultNames = 0
RdfFreeTextRulesSize = 100
IndexTreeMaps = 256
MaxMemPoolSize = 200000000
PrefixResultNames = 0
MacSpotlight = 0
IndexTreeMaps = 64
NumberOfBuffers =<PHONE_NUMBER>21875
MaxDirtyBuffers = 7.602841186523437
[SPARQL]
ResultSetMaxRows = 50000
MaxQueryCostEstimationTime = 600 ; in seconds (increased)
MaxQueryExecutionTime = 180; in seconds (increased)
[HTTPServer]
ServerPort = 3001
Charset = UTF-8
ServerThreads = 15 ; increased from unknown
virtuoso_db/virtuoso.ini
==== Starting Virtuoso server for virtuoso_db on port 3001...
RUNNING: D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/virtuoso-t +configfile virtuoso_db/virtuoso.ini +wait

@Lanskttee The error suggests that the command isn't recognized. Please make sure virtuoso-t is located in D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/ and executable.

@Lanskttee The error suggests that the command isn't recognized. Please make sure virtuoso-t is located in D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/ and executable.

hi, I checked the mode and it is:
-rwxrwxrwx+ 1 lans None 15456464 Aug 16  2018 virtuoso-t
But it still reports:
/bin/sh: line 1: D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource/bin/virtuoso-t: cannot execute binary file: Exec format error

it looks like you are on Windows. probably need to change all paths to Windows format.

@Lanskttee Also virtuoso-t should be a binary executable file, not a shell script. You should be able to run it with ./virtuoso-t. You could try ./virtuoso-t --help to see if there is any output.

@Lanskttee Also virtuoso-t should be a binary executable file, not a shell script. You should be able to run it with ./virtuoso-t. You could try ./virtuoso-t --help to see if there is any output.

it report like this:
$  ./virtuoso-t --help
bash: ./virtuoso-t: cannot execute binary file: Exec format error

it looks like you are on Windows. probably need to change all paths to Windows format.

hi!
When I changed the slash, it just reports like this:
/bin/sh: line 1: D://pycharm//KB-BINDER-main//fujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnu//virtuoso-opensource//bin//virtuoso-t: cannot execute binary file: Exec format error
And backslash like this:
/bin/sh: line 1: D:pycharmKB-BINDER-mainfujian//virtuoso-opensource.x86_64-generic_glibc25-linux-gnuvirtuoso-opensourcebinvirtuoso-t: No such file or directory
It disappeared in the output.
BTW, I do on Window, but I use cygwin to run it.

@Lanskttee I'm not familiar with Windows, but this seems to be a problem with virtuoso installation. You can try reinstalling virtuoso, or building virtuoso 7.2.5 from source (https://github.com/openlink/virtuoso-opensource/releases).

we meet the error
Error HTTP/1.1 404 File not found
The requested URL was not found    URI  = '/'

when open http://localhost:3001

我们遇到了错误
Error HTTP/1.1 404 File not found
The requested URL was not found    URI  = '/'

打开时 http://localhost:3001

Is this problem solved? I also encountered the same problem.


[#4298]fix(flink-connector): fix scala version in flink connector

What changes were proposed in this pull request?
Flink only supports Scala 2.12. All scala API will removed in the future version.
https://issues.apache.org/jira/browse/FLINK-23986
https://issues.apache.org/jira/browse/FLINK-20845
https://issues.apache.org/jira/browse/FLINK-13414
Why are the changes needed?
Fix: #4298
Does this PR introduce any user-facing change?

no

How was this patch tested?

./gradlew flink-connector:publishToMavenLocal -PdefaultScalaVersion=2.13

@FANNG1 Please take a look. Thanks.

we could exclude the Flink connector from Gravitino if the scala version is 2.13, like spark-connector in setting.gradle.kts
if (scalaVersion == "2.12") {
  include("spark-connector:spark-3.3", "spark-connector:spark-runtime-3.3")
  project(":spark-connector:spark-3.3").projectDir = file("spark-connector/v3.3/spark")
  project(":spark-connector:spark-runtime-3.3").projectDir = file("spark-connector/v3.3/spark-runtime")
}

@yuqi1129 do you have time to review this PR?

@yuqi1129 do you have time to review this PR?

I have reviewed it and I'm okay with the change.


Need to find a way to validate public API parameters / structures

Thinking that if we want to integrate in TypeScript, we would need to declare all public APIs in a compatible way.


[ANTLR grammar] Minor observations.

This issue points out some parts of the ANTLR-based Dart grammar that don't seem to be intended to be that way or can perhaps be made a little stricter.

A) Consider localVariableDeclaration -> initializedVariableDeclaration -> declaredIdentifier:

I think a local variable should not be allowed to be annotated as being covariant?
i.e. I think the following is not valid Dart code:
void foo() {
  covariant final i = 0;
}

the same for forLoopParts via its first production again via declaredIdentifier:

B) There are three rules that seem to not be related to the lexical grammar, that are not meant to be start symbols and do not appear on the rhs of any production and are therefore unused. They are:

functionPrimaryBodyPrefix
functionExpressionBodyPrefix
functionBodyPrefix

I think they can either be removed or should not be unused?

C) finalConstVarOrType allows fieldFormalParameters to be const and late, which doesn't look right. Same for simpleFormalParameter via declaredIdentifier.

and same for the first production of forLoopParts via declaredIdentifier.

D) I think the rhs of the comparison in uriTest can be restricted to be a stringLiteralWithoutInterpolation to capture the fact that interpolated strings are not allowed there (as is already being done with uris)

E) The single/double line single/double quote string literals in a singleStringWithoutInterpolation allow for simple interpolations to occur in it:

Fixing this (by having explicit string literals for each one with and without simple interpolations) will introduce an ambiguity in the regular part of the lexical grammar. I'm not sure what the best way would be to express and resolve that ambiguity in ANTLR, but I think, conceptually, we should disambiguate in favor of the non interpolated version of these strings and use both versions in the syntactical grammar as a replacement for the single interpolated version that is being used today?

F) I think onParts would be better expressed as onPart+. This seems to be a valid substitution to me, and would make this part of the grammar easier to understand and visualize.

G) The firstset of libraryDefinition contains FEFF. libraryDefinition is a start symbol. partDeclaration is also a start symbol, but the firstset of partDeclaration does not contain FEFF. Is that intentional? The descriptions of FEFF that I could find online make me think that partDeclaration should also start with an optional FEFF (to match libraryDefinition)?

H) There's a small typo here:
funciton -> function
https://github.com/dart-lang/sdk/blob/78ac63303b57a9804f3e0958c7f9afafb05f484d/tools/spec_parser/Dart.g#L132
And the convention of 4 spaces after the colon and pipe is being violated here:
https://github.com/dart-lang/sdk/blob/78ac63303b57a9804f3e0958c7f9afafb05f484d/tools/spec_parser/Dart.g#L640-L668

(I hope the images are helpful!)

[...] but this approach is easier to specify and has better error recovery.

That makes sense to me, thank you for the explanation.
Some context: I hope to be able to assume that the ANTLR-based grammar specification is sound (parses only syntactically valid Dart programs) and complete (parses all syntactically valid Dart programs) so that I can more easily verify that the grammar that I end up using remains correct as Dart continues to evolve.
Are soundness and completeness (as described above) goals of the ANTLR-based grammar specification?

@lrhn already described the situation very precisely: In some cases, including the modifier covariant, we could choose to make certain situations like covariant top-level variables a syntax error (in which case the grammar would be larger and have more near-duplicated elements) or we could make it a compile-time error which is not a syntax error (which is what we actually do). The main reason for doing this is that the grammar ruleset explodes if we encode too many restrictions into the grammar, compared to the situation where we make it a "normal" compile-time error.
However, there is one more aspect of this: We do not specify which compile-time errors are syntax errors, and which ones are not. The point is that it is up to each Dart parser to make that decision based on its purpose and usage.
If we use a grammar that derives a larger language than the one in the language specification, and then reject the programs that are outside the language using non-syntax compile-time errors, the parser may well have a more helpful behavior during error recovery, and it may be able to give more informative error messages. This means that it's perfectly meaningful for a tool to use a more permissive grammar than the one in the specification.
The only requirement is that if P is a program which is derivable by the specified grammar, and P doesn't have any compile-time errors, then Dart tools should accept P, and if it is executed then it should have the run-time behavior which is specified.
Dart.g is intended to serve as a mechanized checking device on the specification grammar. That is, Dart.g should be equivalent to the sum of the grammar snippets in the language specification. We may then use it to detect parsing success/failure discrepancies with actual Dart code, which will help us to detect elements of the specification grammar that aren't consistent with the behavior of the language tools. We may then correct the behavior of the tools, or change the specification such that they are consistent, whichever is more appropriate.
In that sense, Dart.g as executed via the ANTLR v4 parser generator is intended to be equivalent to the language specification. In particular, both the language specification and the ANTLR parser rely on the ordering of alternatives in each grammar rule to disambiguate, in the case where multiple choices will produce a successful parse.
(It should be noted that the language specification is at Dart 2.10 at this point, and updates to include null safety (in particular, Dart 2.13), are under review. In contrast, Dart.g is much further ahead, e.g., it includes patterns and records. But when the language specification catches up with newer language versions, the two grammars should be equivalent. Issues reporting discrepancies are very welcome. ;-)
With all these things in mind, the soundness and completeness goals are as follows: Dart.g for language version x.y should be consistent with the language specification for language version x.y. Tools may choose to use a more permissive grammar, as long as they filter out the additional programs (the ones that "should have had a syntax error") by means of non-syntax compile-time errors.

Thank you @eernstg for your help. I will now consider the ANTLR-based grammar specification  as a part of the LaTeX-based specification and not as a single independent unit. I made some assumptions that were not true, thank you for correcting them.


proposal: errors: Add All(error) iter.Seq[error]

Proposal Details
This is a follow up to #53435. It is now possible to build a tree of wrapped errors, but there is no convenient way to iterate through all members of the tree. It should just do a simple depth-first traversal of all the errors it finds with Unwrap() error/Unwrap() []error. The main use case for this is some error reporting service that wants to log the concrete types of all the sub-errors it finds in an error.

A corollary: The behavior should probably not be dependent on the wrap status of the passed error, so calling All(err) where err doesn't wrap any other errors should still yield an iter.Seq that just yields err.

Yes, it should handle all of that automatically. I think this is a working implementation:
func All(err error) iter.Seq[error] {
	return func(yield func(error) bool) {
		all(err, yield)
	}
}

func all(err error, yield func(error) bool) bool {
	if err == nil {
		return true
	}
	if !yield(err) {
		return false
	}
	if wrapped := errors.Unwrap(err); wrapped != nil {
		return all(wrapped, yield)
	}
	multi, ok := err.(interface{ Unwrap() []error })
	if !ok {
		return true
	}
	for _, suberr := range multi.Unwrap() {
		if !all(suberr, yield) {
			return false
		}
	}
	return true
}

I'm imagining something like Sentry.io recording the tree of an error for debugging purposes. In those cases, you would want all of the children, but yes you'd also want to know what was the parent and what was the child so you can represent that in the UI. So maybe this is something third parties should do for themselves…

CC @neild

I'm going to close this because I like the proposed AllAs in https://github.com/golang/go/issues/66455 more than plain All and to consolidate conversations to one place.


Run final check to ensure we have all RF4 contracts in OSO

What is it?
All artifacts included in RF4 applications should be available in artifacts_by_project_v1. We can confirm this by querying:
with expected_artifacts as (
  select *
  from `static_data_sources.agora_rf4_artifacts_by_app`
  where application_id in (
    select application_id
    from `static_data_sources.agora_rf4_to_ossd`
    where oso_project_name is not null
  )
  and artifact_type = 'CONTRACT'
)

select
  expected_artifacts.*
from expected_artifacts
left join `oso.artifacts_by_project_v1` abp
  on
    expected_artifacts.artifact = abp.artifact_name
    and expected_artifacts.artifact_source = abp.artifact_source
where abp.project_id is null
order by project_name

Ideally, we get zero results back. Currently there are 69 missing contracts, available here, across 20 project applications.

https://github.com/opensource-observer/oss-directory/pull/415


Copying to clipboard not working

It's creating the gists as it should, they're showing up in my account. But it's not copying a url into my clipboard. Furthermore, it's breaking the text that already was in my clipboard.
Arch Linux x64, atom 1.0.2, gist-it 0.7.0
In addition, I'm unable to create a gist by hitting enter after entering a name. I must click the "Gist It" button with my mouse.

Same here.  Debian Jessie x64, atom 1.02, gist-it 0.7.0.


fix: flutterfire_ui README links

Description
The flutterfire_ui README file had some links broken, as they were pointing to a 'docs' folder instead of 'doc'. This PR intends to fix it.
Related Issues
#8628
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]).
This will ensure a smooth and quick review process. Updating the pubspec.yaml and changelogs is not required.

[*] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[NA] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See [Contributor Guide]).
[NA] All existing and new tests are passing.
[NA] I updated/added relevant documentation (doc comments with ///).
[NA] The analyzer (melos run analyze) does not report any problems on my PR.
[*] I read and followed the [Flutter Style Guide].
[*] I signed the [CLA].
[*] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change.
[*] No, this is not a breaking change.

Thanks!


Incomplete information on "Workloads > kubecf" (no pods)

Frontend Deployment type

[ ] Cloud Foundry Application (cf push)
[x] Kubernetes, using a helm chart
[ ] Docker, single container deploying all components
[ ] npm run start
[ ] Other (please specify below)

Backend (Jet Stream) Deployment type

[ ] Cloud Foundry Application (cf push)
[x] Kubernetes, using a helm chart
[ ] Docker, single container deploying all components
[ ] Other (please specify below)

SUSE CaaSP4 added as "CaaSP Endpoint" (Name: CaaSP4-tf4)
Expected behaviour
When I visit "Workloads > kubecf", status for pods is shown like for other workloads.
Actual behaviour
While information for kubecf is filled, it's incomplete - e.g. for pods it reports "0".
Steps to reproduce the behavior

Use helm-3.1.3
Install CAP-2.0-RC3 (kubecf-2.2.2) to CaaSP4
Install Console-3.2.1-RC1
Add CaaSP as endpoint to Stratos (CaaSP-endpoint, kubeconfig upload)
Navigate to "Workloads > kubecf" and find pods report: "0 pods, 0 containers, 0 ready, 0 not ready"

Log output covering before error and any error statements
kubectl logs stratos-0 proxy -n stratos snippet
...
    stemcells:
    - alias: default
      os: ubuntu-xenial
      version: "621.59"
'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: yaml: did not find expected comment or line break
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: yaml: line 11: could not find expected ':'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: yaml: did not find expected comment or line break
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/eirini.yaml
# Create the bits secret
'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/ingress.yaml
# If the ingress feature (ingress-controller) is not enabled, create public services to allow
# public ingress to the cluster.
'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/ops.yaml
# This template creates a ConfigMap for each ops file under assets/operations.
'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/properties.yaml
# This ConfigMap creates an ops-file containing the user provided properties.
'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: Object 'Kind' is missing in '# Source: kubecf/templates/sync-integration-tests.yaml
# SITS only makes sense when using Diego, for this reason, we only enable it if Eirini is not
# enabled.
'
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "BOSHDeployment" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksJob" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksJob" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksSecret" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksSecret" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksSecret" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
ERRO[Mon Jun  8 13:56:59 UTC 2020] Helm Manifest Parser: Error while decoding YAML object. Err was: no kind "QuarksStatefulSet" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery@v0.0.0-20191001043732-d647ddbd755f/pkg/runtime/scheme.go:101"
WARN[Mon Jun  8 13:56:59 UTC 2020] Passthrough response: URL: https://tf4-c4-lb.cap.suse.de:6443/apis/policy/v1beta1/namespaces/kubecf/podsecuritypolicys/kubecf-default/status, Status Code: 404, Status: 404 Not Found, Content Type: application/json, Length: 174
WARN[Mon Jun  8 13:56:59 UTC 2020] {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
WARN[Mon Jun  8 13:57:00 UTC 2020] Passthrough response: URL: https://tf4-c4-lb.cap.suse.de:6443/apis/policy/v1beta1/namespaces/kubecf/podsecuritypolicys/kubecf-default/status, Status Code: 404, Status: 404 Not Found, Content Type: application/json, Length: 174
WARN[Mon Jun  8 13:57:00 UTC 2020] {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
WARN[Mon Jun  8 13:57:11 UTC 2020] Passthrough response: URL: https://tf4-c4-lb.cap.suse.de:6443/apis/policy/v1beta1/namespaces/kubecf/podsecuritypolicys/kubecf-default/status, Status Code: 404, Status: 404 Not Found, Content Type: application/json, Length: 174
WARN[Mon Jun  8 13:57:11 UTC 2020] {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}

I had a quick look into this and saw similar failures in the backend. It looks like there's multiple issues when trying to pass helm's manfiest for KubeCf's.

Err was: Object 'Kind' is missing in - These are issues with KubeCf, they have objects without kinds. We'd need to chase them and see if this is expected
Err was: no kind "QuarksStatefulSet" is registered for version "quarks.cloudfoundry.org/v1alpha1" in scheme "k8s.io/apimachinery - These are due to non-standard resources, not sure we can do anything about these.. would need to know at build time and add them to the schema we use to pass the manifest (see https://github.com/kubernetes/client-go/issues/193#issuecomment-594440015)
Other errors - These could be errors with how we chop up the manifest or non-standard text in the manifest itself

To solve this we'd probably need to manually chop up the kubecf manifest returned by helm and parse in ever smaller chunks to narrow down on the issue.


Execution order and priority

I would like to be able to prioritize the execution order of certain jobs such that their dependencies are not run until they are needed.
For example, if I have a a rule that makes parts and another that aggregates those parts, I typically see all parts processed first, followed by the aggregation for each target.
I would like to be able to use either of the methods described in this post to have Snakemake only work on the dependent parts when a new aggregated target is needed.
Here is a minimal example:
targets = ["A", "B", "C"]
parts = ["1", "2", "3"]

rule all:
    input: [v + ".final" for v in targets]

rule make_target:
    input: lambda wc: [wc.target + v for v in parts]
    output: "{target}.final"
    priority: 3
    resources: batch=1
    shell: "echo {input:q} > {output}"

rule make_part:
    output: "{target}{part}"
    shell: "echo > {output}"

I would expect the order of the jobs to be:
A1, A2, A3, A.final, B1, B2, B3, B.final, etc.
Regardless if I use --resources batch=1 or not, I get the following order:
A1, A2, A3, B1, B2, B3 ... A.final, B.final, C.final
I am using Snakemake v5.10.0
Thank you for your help!

@johanneskoester with a little guidance, I would be happy to help contribute to this feature.


Add HLS to dev shell

Installing it per-project is preferred as HLS is bound quite tightly to specific versions of GHC.
I had a binary available for download from CI with this. Is that also the case on macOS? I wouldn't want to merge this if it needs to be built from source.

I had a binary available for download from CI with this. Is that also the case on macOS? I wouldn't want to merge this if it needs to be built from source.

How do I test this? I checked out the branch but it didn't look like anything was installed/built by Nix.
My .envrc has use nix.

My .envrc has use nix.

On master now, and with this PR, I think it should instead be use flake.

How do I test this? I checked out the branch but it didn't look like anything was installed/built by Nix.

Possibly because of the above. I'd guess your direnv has you stuck in an old shell.
Once that's fixed you'll either download ~200MB and be on your way, or you'll download and then start building HLS, which would take a while.
The idea is that you remove any global HLS (probably installed via ghcup) and use the one defined in Nix dev shells on a per-project basis.

On master now, and with this PR, I think it should instead be use flake.

Ah, okay. I just tried this on master and it seems like it's building stuff. Once that's done I'll try switching to this branch to see if it just downloads HSL or whether it needs to build it as well.

I just switched branch and it seems to building. I'll keep it running and let you know how long it takes.

Still going… do you want me to keep waiting?

Finished after 2h 19m 49s.

Oh... okay this might be a problem more broadly with macOS on master which would need addressing first.
Here's something else to test. Switch to master and run the following:
$ nix flake update

This updates our flake's inputs, most notably nixpkgs. For me, the result is a ~30MB download of nixpkgs followed by a few seconds of processing and then a ~360MB download of updated project dependencies. Once these have finished downloading there's nothing to build (significant enough to be slower than downloading, anyway) and I'm in the shell with the likes of ghci accessible.
Is your experience the same, or does it start slowly building a bunch of stuff?

Is your experience the same, or does it start slowly building a bunch of stuff?

It seems to be building:

Finished after 24m.

No need to commit, that's just a test to see if you have binaries available on Darwin on a fresh nixpkgs revision. (In the future if you wanted to bump nixpkgs, you'd run that command and then commit the lockfile, similar to Yarn.)
$ nix flake update --override-input nixpkgs github:NixOS/nixpkgs/e06bd4b

I was going to ask you to try this, the latest commit build for Darwin at time of writing, but that's also building for me. :angry:
It may be that haskell.packages.<version> isn't ever cached in Hydra CI, maybe? These are secondary to the primary versions at ghc, haskellPackages, etc, which for the nixpkgs revision on master targets GHC 9.0.
Let's try one more thing. This patch has us target the "main" version of everything Haskell in our nixpkgs revision. It's no good for master right now as it'd bump us up to GHC 9.0, but I'm hoping at least on Darwin you'll see binaries for everything here.
diff --git a/flake.nix b/flake.nix
index ed4e28c..b562d30 100644
--- a/flake.nix
+++ b/flake.nix
@@ -12,9 +12,10 @@
           buildInputs = with pkgs; [
             git

-            haskell.compiler.ghc8107
-            haskell.packages.ghc8107.cabal-install
-            haskell.packages.ghc8107.hspec-golden
+            cabal-install
+            ghc
+            haskell-language-server
+            haskellPackages.hspec-golden

             # For typechecking golden output
             nodejs

$ pbpaste | git apply
warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty
error: Package ‘compiler-rt-libc-7.1.0’ in /nix/store/qdsi66bin3h3yimbn35fxbj7fwdpi739-source/pkgs/development/compilers/llvm/7/compiler-rt/default.nix:96 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty

Oh yeah, that's a separate issue we documented back in https://github.com/unsplash/intlc/pull/162.
Try commenting out haskell-language-server. That doesn't solve this PR, but at least it should mean no building anything, and ghci et al in scope.

I tried that and it looked like nothing needed to build. 😄

One more thing I want to check to round this out, because I'm wondering if HLS is still built with the same M1-unfriendly LLVM package on a newer nixpkgs.
So, on a fresh master...:
diff --git a/flake.nix b/flake.nix
index ed4e28c..b562d30 100644
--- a/flake.nix
+++ b/flake.nix
@@ -12,9 +12,10 @@
           buildInputs = with pkgs; [
             git

-            haskell.compiler.ghc8107
-            haskell.packages.ghc8107.cabal-install
-            haskell.packages.ghc8107.hspec-golden
+            cabal-install
+            ghc
+            haskellPackages.hspec-golden

             # For typechecking golden output
             nodejs

Then:
$ nix flake update --override-input nixpkgs github:NixOS/nixpkgs/e06bd4b

And finally add haskell-language-server back to the dev shell.

warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty
error: Package ‘compiler-rt-libc-7.1.0’ in /nix/store/xc6s7iccz5nkm8vl9nybz698nq2ngxlc-source/pkgs/development/compilers/llvm/7/compiler-rt/default.nix:96 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty

:unamused:
Okay, so we could:

Use mainline GHC et al packages so they should be in the binary cache for all platforms. As per #162 we'll need to check these versions are compatible with ghcup's build matrix.
Conditionally only include HLS in the shell on Linux, and instruct macOS/M1 contributors to source HLS externally such as via ghcup.

There are more exotic solutions like haskell.nix but then folks have to start modifying their global Nix configs for access to these other binary caches. I suppose this doesn't preclude anyone from simply sourcing the dependencies externally and forgoing Nix anyway.
It really would be nicer to have HLS via Nix though for all platforms. I'm going to dig some more.


Refactor new page design and features

Codecov Report

Merging #73 (26bfc2a) into dev (91730d6) will increase coverage by 1.92%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##              dev       #73      +/-   ##
===========================================
+ Coverage   98.07%   100.00%   +1.92%
===========================================
  Files           8         8
  Lines         104        96       -8
===========================================
- Hits          102        96       -6
+ Misses          2         0       -2

Impacted Files
Coverage Δ

apps/publishing/lib/publishing/manage/article.ex
100.00% <ø> (ø)

apps/publishing/lib/publishing/manage.ex
100.00% <100.00%> (+3.33%)
:arrow_up:

apps/publishing/lib/publishing/markdown.ex
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91730d6...26bfc2a. Read the comment docs.


Remove pyload, replaced by official repository

First of all, thank you for your contribution! 😄
☑ Self Check before Merge

[x] I have tested the template using the method described in README.md thoroughly
[x] I have ensured that I put as much default values as possible (except passwords) to ensure minimum effort required for end users to get started.
[x] I have ensured that I am not using the "latest" tag as this tag is dynamically changing and might break the one-click app. Use a fixed version.
[x] I have made sure that instructions.start and instructions.end are clear and self-explanatory.
[x] Icon is added as a png file to the logos directory.

@Vdauphin Hey there! You can go ahead and merge it whenever you'd like, no need to wait for me!


Escape chars for string fields

LGTM. Go ahead with it if you're in a hurry, but please add a task to this week for some unit tests


Icons aren't showing up

Describe the bug
From today onwards (20-07-2024), some icons aren't showing up. Initially, folder icons are invisible, when I open that folder then the folder's icons show up.
Screenshots

Computer information (please complete the following information)

OS edition: [Windows 11 Enterprise]
OS build: [22631.3880]
VSCode version [1.91.1]

Closing as duplicate of #2428