Please make placeholder "Start typing..." customisable

Hello, I really like the papercups.
Would love if the placeholder could be a customisable for i18n.

great idea @webdeb! happy to make that change

I need this too. My site is in French

@BonneauPhilippe I'll add it today :) just focusing on moving the chat window into an iframe first 👍

Hi Alex. Thank you for your quickness. I have put the code in a Raw content element in the header of my Wordpress Theme (Themeco Pro). Here is the result https://dev.ecordl.ca
Philippe
http://ecommunication.ca/
Philippe Bonneau
68, route 132
Saint-Germain
G0L 3G0
581 729-1502
<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: ecommunication.ca http://ecommunication.ca/
Facebook: ecommunication.kamouraska
https://www.facebook.com/ecommunication.kamouraska/Twitter: eco_mmunication https://twitter.com/eco_mmunication
https://www.facebook.com/ecommunication.kamouraska/LinkedIn: philippebonneau https://ca.linkedin.com/in/philippebonneau
Skype: philippebonneau

Le 6 août 2020 à 13:45, Alex Reichert<EMAIL_ADDRESS>a écrit :
@BonneauPhilippe https://github.com/BonneauPhilippe I'll add it today :) just focusing on moving the chat window into an iframe first 👍
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/papercups-io/chat-widget/issues/16#issuecomment-670079283, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCT2WLJV2V2ZL5E55YCDATR7LTZ7ANCNFSM4PWRHF6Q.

@BonneauPhilippe hmm I can't seem to view https://dev.ecordl.ca -- I might not have permission? it's redirecting me to https://ecordl.ca/
can you take a screenshot? is it still not working?

Sorry Alex. I have moved this feature to a dev environment here https://dev.ecordl.ca
http://ecommunication.ca/
Philippe Bonneau
68, route 132
Saint-Germain
G0L 3G0
581 729-1502
<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: ecommunication.ca http://ecommunication.ca/
Facebook: ecommunication.kamouraska
https://www.facebook.com/ecommunication.kamouraska/Twitter: eco_mmunication https://twitter.com/eco_mmunication
https://www.facebook.com/ecommunication.kamouraska/LinkedIn: philippebonneau https://ca.linkedin.com/in/philippebonneau
Skype: philippebonneau

Le 6 août 2020 à 14:29, Alex Reichert<EMAIL_ADDRESS>a écrit :
@BonneauPhilippe https://github.com/BonneauPhilippe hmm I can't seem to view https://dev.ecordl.ca https://dev.ecordl.ca/ -- I might not have permission? it's redirecting me to https://ecordl.ca/ https://ecordl.ca/
can you take a screenshot? is it still not working?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/papercups-io/chat-widget/issues/16#issuecomment-670100352, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCT2WOLXN7LCW4YN5M6DQDR7LZAZANCNFSM4PWRHF6Q.

@BonneauPhilippe I'm afraid I can't access this url https://dev.ecordl.ca 😞

Hi Alex Sorry for the delay. There was a problem with the .htaccess file. You should be able to access the provided link now.
Bonne journée.
Philippe
http://ecommunication.ca/
Philippe Bonneau
68, route 132
Saint-Germain
G0L 3G0
581 729-1502
<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: ecommunication.ca http://ecommunication.ca/
Facebook: ecommunication.kamouraska
https://www.facebook.com/ecommunication.kamouraska/Twitter: eco_mmunication https://twitter.com/eco_mmunication
https://www.facebook.com/ecommunication.kamouraska/LinkedIn: philippebonneau https://ca.linkedin.com/in/philippebonneau
Skype: philippebonneau

Le 6 août 2020 à 13:45, Alex Reichert<EMAIL_ADDRESS>a écrit :
@BonneauPhilippe https://github.com/BonneauPhilippe I'll add it today :) just focusing on moving the chat window into an iframe first 👍
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/papercups-io/chat-widget/issues/16#issuecomment-670079283, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCT2WLJV2V2ZL5E55YCDATR7LTZ7ANCNFSM4PWRHF6Q.

@BonneauPhilippe  @webdeb you should now be able to customize this:
in HTML:
<script>
  window.Papercups = {
    config: {
      // ...
      newMessagePlaceholder: 'Customize me!', // "Start typing..." by default
      // ...
    },
  };
</script>
<script
  type="text/javascript"
  async
  defer
  src="https://app.papercups.io/widget.js"
></script>

in React:
first make sure you have the latest version installed:
"@papercups-io/chat-widget": "^1.0.16"

    <ChatWidget
        // ...
        newMessagePlaceholder='Customize me!', // "Start typing..." by default
        // ...
      />

let me know if you need any help or find any issues!

Thank you Alex. Do you plan a notification system?
Philippe
http://ecommunication.ca/
Philippe Bonneau
68, route 132
Saint-Germain
G0L 3G0
581 729-1502
<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: ecommunication.ca http://ecommunication.ca/
Facebook: ecommunication.kamouraska
https://www.facebook.com/ecommunication.kamouraska/Twitter: eco_mmunication https://twitter.com/eco_mmunication
https://www.facebook.com/ecommunication.kamouraska/LinkedIn: philippebonneau https://ca.linkedin.com/in/philippebonneau
Skype: philippebonneau

Le 7 août 2020 à 11:39, Alex Reichert<EMAIL_ADDRESS>a écrit :
@BonneauPhilippe https://github.com/BonneauPhilippe @webdeb https://github.com/webdeb you should now be able to customize this:
in HTML:

in React:
first make sure you have the latest version installed:
"@papercups-io/chat-widget": "^1.0.16"
<ChatWidget
// ...
newMessagePlaceholder='Customize me!', // "Start typing..." by default
// ...
/>
let me know if you need any help or find any issues!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/papercups-io/chat-widget/issues/16#issuecomment-670578397, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCT2WIHVRFPGM72FIA4UX3R7QN2JANCNFSM4PWRHF6Q.

@BonneauPhilippe yes we plan on improving our notifications :)
what kind of notification system would you like?

In browser notification, by email, mobile app or SMS. I want to be able to work on other tasks but be notified very quickly when someone writes in the chat. If I can be notified before the user click on the plane icon button would be great.
http://ecommunication.ca/
Philippe Bonneau
68, route 132
Saint-Germain
G0L 3G0
581 729-1502
<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: ecommunication.ca http://ecommunication.ca/
Facebook: ecommunication.kamouraska
https://www.facebook.com/ecommunication.kamouraska/Twitter: eco_mmunication https://twitter.com/eco_mmunication
https://www.facebook.com/ecommunication.kamouraska/LinkedIn: philippebonneau https://ca.linkedin.com/in/philippebonneau
Skype: philippebonneau

Le 7 août 2020 à 14:42, Alex Reichert<EMAIL_ADDRESS>a écrit :
@BonneauPhilippe https://github.com/BonneauPhilippe yes we plan on improving our notifications :)
what kind of notification system would you like?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/papercups-io/chat-widget/issues/16#issuecomment-670655978, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCT2WIPMW3MAT32OABB3C3R7RDKPANCNFSM4PWRHF6Q.

@BonneauPhilippe we currently support Slack notifications, and we're working on email next! email should be ready soon
by the way, were you able to fix the "Start typing..." placeholder? can I close this ticket now?

Problem solved. Tx
http://ecommunication.ca/
Philippe Bonneau
68, route 132
Saint-Germain
G0L 3G0
581 729-1502
<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: ecommunication.ca http://ecommunication.ca/
Facebook: ecommunication.kamouraska
https://www.facebook.com/ecommunication.kamouraska/Twitter: eco_mmunication https://twitter.com/eco_mmunication
https://www.facebook.com/ecommunication.kamouraska/LinkedIn: philippebonneau https://ca.linkedin.com/in/philippebonneau
Skype: philippebonneau

Le 7 août 2020 à 15:49, Alex Reichert<EMAIL_ADDRESS>a écrit :
@BonneauPhilippe https://github.com/BonneauPhilippe we currently support Slack notifications, and we're working on email next! email should be ready soon
by the way, were you able to fix the "Start typing..." placeholder? can I close this ticket now?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/papercups-io/chat-widget/issues/16#issuecomment-670684960, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCT2WIXG4KPLIQJ6ABV6KDR7RLEDANCNFSM4PWRHF6Q.

Feel free to open up another issue if you have anymore questions! Or feel free to email me at<EMAIL_ADDRESS>


add context cancelation

Some of these algorithms take a long time to calculate for massive amounts of data. It would be nice if you implemented context cancelation.

Could you please elaborate?
I'm not sure how context cancelation should be integrated into this lib, some example or a pull request would be nice.

For each function that potentially takes a long time to return, add ctx context.Context as the first argument.
Then inside the function, check for if ctx.Err() != nil { ... }

?


AdopterFoster: Add adopted pets to the dashboard

We need to enable adopters to go to their dashboard and view their adopted pets and the files associated with those pets. These files get uploaded by staff and are usually things like medical records, so should be shared with the adopter. The adopter should be able to download the files from the dashboard for a given pet.
We need:

adopted_pets route scoped under org > adopter_fosterer
adopted pets controller under the same scope
Index action that displays the adopted pets by that user who is an adopter
Policy and policy action to make sure the user is an authenticated adopter
design the pet cards and table per the image below
see the table in app/views/organizations/staff/pets/_file_attachment_table.html.erb except we do not need the buttons to attach or delete
the icon used for Adopted Pets in the image below is a little dog. We should stick to what's available in our theme at https://geeksui.codescandy.com/ so take a look there and see if you can find something. I was thinking of the star from this dash nav https://geeksui.codescandy.com/geeks/pages/dashboard-instructor.html ... unless you can get a puppy to work without having to upload a new icon...then even better.

For this issue, just get the above working. We will then want to, later, update the table using turbo stream when you click on a pet to render that pet's files. If you really want to do that here, you can as well. I think we should use the show action on the same controller.
design

Hi @kasugaijin ,
I can work on this issue.. Can you please assign it to me?
Thanks,
Nidhi

@sarvaiyanidhi all yours!


[BUG]: Sharing Schema/Types

What version of drizzle-orm are you using?
0.33.0
What version of drizzle-kit are you using?
0.23.1
Describe the Bug
Hello ! Overall wonderful library, you really helped me understand SQL, and databases.

Context  💭

I'm running a Sveltekit Application (named : app)
I have extracted the schema into a different project (named: package-schema) that i consume inside App by doing npm link Package

Configuration  🛞
App
drizzle-orm : 0.33.0
typescript :  5.6.2"
//tsconfig.json
{
	"extends": "./.svelte-kit/tsconfig.json",
	"compilerOptions": {
		"allowJs": true,
		"checkJs": true,
		"esModuleInterop": true,
		"forceConsistentCasingInFileNames": true,
		"resolveJsonModule": true,
		"skipLibCheck": true,
		"sourceMap": true,
		"strict": true,
		"moduleResolution": "bundler"
	}
}

package-schema
drizzle-orm : 0.33.0
drizzle-zod : 0.5.1
typescript :  5.6.2
{
//package.json
//...
 "devDependencies": {
    "typescript": "^5.5.4"
  },
  "dependencies": {
    "drizzle-orm": "^0.33.0",
    "drizzle-zod": "^0.5.1",
    "zod": "^3.23.8"
  }
}

//tsconfig.json
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ES2022",
    "moduleResolution": "Bundler",
    "declaration": true,
    "outDir": "./dist",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "sourceMap": true,
    "declarationMap": true
  },
  "include": ["src/**/*"],
  "exclude": ["dist", "node_modules"]
}

The ERROR (maybe BUG) 🐛
The error from typescript

// catToMaster come from `package-schema`

const allCatToMaster = await db
		.select()
                // The typescript error is on the line below
		.from(catToMaster)

Argument of type 'PgTableWithColumns<{ name: "catToMaster"; schema: undefined; columns: { catId: PgColumn<{ name: "cat_id"; tableName: "catToMaster"; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; ... 6 more ...; generated: undefined; }, {}, {}>; userId: PgColumn<...>; createdAt: PgCol...' is not assignable to parameter of type 'SQL<unknown> | PgTable<TableConfig> | Subquery<string, Record<string, unknown>> | PgViewBase<string, boolean, ColumnsSelection>'.
  Type 'PgTableWithColumns<{ name: "catToMaster"; schema: undefined; columns: { catId: PgColumn<{ name: "cat_id"; tableName: "catToMaster"; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; ... 6 more ...; generated: undefined; }, {}, {}>; userId: PgColumn<...>; createdAt: PgCol...' is not assignable to type 'PgTable<TableConfig>'.
    The types of '_.config.columns' are incompatible between these types.
      Type '{ catId: PgColumn<{ name: "cat_id"; tableName: "catToMaster"; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; hasDefault: false; isPrimaryKey: false; isAutoincrement: false; hasRuntimeDefault: false; enumValues: [...]; baseColumn: never; generated: undefined; }, {}, {}...' is not assignable to type 'Record<string, PgColumn<ColumnBaseConfig<ColumnDataType, string>, {}, {}>>'.
        Property 'catId' is incompatible with index signature.
          Type 'PgColumn<{ name: "cat_id"; tableName: "catToMaster"; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; hasDefault: false; isPrimaryKey: false; isAutoincrement: false; hasRuntimeDefault: false; enumValues: [...]; baseColumn: never; generated: undefined; }, {}, {}>' is not assignable to type 'PgColumn<ColumnBaseConfig<ColumnDataType, string>, {}, {}>'.
            The types of 'table._.config.columns' are incompatible between these types.
              Type 'Record<string, import("/Users/xxx/Documents/Projets/package-schema/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/Users/xxx/Documents/Projets/package-schema/node_modules/drizzle-orm/column").ColumnBaseConfig<import("/Users/xxx/Documents/Projets/package-schema/node_m...' is not assignable to type 'Record<string, import("/Users/xxx/Documents/Projets/app/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/Users/xxx/Documents/Projets/app/node_modules/drizzle-orm/column").ColumnBaseConfig<import("/Users/xxx/Documents/Projets/app/node_modules/drizzle-orm/co...'.
                'string' index signatures are incompatible.
                  Type 'import("/Users/xxx/Documents/Projets/package-schema/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/Users/xxx/Documents/Projets/package-schema/node_modules/drizzle-orm/column").ColumnBaseConfig<import("/Users/xxx/Documents/Projets/package-schema/node_modules/drizzle-...' is not assignable to type 'import("/Users/xxx/Documents/Projets/app/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/Users/xxx/Documents/Projets/app/node_modules/drizzle-orm/column").ColumnBaseConfig<import("/Users/xxx/Documents/Projets/app/node_modules/drizzle-orm/column-builder")....'.ts(2345)

What I have tried 🤔

I have tried various typescript configuration on app or on package-schema, didn't work.
Moved schema to app THIS IS WORKING and resolving the issue 👍  (But I don't want my schemas to live in the app project

Help needed 🙏

You can point me to the documentation if you think I miss something (even if I have extensively read trough it)
You can point me to an issue, that is related
If needed i can create a minimal repo(s) to reproduce it.

Thanks again for this wonderful projet ! You rocks
Expected behavior
I expect to be able to share schema and types between project without having issues regarding typing.
Environment & setup
vscode :  1.93.1
macos : 15.0 (24A335)

I reproduce this exact issue. I need to share my schema between 2 projects.
The only workaround I found is to duplicate the schema declaration in both projects...

Same typing error here, but for example select returns correct values.

I am having the same issue... tried creating a package for my schema to use between my two web apps

I found a temporary solution. Since the issue seems to be with the fact that TypeScript can not validate them since they are separate packages, one can use NPM workspaces, this way it will use the same "package"

I found a temporary solution. Since the issue seems to be with the fact that TypeScript can not validate them since they are separate packages, one can use NPM workspaces, this way it will use the "same" package

Can you detail your setup please ?
I tried with workspaces without success

I found a temporary solution. Since the issue seems to be with the fact that TypeScript can not validate them since they are separate packages, one can use NPM workspaces, this way it will use the "same" package

Can you detail your setup please ? I tried with workspaces without success

I mostly just followed this guide here.
My current repo structure looks like this:
my-monorepo
- package.json
- node_modules/
- packages/
- drizzle-schema/
- webapp_1
- webapp_2

Would like to see a guide on this topic in drizzle-orm documentation 👍

I use nx and pnpm for my monorepo setup. No codesharing problems so far!
I ran into this problem momentarily because I overlooked a node_modules folder that was accidentially created when running pnpm update recursively.

Hello, this issue still persists with turborepo. Any updates or PRs?

Can confirm the same issue. We have structure similar to @imMadsen, setup with turborepo. Types bug out all the time in the consumer packages, essentially forcing us to use an alternative as we cannot duplicate schemas all across our repo.
Errors look like:
Argument of type 'PgTableWithColumns<{ name: "..."; schema: "..."; columns: { id: PgColumn<{ name: "id"; tableName: "..."}, {}, {}>; ... 38 more .' is not assignable to parameter of type 'SQL<unknown> | PgTable<TableConfig> | Subquery<string, Record<string, unknown>> | PgViewBase<string, boolean, ColumnsSelection>'.

Type 'PgTableWithColumns<{ ... }, {}, {}>; ... 38 more ...' is not assignable to type 'PgTable<TableConfig>'.
    The types of '_.config.columns' are incompatible between these types.
      Type '{ id: PgColumn<{ name: "id"; tableName: ",..."; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; hasDefault: false; isPrimaryKey: true; isAutoincrement: false; hasRuntimeDefault: false; enumValues: [...]; baseColumn: never; generated: undefined; }, {}, {}>; ... 38 more .....' is not assignable to type 'Record<string, PgColumn<ColumnBaseConfig<ColumnDataType, string>, {}, {}>>'.
        Property 'id' is incompatible with index signature.
          Type 'PgColumn<{ name: "id"; tableName: "person"; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; hasDefault: false; isPrimaryKey: true; isAutoincrement: false; hasRuntimeDefault: false; enumValues: [...]; baseColumn: never; generated: undefined; }, {}, {}>' is not assignable to type 'PgColumn<ColumnBaseConfig<ColumnDataType, string>, {}, {}>'.
            The types of 'table._.config.columns' are incompatible between these types.
             Type 'Record<string, import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg@8.11.10_kysely@0.27.4_pg@8.13.1/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg<EMAIL_ADDRESS>is not assignable to type 'Record<string, import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg@8.11.10_pg@8.13.1/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg<EMAIL_ADDRESS>                'string' index signatures are incompatible.
                  Type 'import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg@8.11.10_kysely@0.27.4_pg@8.13.1/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg@8.11.10_kysely<EMAIL_ADDRESS>is not assignable to type 'import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg@8.11.10_pg@8.13.1/node_modules/drizzle-orm/pg-core/columns/common").PgColumn<import("/path/to/project/node_modules/.pnpm/drizzle-orm@0.36.3_@types+pg@8.11.10_pg@8.13.1/node_modules/drizzle-orm/column")....'.
                    Property 'config' is protected but type 'Column<T, TRuntimeConfig, TTypeConfig>' is not a class derived from 'Column<T, TRuntimeConfig, TTypeConfig>'.ts(2345)
(alias) const table: PgTableWithColumns<{
    name: "...";
    schema: "...";
    columns: {
        id: PgColumn<{
            name: "id";
            tableName: "...";
            dataType: "string";
            columnType: "PgText";
            data: string;
            driverParam: string;
            notNull: true;
            hasDefault: false;
            isPrimaryKey: true;
            ... 4 more ...;
            generated: undefined;
        }, {}, {}>;
        ... 38 more ...;
        updated_at: PgColumn<...>;
    };
    dialect: "pg";
}>

In one piece of code we use a factory to return the correct table. Here we get type errors like:
The inferred type of 'tableFactory' cannot be named without a reference to '../../../..//node_modules/drizzle-orm/pg-core'. This is likely not portable. A type annotation is necessary.ts(2742)

By the way, a way to fix this is to simply provide factories for creating a database client directly in the same package that exports the database schema. This is seemingly, because it is required to use the exact same node module for the types to not interfere with each other.

By the way, a way to fix this is to simply provide factories for creating a database client directly in the same package that exports the database schema. This is seemingly, because it is required to use the exact same drizzle node module for the types to not interfere with each other.

Don't know why I didn't think of that, that is a good solution 😅

By the way, a way to fix this is to simply provide factories for creating a database client directly in the same package that exports the database schema. This is seemingly, because it is required to use the exact same drizzle node module for the types to not interfere with each other.

Don't know why I didn't think of that, that is a good solution 😅

Yeah took me a bunch of tries to figure it out eventually haha. Quite a simple fix. All I had to do was to create those factories and rename my package from db-schema to db-client and it worked. Sometimes the best solutions are the easiest and most simple 🤷♂

By the way, a way to fix this is to simply provide factories for creating a database client directly in the same package that exports the database schema. This is seemingly, because it is required to use the exact same drizzle node module for the types to not interfere with each other.

Yep this has been the solution I use too. I have a single package called data-kit that contains both the drizzle schema def and a factory function that generates a DB client. It has drizzle-orm set as a peer dep, and any consumer package/app I have that needs the data-kit installs drizzle-orm and imports the factory function. This also allows you to use other drizzle-orm exports like and, or, etc. (query building utilities) without typescript errors either.


ARROW-13262: [R] transmute() fails after pulling data into R

This fixes a problem with arguments not being defused in our implementation of transmute(), and adds checks for unsupported transmute() arguments for consistency with dplyr's behavior on data frames.

That's sneaky! Looks good to me


add create_time to information_schema.tables (issue #1909)

Add create_time to information_schema.tables (issue #1909)

be/src/exec/schema_scanner/schema_tables_scanner.cpp
fe/src/main/java/org/apache/doris/catalog/Table.java
fe/src/main/java/org/apache/doris/common/FeConstants.java
fe/src/main/java/org/apache/doris/common/FeMetaVersion.java
fe/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
fe/src/test/java/org/apache/doris/catalog/TableTest.java
gensrc/thrift/FrontendService.thrift

Fix some warning when compile type is debug using -Werror flag(because of use deprecated funcrtions)files as follow

be/src/olap/data_dir.cpp
be/src/util/disk_info.cpp
be/src/agent/cgroups_mgr.cpp

@yangzhg
Hi, can you split this PR into two PRs? one for adding create_time and one for compile error?

@yangzhg
Hi, can you split this PR into two PRs? one for adding create_time and one for compile error?

removed compile error code


body { 	perspective: 400px; }  not being prefixed in moz at least

-moz-perspective does exist and function in latest firefox
test drive page currently will generate "-webkit-perspective", but on moz nada
thanks for a brilliant tool. very next-gen :)

Hi, I just tested on Firefox 10 (nightly) and it works fine. Which version did you experience the issue in?

3d transforms are not enabled in firefox 7

This was FF 7.01 on Mac

The property doesn’t seem to be supported there with a prefix. Try it for yourself: Run in the console 'MozPerspective' in document.body.style
I guess it’t not enabled in that version of Firefox, like panayiotisa said.

ah right you are. a little too far ahead of the curve there.
On Thu, Oct 27, 2011 at 8:14 PM, Lea Verou <
<EMAIL_ADDRESS>
The property doesnt seem to be supported there with a prefix. Try it for
yourself: Run in the console 'MozPerspective' in document.body.style
I guess itt not enabled in that version of Firefox, like panayiotisa said.
--
Reply to this email directly or view it on GitHub:
https://github.com/LeaVerou/prefixfree/issues/20#issuecomment-2552114


Output json is not parsable

Hi!
Looks like the output json uses single quotes (') and that causes issues when feeding the json to an external program (jq) or python module (json.tool).
Normal output:
$ ./btcstats.py
{'RECOMMENDATION': 'BUY', 'BUY': 13, 'SELL': 4, 'NEUTRAL': 9}

Feeding to jq:
$ ./btcstats.py | jq .
parse error: Invalid numeric literal at line 1, column 18

Feeding to json.tool:
$ ./btcstats.py | python -m json.tool
Expecting property name: line 1 column 2 (char 1)

Trying to parse on https://jsonlint.com/ returns the following:
Error: Parse error on line 1:
{	'RECOMMENDATION': 'B
--^
Expecting 'STRING', '}', got 'undefined'

Changing the quotes to double (") fixes it.
A workaround is to use sed and replace the single quotes.
$ ./btcstats.py | sed 's/\x27/"/g' | jq .
{
  "RECOMMENDATION": "BUY",
  "BUY": 12,
  "SELL": 5,
  "NEUTRAL": 9
}

ah that makes sense, it didn't actually occur to me to use the builtin json parsing!
Thanks for the prompt response.

Glad to help.


Loading all functions in a library

I was wondering, instead of doing #@ load("config.lib.yml", "base", "base_ref"), is there a way to do #@ load("config.lib.yml", "config") and use the functions as config.base and config.base_ref? I tried the above and it could not load config.lib.yml.

Hey @peterfeifanchen,
You could add the functions to a struct and then return the struct. An example here: https://get-ytt.io/#gist:https://gist.github.com/ewrenn8/df0830f14e9724b82d10fae0442aa095
Eli

Thanks!

Oh I guess also, how do you do it for *.star files if they are just normal starlark functions?

Yep, it can be done the same way. Example here: https://get-ytt.io/#gist:https://gist.github.com/ewrenn8/07f875e7fed11015c563921da75ed0a9

closing this per answer above. feel free to reopen or ping us on slack.


⚠ Turnos 24x7 has degraded performance

In 0162024, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 3613 ms

Resolved: Turnos 24x7 performance has improved in 7da8a20.


GraphQL: Media storage error log with reason

Coverage remained the same at 89.813% when pulling ec2a7227b6ffe882c73156d7e1550efaefccc286 on ms-error-log into a27901d47ca79562a64ac800409bb438cc2687fc on develop.


warning  'Unnecessary type declaration' from openapi/warnings

I have warning message  'Unnecessary type declaration' when I use:
@paramComponent {ExampleParameter} and @bodyComponent
What does it mean and how to fix it ?

I don’t think this warning is coming from openapi-comment-parser. This sounds like a typescript or eslint warning

I guess that's from openapi plugin, in my eslint config I have "plugin:openapi/recommended".
Error attached:

Oh yea looks like a bug with the eslint plugin. Does it generate the correct openapi spec?

Yes, spec is correct. (only warning shows up)


PushAPI Source

We need a push api source

looking at this, my guess is that we need a source that takes as input the HTTP request body and emits batch of docs.
I am starting to think it could probably be better to have the elastic specific parsing in a dedicated source ElasticBulkApiSource rather than having it in the endpoint handler. WDYT?

I think we need 2 rest API endpoint.

One compatible with elastic batch endpoint.
One that just takes ndjson
We then need to store the data in rocksdb, one doc per record, in a way that is agnostic of the rest ingest endpoint.

We then need a single source.

Currently, I am making push service (Mailbox) a member of source.
pub struct PushApiSource {
    push_api_mailbox: Mailbox<PushApiService>,
   ...
}

Making it not possible to get an instance of Mailbox<PushApiService>  inside
impl TypedSourceFactory for PushApiSource {
      async fn typed_create_source(params: PushApiSourceParams, checkpoint: SourceCheckpoint,) {
           ....
      }

@fulmicoton couple of solutions:

Make the push_api_mailbox globally available via a function
Not support constructing the source via config (I don't think end-user needs to see/configure this). Do not implement TypedSourceFactory at all if that is the case
Accept the Queues path and Open a Readable RockDB instance from PushApiSource, (very clunky)

I was in favor of solution (2) but you said we might need it to be able to instantiate PushApiSource via Factory.

I was in favor of solution (2) but you said we might need it to be able to instantiate PushApiSource via Factory.

Just to clarify. This is not me doing overengineering here. I think we do need a factory to spawn an indexing pipeline, so you might need right after this ticket.
You can double check that statement by doing a find usage.
It should lead you to these lines.
https://github.com/quickwit-oss/quickwit/blob/main/quickwit-indexing/src/actors/indexing_pipeline.rs#L384-L386
Solution one seems to be the only one possible.
Actix does that as part of the framework, offering a register for a specific class of service actors (that can be instantiated without  extra params). Fetching an actor can then spawn them in a lazy manner if they are not there.
We might add in the future too.
For the moment, can you add two method in the push-api crate that returns such a singleton.
pub fn init_push_api(queue_path: &Path) -> anyhow::Result<()> {
   // ...
}

pub fn get_push_api_service() -> Option<Mailbox<PushApiService>> {
  // ...
}

Or something like that.
The reason I think we cannot just have one method is that  we need to know the queue path for instanciation.
(Of course, this is just one solution and possibly not the best one.)
The painful thing here is unit testing. By introducing that kind of singleton, unit tests are not nice independent unit anymore.

No, you are not over overengineering, validating this decision was quite important for me in other aspects: In which appropriate crate to place the push_api_source in. Besides, having this conversation makes sure we are on the same line. After writing the test, I also found it was not as obvious as I thought for plugging the Source then I started exploring what you said.


Make all output stable by sorting attributes

Description of changes:
With this change, all JSON, YAML, and raw output will be sorted by attribute keys. Making the output stable provides benefits like being able to diff exports/scans/etc and confusing users less. :)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@StoneDot @itazur PTAL

Thank you for your pull request and I apologize the delay of review.
I've discuss this PR with itazur. We concludes this pull request should be reviewed by design perspective because of potential backward incompatibility. We hope you give us additional time to review this PR.

@StoneDot Sounds good. Looking forward to your decision.
In my opinion, making the output stable only means that people can expect it to be stable now, whereas they couldn't make any such assumption before.

@mlafeldt Thank you for your patience. We want to merge this PR with some modifications.
Required modifications are the following:

Add an option to specify ordering like --sortedjson for this PR.
Original behaviors of output options should be held.
Ensure the ordering of List by test. But if you feel this requires too much work, you can create the issue for future work.

I believe the above modification demands a little effort, but please let me know if you need assistance to implement it.
I want to write down the decision background.
First, the incompatibility introduced by this change can be acceptable for us because dynein is still in an early stage of development. But another consideration point exists. DynamoDB does not provide any guarantee for attribute ordering. Thus, we don't want to impress that the native DynamoDB delivers a guarantee for attribute ordering. This is the reason we decide to add an option for ordering attributes.
If you have any questions regarding the above decision, please comment.

We will close this issue.

DynamoDB does not provide any guarantee for attribute ordering. Thus, we don't want to impress that the native DynamoDB delivers a guarantee for attribute ordering.

dynein doesn't deliver a guarantee for attribute ordering:
$ dy get 1958 "Touch of Evil" --region eu-west-1 --table Movie
{
  "title": "Touch of Evil",
  "info": {
    "actors": [
      "Charlton Heston",
      "Janet Leigh",
      "Orson Welles"
    ],
    (snip)
  },
  "year": 1958
}

$ dy get 1958 "Touch of Evil" --region eu-west-1 --table Movie
{
  "title": "Touch of Evil",
  "year": 1958,
  "info": {
    "actors": [
      "Charlton Heston",
      "Janet Leigh",
      "Orson Welles"
    ],
    (snip)
  }
}

Instead, we can use jq -S to sort result:
$ dy get 1958 "Touch of Evil" --region eu-west-1 --table Movie | jq -S
{
  "info": {
    "actors": [
      "Charlton Heston",
      "Janet Leigh",
      "Orson Welles"
    ],
    (snip)
  },
  "title": "Touch of Evil",
  "year": 1958
}

Alright.


Custom font can be displayed normally in Chrome, but unnormally with other readers, such as MS Edge, Acrobat

Custom font can be displayed normally in Chrome, but unnormally with other readers, such as MS Edge, Acrobat reader.
I try to use Japanese font in my project, it can be displayed normally only with Chrome.
doc.setFont("MS Mincho");
I don't know how to fix it

Did you include a the font like described in the readme?

Did you include a the font like described in the readme?

Yes, I do. But it works with another font file, maybe I used a abnormal ttf file.
Thank U


Revert "feat: fail matchScreenshot when there is a visual diff to ena…

…ble cypress retries config (#524)"
This reverts commit 82c3cc64924ac745de669e9632ac90c0b8cac1da.
:pencil: Description

The comparadise action is failing with node 20 so reverting this change for now.

(node:4919) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/home/runners/actions-runner/_work/_actions/**/comparadise/v1/action/dist/main.js:1
import {createRequire} from "node:module";
^^^^^^
SyntaxError: Cannot use import statement outside a module
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1153:20)
    at Module._compile (node:internal/modules/cjs/loader:1205:[27]
    at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
    at node:internal/main/run_main_module:23:47
Node.js v20.8.1

:link: Related Issues

fixing forward


compile shared library occurs problems

@FloopCZ  Thank you for your project~
I have followed your installation guide trying to build shared library, but meet errors as follows:
compile_linux_protobuf.sh finished successfully!!! CUDA support disabled Warning: ignoring LD_PRELOAD in environment. .......... INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes. Configuration finished Warning: ignoring LD_PRELOAD in environment. WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing. ERROR: /home/user/tensorflow_cc_dynamic/tensorflow_cc/build/tensorflow/third_party/py/python_configure.bzl:285:20: unexpected keyword 'environ' in call to repository_rule(implementation: function, *, attrs: dict or NoneType = None, local: bool = False). ERROR: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '': Extension file 'third_party/py/python_configure.bzl' has errors. INFO: Elapsed time: 18.851s make[2]: *** [tensorflow-stamp/tensorflow_shared-configure] Error 1 make[1]: *** [CMakeFiles/tensorflow_shared.dir/all] Error 2 make: *** [all] Error 2
My server's system is centos7. One fact is that I have successfully built the static library as your guide, and run your demo code successfully. But when use the static library to create a session from a .pb file (as following bold sentence),  the status.ok() is not true. So I guess the static library doesn't support this api. So I have to build shared library but don't know how to fix the problems above mentioned.
Status status = session->Create(graph_def);
Hope for your reply~

Problem solved. refer [https://github.com/tensorflow/tensorflow/issues/8738]
update bazel to 0.4.5 and retry.

Issues solved before I even read them are the best issues.


Use esbuild Instead of Snowpack (and fix build error)

Got a new build system in place to replace Snowpack. It was failing to build so I fixed the build, and then moved to esbuild.
fixes #526
fixes #512
fixes #526
fixes #513
(many of these are duplicates)

thanks for the PR the project uses pnpm for lock file so could you remove the package-lock.json and update the lockfile

I've never used pnpm or corepack before, so let me know if anything is out of place. I think it should be correct though. Had to update husky as v4 was quite old and didn't work with pnpm. Thanks for taking a look at the PR

Just realized I need to use pnpm in some more places, holding off on this for a sec.

Okay, made a bunch of changes and started using esbuild in a better way. Important changes are called out in the commit message.
Let me know if you want anything else changed

@butlerx Anything I can do to help this PR move forward? Would be nice to use 2.7.0


Union member being mapped to parent type disallowing ability to use 'Force Field'

Describe the bug
Whilst decompiling a variable of type MMTIME which is a typedef for mmtime_tag (see screenshots below), no "Force Field" menu is available on the member u in the Decompile view.
To Reproduce
Steps to reproduce the behavior:

Obtain an appropriate 16-bit Windows application (or I've provided the xml output from "Debug Function Decompilation" menu;
Go to a variable of that type (line 44 of my example)
Right click on union variable
See no "Force Field" menu option!

Expected behavior
The ability to use this new feature.
Screenshots

Attachments
Output from "Debug Function Decompilation" menu;
dragmedi_Fep_StatusPlay.zip
Environment (please complete the following information):

OS: Windows 10 21h2
Java Version: 11.0.2
Ghidra Version: 10.2-DEVELOPMENT
Ghidra Origin: locally built

Additional context
Stepping through the ForceUnionAction.isEnabledForDecompilerContext(...) method I found that the datatype for the u member was the typedef MMTIME and not _union_74.conflict.

Interestingly, when moving the cursor across the wType member it too has the datatype of MMTIME and not UINT!


Add Pupilfirst LMS

Pupilfirst LMS is a feature-rich open-source Learning Management System (LMS)
Thank you for contributing! Please confirm this pull request meets the following requirements:

[x] I followed the contributing guidelines: https://github.com/github/explore/blob/master/CONTRIBUTING.md
[x] I have no affiliation with the project I am suggesting (as a maintainer, creator, contractor, or employee).

Which change are you proposing?

[x] Suggesting edits to an existing topic or collection
[ ] Curating a new topic or collection
[ ] Something that does not neatly fit into the binary options above

Editing an existing topic or collection
I'm suggesting these edits to an existing topic or collection:

[ ] Image (and my file is *.png, square, dimensions 288x288)
[x] Content (and my changes are in index.md)

Pupilfirst is a learning management system (LMS) built on Ruby on Rails and Rescript where learning is achieved through focused tasks, directed feedback, an iterative workflow, and community interaction.

Thanks for your contribution @gigincg.
@harigopal - Can you please confirm that Pupilfirst LMS can be listed under the Made in India collection on GitHub Explore?

@mvkaran Yes, that would be nice! 🙂

@mvkaran Yes, that would be nice! 🙂

Thanks for the confirmation!


Multi-uav simulation with qgc

I have successfully built 3 uavs in gazebo, but when I start qgroundcontrol, it can only connect with iris1 and iris2. I don't know whether there are some parameters I missed to change. Do you have any suggesstions?

➤ matthewsharpe commented:
Did you ever find any solution to this? I'm am having the exact same problem.

➤ Hamish Willee commented:
What setup instructions did you follow? This works: http://dev.px4.io/en/simulation/multi-vehicle-simulation.html This also works, but has not yet been included in the developer guide: https://github.com/acsl-mipt/multiple-sitl/tree/for-px4-pr


Container app secret key length limited to 20, but not reported in Bicep

Bicep version
Bicep CLI version 0.13.1 (e3ac80d678)
Describe the bug
When deploying a container app, the secret key is limited to 20, which is reported by az containerapp secret set:

Secret names cannot be longer than 20

This should also be reported when using Bicep to deploy the container app resource.

Related to #1679, but it will also require a swagger update on the container apps side to include naming restrictions:
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/app/resource-manager/Microsoft.App/preview/2022-01-01-preview/ContainerApps.json#L163


skrollr parallax effect work very stuck on android mobile

my page with skrollr get stuck on android mobile in wechat browser, I'm very confused, is there any way to solve the problem?

Getting the following warning message when on mobile:

skrollr.min.js:2 [Intervention] Unable to preventDefault inside passive event listener due to target being treated as passive. See https://www.chromestatus.com/features/5093566007214080

Any ideas how to fix this problem?


queue info for onchain tx

messaging not optimal (ie: shortly instead of now)
need auto refresh in transaction detail page

Could we order the fields (50% opacity for inactive fields) in order of importance?

"transaction queued"
destination
fees
account (no amount duplicity
date
description
type
hash
Cancel option (when possible)


Asynchronous behaviour of SNN model

Hi,
I have a quick question regarding the Asynchronous implementation of Spiking Neural Networks in Python. As Python is a sequential language then how SNN-toolbox can deal with the asynchronous behavior of SNNs, as each layer should have to work independently from the rest of the layers, and how does SNN-toolbox incorporate that async. functionality in Python.

SNN toolbox provides an interface to multiple backends, some of which are actual neuromorphic hardware (Loihi, Spinnaker). When using these, Python is only involved in preparing / deploying the model, and the actual processing is (more or less) asynchronous, depending on the hardware. Aside from these, the toolbox also provides a GPU / CPU backend, which does not emulate or implement asynchronous spike behavior. Instead it simulates sparse spike processing by running a full forward pass at each time step. in this forward pass, the membrane potentials are updated, and where applicable, spikes are generated and passed on to the next layer, where they get added to the membrane potential and may cause a spike, and so on. Then the next time step is done with a full forward pass. Not very efficient, but one can count the number of nonzero activations (spikes) and thus estimate the computational effort (synaptic operations) that would be necessary on dedicated asynchronous hardware.


upgrade node-mbtiles

fixes #39

upgrade mbtiles to 0.10.0
upgrade dev dependencies
lint
commit package-lock

mbtiles upgrade brings a version of sqlite3 that is incompat. w/ node4.

npm WARN engine<EMAIL_ADDRESS>wanted: {"node":">=4.5"}

Looks like we have some code that is incompat. with node4. Offical support for node4 ended 2018-04-30(https://github.com/nodejs/Release). Removing argon.

incompat. code was eslint's: (https://github.com/eslint/eslint/blob/v5.13.0/package.json#L124)


getcwd_syscall() updates

Description
Fixes # (issue)
The following changes include a small fix for checking null pointers, more elaborate comments, and new unit tests for getcwd_syscall().
Type of change

[ ] More detailed comments for getcwd_syscall()
[ ] Added a condition to error-out null pointers passed as buf argument
[ ] New unit tests for getcwd_syscall()

How Has This Been Tested?
To run the tests, we need to run cargo test --lib command inside the safeposix-rust directory.
All the tests are present under this directory: lind_project/src/safeposix-rust/src/tests/fs_tests.rs

Test A - lut_lind_fs_getcwd_valid_args()
Test B - ut_lind_fs_getcwd_invalid_args()

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been added to a pull request and/or merged in other modules (native-client, lind-glibc, lind-project)

@Anway-Agte can you review this before Rupesh does?


chore: drop Algebra.TensorProduct.rightAlgebra'

Generalized upstream in https://github.com/leanprover-community/mathlib4/pull/18089.

Thank you for upstreaming this!


GitHub actions

This is a first cut, I haven't done the NuGet related stuff as I don't have permissions on this repo to setup secrets.
This PR adds:

A GitHub Action to build, run tests of each platform, integration tests on linux only, and collects artifacts
Enables deterministic builds under GitHub Actions, I had to swap out some acceptance test code as it expects a full path in the symbols, this may already be fixed in Shouldy master (Acceptance tests implementation have been removed in favour of a new library), but we are waiting on a pre-release build to try it.
Disables MinVer when running locally in debug, just for speed

Nice one 👍 . I usually use https://github.com/droyad/Assent but since Shouldly was already in here I just used that. Should we wait for Shouldly to support DiffEngine and use that, or just use Assent now? I'm fine either way.

Nice, I didn't know about Assent. Unfortunately they all suffer from the same issue, which is trying to resolve the file path doesn't work. Perhaps I should disable it for test runs?

Ah ok, is that a side effect of using deterministic buiilds? I'm guessing it erases or standardises paths so that the build is the same wherever you run it? I suppose in that case we'd need a lib that can read embedded files as approval sources.

Deterministic builds causes issues with Code Coverage at the moment, so we should defer that until coverlet fixes it.

I'll rollback deterministic builds, it's causing too much pain right now.
On the build script, yeah good point. I like the way GHA presents the individual steps, but not at the cost of having duplicated definitions, I'll use the pwsh script 👍

Looks like the GHA workflow is missing the step to upload to codecov.

That didn't go as well as I'd hoped: https://github.com/justeat/JustSaying/actions/runs/230252521


DBContext in Application not working

To accommodate old tables/code when migrating to the FSH API I need to access a dbContext in the ApplicationLayer until such time as the code can be refactored.  However even for a new simple DBContext (which I had done in the previous project) I get the following error when Injecting into Mediatr.
"Error constructing handler for request of type Med…ontainer. See the samples in GitHub for examples"
public class ContactGetQueryHandler : IRequestHandler<ContactGetQuery, ContactDetailDto>
{
    private readonly IMyDbContext _context;

    public ContactGetQueryHandler(IMyDbContext context) => _context = context;

    public async Task<ContactDetailDto> Handle(ContactGetQuery request, CancellationToken cancellationToken)
    {
        return new ContactDetailDto();
    }
}

public ContactGetQueryHandler(IMyDbContext context) = new ContactGetQueryHandler(DbContext option):  base(option)

Unfortunately that did not help :(

Hi @iang0ld, can you share a gist of your code so someone might be able to help you?
How are you wiring up your IMyDbContext interface during your DI registration?
Here's a screenshot of my code where I Inject an IApplicationDbContext into my Handler in my Application Layer in a project that's similar to this boilerplate.

This is how I wire up my Context.  It works fine in the Integration layer.
Yeh. I have done this before in other projects without a problem.
namespace FSH.WebApi.Infrastructure.Persistence;

public class TitanDbContext : DbContext, ITitanDbContext
{
    public TitanDbContext(DbContextOptions<TitanDbContext> options)
        : base(options)
    {
    }
    public DbSet<Tag> Tags { get; set; }
}

 return services
            .Configure<DatabaseSettings>(config.GetSection(nameof(DatabaseSettings)))
            .AddDbContext<ApplicationDbContext>(m => m.UseDatabase(dbProvider, rootConnectionString))

            .AddDbContext<TitanDbContext>(m => m.UseDatabase(dbProvider, rootConnectionString))

@iang0ld , Glad to see you were able to get this resolved :-).  What ended up being the resolution to this issue with respect to FSH boilerplate?  Can you share with the community?

The ticket was closed in frustration rather than success.


how can i change the timezone of devicehive-java-server deployed by docker?

how can i change the timezone of devicehive-java-server deployed by docker?

i found your com.devicehive.service.time.LocalTimestampService in devicehive-common-service has a problem in method :
public String getDateAsString() {
DATE_FORMAT.setTimeZone(new SimpleTimeZone(SimpleTimeZone.UTC_TIME, "UTC")); // (1)
return DATE_FORMAT.format(getDate());
}
so, this code (1) changes and hard coded the timezone of global const DATA_FORMAT !!!


XGBoost NCCL Failure when using TPOT with RAPIDS

Describe the bug
When using TPOT cuML GPU crashes after a few hours, it ran for ~5.5 hours before crashing as I attempted to reproduce this example provided by a DGX A100 customer. The specific error I got was a NCCL error from XGBoost. I've included the full text of my error message as well as a screenshot from the one the customer had. I monitored the whole run until it crashed using the JupyterLab NVDashboards to try to gather some insights to help. I did not observe GPU memory or host memory filling up over time and neither were full when the crash occurred. This example uses a single GPU (unsure why NCCL is being used behind the scenes).
Steps/Code to reproduce bug
from tpot import TPOTRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

NSAMPLES = 5000
NFEATURES = 50
SEED = 12

# For cuML with TPOT, you must use CPU data (such as NumPy arrays)
X, y = make_regression(
    n_samples=NSAMPLES,
    n_features=NFEATURES,
    n_informative=NFEATURES,
    random_state=SEED,
    noise=200,
)

X = X.astype("float32")

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=12)

# TPOT setup
GENERATIONS = 100
POP_SIZE = 100
CV = 10

tpot = TPOTRegressor(
    generations=GENERATIONS,
    population_size=POP_SIZE,
    random_state=SEED,
    config_dict="TPOT cuML",
    n_jobs=1, # cuML requires n_jobs=1
    cv=CV,
    verbosity=2,
)

tpot.fit(X_train, y_train)

preds = tpot.predict(X_test)
print(r2_score(y_test, preds))

#tpot.export('tpot_regression_cuml_pipeline.py')
#print(tpot.export())
Expected behavior
TPOT should run to completion and then allow the exporting of the best model (commented code at the very end of the example).

Environment details
nvcr.io/nvidia/rapidsai/rapidsai:21.08-cuda11.0-runtime-ubuntu18.04 Docker Image from NGC.
TPOT installed with "conda install -y tpot" inside of rapids conda environment. TPOT version: tpot-0.11.7
Reproduced on DGX A100 40GB by customer and on a DGX Station (32GB V100) - Driver Version: 460.32.03
Error message text
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/base.py in fit(self, features, target, sample_weight, groups)
    827                     per_generation_function=self._check_periodic_pipeline,
--> 828                     log_file=self.log_file_,
    829                 )

/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/gp_deap.py in eaMuPlusLambda(population, toolbox, mu, lambda_, cxpb, mutpb, ngen, pbar, stats, halloffame, verbose, per_generation_function, log_file)
    247
--> 248         offspring = toolbox.evaluate(offspring)
    249

/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/base.py in _evaluate_individuals(self, population, features, target, sample_weight, groups)
   1531                     val = partial_wrapped_cross_val_score(
-> 1532                         sklearn_pipeline=sklearn_pipeline
   1533                     )

/opt/conda/envs/rapids/lib/python3.7/site-packages/stopit/utils.py in wrapper(*args, **kwargs)
    141             if timeout:
--> 142                 with self.to_ctx_mgr(timeout, swallow_exc=True):
    143                     result = self.default  # noqa

/opt/conda/envs/rapids/lib/python3.7/site-packages/stopit/utils.py in __enter__(self)
     72         self.state = BaseTimeout.EXECUTING
---> 73         self.setup_interrupt()
     74         return self

/opt/conda/envs/rapids/lib/python3.7/site-packages/stopit/threadstop.py in setup_interrupt(self)
     59         self.timer = threading.Timer(self.seconds, self.stop)
---> 60         self.timer.start()
     61

/opt/conda/envs/rapids/lib/python3.7/threading.py in start(self)
    851         try:
--> 852             _start_new_thread(self._bootstrap, ())
    853         except Exception:

RuntimeError: can't start new thread

During handling of the above exception, another exception occurred:

XGBoostError                              Traceback (most recent call last)
<ipython-input-4-c4170700a203> in <module>
     14 )
     15
---> 16 tpot.fit(X_train, y_train)
     17
     18 preds = tpot.predict(X_test)

/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/base.py in fit(self, features, target, sample_weight, groups)
    861                     # raise the exception if it's our last attempt
    862                     if attempt == (attempts - 1):
--> 863                         raise e
    864             return self
    865

/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/base.py in fit(self, features, target, sample_weight, groups)
    853
    854                     self._update_top_pipeline()
--> 855                     self._summary_of_best_pipeline(features, target)
    856                     # Delete the temporary cache before exiting
    857                     self._cleanup_memory()

/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/base.py in _summary_of_best_pipeline(self, features, target)
    993             with warnings.catch_warnings():
    994                 warnings.simplefilter("ignore")
--> 995                 self.fitted_pipeline_.fit(features, target)
    996
    997             if self.verbosity in [1, 2]:

/opt/conda/envs/rapids/lib/python3.7/site-packages/sklearn/pipeline.py in fit(self, X, y, **fit_params)
    328         """
    329         fit_params_steps = self._check_fit_params(**fit_params)
--> 330         Xt = self._fit(X, y, **fit_params_steps)
    331         with _print_elapsed_time('Pipeline',
    332                                  self._log_message(len(self.steps) - 1)):

/opt/conda/envs/rapids/lib/python3.7/site-packages/sklearn/pipeline.py in _fit(self, X, y, **fit_params_steps)
    294                 message_clsname='Pipeline',
    295                 message=self._log_message(step_idx),
--> 296                 **fit_params_steps[name])
    297             # Replace the transformer of the step with the fitted
    298             # transformer. This is necessary when loading the transformer

/opt/conda/envs/rapids/lib/python3.7/site-packages/joblib/memory.py in __call__(self, *args, **kwargs)
    350
    351     def __call__(self, *args, **kwargs):
--> 352         return self.func(*args, **kwargs)
    353
    354     def call_and_shelve(self, *args, **kwargs):

/opt/conda/envs/rapids/lib/python3.7/site-packages/sklearn/pipeline.py in _fit_transform_one(transformer, X, y, weight, message_clsname, message, **fit_params)
    738     with _print_elapsed_time(message_clsname, message):
    739         if hasattr(transformer, 'fit_transform'):
--> 740             res = transformer.fit_transform(X, y, **fit_params)
    741         else:
    742             res = transformer.fit(X, y, **fit_params).transform(X)

/opt/conda/envs/rapids/lib/python3.7/site-packages/sklearn/base.py in fit_transform(self, X, y, **fit_params)
    691         else:
    692             # fit method of arity 2 (supervised transformation)
--> 693             return self.fit(X, y, **fit_params).transform(X)
    694
    695

/opt/conda/envs/rapids/lib/python3.7/site-packages/tpot/builtins/stacking_estimator.py in fit(self, X, y, **fit_params)
     65             Returns a copy of the estimator
     66         """
---> 67         self.estimator.fit(X, y, **fit_params)
     68         return self
     69

/opt/conda/envs/rapids/lib/python3.7/site-packages/xgboost/core.py in inner_f(*args, **kwargs)
    434         for k, arg in zip(sig.parameters, args):
    435             kwargs[k] = arg
--> 436         return f(**kwargs)
    437
    438     return inner_f

/opt/conda/envs/rapids/lib/python3.7/site-packages/xgboost/sklearn.py in fit(self, X, y, sample_weight, base_margin, eval_set, eval_metric, early_stopping_rounds, verbose, xgb_model, sample_weight_eval_set, base_margin_eval_set, feature_weights, callbacks)
    745             verbose_eval=verbose,
    746             xgb_model=model,
--> 747             callbacks=callbacks,
    748         )
    749

/opt/conda/envs/rapids/lib/python3.7/site-packages/xgboost/training.py in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, xgb_model, callbacks)
    195                           evals_result=evals_result,
    196                           maximize=maximize,
--> 197                           early_stopping_rounds=early_stopping_rounds)
    198     return bst
    199

/opt/conda/envs/rapids/lib/python3.7/site-packages/xgboost/training.py in _train_internal(params, dtrain, num_boost_round, evals, obj, feval, xgb_model, callbacks, evals_result, maximize, verbose_eval, early_stopping_rounds)
     79         if callbacks.before_iteration(bst, i, dtrain, evals):
     80             break
---> 81         bst.update(dtrain, i, obj)
     82         if callbacks.after_iteration(bst, i, dtrain, evals):
     83             break

/opt/conda/envs/rapids/lib/python3.7/site-packages/xgboost/core.py in update(self, dtrain, iteration, fobj)
   1499             _check_call(_LIB.XGBoosterUpdateOneIter(self.handle,
   1500                                                     ctypes.c_int(iteration),
-> 1501                                                     dtrain.handle))
   1502         else:
   1503             pred = self.predict(dtrain, output_margin=True, training=True)

/opt/conda/envs/rapids/lib/python3.7/site-packages/xgboost/core.py in _check_call(ret)
    208     """
    209     if ret != 0:
--> 210         raise XGBoostError(py_str(_LIB.XGBGetLastError()))
    211
    212

XGBoostError: [20:33:29] /opt/conda/envs/rapids/conda-bld/xgboost_1628090879278/work/src/tree/updater_gpu_hist.cu:793: Exception in gpu_hist: NCCL failure :unhandled cuda error /opt/conda/envs/rapids/conda-bld/xgboost_1628090879278/work/src/common/device_helpers.cu(71)

Stack trace:
  [bt] (0) /opt/conda/envs/rapids/lib/libxgboost.so(+0x5f66d4) [0x7f5de54046d4]
  [bt] (1) /opt/conda/envs/rapids/lib/libxgboost.so(xgboost::tree::GPUHistMakerSpecialised<xgboost::detail::GradientPairInternal<double> >::Update(xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal<float> >*, xgboost::DMatrix*, std::vector<xgboost::RegTree*, std::allocator<xgboost::RegTree*> > const&)+0xa50) [0x7f5de5429e90]
  [bt] (2) /opt/conda/envs/rapids/lib/libxgboost.so(xgboost::gbm::GBTree::BoostNewTrees(xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal<float> >*, xgboost::DMatrix*, int, std::vector<std::unique_ptr<xgboost::RegTree, std::default_delete<xgboost::RegTree> >, std::allocator<std::unique_ptr<xgboost::RegTree, std::default_delete<xgboost::RegTree> > > >*)+0x1ac) [0x7f5de507edac]
  [bt] (3) /opt/conda/envs/rapids/lib/libxgboost.so(xgboost::gbm::GBTree::DoBoost(xgboost::DMatrix*, xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal<float> >*, xgboost::PredictionCacheEntry*)+0x4c6) [0x7f5de5081766]
  [bt] (4) /opt/conda/envs/rapids/lib/libxgboost.so(+0x2871c0) [0x7f5de50951c0]
  [bt] (5) /opt/conda/envs/rapids/lib/libxgboost.so(XGBoosterUpdateOneIter+0x4e) [0x7f5de4f8ddee]
  [bt] (6) /opt/conda/envs/rapids/lib/python3.7/lib-dynload/../../libffi.so.7(+0x69ed) [0x7f60f2cff9ed]
  [bt] (7) /opt/conda/envs/rapids/lib/python3.7/lib-dynload/../../libffi.so.7(+0x6077) [0x7f60f2cff077]
  [bt] (8) /opt/conda/envs/rapids/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x2f4) [0x7f60f1e0f794]

This example uses a single GPU (unsure why NCCL is being used behind the scenes).

Looks like I should try to figure out this one first.

Closing as this happens in cuml.  Feel free to ping me if there's anything I can help.


Don’t attempt to use Bazel for tests.

We can’t use Bazel for this project because due to
https://github.com/bazelbuild/bazel/blob/20b056ae8506e489933cf236b9401cddad90afac/src/main/java/com/google/devtools/build/lib/buildtool/SymlinkForest.java#L257-L258
Bazel doesn’t support source files starting with “bazel-”.  Switch to good old
Make instead.

We should be able to find a workaround.
Can we move the bazel-* files in a subdirectory?

Filed a bug against Bazel: https://github.com/bazelbuild/bazel/issues/10560
The bug is a bit embarrassing.


LiveReload Protocol Error (Svelte, Rollup, Tailwind, Flask Server )

Hello All,
Note: I am coming from the python world
I am attempting to develop a simple site that serves from a py flask server with a svelte client with TailwindCSS.
Suddenly, I LiveReload is failing. The page is constantly refreshing after every change (minor or major). Below are my config files. Thanks in advance for your help.
Error via chrome console
`
livereload.js?snipver=1:2593 LiveReload protocol error (invalid command 'reload', only valid commands are: hello)) after receiving data: "{"command":"reload","path":"/home/projects.d/lib/web-app/client/public/bundle.js","liveCSS":true,"originalPath":"","overrideURL":""}"..`

rollup.config.js
`import svelte from 'rollup-plugin-svelte';
import commonjs from '@rollup/plugin-commonjs';
import resolve from '@rollup/plugin-node-resolve';
import livereload from 'rollup-plugin-livereload';
import { terser } from 'rollup-plugin-terser';
import css from 'rollup-plugin-css-only';
import sveltePreprocess from 'svelte-preprocess';

const production = !process.env.ROLLUP_WATCH;

function serve() {
        let server;

        function toExit() {
                if (server) server.kill(0);
        }

        return {
                writeBundle() {
                        if (server) return;
                        server = require('child_process').spawn('npm', ['run', 'start', '--', '--dev'], {
                                stdio: ['ignore', 'inherit', 'inherit'],
                                shell: true
                        });

                        process.on('SIGTERM', toExit);
                        process.on('exit', toExit);
                }
        };
}

export default {
        input: 'src/main.js',
        output: {
                sourcemap: true,
                format: 'iife',
                name: 'app',
                file: 'public/bundle.js'
        },
        plugins: [
                svelte({
                        preprocess: sveltePreprocess({ postcss: true }),
                        compilerOptions: {
                                // enable run-time checks when not in production
                                dev: !production,
                                css: css => {
                                        css.write('bundle.css');
                                }
                        }
                }),
    // we'll extract any component CSS out into
    // a separate file - better for performance
    css({ output: 'bundle.css' }),

                // If you have external dependencies installed from
                // npm, you'll most likely need these plugins. In
                // some cases you'll need additional configuration -
                // consult the documentation for details:
                // https://github.com/rollup/plugins/tree/master/packages/commonjs
                resolve({
                        browser: true,
                        dedupe: ['svelte']
                }),
                commonjs(),

                // In dev mode, call `npm run start` once
                // the bundle has been generated
                !production && serve(),

                // Watch the `public` directory and refresh the
                // browser on changes when not in production
                !production && livereload('public'),

                // If we're building for production (npm run build
                // instead of npm run dev), minify
                production && terser()
        ],
        watch: {
                clearScreen: false
        }
};
`

tailwind.config.js
`
const colors = require('tailwindcss/colors')

module.exports = {
    content: [
        './public/**/*.html',
        './src/**/*.svelte'
    ],
    theme: {
        extend: {
            fontFamily: {
               ...
            },
        },
    },
    variants: {},
    plugins: [],
}`

svelte.config.js
# svelte.config.js
import preprocess from "svelte-preprocess";

const config = {
  preprocess: [
    preprocess({
      postcss: true,
    }),
  ],
}
`

package.json
`{
   "name": "client",
   "version": "0.0.0",
   "dependencies": {
      "@rollup/plugin-commonjs": "^22.0.2",
      "@rollup/plugin-node-resolve": "^13.3.0",
      "postcss": "^8.4.16",
      "postcss-load-config": "^4.0.1",
      "rollup": "^2.78.0",
      "rollup-plugin-css-only": "^3.1.0",
      "rollup-plugin-livereload": "^2.0.5",
      "rollup-plugin-serve": "^2.0.1",
      "rollup-plugin-svelte": "^7.1.0",
      "rollup-plugin-terser": "^7.0.2",
      "svelte": "^3.49.0",
      "svelte-preprocess": "^4.10.7",
      "tailwindcss": "^3.1.8"
   },
   "scripts": {
      "build": "rollup -c",
      "serve": "rollup -c -w",
      "start": "#no-op",
      "start:dev": "sirv public --single --dev"
   }
}`

Hi! It sounds though like something is calling livereload in the wrong order.
You'll need to debug your setup yourself though. I think it is a good idea to set up a separate repository which is able to reproduce this issue. Once you are able to pinpoint which package is causing the issue, you can refer to your example while reporting it in the issue tracker of the package in question.


Implement serde serialization/deserialization

This makes it much easier to pass the messages as part of protocols, and provides a standard interface for serializing/deserializing all the messages (e.g. to store in DB).
Fixes https://github.com/novifinancial/opaque-ke/issues/166

Closing due to conflicting CLAs


[MXNET-259] Performance improvement of random.shuffle

Description
For multidimensional arrays on CPU, mx.random.shuffle implements Fisher-Yates algorithm which needs n number of swaps of two memory ranges where n is the length of the first axis. Previously the swap is performed by std::swap_ranges. This PR replaces it with a manual swap using std::memcpy. This brings a good performance gain. For example, shuffling of arrays with shape (1000, 10000) is almost ten times faster in linux/gcc. (1D arrays on CPU and arrays on GPU are not in the scope of this PR.)
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@reminisce @marcoabreu


pytube error 'url_encoded_fmt_stream_map', 'pytube.exceptions.VideoUnavailable: This video is unavailable.'

What is the purpose of your issue?

[x] Bug
[ ] Feature Request
[ ] Question
[ ] Other

Description
spotdl v2.0.8
Log
❯ spotdl -s "https://music.youtube.com/watch?v=DLY0w36iR2U" -ll DEBUG
DEBUG:spotdl.command_line.core:45:
Received arguments:
{'song': ['https://music.youtube.com/watch?v=DLY0w36iR2U'], 'list': None, 'playlist': None, 'album': None, 'all_albums': None, 'username': None, 'write_m3u': False, 'manual': False, 'no_metadata': False, 'no_encode': False, 'overwrite': 'prompt', 'quality': 'best', 'input_ext': 'automatic', 'output_ext': 'mp3', 'write_to': None, 'output_file': '{artist} - {track-name}.{output-ext}', 'trim_silence': False, 'search_format': '{artist} - {track-name} lyrics', 'dry_run': False, 'processor': 'synchronous', 'no_spaces': False, 'skip_file': None, 'write_successful_file': None, 'spotify_client_id': '4fe3fecfe5334023a1472516cc99d805', 'spotify_client_secret': '0f02b7c483c04257984695007a4a8d5c', 'log_level': 'DEBUG', 'config': '/home/aakash/.config/spotdl/config.yml', 'remove_config': False}

DEBUG:spotdl.authorize.services.spotify:37:
Setting master Spotify credentials.

DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.

INFO:spotdl.command_line.core:181:
Downloading "https://music.youtube.com/watch?v=DLY0w36iR2U"

DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.

DEBUG:spotdl.metadata_search:194:
Extracting YouTube and Spotify metadata for input YouTube URL.

DEBUG:spotdl.metadata_search:230:
Extracting YouTube metadata for input YouTube URL.

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 4

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 3

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 2

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 1

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 0

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/spotdl", line 8, in <module>
    sys.exit(main())
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/command_line/__main__.py", line 48, in main
    spotdl.match_arguments()
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/command_line/core.py", line 65, in match_arguments
    self.download_track(track)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/command_line/core.py", line 192, in download_track
    metadata = search_metadata.on_youtube_and_spotify()
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata_search.py", line 89, in on_youtube_and_spotify
    metadata = caller()
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata_search.py", line 195, in _on_youtube_and_spotify_for_type_youtube
    youtube_metadata = self._on_youtube_for_type_youtube(self.track)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata_search.py", line 231, in _on_youtube_for_type_youtube
    youtube_metadata = self.providers["youtube"].from_url(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 242, in from_url
    return self.from_url(url, retries=retries)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 242, in from_url
    return self.from_url(url, retries=retries)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 242, in from_url
    return self.from_url(url, retries=retries)
  [Previous line repeated 2 more times]
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'
❯ spotdl -s "https://music.youtube.com/watch?v=DLY0w36iR2U" -ll DEBUG
DEBUG:spotdl.command_line.core:45:
Received arguments:
{'song': ['https://music.youtube.com/watch?v=DLY0w36iR2U'], 'list': None, 'playlist': None, 'album': None, 'all_albums': None, 'username': None, 'write_m3u': False, 'manual': False, 'no_metadata': False, 'no_encode': False, 'overwrite': 'prompt', 'quality': 'best', 'input_ext': 'automatic', 'output_ext': 'mp3', 'write_to': None, 'output_file': '{artist} - {track-name}.{output-ext}', 'trim_silence': False, 'search_format': '{artist} - {track-name} lyrics', 'dry_run': False, 'processor': 'synchronous', 'no_spaces': False, 'skip_file': None, 'write_successful_file': None, 'spotify_client_id': '4fe3fecfe5334023a1472516cc99d805', 'spotify_client_secret': '0f02b7c483c04257984695007a4a8d5c', 'log_level': 'DEBUG', 'config': '/home/aakash/.config/spotdl/config.yml', 'remove_config': False}

DEBUG:spotdl.authorize.services.spotify:37:
Setting master Spotify credentials.

DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.

INFO:spotdl.command_line.core:181:
Downloading "https://music.youtube.com/watch?v=DLY0w36iR2U"

DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.

DEBUG:spotdl.metadata_search:194:
Extracting YouTube and Spotify metadata for input YouTube URL.

DEBUG:spotdl.metadata_search:230:
Extracting YouTube metadata for input YouTube URL.

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 4

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 3

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 2

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://music.youtube.com/watch?v=DLY0w36iR2U)`. Retries left: 1

DEBUG:spotdl.metadata.providers.youtube:228:
Fetching YouTube metadata for "https://music.youtube.com/watch?v=DLY0w36iR2U".

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 97, in prefetch_init
    self.init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 130, in init
    mixins.apply_descrambler(self.player_config_args, fmt)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/mixins.py", line 89, in apply_descrambler
    for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/spotdl", line 8, in <module>
    sys.exit(main())
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/command_line/__main__.py", line 48, in main
    spotdl.match_arguments()
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/command_line/core.py", line 65, in match_arguments
    self.download_track(track)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/command_line/core.py", line 192, in download_track
    metadata = search_metadata.on_youtube_and_spotify()
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata_search.py", line 89, in on_youtube_and_spotify
    metadata = caller()
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata_search.py", line 195, in _on_youtube_and_spotify_for_type_youtube
    youtube_metadata = self._on_youtube_for_type_youtube(self.track)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata_search.py", line 231, in _on_youtube_for_type_youtube
    youtube_metadata = self.providers["youtube"].from_url(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 242, in from_url
    return self.from_url(url, retries=retries)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 242, in from_url
    return self.from_url(url, retries=retries)
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 242, in from_url
    return self.from_url(url, retries=retries)
  [Previous line repeated 1 more time]
  File "/home/aakash/.local/lib/python3.8/site-packages/spotdl/metadata/providers/youtube.py", line 230, in from_url
    content = pytube.YouTube(url)
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 88, in __init__
    self.prefetch_init()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 96, in prefetch_init
    self.prefetch()
  File "/home/aakash/.local/lib/python3.8/site-packages/pytube/__main__.py", line 162, in prefetch
    raise VideoUnavailable('This video is unavailable.')
pytube.exceptions.VideoUnavailable: This video is unavailable.

The traceback again seems points at PyTube. Does this also happen with other tracks?


[FEATURE] Scan n+2 buckets

With the lack of a better naming convention - we now have currentBuckets, nextBuckets and the new afterNextBuckets. Better naming proposals are welcome.

I'm trying to refactor this and encapsulate Bucket logic.

After the refactoring, the bucket subscription logic works as follows:

create and subscribe to all buckets when all stopped or first run
buckets are represented as array of 3 elements [current, next, afterNext]
when new current has changed then

stop and remove old current
start and store afterNext
so we transition from [current, next, afterNext] -> [next, afterNext, newAfterNext] so in general we shift the array and push on top

Pull Request Test Coverage Report for Build 2120

5 of 5 (100.0%)  changed or added relevant lines in 2 files are covered.
20 unchanged lines in 1 file lost coverage.
Overall coverage increased (+3.1%) to 76.407%

Files with Coverage Reduction
New Missed Lines
%

src/Scanner/ChainScanner.ts
20
61.25%

Totals

Change from base Build 2112:
3.1%

Covered Lines:
903

Relevant Lines:
1131

💛  - Coveralls


Shouldn't the 1.46.1 “May 2019 Recovery” release be named “May 2020 Recovery”?

VSCode Version: N/A
OS Version: N/A

Steps to Reproduce:

Check out https://github.com/microsoft/vscode/releases

Does this issue occur when all extensions are disabled?: Yes

Pinging @connor4312


Fe assignment

Documentation with screenshots:
https://valuable-warrior-a8a.notion.site/Assignment-1-80b629f88bec41a89fbdb6fb49ed83a0?pvs=4
Video Link:

Metric
Value

alert_status
:white_check_mark: OK

bugs
8

code_smells
17

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
18

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
8

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
6

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
6

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
6

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
6

code_smells
20

reliability_rating
E (at least 1 Blocker Bug)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Breaking ui

cannot post in mobile


Fix console errors in features affecting the PR merge panel

Fixes #5547
Test URLs

Click on the merge button at the bottom of the page
Click on "Cancel" and reload the page
No console errors should be produced

Screenshots
See the review comments below.

Two related comments that can't be reviews because the files haven't been touched :point_down:

In clear-pr-merge-commit-message, we only need to react to the user opening the panel: when the PR conversation page is first opened, the panel will always be closed, so the user has to open it at least once before merging.
Then on subsequent reloads, any changes the feature made to the commit description will automatically be restored by GitHub.

The session:resume event is also used in prevent-pr-merge-panel-opening. Since this feature doesn't produce any errors because of it, and because it can be triggered several times with little cost, I haven't touched upon it in this PR.
But it's somewhat unreliable, so it will have to be revised too.

The demo videos look good. It would be good to outline the specific situations that our listeners cover, including past fixes we applied, so that we can use it as a test checklist. For example:

Clean PR load, open merge box
Clean PR load, open merge box, edit commit title
Clean PR load, open merge box, edit commit title, reload
Clean PR load, open merge box, new commit comes in
...etc

Or simply list all the "events" that affect the box, like:

opening and closing it
loading the page after editing
etc

This PR can be merged after 22.4.1 as is, though

The demo videos look good. It would be good to outline the specific situations that our listeners cover, including past fixes we applied, so that we can use it as a test checklist.

+1
Actually, I've been thinking that we could really use a permanent list of testing URLs for often-modified features or features that have a lot of edge cases (e.g. clean-repo-sidebebar, clean-conversation-headers, etc.), so we don't have to go fishing through old PRs for them.
Just a Markdown file somewhere in the repo would be good enough.

We can include these details in the feature itself. Adding large comments isn't an issue, we can append them to the file, so they follow the init


Add Documentation

Docs Updated:

Find-PSResource
Get-PSResourceRepository
Install-PSResource (some areas need to be finished)

@SydneyhSmith I'm going to close this PR since we're incorporating help docs as we refactor cmdlets, but we can refer back to these docs when pushing the new changes


add test box to drop user images into

I thought it would be nice if I could drop my own images onto the demo page to test out js-imagediff.

Preview here: http://benkaiser.github.io/js-imagediff/

Nice.

Thank @cesutherland for merging my changes and for bumping the npm version.