Background sync queue callbacks

R: @jeffposnick @prateekbh
Fixes #646
This PR updates the background sync queue callbacks to address the issues brought up in #646. Here's what's changed:

renamed: onResponse => replayDidSucceed
renamed: onRetryFailure  => replayDidFail
added: requestWillEnqueue
added: requestWillDequeue

I've also added a utility method to deprecate (and warn about) old object methods since I'm renaming existing callbacks, but I didn't feel like it warranted a major version bump.

The only callback documentation I saw in the original code was here, so that's where I updated it:
https://github.com/GoogleChrome/workbox/blob/1bf91ec22d666d5ce5b42d505e4cc850ea3222d5/packages/workbox-background-sync/src/lib/background-sync-queue-plugin.js#L10-L18
I agree that the JSDoc comments need to be more clear, but I wasn't sure what conventions we were using for documenting sub-properties (I'm not seeing a lot of consistency, so I was going to bring it up in the next meeting).
If we're going to be following Google JS style, we should be documenting them like this (but again, I didn't see that so I'm not sure if there was a reason we weren't doing it that way, e.g. doc generation, etc.):
https://google.github.io/styleguide/jsguide.html#features-objects-destructuring

I was also going to suggest either I or @prateekbh do a clean up since most of the files in this project don't include the license, and there are many cases where the code doesn't match the style guide.
If I update the gulp-eslint and eslint-config-google versions I notice a lot more errors, most of which can be easily fixed via the --fix option.

(I'm in the middle of putting together a PR that updates some of our linting rules, tracked in #478)

(I'm in the middle of putting together a PR that updates some of our linting rules, tracked in #478)

Cool, happy to help out on that front if you want.

If I update the gulp-eslint and eslint-config-google versions I notice a lot more errors, most of which can be easily fixed via the --fix option.

Happy to do that. I tried adding prettier, but it conflicts with our rules a little too much(would like to discuss in the next meeting)

I've updated the jsdoc comments. @jeffposnick PTAL.

One more issue:
The current workbox-google-analytics API accepts a hitFilter function, that -- if it throws -- aborts the hit and stops further retries. If we want to keep this behavior, we'd have to add an additional feature here to support aborting a retry.
The way queuing currently works, items are left in the queue even after a successful retry; they're only removed during clean up.
If we want to support stopping a request from being replayed, we should probably add a utility method to remove an item from the queue rather than depending on a clean up cycle.
I think this makes sense since I know I was momentarily confused when debugging by the fact that successful replays remained in the queue.

If we want to support stopping a request from being replayed, we should probably add a utility method to remove an item from the queue rather than depending on a clean up cycle

I can take this up, do you want the item to be removed after successful replay or do you have anything else in mind?
P.S. clean up removes the request from IDB after its threshold time. But we can remove the request from the current instance after a successful replay.

@philipwalton  As far as hitFilter is concerned, maybe we can add a requestWillPlay callback much like requestWillDeque, and returning false from this can make the request skip replaying.

@philipwalton, are there "legitimate" use cases for wanting to abort things via an exception in hitFilter? Or was that just an unintended side effect of the previous codebase? It sounds like it might be cleaner not to support that, and add in catch() as needed, along with logging to report the error.

@philipwalton, are there "legitimate" use cases for wanting to abort things via an exception in hitFilter? Or was that just an unintended side effect of the previous codebase? It sounds like it might be cleaner not to support that, and add in catch() as needed, along with logging to report the error.

I'd be OK with not supporting it for the analytics use case. I originally implemented it primarily for compatibility with how tasks work in analytics.js, but I don't see it as a critical feature by any means.

@philipwalton  As far as hitFilter is concerned, maybe we can add a requestWillPlay callback much like requestWillDeque, and returning false from this can make the request skip replaying.

From a generic background sync perspective, I can imagine cases where a developer wouldn't want to replay a hit. Perhaps in cases where they know the current state data is fresher than what this hit will send (though I suppose you could argue they could/should handle that on the back end).
If we add this method I'd recommend the name requestShouldReplay, and if should probably be async since you'd likely need to check IDB to know whether or not it should replay.
..in fact, perhaps all of these should be async :)

@prateekbh and I discussed this offline, and if the rest of the team is OK with @jeffposnick's suggestion to drop support for aborting a replay by throwing in the hitFilter function, then we think we're ready to merge as is.
We decided it's probably better to wait for an actual use case for aborting a replay and design the API around that rather than trying to anticipate one.
So if everyone agrees, I think we should merge this and move the discussion of how to handling the API change in the workbox-google-analytics PR.

SGTM.


How to open slide menu only if swipe from left to right ?

i want to open slide menu only if swipe from left to right.right now if you swipe up and down then menu is appearing, how to stop that ?.
On normal UIVIew if i am swiping or touching from top to bottom then the slide menu is coming.

I am also facing the same issue, I want to open side drawer only when I swipe from left to right. Right now side drawer appears on slight swipe from top to bottom too.


Adding a font question

Note: for support questions, please use one of these channels: Chat: AngularClass.slack or Twitter: @AngularClass

I'm submitting a ...
[ ] question about the decisions made in the repository
How to add font locally from assets/fonts. What configuration have to add and where? Thank you

How I am suppose to add webfont?
I have made folder called "fonts" in assets and I have put my fonts there.
fonts.scss
@font-face {
  font-family: 'Lato';
  src: url('../assets/font/Lato-Regular.eot');
  src: url('../assets/font/Lato-Regular.eot?#iefix') format('embedded-opentype'),
  url('../assets/font/Lato-Regular.woff2') format('woff2'),
  url('../assets/font/Lato-Regular.woff') format('woff'),
  url('../assets/font/Lato-Regular.ttf') format('truetype');
  font-weight: normal;
  font-style: normal;
}

Also I have added configuration for fonts and add it to app.modules.ts
import '../styles/fonts.scss';
My questions what code should I add to webpack to make it work.?
Thank you

on this.


chore(npm): improve ignore list

Platforms affected
Motivation and Context

Description

Testing

Checklist

[ ] I've run the tests to see all new and existing tests pass
[ ] I added automated test coverage as appropriate for this change
[ ] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[ ] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[ ] I've updated the documentation if necessary

Interesting, why is no CI run triggered?


LCD what name?

LCD what name?

LCD Driver is HD44780. Its a very common LCD driver that is used.


Add specific details to Figma Community Navbar

I was going through the resources when I came across Figma resources and a navbar displaying "official website".
Where should the "official website" should be redirecting? Could you please clarify?
I thought that it would redirect me to Figma's official website but it  headed back to home page.
My suggestion is if you could change the label to "Previous" or "Back to Main" because it looks a bit confusing.
🚀 Feature Request
Is your feature request related to a problem? Please describe.

Describe the solution you'd like

)

Describe alternatives you've considered

Additional context

@bhavyasingh75 Thanks for your concern but this resources page is built with notion which is a No-code tool. So, We can't do much with it. So, Closing this issue.


Correctly set "Name" for SHOW STATS output

The output for SHOW STATS was not quite right, it wasn't correctly setting the measurement name -- previously the measurement name appeared as a column in the output. This change fixes that and also adds unit tests. Example output from SHOW STATS is below.
> show stats
name: runtime
-------------
Alloc   Frees   HeapAlloc       HeapIdle        HeapInUse       HeapObjects     HeapReleased    HeapSys         Lookups Mallocs NumGC   NumGoroutine    PauseTotalNs    Sys             TotalAlloc
8743320 9679037 8743320         25100288        10878976        102854          0               35979264        94      9781891 107     44              44741237        41896184        554651152

name: graphite
--------------
proto   batches_tx      bytes_rx        connections_active      connections_handled     points_rx       points_tx
tcp     230

@jwilder

:+1:


X11: implement with_resizable (#540)

[x] Tested on all platforms changed
[x] Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
[x] Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
[ ] Created an example program if it would help users understand this functionality

Setting the min/max to the same value creates a non-resizable window with X11.
regarding #540

FYI, the list of checkboxes got me to go back and edit documentation before creating the PR. ;)

Thanks! Looks good.
This still shows resize cursors on Xfwm4, but since it's fine on KWin, Mutter, Marco, Compiz, IceWM, Awesome, and Openbox, I'll let you off the hook.

Dang, do you have all those window managers installed locally? I only tested on Mutter with Gnome 3.


Missing aliases

The aliases field is missing for several advisories.
Running curl "https://api.osv.dev/v1/vulns/PYSEC-<id> on the following:

pillow : PYSEC-2021-317
apache-superset : PYSEC-2020-224
roundup : PYSEC-2019-201
salt : PYSEC-2019-119
salt : PYSEC-2019-49
pillow : PYSEC-2020-77
pillow : PYSEC-2020-78

The aliases field is missing but it is present in the corresponding YAML on GitHub repository.

Thanks for pointing these out! The underlying bug has been fixed and existing entries should be all fixed soon.

Thanks for flagging. This one is also fixed.


repo: Update build image and toolchains

Signed-off-by: Ryan Northey<EMAIL_ADDRESS>Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/retest

/lgtm deps


Secure API using a JWT instead of token auth

Description of Problem:
Currently, the webserver is either unsecured or secured using token authentication. The token authentication is non-standard. Rather than using our own token-based authentication, it would be good to switch to a standard method for securing these endpoints.
Overview of the Solution:
Instead of using tokens to authenticate a request, we can use JWT tokens. To make the switch easier, we can start using symetric encryption and a PW - this would mimic the token authentication.
This issue is related to https://github.com/RasaHQ/rasa/issues/2689 and should apply to REST input channels as well.
Definition of Done:

[ ] Tests are added
[ ] Feature described the docs
[ ] Feature mentioned in the changlog

This should probably also apply to socketio

JWT can be implemented in rest customer connetor with PyJwt.
@custom_webhook.route("/bot_auth", methods=["POST"])
        async def auth_token(request: Request):
            if not request.json:
                return response.json({"error":"Missing sender"}, 400)

            sender_id = await self._extract_sender(request)
            utcnow = datetime.utcnow() + timedelta(seconds=-5)
            expires = utcnow + timedelta(hours=24)
            try:
                payload = {'iat': utcnow,'sender_id': sender_id, 'role':'user', 'exp':expires}
                bot_token = jwt.encode( payload,
                                    'thisismysecret',
                                    algorithm='HS256').decode("utf-8")
            except Exception as e:
                return response.json({'error':str(e)}, 400)
            return response.json({"bot_token":bot_token}, 200)

decorator function required for authentication

@tmbo You mean that Rasa should generate the jwt tokens since Rasa already values existing tokens?

mhm I am not sure what is missing in our current implementation, because we already support JWT authentication in the Rasa OSS server API. I mean we could check and make sure that in Rasa X we are using jwt instead of tokens when communicating with Rasa - but all in all I think we are good and this can be closed.

There's related request for the action server to validate a token that is passed around other microservices. Could we add an optional jwt value to the rest channel.

yes I think securing the action server in some way is a good push :+1:

@tmbo Is there any reason why the JWT couldn't be stored in the sender id for now?

Would probably break the SQLTrackerStore because that would make the sender id field insanely long

Just a note, a few customers have requested this as a feature enhancement on our end.

Hi, Is anyone working on this? Can I take this enhancement?

@pnandhini11 Yes, fell free to take this one 💯

Can you assign this issue to me?

@wochinge Can some assign this issue to me?

@pnandhini11 Sure! Can you please write down a short implementation proposal before starting the implementation? Would be great if we align on the general direction as this is a pretty significant change.

@wochinge Sure will do.  I will do it over the weekend.

@wochinge Pls correct my understanding about this issue before I get into it and work for a solution.
Rasa already provided an option to secure APIs using JWT authentication.  And I believe its working fine.  Now we need to secure action server and Rest channel with the JWT authentication.

@pnandhini11 Hi Padma, are you able to secure the general RASA bot webhook api? A custom channel is allowed only in RASA-X which is a commercial product.

@b-quachtran What exactly have customers requested? Token authentication with the action server or for the REST channel?


Upgrade to MicroPython 2.1.2

This release contains a fix for a neopixel issue:
https://github.com/microbit-foundation/micropython-microbit-v2/issues/161
Tag:
https://github.com/microbit-foundation/micropython-microbit-v2/releases/tag/v2.1.2
Upgrade tracked by:
https://github.com/microbit-foundation/python-editor-v3/issues/1137

Flashes OK. Serial version number looks good:


feat: server no longer depends on Unity

BREAKING CHANGE: Authenticators get a Server reference in ServerSetup instead of NetworkServer
TODO

[ ] Switch ILogger for one that doesn't depend on UnityEngine
[ ] Test in my project

Few things.

Let's separate everything not part of server change code make things in smaller matches. The Ibehaviour and INetworkBehaviour let's keep out for now dont see it used anywhere.

INetworkServer its used on the mono class still. Is that intentional until rest of the files are changing. Should it not be part of Server class instead?

Few things.

Let's separate everything not part of server change code make things in smaller matches. The Ibehaviour and INetworkBehaviour let's keep out for now dont see it used anywhere.
INetworkServer its used on the mono class still. Is that intentional until rest of the files are changing. Should it not be part of Server class instead?

Sure, can remove it for now.
I kept it on Mono until we have all the classes as standalone, because NetworkIdentity and others depend on INetworkServer directly, and I don't want to break APIs (passing in the pure Server would break API).

Let's try to fix the code smells and bugs so sonar passes.

Let's try to fix the code smells and bugs so sonar passes.

I've done more work on this, will continue and push it tomorrow.


Provide static object slicer

closes #342

For completion, I need to think about how to associate static slicing with client PUT.

Another approach could be exposing the io.Writer interface, then the app will write to it in any way it wants to and all the magic will happen inside of this writer.

Another approach could be exposing the io.Writer interface.

This depends on what we wanna control. For example, successful writing 100B chunk to io.Writer doesn't mean that 100B has been written into the NeoFS. With io.Reader progress tracking becomes more realistic, but requires special support. I thought about this approach and like it

we could have some callback that can be called after each uploaded chunk

@roman-khimov  lets resolve https://github.com/nspcc-dev/neofs-sdk-go/pull/382#discussion_r1121527346, then I'll implement https://github.com/nspcc-dev/neofs-sdk-go/pull/382#discussion_r1119285703 and then we'll see what happens.

For example, successful writing 100B chunk to io.Writer doesn't mean that 100B has been written into the NeoFS

True, but writing 100 bytes into just about anything is not a guarantee until you Close() it and even then some fsyncs can be useful if you really care. Some caching/buffering/delays are kinda expected in this scenario and they can be documented. Another problem that is a bit more natural to solve with io.Writer is application generating content on the fly, like a live video stream for example. Readers/writers can easily be bridged together though.
The advantage of an io.Reader/callback interface is probably in that it also allows to control individual slices and real amount of data written into NeoFS in case it matters for and application. I'm not sure it's a widespread requirement though.

Added new method which returns io.Writer. Currently it show very low performance (much lower than with io.Reader). I'll take a look at how to optimize it

@roman-khimov, i realized that if user has client connection and wants to get data io.Writer, then he needs to construct slicer instance from the client. I've added functionality for this in 2c7c824cad421e1044a244148288256b64b8c555


contentType: ContentType.json,

New Issue Checklist

[ ] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
e.g. flutter / ios / android

Platform Version
e.g.  1.5.0 / 12.0 / 9.0

Dio Version
e.g.  2.1.0 / 1.0.17

Android Studio / Xcode Version
e.g. Android Studio 3.3.2 / Xcode 10.2.1

Repro rate
e.g. all the time (100%) / sometimes x% / only once

Repro with our demo prj
e.g. does it happen with our demo project?

Demo project link
e.g. link to a demo project that highlights the issue

Issue Description and Steps
Please fill in the detailed description of the issue (full output of any stack trace, compiler error, ...) and the steps to reproduce the issue.

contentType: ContentType.json,  undefined name ContentType

Definitely still a problem, though.


Fix pvc.StorageClassName order

If both storage_class_name and image are provided, then storage_class_name gets overwritten by the one derived from the image.
This changes the order of assignment for the inline disk spec and adds a check for the volume resource, so if storage_class_name is provided, it doesn't get overwritten.
Tested it by building the provider locally and verifying that it creates the PVC with the correct StorageClass, both with specifying it and without.

Since The storageclass of image volume cannot be modified,
so the storage_class_name field in harvester_image is a Computed field in v0.5.2
will rename the field storage_class_name to volume_storage_class_name in https://github.com/harvester/terraform-provider-harvester/pull/54
and add a new field storage_class_name as a template to help user to specify storageClass parameters
more info: https://github.com/futuretea/harvester/blob/storage-tiering/enhancements/20220531-storage-tiering.md


[BUG] pip install failed on M1 Mac

pip install graphscope --pre failed with errors.
error.log
Please address this in the next release for M1.

See also: https://github.com/grpc/grpc/issues/31737


feat: Add task request details page

Overview
This PR is a part of stacked PR pattern, The PR below it are:

#402

Desktop view

status waiting
 Show

status approved
 Show

loading
 Show

Mobile View

status approved
 Show

status waiting
 Show

loading
 Show

PR for tests: #407


file not found when mime file doesn't exist

Hi. I installed handlr from aur a few minutes ago and I would get this no matter what arguments i try.
Error: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" })
it was fixed when i created ~/.config/mimeapps.list

@return127 try updating to v0.5, the problem should be fixed now

I'm having a similar error when trying to open URLs with handlr, though the help suggests that handlr open url should be supported:
handlr open https://en.wikipedia.org
No such file or directory (os error 2)

What I am trying to do is change my default web browser from Firefox to Nyxt (using a nyxt.desktop file).


Remove gray background for transparent images

Summary
Removes placeholder (bg color for now) for fully loaded images (we can also do that or cached/small images, but we can improve that once we have a clear vision for the placeholder/loading bar).
Relevant Technical Choices
Small resourceList refactor to prevent transparent images placeholder flickering in some cases.
I've added a new resource with a transparent background to the media gallery and corresponding Percy test for it.
To-do

User-facing changes

Testing Instructions
PNGs with transparent background should be displayed correctly (when fully loaded).
Not like this:

Fixes #4316

Seems to work excellently. There might be a very short flash of large gray square, but it's barely noticeable.

Yep, that part can be improved "once we have a clear vision for the placeholder/loading bar".
This PR is mostly about fixing the full-size image.

Verified in QA


FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

TypeScript Version:  2.4.1 / nightly (2.5.0-dev.20170707)
Code
gulpfile.js
// A *self-contained* demonstration of the problem follows...
gulp.task('compile-ts', () => {
	const ts = require('gulp-typescript');
	const tsProject = ts.createProject('./tsconfig.json');
	tsProject.options.module = 1;
	const dest = tsProject.options.outDir;
	return gulp.src('./src/**/*.ts')
		.pipe(tsProject())
		.pipe(gulp.dest(dest));
});

tsconfig.json
{
        "include": [
                "src"
        ],
        "compilerOptions": {
                "module": "commonjs",
                "target": "esnext",
                "noImplicitAny": false,
                "sourceMap": false,
                "emitDecoratorMetadata": true,
                "experimentalDecorators": true,
                "strictNullChecks": false,
                "noImplicitThis": true,
                "rootDir": "./src/",
                "rootDirs": [],
                "allowJs": false,
                "allowUnreachableCode": false,
                "allowUnusedLabels": false,
                "alwaysStrict": true,
                "baseUrl": "",
                "charset": "utf8",
                "declaration": false,
                "inlineSourceMap": false,
                "allowSyntheticDefaultImports": false,
                "diagnostics": false,
                "emitBOM": false,
                "forceConsistentCasingInFileNames": false,
                "importHelpers": false,
                "inlineSources": false,
                "isolatedModules": false,
                "lib": [
                        "es6",
                        "es7",
                        "dom"
                ],
                "listFiles": true, // default false
                "listEmittedFiles": true, // default false
                "locale": "zh_CN",
                "newLine": "CRLF",
                "noEmit": false,
                "moduleResolution": "node",
                "noEmitHelpers": false,
                "noEmitOnError": false,
                "noImplicitReturns": false,
                "noImplicitUseStrict": false,
                "maxNodeModuleJsDepth": 0,
                "noLib": false,
                "outDir": "./dist/",
                "noFallthroughCasesInSwitch": false,
                "noResolve": false,
                "noUnusedLocals": false,
                "noUnusedParameters": false,
                "paths": {},
                "preserveConstEnums": false,
                "pretty": true,
                "removeComments": false,
                "skipDefaultLibCheck": true, // default false
                "skipLibCheck": true, // default false
                "stripInternal": false,
                "suppressExcessPropertyErrors": false,
                "suppressImplicitAnyIndexErrors": true, // default false
                "traceResolution": true, // default false
                "typeRoots": [],
                "types": [],
                "watch": false
        }
}

Expected behavior:
Actual behavior:
Exception i got:
<--- Last few GCs --->

[24442:0x2def9d0]    58440 ms: Mark-sweep 1400.1 (1465.9) -> 1400.1 (1465.9) MB, 1948.3 / 0.0 ms  allocation failure GC in old space requested
[24442:0x2def9d0]    60312 ms: Mark-sweep 1400.1 (1465.9) -> 1400.1 (1434.9) MB, 1871.6 / 0.0 ms  last resort
[24442:0x2def9d0]    62184 ms: Mark-sweep 1400.1 (1434.9) -> 1400.0 (1434.9) MB, 1871.8 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3d1f48829891 <JS Object>
    2: write [/home/taoqf/feidao/temp/boai-h5-6/1/node_modules/typescript/lib/typescript.js:~9035] [pc=0x3cfdedb6fe70](this=0x1857e443e689 <an Object with map 0xddd4399bc9>,s=0x3c6749682c29 <String[4]: "id">)
    3: pipelineEmitExpression [/home/taoqf/feidao/temp/boai-h5-6/1/node_modules/typescript/lib/typescript.js:~66807] [pc=0x3cfdedbd085f](this=0xf51a038dbf9 <JS Global Object>,node=0x29110b...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [gulp]
 2: 0x13647ec [gulp]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [gulp]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [gulp]
 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [gulp]
 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [gulp]
 7: 0x3cfded28437d
Aborted

More

tsc works fine.
do not depend on fp-ts, fantasyland and monet
if i compile my ts files separately(1 ts file per time), also there is no exception.
i have searched all the issues already and found no clue. like https://github.com/Microsoft/TypeScript/issues/14628#issuecomment-314133082

@sandersn , Could this be fixed?

Two questions:

Can you provide the code or a small repro?
The gulp-typescript setup changes options.module=1. Does tsc still succeed when you set the equivalent module option in tsconfig? It's commonjs right now and I'm not sure that 1 means commonjs

Afterwards, if tsc still works, then it sounds like a gulp-typescript bug. The stack is in emit when it runs out of memory. This is very unusual for vanilla tsc, so I would first suspect a configuration/setup bug in gulp-typescript.
Also, @rbuckton, does that crash location mean anything to you?

1 means comonjs 2 amd 3 umd
I set options.module=1 this because i have multi tasks in gulp.
I guess gulp may use more memory than tsc because it will store files in memory steam.
I have tested if I reduce the number of the file it will succeed.

looks like the gulp typescript + typescript memory usage pushes you over the limit. I would say gulp-typescript should take the first stab at this issue.

@mhegazy , I got this Error just now:
<--- Last few GCs --->

[22042:0x3696020]    34073 ms: Mark-sweep 1403.3 (1447.3) -> 1403.3 (1442.3) MB, 877.7 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 878 ms) last resort
[22042:0x3696020]    34966 ms: Mark-sweep 1403.3 (1442.3) -> 1403.3 (1442.3) MB, 891.5 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2a16e0b9cef1 <JSObject>
    1: declareSymbol [/home/taoqf/.nvm/versions/node/v8.4.0/lib/node_modules/typescript/lib/tsc.js:~18571] [pc=0x1b65f5d1bf8b](this=0x3f86308890d9 <JSGlobal Object>,symbolTable=0x10f596065381 <Map map = 0x17f790d183b9>,parent=0x220cdb20d551 <Symbol map = 0x17f790d19019>,node=0x14238561b021 <Node map = 0x17f790d14e19>,includes=4,excludes=0)
    2: bindPropertyOrMethodOrAccessor [/home/taoqf/.nvm/...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x13740dc [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 7: 0x1b65f58040dd
[1]    22042 abort      tsc

using tsc.

can you share your project

+1
I have same problem.
I can't share my project because our enterprise project.
This is not appear in normal compile environment, but almost appear in below environment.

large or huge size project
If you change the file frequently during typescript compile (with hot module replacement), a crash appears.

@taoqf @swlee60 can you try typescript@next ? The recent change #17984 improved checker resource usage for recursive types references. (eg class List<T> { map<U>(f): List<U> }). This doesn't directly address the problem of huge projects, but it's likely to help since huge projects typically have some specialised collections somewhere.
If that doesn't help, is there some way you can share your project? I don't have any other starting point for debugging since eliminating gulp-typescript from the crash.

@sandersn
I used typescript@next but problem still appears.
Problems occur on both sides of awesome-type-script-loader and ts-loader
$ tsc --version
Version 2.6.0-dev.20170824
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
25546 ******    20   0 2763m 1.5g  16m R 99.6 39.9   3:30.55 node
webpack: Compiling...
<--- Last few GCs --->
551020 ms: Mark-sweep 1288.8 (1408.1) -> 1288.8 (1412.1) MB, 876.2 / 0.0 ms [allocation failure] [GC in old space requested].
551884 ms: Mark-sweep 1288.8 (1412.1) -> 1288.8 (1412.1) MB, 864.3 / 0.0 ms [allocation failure] [GC in old space requested].
552797 ms: Mark-sweep 1288.8 (1412.1) -> 1297.8 (1408.1) MB, 912.2 / 0.1 ms [last resort gc].
553713 ms: Mark-sweep 1297.8 (1408.1) -> 1306.9 (1408.1) MB, 916.7 / 0.1 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x3f6911dcfb51
2: /* anonymous */ [/home1/irteam/dev/partner/booking-web/partner/node_modules/source-map/lib/source-node.js:~336] [pc=0x3c44f4a6f4a0] (this=0x3f6911de6119 ,chunk=0x2bbfebf8e701 <String[22]: \n    font-size: 20px;>,original=0x235420ed6be1 <an Object with map 0x1c0b11e7f8d9>)
3: walk(aka SourceNode_walk) [/home1/irteam/dev/partner/booking-web/partner/node_modules/source...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/home1/irteam/apps/bin/node]
2: 0x1098b2c [/home1/irteam/apps/bin/node]
3: v8::Utils::ReportApiFailure(char const*, char const*) [/home1/irteam/apps/bin/node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/home1/irteam/apps/bin/node]
5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [/home1/irteam/apps/bin/node]
6: v8::internal::TypeFeedbackVector::New(v8::internal::Isolate*, v8::internal::Handlev8::internal::TypeFeedbackMetadata) [/home1/irteam/apps/bin/node]
7: 0xb47d40 [/home1/irteam/apps/bin/node]
8: 0xb480a9 [/home1/irteam/apps/bin/node]
9: 0xb4a4a0 [/home1/irteam/apps/bin/node]
10: 0xb51b46 [/home1/irteam/apps/bin/node]
11: v8::internal::Compiler::Compile(v8::internal::Handlev8::internal::JSFunction, v8::internal::Compiler::ClearExceptionFlag) [/home1/irteam/apps/bin/node]
12: v8::internal::Runtime_CompileLazy(int, v8::internal::Object**, v8::internal::Isolate*) [/home1/irteam/apps/bin/node]
13: 0x3c44e83092a7
Done in 554.33s.

@sandersn same here.
yarn list typescript
yarn list v0.27.5
└─<EMAIL_ADDRESS>Done in 5.03s.

Exception:
<--- Last few GCs --->

[25632:0x2a01020]    41726 ms: Mark-sweep 1403.2 (1456.8) -> 1403.2 (1444.8) MB, 884.6 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 885 ms) last resort
[25632:0x2a01020]    42607 ms: Mark-sweep 1403.2 (1444.8) -> 1403.2 (1444.8) MB, 880.9 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1f2d7e99cef1 <JSObject>
    1: set [native collection.js:~247] [pc=0x1a70fe13f9d3](this=0x24e4b83551 <Map map = 0x26bd5c2183b9>,p=0x38da3313ee71 <String[2]: id>,x=0x6fefaff14f1 <Symbol map = 0x26bd5c232be9>)
    2: declareSymbol [/home/taoqf/.nvm/versions/node/v8.4.0/lib/node_modules/typescript/lib/tsc.js:~18571] [pc=0x1a70fe51a23b](this=0x154578890d9 <JSGlobal Object>,symbolTable=0x24e4b83551 <Map map = 0x26bd5c2183b9...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x13740dc [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Allocate(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node]
 7: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap>, int) [node]
 8: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::EnsureGrowable(v8::internal::Handle<v8::internal::OrderedHashMap>) [node]
 9: v8::internal::Runtime_MapGrow(int, v8::internal::Object**, v8::internal::Isolate*) [node]
10: 0x1a70fe0040dd
Aborted

@sandersn same here.
tsc --version
Version 2.6.0-dev.20170824
$ tsc

<--- Last few GCs --->

[26137:0x3b6b020]    37623 ms: Mark-sweep 1403.3 (1445.3) -> 1403.3 (1440.8) MB, 861.8 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 862 ms) last resort
[26137:0x3b6b020]    38487 ms: Mark-sweep 1403.3 (1440.8) -> 1403.3 (1440.8) MB, 864.1 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xc3bfff9cef1 <JSObject>
    1: declareSymbol(aka declareSymbol) [/home/taoqf/.nvm/versions/node/v8.4.0/lib/node_modules/typescript/lib/tsc.js:~14980] [pc=0xac3a15e8497](this=0xc3bfff82241 <undefined>,symbolTable=0x1eb8b183ecc9 <Map map = 0x4a1246983b9>,parent=0x29b2e1ed5e61 <Symbol map = 0x4a12469a4b9>,node=0x2cc77d79521 <Node map = 0x4a124695a79>,includes=4,excludes=0,isReplaceableByMethod=0xc3bfff82241 <undefined>)
  ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x13740dc [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
 6: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 7: 0xac3a14040dd
Aborted

@sandersn I'm totally ignorant of tsc internals, but I might have a thought worth chasing.  If I understand it correctly, the fix you provided (PR https://github.com/Microsoft/TypeScript/pull/17984) seems incomplete, in at least a couple of ways:

It doesn't (at least from your description) address constrained (extends) types.
It doesn't address cycles of isomorphic generic-interface-containing-generic-method where the method also consumes  the class's type parameters.

FWIW!!!  I suspect these may be subcases of broader sets of unaddressed cases.  In any event, my main reason for writing is to provoke you to think along these lines for missed cases.  If you're not one yourself, you might want to consult a Typescript "language lawyer".

@swlee60 Out of curiosity, does your crash disappear if you disable sourcemaps (source-map is in your crash log, after all)?

@weswigham Do you mean webpack devtool option? Whether I set inline-source-map or eval, heap overflow always comes up.

Interesting. Looks like @taoqf's project runs out of memory in the binder. Not much has changed in the binder recently, though.

@taoqf, what is the version of Typescript that you used before 2.4.1? Your project fails on 2.3 for me as well.

On master (I haven't checked with 2.3 or 2.4), the AST alone brings memory usage to 1.1 GB, which means that every byte of source results in almost 40 bytes in the parse tree. That leaves the compiler about 300 MB to bind and check with. It is unable to do so.
We need to think about architectural ways to reduce parse tree overhead to make sure that Typescript can scale to very large projects. Specifically, how much overhead we incur when parsing common patterns like object literals.
As a workaround, @taoqf, can you split your compilation into multiple compilations? Then you could compile it in multiple pieces. You can use .d.ts files, either generated or manually created, as interfaces between the split compilations.

@taoqf, what is the version of Typescript that you used before 2.4.1?

I always used next version.

<--- Last few GCs --->

   64156 ms: Mark-sweep 1302.7 (1435.3) -> 1298.6 (1435.3) MB, 2648.5 / 0.0 ms [allocation failure] [GC in old space requested].
   66567 ms: Mark-sweep 1298.6 (1435.3) -> 1298.6 (1435.3) MB, 2411.3 / 0.0 ms [allocation failure] [GC in old space requested].
   69175 ms: Mark-sweep 1298.6 (1435.3) -> 1306.1 (1409.3) MB, 2607.0 / 0.0 ms [last resort gc].
   71539 ms: Mark-sweep 1306.1 (1409.3) -> 1313.8 (1409.3) MB, 2363.7 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2dd8a203fa99 <JS Object>
    1: set [native collection.js:~252] [pc=0x27ff7fb0d83b] (this=0x3c34b6f24d71 <a Map with map 0x230c6e7113b1>,r=0x1039b0b29051 <String[13]: 896958,897404>,z=0x1039b0b29079 <a TypeObject with map 0x230c6e796f31>)
    2: getTypeWithThisArgument [/home/jenkins/workspace/Project/web-app/node_modules/typescript/lib/typescript.js:~31577] [pc=0x27ff7fbe98bc] ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [@angular/cli]
 2: 0x10d368c [@angular/cli]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [@angular/cli]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [@angular/cli]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [@angular/cli]
 6: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::Allocate(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [@angular/cli]
 7: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap>, int) [@angular/cli]
 8: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::EnsureGrowable(v8::internal::Handle<v8::internal::OrderedHashMap>) [@angular/cli]
 9: v8::internal::Runtime_MapGrow(int, v8::internal::Object**, v8::internal::Isolate*) [@angular/cli]
10: 0x27ff7e2060c7

@tiger-seo can you open a new issue? There are multiple reasons for the compiler to run out of memory and your reason is likely different from @taoqf.

@taoqf if compilation worked previously, what else changed? Did your project add a lot more files? Did the individual files get a lot bigger? The project you gave me fails on 2.3, 2.4 and 2.5 and I expect it would fail on earlier versions because the AST memory footprint hasn't grown much recently.

It never worked , I compiled in multiple pieces in my project now.

@sandersn , I tried to use gulp-typescript and --max-old-space-size=4096 to run script, it will be ok, it would be very nice if tsc and gulp-typescript could accept this option.

I recently ran into the same issue using ts-loader and webpack after upgrading Typescript from 2.3.4 to 2.5.2. I downgraded to 2.4.2 and 2.4.1 and the issue persisted. With 2.4.0, however, everything works as expected.

@troeggla How big is your project? If the project is less than 10 MB of source, then it's probably a different cause.
Can you

Try building with typescript@next?
If it still fails, open a separate issue.
Share your source with us? I can sign an NDA if necessary.

Randomly getting this on my project
07:28:49 - File change detected. Starting incremental compilation...

<--- Last few GCs --->

[1592:000002842FE68000]  7027823 ms: Mark-sweep 1401.5 (1465.4) -> 1401.5 (1465.4) MB, 4599.6 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000033CF2C1CEA9 <JSObject>
    1: set [native collection.js:~247] [pc=000000C2A8FE6697](this=000000F425FBE6F1 <Map map = 000003AA6FD983B9>,p=000003A9ED17AB21
<String[9]: @Telefone>,x=000003A9ED17AB51 <Type map = 000003AA6FDC6CF9>)
    2: getLiteralType(aka getLiteralType) [C:\nodejs\node_modules\typescript\lib\tsc.js:~24883] [pc=000000C2A8D93E76](this=0000033C
F2C02241 <undefined>,value=000000CEC1A9C961 <String[8]: Telef...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
The terminal process terminated with exit code: 3

not too big, 36 typescript files, but only 120kb in total

@sandersn Thanks for looking into this! Yeah, my project is well below 10 MB (excluding packages though) and the resulting bundle comes to about 1.9MB (without minification). Building with typescript@next compiles as it did with Typescript 2.4.0.
If you want, I can give you access to a private repository.

@troeggla If the nightly typescript is working for you then I think the recent bug fixes apply and we don't need your source.
@pocesar Can you open a separate issue? For only 120k of source you do not have the same issue as @taoqf.
Total RAM doesn't make a difference unless you have configured node to use more than 1.4 GB. You can try raising it to see if the problem goes away, and try lowering it to see if you can make it reproduce reliably. I think the flag is --max-old-space-size=8192 (for 8 GB).
You can also try compiling with --diagnostics to get a count of types and symbols. More than half a million types is a suspicious number.

@sandersn
Hello, When do you think this problem is solved?
My team also has large project, and we changed TypeScript version to 2.4.0 because upper TS 2.4.1 occurs this FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory issue.
We want to use awesome latest TypeScript's features, like 2.5, 2.6, ... or 3.0 and after! But we don't know when is this problem solved, so we can't upgrade TypeScript no more.
If you can, please let me know the progress of this issue.

@cheerun Try typescript@next. We have fixed all but a two of the known issues there, and those fixes will also ship with 2.5.3.
If you have 10s of megabytes of source then you may still have a problem; that's what this issue tracks.

The OP seems to be caused by a large object structure, @sandersn's comment in https://github.com/Microsoft/TypeScript/issues/17112#issuecomment-326684693 summarizes the issues.


DataFrame print table, Main menu and CMake copy CSV files to build directory

In this pull requested I have added a couple features:

Main menu with 8 options, including read, save and show table for grades and students, add and remove students
Pretty print DataFrame table
Export GradeBook data structures to DataFrames
CMakeLists.txt copy CSV files to build directory

@andrewlod this looks like you only shipped the build and some .csv's not the update(s).

My apologies for that. Usually I would just do git add . and all changes, including modifications and deletions would be included, but for some reason, they were not this time. I have included the changes in a new pull request.

Do we make separate testdata for individual tests, use our gradebook program to help the teacher input the grades/scores, or?

I suppose test grades could work the same way as assignment grades, where there is a CSV file, say, testgrades.csv with columns test_id,student_id,grade, and another CSV file testdata.csv with columns test_id,name/description,class_id.
Now for the teacher to update grades efficiently, it would be a lot easier if there was an actual UI (web or even Desktop) where a teacher could easily jump from one student to another (using arrow keys or the mouse, for example), in order to post grades. Another solution would be: GradeBook knows all students, and students could be linked to classes (CSV studentclasses.csv with columns student_id,class_id,semester), and tests are also linked to classes. Whenever a teacher wants to fill in the grades of an ungraded test, GradeBook could query all students of the class the test was applied and ask the grade, one by one, without the teacher having to type the name (or ID) of each student, just the grades. Does that make sense?

Yeah that does make sense. I originally thought about some sort of UI when I made this program as well  as intended for it to actually be used.
I don't know about a Web UI but certainly something useable could be made for the program.
Some sense of security needs to be had too so that someone couldn't get in and change grades scores etc without authorization which could be just username and password and a verification, maybe a couple levels deep.

Since we are thinking about authentication and UI, maybe it would make sense to make it a server (i.e. API) with a front-end. I have some experience with CPPRestSDK and it is cross-platform. There are some advantages to this approach, where people from multiple devices can connect to the service. Alternatively, if the goal is to have a local application only, it is possible to proceed with an approach without a server (like the current state of the project).
As for UI libraries, I have worked with openFrameworks and wxWidgets. Maybe the second one fits better in GUIs, but I am not completely sure. There is also QT, but I have never used it.

I would be concerned with making the program capable.  ie features first before we actually go down the UI route.. but yes, UI.  I am completely ok with a text based UI  akin to 16 bit early 32 bit DOS applications.
API would be good too.


Fix the KR580VM1 backend

I put the bug label here because the actual backend is there, but the reality is there's been no attempt since the Big Refactor to actually get it doing anything

not bothering with kr580vm1 backend since v.0.1.2


[Bug]: curl: (7) Failed to connect to 2a03:2880:f10a:83:face:b00c:0:25de: Network is unreachable

Preflight Checklist

[X] I agree to follow the Code of Conduct that this project adheres to.
[X] I have searched the issue tracker for an issue that matches the one I want to file, without success.
[X] I am not looking for support or already pursued the available support channels without success.

Version
1.11.0
Installation Type
Official Docker Compose
Service Name
Deploy(DongTai-deploy)
Describe the details of the bug and the steps to reproduce it
安装时报错：curl: (7) Failed to connect to 2a03:2880:f10a:83:face:b00c:0:25de: Network is unreachable

Additional Information
No response
Logs
No response

解决了，网络问题

我也出现了这个问题，后来怎么解决的啊


Updated Restic e2e tests & added default plugins e2e tests

/ok-to-test

@dymurray Applied your suggest changes. Please let me know if you notice anything else.

TODO: I would like to revisit the logging from ginkgo rather than using fmt.Print... everywhere. We can revisit that in another PR

/ok-to-test


Investigate distance from host issues

Issue related to player entering / leaving vehicles, fixed in version 0.1.7
Race conditions caused player to briefly use the wrong coordinates


Add mysql2 support

What is the purpose of this pull request?
Added mysql2 support
Is there anything you'd like reviewers to focus on?
Added a new Mysql Adapter
Mysql doesn't support ON CONFLICT (but has ON DUPLICATE KEY UPDATE) and doesn't have RETURNING ID
So, I used ActiveRecord::Base.connection.update(sql) for getting amount of updated rows
Also, Mysql doesn't understand EXCLUDED, so we need to use some method wrap_column_name, which will be defined in every adapter. Because, Mysql understands VALUES(here_column_names), but Postgresql understands EXCLUDED.here_column_name
Checklist

[- ] I've added tests for this change
[+] I've added a Changelog entry
[+] I've updated a documentation

Thanks!


Diverse Lucene options

I am throwing the rest into this one issue as I feel they are a bit more exotic. We can always pull then into their own issues later, should there be more interest. I believe a couple were already suggested too.

fielded searching (e.g. title, author, contents)
flexible faceting, highlighting, joins and result grouping
sorting by any field
allow simultaneous update and searching
fast, memory-efficient and typo-tolerant suggesters

Scott

This issue was moved to orientechnologies/orientdb#5186


Do not hard code registry port

As a registry admin I am able to configure the registry to serve on any port. When using a port other than 5000 I get this error:
    "Invalid format of docker reference: expecting port 5000"

See https://github.com/projectatomic/skopeo/blob/4421e7ea2face14ce4530ed6e6df66b6f94d544f/vendor/github.com/containers/image/openshift/openshift.go#L164
Could we discover the port using ~/.docker/config.json?

Philosophically, no, config.json could contain different registries for the same host, differentiated by a port.
Really what we need here is a way to get the official endpoint from the OpenShift API server, as https://github.com/projectatomic/skopeo/blob/4421e7ea2face14ce4530ed6e6df66b6f94d544f/vendor/github.com/containers/image/openshift/openshift.go#L164 says. Is that still not possible?

Philosophically, no, config.json

(Of course this means “in practice, yes”, but I’d much prefer a real solution to this, ingenious but ultimately risky and ugly workaround. Especially if the hostname ends up being one of the few public IPs of a cluster, this could easily collide with someone’s private registry exposed on the same IP address, couldn’t it?)

Really what we need here is a way to get the official endpoint from the OpenShift API server ... Is that still not possible?

Sadly, no. See https://github.com/openshift/origin/issues/2162

Especially if the hostname ends up being one of the few public IPs of a cluster, this could easily collide with someone’s private registry exposed on the same IP address, couldn’t it?

Yes. Good point.


Workaround for React test renderer not supporting refs

Hi,
I'm running into an issue where one of my third-party components uses refs.
This issue has already been explained and closed in https://github.com/facebook/jest/issues/1332, but what wasn't provided was a workaround.
As it stands, snapshot testing is totally broken for the component in question, as well as its parent components. The component in question is a third-party component for integration with MapBox - is there a way to mock out the ref?
How about simply not testing the MapBox component, and mocking out that component for its parent components? I can live with that component not being tested, but I need to be able to test the components further up the tree.
Would appreciate any guidance. Recently came back to Jest after almost 2 years, and it's gotten tons better since then :)
Node v6.2.2, NPM 3.9.5

This will be part of React 15.4: https://github.com/facebook/react/issues/7740
You can do jest.mock('path/to/MapBox', () => 'MapBox') which should completely stub it out.

@cpojer great, it really help me to resolve the problem. Thanks a lot.


make_csv_dataset fails inside Dataset.interleave()

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):no
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): debian
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.5.0
Python version: 3.8
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: NA
GPU model and memory: NA

Describe the current behavior
make_csv_dataset  fails  when used as a map function inside data.Dataset.interleave()
Describe the expected behavior
It must create datasets
Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
files = tf.data.Dataset.list_files("*.csv")
ds3 = files.interleave(lambda x: tf.data.experimental.make_csv_dataset(x, batch_size=1))

link to colab
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.
Running this code:
files = tf.data.Dataset.list_files("*.csv")
ds3 = files.interleave(lambda x: tf.data.experimental.make_csv_dataset(x, batch_size=1))

I get the following error:
OperatorNotAllowedInGraphError: in user code:

    <ipython-input-5-fcdae26c49e4>:2 None  *
        lambda x: tf.data.experimental.make_csv_dataset(x, batch_size=1))
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/experimental/ops/readers.py:480 make_csv_dataset_v2  **
        filenames = _get_file_names(file_pattern, False)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/data/experimental/ops/readers.py:1135 _get_file_names
        file_names = list(gfile.Glob(file_pattern))
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/lib/io/file_io.py:383 get_matching_files
        return get_matching_files_v2(filename)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/lib/io/file_io.py:449 get_matching_files_v2
        for single_filename in pattern
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:520 __iter__
        self._disallow_iteration()
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:516 _disallow_iteration
        self._disallow_in_graph_mode("iterating over `tf.Tensor`")
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py:496 _disallow_in_graph_mode
        " this function with @tf.function.".format(task))

    OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

@saikumarchalla  friendly ping

@rmothukuru ,
I was able to reproduce the issue in tf v2.4,v2.5 and nightly.Please find the gist here.

@jvishnuvardhan
Any update on this?

Thanks for reporting this @eli-osherovich, this looks like a real bug that I'll work on fixing.
The reason this hasn't come up before is that calling make_csv_dataset inside of interleave isn't usually necessary. make_csv_dataset will incorporate its own interleave when you set num_parallel_reads. i.e., instead of writing
files = tf.data.Dataset.list_files("*.csv")
ds3 = files.interleave(lambda x: tf.data.experimental.make_csv_dataset(x, batch_size=1))

you could write
ds3 = tf.data.experimental.make_csv_dataset("*.csv", batch_size=1, num_parallel_reads=tf.data.AUTOTUNE)

The second version may also be more performant, because it will use multiple threads to read from CSV files in parallel.

Thanks, @aaudiber
May real code is a bit more complex: in my case each CSV file represents a  different class (with big skew between classes) and I wanted to do sort of round-robin sampling.

@eli-osherovich CsvDataset accepts Tensor filenames, so you could use it directly from inside of interleave. make_csv_dataset on the other hand requires string filenames, so that it can infer column names based on the filenames. It would be convenient if make_csv_dataset could support Tensor filenames as well, but right now your best option is to use CsvDataset instead.

Thanks, @aaudiber  I will try CsvDataset, even though, I would prefer the automation of make_csv_dataset


GeographicPosition is obsolete in GeoJSON.Net

See changeset https://github.com/GeoJSON-Net/GeoJSON.Net/commit/81080d3fdb7b7ef9ce2a6f9af9af7d278e860473
GeographicPosition needs to be replaced with Position.

OK thanks @inzanenl. Will fit that within the next hours.

OK. Nuget package published.
@inzanenl Can you test if it is OK, and close if it is ?
Thanks

Seems OK now! I'm closing this issue.


UX: delete a workflow group causes an error without enough information

Describe the bug
If you try to remove  workflow group that is associated with some workflow tasks the backend raise a 422 error but the UI will just report a generic error
To Reproduce
Steps to reproduce the behavior:

create a collection
assign an editor role to the collection setting an user as member of the created group
deposit an item in such collection
try to remove the editor group

Expected behavior
A message saying that there are related workflow tasks and so the group cannot be removed would be more appropriate

would be appropriate to assume that the 422 status is only returned in such scenario as by the current code? if so, we can easily fix that just in the message otherwise this would required a major redesign of the error message on the rest side to include more details

Showing a more specific error in case rest returns a 422 can be done in 2 hours
We'd like to claim this task

@artlowel : Sounds good. Adding estimate & assigning to you.


[Bug]: yarn deploy fails when checkout-ui-extension and theme-app-extension exist in same repo

Please confirm that you have:

[X] Searched existing issues to see if your issue is a duplicate. (If you’ve found a duplicate issue, feel free to add additional information in a comment on it.)
[X] Reproduced the issue in the latest CLI version.

What type of project are you working on?
extension
Expected behavior
Running yarn deploy should work.
Actual behavior
yarn deploy fails.
Theme app extension runs bundle exec theme-check but this is failing for linter issues that should not be present in theme-check for theme app extensions. These linter issues cannot be disabled either (or where is the documentation?).
This is blocking deploying checkout-ui-extensions.
Example error:
error: UndefinedObject: Undefined object app
app is a valid liquid object for theme app extensions, ie: {{ app.metafields.NAMESPACE_HERE }}.
Stack trace
No response
Reproduction steps

Create a repo with various extensions, embedded app, etc
yarn deploy
fails.

Operating System
Mac OS Monterey
CLI version (run shopify version if you're not sure)
latest / 3.x
Shell
iTerm2
Ruby version (run ruby -v if you're not sure)
3.1.2
What language and version are you using in your application?
Ruby 3.1.2

This should be fixed via: https://github.com/Shopify/cli/pull/113
Will re-open if the issue persists once that commit is packaged + released.

This should be fixed via: https://github.com/Shopify/cli/pull/113
Will re-open if the issue persists once that commit is packaged + released.


Use coroutines to create modules in parallel

Are you sure that it is faster? You still have only one hard drive, and that is a bottle neck I belive

Don't understand me wrong, I don't have anything against this PR. Just was curious if you did some measurements. I'll try to do them now.

I am doing that right now, just for curiosity :p
On Fri, Nov 10, 2017 at 12:13 PM Nikita Kozlov<EMAIL_ADDRESS>wrote:

Don't understand me wrong, I don't have anything against this PR. Just was
curious if you did some measurements. I'll try to do them now.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/borisf/java-generator/pull/26#issuecomment-343574329,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AcLuTeCiHuxTo1mEZGROEe5qBU3b1sgvks5s1K5TgaJpZM4QaAVZ
.

You are right, the bottleneck is the drive, but it hides some of the IO latency. On my machine, creating a project with the default settings and 2000 modules takes 19022ms in parallel and 92965ms without coroutines (my machine has a SSD, probably on spinning disk it will not be such an improvement)

Actually, probably it will a better improvement on spinning disks, I am not sure.

2000 modules? My mac almost died trying to achieve this without coroutines. With coroutines it did much better job, but still more then 1 min.

On spinning disks it might be slower, because we are not writing one big stream of bytes, rather a lot of small files. But I'm not sure as well.

Yes, probably with lots of methods, to make files big enough. I am not near to one to test.
Also, I tried coroutines on a single thread (using a newSingleThreadContextand) it did not improve, it seems that one single thread cannot use all disk bandwidth. All this conclusions are dependent on hardware anyway, but I think we can assume that it is better to use them.

Thanks all merging


"JavaScript heap out of memory" when running force:source:retrieve

Summary
I have an existing org which I'm trying to import with my package.xml file. It's a decent sized org but not huge: 160MB of Documents, nothing else significant.
When running force:source:retrieve, I get this:
FATAL ERROR: invalid table size Allocation failed - JavaScript heap out of memory
I have tried running with NODE_OPTIONS=--max-old-space-size=8192 and I have also modified /usr/local/bin/sfdx and /Users/matt/.local/share/sfdx/client/bin/sfdx to add the parameter to NODE_FLAGS, but still no joy.
Steps To Reproduce:

NODE_OPTIONS=--max-old-space-size=8192 /usr/local/bin/sfdx --dev-debug force:source:retrieve --manifest /Users/matt/sites/fnsf-dx/manifest/package.xml

Expected result
Not sure, never run this command. Not expecting an out of memory error though.
Actual result
Full command debug output: https://gist.github.com/mattparlane/bba2c6ef5ee424bbce99320b0f005fd4
Additional information
MacBook-Pro-3:fnsf-dx matt$ /usr/local/bin/sfdx -v
sfdx-cli/7.1.4-79f97a7df8 darwin-x64 node-v10.15.3

MacBook-Pro-3:fnsf-dx matt$ /usr/local/bin/sfdx plugins --core
@oclif/plugin-commands 1.2.2 (core)
@oclif/plugin-help 2.1.6 (core)
@oclif/plugin-not-found 1.2.2 (core)
@oclif/plugin-plugins 1.7.8 (core)
@oclif/plugin-update 1.3.9 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.0.2 (core)
analytics 1.1.2 (core)
generator 1.1.0 (core)
salesforcedx 45.8.2 (core)
├─ force-language-services 45.9.0 (core)
└─ salesforce-alm 45.11.0 (core)

sfdx-cli 7.1.4 (core)

MacOS 10.14.3

@mattparlane I have not seen this issue before and we have large orgs on SFDX. Would you mind posting your question to the Trailblazer DX Community (https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F93A000000HTp1) There are lots of users who may be able to help? If you still experience the problem, we can reopen this issue

MacBook Pro (15-inch, 2018)
Mem 16 GB 2400 MHz DDR4
Getting this error
My org is probably huge, but first need to get it to measure


move: source service tracks recent release branches

Description
I'm finding that framework/* branches diverge from on-chain bytecode when the on-chain code upgrades, but we haven't synced to framework/* branches yet. Specifically we have hourly monitors for framework/testnet and framework/mainnet for syncing, meaning there's a potential time gap when it's not up to date. If there is a divergence, (a) the service won't start currently, (b) monitoring on-chain changes means that syncing from repositories won't succeed for some time gap (which could be up to an hour based on our monitors).
Because of this I want to change the default config to track and monitor latest testnet and mainnet for framework code. It should effectively addresses (a) and (b) because on-chain changes ought to be up to date with testnet and mainnet branches immediately.
Test Plan
Ran locally.

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[ ] protocol change
[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes

... yeah you're right this causes issues @amnn.
The real issue is I thought I could bank on mainnet having the latest framework code if we detect an on-chain upgrade (i.e., if the server is up and we're monitoring changes, we only check in one direction: if on-chain change -> pull branch). The problem is that a branch like mainnet can run ahead (with cherry-picks) before we cut a release.
I'm still not satisfied with using framework/* due to the periodic syncing. Intuitively, if (and as soon as) an on-chain upgrade succeeds for framework packages, then it must be the case that the source of truth for those packages are immediately available somewhere. It looks like the only reliable way to find those sources is by looking at tagged releases. So now my plan is to look up our latest releases for each network upon detecting an on-chain upgrade to framework code. Then sort releases by semantic version, and seek to revalidate to the most recent release per network.

thanks @amnn, yes I think we'll figure out a way to kick the server, until we have an automated way to resolve packages. Using GH releases is probably not the worst idea right now, but needs plumbing.


Adding node errors on Taiko Docs page

Describe the feature request
Many errors occur when running Prove/Proposer on Taiko's Testnet. The following link has various error indications, but there are errors that occur frequently in Discord, including mine

when run a node by mistake, the error display is "error="initialize rpc clients error" repeatedly.
This is an error caused by incorrect L2 or L3 RPC.
Describe alternatives you've considered
It is preferable to add specific examples of errors to the Docs.
A description of how to fix the error is as follows:
"The RPC in your config is wrong.
You have to set the ENDPOINT correctly.
Please refer to the following page to set the correct RPC
https://taiko.xyz/docs/guides/run-a-node/run-a-taiko-node"
Additional context
Additional context here.

No problem

Closed by https://github.com/taikoxyz/taiko-mono/pull/14231 , https://github.com/taikoxyz/taiko-mono/pull/14316 , and will keep updating this page.


[Form Attachments] AJAX removing of attachment record

[Form Attachments] AJAX removing of attachment record.
Fixes https://github.com/bitzesty/qae/issues/358

need to fix for Organisational Charts too


[Bug]: getPageResolver generating incorrect paths on Windows

Steps to reproduce
When using the latest RC version of the plugin all SSR pages in Next.js are failing to find the styled-components dependency imported in _document.ts.
This is happening with and without the esbuild function bundle option. The repo is a private repo so I cannot share this currently, I can see about trying to put together a reproduction repo but that will take some time.
A link to a reproduction repository
No response
Plugin version
^4.0.0-rc.1
More information about your build

[X] I am building using the CLI
[X] I am building using file-based configuration (netlify.toml)

What OS are you using?
Windows
Your netlify.toml file

`netlify.toml`
[build]
command = "yarn build"
publish = ".next"

[functions]
node_bundler = "esbuild"

[[plugins]]
package = "@netlify/plugin-nextjs"

Relevant log output (or link to your logs)

Build logs
1:45:45 PM: f3de4f18 ERROR  Error: Cannot find module 'styled-components'
Require stack:
- /var/task/packages/web/.next/server/pages/_document.js
- /var/task/node_modules/next/dist/server/require.js
- /var/task/node_modules/next/dist/server/load-components.js
- /var/task/node_modules/next/dist/server/next-server.js
- /var/task/packages/web/.netlify/functions-internal/___netlify-odb-handler/handlerUtils.js
- /var/task/packages/web/.netlify/functions-internal/___netlify-odb-handler/___netlify-odb-handler.js
- /var/task/___netlify-odb-handler.js
- /var/runtime/UserFunction.js
- /var/runtime/index.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:93:18)
    at Object.7518 (/var/task/packages/web/.next/server/pages/_document.js:115:18)
    at __webpack_require__ (/var/task/packages/web/.next/server/webpack-runtime.js:25:42)
    at Object.8684 (/var/task/packages/web/.next/server/pages/_document.js:16:75)
    at __webpack_require__ (/var/task/packages/web/.next/server/webpack-runtime.js:25:42)
    at __webpack_exec__ (/var/task/packages/web/.next/server/pages/_document.js:125:39)
    at /var/task/packages/web/.next/server/pages/_document.js:126:70 {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/var/task/packages/web/.next/server/pages/_document.js',
    '/var/task/node_modules/next/dist/server/require.js',
    '/var/task/node_modules/next/dist/server/load-components.js',
    '/var/task/node_modules/next/dist/server/next-server.js',
    '/var/task/packages/web/.netlify/functions-internal/___netlify-odb-handler/handlerUtils.js',
    '/var/task/packages/web/.netlify/functions-internal/___netlify-odb-handler/___netlify-odb-handler.js',
    '/var/task/___netlify-odb-handler.js',
    '/var/runtime/UserFunction.js',
    '/var/runtime/index.js'
  ]
}

Hey. Thanks for the report. I think I am going to need a repro from you, because I've failed to reproduce it myself. See #943 and the deploy preview

I'm thinking it might be due to it being in a monorepo. It is a Sanity project so we have the following structure:
package.json
packages/
  studio/
    package.json
  web/
    package.json

Perhaps the Netlify function isn't able to bundle the packages from the monorepo properly? I should have some time free tomorrow to try and investigate and put together a reproduction. 🤞

Thank you. It shouldn't be a problem with it being a monorepo, as we have several monorepo test sites, but it may still be related. A repro would be useful. It would also be helpful if you could run netlify build locally, and share the content of .netlify/functions-internal/___netlify-odb-handler/pages.js

The pages.js file from running the netlify build locally. Not sure if this is what is generated on the server though, can't see how it could possible resolve from here if it was.
// This file is purely to allow nft to know about these pages. It should be temporary.
exports.resolvePages = () => {
    try {
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/404.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/500.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/[slug].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/_app.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/_document.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/_error.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/formium/[formid].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/indexers/products-full.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/indexers/products-webhook.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/indexers/site-full.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/indexers/site-webhook.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/media.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/api/vacancies.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/applications/[application].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/applications/[application]/[subapplication].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/index.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/media.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/media/[slug].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/products.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/products/[category].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/products/[category]/[subcategory].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/products/[category]/[subcategory]/[product].js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/search-results.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/sitemap.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/sitemap.xml.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/vacancies.js')
        require.resolve('../../../C:\Projects\ClientName\packages\web\.next/server/pages/vacancies/[slug].js')
    } catch {}
}

We're using Azure DevOps for our CI process and decided to try running the same build/release process on a Unix agent and it appears to have resolved the issue for us. There must be something going awry on Windows agents specifically.

Oh yes that does look like it! Thanks for the report. I will rename this issue and prioritise a fix.

Can you confirm which version of Node is being used in the build? It would be useful if you could share the build logs too. We run integration tests on Windows which include snapshots of the pages.js file, so it does seem to generate the correct ones there at least.

Sorry just checked back in on this, the node version would have been 14.16.0. To be honest it should have been updated to use the latest LTS but that was how the release was setup when we last ran it on the Windows agent.
I can share the log directly if you're able to provide a contact email or similar. Would prefer not to post the entire log on here just in case.

I have a probable solution. I'll see if I can get that in a release today.

This should be fixed in rc.2

@ascorbic I've updated to rc.2 and ran the release process on the Windows agent and we're getting the entire site timing out with this error now.
{"errorMessage":"2021-12-10T14:37:39.530Z 43f7001e-3228-4aa8-ad43-fa9a3d2abb6d Task timed out after 10.01 seconds"}

@rshackleton oh dear. Can you share the pages.js file again? Does it still work with the linux agent? I have been thinking about it, and realise that there are svereal things that could cause problems deploying from a Windows agent, because it's building functions that are deployed to Lambda functions running on Linux. I would liek to get builds working properly on Windows, but this is somehting to bear in mind.

We're ok using the hosted Ubuntu agents for now, and we're looking at setting up our own dedicated Ubuntu agent to run alongside our Windows agents so that should be fine for us in the long run.
I'll try and get the logs to you but we're currently trying to fix the stack overflow issue that has come up with installing the latest Netlify CLI. 😫

Would that be this issue?

Yep, we were using the CLI via a global install so we're updating the CI to use it via a dev dependency. That is fixed now and I think the timeout issue might actually be due to me enabling ISR earlier on - I wasn't able to properly check that was working properly because of the release issues. 🤦♂
It has been a bit of a day. 😅

Ok, so we're back up and running. Tested the build on rc.2 on both Ubuntu and Windows agent and both are working great now, no more dependency issue even on the Windows agent!

Oh wow, that's great news! Thanks so much for your help in tracking this down.

No worries, cheers for getting a fix out so promptly!


Italian Hugging Face ASR Models need updates

Describe the bug
Hi, amazing work so far with SpeechBrain!
I am implementing an ASR model focused on the Italian language using SpeechBrain. My first step was to try the pretrained models available on Hugging Face.

speechbrain/asr-whisper-medium-commonvoice-it does not work out of the box when following the README instructions. I managed to make it work by modifying the hyperparams.yaml file as follows:

decoder: !new:speechbrain.decoders.seq2seq.S2SWhisperGreedySearcher
    model: !ref <whisper>
    # bos_index: !ref <timestamp_index>
    # eos_index: !ref <eos_index>
    min_decode_ratio: !ref <min_decode_ratio>
    max_decode_ratio: !ref <max_decode_ratio>

speechbrain/asr-transducer-commonvoice-14-it is not working on CUDA but performs well on Colab. When using the CUDA option, I encounter the following error:

Traceback (most recent call last):
  File "test_script.py", line 63, in transcribe_audio_files
    sentence = asr_model.transcribe_file(audio_path)
  File "path_to_environment/lib/python3.11/site-packages/speechbrain/inference/ASR.py", line 93, in transcribe_file
    predicted_words, predicted_tokens = self.transcribe_batch(
  File "path_to_environment/lib/python3.11/site-packages/speechbrain/inference/ASR.py", line 162, in transcribe_batch
    predicted_tokens, _, _, _ = self.mods.decoder(*inputs)
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "path_to_environment/lib/python3.11/site-packages/speechbrain/decoders/transducer.py", line 149, in forward
    hyps = self.searcher(tn_output)
  File "path_to_environment/lib/python3.11/site-packages/speechbrain/decoders/transducer.py", line 381, in transducer_beam_search_decode
    out_PN, hidden = self._forward_PN(
  File "path_to_environment/lib/python3.11/site-packages/speechbrain/decoders/transducer.py", line 590, in _forward_PN
    out_PN = layer(out_PN)
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "path_to_environment/lib/python3.11/site-packages/speechbrain/nnet/embedding.py", line 118, in forward
    return self.Embedding(x.long())
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/modules/sparse.py", line 190, in forward
    return F.embedding(
  File "path_to_environment/lib/python3.11/site-packages/torch/nn/functional.py", line 2551, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

speechbrain/asr-crdnn-commonvoice-it and speechbrain/asr-crdnn-commonvoice-14-it are working quite well actually. They do not fully complete the transcription but at least provide partial text output.

speechbrain/asr-wav2vec2-commonvoice-it does not perform well on the sample I provided (details in the Colab).

speechbrain/asr-wav2vec2-commonvoice-14-it performs great both on Colab and on CUDA.

Expected behaviour
Working seameless with the pretrained models provided into SpeechBrain HF repos.
To Reproduce
https://colab.research.google.com/drive/1xw7p0QD1Lg6pWTmNwVTERs02Yzo1vxgs?usp=sharing
Environment Details
speechbrain v1.0.2 (local)
Relevant Log Output
No response
Additional Context
If I'm missing something, could you guys point me out to the right docs to follow? Thank you in advance.

speechbrain/asr-transducer-commonvoice-14-it is not working on CUDA but performs well on Colab. When using the CUDA option, I encounter the following error:

Hey, how are you passing the cuda device? Can I see the code? thanks :)

Hi!
yes sure, here is the snippet:
    model_info = MODELS[args.model]
    asr_model = model_info["class"].from_hparams(
        source=model_info["source"],
        savedir=f"pretrained_models/{args.model}",
        run_opts={"device": "cuda"}
    )

The model_info["class"] is EncoderDecoderASR for the transducer. I’m using the same script to test the models.
Thank you for the support!

@Adel-Moumen any news on that?


feat(core): Ability to set global fetch policy & implement ssrForceFetchDelay

Fixes #55 and #14

Hi, thanks a lot for your contribution!
The code looks good to me. Have you tested this version in your project to see if it fixes your 2 issues?
I can get it merged and release a new version after that!

Due to the inactivity, I will be continuing this on #60.
Thanks!

Released in 0.17.0


Comment in yaml-codeblock won't generate html

What version of Hugo are you using (hugo version)?

$ hugo version
hugo v0.139.4+extended linux/amd64 BuildDate=unknown

Does this issue reproduce with the latest release?
Yes, it does.
When using the string:
# more publishable markdown files...

in a yaml-codeblock, like this
      - name: Copy over new files
        run: |
            # Whitelist of all publishable wiki articles
            cp index.md $content
            # more publishable markdown files...

instead of the block an e> gets generated. When removing the comment, everything works fine. I also tried different strings with which, it sometimes worked and sometimes not. I cannot guess pattern, when the error occurs.
Also there is no special output when building the site. I guess there is something wrong with goldmark but I do not fully understand how hugo uses goldmark. So I'm approaching you guys at first.
Thanks for your time. I would be happy to provide more information on this issue.

in a yaml-codeblock, like this

What exactly do you mean by this.

I mean it like this:
Some text explaining the yaml
```yaml
      - name: Copy over new files
        run: |
            # Whitelist of all publishable wiki articles
            cp index.md $content
            # more publishable markdown files...
```

When building this, I just get

Some text explaining the yaml

e>

I tried your example, and it renders fine for me (I tested with both markup.higlight.codeFences true and false).

I looked at the relevant page in your repository:
https://github.com/hmaier-dev/wiki/blob/main/content/Github.md
The first line of the file is:
# How this wiki works

So, despite the .md extension, Hugo treats this as Emacs Org Mode content:
https://github.com/gohugoio/hugo/blob/a32c889a7b1b6cb7b9a80b4fbab342d9b660a988/parser/pageparser/pagelexer_intro.go#L31-L32
And in Emacs Org Mode, the manual summary divider is... # more.

We might want to change this:
https://github.com/gohugoio/hugo/blob/a32c889a7b1b6cb7b9a80b4fbab342d9b660a988/parser/pageparser/pagelexer_intro.go#L31-L32
To this:
case (r == '#' && l.peek() == '+'):
	return lexFrontMatterOrgMode


[FEATURE]: Ability to specify selected fields dynamically

Describe what you want
I have a model on my project that fetching users from the database. Each time I need different set of fields and I would like to define a function with generics so only the exact fields I asked for will be return:
const findByEmail = async <P>(email: string, projection: P) => {
  const rows = await db
    .select(projection)
    .from(users)
    .where(eq(users.email, email))
    .limit(1);

  if (!rows.length) {
    return;
  }

  return rows[0];
}

And then I can ask only for the fields I want:
const userWithIdOnly = await findByEmail(email, { id: users.id });
      // ^ { id: number }

const userWithName =  = await findByEmail(email, { id: users.id, name: users.name });
      // ^ { id: number; name: string }

I encounter 2 problems when trying to implement this:

There is no type I can use to extend P. I'm looking for something like P extends SelectedFields<typeof users>
The current implementation breaks and produce invalid types in response. It seems that Drizzle expecting only static projection types and can't handle generic properly.

Any help will be appreciated.

The selection passed to select is too wide, you could select whatever yo want from the database. like count, max, all kinds of functions, etc.
I guess you just need to manually type the generic parameter and the help the compiler by type casting.
Check this example implementation.

The trick is actually the "partial" generic parameter. That needs to exists somewhere for the inference to work. That's why type casting is needed. I think we can close this issue and if you think we could expose a type helper for something like this, please create a new one with your idea. I even went ahead and extended my last example to accept extra properties with proper types. See here

Hi,
Does not seem to work with MySql.
I tried to convert the provided example implementation but it seems to be missing a parameter for MySqlSelectBase.
Would you mind providing the missing parameter. I tried to follow the never ending suite of extends but could not get it.
Thanks in advance.
example implementation converted to MySql

The types changed a little bit in the last release. You actually need to use 'single' instead of 'partial' and mysql has an extra type parameter. See here

Dynamic selection is "impossible" once you try to do joins. I have tried for hours, even with type casting.

@mjbergman92 , found a way. See https://github.com/drizzle-team/drizzle-orm/issues/3560#issuecomment-2478643565


The function named setExogenuosInputs should be spelled setExogenousInputs

Related to #29

Resolved in 0.5.0


Fix typos

This patch will do nothing; just few cosmetic changes.

I've never noticed. Thank you for your fix.


drivers: pwm: mspm0 add driver

This pr introduces a new device driver to use pwm functionality on some of the available TIMA/TIMG. In the dts file, we can define multiple properties such as:

pinctrl
clock devide ratio
prescaler (if supported)
pwm mode (check docs)
capture compare channel index and channel direction

and more.
The period has to be configured from the application part, as we are leveraging Zephyr's pwm driver API. Also, several of the timers support multiple channels, but are configured to be in different modes that would make things more complicated.

ping!

@Emil-Juhl feel free to take an extra look here. I dropped several dts properties and simplified the logic


Add decodeString piblic API

closes #88

I shall add tests

@davidyuk I've added an additional method, however I'm not really happy with the API now, so any suggestions how to minimize/simply it are very welcome
@marc0olo ping for ideas :)

I'm merging this as I want to revamp the public API, suggestion are still welcome


UI: SBOM table sorting

Needs https://github.com/patternfly-yew/patternfly-yew/pull/72

@ctron @lulf The time to ask for some help/hints has come for me.
I'm trying to add sorting to the SBOM/Packages table and although I know that the sorting query should be something like q=sort=field_name I was unable to come up with a way to connect the on_sort_by callback written in this Draft PR and the actual code where the http request is generated https://github.com/trustification/trustification/blob/main/spog/ui/src/components/sbom/search.rs#L54-L66
May I ask you for some hints please? Any kind of suggestion/hint/idea is very much appreciated.

The actual query string is generated here: https://github.com/trustification/trustification/blob/8ba64fff4f1dd95d52ecb7b449811c91362d62d4/spog/ui/src/components/sbom/search.rs#L52
I think what might work is to amend this struct to contain a list of fields to sort by. We might need a bit of mapping there, as not all combinations of field and sort order are supported. But having this information as part of the struct would allow us to add the list of sort fields to the final query string:
foo in:package

To
(foo in:package) sort:fieldA -sort:fieldB

Where the - prefix means (descending order).

@lulf :point_up:

@carlosthe19916 Do you have the 'raw' queries that gets sent to the backend? The sorting tests pass so I'm just curious if there could be something special about the query.

@carlosthe19916 Thanks, I've pinpointed the issue (related to empty queries). Will submit a PR with a fix shortly.

@lulf your PR https://github.com/trustification/trustification/pull/479 fixed the issue with the "SBOM table".
I can still see an error when I try to sort the table "Advisories" by "severity", is this something to address in another PR? Or perhaps we should not sort by "severity" at all?

Query used:
http://localhost:8083/api/v1/advisory/search?q=sort:severity&limit=10&offset=0&metadata=true&summaries=true

Screencast from 2023-09-05 10-54-21.webm

Hi @ctron , the comments were addressed. One of the builds failed Red Hat Trusted App Pipeline but I don't think it has something to do with the code. Let me know if there are more changes I need to do in this PR. Happy to address any issue :)


Remove version: WinSCP.WinSCP version 6.3.6

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

Close with reason: Duplicate;


Fix kerberos authentication

Overview
This PR fixes TrinoKerberosCredentials.trino_auth() in connections.py to include config, principal, and keytab necessary for kerberos authentication (tested with actual credentials).
Checklist

[x] README.md updated and added information about my change
[x] CHANGELOG.md updated and added information about my change

I squashed the commits into one. Not sure why the checks were failing though. Please let me know if there is any additional changes I need to make.

Thanks, there are some style tweaks which need to be made, see:
flake8 runtests: commands[0] | /bin/bash -c '$(which flake8) --select=E,W,F --ignore=W504 dbt/'
dbt/adapters/trino/connections.py:193:19: E251 unexpected spaces around keyword / parameter equals
dbt/adapters/trino/connections.py:193:21: E251 unexpected spaces around keyword / parameter equals
dbt/adapters/trino/connections.py:194:25: E251 unexpected spaces around keyword / parameter equals
dbt/adapters/trino/connections.py:194:27: E251 unexpected spaces around keyword / parameter equals
dbt/adapters/trino/connections.py:195:22: E251 unexpected spaces around keyword / parameter equals
dbt/adapters/trino/connections.py:195:24: E251 unexpected spaces around keyword / parameter equals
dbt/adapters/trino/connections.py:196:23: E251 unexpected spaces around keyword / parameter equals

Please squash commits and we will be good to merge.

Merged, thank you for your contribution!


Detach event does not fire if device was connected when module was required

Bug Description:
The first detach event doesn't fire if the device was already connected when the process started. This doesn't reproduce if I unplug the device before starting the process and then plug it in afterwards.

Steps to Reproduce:

Plug in a USB device
Start a Node process that adds detach/attach listeners
Unplug the device, detach event should fire but does not

Additional Information

Operating System: macOS Monterey
Runtime (node -v or electron -v): Node 16.16
Node USB Version (npm ls usb): 2.5.2 (also tried 2.4.3)

I've been unable to reproduce this with the latest release, both with devices attached before and after running node on macOS.
I used this, do you have a similar reproducer?
const usb = require('usb').usb;

usb.on('attach', device => console.log('attach', device));
usb.on('detach', device => console.log('detach', device));

setTimeout(process.exit, 10000);

Ah, sorry, I forgot to mention I'm using the WebUSB interface not the legacy one:
const WebUSB = require('usb').WebUSB;

const webUSB = new WebUSB({ allowAllDevices: true });

webUSB.addEventListener('connect', device => console.log('connect', device));
webUSB.addEventListener('disconnect', device => console.log('disconnect', device));

I tried the snippet you provided and that works as I'd expect, so this seems to be a WebUSB interface issue specifically.

Though even with the usb.on event, it seems not to fire if the device is open and actively being used (open/selectConfiguration/claimInterface) when disconnected. My use case is that I'm trying to cleanup references to the device and mark it disconnected when the user unplugs it.

Due to the webusb device requiring more data to be held than is available once the device is disconnected, the backing "store" of devices for it will need to be initiliased as your app starts.
I don't want to do this automatically as this would bring in unnecessary device listing of all potential devices.
Even doing it as the webusb clas is constructed would introduce an async constructor :(
Try listing the websub devices first which will initialise the list of known devices:
const WebUSB = require('./').WebUSB;

const webUSB = new WebUSB({ allowAllDevices: true });

(async () => {
    await webUSB.getDevices();
    webUSB.addEventListener('connect', device => console.log('connect', device));
    webUSB.addEventListener('disconnect', device => console.log('disconnect', device));

    setTimeout(process.exit, 10000);
})();

This may need to be documented or exposed as an 'initialise' call for webusb...

Closing this issue due to inactivity, please create a new issue as required.


Add compatibility with zsh

The bash read command previously used in complete and framework/complete is a bash built-in command which has different implementation with zsh's read. This PR changes the logic to use array instead to maintain compatibility between bash and zsh.
Also, this PR fixes the possibility of infinitely chaining help virtual command. This will happen when one of the subcommands is help. You can try with bcl help, press tab, and chose help again, ad infinitum. e.g.:
$ ./pkictl help help etcd help [TAB]
deploy  help   <-- suggestions

Thanks!


Unable to reinstall the operator fixes #43

@nicolaferraro as you suggested, ignore service recreation

Thanks!


Forms - add

Add select multiple for forms, add styling for mobile and up. http://cdpn.io/Frwuk

@bensaufley's solution worked for me in Foundation 6. Surprised this is a still an issue!


[5.0] Composer update throws error

So this commit to laravel/laravel seems to be throwing the following error when you run composer update:
  [Composer\DependencyResolver\SolverProblemsException]
    Problem 1
      - The requested package laravel/framework could not be found in any ver
  sion, there may be a typo in the package name.
  Potential causes:
   - A typo in the package name
   - The package is not available in a stable-enough version according to you
  r minimum-stability setting
     see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussi
  on> for more details.
  Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further c
  ommon problems.

Running composer self-update and then running composer update fixes this.


Any tips for training on my own dataset / fixing scale from COLMAP?

Hi there,
First of all, thanks for releasing the code for this impressive method!
I have been trying to train this on my own dataset (it's front-facing).
After a bit of experimentation, I can get ok-ish results. To do this, I extracted the poses from COLMAP then used imgs2poses.py in the LLFF repo to obtain the poses_bounds.npy for the 16 cameras in my dataset. Then, I created a .mp4 video for each camera to get the dataset into an identical format as per neural_3d in your repo.
From there, the only change I needed to make to get training to work was to expand the grid_bounds in neural_3d_z_plane.yaml from:
# Grid bounds
aabb: [[-1.0, -1.0, -1.0], [1.0, 1.0, 1.0]]

to:
# Grid bounds
aabb: [[-5.0, -5.0, -5.0], [5.0, 5.0, 5.0]]

Then I get OK results.
So it seems like COLMAP estimated my scene to have a somewhat larger scale than in the neural_3d scenes.
I am wondering if based on this, you had any ideas as to how I can improve performance to be closer to the level of neural_3d? I just have the feeling the code is optimised for a (-1,1) aabb box. But digging into the code, COLMAP, etc., it's not super obvious how I can rescale my scene, or what I should change in hyperreel to make it better suit my scene.
Just thought I would share this result, and yeah if you have any tips/ideas I'd really appreciate it! I'll also share any progress I manage to make on my own.

Hi, I think you may have submitted this issue to the wrong repo?

Yes, my bad haha!


Fix smalls remove percy and call svgo from npx

Build log
Preview build with this PR (b8f4bdf61f079b9c1fc88bb4376bc0790bf94067)

more info:cloud: Storage Free space   5.41 GiB    36.05%:open_file_folder: 0 B    CI_WORK/GitHubActions/roki-web


fix crash on ws comunication

Purpose
EVM PR https://github.com/Taraxa-project/taraxa-evm/pull/135
How does the solution address the problem

Changes made

prnet cleaned-up successfully