Sync XLIFF downloaded from Crowdin UI in browser (clean)

By project and file ID stored in XLIFF attributes (just like for PO it implemented earlier)

Might be better to continue here so just duplicating https://github.com/vslavik/poedit/pull/644#issuecomment-657604181 I posed just few minutes after you already moved to here :)

exportAsXliff: true API param for native XLIFF as well

This part is… complicated.
...

for the broken v2 API that downloads corrupted files

Sorry, but I am not sure what do you mean? Which exactly "v2 API" are "broken" and which files are downloaded "corrupted"

As a permanent solution, though, this shouldn't be done. All the arguments for downloading files in their original form that apply to POs also apply to XLIFFs. Crowdin's XLIFF-converted-to-XLIFF looses information. It's an imperfect representation of the original. It is a fallback solution to editing the real thing.

I am fully consent with you as it should be ideally but as of my today discussion with Crowdin team it's not going to changed in the nearest future. So as of now we have only two options as for exactly that particular case, i.e. if the original of the file downloaded exactly in a browser is exactly XLIFF

Just to disable Sync for it
Sync it with exportAsXliff: true to get it syncable at least this way rather than none

I would prefer 2 unless you are certainly for 1 or propose some better solution

For example, this Xcode-made XLIFF file differs in significant ways
...
Fully got your concern here so fully agree and admit that in case of file opened via Open from Crowdin initially or from local cache later we should always use exportAsXliff: false to handle it natively as is without any changes to it

And BTW, while deeping inside issues attempted to be addressed in this/#644 PR - discovered issue introduced with your refactoring of my #633 in your https://github.com/vslavik/poedit/commit/ea1a6fddc1ba088b12687d212823a7f92198f6bf#diff-a1865a3025926eb82afbd7f4d9f13518R629 which could be discussed in separate PR/issue of course but so far as I see fix of it is closely related to this issue so might be the best place to discuss it here as well.
So let's start from steps to reproduce:

Run Poedit with WXTRACE=poedit.crowdin env var
Open from Crowdin some natively XLIFF file (i.e. original/source of which is XLIFF)
Do either Sync or Save (Save makes sense in any branch in which it still works before broken in #642)
Look into DownloadFile(project_id of trace output
Result: file_extension= there is the name of the file instead of xliff extension which leads to isXLIFFConverted is false which leads to "exportAsXliff: false what is definitely not what we want accordingly to above and https://github.com/vslavik/poedit/pull/644#issuecomment-657604181 discussion

Sorry, but I am not sure what do you mean? Which exactly "v2 API" are "broken" and which files are downloaded "corrupted"

If you "Open from Crowdin" XLIFF file that is natively uploaded as XLIFF (as the example in the sandbox project), without exportAsXliff, it won't download the actual original, but a corrupted version with translations replaced by some weird whitespace. This is "a known bug" according to Crowdin that is "already fixed in testing".
Please don't tell me they changed their mind on fixing it :(

Thanks; squashed first three commits into 983a74dc.

And BTW, while deeping inside issues attempted to be addressed in this/#644 PR - discovered issue introduced with your refactoring of my #633 in your ea1a6fd#diff-a1865a3025926eb82afbd7f4d9f13518R629

Sorry about that. I'm getting the uneasy feeling that this code is crazy fragile — I as much as look at it funny and I break something :(
I don't think extracting data from XLIFF nodes is appropriate fix - it's just another patch over a patch over a patch. The real cause is, in my view, the assumption in CrowdinSyncFile that files must have the double extension; when I removed the .xliff.xliff double-extension, I did it unaware that some other innocuous looking code depends on it:
if (filename.GetExt().CmpNoCase("po") != 0)  // if not PO
    filename.SetFullName(filename.GetName());  // set remote (Crowdin side) filename extension

I think the fundamental mistake is having file extension as DownloadFile (and UploadFile) argument - this is something server-side determined, so we shouldn't rely on it. Is there an API endpoint allowing as to get metadata for a specific file given projectId+fileId?

And BTW, while deeping inside issues attempted to be addressed in this/#644 PR - discovered issue introduced with your refactoring of my #633 in your ea1a6fd#diff-a1865a3025926eb82afbd7f4d9f13518R629

Sorry about that. I'm getting the uneasy feeling that this code is crazy fragile — I as much as look at it funny and I break something :(

Honestly speaking I got the same feeling everywhere in the Poedit code I needed to introduce Crowdin related changes except protocol (http_client) and API crowdi_client related code the only which was quite robustly abstracted from other Poedit code, Anyway it might be just not enough familiarity with your professional style/approaches/patterns/paradigms/semantics just like every engineer has his/her own (just like to one comes to somebody kitchen :) which I must to admit is much better standardized in Qt, GoLang and Rails projects in this terms).

I don't think extracting data from XLIFF nodes is appropriate fix - it's just another patch over a patch over a patch. The real cause is, in my view, the assumption in CrowdinSyncFile that files must have the double extension; when I removed the .xliff.xliff double-extension, I did it unaware that some other innocuous looking code depends on it:
if (filename.GetExt().CmpNoCase("po") != 0)  // if not PO
filename.SetFullName(filename.GetName());  // set remote (Crowdin side) filename extension

It's not much about exactly above but rather about conceptual logic/approach itself (which I already mentioned to you about this "doubled extension" when you asked about on some of reviews so far I remember but I fully understand you might miss it due of so much code reviewed for which thanks a lot to you anyway) but rather about that if currently there are no well established place to keep original file extension except as in filename - then how can we distinguish between original file extension is XLIFF (exportAsXliff: false) and original file with no extension (exportAsXliff: true)? I agree that not so much extension we need to know but just whether to download with exportAsXliff true or false which can be encoded in the name in similar way as project and file id (which actually needed only in case of files opened via Open from Crowdin or from local cache later). But this just a question of preference to approach/style which I chosen just from that point to make original file extension human readable to end user (if user just look on the name of opened file). So if you don't have better ideas here - then what is wrong with doubling extension for you? :).

I think the fundamental mistake is having file extension as DownloadFile (and UploadFile) argument - this is something server-side determined, so we shouldn't rely on it. Is there an API endpoint allowing as to get metadata for a specific file given projectId+fileId?

Here it is https://support.crowdin.com/api/v2/#operation/api.projects.files.post but I and Crowdin team always try to avoid any extra calls if possible to keep necessary data somewhere locally from Poedit / file name / file metadata side. Unless it adds too much complexity to the code what I don't think doubling .xliff extensions the case (but rather point of elegancy I guess but not a real problem/weakness/fragility on itself, at least IMO).

I don't think extracting data from XLIFF nodes is appropriate fix - it's just another patch over a patch over a patch.

What I wrote there is BS. I misread it as looking for the filename, not using it as an indicator meaning only "this is an convert XLIFF".  You're absolutely right about that, will do. Thanks!

@vslavik , just checking if I can help with anything (to answer I just missed or to do/implement) to speed up merging and closing this PR and to speed up release. e.g. https://github.com/vslavik/poedit/pull/647#issuecomment-657606313 related to https://github.com/vslavik/poedit/pull/647/commits/0795c5a6f6ffe3806ae288baa561ea845724aa22 seems unanswered/unresolved yet and if you answered with prefered solution (chosen among proposed or some another idea) - I could proceed with implementing it. Thanks for all your work and efforts!

In 9487af01 — extract correct filename/extension from XLIFF except that the logic slightly differs from what https://github.com/vslavik/poedit/pull/647#issuecomment-657618842 says in that if this can't be extracted, then file extension is trusted (it means file format is natively supported, therefore Poedit didn't change it).
In 4b682728 — applied the workaround for broken native XLIFF support, i.e. 0795c5a, rebased atop the above to make reverting in the future simpler; note that the crowdin_gui.cpp part is no longer needed thanks to 9487af01.

@vslavik , just checking if I can help with anything

I'd appreciate double checking if I didn't screw up anything this time.
Thanks!

In 9487af0 — extract correct filename/extension from XLIFF except that the logic slightly differs from what #647 (comment) says in that if this can't be extracted, then file extension is trusted (it means file format is natively supported, therefore Poedit didn't change it).
In 4b682728 — applied the workaround for broken native XLIFF support, i.e. 0795c5a, rebased atop the above to make reverting in the future simpler; note that the crowdin_gui.cpp part is no longer needed thanks to 9487af0.

Unfortunately it still needs to handle below special cases in case if original/source of downloaded file is XLIFF

As it comes from your https://github.com/vslavik/poedit/pull/644#issuecomment-657252958 starting from your "For example, this Xcode-made XLIFF file differs in significant ways"  and what I just tested on Crowdin to confirm - exportAsXliff: false should be used to download it "as is" just like in case with PO/POT. Since when I implemented closest logic to your in above commit for setting of exportAsXliff in https://github.com/vslavik/poedit/pull/644/commits/dc985930b95380cb40f410dfc0ef74fbb6aeb7d6 - I just did not expect above problem you highlighted. I.e. && ext != "XLIFF" && ext != "XLF" should be added back to here https://github.com/vslavik/poedit/pull/647/commits/0795c5a6f6ffe3806ae288baa561ea845724aa22#diff-ee71506cd9832717af50c2721cc35d85L428
While above addition proposed by me solves above problem so XLIFF is returned really  "as is" (i.e. without any changes or missing/corruption of anything) - it causes XLIFF to do not contain original attribute in that case as well as any other Crowdin specific attributes/nodes (project and file id etc.) and there are nothing to do with that in the nearest time as of discussion with Crowdin team. This breaks only below use case / workflow:

File downloaded from browser via Crowdin UI with "Download in XLIFF" (just recall we are talking only about the case when original/source file of downloaded file is XLIFF) which differs from simple Download in that Download in XLIFF is downloaded as already parsed and recreated by Crowdin from internal structure (i.e. XLIFF-to-XLIFF converted as you called it somewhere) while simple Download - downloads "as is" natively just just like in case with PO/POT with only difference that XLIFF unfortunately does not contain project and file IDs in this case as well as other Crowdin specific attributes
Opened in Poedit
Press Sync
File is successfully uploaded and then downloaded back with exportAsXliff: false  (since it extension is XLIFF  - taken from original attribute or would be same if taken just from local filename) but already "as is" without file and project IDs attributes as well as any other Crowdin specific attributes

Result: Sync button disappears from toolbar
Proposed fix/workaround:  Change this https://github.com/vslavik/poedit/commit/9487af01bb1347d25c9f756f99fc0e4b7ba2aca8#diff-a1865a3025926eb82afbd7f4d9f13518L823 to somethign like

if (!xliffRemoteFilename.empty())
{
    auto ext = filename.GetExt();
    ext.MakeLower();
    if(ext == "xliff" || ext == "xlf)
        filename.SetExt(""); // none or any but just not XLIFF
    else
        filename.Assign(xliffRemoteFilename, wxPATH_UNIX);
}

As it comes from your #644 (comment) starting from your "For example, this Xcode-made XLIFF file differs in significant ways" and what I just tested on Crowdin to confirm - exportAsXliff: false should be used to download it "as is" just like in case with PO/POT.

I think we're moving in circles.
Again:

Yes, I agree! It should behave that way. I want that.
Crowdin API has a bug currently. It doesn't work that way. Try it yourself: revert 4b68272, then open Poedit Testing Sandbox →  AppSandbox.xliff file from Crowdin. Poedit will tell you the file is empty. It shouldn't be.
Hence applying 4b68272 as a workaround for the bug until Crowdin deploys the fix.

While above addition proposed by me solves above problem so XLIFF is returned really "as is" (i.e. without any changes or missing/corruption of anything) - it causes XLIFF to do not contain original attribute in that case as well as any other Crowdin specific attributes/nodes (project and file id etc.) and there are nothing to do with that in the nearest time as of discussion with Crowdin team. This breaks only below use case / workflow:

Note that it does not break it as of now as 4b68272 remains applied.

Proposed fix/workaround:

A better fix is something I in fact started doing instead of 9487af01 before I realized the file extension is still important (this also prompted my question about getting this from API as you may recall): pass wasExportedAsXliff as another boolean argument to DownloadFile(). The logic here is complicated, so the extra verboseness and clarity would only help, and it would unambiguously say what needs to be done.

As it comes from your #644 (comment) starting from your "For example, this Xcode-made XLIFF file differs in significant ways" and what I just tested on Crowdin to confirm - exportAsXliff: false should be used to download it "as is" just like in case with PO/POT.

I think we're moving in circles.
Again:

Yes, I agree! It should behave that way. I want that.
Crowdin API has a bug currently. It doesn't work that way. Try it yourself: revert 4b68272, then open Poedit Testing Sandbox →  AppSandbox.xliff file from Crowdin. Poedit will tell you the file is empty. It shouldn't be.
Hence applying 4b68272 as a workaround for the bug until Crowdin deploys the fix.

No! It works perfectly now with my fix #648 (https://github.com/vslavik/poedit/pull/648/commits/9d68fd54d51b6017abd06bb170779d244ca2219f). See screenshots below or just try it yourself. So it looks like Crowdin fixed that problem you mentioned related to it. Of course if I did not miss something in your explanation / description / steps to reproduce or otherwise sorry if I may be just did not delve into described your problem deeper to reveal and confirm it now fixed!

Moving this topic over to #648. Let's please don't add any more comments on this PR unless they are about the commits applied directly.

A better fix is something I in fact started doing instead of 9487af0 before I realized the file extension is still important (this also prompted my question about getting this from API as you may recall): pass wasExportedAsXliff as another boolean argument to DownloadFile(). The logic here is complicated, so the extra verboseness and clarity would only help, and it would unambiguously say what needs to be done.

Done in 09b50afd — any thoughts?

A better fix is something I in fact started doing instead of 9487af0 before I realized the file extension is still important (this also prompted my question about getting this from API as you may recall): pass wasExportedAsXliff as another boolean argument to DownloadFile(). The logic here is complicated, so the extra verboseness and clarity would only help, and it would unambiguously say what needs to be done.

Done in 09b50af — any thoughts?

Seem everything as discussed and expected now. Thanks a lot!


Not working.

I get many errors while downloading the files..
downloading
d.lua
PANIC: unprotected error in call to Lua API (d.lua:20: unexpected symbol near 'ÿ')

After running it many, many times, I get all the files, but some of them are corrupted.
There might be a memory leak somewhere.
Perhaps you could simply upload the needed .lua files here?

You can download d.lua and i.lua, and init.lua from here:
https://github.com/NodeUSB/lua-examples/tree/master/codes
You also need to compile these files (d.lua, i.lua)

Good job. Worked well. Would be better if we could host the interface locally ;)

Get the errors above no matter what I do.
I'm using ESPlorer to upload the init.lua.
I've also tried download d.lua and i.lua and compiling them, and it still happens.
Mike

Everything worked fine for me. YOU ROCK! (I mean, siriously, you ROCK!)


Clarify PAT instructions in README

Got a bit confused between the fine-grained tokens vs the 'classic' tokens. Seems like GH is trying to encourage people to prefer / use the fine-grained ones, so thought maybe worth specifying that here.
I also added .DS_Store files to the .gitignore file.

Thanks!


Allow recovering from invalid call to set_rules_dir

This fixes issue #36 by clearing PreferenceManager.error when initialisation succeeds. Also a test to demonstrate the issue and prove the fix works.

Looks good. Thanks for the fix!


Use Let's Encrypt for getting a real certificate

I found this container a super easy way to get (and renew) free certificates using Let's Encrypt.
https://hub.docker.com/r/bradjonesllc/docker-nginx-letsencrypt/

Won't do. Using traditional CA instead.


Add test for QPDFObjectHandle::isDirectNull

Rather than merging these as they come, how about if you let me know when you're gone through and added the tests you're going to add? Since these are just tests, I don't have an issue with merging right before I prepare 11.9.0. If there are any non-obvious fixes, I might push it off a little.
Thank you very much for doing this.


Add support for entities folder

References #650
Changes proposed:

[x] Add

Is this icon OK? it shouldn't be red color like models?

@alfonmga If you think it is a bug, better open a new issue to address that 👍

It doesn't look like one of our folder icons.


Syntactic equality fails on top-level terms

There is something different in the encoding of top-level terms that prevents syntactic equality of terms from working correctly. I usually work around this by adding an assert_norm of the definition of the term (this appears in a few places in TLS, in DefineTable.fst and KDF.fst) but I wonder if there is a way to fix it? Itsaka actually ran into this issue trying to prove the CPS fib example of the tutorial.
let a = fun x -> x + 1
let b = fun x -> x + 1
[@expect_failure]
//assert_norm(a == (fun x -> x + 1) /\ b == (fun x -> x + 1))
let t () : Lemma (a == b) = ()

let _ =
  let a = fun x -> x + 1 in
  let b = fun x -> x + 1 in
  let t () : Lemma (a == b) = () in
  ()

This is somewhat by-design. By default top-level definitions are not substituted when queries are encoded to Z3. As a result we need assert_norm or some such normalization to prove equality of top-level a and b (in general assert_norm may be unnecessarily aggressive, so controlled normalization using Pervasives.reveal_opaque or tactics is better). Whereas inner let-bindings are substituted and so, in the encoding of inner lemma t, the postcondition is simply fun x -> x + 1 == fun x -> x + 1, which goes through.

Closing this issue for now, changing this behavior would require some changes to the smt encoding. Instead there is a workaround of using normalization. Please reopen if necessary, thanks!


[Archlinux] error : could not load plugin & error could not determine file type

On GitLab by @Pixead on Dec 21, 2020, 24:03

using the package zathura-pdf-mupdf-0.3.6-3
When opening a document, I get the two following errors :
error: Could not load plugin '/usr/lib/zathura/libpdf-mupdf.so' (/usr/lib/zathura/libpdf-mupdf.so: undefined symbol: gumbo_destroy_output).
error: Could not determine file type.
downgrading to the package  zathura-pdf-mupdf-0.3.6-2 fixes the issue

On GitLab by @yigitsever on Dec 21, 2020, 02:36

same here, switching to poppler backend (zathura-pdf-poppler, 0.3.0-1) fixes the issue as well

closed

Fixed in d0a99a26ab5f80334308607ba5ef9525af718f2a. Please report this issue to the Archlinux bug tracker.


Appium opens the flutter application and then hangs, says port is busy

I have this issue where appium opens my flutter application on a real ipad, and then hangs with the following log. It sucessfully downloads webdriver, only for it to say the port is busy.
debug] [XCUITestDriver@4640 (81100193)] WebDriverAgent information:
[debug] [XCUITestDriver@4640 (81100193)] {
[debug] [XCUITestDriver@4640 (81100193)]   "message": "WebDriverAgent is ready to accept commands",
[debug] [XCUITestDriver@4640 (81100193)]   "state": "success",
[debug] [XCUITestDriver@4640 (81100193)]   "os": {
[debug] [XCUITestDriver@4640 (81100193)]     "testmanagerdVersion": 28,
[debug] [XCUITestDriver@4640 (81100193)]     "name": "iPadOS",
[debug] [XCUITestDriver@4640 (81100193)]     "sdkVersion": "16.4",
[debug] [XCUITestDriver@4640 (81100193)]     "version": "16.6.1"
[debug] [XCUITestDriver@4640 (81100193)]   },
[debug] [XCUITestDriver@4640 (81100193)]   "ios": {
[debug] [XCUITestDriver@4640 (81100193)]     "ip": "<IP_ADDRESS>"
[debug] [XCUITestDriver@4640 (81100193)]   },
[debug] [XCUITestDriver@4640 (81100193)]   "ready": true,
[debug] [XCUITestDriver@4640 (81100193)]   "build": {
[debug] [XCUITestDriver@4640 (81100193)]     "upgradedAt": "1691701695821",
[debug] [XCUITestDriver@4640 (81100193)]     "time": "Oct 20 2023 14:17:06",
[debug] [XCUITestDriver@4640 (81100193)]     "productBundleIdentifier": "XXXXXXX.testing.WebDriverAgentRunner"
[debug] [XCUITestDriver@4640 (81100193)]   }
[debug] [XCUITestDriver@4640 (81100193)] }
[debug] [XCUITestDriver@4640 (81100193)] WebDriverAgent successfully started after 9468ms
[debug] [XCUITestDriver@4640 (81100193)] Event 'wdaSessionAttempted' logged at<PHONE_NUMBER>892 (15:10:14 GMT-0500 (Eastern Standard Time))
[debug] [XCUITestDriver@4640 (81100193)] Sending createSession command to WDA
[debug] [XCUITestDriver@4640 (81100193)] Matched '/session' to command name 'createSession'
[debug] [XCUITestDriver@4640 (81100193)] Proxying [POST /session] to [POST http://<IP_ADDRESS>:8100/session] with body: {"capabilities":{"firstMatch":[{"bundleId":"com.testing.patientHostApp","arguments":[],"environment":{},"eventloopIdleDelaySec":0,"shouldWaitForQuiescence":true,"shouldUseTestManagerForVisibilityDetection":false,"maxTypingFrequency":60,"shouldUseSingletonTestManager":true,"shouldTerminateApp":true,"forceAppLaunch":true,"useNativeCachingStrategy":true,"forceSimulatorSoftwareKeyboardPresence":true}],"alwaysMatch":{}}}
[debug] [XCUITestDriver@4640 (81100193)] Got response with status 200: {"value":{"sessionId":"B62F83A4-68F5-4C9C-9315-719B8B41E68B","capabilities":{"device":"ipad","browserName":"Patient Host App","sdkVersion":"16.6.1","CFBundleIdentifier":"com.testing.patientHostApp"}},"sessionId":"B62F83A4-68F5-4C9C-9315-719B8B41E68B"}
[XCUITestDriver@4640 (81100193)] Determined the downstream protocol as 'W3C'
[debug] [XCUITestDriver@4640 (81100193)] Event 'wdaSessionStarted' logged at<PHONE_NUMBER>131 (15:10:17 GMT-0500 (Eastern Standard Time))
[debug] [XCUITestDriver@4640 (81100193)] Event 'wdaStarted' logged at<PHONE_NUMBER>131 (15:10:17 GMT-0500 (Eastern Standard Time))
[debug] [BaseDriver] The value of 'elementResponseAttributes' setting did not change. Skipping the update for it
[debug] [BaseDriver] The value of 'shouldUseCompactResponses' setting did not change. Skipping the update for it
[FlutterDriver] Attempt #1
[FlutterDriver] Running on iOS real device
[FlutterDriver] Port forwarding to: 59491
[FlutterDriver] Connecting to Dart Observatory: ws://<IP_ADDRESS>:59491/p6YbQqzzpPs=/ws
[FlutterDriver] Connecting to ws://<IP_ADDRESS>:59491/p6YbQqzzpPs=/ws
[FlutterDriver] Listing all isolates: [{"type":"@Isolate","id":"isolates/3935125082630639","name":"main","number":"3935125082630639","isSystemIsolate":false,"isolateGroupId":"isolateGroups/5315755538372737"}]
[FlutterDriver] "ext.flutter.driver" is not found in "extensionRPCs" ["ext.ui.window.impellerEnabled","ext.dart.io.getHttpEnableTimelineLogging","ext.dart.io.setHttpEnableTimelineLogging","ext.dart.io.httpEnableTimelineLogging","ext.dart.io.getSocketProfile","ext.dart.io.startSocketProfiling","ext.dart.io.pauseSocketProfiling","ext.dart.io.socketProfilingEnabled","ext.dart.io.clearSocketProfile","ext.dart.io.getVersion","ext.dart.io.getHttpProfile","ext.dart.io.getHttpProfileRequest","ext.dart.io.clearHttpProfile","ext.flutter.exit","ext.flutter.connectedVmServiceUri","ext.flutter.activeDevToolsServerAddress","ext.flutter.timeDilation","ext.flutter.debugDumpRenderTree","ext.flutter.debugDumpSemanticsTreeInTraversalOrder","ext.flutter.debugDumpSemanticsTreeInInverseHitTestOrder","ext.flutter.profileRenderObjectPaints","ext.flutter.profileRenderObjectLayouts","ext.flutter.debugDumpApp","ext.flutter.debugDumpFocusTree","ext.flutter.showPerformanceOverlay","ext.flutter.didSendFirstFrameEvent","ext.flutter.didSendFirstFrameRasterizedEvent","ext.flutter.profileWidgetBuilds","ext.flutter.profileUserWidgetBuilds","ext.dart.io.getOpenFiles","ext.dart.io.getOpenFileById"]
[FlutterDriver] Waiting 3 seconds before trying...
[FlutterDriver] Attempt #2
[FlutterDriver] Running on iOS real device
[FlutterDriver] Port #59491 is busy. Did you quit the previous driver session(s) properly?
[FlutterDriver] Did not know how to release port #59491 in 15004ms
[debug] [FlutterDriver] Error: The port :59491 is occupied by an other process. You can either quit that process or select another free port.
[debug] [FlutterDriver]     at waitForPortIsAvailable (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:83:11)
[debug] [FlutterDriver]     at getObservatoryWsUri (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:111:3)
[debug] [FlutterDriver]     at connectSocket (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/observatory.ts:33:28)
[debug] [FlutterDriver]     at async Promise.all (index 1)
[debug] [FlutterDriver]     at FlutterDriver.createSession (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/session.ts:36:47)
[debug] [FlutterDriver]     at AppiumDriver.createSession (/opt/homebrew/lib/node_modules/appium/lib/appium.js:374:35)
[FlutterDriver] Connecting to Dart Observatory: undefined
[FlutterDriver] {}
[FlutterDriver] Check Dart Observatory URI no URI found in the device log
[FlutterDriver] Waiting 3 seconds before trying...
[FlutterDriver] Attempt #3
[FlutterDriver] Running on iOS real device
[FlutterDriver] Port #59491 is busy. Did you quit the previous driver session(s) properly?
[FlutterDriver] Did not know how to release port #59491 in 15002ms
[debug] [FlutterDriver] Error: The port :59491 is occupied by an other process. You can either quit that process or select another free port.
[debug] [FlutterDriver]     at waitForPortIsAvailable (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:83:11)
[debug] [FlutterDriver]     at getObservatoryWsUri (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:111:3)
[debug] [FlutterDriver]     at connectSocket (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/observatory.ts:33:28)
[debug] [FlutterDriver]     at async Promise.all (index 1)
[debug] [FlutterDriver]     at FlutterDriver.createSession (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/session.ts:36:47)
[debug] [FlutterDriver]     at AppiumDriver.createSession (/opt/homebrew/lib/node_modules/appium/lib/appium.js:374:35)
[FlutterDriver] Connecting to Dart Observatory: undefined
[FlutterDriver] {}
[FlutterDriver] Check Dart Observatory URI no URI found in the device log
[FlutterDriver] Waiting 3 seconds before trying...
[FlutterDriver] Attempt #4
[FlutterDriver] Running on iOS real device
[FlutterDriver] Port #59491 is busy. Did you quit the previous driver session(s) properly?
[FlutterDriver] Did not know how to release port #59491 in 15002ms
[debug] [FlutterDriver] Error: The port :59491 is occupied by an other process. You can either quit that process or select another free port.
[debug] [FlutterDriver]     at waitForPortIsAvailable (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:83:11)
[debug] [FlutterDriver]     at getObservatoryWsUri (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:111:3)
[debug] [FlutterDriver]     at connectSocket (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/observatory.ts:33:28)
[debug] [FlutterDriver]     at async Promise.all (index 1)
[debug] [FlutterDriver]     at FlutterDriver.createSession (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/session.ts:36:47)
[debug] [FlutterDriver]     at AppiumDriver.createSession (/opt/homebrew/lib/node_modules/appium/lib/appium.js:374:35)
[FlutterDriver] Connecting to Dart Observatory: undefined
[FlutterDriver] {}
[FlutterDriver] Check Dart Observatory URI no URI found in the device log
[FlutterDriver] Waiting 3 seconds before trying...
[FlutterDriver] Attempt #5
[FlutterDriver] Running on iOS real device
[FlutterDriver] Port #59491 is busy. Did you quit the previous driver session(s) properly?
[FlutterDriver] Did not know how to release port #59491 in 15002ms
[debug] [FlutterDriver] Error: The port :59491 is occupied by an other process. You can either quit that process or select another free port.
[debug] [FlutterDriver]     at waitForPortIsAvailable (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:83:11)
[debug] [FlutterDriver]     at getObservatoryWsUri (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/ios.ts:111:3)
[debug] [FlutterDriver]     at connectSocket (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/observatory.ts:33:28)
[debug] [FlutterDriver]     at async Promise.all (index 1)
[debug] [FlutterDriver]     at FlutterDriver.createSession (/Users/ani/.appium/node_modules/appium-flutter-driver/lib/sessions/session.ts:36:47)
[debug] [FlutterDriver]     at AppiumDriver.createSession (/opt/homebrew/lib/node_modules/appium/lib/appium.js:374:35)

Please attach the full appium log. I guess that was fixed in https://github.com/appium/appium-flutter-driver/blob/main/driver/CHANGELOG.md#223

Closing the ticket as no response.


下载句法解析时出错

Describe the bug
下载句法解析时出错，提示https://file.hankcs.com/hanlp/dep/biaffine_ctb7_20191229_130325.zip was created with hanlp-2.0.0, but you are running 2.0.0-alpha.12. T
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
import hanlp
syntactic_parser = hanlp.load(hanlp.pretrained.dep.CTB7_BIAFFINE_DEP_ZH)

Describe the current behavior
Downloading https://file.hankcs.com/hanlp/dep/biaffine_ctb7_20191229_130325.zip to /Users/wuhaixu/.hanlp/dep/biaffine_ctb7_20191229_130325.zip
100.00%, 65.8 MB/65.8 MB, 219 KB/s, ETA 0 s
Extracting /Users/wuhaixu/.hanlp/dep/biaffine_ctb7_20191229_130325.zip to /Users/wuhaixu/.hanlp/dep
Downloading https://github.com/SUDA-LA/CIP/archive/master.zip#BPNN/data/embed.txt to /Users/wuhaixu/.hanlp/thirdparty/github.com/SUDA-LA/CIP/archive/master.zip
100.00%, 0 KB/0 KB, 0 KB/s, ETA 0 s
Extracting /Users/wuhaixu/.hanlp/thirdparty/github.com/SUDA-LA/CIP/archive/master.zip to /Users/wuhaixu/.hanlp/thirdparty/github.com/SUDA-LA/CIP/archive
Failed to load https://file.hankcs.com/hanlp/dep/biaffine_ctb7_20191229_130325.zip. See stack trace below
Traceback (most recent call last):
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/utils/component_util.py", line 36, in load_from_meta_file
    obj.load(save_dir)
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/common/component.py", line 231, in load
    self.build(**merge_dict(self.config, training=False, logger=logger, **kwargs, overwrite=True, inplace=True))
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/common/component.py", line 242, in build
    loss=kwargs.get('loss', None)))
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/components/parsers/biaffine_parser.py", line 35, in build_model
    self.transform) if pretrained_embed else None
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/layers/embeddings/__init__.py", line 33, in build_embedding
    layer: tf.keras.layers.Embedding = tf.keras.utils.deserialize_keras_object(embeddings)
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 305, in deserialize_keras_object
    return cls.from_config(cls_config)
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 519, in from_config
    return cls(**config)
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/layers/embeddings/word2vec.py", line 119, in __init__
    word2vec, _output_dim = load_word2vec(filepath)
  File "/Users/wuhaixu/anaconda3/lib/python3.6/site-packages/hanlp/utils/io_util.py", line 410, in load_word2vec
    with open(realpath, encoding='utf-8', errors='ignore') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/wuhaixu/.hanlp/thirdparty/github.com/SUDA-LA/CIP/archive/master/BPNN/data/embed.txt'
https://file.hankcs.com/hanlp/dep/biaffine_ctb7_20191229_130325.zip was created with hanlp-2.0.0, but you are running 2.0.0-alpha.12. Try to upgrade hanlp with
pip install --upgrade hanlp

System information

Mac OS
Python 3.6.9
HanLP version: 2.0.0-alpha.12

[x] I've completed this form and searched the web for solutions.

感谢反馈，已经修复，请参考上面的commit。
如果还有问题，欢迎重开issue。


Can we do train_test_split() on SQuAD-like json file?

We might use directly sklearn train_test_split.
@andrelmfarias as discussed, to be done after annotation.

We actually don't need this because we are not fine-tuning the reader on our annotated dataset (we only annotate it for evaluation atm).
If later our dataset annotations size increases and justify a fine-tuning to gain in performance, we will do:
from sklearn.model_selection import train_test_split

train, test = train_test_split(json_data) # json_data is the annotated dataset

After applying this I we want to save as train.json and test.json then what should be do?

@fmikaelian @kusumlata123 @andrelmfarias
So I was also dealing with this kind of an issue - since the free tier of GPU-based instances might not be able to extract features of huge SQuaD format datasets. I tried to work around with something relevant to perform train_test_split and maintain SQuAD format  -

if you take a look at this method, there is a certain format how the split file needs to be before it can be used to extract features - https://github.com/huggingface/transformers/blob/main/src/transformers/data/processors/squad.py#L607

The format of the split data needs to be in dict type (so json format is a must) and also the context,qas needs to be nested in data key of the split json file (much similar to squad dataset samples)

My implementation -
import json
from sklearn.model_selection import train_test_split

# Load JSON file
with open('data.json', 'r') as file:
    json_data = json.load(file)

# Split the data into train and test subsets
train_data, test_data = train_test_split(json_data['data'], test_size=0.2, random_state=42)

# Create a new dictionary with train and test subsets
split_data = {
    'version': data['version'],
    'data': train_data
}

# Save the train and test subsets as separate JSON files
with open('train_reduced1.json', 'w') as train_file:
    json.dump(split_data, train_file)

Feel free to modify it at your convenience.
Cheers


config: Deny CARGO_HOME in [env] table (fixes #11590)

Fixes #11590

Thanks!
@bors r+

:pushpin: Commit 17a2bce2f10d5fdabbbf9aa637b2263f8da9ea85 has been approved by ehuss
It is now in the queue for this repository.

:hourglass: Testing commit 17a2bce2f10d5fdabbbf9aa637b2263f8da9ea85 with merge cf410cb33715614b4c4afe0b6ffa64717ed526d9...

:sunny: Test successful - checks-actions
Approved by: ehuss
Pushing cf410cb33715614b4c4afe0b6ffa64717ed526d9 to master...


fix(mse): module config works with AOT compiler

BREAKING CHANGE: module configuration API has changed to better work with the AOT compiler. Module now defaults with the setting capLevelToPlayerSize set to true.

:tada: This PR is included in version 11.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Support for Web Integration Testing in Flutter

Use case

Currently, Flutter's integration testing package does not fully support web platform testing.
I ran an integrated test to import and map the data from Firebase, but there was a problem
How about modifying the official document? The official document guides on the web integration test.
https://docs.flutter.dev/testing/integration-tests
Proposal
I would like to request that the Flutter team provides full support for integration testing on the web platform.
Specifically, the integration tests should be able to:

Run seamlessly in different web browsers such as Chrome, Firefox, and Edge.
Provide detailed feedback and logs when running tests in a web environment.
Support common web-specific scenarios, such as handling of different screen sizes and input types.

[✓] Flutter (Channel stable, 3.22.3, on macOS 14.1.1 23B81 darwin-arm64, locale en-KR)
    • Flutter version 3.22.3 on channel stable at /Users/i/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b0850beeb2 (4 weeks ago), 2024-07-16 21:43:41 -0700
    • Engine revision 235db911ba
    • Dart version 3.4.4
    • DevTools version 2.34.3

[!] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/i/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /usr/bin/java
    • Java version OpenJDK Runtime Environment AdoptOpenJDK-16.0.1+9 (build 16.0.1+9)
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15E204a
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (not installed)
    • Android Studio not found; download from https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).

[✓] VS Code (version 1.87.2)
    • VS Code at /Users/i/Downloads/Visual Studio Code.app/Contents
    • Flutter extension version 3.94.0

[✓] Connected device (3 available)
    • macOS (desktop)                 • macos                 • darwin-arm64   • macOS 14.1.1 23B81 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 14.1.1 23B81 darwin-arm64
    • Chrome (web)                    • chrome                • web-javascript • Google Chrome 127.0.6533.120
    ! Error: Browsing on the local area network for Jonghyuck’s iPhone. Ensure the device is unlocked and attached with a cable or associated with the same local area network as this
      Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)

[✓] Network resources
    • All expected network resources are available.

Closing this as a duplicate of https://github.com/flutter/flutter/issues/117955


fix: makes types and abi available to publish

Previously, /types and /ERC677Data.json have been added to the files property in package.json so that these are present in the package after it is installed in a project. This has been reflected in the documentation, but presumably mistakenly not removed from .gitignore, thus never pushed to the repo itself.
This PR changes that, whilst removing the type generation from the installation process, as these will be already present.
Resolves https://github.com/rsksmart/erc677/issues/10

Checks fail to npm i with:

Could not install from "node_modules/web3-providers-ws/websocket@github:web3-js/WebSocket-Node#polyfill/globalThis" as it does not contain a package.json file.

Checks fail to npm i with:

Could not install from "node_modules/web3-providers-ws/websocket@github:web3-js/WebSocket-Node#polyfill/globalThis" as it does not contain a package.json file.

Which points me out to: Why are we not doing npm ci in the CI @ilanolkies? (btw, I do know that that is not the issue, in case you're wondering)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 90.909%

Totals

Change from base Build 764899969:
0.0%

Covered Lines:
9

Relevant Lines:
9

💛  - Coveralls

I'm not entirely clear on why the slither scan is failing @ilanolkies, but maybe @luisfontes19 could shed some light on this.


External media libraries

Third party media integrations are here!! 🎉 🎉
I wrote this in about three days, so go easy 😅
This PR includes the new registerMediaLibrary API and our first integration: Uploadcare. If you haven't heard of them, go check 'em out! Integration docs are included as well.
We originally expected third party media library integrations to be API calls underneath our built in media library, but this PR takes a different route: allowing UI libraries created by services like Uploadcare and Cloudinary to replace our media library entirely. The goals of this PR were:

Get third party file services working fast.
Allow the integrations themselves to be tiny.
Allow projects to opt in by updating config, nothing more.

Read on for details, or try the live demo from this PR's deploy preview.
Prior art
Uploadcare built their own Netlify CMS integration back in June (shoutout to @dmitry-mukhin 🙌 ). They worked around a lot of limitations to provide a script that integrated their widget into Netlify CMS despite our lack of an API to allow it. Beside the Uploadcare specific bits, the integration mostly centered around a custom widget for the editor and an editor component to replace our auto-registered image component in the Markdown editor. Our work on the new API benefited heavily from their work, as it helped to clarify the challenges facing this kind of integration while proving that using external interfaces within the CMS really can provide a solid experience.
Retrofitting FTW
Requiring every integration to create a custom widget, editor component, and undoing our dependency on the built in image editor component requires extra time and maintenance. So we went with retrofitting instead. The current image and file widgets, which most Netlify CMS projects are already using, interact with the built in media library through methods that hook into our Redux actions. They're not actually opinionated on what that media library is, only that it provides a URL value. This applies to the existing image editor component as well.
Retrofitting means that our image and file components, along with any widgets that interact with the media library, can now control whatever media library is registered. No custom widgets necessary.
API crash course
The API is not public yet, but it will be soon. It looks like this:

registerMediaLibrary accepts an object with properties name and init

// netlify-cms will do this automatically
import uploadcare from 'netlify-cms-media-library-uploadcare'
CMS.registerMediaLibrary(uploadcare)

init is a function that is only called once, and should return an API object with these methods:

show: called when the media library needs to open, args include:

id: the uuid of the calling control, if any
value: the current value of the control, if any
config: the media library config from the calling control to apply to this specific instance, if any
imagesOnly: boolean indicating whether this instance should only accept images

hide: called when the media library needs to close, may not be necessary for some integrations that block page interactions (possibly the case for all of them)
onClearControl: called when the calling control has it's value cleared, args include:

id: the uuid of the calling control

onRemoveControl: called when the calling control is about to unmount, args include:

id: the uuid of the calling control

enableStandalone: method that should return a boolean indicating whether the media library should be opened without a calling control, e.g. via the global Media button

Bonus: you can now upload multiple images!
Only with external media libraries, and only with the ones that allow it, but still. Uploadcare's configuration docs show a multiple option for using their multiple selection widget - setting this configuration is all that's needed, the integration handles using the right widget and the image/file editor widgets can now handle multiple values!
Using the Uploadcare integration
Uploadcare's widget isn't a media library - it doesn't show things you've already uploaded - so it returns false from it's enableStandalone method. This means the Media button in the nav is removed when using Uploadcare. #notabug. The integration loads dependencies via injected script tags because they're using a bit of ES3 that breaks our builds, and so could not be fetched via npm.
Other than that, the docs cover how to use. It's just config changes!
That's all folks
We're really excited to get this out into the wild in Netlify CMS 2.1.0! Please check out the PR and the docs, test it out, etc - reviews and feedback are much appreciated ❤
Closes #432.
TODO

[x] fix image fields in editor component

Deploy preview for netlify-cms-www ready!
Built with commit 7b0aba1cd15e9ad43021bc19210ed14bfa8989e3
https://deploy-preview-1602--netlify-cms-www.netlify.com

Deploy preview for cms-demo ready!
Built with commit 2ba7feb50864fd46892586401551a4cccf8ba02b
https://deploy-preview-1602--cms-demo.netlify.com

Cool, sounds and looks promising

@erquhart Nice work! Custom media library API is exactly what we so lacked.
In the provided demo, there is an error when I click on "Choose an image" in the body editor:
mediaLibrary.js:63 Uncaught TypeError: Cannot read property 'toJS' of undefined
    at A (mediaLibrary.js:63)
    at index.js:9
    at bindActionCreators.js:3
    at t (withFileControl.js:131)
    at Object.<anonymous> (react-dom.production.min.js:15)
    at Object.invokeGuardedCallback (react-dom.production.min.js:16)
    at Object.invokeGuardedCallbackAndCatchFirstError (react-dom.production.min.js:16)
    at C (react-dom.production.min.js:20)
    at R (react-dom.production.min.js:22)
    at N (react-dom.production.min.js:22)

export function openMediaLibrary(payload = {}) {
  return (dispatch, getState) => {
    const state = getState();
    const mediaLibrary = state.mediaLibrary.get('externalLibrary');
    if (mediaLibrary) {
      const { controlID: id, value, config, forImage } = payload;
      mediaLibrary.show({ id, value, config: config.toJS(), imagesOnly: forImage }); // this line
    }
    dispatch({ type: MEDIA_LIBRARY_OPEN, payload });
  };
}

Uploadcare's widget isn't a media library - it doesn't show things you've already uploaded - so it returns false from it's enableStandalone method.

Despite the fact that Uploadcare is not a media library, we can store user-uploaded UUIDs in LocalStorage or globally in the repository as a json file. And provide those files with Custom Tab in the Uploadcare Widget, like EffectsTab. What do you think about it?

@nd0ut: I was considering this direction, but your suggestion may be a better tack indeed:

For my use-case the user needs the ability to select and add multiple images into something like a List widget. I am wondering how and if we can treat Uploadcare as a kind of assetStore to fetch files and pass them to the built-in MediaLibrary.
I've been diving into the Uploadcare docs and it seems feasible to use their REST API get-list to populate the built-in Media Library with infinite scrolling.
I am considering exploring this direction by hijacking the loadMedia action in a PR. Perhaps that ground-work could be abstracted into a API for treating services like Uploadcare as an asset store?
Or is there another tack?

Today I'll start looking into your suggestion to use the Custom Tab in the Uploadcare Widget and share my findings, as I need to provide a Media Library experience and 3rd party storage asap!

Based on @nd0ut's suggestion I've followed the Uploadcare docs and set up a Custom Tab that passes in an array of UUID's via the config.yml for mock data. This approach works extremely well as a simple Media Library.

Currently vanilla Javascript to appends the images to the uploadcare modules DOM, see here.
However, features which we have in the built-in Media Library like pagination and search will be sorely missed, so I am considering if and how I could inject a React Component into the Uploadcare widget.
I am also wondering how to best populate the tab with real data. A JSON file checked in to the repo or a call to the Uploadcare API to fetch paginated lists of files.
@erquhart: I am curious to hear your thoughts and if you'd have any pointers on how to proceed!

@owenhoskins still having a hard time envisioning that kind of interface working for sites with a lot of images like the one you're working on. Your editors would need to manually scroll through 5,000+ images to find the one they're looking for. Wouldn't you need a folder hierarchy, and possibly a nested one?

Thanks for your consideration @erquhart! In this use-case, the file name carries an artist code which acts as a kind of folder system when searching by filename for that code. So far that's worked quiet well and recreating the filename search feature would fulfill this particular use-case.
Uploadcare's Group API is interesting but it is immutable and as such it doesn't seem to be a route to a folder system (or nested one).
If we went the route of saving UUIDs to a json file in source control we could create some kind of folder system based on which "post" the file was uploaded within, for example. Thoughts?

Uploadcare's Group API is absolutely not supposed to be used as some kind of folder hierarchy.
I like an idea to use json with UUIDs and metadata. This allows to describe the hierarchy in any needed form, linked with posts or with users, implement access-control and searching.

Fixed the editor component issue - I'm considering this ready to merge, as the other improvements mentioned here can be added on afterward.
Going once, going twice?

This isn't an out-of-the-box widget?

Ok, I thought I would have to stop using CDN links to use this.

Hello all. (especially @erquhart & @nd0ut)
I was considering opening another issue or pull request to share my progress here but I thought I'd start with a follow up comment here.
I've integrated the Search component from the built-in Media Library into a Custom Tab in the Uploadcare widget. Currently I am calling the Uploadcare API to populate the components state, which enables search by filename and is fed into a react-virtualized implementation to lazy-load and infinite-scroll the list of images.
I also recognized that the netlify-cms-widget-image when used with the Uploadcare Preview tab makes for a very useable drag-and-drop gallery manager, nice!

Calling the Uploadcare API in this way requires passing both the public/private key in front-end code so this isn't a viable production option.
The next step would be storing UUIDs / metadata in JSON files but I am not sure how to proceed. Please advise!

@owenhoskins Hello! Nice work!
I just deal with the same problem, but on the other hand - I've started from storing JSON file index. This is done with persistMedia action which stores passed File to the store (git repo on production or local storage on dev). You can view my code here. It is working but completely not ready so I didn't create PR.

@owenhoskins can you make a PR with your code? I think we could collaborate on this.

@nd0ut I took your branch for a spin -- it's so very satisfying to have the uploads persisted in a JSON file. Great work!
I stumbled a bit getting started because the local storage works as described but when I added a backend repo I encountered JS errors e.g. getState().mediaLibrary.get('files') at file-index.js:15 returning undefined.
After I protected against undefined in a few instances of the media library actions and reducers my the JSON file persisted to the repo.
So it looks like all calls to get('files') return undefined until there is actually a file at assets/uploads/uploadcare-index.json in the repo . Sound plausible?
Other then that I have encountered a few aspects which are probably what you were referring to when you described the branch as 'completely not ready'.
I would love to lend a hand and start integrating the UI that I've been working on. Do you have any suggestions for a communication channel/workflow to collaborate on this? Thanks!

@owenhoskins
I understood what the problem was with that state.medialLibrary.files is undefined. You just need to call loadMedia action to load media with provider, after that files will be initialised.

Could someone help?
I am trying to set up a free account with Uploadcare, i can see the website lists a free tier(https://uploadcare.com/pricing/) but when signing up, it only provides a 14 day trial. there is no other options.

@franva I suggest you contact Uploadcare support https://uploadcare.com/community/c/support/7

Hi @franva, I'm Alex from Uploadcare support. Sorry for the confusion. We're re-working our registration flow and have temporarily removed the option to choose the free plan on signup. We still offer this plan, though, and will be happy to provide you with it. Drop us a line at<EMAIL_ADDRESS>and we'll set it up for you shortly.


getting the sent message id

It'd be nice when you call send, that the message id was returned in the response.
Then I could match back to unsubscribes in my system.

Hi Daniel,
You can call addUniqueArg() on the SendGrid Email object to add a unique
identifier to the email.  This can be used to link the email back to a
specific account.  We do this for iDimensionz's Communiqué email marketing
system to track all statistics (via the web hooks).
Best regards,
Jim Mullen
On Fri, Apr 29, 2016 at 5:59 AM, Daniel Platt<EMAIL_ADDRESS>wrote:

It'd be nice when you call send, that the message id was returned in the
response.
Then I could match back to unsubscribes in my system.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/sendgrid/sendgrid-php/issues/230

@hackzilla
@idimensionz solution is correct (thanks Jim!)
Here is how you would add a custom argument using our new v3 /mail/send endpoint: https://github.com/sendgrid/sendgrid-php/blob/v3beta/examples/helpers/mail/example.php#L50


[Feature] support ANSI standard keyword for current_xxx

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
add below keyword
CURRENT_DATE
CURRENT_TIME
CURRENT_TIMESTAMP
LOCALTIME
LOCALTIMESTAMP
CURRENT_USER

snow flake doc: https://docs.snowflake.com/search?q=To comply with ANSI standards
Use case
No response
Related issues
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

May I be assigned to this issue?
This would be my first contribution to the project.
If I understand correctly, we want the sql parser/scanner/analyzer to run the same function for the following query
select CURRENT_DATE; is interpreted the same as select CURRENT_DATE();
and similarly for the other keywords mentioned.

I can try it

I can try it


Merged consul and config-seed into one docker

To avoid high memory peaks during docker deployment, we have merged consul configuration and config-seed.
Changes in dockerfile to launch a script that do the work
This PR requires the PR in core-config-seed

Closed by mistake


iOS Pod file requires manual link to React pod in node_modules

After following your iOS install directions. I kept getting a warning when installing the pod: [!] React has been deprecated. This also led to build issues in the React Native packager:
Ambiguous resolution: module ".../index.js" tries to require 'react-native', but there are several files providing this module. You can delete or fix them:

.../MyProject/ios/Pods/React/package.json

.../MyProject/node_modules/react-native/package.json

I got around the issue by manually linking the React pod to the node_modules directory so that it wasn't downloaded by the AppsFlyer pod.
pod 'React', :path => '../node_modules/react-native', :subspecs => [
  'Core',
  'DevSupport'
]
pod 'react-native-appsflyer', :path => '../node_modules/react-native-appsflyer'

post_install do |installer|
   installer.pods_project.targets.each do |target|
      # It also wouldn't work without this
      if target.name == "React"
        target.remove_from_project
      end
   end
end

Do you guys need the React dependency in the podspec file given that this already is a react native project?

note: I'm a user of appsflyer, but can explain this.
Recent cocoapods don't add something into pod include paths if it not a dependency of pod. So it is required specify React as dependency in react-native-appsflyer.podspec.
Most working pods for react-native follow this pattern (firebase, react-native-navigation, sentry etc...)
I'd personally recommend to use cocoapods for all deps including react-native, it simplifies maintenance of project for ios.


Unable to load some chinese named roms.

very strange problem, I got rg351 from aliexpress, all hacked games in gbah, nesh, genh etc folder with chinese names wont load even I move them to the right folder.  They are working fine after I unzipped them.  attached are 2 x gba roms that wont load. Maybe some problem with the font?
The file explorer also cant show the chinese character.

I have removed the rom as they are illegal to load on here.  The issue is probably related to incompatibility with the file system and characters.  The solution is to either change the name or do as you already did.


.Net Core Support

Kindly consider updating the library to be supported by .Net Core

I cannot use the library because .Net Core is not supported.
I get the following error:
Newtonsoft.Json.JsonReaderException: 'Unexpected character encountered while parsing value: T. Path '', line 0, position 0.'


Bash shell only?

Can this be used via command line only? I really want an SSR server on my Pi with a http proxy like this project has, but I don't want to install a GUI just to run this...

I'm currently developing a new SSR client based on web gui and nodejs, which will be available on headless platform.  you can check it out on my GitHub page, ssr_cli. The development will be completed in days.
xVan Turing
<EMAIL_ADDRESS>---Original---
From<EMAIL_ADDRESS>Date: Mon, Aug 17, 2020 23:00 PM
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS>Subject: [shadowsocksrr/electron-ssr] Bash shell only? (#110)
Can this be used via command line only? I really want an SSR server on my Pi with a http proxy like this project has, but I don't want to install a GUI just to run this...
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

I can’t find that repo? I can contribute if you pass me the link.

The link is here. The project is still in a very early stage, it may be hard to contribute for others.


tiles.transform

Currently, it is a bit cumbersome to set the scale and translate on the tile image container.
raster.attr("transform", "scale(" + tiles.scale + ")translate(" + tiles.translate + ")");

It can be particularly confusing because the scale must come first, which is not that common. It would be nice to be able to say this instead:
raster.attr("transform", tiles.transform);

Here's a prototype implementation of this where setting the transform property on the returned tiles array looks like this:
tiles.transform = d3.zoomIdentity
  .scale(k)
  .translate(x / k, y / k);

Without a dependency on d3-zoom, setting transform could look like this:
tiles.transform = {
  k: k,
  x: x / k,
  y: y / k,
  toString: function() {
    return "translate(" + this.x + "," + this.y + ") scale(" + this.k + ")";
  }
};

Duplicating toString from d3-zoom.

The implementation could also look like this, a mini d3-zoom transform:
function Transform(k, x, y) {
this.k = k;
this.x = x;
this.y = y;
}
Transform.prototype = {
toString: function() {
return "translate(" + this.x + "," + this.y + ") scale(" + this.k + ")";
}
};
...
/// inside tile(t)
tiles.transform = new Transform(k, x, y);

I noticed this snippet in some of the updated examples, maybe ideas from it could be added into the library.

providing tiles.transform with scale / 256, intended for direct use with 256 X 256 image elements.
The rounding logic for crisp tile boundaries (at least I think that's what that is for..).

function stringify(scale, translate) {
  var k = scale / 256, r = scale % 1 ? Number : Math.round;
  return "translate(" + r(translate[0] * scale) + "," + r(translate[1] * scale) + ") scale(" + k + ")";
}

A similar comment, three years later: https://github.com/d3/d3-tile/pull/53#issuecomment-513263755
Adding this issue to 1.0 to see if we can figure out something more convenient.

I’m going to punt on this for 1.0 because I don’t have a great idea for making this easier. The main challenge is that there are many valid ways of rendering tiles, and most of the convenience methods would have to assume a particular rendering method (such as Canvas, or an SVG G with a transform), and I don’t want this library to be overly prescriptive in terms of rendering strategy.


Containerize using Docker

Using reference Dockerfile from https://github.com/ba-st/docker-pharo

Done https://github.com/sebastianconcept/matter/pull/19


Drain Response.Body to enable TCP/TLS connection reuse (4x speedup)

On successful queries (on errors ReadAll runs in CheckResponse), the
Response.Body is not read all the way to the EOF as json.Decoder is used,
which stops at the end of the object, which is probably followed by a \n.
Then, Response.Body.Close() is called.  When that happens with a non-drained
Body, the TCP/TLS connection is closed (which makes sense, in case there was
still a lot to read from the network), stopping the Transport from reusing
it.  Also, a library user can't drain the Body themselves because Close() is
called before passing the Response to the user.
Since we know GitHub API responses only carry a single JSON object, draining
before closing is free, and saving all those handshakes causes huge savings
in latency, bandwidth and CPU for both the client and the poor GitHub
servers.
Here's the result of running the benchmark below on a ADSL connection:
before: 2m3.001599093s
after: 33.753543928s

func main() {
    ts := oauth2.StaticTokenSource(
        &oauth2.Token{AccessToken: os.Getenv("GITHUB_TOKEN")},
    )
    tc := oauth2.NewClient(oauth2.NoContext, ts)
    gh := github.NewClient(tc)
    start := time.Now()
    for i := 0; i < 100; i++ {
        gh.Repositories.Get("FiloSottile", "gvt")
        fmt.Print(".")
    }
    fmt.Printf("\n%s\n", time.Now().Sub(start))
}

Hey Filippo, this is really cool! I have one question, I'll post it inline.

Maybe it's a good idea to add a test for this?

I don't have time to write one today (1:30am :astonished:), but I think one can be written like this

make one request
replace Dial in the Transport with panic()
make another request

But it might be flappy, as no behavior is guaranteed here AFAICT.
Can't think of a simpler one.

I don't think it's a good idea to have a test for this, unless someone can come up with a reliable one. This is just normal behavior, and it's explained well in the comment in code. This change is not about correctness, it's about an optimization.

Thanks for the patch @FiloSottile!

make one request
replace Dial in the Transport with panic()
make another request

Why not just mocking with an httptest server any of the github endpoints and use go-github to make the request?. I'm not familiar with the library (I came here through the gophers slack) but I believe it would be fairy simple to do this and assert that the connection is being reused (I've done this in the past with some other projects. )
Thoughts?

Hi, author of net/http here. I cannot reproduce this result.
The underlying reader (the one beneath the oauth2.Transport's onEOFReader) is returning (83, io.EOF), which is successfully recycling the TCP connection, even without your patch.
Are your measurements actually from that code, or similar-ish code?
Which version of Go?
Which version of oauth2?

@bradfitz, I tried to reproduce this just now, and I was able to reliably see a significant performance difference.  As you can see, it makes a significant difference each time.
The fast runs (14-18 seconds) are with this change applied, and the slow runs (42-73 seconds) were with //io.Copy(ioutil.Discard, resp.Body) commented out.

Any chance you could try that experiment using Go tip? Brad was testing with (I believe) a version pretty close to tip. I wonder if there's something that changed in net/http since go1.6.

Sure, I can try with tip. It'll be a few moments because I'll have to get it and build it.

Actually, never mind. I think I can reproduce it now at tip, on darwin/amd64 too.

 Thanks @shurcooL!

FWIW, I had already started, and I was curious, so I tried 1.6 and tip in a Linux VM (Ubuntu 14.04), and I was able to reproduce it there too. Both with 1.6 and current tip (https://github.com/golang/go/commit/45d334ecf1b2bcbf0f8667d4c772ef3db0e03587). See screenshot.

@shurcooL and @dsymonds answered most questions already, but just to confirm I tested with 1.6 and tip on OS X, with and without oauth2, with and without using DefaultTransport.

Sorry for being slightly OT but why doesn't Close() drain the body itself?

@slimmy: because closing a connection would then potentially block for a long time - or forever - if the other end of a connection has a lot of / infinite bytes still to be sent. Imagine calling Close only to block on receiving the rest of a 10 GB file you didn't want.
As mentioned in the PR:

Response.Body.Close() is called. When that happens with a non-drained
Body, the TCP/TLS connection is closed (which makes sense, in case there was
still a lot to read from the network)

@kofalt: That makes sense, but it doesn't seem like something that would actually happen a lot in practice. Maybe it should be configurable to make Close() drain the body.

@slimmy
Because of bad actors in internet that would send infinite number of bytes to your site.

@devlo: I get that, but everyone using io.Copy(ioutil.Discard, resp.Body) to enable the reuse of the TCP connection would be susceptible that as well no?

@slimmy
You can use io.LimitedReader in this case to mitigate that.

@devlo
I'm just trying to understand it all in the context of this PR. To achieve maximum performance you have to make sure you read all of the response body (so that the TCP connection can be reused). But in doing so you are risking large (possibly infinite) streams of data. So the solution is to only utilize this "trick" when you are fetching data from trusted sources (i.e GitHub as in the case of this PR)?

I'm trying to reproduce this issue in its purest form (ie: Without any dependencies beyond the standard packages), but so far am unable to. I want to fully appreciate if this is a problem for anyone using net/http or if it's limited to this project.
Here is my attempt (results in the readme): https://github.com/kevin-cantwell/tcpdraintest
It appears the performance differences are greatest when the number of requests is low. I'm reusing the default http client and my json responses are empty objects with a few extra newline characters to drain. Am I doing it wrong?

Ok, I think I understand a bit more about this. Seems like this change makes the biggest difference when there is a large number of bytes remaining in the body after json parsing. A few extra bytes doesn't seem to matter much, even at high request volumes. But if you've got hundreds or thousands of extra bytes after json decoding is done, then yes, this could seriously affect performance. My question then is: without committing to draining the entire remaining bytes (which as @kofalt pointed out, could be GB of data), how far should one read before just closing the response? Because unless you read quite far, the performance gains might not be visible.
Another question I might raise is: Is it even reasonable to expect there to be lots of bytes after a terminating json character? There could easily be a newline or two, but hundreds or thousands of them? I'm skeptical that any api would serve that.
Again, if I'm missing the point somehow, please illuminate me! :)

Ok, I think I understand a bit more about this. Seems like this change makes the biggest difference when there is a large number of bytes remaining in the body after json parsing.

In the GitHub cases above, there are actually no remaining bytes after json parsing, so I don't think your statement here is correct.  One current working guess is that the io.EOF hasn't been reached, and that's what is causing the connection to not be reused.

But in doing so you are risking large (possibly infinite) streams of data. So the solution is to only utilize this "trick" when you are fetching data from trusted sources (i.e GitHub as in the case of this PR)?

@slimmy, have you seen this inline https://github.com/google/go-github/pull/317#discussion_r57536827? A proposed suggestion there is to "read, say, 512 bytes with a 50ms timeout happening on close, which would probably be good enough for cases like this one."

I'm trying to reproduce this issue in its purest form (ie: Without any dependencies beyond the standard packages), but so far am unable to. I want to fully appreciate if this is a problem for anyone using net/http or if it's limited to this project.

@kevin-cantwell, an extremely important aspect of this is the fact that these are TLS connections being made, since the protocol used is https, not http. TLS connection handshake is quite expensive, so reusing TLS connections made to a remote server will offer great savings. Reusing non-TLS connections to localhost will have very little gains, and what's what your test appears to be trying to do from what I can tell.

TLS connection handshake is quite expensive, so reusing TLS connections made to a remote server will offer great savings. Reusing non-TLS connections to localhost will have very little gains, and what's what your test appears to be trying to do from what I can tell.

@shurcooL That makes sense, thanks! And when I use an https api I can confirm the performance gains:
Output:
$ go version
go version go1.6 darwin/amd64

$ go run main.go https://api.github.com/repos/google/go-github 10
Without drain (x10): 1.302103099s
   With drain (x10): 282.559111ms

$ go run main.go https://api.github.com/repos/google/go-github 100
Without drain (x100): 8.339789738s
   With drain (x100): 2.694648312s

$ go run main.go https://api.github.com/repos/google/go-github 1000
Without drain (x1000): 1m16.253147895s
   With drain (x1000): 32.260955511s

Source code:
package main

import (
	"encoding/json"
	"fmt"
	"io"
	"io/ioutil"
	"net/http"
	"os"
	"strconv"
	"time"
)

func main() {
	url := os.Args[1]
	n, _ := strconv.Atoi(os.Args[2])

	client := http.DefaultClient

	d1 := run(client, url, n, false)
	d2 := run(client, url, n, true)

	fmt.Printf("Without drain (x%d): %v\n", n, d1)
	fmt.Printf("   With drain (x%d): %v\n", n, d2)

	os.Exit(0)
}

func run(client *http.Client, url string, n int, drain bool) time.Duration {
	var total time.Duration
	for i := 0; i < n; i++ {
		start := time.Now()
		resp, err := client.Get(url)
		if err != nil {
			panic(err)
		}
		dec := json.NewDecoder(resp.Body)
		var body map[string]interface{}
		if err := dec.Decode(&body); err != nil {
			panic(err)
		}
		timing := time.Since(start)
		total += timing

		if drain {
			io.Copy(ioutil.Discard, resp.Body)
		}
		resp.Body.Close()
	}
	return total
}

Okay, I can reproduce now. I had a typo.
As background, Go used to auto-drain (around Go 1.0 and Go 1.1 era) but it was problematic for a number of reasons and we changed the behavior to be more predictable but also pushed io.EOFs earlier for many readers and made the final Read-with-EOF on the connection also auto-recycle the TCP connection for people who forget to Close. It seems there's a Reader not doing early (n, io.EOF) instead of (n, nil)+(0, io.EOF) somewhere.
I see the final 0-byte HTTP/1.1 EOF chunk arriving in the final read:
2016/03/30 09:23:20 transport-1.Read(4096) = 1370, <nil> = "HTTP/1.1 200 OK\r\nServer: GitHub.com\r\nDate: Tue, 29 Mar 2016 22:22:34 GMT\r\nContent-Type: application/json; charset=utf-8\r\nTransfer-Encoding: chunked\r\nStatus: 200 OK\r\nX-RateLimit-Limit: 5000\r\nX-RateLimit-Remaining: 4946\r\nX-RateLimit-Reset:<PHONE_NUMBER>\r\nCache-Control: private, max-age=60, s-maxage=60\r\nVary: Accept, Authorization, Cookie, X-GitHub-OTP\r\nETag: W/\"ee6421a9addcf34be8071a9b78863cb0\"\r\nLast-Modified: Tue, 29 Mar 2016 10:05:03 GMT\r\nX-OAuth-Scopes: repo\r\nX-Accepted-OAuth-Scopes: repo\r\nX-GitHub-Media-Type: github.drax-preview; format=json\r\nAccess-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval\r\nAccess-Control-Allow-Origin: *\r\nContent-Security-Policy: default-src 'none'\r\nStrict-Transport-Security: max-age=31536000; includeSubdomains; preload\r\nX-Content-Type-Options: nosniff\r\nX-Frame-Options: deny\r\nX-XSS-Protection: 1; mode=block\r\nVary: Accept-Encoding\r\nX-Served-By: 7f48e2f7761567e923121f17538d7a6d\r\nContent-Encoding: gzip\r\nX-GitHub-Request-Id: 4A7D38D6:36CB:143A0579:56FB002A\r\n\r\n4aa\r\n\x1f\x8b\b\x00\x00\x00\x00\x00\x00\x03\xa5X]o\xdb6\x14\xfd+\x86^焲\x13w\xb3\x80\xa1{XV\x14Xۡˆa/\x06-\xd1\x12\x11I\x14H\xcaA*\xe4\xbf\xef\\\x91\x92eé\x13\xee%\x91%\x9e\xc3\xc3\xfbA\xde\xcb.\x92Y\x94\xdc.ַ\xab\xe5z1\x8fj^\x89(\x89\xf2\xbd\x8d\xe6Ѯ-ˍ\u007f\xf3\x9b,՟\xcaZY\n澪\xc7Z\xe8(\xe9\xa2R\xe5\xb2\x06h2\x04`\xe2],\x97\xa0\xbd\x9dG|\xcf-כV\x97\x18WXۘ\x841\xf7\xd2\\\xe7\xd2\x16\xed\xb65B\xa7\xaa\xb6\xa2\xb6ש\xaaX\xcb<\xfa\xfd\xfe\xe7\x1b\xf0\xe5ړ\x10q\x84\x17'd\x8d\xf4D\x0e\r6Î\x15\x15\xb6*O$\xb8\xa9{\xc4\xf1\u061d*K\xf5\b\x8eS\xcd\x17\xa6a#\x90\xecד\xc8:\x0f#\x01\xb0c\xca\x16\x02\x86\xc3r\x9e\xc9\b\xd2\xd87K\xeaA\x1d\xa3\u007f\x1b\x99\x11\x8d\x813\xb4"
2016/03/30 09:23:20 transport-1.Read(4096) = ....
2016/03/30 09:23:20 transport-1.Read(4096) = 949, <nil> = "\xc8\xde*\xcb\xc3 \x8a\\\xff\xdc1-\x1a\xd5\xf3\xb5[\x93j\xd9X\xa9\xea7\xcb;\x02\x83L\xe9\x9c\xd7\xf2\x1b\x0f\"\x03\u0600\x83\x84\xbdYH\x0f\x02X\xec\x11\x83oF;T\xc7\x1a-\xf7<}\"\xb3h\x91\n\xb9\x87\x9d\xc3\x18O\xe0 \xb4O\re\xe7_\x88\x06\xb2\xba\xb4bó\x8a\x92o\xc7K#\x9e\xe7Q?\xbbŠ\xfe\xc5\x1c\xb9\xf6ʘ\xf7I\x9d\x89ёn\x1b\x98I3C\b\xcer5\x83Y2\xa5\x11\x953\xabT9\xdb)\xdd\u007f\xf9\xf0e\xb1\xfa\xfb\xee\xf3\xaf_\xbe\xde\xfd\xf3\xc7\xdd\u05cf\x9f\xee>\xdf\xcfg[nD6S\xf5,\xdf^9 \x14\x03\xf20J\xfbn\xfe\xf6\xbe\x98\xe6\xa4\xd7G\f\x17\\s\x16\x8a\xbc\x04\x10\x12\x1e\xc4S\x10\x9ep\x1d\xc3_\x9fB)r\x9bo\x95\xe6V]\xda$\xce\v:\"\xe8\xd8\xf4'Ŏ\x15\xbc\n\x12\xda\x03AP(\x15f\xa9\x1e\b\x02iL+^\x15\xba\xe7\x17\xd8\xe3\r\x1b\xf2\xa2n\xab\xad\xdb\xc4^\x93\r\xe7)\x1d\x12ڸ12\xaf\x85\b\xb2\xd0\b\xeeذ\xafn5\xaf\xd3\"\x8cn\xc0v\xcc=\xf5\xde\xe3y\x904\xc2\x01\xbe-\xd56\b\x8f\xf3\x8c\xf5\xe0\x8e\x99\x82\xbb\x13\xc3nB\xd5\x10\x1ba\x8fȴ\xd8\x05K#\xecHfu\xa0\xffzY\x04\x1e\xa9p0Y\xb82H׀e\x9d\xb7X\xc9\xeb\xbc\xe5y\x18\xdb\b\x86\x17\xe9\xb8\xcc\xf9\xb7\x8bE\xc4\xf9X?\xa0AE\x85\x91\x96\xdb6|\xb39\xe0I\x99;\xaf\x91\x8fa&;\xc0\x0fd\xfd\xe1\xff\xfd\x8aⅅN\x8a\x87~\xa9U%/\x1d\xbf\xe7\x99<\xf4(\\\xff\a\x1d\xc5\xd9)%\xfd\xbe\\\x1d\xbc,\x8f\xb0\x1d;\xec\x85n\x93\xf5\xac!\xd6\xf3\xbb\xec\xa0k\xca\xed\xab\xe9 \x17\x0fX\xd6\xfd\xd0p[\xd0N\x82)\x1a\xaeE\x88H\x0fe\x1dU\x05\xcf\xd7\xd7\xd7]!x_\x89VB\af\x9aC\x82\x82\xeb\xb4@\x95\x15\xa2\xab\x1b\xb0\xa8\x0e*n\xfb\x8avG\xb22T\xb8\xa5\xe2Y\x90\xedF0\x88\x9c{B\xb49\xe4ԟ\rڱ A=p\xcaT\xa1\x8b3V\xd5a{\xdc\x01=嬕\x95;\x99\xbe\xa6l?\x9f\x1eG\x04\xdd{#\xebT\xccyY\xce\x11uV\xa6\x12q\x88ړ\xbc\x83\xc2K\x84Y\xc2!!\x1bm\xae\xab\xd0K\x81\x90\f\xb2\xaa\x16\x0e\xdb1\xd7Te\xa2)\xd5S\xf0\xee0\x81S\xb2i\x812>\xdbp\x8b2|\x19/VW\xf1\xfa*^\xddǋd\xf5.\x89\x97\xffbL\xdbdGc\xde]\xc57W\xcb\xf5\xfd\"N\xe2U\x12\xdfИ\xa65ń\xc6\r\xf9\xe9~qC4\xb71\r\xc16\xe7\xe3\x13Oh\xcb\xcf\xf7\xc4T\u007fS\x83\r\x801\xc5\x01\xf0\xcbaxrrG\xe0\x87\xa7%\x02\xed$\x03.ϱ?=K^\x86@R\xa1*\xd1\xe0\xccv7\x03F~\xc3\xd3\"^\x1d\x9d\xc1\xa9jkXs\xb9^ΣGnQ\xee\xe1\xf4\x9b\xbe\x1c\xcen\x90|PD\xca\xcd\xc6ea\x94X\xddR7\x857\x87잼|\x94\x0fr:\x88\xb4\x98\xe1\x85\xebY\x86\x99p\x1bRI\xad\x95\xbf\r\xa9\x91\x9b\xe8y\x1bQ\xfb\xb9\x06Ik\xd7.\x81e\t\xc8d@\x94\xe0Ӱ\x00\xbf\x9eL\xecx[ڍ\xab@\xb1\x80\x8a\x1b\xdb7\x89\xee\xe1̇F\xe8\n\xab\xa3\x8e\x9d.q\xa6}\xa4\v\x9b\xb1Q\xa3\xedí\x05\rf)SQ\x1b\x98\xb7\xa3V\x8af\xeac\xc2\xdf\x12}\xfax?\xfbݏ\xb8pC\xe3\x89\fs\x04;\xc4{\x8b;\x89q\xa2Z\xd8G\xf4l\xa3\x87`\x84i\xd9\xe2\u0379\xf8\xf1\xf9?\xb7a\x95\x03\xc3\x12\x00\x00\r\n0\r\n\r\n"
Read on *http.bodyEOFSignal = 512, <nil>
Read on *http.bodyEOFSignal = 1024, <nil>
Read on *http.bodyEOFSignal = 2048, <nil>
Read on *http.bodyEOFSignal = 1219, <nil>
Draining...
tryPutIdleConn = <nil>

(Scroll to the right forever and you can see \x00\x00\r\n0\r\n\r\n" at the end)

@bradfitz
This one?
https://golang.org/src/encoding/json/stream.go?#L124

I sent out https://go-review.googlesource.com/21290
@bashtian, what about that line?

Actually, better fix: https://golang.org/cl/21291

@FiloSottile unbounded io.Copy calls should be avoided in this case. io.CopyN or just a Read with, say, and upper bound of 1 KiB would be much, much safer -- if the server side has a bug that causes it to send megabytes or worse of extra data down the socket, you don't want to read all that just as an optimization for reusing the connection (you really want to have the constraint of: "if there's not much left in the pipe, then reuse. Otherwise, force close")

@extemporalgenome good idea about using io.CopyN instead here.  I'll make that change.

However, the docs on http.Client.Do only mention Close, not Read:

If the returned error is nil, the Response will contain a non-nil Body which the user is expected to close. If the Body is not closed, the Client's underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent "keep-alive" request.

Just to clear things up for others that ended up here - @pschultz the linked go doc have been updated
The response body SHOULD be drained/read to completion! The new docs (on your link now) make it more clear.

@karelbilek  It seems that Close() will drain the body automatically.


Permission level fe

Description
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[X] New feature (non-breaking change which adds functionality)

Change Status

[X] Complete, tested, ready to review and merge

How Has This Been Tested?

[X] Used localhost to test that new feature works as desired

Checklist

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[ ] My code has been reviewed by at least one peer
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] There are no merge conflicts

 Give BE access to permission level in url - FE


build: upgrade to node lts 20.9.0

Upgrading to the latest version of node - 20.9.0

Description
This will align with the changes in kommander-ui to bring us to the latest version of node.
Encountered this error after the upgrade initially:
TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received an instance of Object
Found a suggested solution here and it worked to include the conversion to a string.

Which issue(s) does this PR relate to?

Testing

Trade-offs

Screenshots

Checklist

[ ] This PR is associated with a JIRA and it is mentioned in the commit message footer ("Closes …")
[ ] Significant changes have been tested downstream to avoid breaking changes
[ ] This PR contains breaking changes and states in the commit message body ("BREAKING CHANGE: …")
[ ] I have reviewed the changes and provided detail to the sections above

Do we use this Dockerfile even anywhere?

You know what, no, it looks like was back from the Jenkins set up for CI but with gh actions and gh pages we should be okay here. I've removed this.


pthread: fix pthread exit error when set DETACHED

Summary
pthread: fix pthread exit error when set DETACHED
Impact
pthread exit
Testing
VELA

Let's ignore the unrelated ci broken which is fixed by https://github.com/apache/nuttx/pull/8155.


about description.py for nuscenes

i found another error, it seams that the description.py in WorldDreamer just descript nuplane dataset , and could you please tell how to descript nuscenes, thanks a lot ~.
and BWT  could you please provide the code to make demo.pth~

nuScenes  and nuPlan are using the same pipeline.  just modify these lines to read in nuscenes datasets:
https://github.com/PJLab-ADG/DriveArena/blob/ba7dd86b606b92710e111e55fa639ebf6ebe0ed4/WorldDreamer/tools/description.py#L87-L99

thanks for your reply , and i'm wondering that if you can provide the code to provide demo_data.pth ~


chore: Shrink docker containers (pnp in final artefacts, amongst other things).

From: /yarn-project/yarn-project-base/Dockerfile
This base Dockerfile is for:

Caching the workspace dependencies.
Running workspace checks (package.json inheritence, tsconfig.json project references).
Bringing in any upstream artefacts such as circuits.wasm and L1 contracts.
Performing any code generation that doesn't require the workspace code to do so (generate L1 artefacts).

When installing workspace dependencies, there are issues with doing this naively:

We only ever want to re-download and install workspace dependencies in the event that they change.
If a developer only changes some code, we want this step of the build to become a noop and to reuse existing image.
Dockerfile.dockerignore is tailored to specifically only bring in files needed to install the dependencies.
NOTE: For this dockerignore file to be used, you MUST be using BUILDKIT in more recent versions of Docker.
Best to make sure you're on docker >= 24. On mainframe run restart_docker, it should auto-upgrade.
A non pnp install is many 100's of MBs larger, impacting build times and bloating the final container size.
We use .yarnrc.prod.yml to enable pnp in docker. This file is simply copied over the top of .yarnrc.yml.
Tooling such as prettier/eslint is a pain with pnp, so we only want to enable it towards the end of the build.
We want to disable yarn from accessing the net after initial package download as it prevents a class of build bugs.
We want to prune dev dependencies as this can significantly reduce the final container size even further.
Yarn devs won't provide the ability to prune dev dependencies from the local project cache:
https://github.com/yarnpkg/berry/issues/1789
This means we need a global cache, so we can clean the cache and reinstall prod modules without re-downloading.
The default global cache and local cache are simply copies, not hardlinks, thus doubling the storage of modules.
To work around, we will construct a global cache from the local cache using hard links (requires a hacky rename).
Hardlinks created in layers different to the layer in which the file is initially created, end up as copies as well!
To work around, we recreate the global cache as hardlinks in the same layer that the local cache is populated.
As file creation and hardlinks are created within the same layer it takes no additional space.
If hardlinked files are COPY'd in independent layers, then again you end up with copies, doubling the storage.
To work around, we co-locate both the global and local cache under /usr/src, so they can both be copied at the
same time when dealing with multi-stage docker builds.

So, here in the base image we do a regular non-pnp, big 'ol node_modules install.

/usr/src/.yarn/cache (global cache) is populated with zipped packages.
/usr/src/yarn-project/.yarn/cache (project local cache) is populated with copy of zipped packages.
Packages are unzipped into various node_modules folders, taking lots of space.
We then erase the global cache, and recreate each file as a hard link, reducing the zipped package storage by half.

That's all we want to do here r.e. dependency installation. In yarn-project we will:

Copy in the rest of the workspace files.
Perform any code generation steps that require the workspace code (generate L2 contract wrappers).
Build all the code, and runs formatting checks.
Enable yarn pnp by replacing .yarnrc.yml with .yarnrc.prod.yml.
Perform a final yarn install to remove all node_modules folders and generate .pnp.cjs etc.
Create the final slim yarn-project image by copying /usr/src into a fresh image.

When we build a downstream docker image, we:

Do any project specific work in a "builder" stage.
Erase the local cache with a yarn cache clean. Files will remain in global cache due to hard link.
Do a yarn workspaces focus --production to install production dependencies from the global to local cache.
Remove the global cache from /usr/src/.yarn.
The above can be done with:
RUN yarn cache clean && yarn workspaces focus --production > /dev/null && rm -rf /usr/src/.yarn
Create final slim image by copying /usr/src into a fresh image.

At the time of writing the following 3 images have the following sizes:

yarn-project-base: 1.11GB
yarn-project: 467MB
end-to-end: 360MB

This is probably near optimal.

I still have reservations about enabling PNP as I do not feel the community is rallying around it, feels like another 'third party alt node thingy'. But I'm good with merging this if you feel it's worthwhile


Upgrade to Laravel 11, PHP 8.2.7, PHP-FPM 3.1.6

It's working and had a successful test deployement to Render.
The current version of this repo is using a dangerous unsupported version of PHP, and a very old version of Laravel. I hope that by updating it to the most current version it will encourage people to upgrade their existing projects, or to start new proejcts with current versions.

Just put up a separate PR to update our docs page


Custom Font Family

Validations

[X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that requests the same enhancement

Problem
I'm really hoping that you guys provide a setting option for us to modify the font family font size of the GUI. And also, the latex/katex math font is not working at least for my model's generation. Not technically a bug but it would be nice to see if there is a solution.
Solution
A custom css would be really beautiful.

@uncle-yi Do you have interest in setting the font outside of the one used by the IDE? My thought is that this should match the IDE font/size as much as possible, and the font size could be tweaked via VS Code as well. Let me know your thoughts.
And for the katex problem, is it not rendering, or does the font just look bad? I know that sometimes models will output without the surrounding $$ tags so that it doesn't render

@uncle-yi Do you have interest in setting the font outside of the one used by the IDE? My thought is that this should match the IDE font/size as much as possible, and the font size could be tweaked via VS Code as well. Let me know your thoughts.
And for the katex problem, is it not rendering, or does the font just look bad? I know that sometimes models will output without the surrounding $$ tags so that it doesn't render

I m currently using VSCode (1.88.1), and after I set the editor font, the code (inline and block) that has been generated has changed to corresbonding font, but the main text hasnt changed even if i use the fonted extension to alter the VSCode's UI font.
The font size do match the IDE's setting, but sometimes you do want them to have different font size for productivity or just for the pleasure (This is a bit nitpicking).
As for the katex problem, you are absolutely right. After I asked the model to use the $$ tag the font has appeared. (I was using Gemini, and also a model named moonshot by a chinese mainland provider, they both output math formula without the strict markdown syntax by default, I dont know how they manage to bypass that in their official website, but there could be some way to present that correctly even without the tag)

Custom font with font size will really be apreciated!

The font generated by the plugin is rather small, it would be nice to be able to customise the font size

Is there a way to change the font size in JetBrains IDEs? The font is super small on my screen, and zooming with Ctrl+mouse wheel or Ctrl++ wheel didn't work.


koord-scheduler: add a parameter to mark whether scheduling is allowed on node with expired nodemetric

Ⅰ. Describe what this PR does
add enableScheduleWhenNodeMetricsExpired parameter for loadaware scheduler plugin to mark whether scheduling is allowed on node with expired nodemetric.
it be used to address the following issues：

not allow scheduling when nodemetric expired：avoid pods being scheduled to nodes with high load.
allow scheduling when nodemetric expired：the actual load of the node is very low but the nodemetric is very high. It may be blocked by the filter phase,  causing the node to be unable to schedule a new pod anymore. FYI:  https://github.com/koordinator-sh/koordinator/pull/1563

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it
Ⅳ. Special notes for reviews
V. Checklist

[ ] I have written necessary docs and comments
[ ] I have added necessary unit tests and integration tests
[ ] All checks passed in make test

/lgtm

/lgtm


all: Avoid unnecessary "append" loops

In cases where append(a, b...) can be used, don't replicate the same
functionality with a for loop.
Modify the deepcopy plugin to generate code that uses the former syntax.
Fix an instance in allocator. Also change a slice == nil check to
len(slice) == 0, which caused some fun to debug problems.
@stevvooe @mrjana

LGTM

Current coverage is 55.10% (diff: 94.28%)

Merging #1633 into master will decrease coverage by 0.24%

@@             master      #1633   diff @@
==========================================
  Files            91         91
  Lines         15034      15000    -34
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           8322       8266    -56
- Misses         5620       5647    +27
+ Partials       1092       1087     -5

Powered by Codecov. Last update 9cb1eae...ae293d3

LGTM