[V3][iOS][CodeScanner] CodeScanner only captures one barcode at a time🐛

What's happening?
I have built a barcode scanner using useCodeScanner but in IOS it scans only one code at a time. It works perfectly fine in Android.
Reproduceable Code
const device = useCameraDevice('back');
const codeScanner = useCodeScanner({
    codeTypes: ['qr', 'ean-13', 'ean-8', 'upc-e', 'code-128', 'code-39', 'pdf-417', 'data-matrix'],
    onCodeScanned: (codes, frame) => {
      if (codes && codes.length === 0) {
        return;
      }
      const barcodes = codes.filter(barcode => isValidBarcode(barcode));
      if(barcodes && barcodes.length !== 0) {
          if (barcodes.length >= 1) {
console.log(barcodes.length)
           }
}
    }
  });
 <Camera
        style={styles.preview}
        codeScanner={codeScanner}
        device={device}
        isActive={isCameraActive}
        ref={rnCameraRef}
        torch={flashMode ? 'on' : 'off'}
      />

Relevant log output
I cannot attach log as it for my company but I don't see any issue in the log as well.

Camera Device
{
“hardwareLevel":"full",
   "hasFlash":false,
   "hasTorch":false,
   "id":"com.apple.avfoundation.avcapturedevice.built-in_video:0",
   "isMultiCam":false,
   "maxExposure":8,
   "maxZoom":95.625,
   "minExposure":-8,
   "minZoom":1,
   "name":"Back Camera",
   "neutralZoom":1,
   "physicalDevices":[
      "wide-angle-camera"
   ],
   "position":"back",
   "sensorOrientation":"landscape-right",
   "supportsFocus":true,
   "supportsLowLightBoost":false,
   "supportsRawCapture":false,
   "formats": {"autoFocusSystem":"contrast-detection",
         "fieldOfView":54.26699447631836,
         "maxFps":30,
         "maxISO":768,
         "maxZoom":153,
         "minFps":2,
         "minISO":24,
         "photoHeight":2448,
         "photoWidth":3264,
         "pixelFormats":[
            "Array"
         ],
         "supportsDepthCapture":false,
         "supportsPhotoHdr":false,
         "supportsVideoHdr":false,
         "videoHeight":144,
         "videoStabilizationModes":[
            "Array"
         ],
         "videoWidth":192
      }
}

Device
iPad (7th Generation) (iOS 16.2)
VisionCamera Version
3.7.0
Can you reproduce this issue in the VisionCamera Example app?
I didn't try (⚠ your issue might get ignored & closed if you don't try this)
Additional information

[ ] I am using Expo
[X] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

Using this package in a production environment is a significant risk, especially considering the ongoing issues with barcode scanning. Why is it still challenging to release a stable version with reliable barcode reading capabilities?

@hilkbahar I'm not sure if you're serious or not, but if you are serious: This package is running fully stable in multiple production apps. You can use Frame Processors to do a ton of magic, including fully custom QR code scanning with MLKit, ZXing or whatever library you prefer.
I built CodeScanner/useCodeScanner afterwards because some people found Frame Processors to be too difficult, so it is an easy to use alternative that does not require Frame Processors or Worklets.
The CodeScanner API exposes the recommended underlying native platform features, Vision on iOS and MLKit on Android.
For me, both of those APIs worked fine for scanning a simple QR code. That's what it was tested on. It also works on other types of codes.
If you need to scan multiple codes, increase performance, or whatever - you can always use a custom Frame Processor to achieve what you need.

Why is it still challenging to release a stable version with reliable barcode reading capabilities?

This is a free library you are using to gain profit for yourself/your company. If you want something fixed, pay for it and don't complain to authors and maintainers like me who do it in their free time.

@mrousavy - Thank you for the response. From custom Frame processors do you mean to use vision-camera-code-scanner which uses scanBarcodes and does v3 visioncamera supports vision-camera-code-scanner?

Yep, either vision-camera-dynamsoft-barcode-reader by @xulihang or vision-camera-code-scanner by @rodgomesc - not sure if there are more?

Ah sorry there's also vision-camera-barcode-scanner which uses MLKit on Android and Vision on iOS, by @mgcrea. This looks really stable and maintained to me.

I just tried it and you are right, it captures only one ean-13 code at a time, but can capture 4 or more qr codes at a time.
I am not sure why this is happening, but I am using the underlying platform native APIs and there's nothing I can do. This is an Apple bug, so file a bug using their bug reporter. The related class is called AVCaptureMetadataOutput, this can only capture 1 ean-13 code at a time.
Maybe it will get fixed in the next iOS update :)


New version: SDCC.SDCC version 4.0.0

Pull request has been created with Komac v1.11.0 :rocket:
Microsoft Reviewers: Open in CodeFlow

Resolves: #99198

/AzurePipelines run


jenkinsfile: Only cascade master builds

My comment here is the same as on the server repo, but it especially applies here. Given that we don't do any releases without both a pass from client and KBFS, then any failure triggered by client on kbfs-server will show up in kbfs-server tests, and the release will be blocked. But bad PRs will potentially show up since we aren't running the tests on them. So it seems you get the worst of all worlds here, where bad commits will break kbfs-server master, and you get a higher chance of a kbfs-server flake blocking releases.

@mmaxim: kbfs-server flakes should basically not happen any more. I've increased the retries to 5 on all integration tests.
Additionally, in this repo it's even more important than in kbweb to have master cascade, since we build the kbfs container from the merged client container.

@mmaxim: maybe we can have PRs run kbfs only (and not kbfs-server)?

I thought KBFS repo just used mocks for everything, so there wasn't much chance client could affect it? Either way, I'm fine with this, just wanted to bring up the point from above.


Disable synchronization of selected files

I have files on each device with Obsidian (2 PC and mobile) that I don't want to push to GitHub. On PC I use git plugin and .gitignore file for that purpose. I tried searching in settings and creating .gitignore files like on PC, but it doesn't seem to work. Is it even possible?

It doesn't look like the plugin supports excluding files. The only workaround I've seen is converting folders / files to hidden ones (by prepending "." to their name).
It isn't perfect but if you don't have any other plugin dependent on the folder / file not being hidden, it can help.


[Event Request] table 7321 "Warehouse Shipment Line" - OnBeforeValidateQtyToShipBase

We would like to add a new event OnBeforeValidateQtyToShipBase in the following trigger:
    field(22; "Qty. to Ship (Base)"; Decimal)
    {
        Caption = 'Qty. to Ship (Base)';
        DecimalPlaces = 0 : 5;

        trigger OnValidate()
        var
            IsHandled: Boolean;
        begin
            //A200F0000.Event18.03.ns
            IsHandled := false;
            OnBeforeValidateQtyToShipBase(Rec, xRec, CurrFieldNo, IsHandled);
            if IsHandled then
                exit;
            //A200F0000.Event18.03.ne
            Validate("Qty. to Ship", CalcQty("Qty. to Ship (Base)"));
        end;
    }

Event Signature:
[IntegrationEvent(false, false)]
local procedure OnBeforeValidateQtyToShipBase(var WarehouseShipmentLine: Record "Warehouse Shipment Line"; xWarehouseShipmentLine: Record "Warehouse Shipment Line"; CallingFieldNo: Integer; var IsHandled: Boolean)
begin
    //A200F0000.Event18.03
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


implement sql migrations

Closes #145
Design overview
There are three types of migrations involved in migrations:

lock.json: contains the latest version of the SQL schema. Every new generation, takes it from there and diffs the current schema inferred from the models against it.
TS_name.up.sql: an up script for a migration.
TS_name.down.sql: a down script for a migration.

Flow

Lock file is read.
Models are processed and converted to a SQL schema (tables and columns).
Diff between lock schema and current schema produces the change set for the up migration.
With the up migration, the inverse of the changes are made and this is the down migration.
With the up and down migrations and the current schema, three files are generated:

New lock with the current schema.
New up script.
New down script.

Caveats

A new struct tag for fields is introduced: sqltype. There are some cases in which we can't know which SQL corresponds to a Go type, for example, in sql.Valuers. In this case, the generation will fail and the user will have to use a custom type.
There are some cases where tracking certain changes are hard for us to do automatically, and thus, require manual change. There is a type of change named ManualChange that generates invalid SQL syntax (with a human-readable message so the user knows why the manual change is required) so the user can't run the migrations until a proper migration is done manually.
Renames can't be tracked in neither tables nor columns, so they are a DROP+CREATE instead.

Missing features

This only introduces the generation of migrations, not their execution! That will be done in a followup PR.
Once this is merged and finalized I will make a followup PR with the documentation for this feature.

Done @Serabe @ajnavarro

Thank you, @erizocosmico


Firestore startup overloads logs (to crash) when Airplane mode is on

[REQUIRED] Please fill in the following fields:

Unity editor version: 2019.3.8
Firebase Unity SDK version: 6.15.2
Source you installed the SDK: Unity Package Manager
Problematic Firebase Component: Firestore
Other Firebase Components in use: Firebase Core, Auth (and External Dep Manager 1.2.155)
Additional SDKs you are using: NA
Platform you are using the Unity editor on: Mac OSX - Catalina 10.15.6 (19G2021)
Platform you are targeting: iOS
Scripting Runtime: IL2CPP

[REQUIRED] Please describe the issue here:
When I turn on Airplane Mode on my iPhone 11, and try to launch my iOS app, it exits immediately. The same does not occur on Android, which is functioning as expected. I seem to be hitting an issue with Firestore connectivity.
2020-09-10 12:04:23.497713-0500 app[2760:677481] Task <844D54D1-564F-4973-A493-6AEBB57CA104>.<1> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The Internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x283e0ce10 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <844D54D1-564F-4973-A493-6AEBB57CA104>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <844D54D1-564F-4973-A493-6AEBB57CA104>.<1>"
), NSLocalizedDescription=The Internet connection appears to be offline., NSErrorFailingURLStringKey=https://securetoken.googleapis.com/v1/token?key=AIzaSyDZ9NJBOqY4Zxv3wDEG7wdV-uwZfxSmVPU, NSErrorFailingURLKey=https://securetoken.googleapis.com/v1/token?key=AIzaSyDZ9NJBOqY4Zxv3wDEG7wdV-uwZfxSmVPU, _kCFStreamErrorDomainKey=1}
libc++abi.dylib: terminating with uncaught exception of type firebase::firestore::util::FirestoreInternalError

This is the full log.
I have a system enabled called LunarConsole which allows me to display Debug logs/exceptions on mobile. This has been vital as I’m developing an ARFoundation app and Unity still hasn’t released a native AR Remote solution, making debugging challenging. As far as I know it essentially listens for logs. When LunarConsole is uninstalled, the bug does disappear. I found an old bug on the Flutter repo that mentions a potentially similar issue that occurs when “the firebase plugin just happens to log enough in airplane mode to trigger the crash.” Could Firestore be logging excessively and blowing up LunarConsole?
Steps to reproduce:
Have you been able to reproduce this issue with just the Firebase Unity quickstarts (this GitHub project)?
Quickstart doesn’t utilize the same Unity/FB versions as our environment.
What's the issue repro rate? (eg 100%, 1/5 etc)
100%
What happened? How can we make the problem occur?
This could be a description, log/console output, etc.

Install Firestore
Install LunarConsole (asset store) (github)
Turn on Airplane mode
Launch the app on iOS.

If you have a downloadable sample project that reproduces the bug you're reporting, you will
likely receive a faster response on your issue.
Relevant Code:
The code is simply including / engage with Firestore while in Airplane Mode

Hi @jonahgoldsaito,
Most of the warnings logged are from Firebase Auth in its inability to reach a particular service, but generally speaking there's nothing that sticks out as a critical error here.  The first impression is that the logs don't seem to be too large, to be honest.
You might want to (enable more logging)[https://firebase.google.com/docs/reference/unity/class/firebase/firestore/firebase-firestore#class_firebase_1_1_firestore_1_1_firebase_firestore_1aad122e86847cc114fb99685c259b9918] to see if there's other Firestore issues taking place.  You can do this via:
FirebaseFirestore.LogLevel = LogLevel.Debug;

With added logging volume it would be interesting to see if Lunar Console crashes at roughly the same amount of output or not.


Fix: 8824 Edit Icon Alignment fix in the links page.

Fixes Issue

Closes #8824
Changes proposed

Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[ ] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Screenshots

How it will look after merging this PR

How it looks now

Note to reviewers
This modification does not impact other pages using the Edit component as the component now accepts a new prop i.e className for modifications instead of changing the exisiting classes.

Please take into consideration the comments here.


Some issues with GoogleDrive, OneDrive for Android and iOS

It seems there is merged conflicts to be solved first

I'm a little bit confused. What should I do?

I would like to get all the fixes for issues that are on the original repo into this one. I am working on setting up the CI and CD for this library and then this will become the primary source for the plugin which can be maintained from here on out.
Since another PR is already merged, I figure your PR isn't directly mergeable. I would like to ask you to implement your fix into this code as well, so we will have a complete and working library ASAP.

It means I should fork your repo, change the code and start pull request? Correct?

Correct

Ok. I'll try


Typo in project description

Step by steps ...

should be

Step by step ...

🎵 day by day 🎵

Thanks!!
Sadly I don't have permission to change this :(
@aelnaiem can you either up my permissions or fix the typo? Thanks :D

@evykassirer @aelnaiem How is this going? The list of opened issues is growing; let's try to close some up :-)

ah, ahmed fixed it - great!


Lab 06: Creation of Private IP fails

In terraform/Lab_06_Site_to_Site_Virtual_Private_Network, terraform apply fails with:
│ Error: 404-NotAuthorizedOrNotFound, Authorization failed or requested resource not found.
│ Suggestion: Either the resource has been deleted or service Core Private Ip need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/core_private_ip
│ API Reference: https://docs.oracle.com/iaas/api/#/en/iaas/20160918/PrivateIp/CreatePrivateIp
│ Request Target: POST https://iaas.us-ashburn-1.oraclecloud.com/20160918/privateIps
│ Provider version: 6.18.0, released on 2024-11-13. This provider is 4 Update(s) behind to current.
│ Service: Core Private Ip
│ Operation Name: CreatePrivateIp
│ OPC request ID: 5275e9603840f884ffe9c5f9f2fea3ba/174615259E65AC23559169AAFEC0FB32/15865867E58242384B2290F1A2D25755
│
│
│   with oci_core_private_ip.IAD-NP-LAB06-VMCPE-Private-IP,
│   on ashburn_compute.tf line 85, in resource "oci_core_private_ip" "IAD-NP-LAB06-VMCPE-Private-IP":
│   85: resource "oci_core_private_ip" "IAD-NP-LAB06-VMCPE-Private-IP" {

Updated vnic_id parameter to reference VNIC of VNIC Attachment instead of OCID for VNIC Attachment.


[concurrency] Task cancellation and deadlines

rdar://70141994
Work in progress adding API stubs for task cancellation and deadlines.
Not all of the API is fleshed out here, and we may need to discuss the general usage patterns still a bit.
(~Depends on https://github.com/apple/swift/pull/34391 which can be merged at will.~ Merged and rebased)

@swift-ci please smoke test

@swift-ci please test

@swift-ci please smoke test

There's a failure but no way it's related to this PR:
15:08:18 ******************** TEST 'Swift(linux-x86_64) :: Sema/type_checker_crashers_fixed/rdar27830834.swift' FAILED ********************
15:08:18 Script:
15:08:18 --
15:08:18 : 'RUN: at line 1';   /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/buildbot_linux/swift-linux-x86_64/bin/swift-frontend -target x86_64-unknown-linux-gnu  -module-cache-path /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/buildbot_linux/swift-linux-x86_64/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache -swift-version 4  -ignore-module-source-info  -typo-correction-limit 10  /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/swift/validation-test/Sema/type_checker_crashers_fixed/rdar27830834.swift -typecheck
15:08:18 --
15:08:18 Exit Code: 1
15:08:18
15:08:18 Command Output (stderr):
15:08:18 --
15:08:18
/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/swift/validation-test/Sema/type_checker_crashers_fixed/rdar27830834.swift:4:15: error: type 'Dictionary<String, String>.Element' (aka '(key: String, value: String)') cannot conform to 'Hashable'
15:08:18 _ = "\(d.map{ [$0 : $0] })"
15:08:18               ^
15:08:18 /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/swift/validation-test/Sema/type_checker_crashers_fixed/rdar27830834.swift:4:15: note: only concrete types such as structs, enums and classes can conform to protocols
15:08:18 _ = "\(d.map{ [$0 : $0] })"
15:08:18               ^
15:08:18 /home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/swift/validation-test/Sema/type_checker_crashers_fixed/rdar27830834.swift:4:15: note: required by generic struct 'Dictionary' where 'Key' = 'Dictionary<String, String>.Element' (aka '(key: String, value: String)')
15:08:18 _ = "\(d.map{ [$0 : $0] })"

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test


Add prometheus metrics + grafana charts for node latency

Indexer speed (esp. when reindexing) depends on node responsiveness a lot. Instrument calls (GetBlock(), GetEvents(), etc) in storage/oasis/{consensus,runtime}.go to measure how long the node takes to respond, and plot them in grafana.

https://app.clickup.com/t/3ufk8kr

The existing blockfetch metrics are sufficient at this time:
https://github.com/oasisprotocol/nexus/blob/12a1ea7e900205d33f647b36574a623ec2057009/analyzer/consensus/consensus.go#L332
https://github.com/oasisprotocol/nexus/blob/12a1ea7e900205d33f647b36574a623ec2057009/analyzer/runtime/runtime.go#L191


Accounting for unknown (because of missing data) but probably existing power

What's needed?
When there's no data for some components, we should stop assuming that everything is None, because that could be very costly in big installations.
When PV inverters are not responding, we need to assume they're probably operating at maximum power.
We need to have an estimate of what could be going on with these PV inverters when they're not sending data.  This should be available from the PVPool.
Then this needs to be extended to other pools.
Proposed solution
No response
Use cases
No response
Alternatives and workarounds
No response
Additional context
No response

When PV inverters are not responding, we need to assume they're probably operating at maximum power.

I would say what the assumption is should be left to be decided by the user, because different applications might need to make different assumptions. For example when shaving peaks, you probably want to assume no power is generated by missing PVs and discharge batteries more preemptively to avoid peaks as much as possible.
My suggestion is to make what the assumption is for missing data configurable per pool, so the user creating the pool can say this pool should assume a component is providing maximum power or not power depending on what its safety requirements are.

This is also related to:

https://github.com/frequenz-floss/frequenz-sdk-python/issues/1021

They are about different types of missing data (or data quality), I'm not sure if once this issue is resolved, #121 is still necessary. @cwasicki ?

(fixed your link)
For the case of unreachable inverters this one might be sufficient, but I see #1021 useful to more problems with the data.


Unable to override login credentials for already initialized database

I run the command:
docker run -it \
                     -p 5432:5432 \
                     -e POSTGRES_PASSWORD=first \
                     -e POSTGRES_USER=first \
                     -v `pwd`/temp:/var/lib/postgresql/data \
                      postgres:16

And I can log into the database with the username first and the password first:
~$ PGPASSWORD=first psql -h <IP_ADDRESS> -U first -d first
psql (15.6 (Debian 15.6-0+deb12u1), server 16.1 (Debian 16.1-1.pgdg120+1))
WARNING: psql major version 15, server major version 16.
         Some psql features might not work.
Type "help" for help.

first=#

I then stop that container and run the following command in the same directory:
docker run -it \
                     -p 5432:5432 \
                     -e POSTGRES_PASSWORD=second\
                     -e POSTGRES_USER=second \
                     -v `pwd`/temp:/var/lib/postgresql/data \
                     postgres:16

And I cannot log in as second!! (I can still log in as first)
~$ PGPASSWORD=second psql -h <IP_ADDRESS> -U second
psql: error: connection to server at "<IP_ADDRESS>", port 5432 failed: FATAL:  password authentication failed for user "second"

@CitationGraham that is by design. This is how database initialization works.

@LaurentGoderre I am not sure the inability to override the username or at least the password after first running makes any sense in a Docker context...
You can hack around with the ENTRYPOINT to accomplish the same outcome; the expectation is to be able to do it more simply and cleanly with the same ENV used earlier in the process.

did you tried doing that without the volume?

See also https://hub.docker.com/_/postgres, especially:

Warning: the Docker specific variables will only have an effect if you start the container with a data directory that is empty; any pre-existing database will be left untouched on container startup.

I agree that not being able to reconfigure is a bit unusual, but a database without persistence is even more unusual and users/authentication are a property of the initialized database (and maintained via the data directory/database itself after initialization). :smile:

@CitationGraham creating another user wouldn't work in your use case?


Where is DaggerJockeyComponent?

Can't find DaggerJockeyComponent referenced in JockeyApplication.
Also, can you please direct me to a tutorial on these new libraries like inject, dagger etc. Changes made in past few weeks are too big for me to follow.

Don't worry, Dagger and RxJava are a lot to take in. Hopefully this should clear some things up:
Where is DaggerJockeyComponent?
DaggerJockeyComponent is an automatically generated class built by Dagger at compile time with apt. It contains a ton of boilerplate code to implement the JockeyComponent interface.
Dagger
Dagger is a dependency injection library. A dependency is anything that another class needs to work. For example, there are a ton of Activities and Fragments in Jockey that need access to the music library. They don't really care where it comes from or how it works, just that it returns usable data. By using dependency injection, we can abstract the old Library class with its static helper methods into a data store object (in this case MusicStore and PlaylistStore). This allows us to easily swap the data source in the future and to create unit tests for the UI logic (we could accomplish this by changing the component to a fake data source we can test against).
There's a couple of really important interfaces that Dagger uses to get injection working. JockeyGraph is a basic interface that tells Dagger what classes it can inject dependencies into. JockeyGraph doesn't do anything in and of itself – it's just a convenience interface to make it really easy to make new Components. All of the classes that JockeyGraph has inject() methods for have protected fields annotated with @Inject. They can't be private, because Dagger doesn't use reflection to inject dependencies. When the class has been set up and we call inject(this), all of those fields will be populated by Dependencies from our Component.
JockeyComponent is another interface that extends JockeyGraph. JockeyComponent is annotated with modules including ContextModule, MediaStoreModule, and LastFmModule, which Dagger uses to figure out how to create the dependencies. When building the project, the Dagger compiler creates a new class, DaggerJockeyComponent, that implements JockeyComponent. This generated class contains all the code to satisfy dependencies annotated with @Inject by implementing all of the inject() methods inherited from JockeyGraph. Because this class is generated, there's no reflection at runtime, and it's as fast as if you had written the code yourself.
Dagger figures out where to get dependencies from using Modules. Jockey has a few of them as I've mentioned before, but the gist is that they provide instances to Dagger with methods annotated as @Provides. We can add the @Singleton annotation on top of that to make sure that we only create one instance of any given object. For example, all of Jockey's data stores are annotated as singletons so that we don't waste memory or time generating a new data store every time it's needed in the UI.
RxJava
RxJava is a Java implementation of the Reactive programming pattern (and people really love explaining it with marble diagrams, which I found pretty unintuitive when learning it for the first time, so I'll avoid them completely here).
In RxJava, everything is based around the Observable class. An Observable can be thought of as a stream of objects. At any given time, the Observable can emit a new item, emit an exception, or complete by terminating the stream. In both the event of an exception or completion, the Observable can never emit new items.
To be notified about an Observable's status, you subscribe to it with Observable.subscribe(Subscriber), Observable.subscribe(onNext, onError, onComplete), or something similar. Jockey uses the the second method, where each of the parameters is a functional interface (which we can supply using fancy Java 8 Lambda Expressions thanks to Retrolambda). As you may have guessed, any time the original Observable gets a new item, our onNext callback gets invoked – same thing with onError being called when an exception is thrown and onComplete when the Observable won't be giving us any new items (or when we unsubscribe ourselves).
This is great and all, but why bother? Well, for starters, we can get rid of the gross Library.OnLibraryChangeListener callback. If we treat our music library like an Observable<List<Song>> (an Observable whose object stream comprises of Song lists), we'll get new library data as soon as it gets sent to our Observable. The other reason that RxJava is used so often is because of multithreading. RxJava can handle passing data between threads with no implementation apart from telling it what threads to use. We can do this by calling Observable.subscribeOn(Scheduler) to change what thread the work happens on, and Observable.observeOn(Scheduler) to change what thread the results are sent to (which is important since we can only update the UI on the main thread). Jockey doesn't do this yet, but it would be very straightforward to migrate database calls to the MediaStore to a background thread in the future.
RxJava Operators
An operator is a way of changing the object stream produced by an Observable. There's a ton of these and I can't even begin to go into detail with all of them, so here's a few of the more common ones: Observable.map(...), Observable.flatMap(...), Observable.skip(int), Observable.take(int), and many others. Notably, map and flatMap are really flexible and can be hard to differentiate between them, so if you're interested, I recommend a guide called RxChisel that talks about what map and flatMap do and gives practical examples of each.
Another important operator is Observable.compose(...). With the help of RxLifecycle, we can bind our Observable to the corresponding Activity's lifecycle to ensure that the Subscription doesn't leak the activity, and that it won't try to do anything dangerous like updating the UI after the activity has been destroyed. This can be done in one line with Observable.compose(bindToLifecycle()) with no extra work on our part.
Types of Observables
There's a bunch of basic Observables that are included in RxJava. You can make your own, but it's advised against because it's really easy to get wrong, and you can do so much using the tools already at your disposal. The most basic Observables can be made with Observable.empty(), an Observable that calls its subscriber's onComplete method as soon as it's subscribed to; Observable.error(Throwable), which calls its subscriber's onError method as soon as it's subscribed to, and Observable.just(Object), which calls its subscriber's onNext method as soon as it's subscribed to, and then completes itself.
Another powerful kind of Observable are Subjects. A Subject is both a Subscriber and an Observable. The behavior of a Subject depends on which type you use, but right now Jockey only uses the BehaviorSubject, which emits the most recent item it has observed and all subsequent observed items to each subscribed Observer. Jockey uses BehaviorSubjects in MusicStore and PlaylistStore as a way to keep the most recent library. When the list of all songs, for example, is requested, we can return the corresponding subject so that it can be subscribed to. If we haven't loaded it yet, we can start loading the music library and post the status to the returned BehaviorSubject which will update the UI as soon as data is received. If the music library changes for any reason, updates to the BehaviorSubject will call the subscriber's onNext method, which can go ahead and update the displayed data with no extra work.

Hopefully this clears things up. There's a ton of other resources for the two libraries I talked about above, and RxJava especially can be incredibly powerful compared to traditionally firing up an AsyncTask or something.
Feel free to post other questions about Jockey related to Dagger or RxJava here, or start a separate thread.


Min and max validation rule

Add a new validation rule to support minimum and maximum values
Specs

https://github.com/Big-Life-Lab/PHES-ODM-Validation/blob/main/validation-rules/less_than_min_value.md
https://github.com/Big-Life-Lab/PHES-ODM-Validation/blob/main/validation-rules/greater_than_max_value.md

From reading this spec, and reading the cerberus spec, it seems like Cerberus's min/max already is what we want. "The ODM dictionary is currently written so that the defined minimum values are inclusive." min is also inclusive, meaning that min=10 would include 10. Am I missing something?

It looks like cerberus uses gt and lt and not something like gte and lte, https://docs.python-cerberus.org/en/stable/validation-rules.html#min-max? Unless I'm missing something.

Yeah, but I'm guessing they are used the other way around, for example: if x < min: return error, where < is __lt__. I can test it to confirm.

Yup, why don't you test it and let me know

Blocked by #65

I've confirmed that Cerberus uses inclusive min/max. It can easily be tested with the following piece of code:
from cerberus import Validator
v = Validator()
schema = {'age': {'type': 'integer', 'min': 1, 'max': 2}}
assert v.validate({'age': 1}, schema) is True
assert v.validate({'age': 2}, schema) is True


Adds better grid styling, tooltips, and additional information to Grid and panels

Adds additional information to each grid element, tooltips, and the current snow coverage. There is also a preliminary status panel for the overall simulation.

 Evo: Improve information provided by the basic grid view


Add AuthCall variant to CallScheme

Related https://eips.ethereum.org/EIPS/eip-3074

@Rjected does this enum need to be used in the handler in Alphanet to be useful?

@Rjected does this enum need to be used in the handler in Alphanet to be useful?

It's not used in the handler, no, I'm not sure if this is needed actually

it looks like it exists in the branch of revm used in alphanet-foundry:
https://github.com/wevm/revm/blob/66a499fc281b493a34ee42069309cf7aed363052/crates/interpreter/src/inner_models.rs#L109
warning: old revm
Still need to understand how this is used to determine if it's necessary

I will close this for now, and we can always revisit it in future.
There is possibility that EIP-3074 is not going in


ARROW-10405: [C++] IsIn kernel should be able to lookup dictionary in string

This is to resolve ARROW-10405.

Thanks for doing this. Can you add tests on the C++ side? I expect tests for IsIn and IndexIn, with non-trivial dictionary arrays, also in cases where not all dictionary values are referred by indices.

Added for string and float32 types. Should we check more?

Also note that the current implementation (in scalar_set_lookup.cc) is suboptimal as it decodes the dictionary array and runs the set lookup for each decoded dictionary value. It would be more efficient to compute and cache lookup results for distinct dictionary elements. That can be for a later JIRA, though.


Check for Forge tag for wheat

Allows for more cross-mod compatibility with other variants of wheat. Perhaps reverting the name of flour to "flour" instead of "wheat flour" would be more inclusive as well.

In hindsight, dropping wheat seeds may be problematic if the Forge tag for all wheat variants is checked. It is up to you though.

Right, I think it's better if we add individual compatibility recipes for other mods' crops here. That way we can drop the correct seeds and potentially other specific drops.


Extract all logic that is not game specific from ClientGame.js

ClientGame.js is currently mostly a dump for everything that the client should do and it includes helper functions such as lerp and interpolate , network functionality and game logic in the same file.
I can see at least 2 different concerns which should be abstracted out of the ClientGame.js:

Helper Methods: this includes functions like interpolate(), lerp() etc. I think the usage of those should be optional because I may be using a client game library which most probably already provides this functionality for me.

Abstract Client Methods: these would be moved to AbstractClientGame and called by CustomClientGame.js. This would serve to abstract the interface with the engine so it would include things like all the network layer code . For instance:  CustomClientGame.updateInput() would have my custom game logic to process game inputs because this changes on evry game. After the custom code is done, CustomGame.updateInput() would call AbstractClientGame.updateInput(inputData) which would call network.sendData(inputData), thus the game logic doesn't need to know how to interface with the network layer

I will provide more suggestions on what could be extracted when I have a better understanding of the code but I'm having a hard time figuring out how it all works.
I apologize for the long post, thanks for reading!!  :smiley:

Thanks for the suggestions :smile: . I was going to start with issue #2 you posted. I'll probably do some refactoring and take these thing into consideration.


Support Baysor>=7.0.0

Hi @lguerard, I added a test to check if the cell IDs are string
In the new 0.7.0 version, the cell IDs are strings, and we need to use the obs_names instead of CellID to compare to the loom matrix
Let me know if this works for you!

Thanks a lot for looking into that !
And thank you for including the dask backend so make it easier to do parallelization. I just tried with write_patches_cells but I'm getting an error, I'm not familiar at all with dask so I'm not sure what it means..
RuntimeError: Can't redefine method: __getstate__ on class: __torch__.torch.utils.mkldnn.MkldnnBatchNorm (of Python compilation unit at: 000001A8C4E17ED0)

Could you open an issue with a detailed error log?
Since I'll merge this PR soon, it will be better to keep track of this issue

I'm reinstalling my environment right now, I'll create an issue if I encounter it again.
But anyway, you removed the settings file and the dask support for now no ?

Merging this PR, thanks @lguerard!
And please let me know regarding the issue with the dask parallelization (see my comment above)


Make autocomplete-java package extensible

Make autocomplete-java package extensible so that other plugins may extend functionality of core autocomplete-java. Implement some of the autocomplete-java features as separate packages.

See #14


Re-Introspection

We want to remove one of the remaining big pain points of Introspection: If you manually adjusted your data model after introspection (to rename fields or models, or add default IDs for example) these changes are currently lost once you introspect again ("re-introspect", for example, because your database was changed manually or migrated by another tool).
This project consists of many individual problems that we are tackling one by one:

[x] https://github.com/prisma/prisma/issues/2503: Keep manual changes to relation field names
[x] https://github.com/prisma/prisma/issues/2545: Keep @map and @@map
[ ] https://github.com/prisma/prisma/issues/2534: Keep defaults that are not persisted in the database
[ ] https://github.com/prisma/prisma/issues/1997: Keep order of models and fields of existing Prisma schema

Some more issues are listed related to this under: https://github.com/prisma/prisma/labels/topic%3A re-introspection

With 2.1.0 we shipped the experimental flag --experimental-reintrospection  that can be used in npx prisma introspect --experimental-reintrospection. Behind that flag we implemented the first two pieces of Re-Introspection, https://github.com/prisma/prisma/issues/2503 and https://github.com/prisma/prisma/issues/2545

Thanks for putting the effort into this feature. It is greatly appreciated!
We have only had a few issues with re-introspection:

Renamed relations which is fixed and working in v2.1.1
@updatedAt is not maintained. I thought there was an issue for this but I can't seem to find it now. Similar to the @map issue.
Error with renamed relations to the same table. I have created an issue and explained it here: https://github.com/prisma/prisma/issues/2844

I tested --experimental-reintrospection on the database @nikolasburk posted in another issue, where he used @@map and renamed relation fields: https://github.com/prisma/prisma/issues/2425#issuecomment-649709485 All changes were kept 🚀

So far it works great for model classes. It would be great if can keep the changes made to enum class name as well.

When using  --experimental-reintrospection flag, I got [introspection-engine\connectors\sql-introspection-connector\src\re_introspection.rs:346:25] called 'Option::unwrap()' on a 'None' value error, without it, it run as expected.

Can you please create a new issue @Akxe and include the full schema and information we need to reproduce this? Thanks.

@janpio Here you go! https://github.com/prisma/prisma/issues/2893

When using the reintrospection when the schema is located in a custom directory, the dotenv from the same directory is not read by the Prisma CLI.
https://github.com/prisma/prisma/issues/2924

Quick update:
We just merged the last planned feature here 🚀
If you install the dev version with npm install @prisma/cli@dev you will get the state we are planning to launch with 2.6.0 as part of the prisma introspect functionality. Further improvement would be merged into the main feature then in the future directly if we find anything else to do.
For now you can still test it with: npx prisma introspect --experimental-reintrospection
It would be great if you all could test this again, before we go into the final integration with https://github.com/prisma/prisma/issues/3331 and see if your use cases are covered now - and let us know if not.
(Pinging some people from this and other related issues for visibility: @Akxe @ExiaSR @theneshofficial @kennytraction @theRocket @louisrli @harryttd)

Sounds like you're on the tail end of dev work here, but any chance of getting eyes on #3076? tl;dr - reintrospection is incorrect in a particularly bad way when using partial unique indexes, having Prisma ignore them rather than incorrectly assume they're globally unique would be a large improvement in usability for us.

@kennytraction Ahem, there should be a 2.6.0-dev.11 now - seems I was to quick to post here.

@kennytraction Ahem, there should be a 2.6.0-dev.11 now - seems I was to quick to post here.

No problem. With that build it seems all of our issues have been addressed.
Thanks!

Finished via https://github.com/prisma/prisma/pull/3445 which closed https://github.com/prisma/prisma/issues/3331 🚀


First Version suggestion

Hi,
I saw in the first version storing of index is done (Store to levelDB or any other db system #189) however it has been dropped due to slowness.

How much memory we would need if we need to load 1GB of data to index and load?
If i store the index in indexeddb or watermelondb or pouch DB of 1GB of data, will it will be slow on retrieval?
On retrieval do we need to have the equivalent amount of RAM?

How much slowness and data you have tested in the first version, any idea.
Please advise we want to build a search engine which would have maximum of 1 GB of .txt file  which needs to be stored and searched for a keyword with type ahead.
Thanks.

Hi @prog20901,
Since lunr.js is an in-memory search index, I think 1GB of data would probably be too much for a user's browser to handle.  Lunr's serialization and deserialization operates entirely on the level of the index (rather than individual terms in the inverted index), so as far as I know, it wouldn't be possible to only load parts of the index into memory as needed.  Have you considered a search database such as ElasticSearch for your needs?

I mean, that's certainly possible - I think it would probably be a substantial amount of work, though, and I don't know how performant it would be.
Let's back up a step, though - what are you trying to build that requires a 1GB full text search index that lives entirely within a user's browser?  Maybe we can try to find another solution!


Add more file-like methods to the python bindings

This adds the following methods to the File type in the Python bindings:

flush
readable
writable
seekable
closed
readinto

For AsyncFile, the following were added:

readable
writable
seekable
closed

Manually testing was done locally, but I couldn't get the unit tests to run due to environment dependency issues. Hopefully CI will trigger here.

This partially addresses #4384. It assumes that all readable services support seeking, which may not be valid, but it mirrors the implementation seek.

It assumes that all readable services support seeking

Can you use https://docs.rs/opendal/latest/opendal/struct.Capability.html#structfield.read_can_seek?

This is an ongoing RFC related to this: https://github.com/apache/opendal/pull/4382

I've added the capability check to the Python bindings, but is there anything specific I should do w.r.t. the RFC? Or is it just a blocker for this PR?

Or is it just a blocker for this PR?

After this RFC been implemented (which is already going on), we might need to change the way of implement File in python.

Hi, @mobiusklein, sorry for the long waiting. I feel like this PR is good to go. Would you like to fix the CI issues so we can get it merged?

Is it correct to say that seek operations are now supported universally since the behavior is no longer described by the Capability type? I don't see any documentation about it except for an outdated note in https://opendal.apache.org/docs/rust/opendal/struct.Capability.html#naming-style.

Is it correct to say that seek operations are now supported universally since the behavior is no longer described by the Capability type?

Yes!

Thank you. I've refactored the bindings to reflect that. The Capability struct the File types carry around now probably is no longer relevant but I left it there for the future. I'll have to re-digest what the capabilities mean but on my first pass I don't see a way they'd intrude into the Python API.

Things appear to be all set now. Thank you.


Make main button moveable

http://steamcommunity.com/workshop/filedetails/discussion/583429740/341537671987505705/

Done in 1.8.0


Node-sass install breaks CRA with TS template

Describe the bug
When installing node-sass into a CRA with typescript project, in usually integrates node-sass automatically for you. Recently mkdirp in the deps is failing and not removing certain files. This is causing the entire module tree to break.
Did you try recovering your dependencies?
I did, and it resolves it, so this is a CRA issue not node-sass.
Environment
Windows 10 Home, version 2004 build 19041.
Chrome browser.
Node-sass version: 4.14.1
CRA version: 3.4.1
NodeJS version: 13.9.0
Steps to reproduce

Run: npx create-react-app  --typescript
Run: npm i node-sass
Run: npm run start (alternatively VSC recognises the errors in any import of react)

Expected behavior
Node-sass should automatically install safely and allow the use of SCSS files.
Actual behavior
Installs after a very long period, but leaves a plethora of errors that mkdir (a dependency of CRA or node-sass) is unable to remove files? The files are some other modules. I would post a picture but it's crashing my PC at the minute when I attempt to install it freshly.
Reproducible demo
As mentioned above, it's literally crashing my PC unfortunately. If anyone could create the reproducible demo by just simple creating cra with TS and then install node-sass and running the app I'd greatly appreciate it!

Try installing "sass" instead of "node-sass", it is intended as a 100% compatible version.

Try installing "sass" instead of "node-sass", it is intended as a 100% compatible version.

Ahh ok, I've used node-sass previously with no issues. But thanks for the update, appreciate it!


CH EMED Extension Substitution - ValueSets not equal to CDA (Oliver Egger, ahdis)

ch.fhir.ig.ch-emed#1.0.0 /StructureDefinition-ch-emed-ext-substitution.html
Substitution of the medication
Binding: ActSubstanceAdminSubstitutionCode (preferred)
Points to the HL7 V3 ValueSet: https://terminology.hl7.org/2.1.0/ValueSet-v3-ActSubstanceAdminSubstitutionCode.html which contains the code F (Formulary), the code F is not within https://art-decor.org/art-decor/decor-valuesets--ch-pharm-?id=2.16.7<IP_ADDRESS>.1.11.81&effectiveDate=2020-07-10T00%3A52%3A42
Oliver Egger, ahdis

Value “F” from the substitution value set was removed because there was no usage in Switzerland,

Feedback eHS: Wert "F" kann wieder aufgenommen werden


ubuntu-image: Make root README.md better for pkg.go.dev.

Background
The pkg.go.dev site auto-imports info from a (participating) repo’s main README. The page generated for the ubuntu-image project is at pkg.go.dev/github.com/canonical/ubuntu-image.
Task
In order for the 'landing page' on pkg.go.dev to provide useful info, the repo's README should be modified. The README needs to serve two purposes:

Introduction to the project and meta-info about the repo (i.e. what you would expect from a typical repo README).
Getting started instructions and links to relevant info for visitors of the page on pkg.go.dev.

Additionally, the README should feature a 'Snap Store card' linking to the ubuntu-image page on the Snap Store (see the Create embeddable card button on that page).

Nice. Let me know if you need anything.

One note: Another contributor has already started on a related ubuntu-image issue (#20), and they submitted a PR that touches the README file. In order to avoid any dup[lication of efforts and/or potential conflicts, I suggest you either have a look at the pending PR, or wait until it's merged.

Thank you for the update @rkratky. I will work on another issue pending the merging of the other PR.

@activus-d The PR that could potentially cause conflicts has been merged, so if you'd like to pick this issue up, now's the time :-)

Thank you for the update @rkratky. I will start working on the issue.

Thank you for the update @rkratky. I will start working on the issue.

Cool. Don't hesitate to let me know if you need anything!

@rkratky I want to start working on the issue now but I don't think I have the permission to assign issues to myself as I can't find the assign yourself button.

Oh, I didn't realize that. Let me do that.

@rkratky Is the information contained here suitable for getting started?

@rkratky I've made a pull request to solve this issue and it's now ready for your review.

Fixed by https://github.com/canonical/ubuntu-image/pull/210


Add eBPF map information to the generated elf file

The following steps may be involved:

Add eBPF map information to the map segment
Add the relocate information of eBPF map
Write test cases, involving a simple eBPF program, using an eBPF map, and using libbpf-rs to load and pass.

For details, please refer to the logic of Libbpf processing. The functions that may be involved in libbpf include bpf_object__elf_collect (collecting elf information, such as eBPF program instructions, map information, relocation information, etc.) and bpf_object__relocate_data (mainly relocating map information, that is, the map's fd )

/assign

This part seems complicated. I'll see if the libbpf can provide a method to construct a bpf_program instance to facilitate our loading of the eBPF program.
[1]<EMAIL_ADDRESS>


ozw: Removed add_dependencies call from ozw CMakeLists

Small change to get rid of a warning in newer cmake versions.
Versions of cmake (>= 3.0) throw a warning on the add_dependecy
method for non-existant dependencies (add_dependency call before
target_link_libraries call).
Removed the call to add_dependency since target_link_libraries should
provide the same functionality.
Signed-off-by: Noel Eck<EMAIL_ADDRESS>

Should be extended to Modbus drivers too, since they also link externally.

Merged!


Compile stuck at optimize chunk assets

I've started having problems to compile my build when --optimize-minimize is enabled.
I see that I've reached +/-205 build modules, and then the compilation gets stuck at 80% optimize chunk assets for a very long time (+40minutes).
If I wait enough, it ends up node failing with "fatal error: js allocation failed - process out of memory"
I guess it has to do with the uglifyjs plugin as I said this only happens when using --optimize-minime flag (When compiling through my grunt task, this also fails).

I've just come across the same problem with a large project with 26 entry points. If I disable source maps it works but hangs with source maps enabled. My current hypothesis is that because webpack keeps all the sourcemaps in memory as it is compiling it results in node hitting the 1.4Gb limit in v8. I'm not familiar with how the internals work but can the maps not be flushed to the files on disk instead of keeping them in memory?

I've managed to solve the issue by externalising some of the big libraries (React and jQuery). That seems to have reduced the memory usage considerably and I can now compress the bundles with source maps.

Probably it's sufficient to flag libraries with noParse

@sokra so I've created a fairly contrived example repo that manages to reproduce the problem here: https://github.com/jkimbo/webpack-memory-issue
It has 20 entry points each of which requires fairly large libraries and then it runs it through the uglify plugin with sourcemaps. I realise it is in no way optimised but hopefully it can help debug this issue.

I am seeing a similar issue as well.

I have the same problem without UglifyJS (continuous development build). The problem occurs after a couple of rebuilds and I believe I have an idea of where it leaks memory.
In my case, the count of ModuleNotFoundError objects created is growing at an extremely high pace. That error is thrown because of moment.js so I excluded it from parsing (noParse) and as a result the heap size stays stable (instead of growing further). If I read the dump in node-inspector correctly, it might be in the cache object of the Compilation module.
That's as much as I could find out so far.

You can try to disable the cache with cache: false.

Having the same.

Ended up downgrading webpack-dev-server to 1.7.0 and webpack to 1.7.3 to make it work.

Regarding the original issue:
new UglifyJsPlugin({
  sourceMap: false
})

should help...

Have the same issue here, without UglifyJsPlugin.
Do you have any idea when can this be fixed?
anyway. Thanks for the webpack! incredible pice of art.

Same issue, sourceMap: false doesn't make any difference, removing UglifyJS plugin helps.

I don't think it's UglifyJS or memory; my webpack config suddenly started hanging at only module 67, when it was successfully building over 400 modules before.  Ironically it still works with webpack dev server and various plugins; but my config that's hanging has no plugins.

Here's my config that hung it.
webpack.config.base.js (if I use this config by itself it doesn't hang):
var path = require('path');
var webpack = require('webpack');

module.exports = {
  devtool: 'source-map',
  entry: [
    'babel/polyfill',
    './app',
  ],
  output: {
    path: path.join(__dirname, 'dist'),
    filename: 'bundle.js',
    publicPath: '/assets/',
  },
  resolve: {
    extensions: ['', '.js', '.jsx'],
  },
  module: {
    loaders: [
      {
        test: /\.jsx?$/,
        loader: 'babel?stage=0',
        exclude: /node_modules|vendor/,
      },
      {
        test: /\.css$/,
        loader: 'style-loader!css-loader'
      },
      {
        test: /\.sass/,
        loader: 'style-loader!css-loader!sass-loader?indentedSyntax&outputStyle=expanded&' +
        "includePaths[]=" +
        (path.resolve(__dirname, "./node_modules"))
      },
      {
        test: /\.scss/,
        loader: 'style-loader!css-loader!sass-loader?outputStyle=expanded&' +
        "includePaths[]=" +
        (path.resolve(__dirname, "./node_modules"))
      },
      {
        test: /\.(png|jpg)$/,
        loader: 'url-loader?limit=8192'
      },
    ],
  },
};

webpack.config.js:
var path = require('path');
var webpack = require('webpack');
var config = require('./webpack.config.base');
var _ = require('lodash');

var getWindowReact = path.resolve(__dirname, 'getWindowReact');

module.exports = _.assign({}, config, {
  resolve: _.assign({}, config.resolve, {
    alias: _.assign({}, config.resolve.alias, {
      'react$': getWindowReact,
      'react/addons$': getWindowReact,
    }),
  }),
});

getWindowReact.js:
// my boss wants me to use React from a CDN
export default window.React;

@jhnns unfortunately noParse won't work for React because it uses require extensively

@sokra I was thinking, if only I could make Webpack print out the modules one by one as it encounters them, then I could see which one it's hanging on, and maybe that would give me a clue what to investigate next.

Another note, my compilation gets stuck before the optimizing chunk assets part.  Who knows, there might be several unrelated issues here.

I have exactly the same issue, and also at the exact same percentage (80%). Dev builds work fine.

@Monokai As far as I've seen it always says 80% for optimizing chunk assets.

@jedwards1211 @Monokai  I faced the same problem. The main thing on my side is a bug in the libsass transpiler. Following workaround solves the problem for me:
UV_THREADPOOL_SIZE=100 npm run build
Maybe this tweek of the node internal threadpools helps.

@johann-sonntagbauer Fortunately there's a new release of sass-loader with a workaround that doesn't require UV_THREADPOOL_SIZE=100, info about it here: https://github.com/jtangelder/sass-loader/pull/132

@johann-sonntagbauer and I thought node-sass couldn't hang webpack at the "optimizing chunk assets" stage, only beforehand.  Am I wrong?

@jedwards1211 thx for the note about jtangelder/sass-loader#132 :+1:

@johann-sonntagbauer @jedwards1211 Strange, because I'm using Less, instead of Sass. Could it be that the Less loader has the same issues?

@Monokai sorry that i have overlooked that detail. simply thought that the error looks quite similar to mine and wanted to share the workaround. To be honest I don't dig into it. For the less compiler i can't give any hint, because I have never used it. Perhaps there could be a similar issue but I don't know, sorry.

@Monokai I don't know.  As far as I understand node-sass' issue comes from doing the parsing in C and using libuv to run renderers in parallel.  I'm guessing nothing inside the Less loader does that?   It's crazy stuff, I don't know when one needs to run renderers in parallel, but it certainly seems unnecessary for a Webpack loader.

If this is an Uglify issue, maybe its related to this: https://github.com/mishoo/UglifyJS2/issues/636 ? Doesn't look like they've released it on npm yet tho.

I had the same issue. After updating the css-loader to 0.19.0 version, it disappeared.

@raulferras Safe to close?

I just updated to<EMAIL_ADDRESS>and that fixed it for me!

@bebraw The css-loader can't be the only problem here. My issue, for example, seems to be related to uglify, since I'm not using the css-loader.

@fresheneesz I think the uglify dependency of webpack was updated a while ago. Hopefully that fix you linked to made it in. Perhaps it's worth giving it a go again.

I had an issue also related to UglifyJS. Mainly because I have a minified lib in modules, so Uglify takes like 5 minutes to parse it. Notice that it still works, just takes longer to process everything.

Try disabling minimize for some of the loaders. I had that issue with html and adding html?minimize=false to the loaders solved it.

I'm getting a similar problem after adding d3 to a very simple hello world repo.
I've let it run for 10 minutes but it didn't seem like it was proceeding and I was afraid my laptop might overheat so I shut it down.
https://github.com/SenHeng/ng1xts/commit/4277d1ccb04134534779e8b2fa0a520b59d3574d

@raulferras did you ever resolve this issue?

I still have this problem with webpack 1.12.9, which uses uglify-js 2.6.1. Honestly I don't know what to do with this other than put up with my build time being literally 100 times longer than it was before.

Using
new UglifyJsPlugin({
    sourceMap: false
})

Burns 100% of the cpu for >45minutes before I gave up and killed the process.
Running uglifyjs on webpack's output completes in a matter of seconds.

Adding some logs into the UglifyJsPlugin.js file, I see that its stopping and taking forever at line 88: ast = ast.transform(compress);. In my current test, its hanging for perhaps 2 minutes before moving on to the next file and hanging again.

I narrowed it down to packages that seem to have already been compressed/minified.
Best,
Jon
On Jan 6, 2016, 9:32 PM -0500<EMAIL_ADDRESS>wrote:

Adding some logs into theUglifyJsPlugin.js file(https://github.com/webpack/webpack/blob/master/lib/optimize/UglifyJsPlugin.js), I see that its stopping and taking forever at line 88:ast = ast.transform(compress);. In my current test, its hanging for perhaps 2 minutes before moving on to the next file and hanging again.
—
Reply to this email directly orview it on GitHub(https://github.com/webpack/webpack/issues/537#issuecomment-169527635).

I definitely don't have any already-minified sources in my package.

Faced same issue of 80% optimize chunk assets
I'm not using Uglify plugin
Works when I don't include devtool - (but that is very much required!)
Looks like as suggested above source-map might be causing this.
Any workarounds or fixes?

@sokra : Any updates on this 80% optimize chunk assets issue?

I was running into a similar issue when trying to run a production config on a remote server. It worked fine locally, but the process would hang at 78% optimize chunk assets then be killed after some time. setting sourceMap: false seemed to work, though I now get the following UglifyJs error: TypeError: Cannot read property 'sections' of null

@ericgio Where are you using sourceMap: false? I don't see that option in any of the docs. Trying it at the top level of the config, I don't see a difference.

@fresheneesz: I added it to the UglifyJS plugin config as suggested a few posts up. I actually found the root cause of my problem, which may be what others were experiencing. It turns out I was using the UglifyJsPlugin in the config file but also running webpack -p, so optimizing twice. When I removed the -p from the CLI command, everything worked. More here: https://github.com/webpack/webpack/issues/1385#issuecomment-160553651

I'm not using UglifyJsPlugin and also not running webpack with the param p but it's still stuck at 80% optimize chunk assets
My project is quite big though - Will that matter?

@abhinavsingi Ah thanks. Yeah it doesn't help for me, unfortunately. I'm also not running from CLI. Here are my options for reference:
{ context: '/home/vagrant/backend',
  entry: '././git/frontend/App',
  output:
   { path: '/home/vagrant/backend/git/frontend/public',
     filename: 'App.bundle.[chunkhash].js',
     pathinfo: false,
     jsonpFunction: undefined },
  plugins:
   [ OccurrenceOrderPlugin { preferEntry: true },
     DedupePlugin {},
     UglifyJsPlugin { options: { sourceMap: false } },
     ProgressPlugin { handler: [Function] },
     ContextReplacementPlugin {
       resourceRegExp: /.*$/,
       newContentResource: undefined,
       newContentRecursive: undefined,
       newContentRegExp: /NEVER_MATCH^/ },
     BannerPlugin { options: { raw: true, entryOnly: true }, banner: '' } ],
  cache: false,
  devtool: undefined,
  watch: false,
  module: { wrappedContextCritical: true } }

If you using postcss-functions you must apply they before another postcss modules. I have this bug in this context.
How you can debug this problem:

comment modules
In order disable loaders (babel, scss, postcss, less etc). Get sources without processing.

Also you can have this problem if you are using extract plugin and it has incorrect loaders. For example without first array argument ['css', 'scss|less|etc']

Try to remove dev-tool: source-map, then it will be much faster. But I'm still trying to figure out how can keep source-map.

I am also experiencing this problem when Webpack tries to compress node includes which have already been minified. Has anyone figured out a decent workaround for this?

I haven't : (

I had the same problem as everyone, adding UglifyJsPlugin { options: { sourceMap: false } } didn't work sadly, the problem was my pre-minified Three.js dependency. I was requiring Three.min.js and changing it to the src/Three.js files fixed for me instantly, thanks everyone for all the clues

I was able to resolve the "additional chunk assets" slowness by adding this line in webpack-config:
module: {
   noParse: new RegExp("node_modules"),
   ....

@JsonHunt are you sure that is what resolved it and that it's the same issue? It didn't do anything for my project (https://github.com/Restuta/rcn.io main-page branch)

@Restuta Try specifically excluding dev dependencies. I am seeing massive speed boost. See my updated comment above

@JsonHunt already excluded https://github.com/Restuta/rcn.io/blob/main-page/webpack.config.prod.js#L104 this doesn't solve this particular issue of chunk assets hanging

@Restuta noParse requires an array of regular expressions, not strings

@JsonHunt it works with strings as well, you can see it yourself, if I would remove those it would not compile

If still getting stuck/crashing due to OOM at 78%, after removing any pre-minified assets, you may want to try running WebPack with more memory than Node's default and a larger thread pool. You can use the UV_THREADPOOL_SIZE environment variable to set the size of the thread pool, and Node's --max_old_space_size=SIZE_IN_MB argument to increase the Node's memory limit.
For example:
UV_THREADPOOL_SIZE=100 node --max_old_space_size=4096 node_modules/webpack/bin/webpack.js --config config/webpack/production.js

Additionally, I use the following configuration for the UglifyJS plugin for production and it works fine when given enough resources as noted above...it just takes a good bit longer than desired:
new webpack.optimize.UglifyJsPlugin({
            sourceMap: false,
            minimize: true,
            compress: {
                drop_debugger: true,
                warnings: false,
                drop_console: true
            }
})

I know much of this has been mentioned in this issue, but wanted to put the core workarounds that I found together in one place for people searching.

not to minimize for compress-already code, that will make the process extremely slow. This is an known issue for uglifyjs, see this https://github.com/mishoo/UglifyJS2/pull/1024, the pull request has just been merged to master, not been published to npm I think.

We're also running into this issue and have worked-around exactly as @dougcalobrisi suggests. It's slow, but at least it doesn't die.

@Tzaphkiel - Does the big dependency you added to your project have any pre-minified files? If so, that is likely the cause, as @sunlei33 recently mentioned about the UglifyJS issue.
If not, try using the workarounds for increasing the number of threads and allocated memory for the node process running webpack, as I summarized above. I have encountered the exact same issues.

@dougcalobrisi  - Ok, thx, that might be it, the project I'm trying to include is alloy-editor which contains ckeditor, etc. and has indeed minimized files.
Is there a plan to fix this ? I saw something about UglifyJS having a fix in master but nothing released in npm yet... the dep. would still have to be adapted in webpack dependency no ?
My workaround for the moment is simply to turn off compress in the plugin definition for UglifyJS...
This is tedious and impractical :(

@Tzaphkiel - You could clone the current master repo for UglifyJS and then use npm link to create a globally installed version of the current master of UglifyJS, then delete the current UglifyJS dep in node_modules/ of your project and then install the linked version.
I haven't actually tried this, it's just off the top of my head, but should work. Something like this:
#clone the master repo and link it
mkdir ~/modules
git clone git://github.com/mishoo/UglifyJS2.git ~/modules/uglifyjs
npm link ~/modules/uglifyjs

#move to your project, remove the old uglifyjs dep, link the new one, and confirm the symlink shows
cd <path of your project>
rm -rf node_modules/uglifyjs
npm link uglifyjs
ls -l node_modules/uglifyjs

At that point your project should have the uglifyjs dependency in your project's node_modules symlinked to the globally installed version of the current master branch of UglifyJS. You'll want to unlink it once they have pushed a new release to npm.
Let me know if that works or not for you. As I said I haven't actually tested it, so no guarantees. Likely very helpful for others if it works, though.

Just happened to me. And I tested it again after removing devtool: inline-source-map, and the build compilation worked great and much quicker.

watch: true

make this happens. Disable watch solved problem for me.

@dougcalobrisi how can I know webpack's default memory and thread pool that Node provides ? And how much should I set will be best ? UV_THREADPOOL_SIZE=100 or UV_THREADPOOL_SIZE=10 ? The same to max_old_space_size

@FrendEr - I believe the default heap allocation is 1.76gb for a 64bit environment. You could check it using v8.getHeapStatistics() - docs: https://nodejs.org/api/v8.html
As for the default thread pool size, I believe that is coming from libuv which has a default of 5 threads.

@dougcalobrisi thanks! I use UV_THREADPOOL_SIZE=100 node --max_old_space_size=4096 to make my webpack run 8s reduce to 6.5s at additional chunk assets, but it still hangs at 78%.

@sunlei33<EMAIL_ADDRESS>has merge the issue#1024 and publish, but it has no effect on build optimization

compilation.plugin("additional-chunk-assets", function() {
  handler(0.78, "additional chunk assets");
});

I found the code in ProgressPlugin.js, I think UglifyJS takes much time to optimize code, but the reson for webpack hang at 78% just because the code above. It won't update before it turn to next step call optimize-chunk-assets

Had this same issue, building was taking 10mins+ and as a few people have noted, i took out my minified dependencies and it now builds in ~10s :D.

Had the same general issue as @theatlasroom and when the same direction to try and troubleshoot which oddly enough solved the issue.

 Increasing swap fixed that problem for me. It still hangs for a bit at 78-80% tho.

@sokra Could you comment on why you closed this? Has a fix from uglify been pulled in to fix the problem with already-compiled files?

removing the --progress option worked for me

In some cases and digging, i've fixed mine by insuring that the names has unique entries..
if (Mix.extract) {
    module.exports.plugins.push(
        new webpack.optimize.CommonsChunkPlugin({
            names: Mix.entryBuilder.extractions.concat([
                path.join(Mix.js.base, 'manifest').replace(/\\/g, '/')
            ]).filter(function (item, pos, self) {
                return self.indexOf(item) == pos;
            }),
            // minChunks: ({resource}) => /node_modules/.test(resource)
            minChunks: Infinity
        })
    );
}

Hope it helps.

webpack asset optimization continuously loops on 'ng serve'. I was expecting it would do once, then start my app at localhost:4200
Any Ideas?
Did npm update, and got all latest node_modules. Here is my
// package.json
{
"name": "angular-quickstart",
"version": "1.0.0",
"description": "package.json from the documentation, supplemented with testing support",
"angular-cli": {},
"scripts": {
"start": "tsc && concurrently "tsc -w" "lite-server" ",
"e2e": "tsc && concurrently "http-server -s" "protractor protractor.config.js" --kill-others --success first",
"lint": "tslint ./app/**/*.ts -t verbose",
"lite": "lite-server",
"pree2e": "webdriver-manager update",
"test": "tsc && concurrently "tsc -w" "karma start karma.conf.js"",
"test-once": "tsc && karma start karma.conf.js --single-run",
"tsc": "tsc",
"tsc:w": "tsc -w"
},
"keywords": [],
"author": "",
"license": "MIT",
"dependencies": {
"@angular/animations": "^4.1.0",
"@angular/common": "^4.1.0",
"@angular/compiler": "^4.1.0",
"@angular/compiler-cli": "^4.1.0",
"@angular/core": "^4.1.0",
"@angular/forms": "^4.1.0",
"@angular/http": "^4.1.0",
"@angular/platform-browser": "^4.1.0",
"@angular/platform-browser-dynamic": "^4.1.0",
"@angular/platform-server": "^4.1.0",
"@angular/router": "^4.1.0",
"@ng-bootstrap/ng-bootstrap": "^1.0.0-alpha.15",
"angular-in-memory-web-api": "~0.2.1",
"angular2-jwt": "^0.1.28",
"angular2-logger": "^0.5.1",
"auth0-lock": "^10.10.2",
"bootstrap": "4.0.0-alpha.5",
"core-js": "^2.4.1",
"jquery": "1.9.1 - 3",
"reflect-metadata": "^0.1.8",
"rxjs": "5.0.0-rc.4 - 5.0.1",
"systemjs": "0.19.40",
"tether": "^1.3.7",
"typescript": "^2.3.2",
"zone.js": "^0.7.2"
},
"devDependencies": {
"@types/jasmine": "^2.5.36",
"@types/node": "^6.0.46",
"@types/selenium-webdriver": "^2.53.33",
"angular-cli": "^1.0.0-beta.22-1",
"canonical-path": "0.0.2",
"concurrently": "^3.1.0",
"http-server": "^0.9.0",
"jasmine-core": "~2.4.1",
"jasmine-spec-reporter": "^2.7.0",
"karma": "^1.3.0",
"karma-chrome-launcher": "^2.0.0",
"karma-cli": "^1.0.1",
"karma-jasmine": "^1.0.2",
"karma-jasmine-html-reporter": "^0.2.2",
"lite-server": "^2.2.2",
"lodash": "^4.16.4",
"protractor": "~4.0.13",
"rimraf": "^2.5.4",
"ts-node": "^1.7.2",
"tslint": "^3.15.1",
"typescript": "~2.0.10",
"webdriver-manager": "^11.1.0",
"@types/google-maps": "^3.2.0"
},
"repository": {}
}
Chouhan:mockii_node rakeshchouhan$ ng serve
As a forewarning, we are moving the CLI npm package to "@angular/cli" with the next release,
which will only support Node 6.9 and greater. This package will be officially deprecated
shortly after.
To disable this warning use "ng set --global warnings.packageDeprecation=false".
Your global Angular CLI version (1.0.0-beta.28.3) is greater than your local
version (1.0.0-beta.22-1). The local Angular CLI version is used.
To disable this warning use "ng set --global warnings.versionMismatch=false".

                                                                                Hash: 2a5559aa5e7694288411
Time: 359ms
chunk    {0} 0.chunk.js, 0.bundle.map 54 kB {7} {6} {5} {2} {4} {3} {1}
chunk    {1} 1.chunk.js, 1.bundle.map 22.1 kB {3} {4} {2} {5} {6} {0} {7}
chunk    {2} 2.chunk.js, 2.bundle.map 17.8 kB {5} {4} {3} {1} {6} {0} {7}
chunk    {3} 3.chunk.js, 3.bundle.map 4.62 kB {1} {4} {2} {5} {6} {0} {7}
chunk    {4} 4.chunk.js, 4.bundle.map 3.77 kB {2} {3} {1} {5} {6} {0} {7}
chunk    {5} 5.chunk.js, 5.bundle.map 4.43 kB {6} {2} {4} {3} {1} {0} {7}
chunk    {6} 6.chunk.js, 6.bundle.map 4.43 kB {0} {5} {2} {4} {3} {1} {7}
chunk    {7} main.bundle.js, main.bundle.map (main) 172 kB {9} [initial]
chunk    {8} styles.bundle.js, styles.bundle.map (styles) 9.72 kB {10} [initial]
chunk    {9} vendor.bundle.js, vendor.bundle.map (vendor) 2.84 MB [initial]
chunk   {10} inline.bundle.js, inline.bundle.map (inline) 0 bytes [entry]
webpack: Compiled successfully.

@chouhan - Questions regarding the angular-cli belong in whatever medium the have designated for support requests.

@d3viant0ne - I did not get you. Are you expecting me to rewrite the question in a proper format?
I am looking for a solution to the above issue posted.

I'm not expecting you to rewrite the question, I am saying you are posting it in the wrong place.
Your question is related to the angular-cli of which webpack is a part but the right place to look for a solution related to ng serve is https://github.com/angular/angular-cli.
There is a ton of code on top of the cli's default webpack configuration that could be related to the output you are getting and considering you can't directly modify the cli's webpack configuration, this is an issue that will need to be handled by that team in whatever forum they have designated for support requests.
That said, first off I would get off of the beta build of the CLI and onto @angular/cli@latest. There were a significant number of performance improvements in the release candidates that may have a positive effect on the above which is exactly what they are going to tell you.
Secondly, don't mismatch the global cli version with the application cli version particularly with the beta builds. There are numerous resolved issues pre-final release caused by version mismatches.
Beyond those two things, you need to ask your question in the proper forum. i.e. the angular-cli repository.


Update @types/react-datepicker to the latest version

Is your feature request related to a problem? Please describe.
the latest @types/react-datepicker is the 2.9.4
https://www.npmjs.com/package/@types/react-datepicker
Describe the solution you'd like
Update the @types definition to the 2.10.0 version
Thank you

I added showQuarterYearPicker to it today, as we had a need to use that in our React-TypeScript project. I'll go through and add the other missing items in the coming days.

Thank you very much @WtheFourth

Since this updated I'll close the issue!! Thank you!
https://www.npmjs.com/package/@types/react-datepicker
https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/react-datepicker/index.d.ts


onDomCreation lifecycle hook.

I'm submitting a ...  (check one with "x")
[    ] bug report => search github for a similar issue or PR before submitting
[ x ] feature request
[    ] support request => Please do not submit support request here, post on Stackoverflow or Gitter

I would like to have a lifecycle hook "onDOMCreation" to include any tweaks to the rows I'm about to see.
What is the use case for changing the behavior?
I'm using a function on all rows to turn children objects into properties (to spread them across columns), and it takes a lot of time. I would like to do that only on the rows I'm seeing.

I do not think that is possible. Angular only provides onAfterViewInit which might be what your looking for.


Fix animation transition around the unit circle in Android compass example

For example if the previous angle was 269 and the new angle is -84, then the current code would rotate the needle in a clockwise direction. This fix would add 360 to -84, so the new angle would be 276, which would result in the correct behaviour of the needle rotating anti-clockwise.

@tshirtman no worries :)


add compressed option to masternode genkey

This allows anyone to generate / display compressed private keys using masternode genkey. Prior to this change only the uncompressed format is displayed.
I don't like the "string that looks like bool" option but I've tried and unsure how else to get this to a true bool since genkey is a subcommand. Might have to change the help text to reflect this, I'm up for suggestions.

Thanks for the tip! The new method seems to work perfectly for this case.


Logo for our website

We want a logo for our website matching our website idea.
Design a logo for us and get it reviewed, on approval submit a PR.
Show us your logo ss before submitting a PR

Ok, @Alam9835  Before moving forward carefully read the description of the issue and when you are done start working. feel free to ask you doubts.

I have designed the logo please suggest necessary changes


Add support for Gopher Phlog's

Since vee is allready available on freeshell.org, it would be nice if vee can also generate the gophermap and the raw txt files of the individual posts, such that they can be served over Gopher .

@vg sorry for the extreme delay. I need to research this, but seems worth doing!  The raw text files are available in the .vee/*.raw (each post has a corresponding ".raw" file).

@vg I am looking at old issues, can you provide me some informatino on generating gophermaps? E.g., what would they look like for vee posts?

https://gopher.zone/posts/how-to-gophermap/
https://sdfeu.org/w/tutorials:gopher
https://dev.to/dotcomboom/the-gopher-protocol-in-brief-1d88

Excellent, thank you!

@vg I am working on this now that I understand what needs to be done; doesn't seem like too much work. Biggest issue is going to be  code organization.
vee -G to :

generate .txt instead of .html (or just both?)
update gopherlog rather than index file (and also HTML index?)
vee-rebuild to rebuild gopherlog based on *.raw files available

Let me know what you think. Do people maintain both gopher content and HTML content in the same directory?

current status, see commit message for more info: 078760b6e4cf0f477e55c754fb0f

Getting close, 6aae6ac1735e4dc25e5033acc471b31675bcc7ae:
* -g|-G seem to be working pretty good
* gophermap is rebuilt whenever -g|-G is used so that the latest posts appear first
* updated post management to fully utilize veels and veecat, need to test more

Overall lookin pretty good. Since this is a pretty disruptive change and took the
liberty to refactor a few things, this might end up being vee 2.0.

@vg - kindly check out master at your convenience. The changes were great enough to make a full version jump. I tagged this 2.0-rc1 after testing it on SDF's gopher offers. I am very interested in your feedback. Thank you!


🐛 Fix Tenable deduplication setting

see issue #9612
I observed with this retest that the deuplication setting is not good enough for Tenable. Thus, I fixed this as well.

Someone asked about using description in the dedup algorithm for Tenable, here's why:
Tenable parser aggregates finding by endpoint, and appends the description of the new finding to the original:
CSV: https://github.com/DefectDojo/django-DefectDojo/blob/2181c61ef50f7b11d63f4612c1064fec89770760/dojo/tools/tenable/csv_format.py#L157C2-L161C59
XML: https://github.com/DefectDojo/django-DefectDojo/blob/2181c61ef50f7b11d63f4612c1064fec89770760/dojo/tools/tenable/xml_format.py#L226C1-L228C71

Someone asked about using description in the dedup algorithm for Tenable, here's why:
Tenable parser aggregates finding by endpoint, and appends the description of the new finding to the original: CSV: https://github.com/DefectDojo/django-DefectDojo/blob/2181c61ef50f7b11d63f4612c1064fec89770760/dojo/tools/tenable/csv_format.py#L157C2-L161C59 XML: https://github.com/DefectDojo/django-DefectDojo/blob/2181c61ef50f7b11d63f4612c1064fec89770760/dojo/tools/tenable/xml_format.py#L226C1-L228C71

We just ran into this problem.... When reimporting Tenable scan results often the affected hosts for certain findings are patched.. however because these hosts are in the description, and the description is now default in the deduplication algorithm Defectdojo no longer recognises them as the same finding. So instead of correctly updating the finding, Defectdojo just creates new ones.  I really see no good reason why this should be the default setting ? can @manuel-sommer  explain what Deduplication bug was fixed by changing this ?

I don't remember which one was fixed by this. Can you please open up a new issue with steps to reproduce your problem. Unsubscribing here.

@lme-nca
You can change dedup settings for any parser you want local to your DefectDojo instance by adding a line to local_settings.py
There's an example of that in the repo at https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/settings/template-local_settings
I'm curious why you changing the settings for your instance won't solve this problem for you.


Cache ord-editor layers in tests

With 100% caching (no changes to any layers), the build time drops from 6:45 to 1:16.

Hold off on reviewing; the subsequent editor tests aren't using the cache like I expected.

Hold off on reviewing; the subsequent editor tests aren't using the cache like I expected.

PTAL; with 100% caching (no changes to any layers), the test time drops from 7:24 at HEAD to 2:51 in this PR.


clean up some old code

Signed-off-by: chentong319<EMAIL_ADDRESS>Two changes:

Detect optional input. The NoneType is originally used to represent the optional input when it is not presented. Later, some onnx model used 0 length tensor for that purpose. We have function isFromNone for both case.  This change replace the NoneType check for some old code with isFromNone. Also isFromNone is changed to check the type, not how the value (definition point) is defined because a Value may not have a definition point in constructing process.
Delete unnecessary use of TypeConverter. The TypeConverter should be called automatically on the input by Conversion process, according to the document. Old code did not use type converter and the explicit type conversion was replaced with type converter when type converter is added. Such code is not needed.

Question: did you also eliminate all the isa<NoneType>() ?

The NoneType is originally used to represent the optional input when it is not presented. Later, some onnx model used 0 length tensor for that purpose. We have function isFromNone for both case.  This change replace the NoneType check for some old code with isFromNone. Also isFromNone is changed to check the type, not how the value (definition point) is defined because a Value may not have a definition point in constructing process.

I think it's a really good idea to clean up the inconsistencies about optional inputs and NoneType / isFromNone.
I think the best long term solution is to make all optional inputs NoneType so that !value.getType().isa<NoneType> is always the correct and unambiguous way to check if a value has a value. Using an empty tensor seems wrong because empty tensors are valid values in most situations as most tensor operations are well-defined for empty tensors.
This PR may be a good step forward to reduce confusion, but I'm curious if people agree that it's a good long term direction to standardize on NoneType or not as the single way to express if an optional input is absent or present?

It is the implementation detail to use NoneType or zero-sized tensor to represent the optional input. The zero-size tensor has extra info for the element type. The new OptionalType contains further more type info for the tensor. This PR is intended to encapsulate the implementation for possible future evolving.

Picking up on @sorenlassen message. I understand that we appear have multiple ways to represent optional inputs. The useful revision here provide a method that return the right results no matter how we represent optional inputs.
Soren is asking if we could normalize to one method only? Are there technical difficulties that forces us to have the one vs the other?

That's doable.

I support that we land this PR in its current form and open an issue where we can discuss and try to agree to normalize to one method only, and then one of us can implement anything we agree on in a follow-on PR

Jenkins Linux ppc64le Build #8409 [push] clean up some old code (... started at 10:41

Jenkins Linux amd64 Build #9345 [push] clean up some old code (... started at 09:36

Jenkins Linux s390x Build #9363 [push] clean up some old code (... started at 10:36

Jenkins Linux amd64 Build #9345 [push] clean up some old code (... passed after 1 hr 30 min

Jenkins Linux s390x Build #9363 [push] clean up some old code (... passed after 1 hr 57 min

Jenkins Linux ppc64le Build #8409 [push] clean up some old code (... passed after 2 hr 58 min

I think the best long term solution is to make all optional inputs NoneType so that !value.getType().isa<NoneType> is always the correct and unambiguous way to check if a value has a value.

I implemented this in PR #2133


Improved sea ice animation notebook

@libbyrose I think this PR and branch may have been overcome by the PR4 I just did pulling the seaice-animation branch into our local main repo?

Yep looks all good, thanks


Add Webappboost boilerplate

Hi,
I would like to add my recently launched Next.js boilerplate, it uses shadcn/ui as a library. Thank you!
Kind regards,
Ilya

I'm closing this PR because it is an entirely paid project, the idea of the repository is to bring together free projects for open-source contributions


Bug Report

Environment
Windows build number:  [Version 10.0.18362.175]
Windows Terminal version (if applicable): Version 0.2.1715.0

Steps to reproduce

1.) Change the fontFace to "Verdana" - A native Windows font
2.) Save, and return to the terminal
3.) Zoom in slightly
Expected behavior
The font is readable
Actual behavior
The font is squashed, and difficult to read

We offer no warranty for when you use a proportional font in a monospaced environment.

For what it’s worth, this is by design. We aren’t going to support proportional fonts.


Update babel and add watch script

This adds babel 6.0 and requisite plugins.
The stage-2 plugin is for object spread syntax. :dancer:

Awesome!! This works great :+1:


Prefetch + Other Options Configurable

Currently we have prefetch count hard coded. We need to explore optimal values for this and come up with a good default, and also make this (and any other important knobs) configurable in host.json.

The two properties that are important to surface are number of samples and the target queue length (in addition to what @mathewc called out)


we shouldn't be caching tx receipts in non-safe-mode

https://github.com/AcalaNetwork/bodhi.js/blob/80103cc49926c33e042b77fc6ec4d61f2d0d2a84/eth-providers/src/utils/BlockCache.ts#L3
If the transaction is on the forked block and does not exist on the longest chain, then our cache will give the wrong status.
Also need to confirm how subquery handles the fork problem

subql only indexes finalized block, so it should be fine.
For cache I guess we will first need to define the expected behaviors for eth_getTransactionReceipt:

when a fork just happens and sibling block is not finalized
when sibling block is already finalized

actually I think we should still return the receipt in a fork, and it's up to the clients to check for finalization/fork. i.e. for ETH they need to wait for a couple confirmations, and for EVM+ they can use eth_isBlockFinalized

@xlc @ntduan any insights on this?

frontier returns null for non-canonical receipt though
https://github.com/paritytech/frontier/pull/474/files

subql only cares about finalized blocks so eventually we will prune them.
I think it is not wrong to return receipt for tx in forks for some period of time. At least we have not yet received any requests about this so it should be ok.

I think we should still return the receipt in a fork, since eth_getTransactionReceipt only cares if the tx is mined, without knowledge about it's finalization.

If a transaction is in a forked block, there are 3 situations:

Included in the longest chain and block of the same height
After replay, it is included in the last block of the longest chain
The node is doing evil, and the transaction is not included in the longest chain
If we have cached transaction receipts, we also need to maintain fork processing (delete or update). If not cached, it is up to the node to maintain the transaction state

It's up to the clients/dapps to check for finalization/fork, if they ever care. For ETH they just for a couple confirmations, and for EVM+ they can use eth_isBlockFinalized.

Dapp does not need to deal with forks, after confirming the height, verify that the transaction exists and the status is successful.
Take wormhole and Ethereum as an example, after a few blocks, call eth_getTransactionReceipt to verify that the transaction status is successful. If we cached the transaction in non-safe-mode, and the transaction does not exist in the longest chain, then our cache status will return success.

ChatGPT told me the code is helpful for us
const { ApiPromise, WsProvider } = require('@polkadot/api');

async function subscribeToFinalizedBlock() {
  const wsProvider = new WsProvider('wss://rpc.polkadot.io');
  const api = await ApiPromise.create({ provider: wsProvider });

  api.derive.chain.subscribeNewHeads(async (header) => {
    const finalizedHeader = await api.derive.chain.getFinalizedHead();
    console.log(`Latest block: ${header.number}, Finalized block: ${finalizedHeader.number}`);
  });
}

subscribeToFinalizedBlock();

Not sure if it's 100% correct but makes sense to me. I think the main idea is that a receipt alone can never guarantee the tx's status.

For wormhole we provided a safe mode just for their convenience, but they should be able to check finalization without safe mode. Also most apps doesn't care about finalization, for example, when we send a tx in metamask, or make a swap in uniswap, they will mark the tx as complete as soon as we can get a receipt (1 confirmation). It won't wait 12 confirmations, or monitor for forks, since it doesn't care.
So it's up to the dapps to monitor for fork or finalization, if they care, such as wormhole.

can you clarify what do you mean by "inaccurate"? If I didn't miss anything, I believe all info in receipts should be accurate, even if it's a stale receipt.
But I agree there will be some inconsistency between cache and subql. Cache will strictly have more receipts than subql, which are unfinalized ones (remember we call cache as "unfinalizedCache" a ling time ago). But such inconsistency doesn't seem to be an issue, since as we mentioned above, receipt doesn't guarantee finalization, so it's normal if we find a receipt today (in cache), and can't find it tomorrow (in subql).

it's normal if we find a receipt today (from cache), and can't find it tomorrow (from subql).

However we can guarantee it only happens for stale receipt, for finalized receipt we will always be able to find them.

it's normal if we find a receipt today (from cache), and can't find it tomorrow (from subql).

@zjb0807  would this break blockscout? It's not "wrong behavior" but might cause issue for blockscout. I guess we can stick with current behavior for now, and if there is any issue (from blockscout or complained by other users), we can add a layer to cache that checks if receipt is stale before returning

so it's fine if we find a receipt today (from cache), and can't find it tomorrow (from subql).

It's very bad for blockscout and exchange.

People confirm the success of a transaction by checking the status and confirmation count of the receipt. If eth_getTransactionReceipt returns successfully within maxCachedBlocks blocks, but fails on the maxCachedBlocks+1 block, it would be very bad.

can you clarify what do you mean by "inaccurate"? If I didn't miss anything, I believe all info in receipts should be accurate, even if it's a stale receipt.

If a tx is in a forked block, there are 3 cases:

Included in the longest chain and block of the same height, different block hash
After a replay, it is included in a new block
The node is doing evil, and the tx is not included in the longest chain(can't be replayed)

so block height、block hash、status are inaccurate.


Discoverability of finding a streamer profile when an AKA is used as the stream name

I've been trying to find some developer profiles when I stumble across them on Twitch, and some streamers are entering their profile on here under their IRL name rather than their online name. For example David Poindexter goes by RobertTables,
So, I've contemplated adding their streamname into the visible part of the entry so that they can be found by doing CTRL-F in the browser.
This does move away from the clean aesthetic that @bnb went for originally, but I feel that the extra discoverability is beneficial.

I applied this potential change on the Twitch, GitHub and Twitter links.
The other option I was thinking about was an AKA section.
Please have a look at my test branch at : https://github.com/surlydev/awesome-developer-streams/tree/test/twitch-stream-names-visible and give me your opinion.

This seems fine to me, but I don't really have time to work on this repo anymore so take my opinion for what it's worth.
Carry on and keep up the good work, @surlydev!


Unable to login to Stoic wallet with web3Auth

Hello, I have a problem to connect to my Stoic Wallet with web3auth. When i am try to connect it opens to choose what account is linked Google, Github, Facebook etc. and as the wallet is connected to google account i selected it and it starts to load but nothing more happens please help. Also when i am trying to get the recovery phrase from web3auth for my Stoic wallet it does not copies.

This is due to version bump of TORUS wallet from 0.9 to v5
Working on a fix.


Run golangci-lint on examples

Description
golangci-lint doesn't lint the examples directory https://github.com/golangci/golangci-lint/issues/416
Upstream hasn't responded yet, but for now I am just going to run golangci-lint twice so we can fix the immediate issue
Reference issue
Resolves #531 and Resolves #470

Pull Request Test Coverage Report for Build 2568

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at ?%

Totals

Change from base Build 2562:
0.0%

Covered Lines:

Relevant Lines:
0

💛  - Coveralls

Pull Request Test Coverage Report for Build 2566

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at ?%

Totals

Change from base Build 2562:
0.0%

Covered Lines:

Relevant Lines:
0

💛  - Coveralls


Error: nucleo_f401re board doesn't support eth library!

Hi,
I'm trying to use ESP8266Interface library in my mbed project, using nucleo_f401 board. And I got this message when building:
Error: nucleo_f401re board doesn't support eth library!
Apparently, this script that causes that:
platformio/platformio/builder/scripts/frameworks/mbed.py
In the library, there's a class called TCPSocketConnection. I try to rename the file (.h and .cpp) to TCPSocketConn and that error ceases to exist. But I don't think that should be the solution, as I need to always rename the class for the new projects.
Please advice. Thanks.

Please install develop version and wait for the further steps from @valeros.

Hi @andriyadi
ESP8266Interface library currently in beta and has a little bit unusual structure.
I can offer your working project with some modifications:
https://dl.dropboxusercontent.com/u/10025728/ESP8266Interface.zip

I can see that the develop version now offers warning, instead of error.
Warning: nucleo_f401re board doesn't have native support for 'eth' library!
But build is still error like this:
~/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/4.8.3/../../../../arm-none-eabi/bin/ld: cannot find -leth
I'll try @valeros solution.

@andriyadi
cannot find -leth
Please, reinstall develop version.

Hi @valeros
I can confirm that the latest develop version works!
Thanks!


Integrate WaloViz!

Hi there! I'm the creator of WaloViz, and by chance we both launched less than a week apart!
AudioSample looks great and does lots of really cool things, but I see you've struggled with exactly the same problems as I did, that visualizing a spectrogram of any sort with an audio widget is either too difficult, not interactive or looks ugly.
I think there's an incredible opportunity here for a wonderful collaboration, you help me optimize and improve the compatibility of WaloViz with your backend knowledge and I'll help you introduce WaloViz into AudioSample as a visualization with my frontend knowledge.
Looks like a WIN-WIN to me, what do you say?

@nirdeepdub  what do you think?

Sounds like a great idea.