[VERBOSE-2:profiler_metrics_ios.mm(203)] Error retrieving thread information: (os/kern) invalid argument

Steps to Reproduce

Getting this error [VERBOSE-2:profiler_metrics_ios.mm(203)] Error retrieving thread information: (os/kern) invalid argument  without pressing on any button/function. and this error doesn't show up every time rarely to show, but I want to know from where does this error is getting.
Expected results:
Launching the app without getting any errors.
Actual results:
Launching the app and sometimes(rarely) I'm getting this error.
[✓] Flutter (Channel stable, 3.3.9, on macOS 13.0.1 22A400 darwin-arm, locale
    en-LB)
    • Flutter version 3.3.9 on channel stable at
      /Users/hassanyassin/Desktop/Flutter/Development/Tools/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b8f7f1f986 (7 days ago), 2022-11-23 06:43:51 +0900
    • Engine revision 8f2221fbef
    • Dart version 2.18.5
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/hassanyassin/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      11.0.13+0-b1751.21-8125866)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14B47b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      11.0.13+0-b1751.21-8125866)

[✓] Connected device (3 available)
    • iPhone (19) (mobile) • 08f6d5d4e6ac65273a8757fc2424f2f593e6ca56 • ios
      • iOS 16.1.1 20B101
    • macOS (desktop)      • macos                                    •
      darwin-arm64   • macOS 13.0.1 22A400 darwin-arm
    • Chrome (web)         • chrome                                   •
      web-javascript • Google Chrome 107.0.5304.110

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

hello @huycozy , I'm not understanding this warning message from where it's coming, because it's happening in different screens and rarely it happens and without pressing on anything or function, so I don't know if there is a function that leads for this error(I don't think so because it's happening without pressing on any tap).

Hi @hassanyassiin
The issue should be actionable with clear steps to reproduce it. I would suggest you keep an eye on it if you are able to replicate it consistently and see if it has any pattern. If you are unable to see it again, feel free to close it and write a comment when you have clear steps to reproduce. Thanks!

I also get it now with a default flutter app but I added to it firebase storage / firebase core and the image picker package.
// I don't have the steps to reproduce the problem, but it's happening suddenly and rarely even with the default flutter app and without pressing on anything

@hassanyassiin
Do you have M1 or Intel machine ?
Can you switch to latest master and try again to see if it occurs on that version too ?

@darshankawar  This error doesn't appear on the Master channel.
But I want to use the stable channel so how to avoid this error on the stable channel.
Yes I'm using MacBook Pro (M1 Pro chip)

Can you also confirm if this is occuring using iOS or Android or on both platforms ?

Can you also try to run the app directly from terminal / command line instead of from IDE and see if you still get same log with it too ? I don't think this is specific to firebase_storage or any other plugin but could be related to M1.

Thanks for the update. I am unable to replicate this behavior using use case mentioned above, but based on the report and log, keeping it open and labeling for team's attention and insights.
Launching lib/main.dart on iPhone 13 Pro Max in debug mode...
Running pod install...
Running Xcode build...
Xcode build done.                                           115.5s
Debug service listening on ws://<IP_ADDRESS>:49977/aJQpLXJT0-Y=/ws
Syncing files to device iPhone 13 Pro Max...

The log seems to being emitted here and seems benign. @gaaclarke @iskakaushik Perhaps we can suppress it?

@chinmaygarde I've noticed that I got this error when I add a package that makes http requests like firebase/http etc.. because I run default flutter app with only http package and this error occurs, then I tried it with firebase then this error occurs also, when I removed all the packages that makes http request, I've never seen this error again!.
Can you please let me know what should I do to solve this issue before publishing my app? because I'm trying to publish my app after 2 weeks.
Thanks !

I have encountered the same problem
Launching lib/main.dart on iPhone 15 Pro in debug mode...
main.dart:1
Xcode build done.                                           66.8s
(488) [VERBOSE-2:FlutterDarwinContextMetalImpeller.mm(35)] Using the Impeller rendering backend.
(171) [VERBOSE-2:profiler_metrics_ios.mm(203)] Error retrieving thread information: (os/kern) invalid argument
[VERBOSE-2:FlutterDarwinContextMetalImpeller.mm(35)] Using the Impeller rendering backend.
Lost connection to device.
Exited

Unable to build iOS app, app is crashing.

Circling back on this, I am unable to build my app for iOS because of this problem, Is there anything I am missing?

Hey there, I am also having the same problem not able to build the app for iOS.
Log -
'''
(488) [VERBOSE-2:FlutterDarwinContextMetalImpeller.mm(35)] Using the Impeller rendering backend.
(171) [VERBOSE-2:profiler_metrics_ios.mm(203)] Error retrieving thread information: (os/kern) invalid argument
[VERBOSE-2:FlutterDarwinContextMetalImpeller.mm(35)] Using the Impeller rendering backend.
'''

Same issue
[ERROR:flutter/shell/platform/darwin/ios/framework/Source/profiler_metrics_ios.mm(203)] Error retrieving thread information: (os/kern) invalid argument

If this helps
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.16.9, on macOS 14.2.1 23C71
    darwin-arm64, locale en-IN)
[✓] Android toolchain - develop for Android devices (Android SDK
    version 33.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.1)
[✓] VS Code (version 1.85.2)
[✓] Connected device (5 available)
[✓] Network resources

Same issue while performing flutter run
Launching lib/main.dart on iPhone 15 Pro in debug mode... Running Xcode build...                                                    └─Compiling, linking and signing...                      2,749ms Xcode build done.                                           11.1s [ERROR:flutter/shell/platform/darwin/graphics/FlutterDarwinContextMetalImpeller.mm(42)] Using the Impeller rendering backend. [ERROR:flutter/shell/platform/darwin/graphics/FlutterDarwinContextMetalImpeller.mm(42)] Using the Impeller rendering backend. Lost connection to device.
if this helps
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.19.3, on macOS 14.1.2 23B92 darwin-arm64, locale en-NP)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
[✓] Android Studio (version 2023.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.2)
[✓] VS Code (version 1.85.1)
[✓] Connected device (3 available)
[✓] Network resources

• No issues found!
sbhusal@DHNPLAP046-sbhusal app_fluter % flutter doctor -v
[✓] Flutter (Channel stable, 3.19.3, on macOS 14.1.2 23B92 darwin-arm64, locale en-NP)
    • Flutter version 3.19.3 on channel stable at /opt/homebrew/Caskroom/flutter/3.19.3/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ba39319843 (2 weeks ago), 2024-03-07 15:22:21 -0600
    • Engine revision 2e4ba9c6fb
    • Dart version 3.3.1
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/sbhusal/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15E204a
    • CocoaPods version 1.15.2

[✓] Android Studio (version 2023.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)

[✓] IntelliJ IDEA Ultimate Edition (version 2023.2.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.85.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.84.0

[✓] Connected device (3 available)
    • sdk gphone64 arm64 (mobile) • emulator-5554                        • android-arm64 • Android 14 (API 34) (emulator)
    • iPhone 15 Pro (mobile)      • 65FB77C6-4580-4682-8959-43901FA1E60D • ios           • com.apple.CoreSimulator.SimRuntime.iOS-17-4 (simulator)
    • macOS (desktop)             • macos                                • darwin-arm64  • macOS 14.1.2 23B92 darwin-arm64

[✓] Network resources
    • All expected network resources are available.

• No issues found!

The log itself is benign and the profiler will try again if it can't get the information. I've remove the log in https://github.com/flutter/engine/pull/52227
That log should not interfere with launching or publishing the app.
The reports of the asserts in platforms views or anything related to Impeller are unrelated. Please file separate issues with reproduction steps if you have them.


Config changes to reflect the default branch from master to main. Please merge to complete the default branch renaming job.

Auto branch renaming content.

Docs Build status updates of commit bf8d512:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


plugin error after a few queries

Describe the bug
plugin seems to keep connections open and crash.  After a few queries the plugin seems to error out (and return no results) and the only fix seems to be restarting steampipe
Steampipe version (steampipe -v)
Example: v0.23.1
Plugin version (steampipe plugin list)
Example: v0.4.0
To reproduce
> select id,dns_names, issuer ->> 'Organization', subject from crtsh_certificate where query = '<redacted>'   and not_before < now()
  and not_after > now();
+-------------+-----------------------------------------------------------+----------------------+----------------------------------------------------------------------------------->
| id          | dns_names                                                 | ?column?             | subject                                                                           >
+-------------+-----------------------------------------------------------+----------------------+----------------------------------------------------------------------------------->
...<redacted>
+-------------+-----------------------------------------------------------+----------------------+----------------------------------------------------------------------------------->
> select id,dns_names, issuer ->> 'Organization', subject from crtsh_certificate where query = '<redacted too>'   and not_before < now()
  and not_after > now();

Error: crtsh: pq: no more connections allowed (max_client_conn) (SQLSTATE HV000)

+----+-----------+----------+---------+
| id | dns_names | ?column? | subject |
+----+-----------+----------+---------+
+----+-----------+----------+---------+

Expected behavior
Query run
Additional context
plugin log
2024-05-29 19:25:41.979 UTC [ERROR] steampipe-plugin-crtsh.plugin: [ERROR]<PHONE_NUMBER>328: crtsh_ca_issuer.listCaIssuer: connection_error="pq: no more connections allowed (max_client_conn)"
2024-05-29 19:25:41.980 UTC [ERROR] steampipe-plugin-crtsh.plugin: [ERROR]<PHONE_NUMBER>328: crtsh_certificate.listCertificate: connection_error="pq: no more connections allowed (max_client_conn)"
2024-05-29 19:25:41.980 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: doList callHydrateWithRetries (crtsh-1717010739328) returned err pq: no more connections allowed (max_client_conn)
2024-05-29 19:25:41.980 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: QueryData StreamError pq: no more connections allowed (max_client_conn) (crtsh-1717010739328)
2024-05-29 19:25:41.980 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: streamRows execution has failed: crtsh-1717010739328 - calling queryCache.AbortSet (crtsh: pq: no more connections allowed (max_client_conn))
2024-05-29 19:25:41.981 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: QueryCache AbortSet - aborting request  with error crtsh: pq: no more connections allowed (max_client_conn) (1 subscriber) (crtsh-1717010739328)
2024-05-29 19:25:41.981 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: queryData.streamRows returned error: crtsh: pq: no more connections allowed (max_client_conn)
2024-05-29 19:25:41.981 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: executeForConnection crtsh returned error crtsh: pq: no more connections allowed (max_client_conn), writing to CHAN
2024-05-29 19:25:41.981 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: error channel received crtsh: pq: no more connections allowed (max_client_conn)
2024-05-29 19:25:42.004 UTC [WARN]  steampipe-plugin-crtsh.plugin: [WARN]<PHONE_NUMBER>328: readAndStreamAsync failed to read previous rows from cache: crtsh: pq: no more connections allowed (max_client_conn) publisher crtsh-1717010739328 (crtsh-1717010739328)

database log
2024-05-29 19:25:39.085 UTC [WARN]  hub: RestrictionsToQuals: failed to convert 2 restrictions to quals
2024-05-29 19:25:41.981 UTC [WARN]  hub: stream receive error rpc error: code = Unknown desc = crtsh: pq: no more connections allowed (max_client_conn) (0xc000aebb20)
2024-05-29 19:25:41.981 UTC [64283] ERROR:  rpc error: code = Unknown desc = crtsh: pq: no more connections allowed (max_client_conn)
2024-05-29 19:25:41.981 UTC [64283] STATEMENT:  select id,dns_names, issuer ->> 'Organization', subject from crtsh_certificate where query = '<redacted too>'   and not_before < now()
	 and not_after > now();

Hi @tinder-tder,
Thanks for raising the issue. I have a couple of follow-up questions:

When you say "after a few queries," do you recall approximately how many attempts it takes before the issue occurs?
In the plugin, we are connecting to the database using the connection string postgres://guest@crt.sh:5432/certwatch?binary_parameters=yes, which establishes a connection with the database.
The database at crt.sh:5432 can only handle a certain number of concurrent connections, and sometimes all of the connection slots are in use. I found an article in a public forum that discusses this issue: Google Groups.
Could you please confirm if you are getting this error consistently after a few attempts or only occasionally?

Thanks!

@ParthaI consistently, it only takes one query really with a follow up not long after (assuming its not cached).  If its a rate limit itssue maybe the error can be clearer.  or it can try and hold the connection open for cache period (dunno if that fits in the acceptable use policy)?

Hello @tinder-tder, I tried the following queries multiple times. However, I could not replicate the error. All the time I am getting the result successfully. It would be great if you could share the actual query you are trying to run. That may help us with replicating the error.
Queries:
1.
select
  dns_names,
  not_after
from
  crtsh_certificate
where
  query = 'steampipe.io';

2.
select id,dns_names, issuer ->> 'Organization', subject from crtsh_certificate where query = 'steampipe.io'   and not_before < now()
  and not_after > now();

If its a rate limit itssue maybe the error can be clearer. or it can try and hold the connection open for cache period (dunno if that fits in the acceptable use policy)

I don't think it is a rate limit error.
By the way, I have adjusted the client creation logic for establishing the connection with the following changes:

Set the maximum number of concurrently open connections to 10.
Set the maximum number of idle connections to 5.
Set the maximum amount of time a connection may be reused to 30 minutes.

I have hardcoded these values for now and pushed the changes to the issue-28 branch.
Please take a stab at it in the issue-28 branch and share your observations with us to see if it helps.
Steps to try it out in the issue-28 branch:

Clone the repository: git clone https://github.com/turbot/steampipe-plugin-crtsh.git
Navigate to the project directory: cd steampipe-plugin-crtsh
Check out the issue-28 branch: git checkout issue-28
Build the project: make
Run the query to test the changes.

If it helps, we can consider making those hardcoded values dynamic by configuring them in the connection config.
Thank you!

@ParthaI I tried the same queries with the current version and couldnt hit the error today too (maybe it was on their end).  i tested the issue-28 branch too and it didnt error out
the domains tested yesterday and today were
tinder ops.net
match security.net
tinder security.net

Thanks @tinder-tder, for the confirmation. I am closing the issue since it is resolved. Please don't hesitate to reopen the issue/create a new one if you encounter any more issues. Thanks!


Parsing a Path parameter with a trailing extension

Discussed in https://github.com/poem-web/poem/discussions/380

Originally posted by phaer September  2, 2022
Hello,
I am trying to use a Route like "/notebooks/:id.pdf but, it seems that poetry interprets this as a parameter called "id.pdf" instead of an parameter "id", followed by the string ".pdf". Using a regex like /notebooks/:id<[^\\.]>.pdf does not work as expected either.
Swagger UI confirms that the parameters aren't interpreted as expected:

Is this something supported by poem? Am I missing something?

I reviewed the code, this is not a bug, the current matching algorithm cannot support extensions.
I need to think about how to do it. 😀

I thought about it and suggested the following solution:
#[oai(path = `/notebooks/:id`)]
async fn handler(id: Path<String>) -> Result<()> {
    if id.ends_with(".svg") {
        // do something
    } else {
        Err(Error::from_status(StatusCode::NotFound))
    }
}

Thanks for the quick response! But its more a work-around than a solution in my opinion as, ideally, I'd like to have three different handlers: "/notebooks/:id", "/notebooks/:id.pdf" and "/notebooks/:id.png". With this work-around one has to cram all 3 of them into a single handler and do the rest of the matching myself.


Using Futures takes a lot of JIT compilation time

1. Description
I've been checking what resources are being spent on in my application and found that a significant amount of time is spent on JIT compilation. As it turns out, the reason is the use of Futures.
2. Exception
Test application metrics (the same thing happens in a real application):
Time spent in JIT (ms / 1 sec)                                        98.998

It would be great if EntityFramework-Plus would cache compilation results so that performance would increase dramatically.
3. Fiddle or Project
Even the simplest example reveals the problem:
        static void Test()
        {
            using var context = new MyContext();
            var usersFuture = context.Users.Future();
            var companiesFuture = context.Companies.Future();
            _ = usersFuture.ToList();
            _ = companiesFuture.ToList();
        }

Full code - https://gist.github.com/ArtemAvramenko/46afef540b1def14d42a9c24f272c745
4. Any further technical details
I used dotnet-counters to collect the metrics.
Update: If you remove Futures from the code, the "Number of Methods Jitted" metric stops growing and "Time spent in JIT" becomes zero.

Hello @ArtemAvramenko ,
Are you including the time that EF Core use to JIT compile itself? Our library is very small compared to everything that EF Core had to compile to execute the first query. Even if you use our library, we still use EF Core and you cannot include the part when calculating the time of JIT.
Best Regards,
Jon

@JonathanMagnan I certainly removed Futures for comparison. CPU load drops several times after that and Time spent in JIT becomes equal to zero.

I certainly removed Futures for comparison

Did you add these 2 lines to have a good comparison?
var users = context.Users.ToList();
var companies = context.Companies.ToList();

From what I remember, this is when you make the first query that EF Core will JIT compile most of his method. I feel it weird that the JIT compile will be close to zero without our library. We will surely investigate more if you tell me this is the behavior you still have.

@JonathanMagnan Yes, that's exactly the code I was comparing it to. With it, JIT compilation is called only at the first call. But if Futures are added, then each time Test() is called, new +4 methods are generated.

The problem remains in version 8.x

EF version: [EF Core v8.0.0]
EF Plus version: [EF Plus Core v<IP_ADDRESS>]
Database Server version: [SQL Server 2022]
Database Provider version (NuGet): [Microsoft.Data.SqlClient v5.1.1]

Hello @ArtemAvramenko ,
We made some progress in finding out exactly where the JIT issue started, but to be honest, I'm not sure if we will be able to fix it or not.
We will look one more time to find out exactly if we can skip a line of code that causes it or not.

Hello @ArtemAvramenko ,
We made more investigation this week, and I believe we will simply close as work as intended
When creating a query, EF Core compiles it and caches the query plan. So the next time the same query is called, it doesn't longer require JIT compile or at least calls some method that causes a JIT compile, such as the one found here: https://github.com/dotnet/efcore/blob/397dc68aa9599f3fd7bf04b5e447614dfd3f90c2/src/EFCore/Query/QueryCompilationContext.cs#L169
So it explain why the JIT compile is 0 when running the query multiple times without our library.
On our side, the query is always re-created as we combine them. We don't cache the plan either. So we always need to execute some of the code and the visitor found in my previous link every time.
Could it be possible to improve this to reduce the JIT compile time? Surely, but at this point, I believe we are currently too deep in methods specific to EF Core to really make it happen.
Let me know if at least the explanation makes sense.
Best Regards,
Jon

Hello @JonathanMagnan,
I imagined it was possible to cache the result without recompiling again. But obviously there will be a lot of pitfalls, such as different SQL that EF generates depending on the parameter values. This and many other things will have to be taken into account when creating the cache.
At the moment, I will try to remove the use of EntityFramework-Plus and see how negatively it will affect the load on the database. A slight increase in the load on the database will be acceptable to me, as it will significantly reduce the application's load on the CPU.


proof of reserve for bitcoin in channels

We'd like to add proof of reserve on our banking application. one of the unknown I have currently lies on the side of the bitcoin that are in a channel.
Is there a way to sign a message to show proof ownership of a portion of the UTXO that is within a channel?
I'm sure there is no API yet but I'm currently wondering if this would even be possible with the current spec. maybe the other party would need to sign a message as well? Or maybe the forced-closed-ready-to-broadcast transaction could be used?
I'm sure there are also side things to think about like escrow (would not be an issue with the new channel) but I'm trying to get sense of the big picture first.
The track on our side for this item. An ongoing PR on the BDK side.

You can sign a message on the channel output to_local key, if it's a static key all you have to do is export the keys into bitcoin core and then find the one that matches the channel key and then use that to sign
At a high level it's going to be pretty tricky to prove conclusively because channels are stateful in their own domain. How do I know that this proof hasn't already been invalidated by a future channel state?

At a high level it's going to be pretty tricky to prove conclusively because channels are stateful in their own domain. How do I know that this proof hasn't already been invalidated by a future channel state?

when the channel is being updated, there are some state in the script that are related to the block height IIRC?
if we can force an update of the channel somehow (I don't think we can... but not sure? maybe could trigger a msat payment when it's needed), we would have the last block height and this could be used as the time value at which the snapshot is done for the proof of reserve.

You can update a channel more frequently than per block so even if this were the case I don't think it would be a solution
Maybe you could do something with hold payments: the auditor routes a payment through your channel back to themselves, proving that there is coins

Maybe you could do something with hold payments: the auditor routes a payment through your channel back to themselves, proving that there is coins

but if you have a lot of channels would this be working? it would basically block the channel while the hold payment is ongoing assuming the auditor want to see the full capacity of the channel ? and the time to loop all the channels the liquidity could also have changed?

You can update a channel more frequently than per block so even if this were the case I don't think it would be a solution

I think the proof of reserve implies a snapshot at a particular time. there are ways to gamify proof of reserve: a platform could borrow the coin they need to cover their assets just prior to the snapshot and send it back to the lender just after snapshot has been taken.
but if the snapshot is done at a regular interval (let's say every day, or multiple times a day) then this becomes much harder to gamify, compared to if it's done every 6 months for instance. but for that to be implemented it should be a solution that would not be too intrusive.

Is there a way to sign a message to show proof ownership of a portion of the UTXO that is within a channel?

Given that the latest commitment shows the latest snapshot of the current channel balance distribution, you'd just need to show that commitment, and then send your half of the signature to the "auditor". It can't really be gamified as you describe, since the auditor would likley attempt to pin the set of channels to your advertised node. Public chanenls are easy since all nodes already have a proof (the channel announcement) that the channel is anchored at your node. Private channels aren't too different as you can show that there exists a channel which you can sign for.
In terms of what you need today to do something like this: the SCB file already contains the key descriptor information you need to generate an ad-hoc signature using the multi-sig key. From there, you'd need to obtain the latest commitment transaction which is already stored on disk, but not yet exposed on the RPC interface. If the ListChannels call returns the current unsigned commitment transaction then you can generate a manual signature. To err on the safe side, you likely want to sign a challenge that includes the commitment transaction and channel information, so your signature can't be levreage to breach the channel in the unlikely case that the auditor is colluding with your channel counter-party.


[Suggestion] multi-staging build in quic-go example Dockerfile

Background
I encountered an issue when trying to build the quic-go example using the Docker build command:
$ docker build -t quic-go-example .
[+] Building 0.1s (4/4) FINISHED
 => [internal] load build definition from Dockerfile                                             0.0s
 => => transferring dockerfile: 178B                                                             0.0s
 => [internal] load .dockerignore                                                                0.0s
 => => transferring context: 2B                                                                  0.0s
 => [internal] load build context                                                                0.0s
 => => transferring context: 2B                                                                  0.0s
 => ERROR [1/1] ADD main /main                                                                   0.0s
------
 > [1/1] ADD main /main:
------
failed to compute cache key: "/main" not found: not found

Concerns
The current Dockerfile implementation requires a pre-build step in the local environment, which complicates the process for users.
Proposed Solution
I believe the example implementation should be straightforward to run, ideally with a single command like docker build -t quic-go-example .. This can be achieved by using multi-stage builds in the Dockerfile.
Benefits

Reduces the complexity for users, allowing them to build and run the example with a single command.
Improves the user experience by providing a self-contained Docker build process.
Decreases the likelihood of build errors related to missing files or incorrect paths.

I hope this suggestion is helpful, and I look forward to your feedback.

I think we should just remove it. There's little value in having a Dockerfile here in the first place.


CI/CD: build shelter rpm failed

In this test, the process of build shelter rpm is failed due to a network problem.
Config git proxy can solve this problem, the detailed information refer to this link

@liying-000 Please track this issue.

If solves the network problem, then rpm -ivh shelter.rpm still has failed dependencies problem.


Ultimate Performance installed twice..

Accidentally clicked twice on "Add Ultimate Performance Profile" & now it shows up twice. Tried to use "Remove Ultimate Performance Profile" but get "The power scheme, subgroup or setting specified does not exist." in PowerShell. (Even after restarting)
any fix for this?

In this case you need to do it manually as explained here 👎🏽
You can remove it from the power options page by following these steps :
1- Open power option menu by pressing 'WindowsKey + r' to open run command window, then a window will popup type in the box "powercfg.cpl"  then press enter.
2- The power options menu will appear now , on ultimate performance power plan choose 'change plan settings'
3- You'll see option says 'delete this plan' press it and you will be done.

Just made a new commit #766 that fixes duplicate power profile and the remove button now works properly


undefined is not protected

steps to reproduce the bug:

open http://listjs.com/examples.html
open javascript:undefined = 42; void 0;
click on a button
a JavaScript error is displayed in the console: "Uncaught RangeError: Maximum call stack size exceeded" (on chrome)

Ouch, thanks for the heads up!

You're welcome.

Now its protected!


Mp4 recording not working automatically

Hello guys,
Thank you very much for the nice project.
I followed the installation instructions, but here what is happening:
1- Start the meeting > start the recording > end the meeting.
2- tail -f /var/log/bigbluebutton/post_publish.log
I, [2021-02-17T18:14:52.654513 #26989]  INFO -- : Task: Getting meeting metadata
I, [2021-02-17T18:14:52.675215 #26989]  INFO -- : MP4 conversion started ID : true
I, [2021-02-17T18:14:53.148714 #26995]  INFO -- : Task: Getting meeting metadata
I, [2021-02-17T18:14:53.164974 #26995]  INFO -- : Recording Ready Notify for [ID] starts
I, [2021-02-17T18:14:53.165110 #26995]  INFO -- : Recording Ready notify ends
3- nothing is showing in the directory: /var/www/bigbluebutton-default/recording
I should start the process manually by using the command below:

node bbb-mp4.js https://bbb-fqdn/playback/presentation/2.0/playback.html?meetingId=ID
./bbb-mp4.sh ID
to be able to see the mp4 files.

I am expecting all process will be working automatically, can you please help to figure out what is wrong here.

Hi,
Have you installed using bbb-mp4-install.sh as detailed here?
If yes, meeting would get automatically converted into MP4.
You don't need to do step 1 or 2 that you mentioned. bbb-mp4.sg would get called by post-publish.rb, after a recording is published.
May be your recording is still getting published or getting converted. Can you check after some time (few hours)?
If your meeting is published, it should be visible here: https://bbb-fqdn/playback/presentation/2.0/playback.html?meetingId=ID
Once published, MP4 should get converted in (meeting_duration + some more time).
Let me know if you still face difficulties after waiting for publishing/conversion to get over.

I reviewed the installation steps again, but I am still not able to get the it done.
I will remove everything and start over again.
I noticed one thing in some meeting, it keep redirecting to the HigherEdLab.com!

Hi @Moh23ce , Have you done this?
# give bigbluebutton user sudo access
usermod -aG sudo bigbluebutton

# make sudo access passwordless
sudo visudo
# add the following line at the end of the file
bigbluebutton ALL=(ALL) NOPASSWD: ALL

Hello @arunkumar141997,
Yes I did all steps and repeated it over again.
I am still in the same situation.
Thank you guys for your efforts.
You may close the issue, as it might be coming from my dev environment.

@Moh23ce Can you please try the latest code?
We have simplified playback.html by redirecting after the page loads. So it should work in all cases.

Hi @manishkatyan
Thank you very much for the nice project.
I followed the installation instructions, but here what is happening:
/var/www/bbb-mp4/webm/.webm: No such file or directory
Let me know
Thanks in advance

@Moh23ce Have to done this - cp .env-example .env
If you still face issues, I can look at your BBB server setup. Send me a private message at manishkatyan   @   gmail   .   com (email without spaces).

i have the same problem.
I, [2021-03-20T21:36:08.066568 #19164]  INFO -- : Task: Getting meeting metadata
I, [2021-03-20T21:36:08.068041 #19164]  INFO -- : MP4 conversion started for 7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172: true
I, [2021-03-20T21:36:08.958849 #19289]  INFO -- : Task: Getting meeting metadata
I, [2021-03-20T21:36:08.959422 #19289]  INFO -- : Recording Ready Notify for [7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172] starts
I, [2021-03-20T21:36:08.959471 #19289]  INFO -- : Recording Ready notify ends
I, [2021-03-20T21:36:09.246636 #19301]  INFO -- : Recording transferring to Scalelite starts
I, [2021-03-20T21:36:09.247729 #19301]  INFO -- : Transferring recording for 7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172 to Scalelite
I, [2021-03-20T21:36:09.247975 #19301]  INFO -- : Found recording format: presentation/7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172
I, [2021-03-20T21:36:09.248003 #19301]  INFO -- : Creating recording archive
I, [2021-03-20T21:36:09.252541 #19301]  INFO -- : Transferring recording archive to /mnt/bbb-recordings
I, [2021-03-20T21:36:09.381607 #19301]  INFO -- : Recording transferring to Scalelite ends
when i run node bbb-mp4.js https://bbb/playback/presentation/2.0/playback.html?meetingId=xxx
Invalid recording URL! return. but when i click to link it opens the presentationlink correctly.
Anyway when i change the bbb-mp4.js link check regex then another error will throw:
Failed to launch the browser process! spawn /usr/bin/google-chrome ENOENT
Thanks in advance

@nzmhtpgl  Try our new updated code its fully dockerized. Also You can usesh bbb-mp4.sh meetingIDfor manual converion (assuming you have created bbb-mp4:v1 docker image by running bbb-mp4-install.sh

when i use sh bbb-mp4.sh 7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172
there is an output : a336ca3ea0e9e3b0b524b46a3e44c14e9feb9bb2b7ea9c752eae68f71ee5e5d3
and there is no file in /var/www/bigbluebutton-default/recording/
this is last verison bbb-mp4:v1

@nzmhtpgl  run as sudo user, normal user might not have write access to that location.

already run as sudo user

@nzmhtpgl  what does docker logs say?

journalctl -fu docker.service
Mar 21 15:26:04 ip-10-1-15-48 dockerd[3088]: time="2021-03-21T15:26:04.642605223Z" level=info msg="ignoring event" container=1940975efe2db425d604a781009ee60152673bb9003923600ec2044c69b9acd5 module=libcontainerdnamespace=moby topic=/tasks/delete type="*events.TaskDelete"

@nzmhtpgl  when you run sh bbb-mp4.sh meetingID a container will be created with name as your meeting id . Then if you do docker logs MEETINGID -f you see the  logs related to bbb-mp4

its output like this: Error: No such container: 7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172
when i exec docker container ls it is empty also. i check docker images bbb-mp4       v1  is there.
when i run docker run bbb-mp4 id output is like that
Starting pulseaudio...
Waiting for pulseaudio to start...
URL undefined!

@nzmhtpgl  yeah its because docker run needs  some extra commands and env . You can see the docker run command in bbb-mp4.sh

URL undefined! my bbb domain name  bbb.domain.net not .com . All the error could it be related to this?

That shouldn't  matter because we check url based on bbb domain name from .env

@nzmhtpgl Do you mind posting .env here?

playbackFile=playback_default.html
BBB_DOMAIN_NAME=vc-zfp7um.medixcall.net
COPY_TO_LOCATION=/var/www/bigbluebutton-default/recording

@nzmhtpgl sudo docker run --rm --name bbb-mp4 -v /var/www/bigbluebutton-default/recording:/usr/src/app/download --env-file .env --env REC_URL=https://vc-zfp7um.medixcall.net/playback/presentation/2.0/playback_default.html?meetingId=MEETING_ID bbb-mp4:v1   Repalce MEETING_ID with your meeting id and run this. Please choose small meeting, less than 5 minutes as above will run docker without -d (deamon mode). If possible provide me logs.

Starting pulseaudio...
Waiting for pulseaudio to start...
PAGE LOG: ==Getting URL params
PAGE LOG: ==DOM content loaded
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Resizing components
PAGE LOG: ==Processing metadata.xml
PAGE LOG: Failed to load resource: the server responded with a status of 404 (Not Found)
PAGE LOG: Failed to load resource: the server responded with a status of 404 (Not Found)
PAGE LOG: ==Found video /presentation/7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172/video/webcams.webm
PAGE LOG: ==Found deskshare /presentation/7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172/deskshare/deskshare.webm
PAGE LOG: ==Loading playback
PAGE LOG: ==Hide playback content
PAGE LOG: ==Loaded logo from logo.png
PAGE LOG: ==Loading video
PAGE LOG: ==Loading acorn media player
PAGE LOG: ==Loading deskshare
PAGE LOG: ==Processing shapes.svg
PAGE LOG: ==Processing panzooms.xml
PAGE LOG: ==Processing cursor.xml
PAGE LOG: ==Processing deskshare.xml
PAGE LOG: ==Processing captions.json
PAGE LOG: Failed to load resource: the server responded with a status of 404 (Not Found)
PAGE LOG: ==Couldn't load presentation_text.json JSHandle@object
PAGE LOG: ==Processing slides.txt
PAGE LOG: ==All medias can be played
PAGE LOG: ==Linking chat to media
PAGE LOG: ==Running popcorn
PAGE LOG: 'webkitRequestAnimationFrame' is vendor-specific. Please use the standard 'requestAnimationFrame' instead.
PAGE LOG: ==Generating thumbnails
PAGE LOG: ==Setting title on thumbnails
PAGE LOG: ==Defining start time
PAGE LOG: ==Loading done
PAGE LOG: ==Show playback content
PAGE LOG: ==Resizing components
45.341
7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172
stderr: ffmpeg version 4.3.2-0york0~18.04 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --prefix=/usr --extra-version='0york0~18.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libzimg --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
stderr:   libavutil      56. 51.100 / 56. 51.100
libavcodec     58. 91.100 / 58. 91.100
libavformat    58. 45.100 / 58. 45.100
libavdevice    58. 10.100 / 58. 10.100
libavfilter     7. 85.100 /  7. 85.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  7.100 /  5.  7.100
libswresample   3.  7.100 /  3.  7.100
libpostproc    55.  7.100 / 55.  7.100
stderr: [x11grab @ 0x559de1dcf3c0] Stream #0: not enough frames to estimate rate; consider increasing probesize
stderr: Input #0, x11grab, from ':102':
Duration: N/A, start:<PHONE_NUMBER>.617052, bitrate:
stderr: 983040 kb/s
Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1280x800, 983040 kb/s, 30 fps, 1000k tbr, 1000k tbn, 1000k tbc
stderr: Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'pulse':
Duration: N/A, start:<PHONE_NUMBER>.502942, bitrate: 1536 kb/s
Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
stderr: Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
stderr: [libx264 @ 0x559de1df9100]
stderr: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
stderr: [libx264 @ 0x559de1df9100]
stderr: profile High, level 3.2
stderr: [libx264 @ 0x559de1df9100]
stderr: 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=22.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
stderr: Output #0, mp4, to '/usr/src/app/download/7607e7260f66a3d4390e866d32c3e3ebba698947-1616275825172.mp4':
stderr:   Metadata:
encoder         : Lavf58.45.100
stderr:     Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 1280x800, q=-1--1, 30 fps, 15360 tbn, 30 tbc
Metadata:
encoder         : Lavc58.91.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder         : Lavc58.91.100 aac

your mp4 should be present at /var/www/bigbluebutton-default/recording Is it present?

@nzmhtpgl  If you still have issue please make sure /var/www/bigbluebutton-default/recording directory is exist , check if the recording is available . Or  I can take a look at your bbb-mp4 setup if thats ok.

ok it is presented as mp4. file created . but mp4 file is not playable. when i check its size 8.0kb. my presentation is about 30sec normally.

@nzmhtpgl  It happens because of multiple times trying to convert the same recording .To resolve this Remove that mp4 and re convert it using sh bbb-mp4.sh meeting id

Closing as no response from the user


Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event).....

When testing a localhost version of "http://dwmkerr.github.io/angular-modal-service/" which is in the attached ZIP file setup in Visual Studio 2013
(localhost URL is http://localhost:51721/AngularModalServiceSamples2.html), I encountered the error only in IE9 for ng-Animate method "onAnimationProgress(event)". Attached is the screen capture image and the ZIP file to reproduce the problem in localhost using Visual Studio 2013.

I have the ZIP file to reproduce the problem but could not upload the Project zip file as Github does not allow me to upload the zip file.

Do you think you could reproduce the problem in a minimal demo in a plnkr etc? It's very cumbersome to run a big project locally.

I am trying to inline the code and test it. I will get back to you once I reproduced it.
Edwin
x738177
From: Martin Staffa<EMAIL_ADDRESS>Sent: Tuesday, October 27, 2015 3:21 PM
To: angular/angular.js
Cc: Fajardo, Edwin
Subject: {NW} Re: [angular.js] Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event)..... (#13182)
Do you think you could reproduce the problem in a minimal demo in a plnkr etc? It's very cumbersome to run a big project locally.
—
Reply to this email directly or view it on GitHubhttps://github.com/angular/angular.js/issues/13182#issuecomment-151615478.

It is in Plunker : http://plnkr.co/edit/NTd0MG1YoLoEqQywxyo2?p=preview
However, when in Plunker it does not reproduce the error, only when I
launch the page in Visual Studio 2013 running in localhost do I get the error.
Edwin
x738177
From: Fajardo, Edwin
Sent: Tuesday, October 27, 2015 3:29 PM
To: 'angular/angular.js'; 'angular/angular.js'
Subject: RE: {NW} Re: [angular.js] Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event)..... (#13182)
I am trying to inline the code and test it. I will get back to you once I reproduced it.
Edwin
x738177
From: Martin Staffa<EMAIL_ADDRESS>Sent: Tuesday, October 27, 2015 3:21 PM
To: angular/angular.js
Cc: Fajardo, Edwin
Subject: {NW} Re: [angular.js] Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event)..... (#13182)
Do you think you could reproduce the problem in a minimal demo in a plnkr etc? It's very cumbersome to run a big project locally.
—
Reply to this email directly or view it on GitHubhttps://github.com/angular/angular.js/issues/13182#issuecomment-151615478.

Issue #13182 is the same as Github Issue #10387. Plunker URL is
http://plnkr.co/edit/NTd0MG1YoLoEqQywxyo2?p=preview
but the HTML and template html’s need to run in Visual Studio 2013
as localhost to reproduce the problem as it only occurs in Visual Studio 2013.
Edwin
x738177
From: Martin Staffa<EMAIL_ADDRESS>Sent: Tuesday, October 27, 2015 3:21 PM
To: angular/angular.js
Cc: Fajardo, Edwin
Subject: {NW} Re: [angular.js] Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event)..... (#13182)
Do you think you could reproduce the problem in a minimal demo in a plnkr etc? It's very cumbersome to run a big project locally.
—
Reply to this email directly or view it on GitHubhttps://github.com/angular/angular.js/issues/13182#issuecomment-151615478.

If it's the same as #10387, we should continue the discussion there.

I was trying to add to 10387 but only allow me to create a new entry.
Edwin
x738177
From: Martin Staffa<EMAIL_ADDRESS>Sent: Tuesday, October 27, 2015 7:41 PM
To: angular/angular.js
Cc: Fajardo, Edwin
Subject: {NW} Re: [angular.js] Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event)..... (#13182)
Closed #13182https://github.com/angular/angular.js/issues/13182.
—
Reply to this email directly or view it on GitHubhttps://github.com/angular/angular.js/issues/13182#event-447443590.

@erfxyz You cannot comment on the other issue?

Yes, I cannot even add any comment and your link forced me to create a new issue.
Also, I tried installing through nugget Angular UI Bootstrap on the same Visual Studio 2013
solution, the problem went away, so this might be that nugget also updated the Angular JS
and Bootstrap components. I tested the same HTML page in our Project Visual Studio 2013
last night and the problem is NOT  OCCURRING anymore.
I have just arrived at the office this morning, tested the same page today and it is now
working perfectly in localhost Visual Studio 2013.
Edwin
x738177
From: Martin Staffa<EMAIL_ADDRESS>Sent: Wednesday, October 28, 2015 5:18 AM
To: angular/angular.js
Cc: Fajardo, Edwin
Subject: {NW} Re: [angular.js] Unable to get property 'toFixed' of undefined or null reference in ng-animate.js method onAnimationProgress(event)..... (#13182)
@erfxyzhttps://github.com/erfxyz You cannot comment on the other issue?
—
Reply to this email directly or view it on GitHubhttps://github.com/angular/angular.js/issues/13182#issuecomment-151775981.


Feat/auth

Merging from dev back into feat/auth to complete and remove the branch

Using admin privileges to squash and merge so that I can delete this branch


Clean isp labels

Removes many of the strings listed in #119. Incorporates into existing isp transforms.

ping @pbeshai


Using VS2019 - App returns 404 in Azure until you update Microsoft.NET.Sdk.Functions package

To get it working in Azure:

update the application to .NET Core 3.0 in application properties (may or may not be required)
update the Microsoft.NET.Sdk.Functions package to the latest (3.0.2 at time of writing)
republish

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6e88ea08-81ef-2b5a-b9d6-539e85592672
Version Independent ID: f2d8b7b4-906a-d1f5-5201-fc22a7890955
Content: Create your first function in Azure using Visual Studio
Content Source: articles/azure-functions/functions-create-your-first-function-visual-studio.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@Another-Phil-Nhsbt Thank you for your feedback! We will review and provide an update as appropriate.

@Another-Phil-Nhsbt I tried the steps mentioned in the document and it worked as expected. Below is the configuration in my repro:

This channel is for driving improvements towards MS Docs, Since this issue is not related directly to the doc,  I would recommend you to create a thread on the forums- MSDN / Stackoverflow in case you want to understand the reason of getting 404. Once you post your issue on forums, it will have visibility across the community which is a better suited audience for such types of issues.
we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Feat/chunked encoding poc

Codecov Report

Merging #695 (06b94aa) into main (0d108b5) will decrease coverage by 3.17%.
The diff coverage is 70.83%.

:exclamation: Current head 06b94aa differs from pull request most recent head 258aa25. Consider uploading reports for the commit 258aa25 to get more accurate results

@@            Coverage Diff             @@
##             main     #695      +/-   ##
==========================================
- Coverage   56.23%   53.06%   -3.18%
==========================================
  Files          63       65       +2
  Lines        1700     1992     +292
  Branches       41       47       +6
==========================================
+ Hits          956     1057     +101
- Misses        744      935     +191

Impacted Files
Coverage Δ

zio-http/src/main/scala/zhttp/http/Headers.scala
89.47% <0.00%> (-5.77%)
:arrow_down:

zio-http/src/main/scala/zhttp/http/HttpData.scala
46.66% <ø> (-3.34%)
:arrow_down:

...io-http/src/main/scala/zhttp/service/Handler.scala
55.93% <54.54%> (+1.93%)
:arrow_up:

...n/scala/zhttp/service/ChunkedOutBoundHandler.scala
90.00% <90.00%> (ø)

zio-http/src/main/scala/zhttp/http/Response.scala
37.50% <100.00%> (-1.53%)
:arrow_down:

.../scala/zhttp/http/headers/HeaderConstructors.scala
15.24% <0.00%> (-0.42%)
:arrow_down:

...rc/main/scala/zhttp/http/headers/HeaderNames.scala
100.00% <0.00%> (ø)

...c/main/scala/zhttp/http/headers/HeaderValues.scala
100.00% <0.00%> (ø)

.../main/scala/zhttp/service/EncodeClientParams.scala
100.00% <0.00%> (ø)

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d108b5...258aa25. Read the comment docs.


Media queries optimisation

It could be considered to add a grouping media queries plugin to optimisations:
https://github.com/Se7enSky/group-css-media-queries
But this one is not postcss-based as I can see.

Duplicate of https://github.com/ben-eb/cssnano/issues/111.


Delete 'Read-only' from Range.Rows and edit

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

VBA/Excel-VBA/articles/range-rows-property-excel.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

VBA/Excel-VBA/articles/range-rows-property-excel.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

All content in this repository has been migrated to a new repository (https://github.com/MicrosoftDocs/VBA-Docs). The changes in this PR have been applied in the new repository (as part of pull request #61), so this PR is being closed. The changes will be published to the live docs soon. Thank you for your contribution!


uuid error

uncaught typeError: (0 , _uuid.default) is not a function

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Install '...'
Run '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Runtime

OS: [e.g. macOS, Linux]
Runtime: [e.g. Node.js, Chrome, Safari, react-native, AWS Lambda]
Runtime Version: [e.g. 14.2.0]

Additional context
Add any other context about the problem here.

uncaught typeError: (0 , _uuid.default) is not a function

How are you importing uuid?

See https://github.com/uuidjs/uuid/issues/448#issuecomment-628028997


Bloqs adjoint

[ ] abstract method that returns the adjoint of a bloq
[ ] implementation for composite bloq that reverses the contents and calls adjoint on each
[ ] adjoint of registers to flip left and right
[ ] Maybe all bloqs should have an attribute that says whether they're the adjointed version or not?

Done in #565  and related


Use prefix for voyager tables

Yeah, it's me again...
Description of problem
I think users should be able to use whatever names they like for their application's tables and models. Laravel only defines 3 tables by default:

migrations
users
password_resets

They can all be configured:

password_resets in config/auth.php
migrations in config/database.php
users: change the migration and the table name in the model

This allows developers to have a migrations table to store information about birds migrations if they like.
Proposed solution
Since the installation process does not really allow users to edit the migrations (artisan voyager:install runs the migrations from the vendor directory), use a prefix for the tables specific to Voyager.
Alternatives considered
If you want a menu table for a restaurant, use a different slug. Inconvenient: it will be displayed to a user who does not even know there is a menu menu which is used for another purpose.

Duplicate of #1014
I think you are looking for this #4590

Since the installation process does not really allow users to edit the migrations (artisan voyager:install runs the migrations from the vendor directory), use a prefix for the tables specific to Voyager.

Migrations can be overriden, you just need to copy it in your migration folder.
Finding issues and giving suggestions is appreciated but please search before if there is already an issue / PR opened.


:robot: triggering CI on branch 'redhat-v2.1.1' after synching from upstream/v2.1.1

/approve
/lgtm


Re-add PACU model

Re-add PACU model

store in mlflow?
add column to front-end

this is the same as https://github.com/HYLODE/HySys/issues/511
we should probably close the other and hold this one

Ah - if that issue is being closed:
I'd started this (not on FlowEHR or HySys) with a system that allows very quick change of model selection in this branch.
The relevant lines of code are here.
The predictions aren't great but it does work. branch hrishee/abacus which has essentially the same code (except with an xgboost model) is running on gae08:8343 now.

Model is deployed on the abacus deployment from @hqva - needs merging in


read method on filesystem/File throws UnsupportedEncoding error when reading a .jpg file (and many other file types)

Prerequisites

[x] Can you reproduce the problem with Debug -> Reload Without Extensions?
[x] Did you perform a cursory search to see if your bug or enhancement is already reported?
[x] Did you read the Troubleshooting guide?

Description
When using the extension API to read a file that is not a text file (specifically a jpg), the read method will fail with the error UnsupportedEncoding.
It seems to be impossible to read some encodings with the filesystem/File API.
Stack trace:
UnsupportedEncoding
(anonymous function) @ mycode.js:41
(anonymous function) @ /filesystem/File.js:108
(anonymous function) @ /filesystem/impls/appshell/AppshellFileSystem.js:368

Expected behavior: [What you expected to happen]
File should be read.
Actual behavior: [What actually happened]
Error is thrown.
Versions
Mac OSX 10.11.6
Brackets 1.9.0

I'm noticing there is an option to set encoding for the read method, but there is no documentation of how to correctly determine the file encoding type.

Could you try opening the same file in the  latest pre-release build?
Also it would be great if you could share the image also, so that I can test the workflow.

@RationalCoding in 1.10 pre-release we have added auto detection of encoding of file if it is not UTF-8 readable, then it will open file with the detected encoding and then from the bottom right dropdown you can change the encoding, and it will try to reload the file with the newly selected encoding.

And as an extension author, if you know the encoding before hand, then you can set file._encoding to that encoding and then try to call read function, it will consider this encoding only while reading the file.

Hi @saurabh95. Thanks for the reply.
I'm getting several unrelated errors in my extension with the pre-release, but I'll try to test on a simpler case.
The file._encoding trick seems to work, is there a way to also write to a file with a specific encoding?

If you set the file encoding, using file._encoding, then the same encoding will be used in the write file.

Are the supported encodings the same as Node.js?

It actually looks like the problem goes deeper than simply setting the encoding. The appshel filesystem will throw a ERR_UNSUPPORTED_ENCODING regardless of the encoding set. It seems to be determining the encoding type from the file or something?
Is appshell.fs the Node fs module or something else?

I tried bypassing the method with getDocumentText, but seeing as it calls on the same filesystem read method, and images can never be "opened" in the editor, it fails as well.

Does Brackets still not support binary files? This might be the issue.

I think lack of binary support was the problem. I ended up using a Node script with 'binary' encoding and it works great. Thanks for your assistance in debugging this!


No module named 'vnpy_ctp.api.vnctpmd'

hello, I have install vnpy_ctp,but there is error is No module named 'vnpy_ctp.api.vnctpmd'. I open the file init.py it is that
from .vnctpmd import MdApi
from .vnctptd import TdApi
from .ctp_constant import *_
it not find MdApi, TdApi
Program environment：macos, python3.7

ctp不支持macos系统


--index-url from requirements.txt is ignored

--index-url is ignored in requirements.txt, no issue with --extra-index-url parameter.
Look like it's related to the way IndexUrl argument is used. This argument is always initialized with a value (here) and combine always keep the previous value if present (here).
If I invert the expression in combine, my --index-url in requirements.txt is correctly used; but this is maybe not the expected way to fix it because command line parameter (if really defined) should take precedence ?
This affect both pip install and pip  compile commands.
Fail with this requirements.txt:
--index-url https://my-pip-index.org/artifactory/api/pypi/pypi-custom-release/simple/

my-custom-package

Work with:
--extra-index-url https://my-pip-index.org/artifactory/api/pypi/pypi-custom-release/simple/

my-custom-package

With both commands:
uv -v --no-cache pip install requirements.txt
uv -v --no-cache pip compile requirements.in

To set the --index-url, the current workaround is to always set in on command line like:
uv -v --no-cache pip install --index-url https://my-pip-index.org/artifactory/api/pypi/pypi-custom-release/simple/ requirements.txt

(description edited because  it's more related to arguments default values rather than IndexLocations default values)

It is possible to add private PyPI in the pyproject.toml in a standardized way that uv understands? That is quite imporant in many corporate settings.  poetry supports this via the non-standard [[tool.poetry.source]] so that works well for corporations.

Not yet, but we will support persistent configuration of registries in the future.

Thanks for the quick fix

Not yet, but we will support persistent configuration of registries in the future.

@zanieb is there a tracking issue for that feature I can subscribe to?


Add a setting to disable stdlib check

Add a setting named "black-formatter.disableStdlibCheck" in package.json. Default is false.
Read this setting from "setting.ts"
Update the interface in setting.ts to pass in the new value.
Update the defaults in lsp_server.py for the new setting value.
Update the condition in lsp_server.py when is_stdlib is called to check setting first.
Update tests for setting.ts
Update tests for python

We are reserving this issue for Grace Hopper's Open Source Day, which will take place on September 22nd, 2023. If you're part of the event and would like to submit a contribution to this issue, please let us know, and we'll be happy to assign it to you.
For other community members who are not participating in the event, we appreciate your interest in contributing! We kindly request that you wait until after September 22nd to work on this issue. If no one from the event has been assigned to it by then, we'll gladly assign it to you. Thank you for your understanding and support!

Hi @luabud I would love to work on this issue if possible. I also left a message on slack as Michelle Cher. Thanks!

Hi @luabud I am attending GHC OSD and I am interested in this issue. Would it be possible for me to work on this issue? Left a message on Slack as well.

@CosetteX it looks like this has been assigned to @MichelleChernyi- if you could find another one to work on that would be great, let us know if you need help finding one!


Simple DSA endpoint flask app

We can wait to merge this until after the luna move

Codecov Report

Merging #215 (506395f) into master (c06139a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #215   +/-   ##
=======================================
  Coverage   48.56%   48.56%
=======================================
  Files          60       60
  Lines        5741     5741
=======================================
  Hits         2788     2788
  Misses       2953     2953

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c06139a...506395f. Read the comment docs.


Fix GetDeployment function when Devfile SchemaVersion is less than 2.1.0

Signed-off-by: Parthvi Vala<EMAIL_ADDRESS>What does this PR do?:

When GetDeployment is called for a Devfile schemaVersion less than 2.1.0. Currently it fails with error "top-level attributes is not supported in devfile schema version 2.0.0".
Which issue(s) this PR fixes:

PR acceptance criteria:
Testing and documentation do not need to be complete in order for this PR to be approved. We just need to ensure tracking issues are opened.

Open new test/doc issues under the devfile/api repo
Check each criteria if:
There is a separate tracking issue. Add the issue link under the criteria
or
test/doc updates are made as part of this PR
If unchecked, explain why it's not needed

[ ] Unit/Functional tests

[ ] QE Integration test

[ ] Documentation

[ ] Client Impact

[ ] Gosec scans

How to test changes / Special notes to the reviewer:
Use the following devfile for example:

devfile.yaml
```yaml
commands:
- exec:
    commandLine: npm install
    component: runtime
    group:
      isDefault: true
      kind: build
    workingDir: ${PROJECTS_ROOT}
  id: devbuild
- exec:
    commandLine: npm install
    component: runtime
    group:
      kind: build
    workingDir: ${PROJECTS_ROOT}
  id: build
- exec:
    commandLine: npm start
    component: runtime
    group:
      isDefault: true
      kind: run
    workingDir: ${PROJECTS_ROOT}
  id: devrun
- exec:
    commandLine: npm start
    component: runtime
    group:
      kind: run
    workingDir: ${PROJECTS_ROOT}
  id: run
components:
- container:
    endpoints:
    - name: 3000-tcp
      targetPort: 3000
    image: registry.access.redhat.com/ubi8/nodejs-12:1-36
    memoryLimit: 1024Mi
    mountSources: true
  name: runtime
metadata:
  language: nodejs
  name: vwcwtk
  projectType: nodejs
schemaVersion: 2.0.0
starterProjects:
- git:
    remotes:
      origin: https://github.com/odo-devfiles/nodejs-ex.git
  name: nodejs-starter
```

I reproduced this issue with odo. You can use odo dev with the Devfile and see the error.
I'll add the unit test once the changes have been approved.

@kim-tsao What would it take to merge this PR? I see all checks pass and it has both lgtm and approved labels.

@kim-tsao What would it take to merge this PR? I see all checks pass and it has both lgtm and approved labels.

@valaparthvi I realized you do  not have permissions to merge this.  It's merged now


70B SFT out of memory?

I'm using the 8xa100 GPU(80G) to do the SFT with the exact same parameter as in the instruction. If I'm using the stage2.json, it shows GPU OOM, but if i change to stage3.json, it works fine and shows each gpu only use about 11G meomory? Is it expected that stage2 and stage3 have a such big difference on the GPU memory usage?

Hi,
It is true that stage2.json should be OOM for 70B model SFT. But if stage3.json, the GPU memory on my machine is about 70GB. It is weird that it takes only 11GB.
Regards,
Yukang Chen

I will close this issue as it has been inactive for several days. Please feel free to re-open it if there are any others to discuss.


Bolt app ignoring im_close event while it's received at /slack/events

The Slack SDK version
Typescript Bolt app using versions:
"@slack/bolt": "^3.3.0",
"@slack/web-api": "^6.8.1",

Node.js runtime version
v16.20.0
OS info
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy
Steps to reproduce:

Subscribe to im_close event in a bolt-app using the above versions
Close the bot channel
Observe incoming requests
Observe logging output from im_close event handler

Expected result:
The event handler for im_close should be called
Actual result:
The event handler was not called even though the proxy (ngrok) does recieve a request to /slack/events. I can't even seem to catch it with a global middleware like:
app.use(async ({ next, body, ...rest }) => {
   console.log('\n\n')
   console.log(body)
   await next()
})

I don't have any issues receiving or handling any other events I have subscribed to in the events management pane in the slack web platform.

Did this get resolved somehow?  I'm seeing the exact same issue?  Was this just fixed in a newer version of the framework?


Helm upgrade --install thinks a release still exists after deleting

Output of helm version:
version.BuildInfo{Version:"v3.3.4", GitCommit:"a61ce5633af99708171414353ed49547cf05013d", GitTreeState:"dirty", GoVersion:"go1.15.2"}
Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.9-eks-4c6976", GitCommit:"4c6976793196d70bc5cd29d56ce5440c9473648e", GitTreeState:"clean", BuildDate:"2020-07-17T18:46:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.):
EKS
I'm attempting to update a Deployment object with the correct apiVersion required due to a kubernetes upgrade, but helm is complaining it's unable to perform the diff:
Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" helm.go:84: [debug] unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"
So I tried to delete the release and the history so it wouldn't need to perform any diff, but for some reason I'm continuing to get the above error.  I can't find the history for this release either, but it's clearly still keeping it somewhere.
helm -n dev3 history pcs-frontend-dev3 Error: release: not found
Attempts to create a new release results in a failure due to resources associated with the previous release still existing:
Error: rendered manifests contain a resource that already exists. Unable to continue with install: ConfigMap "pcs-frontend" in namespace "dev3" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-name" must equal "pcs-frontend-test": current value is "pcs-frontend-dev3"
The first time I tried to create a new release the CM actually did exist so I deleted it.  Subsequent runs resulted in the same error, despite the fact kubectl -n dev3 get cm does not return any cm associated with the pcs-frontend-dev3 helm release.
To reproduce, I suppose you would need an eks cluster capable of having a deployment using extensions/v1beta1 apiVersion managed by a helm release.  You would then need to upgrade the cluster so extensions/v1beta1 is no longer a valid deployment apiVersion, then follow the steps above.

I think I see the problem. This isn't saying the Helm release is present but that there's a Kubernetes manifest (resource) in the cluster that has the same name as one of the generated ones. If Helm were to install it an existing Kubernetes resource would be overwritten so it is bailing out with an error.
Does that make sense?

My specific situation was due to PEBKAC.
However in order to redeploy the chart I had to delete the helm release and all manifests.


Skip password changed email

It doesn't seem like there is an equivalent to @skip_confirmation_notification, and the instance variables set in the confirmations model.  Is it possible to bypass these emails on a case by case basis?

This would be of interest to a few people I imagine as there is currently a conflict with devise_invitable where password_changed emails are fired off when someone accepts and invite and chooses a password for the first time.
https://github.com/scambra/devise_invitable/issues/679
Allowing a flag of this nature would allow devise_inviteable to bypass the email when accepting the invitation.


xiaomi show request dialog twice,version 4.1.0

Environment

version 4.1.0
xiaomi8 & xiaomi Mix2s

we've already removed Xiaomi specific code from 4.1.0 so that we recommend user to define a workaround on your side. See https://github.com/permissions-dispatcher/PermissionsDispatcher/issues/544


added Cordova CI 2017

There was an error while validating job 180bae2a-1f10-486a-8cf2-18ffc6b13d7f Message:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
Details:Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase+ContextCommandException:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
---> System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Error clonning repository ---> LibGit2Sharp.NotFoundException: Branch 567 not found. Available branches: origin/master
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetBranch(Repository repo, String name)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass47_2.b__5(KeyValuePair2 branch)    at System.Collections.Generic.List1.ForEach(Action1 action)    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass47_1.<CloneRepository>b__2()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func1 func)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)    --- End of inner exception stack trace ---    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository()
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetVersionAsync(String path)
at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)    --- End of inner exception stack trace ---    --- End of inner exception stack trace ---    at Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase.ThrowIfErrorsLogged()    at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)
at Microsoft.LightweightJobEngine.Core.Language.JobWorkflow1.RunCommands(Guid jobId, IDictionary2 context, CancellationTokenSource cancelSignal)

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Problem with drawing primitives collections of labels, billboards and points, memory leak?

Hey! so this is the second time i am in this situation, the first time was about half a year ago but mange to handle this but this time i got no idea what is goin on..
I use cesium 1.48
so i am trying to draw complex drawings for alot of elements.. over 40k
each element hold multiply points, billboards and labels but only 1 polyline
so draw into 4 collections:
1 entity
x number of labels
x number of billboards
x number of points
save them and all is good..
then problem is when i delete my element which means i need to delete from 4 collections.
i dont expect my app to run fast with all the drawing of 40k elements
but when i remove them i want my app memory to get back close the staring point..
then momery indicate other..
alot of the momey which stay is stuff in the:
_spareBillboards
_billboardsToUpdate
_pointsToUpdate
and stuff like this in the memory heap test the chrome offers
even when i get into the console and debug the cesium scene i see these number..
and no, they wont clean up from them self's even after a long time and other drawings
i created a sandcastle example which show this memry issue with 999 labels
this is right after i finish drawing 999 labels:

and this is right after i finish removing them 1 by 1

here is my code

i really need help with this junk objects
SandCastle:
i couldnt get the copy link from the sandcastle :(
but the code i used is above
Thank you in advance!

Thanks for bringing this up @EdenEdk!  For labels, one billboard is used per character to visualize the label.  Once the labels are removed, we keep these billboards around to be reused later when you create another label, because reusing a billboard has better performance than always creating new ones. However, I do agree it doesn't make sense to keep around such a large number of these objects around indefinitely.
We might want to consider using some sort of cache with a fixed capacity and an expiration instead.  For reference, I added an LRU cache in this PR: https://github.com/AnalyticalGraphicsInc/cesium/pull/6284/files
It never made it into production because we ended up going with a different solution, but the implementation is there if we ever want to use it.

ok i got you!
so in th mean while, you got any suggestions for me?
what should i change?

@hpinkos
Hello, I'm EdenEdk's collegue.
In order to resolve this issue, if we wanted to get dirty and clear the caching arrays as you suggested, we would have had to clear multiple arrays, such as _spareBillboards , _backgroundBillboards , etc..
We've successfuly done that, and the memory got cleared, but again, did not want to get our code involved with cesium's code.
Our application is based on batched data, so instead of doing as mentioned, we decided to implement some logic that will manage multiple collections, even though it is not recommended, we figured that managing around 30 collections max for tens of thousands of data would not be that painfull.
Whenever some data-batch needs to be removed, after removing the specific collection's primitives, we could just envoke scene.primitives,remove(collection), so your cache-engine behavior does not effect our application memory, and we do not have to get involved with cesium's code. (except for scene._frustumCommandsList which is not related to the collection itself and needs to get manually cleared...)
Using this solution, did solve the memory issue, but made us experience a performance issue now.
Chrome devtool revealed that over 80% of the Total-Time was spent on PrimitiveCollection.remove.
What do we do now?
Why is PrimitiveCollection.remove so slow and how do we get around it?
Thanks in advance for your help!

@kesemdavid I'm not sure why PrimitiveCollection.remove would be taking so long, but it could potentially be related to #3018.  BillboardCollection works very similarly to PointPrimitiveCollection and I imagine it could have the same problem with destroying the pickId

@hpinkos
Thanks for the fast reply!
I should clear out I'm not talking specifically about BillboardCollection, we've implemented the multiple-collection-management generically for all of the application collections.
That mean, for each 'data-batch' there'll be a labelCollection, a billboardCollection, and a pointCollection, and those three will be removed one after another when that data-batch needs to be removed from the app.
Looking at PrimitiveCollection.remove implementation, a javascript delete operator is being used, which is obviously slow, except for that, the only thing that seems suspicious is the primitive.destroy().
I did not have time to deep dive into that destroy, but I'll highly appreciate any insights you can provide.
Thank you again.

Oh gotcha.  But yeah it seems like all of our collections have a similar problem because they all use pickIds in a similar way, and removing those seems to be the cause of the lag.  See the workaround proposed in https://github.com/AnalyticalGraphicsInc/cesium/issues/3018#issuecomment-144071807 to see if that fixes the problem you're seeing


HGTV - Unsupported URL

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.12.09. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.12.09

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[x] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

It appears the URL scheme that hgtv uses for its various shows has changed. From what I can tell in hgtv.py it's expecting hgtv.com/shows/.... whereas now it's watch.hgtv.com/tv-shows/....
I have log in information available as the example video is behind the cable subscriber log in.

youtube-dl 'https://watch.hgtv.com/tv-shows/house-hunters-international/full-episodes/home-base-in-belgium' -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://watch.hgtv.com/tv-shows/house-hunters-international/full-episodes/home-base-in-belgium', u'-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.12.09
[debug] Python version 2.7.10 (CPython) - Darwin-18.2.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.2.4, ffprobe 3.2.4
[debug] Proxy map: {}
[generic] home-base-in-belgium: Requesting header
WARNING: Falling back on generic information extractor.
[generic] home-base-in-belgium: Downloading webpage
[generic] home-base-in-belgium: Extracting information
NOT TEACHABLE
ERROR: Unsupported URL: https://watch.hgtv.com/tv-shows/house-hunters-international/full-episodes/home-base-in-belgium
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 2340, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2542, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2531, in _XML
    parser.feed(text)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 1, column 72
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 793, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 508, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 3323, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: https://watch.hgtv.com/tv-shows/house-hunters-international/full-episodes/home-base-in-belgium

Let me know how I can provide login details if needed

Yeah realized it looks like https://github.com/rg3/youtube-dl/pull/17959 fixes it. Just trying to figure out how to test it. (New to all this)


Test messenger configuration

Follow up #417

Added test config for all other formats as well

Added test config for all other formats as well


Add an example of Arbitrary return data

In the page, there is an arbitrary return value section. There should be a selectbox with emoticons, where it displays the emoticons, like how github does it in text boxes.

It should include the apple emoticons, slack emoticons, github emoticons and donger emoticons :smile:

Now that I think about it, let's just do the Github emojis and the donger emoticons.
I think the different sources should be categorized as such - and the category should be displayed. I've  attached a sketch. When you select one, it should only display the image or the donger.

p.s. I'm not sure what to name the dongers, maybe don't worry about naming them for now, just with numbers?

@rexlane if you type in :party on github, it suggests :tada: :tada: - that's a feature Soulheart has with aliases. Are github's aliases listed anywhere?


RFC: The Next.js "App Router", React Server Component & "SSR with Suspense" story

The RFC
You can read the rendered RFC here.
The examples
You can find some example applications in the examples folder.

The polls-demo is a simple poll app that showcases both Server Components and Client Components in a single app. It shows the difference between using Apollo Client in Server Components and Client Components.
The hack-the-supergraph-ssr example is an example exploring the "SSR and data transporting" angle more in-depth. It also uses the @defer-related Links discussed in the last part of the RFC.
You can change the time deferred responses from the server take by adjusting it in the "Demo Settings" in the top-right corner. The link is configured so that any fragment coming in after 100ms will not be part of SSR, but be fetched on the client. Keep in mind that this uses a real network connection, so there might be some network delay on top of the number you choose here.
The app-dir-experiments is more of an "experimenting around" app I used for building the package

Points to look out for

Starting the examples for the first time in next oftentimes leads to a hydration mismatch. This is due to useId not matching up between client and server. I assume that this could be a Next.js bug and have filed a bug report here: https://github.com/vercel/next.js/issues/48284
We track this in this issue in #16

Absolutely phenomenal work on this! 👏 I've done an initial skim and intend to read it more in depth soon.

Getting data from RSC into the SSR pass
I've discussed the possibility of using the RSC cache to rehydrate the SSR cache with @gaearon and we agree that it mostly doesn't make sense. The base assumption was that something in the RSC cache would be valuable for the SSR pass. If this were the case, the same data could be rendered by RSC and SSR/Browser components. But while the latter could update dynamically on cache updates, the former could not update without the client manually triggering a server-rerender.

I'd like to add a case to this discussion, what about prefetching data in RSCs that are only used in Client Components? One nice thing about RSCs is that they provide a framework-agnostic central way to prefetch. This would rely on de/rehydration of Promises, but does seem like a valid use case for wanting to reuse parts of the cache (even though they haven't finished fetching yet)?
Any parts only fetched in Client Components still suffer from client-side request waterfalls unless they are explicitly prefetched, so if we can't prefetch those in RSCs, where do we prefetch them (solvable, but would be cumbersome and different per framework/app)? It would be like using RSCs + getInitialProps so to speak.
Another reason to support it could be for data that is shown both in RSCs and Client Components, but that is never refetched (infinite stale time), or only refetched as part of a RSC refetch. In this case you could pass down a SC to the point where the CC is and show the data that way instead, but this might be cumbersome.
I'm sure I could come up with other cases (valid or not). I do agree with the original premise that it mostly doesn't make sense though, for the general case.

I'd like to add a case to this discussion, what about prefetching data in RSCs that is only used in Client Components? One nice thing about RSCs is that they provide a framework-agnostic central way to prefetch. This would rely on de/rehydration of Promises, but does seem like a valid use case for wanting to reuse parts of the cache (even though they haven't finished fetching yet)?

This is something that I want to keep exploring, and yes, passing a Promise from the RSC into the Client Component could be very promising here.

Any parts only fetched in Client Components still suffer from client-side request waterfalls unless they are explicitly prefetched, so if we can't prefetch those in RSCs, where do we prefetch them (solvable, but would be cumbersome and different per framework/app)? It would be like using RSCs + getInitialProps so to speak.

Waterfalls are less of a problem if you use fragment composition, so that's generally a pattern that we want to make more popular with Apollo Client.
As for the alternative to getInitialProps - that would be the SSR render of your client components. Of course, that would only apply for the first page that a user is accessing - but on the other hand, it probably doesn't play too much of a difference if these requests happen from the server or the client, after the page has been hydrated once and is "live".

Another reason to support it could be for data that is shown both in RSCs and Client Components, but that is never refetched (infinite stale time), or only refetched as part of a RSC refetch. In this case you could pass down a SC to the point where the CC is and show the data that way instead, or pass down the raw value, but this kind of prop drilling might be cumbersome.

This one could lead to a bit of a slippery slope, and in the long run probably end up with inconsistencies if developers are not very careful with it - but at the same time I do see some use cases.
Maybe we could add something like a repository-aware eslint plugin that could make sure that data prefetched in RSC is only ever queried from other server components, and client components make sure to always use the cache-only fetchPolicy 🤔

Absolutely phenomenal work on this! 👏 I've done an initial skim and intend to read it more in depth soon.
Thanks, and please let us know if you have any further thoughts!

so we would probably end up with a  component that would have to be rendered out. (passing it into the Provider doesn't make sense, since that is in the Layout and you probably want to prefetch at least on a per-page level.)

It's nice to hear that this is something you are thinking about too. For React Query we are renaming <Hydrate> to <HydrationBoundary> in v5 precisely because of this, the idea being that you might want several throughout your tree, even further down in nested layouts. I hadn't thought about an explicit <Prefetch> component, that's worth considering!

Waterfalls are less of a problem if you use fragment composition

That's true! I was stuck thinking about the general case.

Of course, that would only apply for the first page that a user is accessing - but on the other hand, it probably doesn't play too much of a difference if these requests happen from the server or the client, after the page has been hydrated once and is "live".

Yeah, for the SSR pass this is no more a problem than for RSCs (depending on where the RSC and the SSR servers are located compared to your backend/DB, if RSC is co-located with those and SSR lives on the edge, doing it in SSR would be worse). For client side page transitions though this would lead to the same bad performance as request waterfalls have today. Part of the motivation for RSCs is to "solve" request waterfalls, but they can't do that for data used in Client Components unless we also prefetch that data there, if that makes sense? (Though fragment composition works around that!)

This one could lead to a bit of a slippery slope, and in the long run probably end up with inconsistencies if developers are not very careful with it - but at the same time I do see some use cases.

Definitely! My thinking though is that when you consider all of the above things together, and think about how complex this area is to reason about, it would be really nice to provide developers with a single coherent API for prefetching stuff.
If we could say "If you are using RSCs, just make sure you prefetch everything in those" and make that work well, that would be golden. Again, I do think fragments are a good alternative solution for this for Apollo (especially considering de/rehydrating promises is pretty alpha), so I'm talking more high level/generally here. 😃

Notable things that have happened since this RFC:
Support for useBackgroundQuery and useReadQuery #38 (in SSR)
Usage for these is
function Child({ queryRef }) {
  const { data } = useReadQuery(queryRef);

  return <div>{data.foo.bar}</div>;
}

function Parent() {
  const [queryRef] = useBackgroundQuery(query);

  return (
    <Suspense fallback={<SuspenseFallback />}>
      <Child queryRef={queryRef} />
    </Suspense>
  );
}

The difficulty here is that child components suspense, while the parent starts the request. To get this to work, we simulate an ongoing request in the browser while the real request happens on the server.
restart simulated streamed request on stream close #62  (in SSR)
With these simulated queries, there is always the risk that for some reason, they are started on the server, but not actually consumed (the useReadQuery might be behind a conditional) - which means that the stream closes before we can actually transport the data over.
We have no way of detecting a stream close on the server at this moment (hello, feature request!), so that request will just keep going on the server. But on the browser, we can wait for the .ready event which means that definitely no more HTML will be streamed in. In that case, we have to restart the request in the browser so we at least get a result eventually.


how  to get token when the request need

as wrong tip   :
osdOptions = self.media2_service.GetOSDOptions(token)
File ".r\venv\lib\site-packages\onvif2\client.py", line 26, in wrapped
raise ONVIFError(err)
onvif2.exceptions.ONVIFError: Unknown error: The ReferenceToken type doesn't accept collections as value
when I want to get OSDOptions :media2_service.GetOSDOptions(), should be insert token  ,but I can't get it .help me  tks

you can use https://github.com/FalkTannhaeuser/python-onvif-zeep , try is it ok?
and，what the camer model？ hik 、dahua  、sony ?