Provide a way for the incremental rendering to hand over rendering once client JS kicks in.

Once the client's JS starts, the incremental renderer should stop accepting commands and close the connection.

Need some way of comparing the client-side's rendering to what has been done through the streaming commands so it can flip at the right time.
This means there needs to be some way for the incremental renderer and done-autorender to talk to each other. Having a hard time thinking of what that will look like at the moment.

I think it's ok for done-autorender to know about the incremental renderer, since it already knows about server rendering during reattachment (this is what it reattaches asynchronously).
So my thinking is that the incremental renderer client code can provide the API through a global.

I'm going to make this an ugly global, but I don't have a better idea.
doneSsrAttach(fragment).then(function(){
  // We're at the same point now, can go ahead and attach.
  document.body.appendChild(fragment);
});

Working on done-autorender portion first: https://github.com/donejs/autorender/issues/59

Issue moved to donejs/done-ssr-incremental-rendering-client #2 via ZenHub


Nix: Use npmlock2nix to handle npm dependencies in purescript builds

Motivation: This PR introduces the use of npmlock2nix for dealing with npm dependencies and includes changes in purescript.nix.
Overview of the relevant changes:

Adds npmlock2nix, exposed via plutus.lib.npmlock2nix
Adds package-lock.json files for plutus/marlowe/scb playground (created by npm i)
Removes yarn/yarn2nix files
Adds plutus.lib.buildNodeModules which uses npmlock2nix to create a node_modules output
Replaces manual purs invocation with ${spagoPackages.buildSpagoStyle}
Several simplifications to purescript.nix as a result of a combination of all of the above
drive-by fix: Use nix-gitignore for source cleaning
Added comments

Notes:

The benefit of npmlock2nix is that it parses the package-lock.json directly so there is no need to maintain yet another external source (yarn.nix) which has to be manually created.
This PR uses a branch of npmlock2nix which i can hopefully merge into master soon
For this PR to be ready i'll have to update some documentation referring to yarn
I actually tried this on macOS - it does build but i don't currently have a setup to really test this
The only platform specific code now relates to node_modules.nix because darwin needs xcodebuild and CoreServices to build stuff successfully with node-gyp
We don't have to rely on fetching any binaries anymore: everything builds nicely

I think/hope this PR leads to a nice and tidy purescript.nix and also a nice separation of concerns with the npm stuff happening in node_modules.nix which can be changed separately. The purescript build only cares about getting some nodeModules derivation. This also makes it easier to swap out npmlock2nix if that was necessary down the road.
Let me know what you think.

Pre-submit checklist:

Branch

[x] Commit sequence broadly makes sense
[x] Key commits have useful messages
[x] Relevant tickets are mentioned in commit messages

PR

[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

If you updated any cabal files or added Haskell packages:

[x] $(nix-build default.nix -A pkgsLocal.updateMaterialized) to update the materialized Nix files
[x] Update hie-*.yaml files if needed

If you changed any Haskell files:

[x] $(nix-shell shell.nix --run fix-stylish-haskell) to fix any formatting issues

If you changed any Purescript files:

[x] $(nix-shell shell.nix --run fix-purty) to fix any formatting issues

Pre-merge checklist:

[x] Someone approved it
[ ] Commits have useful messages
[ ] Review clarifications made it into the code
[ ] History is moderately tidy; or going to squash-merge

So just to understand the workflow, how do I add a new node package? How do I run a local env, presumably just from a nix-shell I can run npm whatever?

So just to understand the workflow, how do I add a new node package? How do I run a local env, presumably just from a nix-shell I can run npm whatever?

Exactly. I will update/add documentation about that. The idea is that the nix tooling is now basically a very thin transparent layer and only depends on the npm lock file. So you just deal with npm as you would outside of any npm context (which updates the lockfile). The nix build will pick it up.

Looking good so far, just need to fix hydra and update the docs.

I have fixed all docs, and by now the feature branch of npmlock2nix that i was using has been merged into master. Everything works fine locally.
At this point i just need hydra to actually process this...

I think you're going to hit the same eval error again:
error: "access to path '/nix/store/0h2yg54rvqjpibgqx9p1ib78k5qkfklj-source/package-lock.json' is forbidden in restricted mode"

I had hoped to merge this much earlier but i was thrown back by forgetting about the restricted evaluation mode that hydra applies. What this means is that i had to create a way to pass in hashes for github dependencies in npmlock2nix. Currently only marlowe-playground uses some Github dependencies and the node_modules build thus looks as follows:
{
  nodeModules = buildNodeModules {
    projectDir = nix-gitignore.gitignoreSource [ "/*.nix" "/*.md" ] ./.;
    packageJson = ./package.json;
    packageLockJson = ./package-lock.json;
    githubSourceHashMap = {
      shmish111.nearley-webpack-loader."939360f9d1bafa9019b6ff8739495c6c9101c4a1" = "1brx669dgsryakf7my00m25xdv7a02snbwzhzgc9ylmys4p8c10x";
      znerol.libxmljs."0517e063347ea2532c9fdf38dc47878c628bf0ae" = "0g3kgwnqfr6v2xp1i7dfbm4z45inz1019ln06lfxl9mwxlc31wfg";
    };
  };
}

So the GitHub dependencies are https://github.com/shmish111/nearley-webpack-loader and https://github.com/znerol/libxmljs. I added something to the READMEs explaining how this is to be used:
If you need to use a npm dependency from GitHub you basically just add it analogous to what you can see above and provide a zeroed sha256, start a build and then replace the sha256 with the one you get from the error message. Done.

I think you should be able to find a way to add this to the updateClientDeps script no? Not good having to do this as it is reasonably likely to fail on Hydra in the future, no?

I think you should be able to find a way to add this to the updateClientDeps script no?

So the whole idea with tools like npmlock2nix (and also poetry2nix for example) is to be transparent layer on top of the native package manager and avoid any intermediate representation i.e generated nix code in the first place. Now, unfortunately, hydra uses restricted evaluation mode which means fetching some github dependency without sha256 is impossible (as this is not being provided by npm). On the bright side, there is only one github npm dependency (your webpack-loader) which in turn pulls in some custom version of libxmljs on Github.
Now theoretically we could..

...store the tiny attribute set that we are passing to buildNodeModules in a file
...write some tool that traverses package-lock.json to extract github references
...have the tool fetch those and determine the sha256 to then update the nix file
...track such a file per client
...somehow add this to updateClientDeps

I say "theoretically" because i'd argue this doesn't actually improve the user experience but does introduce more tooling/complexity which then needs to be written and maintained..

Not good having to do this as it is reasonably likely to fail on Hydra in the future, no?

If you...

Introduce a new Github dependency without providing a hash for it: hydra evaluation will fail and it is going to report "that accessing  is not allowed in restricted eval mode"
Update the revision of a GitHub dependency without updating the sha256 : the build is going to fail (also locally) with an error showing expected sha256 versus actual sha256

In both of these cases the fix is just a matter of adding/updating an entry to githubSourceHashMap which only takes a minute. I would argue that is easier than creating extra tooling, and tracking an extra file for 2 hashes. After all the error is hidden away somewhere or is otherwise completely opaque, hidden or whatever else..
PS: I had a look at the git history for the package.json files. Seems like they are updated very infrequently and of course all of the above only* matters when adding GitHub dependencies. Otherwise nothing has to be specified/hard-coded anyhow.
PPS: We could also add https://github.com to the restricted-uris config of the hydra machine and do none of the above :trollface:  :joy:

unfortunately, hydra uses restricted evaluation mode

I would argue that that should be "fortunately" 😛
Ok, I see the costs are less leaving it as it is, just be aware that waiting for Hydra to evaluate something is not low cost, so if someone forgets to do this and pushes, it will take possibly hours before they discover the issue (esp since evaluation failures are (were?) not reported to github). On the plus side this is not likely to happen very often at all.


MediaPipeException: FAILED_PRECONDITION: Graph has errors

Plugin Version or Commit ID
0.12.0
Unity Version
2022.3.4f1
Your Host OS
Windows 10 Enterprise
Target Platform
Android
Description
MediaPipeException: FAILED_PRECONDITION: Graph has errors:
Calculator::Open() for node "poselandmarkgpu__poselandmarkbyroigpu__poselandmarkmodelloader__LocalFileContentsCalculator" failed: Failed to read /storage/emulated/0/Android/data/com.DefaultCompany.ESPORTS_MOLE/files/pose_landmark_full.bytes
when i build .apk file and i try to access files got this error.
I already add below code to custom AndroidManifest.xml file.
uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION"
uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"
uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"
but i still got it. how to fix it?
anybody help me? thanks
Code to Reproduce the issue
Additional Context
No response

Failed to read /storage/emulated/0/Android/data/com.DefaultCompany.ESPORTS_MOLE/files/pose_landmark_full.bytes

Maybe you haven't included pose_landmark_full.bytes to your apk or forgot to save it to the file system (using StreamingAssetResourceManager).
Please read https://github.com/homuler/MediaPipeUnityPlugin/wiki/Getting-Started#load-model-files for more details.


FontWeight.w500 not work for Chinese font and device

Chinese Font do not bold, when i use fontWeight: FontWeight.w500 on some Chinese Android devices. Maybe the mobile handset supplier has changed something，but w100  - w500 has the same effect.

Doctor
[✓] Flutter (Channel unknown, v1.9.1+hotfix.2, on Mac OS X 10.14.4 18E226, locale zh-Hans-CN)

[!] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    ! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses
[✓] Xcode - develop for iOS and macOS (Xcode 10.3)
[✓] Android Studio (version 3.5)
[!] IntelliJ IDEA Community Edition (version 2019.1.3)
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
[!] VS Code (version 1.40.2)
    ✗ Flutter extension not installed; install from
      https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[✓] Connected device (1 available)

Hello @pxc0215 !
Thank you for posting an issue.
Can you please also attach an example that can be used to reproduce the issue?

This is my code:
`import 'package:flutter/material.dart';
void main() => runApp(MyApp());
class MyApp extends StatelessWidget {
// This widget is the root of your application.
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
primarySwatch: Colors.blue,
),
home: MyHomePage(title: 'Flutter Demo Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
MyHomePage({Key key, this.title}) : super(key: key);
final String title;
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State {
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text(widget.title),
),
body: Center(
child: Column(
children: [
Text(
"这是 FontWeight.w100 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w100),
),
Text(
"这是 FontWeight.w200 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w200),
),
Text(
"这是 FontWeight.w300 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w300),
),
Text(
"这是 FontWeight.w400 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w400),
),
Text(
"这是 FontWeight.w500 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w500),
),
Text(
"这是 FontWeight.w600 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w600),
),
Text(
"这是 FontWeight.w700 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w700),
),
Text(
"这是 FontWeight.w800 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w800),
),
Text(
"这是 FontWeight.w900 的字号",
style: TextStyle(
color: Colors.black,
fontSize: 25,
fontWeight: FontWeight.w900),
),
],
),
), // This trailing comma makes auto-formatting nicer for build methods.
);
}
}
`
And，the display in my android phone is below:

As you can see, the Chinese has the same effect from w100 to w500，but the show of letters and numbers was markedly different. Why?

As above, my phone is a Chinese-made phone, maybe the mobile handset supplier has changed something，but there has the same problem on Android Virtual Device (Nexus 6), as follows.

How can I solve this. Very thanks!

@janmoppel Jan Moppel   Please help me

/cc @GaryQian

It looks like your device just simply lacks the multiple levels of bold fonts. This is sometimes the case as a full CJK font can be around 20-30mb, and keeping multiple bold/regular/light versions of the same font can take up significant storage space.
If bold is extremely important, I would recommend either setting it to a heavier weight to use the fakebold as mentioned above, or install your own font family that has true bold fonts for all the weights you want to use. Note that this would likely increase your app size by the size of the fonts.

So why are the character . and the point in character i with font weight 700 and 800 are rectangles on Android devices?

@WeiCongcong Can you post a screenshot of what you are seeing? The . in the i character is not a separate glyph and should be in sync with the body of the i.

@WeiCongcong Can you post a screenshot of what you are seeing? The . in the i character is not a separate glyph and should be in sync with the body of the i.

I noticed the problem in the screenshots posted in this issue.

@GaryQian Is the problem solved?


No web UI login page after TRAINS server installation (0.16.0)

I have installed the TRAINS server version 0.16.0 on a newly build Linux server. Previously I was using 0.15.1, and it was working with no issues. However, the new installation has no login page. The only page has example projects and experiments.
I followed the Trains Server Troubleshooting section "Why is web login authentication not working?" but it didn't solve the issue.
Below is the apiserver.log.
[2020-08-20 02:31:46,726] [8] [INFO] [trains.es_factory] Using override elastic host elasticsearch
[2020-08-20 02:31:46,727] [8] [INFO] [trains.es_factory] Using override elastic port 9200
[2020-08-20 02:31:46,992] [8] [INFO] [trains.redis_manager] Using override redis host redis
[2020-08-20 02:31:46,992] [8] [INFO] [trains.redis_manager] Using override redis port 6379
[2020-08-20 02:31:47,081] [8] [WARNING] [trains.schema] failed loading cache: [Errno 2] No such file or directory: '/opt/trains/server/schema/services/_cache.json'
[2020-08-20 02:31:47,081] [8] [INFO] [trains.schema] regenerating schema cache
[2020-08-20 02:31:49,679] [8] [INFO] [trains.server] ################ API Server initializing #####################
[2020-08-20 02:31:49,679] [8] [INFO] [trains.database] Initializing database connections
[2020-08-20 02:31:49,680] [8] [INFO] [trains.database] Using override mongodb host mongo
[2020-08-20 02:31:49,680] [8] [INFO] [trains.database] Using override mongodb port 27017
[2020-08-20 02:31:49,681] [8] [INFO] [trains.database] Registering connection to auth-db (mongodb://mongo:27017/auth)
[2020-08-20 02:31:49,682] [8] [INFO] [trains.database] Registering connection to backend-db (mongodb://mongo:27017/backend)
[2020-08-20 02:31:49,684] [8] [WARNING] [elasticsearch] GET http://elasticsearch:9200/_template/events* [status:N/A request:0.001s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 233, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd4ec93f208>: Failed to establish a new connection: [Errno 111] Connection refused
[2020-08-20 02:31:49,687] [8] [WARNING] [elasticsearch] GET http://elasticsearch:9200/_template/events* [status:N/A request:0.001s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 233, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd4ec93f2e8>: Failed to establish a new connection: [Errno 111] Connection refused
[2020-08-20 02:31:49,688] [8] [WARNING] [elasticsearch] GET http://elasticsearch:9200/_template/events* [status:N/A request:0.001s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 233, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd4ec93f3c8>: Failed to establish a new connection: [Errno 111] Connection refused
[2020-08-20 02:31:49,690] [8] [WARNING] [elasticsearch] GET http://elasticsearch:9200/_template/events* [status:N/A request:0.001s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 233, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd4ec93f4e0>: Failed to establish a new connection: [Errno 111] Connection refused
[2020-08-20 02:31:49,690] [8] [WARNING] [trains.initialize] Could not connect to es server. Retry 1 of 4. Waiting for 30sec
[2020-08-20 02:32:19,796] [8] [INFO] [trains.initialize] Applying mappings to ES host: [ConfigTree([('host', 'elasticsearch'), ('port', '9200')])]
[2020-08-20 02:32:19,932] [8] [INFO] [trains.initialize] [{'mapping': 'events_log', 'result': {'acknowledged': True}}, {'mapping': 'events_plot', 'result': {'acknowledged': True}}, {'mapping': 'events', 'result': {'acknowledged': True}}, {'mapping': 'events_training_debug_image', 'result': {'acknowledged': True}}]
[2020-08-20 02:32:19,933] [8] [INFO] [trains.initialize] Applying mappings to ES host: [ConfigTree([('host', 'elasticsearch'), ('port', '9200')])]
[2020-08-20 02:32:20,022] [8] [INFO] [trains.initialize] [{'mapping': 'queue_metrics', 'result': {'acknowledged': True}}, {'mapping': 'worker_stats', 'result': {'acknowledged': True}}]
[2020-08-20 02:32:20,022] [8] [INFO] [trains.mongo.initialize.migration] Started mongodb migrations
[2020-08-20 02:32:20,026] [8] [INFO] [trains.mongo.initialize.migration] Finished mongodb migrations
[2020-08-20 02:32:20,125] [8] [INFO] [trains.server] Exposed Services: auth.create_credentials auth.create_user auth.edit_user auth.fixed_users_mode auth.get_credentials auth.get_token_for_user auth.login auth.logout auth.revoke_credentials auth.validate_token debug.ping events.add events.add_batch events.debug_images events.delete_for_task events.download_task_log events.get_multi_task_plots events.get_scalar_metric_data events.get_scalar_metrics_and_variants events.get_task_events events.get_task_latest_scalar_values events.get_task_log events.get_task_metrics events.get_task_plots events.get_vector_metrics_and_variants events.multi_task_scalar_metrics_iter_histogram events.scalar_metrics_iter_histogram events.vector_metrics_iter_histogram models.create models.delete models.edit models.get_all models.get_all_ex models.get_by_id models.get_by_task_id models.get_frameworks models.make_private models.make_public models.set_ready models.update models.update_for_task organization.get_tags projects.create projects.delete projects.get_all projects.get_all_ex projects.get_by_id projects.get_hyper_parameters projects.get_model_tags projects.get_task_tags projects.get_unique_metric_variants projects.make_private projects.make_public projects.update queues.add_task queues.create queues.delete queues.get_all queues.get_all_ex queues.get_by_id queues.get_default queues.get_next_task queues.get_queue_metrics queues.move_task_backward queues.move_task_forward queues.move_task_to_back queues.move_task_to_front queues.remove_task queues.update server.config server.endpoints server.get_stats server.info server.report_stats_option tasks.add_or_update_artifacts tasks.clone tasks.close tasks.completed tasks.create tasks.delete tasks.delete_configuration tasks.delete_hyper_params tasks.dequeue tasks.edit tasks.edit_configuration tasks.edit_hyper_params tasks.enqueue tasks.failed tasks.get_all tasks.get_all_ex tasks.get_by_id tasks.get_configuration_names tasks.get_configurations tasks.get_hyper_params tasks.get_types tasks.make_private tasks.make_public tasks.ping tasks.publish tasks.reset tasks.set_requirements tasks.started tasks.stop tasks.stopped tasks.update tasks.update_batch tasks.validate users.create users.delete users.get_all users.get_all_ex users.get_by_id users.get_current_user users.get_preferences users.set_preferences users.update workers.get_activity_report workers.get_all workers.get_metric_keys workers.get_stats workers.register workers.status_report workers.unregister
[2020-08-20 02:32:20,332] [8] [WARNING] [trains.service_repo] Returned 400 for users.get_preferences in 2ms, msg=Invalid user id: id=0cf9bd46c2af410f8a182a5c8b4d2b13, company=d1bd92a3b039400cbafc60a7a5b1e52b
[2020-08-20 02:32:20,370] [8] [WARNING] [trains.service_repo] Returned 400 for users.get_current_user in 4ms, msg=Invalid user (failed loading user)
[2020-08-20 02:32:20,374] [8] [INFO] [trains.service_repo] Returned 200 for projects.get_all_ex in 11ms
[2020-08-20 02:32:20,423] [8] [INFO] [trains.service_repo] Returned 200 for projects.get_all_ex in 22ms
[2020-08-20 02:32:20,446] [8] [INFO] [trains.service_repo] Returned 200 for tasks.get_all_ex in 37ms
[2020-08-20 02:32:20,698] [8] [INFO] [trains.service_repo] Returned 200 for auth.login in 9ms
[2020-08-20 02:32:20,717] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_all in 3ms
[2020-08-20 02:32:20,725] [8] [INFO] [trains.service_repo] Returned 200 for workers.get_all in 1ms
[2020-08-20 02:32:20,731] [8] [INFO] [trains.service_repo] Returned 200 for workers.get_all in 1ms
[2020-08-20 02:32:20,738] [8] [INFO] [trains.service_repo] Returned 200 for workers.get_all in 1ms
[2020-08-20 02:32:20,745] [8] [INFO] [trains.service_repo] Returned 200 for workers.get_all in 1ms
[2020-08-20 02:32:20,763] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_by_id in 3ms
[2020-08-20 02:32:20,777] [8] [INFO] [trains.service_repo] Returned 200 for workers.register in 7ms
[2020-08-20 02:32:20,842] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 51ms
[2020-08-20 02:32:23,833] [8] [INFO] [trains.service_repo] Returned 200 for workers.status_report in 21ms
[2020-08-20 02:32:25,867] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 13ms
[2020-08-20 02:32:30,889] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 10ms
[2020-08-20 02:32:35,910] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 10ms
[2020-08-20 02:32:40,931] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 10ms
[2020-08-20 02:32:42,249] [8] [INFO] [trains.service_repo] Returned 200 for auth.fixed_users_mode in 0ms
[2020-08-20 02:32:42,307] [8] [WARNING] [trains.service_repo] Returned 400 for users.get_preferences in 2ms, msg=Invalid user id: id=0cf9bd46c2af410f8a182a5c8b4d2b13, company=d1bd92a3b039400cbafc60a7a5b1e52b
[2020-08-20 02:32:42,618] [8] [WARNING] [trains.service_repo] Returned 400 for users.get_current_user in 3ms, msg=Invalid user (failed loading user)
[2020-08-20 02:32:42,682] [8] [INFO] [trains.service_repo] Returned 200 for projects.get_all_ex in 3ms
[2020-08-20 02:32:42,689] [8] [INFO] [trains.service_repo] Returned 200 for projects.get_all_ex in 14ms
[2020-08-20 02:32:42,882] [8] [INFO] [trains.service_repo] Returned 200 for tasks.get_all_ex in 12ms
[2020-08-20 02:32:45,948] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 10ms
[2020-08-20 02:32:50,971] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 12ms
[2020-08-20 02:32:54,070] [8] [INFO] [trains.service_repo] Returned 200 for workers.status_report in 19ms
[2020-08-20 02:32:55,994] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 11ms
[2020-08-20 02:33:01,015] [8] [INFO] [trains.service_repo] Returned 200 for queues.get_next_task in 10ms

The log says GET http://elasticsearch:9200/_template/events* was unsuccessful, but I can get the results on local and remote machines by entering http://<ip_addr>:9200/_template/events* in a web browser:
{"events_log":{"order":1,"index_patterns":["events-log-*"],"settings":{},"mappings":{"properties":{"msg":{"index":false,"type":"text"},"level":{"type":"keyword"}}},"aliases":{}},"events":{"order":0,"index_patterns":["events-*"],"settings":{"index":{"number_of_shards":"1"}},"mappings":{"_source":{"enabled":true},"properties":{"@timestamp":{"type":"date"},"task":{"type":"keyword"},"metric":{"type":"keyword"},"variant":{"type":"keyword"},"iter":{"type":"long"},"type":{"type":"keyword"},"worker":{"type":"keyword"},"value":{"type":"float"},"timestamp":{"type":"date"}}},"aliases":{}},"events_plot":{"order":1,"index_patterns":["events-plot-*"],"settings":{},"mappings":{"properties":{"plot_str":{"index":false,"type":"text"}}},"aliases":{}},"events_training_debug_image":{"order":1,"index_patterns":["events-training_debug_image-*"],"settings":{},"mappings":{"properties":{"key":{"type":"keyword"},"url":{"type":"keyword"}}},"aliases":{}}}

Hi @apwojcik,

The log says GET http://elasticsearch:9200/_template/events* was unsuccessful, but I can get the results on local and remote machines by entering http://<ip_addr>:9200/_template/events* in a web browser:

These errors are transient and related to the time it takes the ElasticSearch docker image to initialize (the trains-apiserver simply keeps trying to communicate until it succeeds).
Your issue might be related to browser cache (assuming you're using the same browser as you did with the previous server, and you access the new server on the same host address) - if so, you can try removing all cookies, reloading the UI using Ctrl-F5 and trying to login again.
If that doesn't help, can you please open your browser's developer tools panel, go to the Network tab and try reloading the UI (Ctrl-F5) - I assume some REST calls will return an error, and I'd like to see which ones 🙂

Hi @jkhenning,
Thank you for your help! Issue resolved  - after clearing browser cache, and cookies all started working as expected.


Sort by logged in user

Closes #625

Unfortunately this still does not work quite right, it does not actually change the ordering of the list in admin/_topic_options.html.erb

you here?

now that i know, you wanna take this private?

now that i have at least three of the same person here. we can have a conversation


Command queue is now scrollable

Thanks for this awesome package. I hope you appreciate my suggestion.

No big change, please submit 1 commit.

Done


canInsertTextAfter exception for composition

https://github.com/facebook/lexical/pull/5363 parity for canInsertAfter (see the other PR and https://github.com/facebook/lexical/issues/5162 for more details). On the playground, hashtags can be written after so I switched it to false to test.
Before
https://github.com/facebook/lexical/assets/193447/9c00e7f5-5dd7-441f-a587-767124793c2f
After
https://github.com/facebook/lexical/assets/193447/73969e80-0eba-498e-bf6e-0073d33be1a8

Yea makes sense.


Upsampling causing CudaException code 2

The following breaks with CUDA:
_faceRecognition.FaceLocations(image, 1, _model).ToList();

The following works w/wo CUDA:
_faceRecognition.FaceLocations(image, 0, _model).ToList();

In python I do 1 upsample and it works fine.
Looks like a memory leak?
cc @takuya-takeuchi

I was not able to reproduce issue.
I tested FaceLocations methjod on the following pattern

numberOfTimesToUpsample

0 or 1

model

cnn or hog

image

https://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/Official_portrait_of_President_Obama_and_Vice_President_Biden_2012.jpg

Did you face this issue everytime when passing 0 for numberOfTimesToUpsample?

@takuya-takeuchi try with a large input image. 0 always works, 1 always blows up for large images.

Model CNN

Hi,
I got the exact same error when trying to run CUDA with the following image and upsampling set to 1:
https://media.gettyimages.com/photos/-picture-id906162?s=2048x2048

Actually, it happens to me even with upsampling set to 0 given big enough images. For example: one image of size 1568 x 1981 and another one of size 1675 x 2048 processed within the same runtime process.

The following sample program can reproduce issue.
using System;
using DlibDotNet;
using FaceRecognitionDotNet;

namespace test
{
    class Program
    {
        static void Main(string[] args)
        {
            const string filename = "gettyimages-906162-2048x2048.jpg";
            var fr = FaceRecognition.Create(@"D:\Works\OpenSource\FaceRecognitionDotNet\test\FaceRecognitionDotNet.Tests\bin\Release\netcoreapp2.0\Models");
            using (var image = FaceRecognition.LoadImageFile(filename))
                for (var num = 0; num < 5; num++)
                    try
                    {
                        foreach (var face in fr.FaceLocations(image, num, Model.Cnn))
                            Console.WriteLine($"numberOfTimesToUpsample={num}: {face.Left}, {face.Top}, {face.Right}, {face.Bottom}");
                    }
                    catch (CudaException e)
                    {
                        Console.WriteLine($"CUDA Error code: {e.ErrorCode}");
                    }
        }
    }
}

The above program output the following message
numberOfTimesToUpsample=0: 317, 285, 600, 568
numberOfTimesToUpsample=1: 321, 273, 614, 566
CUDA Error code: 2
CUDA Error code: 2
CUDA Error code: 2

CUDA Error code 2 is cudaErrorMemoryAllocation = 2
My dev machine is GTX 1080 and it has 8GB GPU RAM.
So FaceRecogtnitionDotNet.CUDA could throw exception.
In other words, FaceRecogtnitionDotNet works fine but it very slow.
The above program consumed more than 15G ram by FaceRecogtnitionDotNet cpu.
However, @turowicz said 'In python I do 1 upsample and it works fine.'

I tried example program.
from PIL import Image
import face_recognition

# Load the jpg file into a numpy array
image = face_recognition.load_image_file("gettyimages-906162-2048x2048.jpg")

# Find all the faces in the image using a pre-trained convolutional neural network.
# This method is more accurate than the default HOG model, but it's slower
# unless you have an nvidia GPU and dlib compiled with CUDA extensions. But if you do,
# this will use GPU acceleration and perform well.
# See also: find_faces_in_picture.py
for num in range(0, 5):
    face_locations = face_recognition.face_locations(image, number_of_times_to_upsample=num, model="cnn")

    print("I found {} face(s) in this photograph.".format(len(face_locations)))

    for face_location in face_locations:

        # Print the location of each face in this image
        top, right, bottom, left = face_location
        print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))

It occurs the following output.
>python test.py
I found 1 face(s) in this photograph.
A face is located at pixel location Top: 285, Left: 317, Bottom: 568, Right: 600
I found 1 face(s) in this photograph.
A face is located at pixel location Top: 273, Left: 321, Bottom: 567, Right: 615
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    face_locations = face_recognition.face_locations(image, number_of_times_to_upsample=num, model="cnn")
  File "D:\Works\Python\envs\face_recognition\lib\site-packages\face_recognition\api.py", line 116, in face_locations
    return [_trim_css_to_bounds(_rect_to_css(face.rect), img.shape) for face in _raw_face_locations(img, number_of_times_to_upsample, "cnn")]
  File "D:\Works\Python\envs\face_recognition\lib\site-packages\face_recognition\api.py", line 100, in _raw_face_locations
    return cnn_face_detector(img, number_of_times_to_upsample)
RuntimeError: Error while calling cudaMalloc(&data, new_size*sizeof(float)) in file D:\Works\Lib\Dlib\19.17\dlib\cuda\gpu_data.cpp:218. code: 2, reason: out of memory

It is similar with C# example on the  previous post.

Looks like its a valid behaviour then.


Question: Is it possible to use config.json with a Snap install ?

Question
Hello,
Is it possible to use config.json when managing Semaphore via Snap ?
Indeed, some options like oidc_providers are not (yet?) available in Snap configuration options, and documentation says that "Snap configurations should be used for when Semaphore was installed via Snap."...
Thank you for your help and for maintaining this project
Related to
Configuration, Documentation

I also faced the same issue; but I found /root/snap/semaphore/common/config.json, which is managed by Snap, but if you aren't updating snap / snap config it seems to be working well

Finally I choosed the manual install, I faced some permission issues and wanted to understand how it worked underneath


rgp-cli: init at 0.2.0

Motivation for this change
Cargo.lock isn't checked in, for now we patch it, but I opened a issue https://github.com/facundoolano/rpg-cli/issues/26
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Added a release notes entry if the change is major or breaking
[x] Fits CONTRIBUTING.md.


Can you also add base images for openjdk 10 and openjdk 11

https://github.com/Transmode/gradle-docker/blob/3d22f0904058a3530ff3eda31ea54f78377ec01e/src/main/java/se/transmode/gradle/plugins/docker/JavaBaseImage.java#L27

Sorry, I saw you have a commit for the most recent versions already. Hope they will be merged soon

shouldn't this work:
docker {
baseImage = 'openjdk:11-jre-slim'
...
}
It fails on my because of
Caused by: java.lang.IllegalArgumentException: No Java base image for the supplied target 11 found.


v.3.0.2610 (nightly build) does not save previous location & size of the main window

This nightly build doesn't seem to remember/save the previous main Process Hacker window location & size on the screen and resets it upon every launch:

Not confirmed. Try checkif IsConfigReadOnly. Example cuse CMD console and type this OS command
cd /d
attrib *.settngs,xml
and read attrib output. Files this your see "R" symbol is read only. Then use command  attrib -r filename. Also your can don't have write rings to PH config - then edit ACL use any OC tools.
But, possible your have locally source for this problem...

@VictorVG Oh, I see what’s going on. If it’s saving its configuration in a file in its installation folder, and if such folder is %ProgramFiles%, the config file will not be accessible for writing if PH was not started as administrator.
I’m not sure if this was intentional, but it might be better to save configuration file in the %appdata% folder instead.

By default config save to %appdata%. And% ProgramFiles% is not writable by users - see its ACL. Otherwise, someone will make mistakes. "Foolproof", albeit effective.

@VictorVG that's what makes it not save previous window location if PH is not started under administrative account. I wouldn't call it "albeit effective" though.

Process Hacker source, ./ProcessHacker/main.c::1038:
    if (!PhStartupParameters.NoSettings)
    {
        static PH_STRINGREF settingsPath = PH_STRINGREF_INIT(L"%APPDATA%\\Process Hacker\\settings.xml");
        static PH_STRINGREF settingsSuffix = PH_STRINGREF_INIT(L".settings.xml");
        PPH_STRING settingsFileName;

        // There are three possible locations for the settings file:
        // 1. The file name given in the command line.
        // 2. A file named ProcessHacker.exe.settings.xml in the program directory. (This changes
        //    based on the executable file name.)
        // 3. The default location.


Removed interest-cohort is still referenced in README

Python Version
3.7.12
Django Version
2.2.27
Package Version
4.8.0
Description
Feature interest-cohort has been deleted in the latest release but it is still listed under README.
Copy-pasting the settings from README, leading to the not working solution:
  File "/home/docker/venv/lib/python3.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
    return WSGIHandler()
  File "/home/docker/venv/lib/python3.7/site-packages/django/core/handlers/wsgi.py", line 135, in __init__
    self.load_middleware()
  File "/home/docker/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 37, in load_middleware
    mw_instance = middleware(handler)
  File "/home/docker/venv/lib/python3.7/site-packages/django_permissions_policy/__init__.py", line 83, in __init__
    self.header_value  # Access at setup so ImproperlyConfigured can be raised
  File "/home/docker/venv/lib/python3.7/site-packages/django/utils/functional.py", line 80, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/docker/venv/lib/python3.7/site-packages/django_permissions_policy/__init__.py", line 101, in header_value
    raise ImproperlyConfigured(f"Unknown feature {feature}")
django.core.exceptions.ImproperlyConfigured: Unknown feature interest-cohort

Could you please update the README for the supported features
P.S. also cannot find in the spec this feature: ch-ua-wow64. Checked here

Done. Next time please just open a PR for a one line change like this.

P.S. also cannot find in the spec this feature: ch-ua-wow64. Checked here

The spec is normally out of date with the hints shipped in Chrome. I don't always know what they are.


Wip/refactor

This is a work in progress.  Basically a fundamental refactor with great performance improvements.  Unfortunately at the moment there is a 2us hit in the serializer that I am trying to track down.
Also, some files have multiple classes and I still need to format the code.
Here are the current stats on my machine.
// * Summary *

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-4820K CPU 3.70GHz, ProcessorCount=8
Frequency=3613278 Hz, Resolution=276.7570 ns, Timer=TSC
  [Host]     : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1586.0
  DefaultJob : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1586.0

                            Method |       Mean |    StdDev |  Gen 0 | Allocated |
---------------------------------- |----------- |---------- |------- |---------- |
   SerializationClassWithPrimitive | 52.4981 us | 0.2014 us | 4.7363 |  37.58 kB |
 DeserializationClassWithPrimitive | 57.4954 us | 0.0766 us | 2.7832 |   27.8 kB |

Thank!


more examples

https://github.com/partrita/justPy

Closing since upstream https://github.com/justpy-org/justpy/issues/555 addresses the examples with over 200 examples from the justpy tutorial at https://justpy.io/


Add testing and support for Python 3.12

I have confirmed that tests pass with Python 3.12, it just isn't available in GitHub Actions yet so merging.


GAME_MASTER 0.75.0

Finally picked up a development device, @celandro -- very easy to regenerate -- great work on this. It's helped me a lot so thank you and happy to contribute a tiny amount of work back.
Summary
Adds GAME_MASTER 0.75.0

Adds Legedary Raid Ticket
Replaces Snarl with Hidden Power (Suicune)
Safari Zone Medal
Mewtwo and Legendary Dog Capture Rate adjustments

Consistent with what everyone else has seen.

Lgtm thanks!


Error inconsistence

In Produce, we throw error when Produce return non 0 (librdkafka know it won't be produced), but complete and fill the Error property of Message when it's a non immediate error
So in the case the topic is unknown, we will have an unthrown exception sometimes (a message with the error field not null, because it's not flagged as permanent unknown topic), then throw an exception - for the same call with the same argument
This seems shaky in the .NET client.
What about never throwing in the Produce and instead add a HandleException in IDeliveryHandler, which would be call instead of throwing? Produce would never throw in this case, the only errors from producer would come from OnError (and/or OnPoduceError if we add this- but that's also the problem between which one we chose)
I think there is many place now where we can get error, and we should document them more - in the message, OnConsumeError, OnError, throwing when producing...
I would go with this:
OnError: error in the internal library. Not bound to producing or consuming a specific message. Broker unavailable, network failure... The error ismore or less severe and could be only informative (link to a post in librdkafka)
OnConsumeError: failed to consume a message. This may or may not be specific to a message (in which case topic, partition and offset will be set accordingly in Message). See the Message.Error for more details
ProduceAsync: will not throw because of Confluent.Kafka (may because of async .NET framework) The delivery report may contain error, in which case it will be passed through IDeliveryHandler.HandleDeliveryReport(Message) with Error field set or IDeliveryHandler.HandleException(Error)

yep - this needs some thought, but I suspect we can do better here.
We could possibly just call HandleDeliveryReport on the IDeliveryHandler immediately with the error value and other info we have that was supplied to the produce method.
Initial reaction to OnProduceError is not into that idea.

Would it make sense to sort this out in librdkafka instead?
We could add a configuration property that propagates these types of error (or all, QUEUE_FULL too?) through the delivery report too.

Could make sense. But I see the point in librdkafka to return directly rather than using deliveryreportcallback for immediate failure - let you see if it's better to have a single error endpoint or not in C, we can easily handle it on our side, just that it's message in .NET client to have so many errors endpoints and behaviour (throw vs not throw...)
I'm wondering by the way, why does librdkafka return RD_KAFKA_RESP_ERR__UNKNOWN_TOPIC marked as permanent state in the doc? If we create the topic after calling rd_kafka_produce, won't it be non-permanent?

Temporary errors are those that stand a chance of resolving "automatically" within reasonable time:
connection errors, temporary states (no leader for partition), etc.
Permanent errors are the opposite, those that will require manual intervention:
if a topic does not exist and is not auto-created then it'll require some external entity (such as a human) to create it, we'll not hold our breath waiting for that to happen.

Understood, thanks :)

Well, preventing ProduceAsync to throw is an issue when user don't provide an IDeliveryHandler as we can't inform him - but not providing IDeliveryHandler can also be considered as a "fire and forget", so this seems ok.
As this is related, why was disableDeliveryReport introduced ?
If we enable it and use the Task ProduceAsync, then the task is not awaitable, so there is a problem ( #201 )
If we pass null to void ProduceAsync, we achieve the same things than enabling this config (and as it returns void, it's more logical)
=> I would remove it and let the user pass null or not, or use the appropriote config.

Non-zero return would be treated as either:

a Task exception, called by IDeliveryHandler.HandleException
a DeliveryHandler with Message.Error setted (I would go for this one)

Task ProduceAsync would directly set an exception (via one of the both option above) to inform there will be no callback if callbacks are disabled

This way:

No more throw
return Task always complete, eventually with Error

disableDeliveryReport was added as an optimization. You are right - there is absolutely no need for it as the code currently stands (I'm guessing at one point i wasn't checking for deliveryHandler != null. We should remove this.


Item converters

Hi,
I have used ddeboer/data-import a while ago on several projects with great success and I am now trying to use portphp on a new one.
One feature I regularly used in data-import were the Item converters which made it easy to transform items before writing them.
I have read in the documentation that these converters were removed.
How can I get similar functionality now?
For example:

How can I remap my item keys (MappingItemConverter) to the field names used in my doctrine entities?
How can I transform my item values to other values (which was previously done using CallbackItemConverter)?

I think you can use ConverterStep http://portphp.readthedocs.io/en/latest/workflow/#mappingstep and MappingStep http://portphp.readthedocs.io/en/latest/workflow/#converterstep

@aaa2000 is completly right. This functionality is not removed rather moved to the new step approach.

Thanks!


Add image manipulation cog

This is the start of what I plan on making a much larger project. All commands are documented in the code.

There are a lot of commands, but if you want i can provide screenshots

Keep developing this branch, I'll push to prod when there's a few more commands.
Take all the time you need

This bot no longer has any integration with the mattias krantz discord server 😄 feel free to edit out those commits.

bro
sad

ok done


consolidate to a single package.json

[x] Move all dependencies to a single top-level package.json so that a single npm install is all that's needed

[x] extension
[x] shell
[ ] devtools

[x] Redirect paths to the top-level node_modules directory
[x] Move shell/package.json non-build logic (e.g. start, stop targets) to either sigh or the top-level package.json
[x] Redirect Travis to this build target

All done.
There's a big piece of work left to finish consolidating into sigh that'll happen as part of https://github.com/PolymerLabs/arcs/issues/926.


Support multiple DNS servers

Description of the change
Adds support for configuring multiple DNS servers (for example, primary and secondary) for the gateway admission controller. This kind of configuration is standard practice to add redundancy to the name resolution.
The change is backwards compatible with the current implementation, and allows specifying the DNS as DNS: "<IP_ADDRESS>" or DNS: "<IP_ADDRESS>,<IP_ADDRESS>".

This a great extension. Thanks.
We will also need a documentation update of the Helm chart. Would you be able to do a PR there as well?

Hi @angelnu sorry for the late reply, I have created https://github.com/angelnu/helm-charts/pull/70 updating the documentation.


Satellite track changes beyond 24 hours.

Hello Shane,
I am using the tle2czml creator to display multiple satellites over days. When you set the end time greater than 24 hours beyond the start time and then view the satellite orbit in the period past the first 24 hours, the satellite track changes from a single line to multiple lines. Do you know why this is and how I could keep the track as one line?
Thanks
Joel

Fixed


[diem framework] delete tiered minting logic

Tiered minting is no longer a requirement, so this PR eliminates all logic related to tier restrictions.
The tier_index parameter to the tiered_mint public function + transaction script remain for backward compatibility, but will now be ignored.
In addition, tiered_mint<T>(dd_address, ...) will now delete the TierInfo<T> resource under dd_address. We can delete the TierInfo resource altogether once there are no longer any left in the system.

/land


b2SetLengthUnitsPerMeter or any equivalent doesn't seem to exist yet

It looks like the function b2SetLengthUnitsPerMeter or an equivalent doesn't yet exist. If it does, what's the function name?

I believe b2SetLengthUnitsPerMeter  is a feature of the new box2d v3 . box2d-netstandard is a C# port of "box2d v2"

I see. I'll close this issue :)


time until next class not working

How to reproduce

Run current master branch
have no current class
Press the fab
you will see that it does not display the time until the next class starts

Bug resolved.


terraform fmt being run in .terraform dir

Not sure why yet but this hook runs in the .terraform dir for me but from the code it looks like that dir should be ignored?
I will investigate further and post an update.

Make sure that you are using correct version of the hook. Also, you can try to clean ~/.cache/pre-commit and run pre-commit install to see if it resolves the issue.

I am using the latest version of the hook (1.4.0).
But I do still get this behaviour. If I have an existing tf repo with a .terraform dir and I run pre-commit run -a it will run terraform fmt on files in the .terraform dir too (downloaded modules) - I'm trying to determine if it is only happening on internal modules or any modules.
I'll check if there is a verbose/debug mode for pre-commit to see if it's apparent why this happens for me.

Do you have .terraform in your .gitignore? If not, you should.
In fact it should not be related to the issue you specify, because those files should be ignored by pre-commit hook.

Yeah I always include .terraform/ in my repo gitignores.
I wonder if this is perhaps a pre-commit issue rather than an issue with this hook where (for me) it is not ignoring the files it is meant to...?
Still testing (in between real job! :grin:)

pre-commit passes the correct set of files to terraform_fmt.sh.
However, terraform_fmt.sh calls dirname on each file to create a set of paths:
paths[index]=$(dirname "$file_with_path")

These directories are then passed to terraform fmt.
If there exists a .terraform directory within any of those directories then they too will be processed by terraform fmt.
Why is terraform fmt ran on the directory containing each file rather than just the file itself?

This can be replicated by creating a directory with the following structure:
.
├── .pre-commit-config.yaml
├── .terraform
│   └── invalid.tf
└── valid.tf

where invalid.tf is a syntactically incorrect Terraform file such as:
invalid

The standard pre-commit requirements will need to be performed (install pre-commit, terraform, this hook, git stuff, etc).
Running pre-commit run produces:
$ pre-commit run
Terraform fmt............................................................Failed
hookid: terraform_fmt

Error running fmt: In .terraform/invalid.tf: key 'invalid' expected start of object ('{') or assignment ('=')

I can confirm the behaviour reported by @samgd .

terraform fmt does indeed include the capability to run against a single file (even though it isn't documented) so continuing from what @samgd has added I think the hook could be improved by:

only running against files and not by directory - we only want to act on files included in the commit and should ignore the rest of the directory
ignoring the .terraform dir

If that sounds acceptable I would be happy to start a PR?

@sysadmiral @samgd thanks for keeping this issue alive :)
pre-commit sends a list of changed files to process to the hook, and then the hook (terraform_fmt.sh) creates a list of unique directories where changed files are and runs terraform fmt in each of these directories. It is much faster than running terraform fmt on files one by one.
I wonder whether pre-commit tracks changes in .terraform directory even when it should be ignored by .gitignore?
Does exclude work as expected in config - https://github.com/antonbabenko/pre-commit-terraform/blob/master/.pre-commit-hooks.yaml#L7 ? Maybe regexp is bad.
If exclude from above is ok, terraform_fmt.sh should force exclude the content of .terraform (and .terragrunt-cache for the same reason).

Are there any updates on this issue? I see the same issue with tfsec not ignoring the .terraform directory.

 to unstage)
        new file:   .pre-commit-config.yaml
        new file:   .terraform/invalid.tf
        new file:   valid.tf

22:19 test-8 git:(main +)
➜ pre-commit run -a

Terraform fmt............................................................Passed

22:19 test-8 git:(main +)
➜ pre-commit --version
pre-commit 2.14.0

22:19 test-8 git:(main +)
➜ terraform --version | head -n 1
Terraform v1.0.6

22:19 test-8 git:(main +)
➜

And not reproduced for terrafrom_tfsec too.
22:25 test-8 git:(main +)
➜ pre-commit run -a

Terraform validate with tfsec............................................Passed

Feel free to reopen the issue with details about your environment and reproduction steps if the issue is still present.


[BUG]: Unable to download subtitle. Access Denied

Program version
5.1.5
Operating System
Windows
Type
GUI
Service
Crunchyroll
Command used
aniDL.exe
Show ID
G6MGP0K46
Episode
3
Console Output
[DEBUG] Got resolve options: {"id":"G6MGP0K46","q":0,"e":"1,2,3","dubLang":["jpn"],"dlsubs":["all"],"fileName":"[${service}] ${showTitle} - S${season}E${episode} [${height}p]","dlVideoOnce":false,"all":false,"but":false,"noaudio":false,"novids":false,"simul":false}
[E1] Ghost Stories (Subbed) - Season 1 - The Ghosts Revive Tonight!! Amanojaku [☆ English, ☆ Japanese]
[E2] Ghost Stories (Subbed) - Season 1 - The Hand that Came Out of the Toilet... Red Paper, Blue Paper [☆ English, ☆ Japanese]
[E3] Ghost Stories (Subbed) - Season 1 - Raise the Curtain!! Cursed School Arts Festival Kutabe!! [☆ English, ☆ Japanese]
[E4] Ghost Stories (Subbed) - Season 1 - Requiem from the Dead — Elise [☆ English, ☆ Japanese]
[E5] Ghost Stories (Subbed) - Season 1 - The Bloody Sports Festival — Datto!! [☆ English, ☆ Japanese]
[E6] Ghost Stories (Subbed) - Season 1 - The Demon's Hand Splits the Door — Night of Tragedy [☆ English, ☆ Japanese]
[E7] Ghost Stories (Subbed) - Season 1 - The Soul-Stealing Mirror!! Utsushimi [☆ English, ☆ Japanese]
[E8] Ghost Stories (Subbed) - Season 1 - The Circuit Connects to Hell — Demon of the Underworld [☆ English, ☆ Japanese]
[E9] Ghost Stories (Subbed) - Season 1 - The Corpse that Roams in the Night — Shirotabi [☆ English, ☆ Japanese]
[E10] Ghost Stories (Subbed) - Season 1 - The Tunnel with No Exit — Anamaneki [☆ English, ☆ Japanese]
[E11] Ghost Stories (Subbed) - Season 1 - The Talking Merry Doll!! Shadow of Terror [☆ English, ☆ Japanese]
[E12] Ghost Stories (Subbed) - Season 1 - The Nurse Who Tells Your Death — Mother's Feelings [☆ English, ☆ Japanese]
[E13] Ghost Stories (Subbed) - Season 1 - The Picture that Swallows People — Da Vinci [☆ English, ☆ Japanese]
[E14] Ghost Stories (Subbed) - Season 1 - The Life-Taking Psychic Photo — Railway Crossing of Evil [☆ English, ☆ Japanese]
[E15] Ghost Stories (Subbed) - Season 1 - The Devil's Spell — Rite of Darkness [☆ English, ☆ Japanese]
[E16] Ghost Stories (Subbed) - Season 1 - The Apartment that Eats People!! Nest of Evil Spirits [☆ English, ☆ Japanese]
[E17] Ghost Stories (Subbed) - Season 1 - Terror at Bloodstain Lake!! Apparition in the Snow [☆ English, ☆ Japanese]
[E18] Ghost Stories (Subbed) - Season 1 - Akane-san of the Broadcasting Room!! Voice of the Dead [☆ English, ☆ Japanese]
[E19] Ghost Stories (Subbed) - Season 1 - The Headless Horsemen!! Curse of Death [☆ English, ☆ Japanese]
[E20] Ghost Stories (Subbed) - Season 1 - Farewell, Amanojaku — The Descent of Ohma [☆ English, ☆ Japanese]

------------------------------

[S1E1] - The Ghosts Revive Tonight!! Amanojaku [✓ Japanese]
[S1E2] - The Hand that Came Out of the Toilet... Red Paper, Blue Paper [✓ Japanese]
[S1E3] - Raise the Curtain!! Cursed School Arts Festival Kutabe!! [✓ Japanese]
USER: Sean1250<EMAIL_ADDRESS>[DEBUG] Got resolve options: {"id":"G6MGP0K46","q":0,"e":"1,2,3","dubLang":["jpn","eng"],"dlsubs":["all"],"fileName":"[${service}] ${showTitle} - S${season}E${episode} [${height}p]","dlVideoOnce":false,"all":false,"but":false,"noaudio":false,"novids":false,"simul":false}
[E1] Ghost Stories (Subbed) - Season 1 - The Ghosts Revive Tonight!! Amanojaku [☆ English, ☆ Japanese]
[E2] Ghost Stories (Subbed) - Season 1 - The Hand that Came Out of the Toilet... Red Paper, Blue Paper [☆ English, ☆ Japanese]
[E3] Ghost Stories (Subbed) - Season 1 - Raise the Curtain!! Cursed School Arts Festival Kutabe!! [☆ English, ☆ Japanese]
[E4] Ghost Stories (Subbed) - Season 1 - Requiem from the Dead — Elise [☆ English, ☆ Japanese]
[E5] Ghost Stories (Subbed) - Season 1 - The Bloody Sports Festival — Datto!! [☆ English, ☆ Japanese]
[E6] Ghost Stories (Subbed) - Season 1 - The Demon's Hand Splits the Door — Night of Tragedy [☆ English, ☆ Japanese]
[E7] Ghost Stories (Subbed) - Season 1 - The Soul-Stealing Mirror!! Utsushimi [☆ English, ☆ Japanese]
[E8] Ghost Stories (Subbed) - Season 1 - The Circuit Connects to Hell — Demon of the Underworld [☆ English, ☆ Japanese]
[E9] Ghost Stories (Subbed) - Season 1 - The Corpse that Roams in the Night — Shirotabi [☆ English, ☆ Japanese]
[E10] Ghost Stories (Subbed) - Season 1 - The Tunnel with No Exit — Anamaneki [☆ English, ☆ Japanese]
[E11] Ghost Stories (Subbed) - Season 1 - The Talking Merry Doll!! Shadow of Terror [☆ English, ☆ Japanese]
[E12] Ghost Stories (Subbed) - Season 1 - The Nurse Who Tells Your Death — Mother's Feelings [☆ English, ☆ Japanese]
[E13] Ghost Stories (Subbed) - Season 1 - The Picture that Swallows People — Da Vinci [☆ English, ☆ Japanese]
[E14] Ghost Stories (Subbed) - Season 1 - The Life-Taking Psychic Photo — Railway Crossing of Evil [☆ English, ☆ Japanese]
[E15] Ghost Stories (Subbed) - Season 1 - The Devil's Spell — Rite of Darkness [☆ English, ☆ Japanese]
[E16] Ghost Stories (Subbed) - Season 1 - The Apartment that Eats People!! Nest of Evil Spirits [☆ English, ☆ Japanese]
[E17] Ghost Stories (Subbed) - Season 1 - Terror at Bloodstain Lake!! Apparition in the Snow [☆ English, ☆ Japanese]
[E18] Ghost Stories (Subbed) - Season 1 - Akane-san of the Broadcasting Room!! Voice of the Dead [☆ English, ☆ Japanese]
[E19] Ghost Stories (Subbed) - Season 1 - The Headless Horsemen!! Curse of Death [☆ English, ☆ Japanese]
[E20] Ghost Stories (Subbed) - Season 1 - Farewell, Amanojaku — The Descent of Ohma [☆ English, ☆ Japanese]

------------------------------

[S1E1] - The Ghosts Revive Tonight!! Amanojaku [✓ English, ✓ Japanese]
[S1E2] - The Hand that Came Out of the Toilet... Red Paper, Blue Paper [✓ English, ✓ Japanese]
[S1E3] - Raise the Curtain!! Cursed School Arts Festival Kutabe!! [✓ English, ✓ Japanese]
USER: Sean1250<EMAIL_ADDRESS>[DEBUG] Got download options: {"id":"G6MGP0K46","q":0,"e":"4","dubLang":["jpn","eng"],"dlsubs":["en"],"fileName":"${showTitle} - S${season}E${episode} - ${title}","dlVideoOnce":false,"all":true,"but":false,"noaudio":false,"novids":false,"simul":false,"ids":["GR8VX8D1R","GR8VX8D1R"],"title":"Requiem from the Dead — Elise","parent":{"title":"Ghost Stories (Subbed)","season":"1"},"image":"https://www.crunchyroll.com/imgsrv/display/thumbnail/1200x675/catalog/crunchyroll/19ead2bc36614d9d28a2e8262baf5d79.jpg","episode":"4"}
[E1] Ghost Stories (Subbed) - Season 1 - The Ghosts Revive Tonight!! Amanojaku [☆ English, ☆ Japanese]
[E2] Ghost Stories (Subbed) - Season 1 - The Hand that Came Out of the Toilet... Red Paper, Blue Paper [☆ English, ☆ Japanese]
[E3] Ghost Stories (Subbed) - Season 1 - Raise the Curtain!! Cursed School Arts Festival Kutabe!! [☆ English, ☆ Japanese]
[E4] Ghost Stories (Subbed) - Season 1 - Requiem from the Dead — Elise [☆ English, ☆ Japanese]
[E5] Ghost Stories (Subbed) - Season 1 - The Bloody Sports Festival — Datto!! [☆ English, ☆ Japanese]
[E6] Ghost Stories (Subbed) - Season 1 - The Demon's Hand Splits the Door — Night of Tragedy [☆ English, ☆ Japanese]
[E7] Ghost Stories (Subbed) - Season 1 - The Soul-Stealing Mirror!! Utsushimi [☆ English, ☆ Japanese]
[E8] Ghost Stories (Subbed) - Season 1 - The Circuit Connects to Hell — Demon of the Underworld [☆ English, ☆ Japanese]
[E9] Ghost Stories (Subbed) - Season 1 - The Corpse that Roams in the Night — Shirotabi [☆ English, ☆ Japanese]
[E10] Ghost Stories (Subbed) - Season 1 - The Tunnel with No Exit — Anamaneki [☆ English, ☆ Japanese]
[E11] Ghost Stories (Subbed) - Season 1 - The Talking Merry Doll!! Shadow of Terror [☆ English, ☆ Japanese]
[E12] Ghost Stories (Subbed) - Season 1 - The Nurse Who Tells Your Death — Mother's Feelings [☆ English, ☆ Japanese]
[E13] Ghost Stories (Subbed) - Season 1 - The Picture that Swallows People — Da Vinci [☆ English, ☆ Japanese]
[E14] Ghost Stories (Subbed) - Season 1 - The Life-Taking Psychic Photo — Railway Crossing of Evil [☆ English, ☆ Japanese]
[E15] Ghost Stories (Subbed) - Season 1 - The Devil's Spell — Rite of Darkness [☆ English, ☆ Japanese]
[E16] Ghost Stories (Subbed) - Season 1 - The Apartment that Eats People!! Nest of Evil Spirits [☆ English, ☆ Japanese]
[E17] Ghost Stories (Subbed) - Season 1 - Terror at Bloodstain Lake!! Apparition in the Snow [☆ English, ☆ Japanese]
[E18] Ghost Stories (Subbed) - Season 1 - Akane-san of the Broadcasting Room!! Voice of the Dead [☆ English, ☆ Japanese]
[E19] Ghost Stories (Subbed) - Season 1 - The Headless Horsemen!! Curse of Death [☆ English, ☆ Japanese]
[E20] Ghost Stories (Subbed) - Season 1 - Farewell, Amanojaku — The Descent of Ohma [☆ English, ☆ Japanese]

------------------------------

[S1E4] - Requiem from the Dead — Elise [✓ English, ✓ Japanese]
Requesting: [GR8VX8D1R] Ghost Stories (Subbed) - 4 - Requiem from the Dead — Elise
[ERROR] 403: Forbidden
[ERROR] <?xml version="1.0" encoding="UTF-8"?>
<Error>AccessDenied<Message>Access Denied</Message><RequestId>PFSDK0EQAPKGGKV7</RequestId><HostId>lImXcxKs4n+/2q3WWmmU2TXys1whtyF+P7bdd42uzyahrHxjYpmSRfjrCr48Gf/7WeQaSjRGpv4=</HostId></Error>
[WARN] Chapter request failed, attempting old API
Old Chapter request successful
Selecting raw stream
Full stream found! (✓1: adaptive_chrome_dash_drm/eng/- )
Downloading video...
Playlists URL: https://cr-play-service.prd.crunchyrollsvc.com/v1/manifest/G6Q4QXJPR/evs3/67ac2c4cb48341ba936998a950837dc4/assets/p/1urrqsb9avagsdb_,2411999.mp4,2411997.mp4,2411995.mp4,15866.txt,.urlset/manifest.mpd?playbackGuid=11-369aaed9-ab25-49a6-9d98-79825da78a67&accountid=3211cb96-01fc-590e-a9e1-20e819a293c6 (adaptive_chrome_dash_drm/eng/-)
Servers available:
        a-vrv.akamaized.net
Available Video Qualities:
        [1] 320x240 (485KiB/s)
        [2] 480x360 (968KiB/s)
        [3] 640x480 (1932KiB/s)
Available Audio Qualities:
        [1] 64kB/s
        [2] 96kB/s
Selected quality:
        Video: 640x480 (1932KiB/s)
        Audio: 96kB/s
        Server: a-vrv.akamaized.net
Stream URL: https://a-vrv.akamaized.net/evs3/67ac2c4cb48341ba936998a950837dc4/assets/p/1urrqsb9avagsdb_,2411999.mp4,2411997.mp4,2411995.mp4,15866.txt
Total parts in video stream: 354 (36*10)
Saving stream to «D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-G6Q4QXJPR.video.enc.m4s»...
Download and save init part...
Init part downloaded.
10 of 354 parts downloaded [3%] (30s | 8.7Mb/s)
20 of 354 parts downloaded [6%] (40s | 9.0Mb/s)
30 of 354 parts downloaded [8%] (41s | 9.1Mb/s)
40 of 354 parts downloaded [11%] (38s | 9.4Mb/s)
50 of 354 parts downloaded [14%] (35s | 9.5Mb/s)
60 of 354 parts downloaded [17%] (31s | 9.7Mb/s)
70 of 354 parts downloaded [20%] (29s | 9.9Mb/s)
80 of 354 parts downloaded [23%] (27s | 10Mb/s)
90 of 354 parts downloaded [25%] (25s | 10Mb/s)
100 of 354 parts downloaded [28%] (24s | 10Mb/s)
110 of 354 parts downloaded [31%] (24s | 10Mb/s)
120 of 354 parts downloaded [34%] (22s | 10Mb/s)
130 of 354 parts downloaded [37%] (21s | 10Mb/s)
140 of 354 parts downloaded [40%] (20s | 10Mb/s)
150 of 354 parts downloaded [42%] (19s | 10Mb/s)
160 of 354 parts downloaded [45%] (18s | 10Mb/s)
170 of 354 parts downloaded [48%] (17s | 10Mb/s)
180 of 354 parts downloaded [51%] (16s | 10Mb/s)
190 of 354 parts downloaded [54%] (15s | 10Mb/s)
200 of 354 parts downloaded [56%] (14s | 10Mb/s)
210 of 354 parts downloaded [59%] (13s | 10Mb/s)
220 of 354 parts downloaded [62%] (12s | 10Mb/s)
230 of 354 parts downloaded [65%] (11s | 10Mb/s)
240 of 354 parts downloaded [68%] (11s | 10Mb/s)
250 of 354 parts downloaded [71%] (10s | 10Mb/s)
260 of 354 parts downloaded [73%] (9s | 10Mb/s)
270 of 354 parts downloaded [76%] (8s | 10Mb/s)
280 of 354 parts downloaded [79%] (7s | 10Mb/s)
290 of 354 parts downloaded [82%] (6s | 10Mb/s)
300 of 354 parts downloaded [85%] (5s | 10Mb/s)
310 of 354 parts downloaded [88%] (4s | 10Mb/s)
320 of 354 parts downloaded [90%] (3s | 10Mb/s)
330 of 354 parts downloaded [93%] (2s | 10Mb/s)
340 of 354 parts downloaded [96%] (1s | 10Mb/s)
350 of 354 parts downloaded [99%] (0s | 10Mb/s)
354 of 354 parts downloaded [100%] (0s | 10Mb/s)
Total parts in audio stream: 354 (36*10)
Saving stream to «D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-G6Q4QXJPR.audio.enc.m4s»...
Download and save init part...
Init part downloaded.
10 of 354 parts downloaded [3%] (2s | 7.4Mb/s)
20 of 354 parts downloaded [6%] (2s | 8.4Mb/s)
30 of 354 parts downloaded [8%] (2s | 8.9Mb/s)
40 of 354 parts downloaded [11%] (2s | 9.0Mb/s)
50 of 354 parts downloaded [14%] (2s | 9.1Mb/s)
60 of 354 parts downloaded [17%] (2s | 9.2Mb/s)
70 of 354 parts downloaded [20%] (2s | 8.2Mb/s)
80 of 354 parts downloaded [23%] (2s | 7.5Mb/s)
90 of 354 parts downloaded [25%] (2s | 7.6Mb/s)
100 of 354 parts downloaded [28%] (2s | 7.7Mb/s)
110 of 354 parts downloaded [31%] (2s | 7.8Mb/s)
120 of 354 parts downloaded [34%] (2s | 7.9Mb/s)
130 of 354 parts downloaded [37%] (1s | 8.0Mb/s)
140 of 354 parts downloaded [40%] (1s | 8.0Mb/s)
150 of 354 parts downloaded [42%] (1s | 8.0Mb/s)
160 of 354 parts downloaded [45%] (1s | 8.1Mb/s)
170 of 354 parts downloaded [48%] (1s | 8.2Mb/s)
180 of 354 parts downloaded [51%] (1s | 8.1Mb/s)
190 of 354 parts downloaded [54%] (1s | 8.2Mb/s)
200 of 354 parts downloaded [56%] (1s | 8.3Mb/s)
210 of 354 parts downloaded [59%] (1s | 8.3Mb/s)
220 of 354 parts downloaded [62%] (1s | 8.3Mb/s)
230 of 354 parts downloaded [65%] (1s | 8.4Mb/s)
240 of 354 parts downloaded [68%] (1s | 8.4Mb/s)
250 of 354 parts downloaded [71%] (1s | 8.4Mb/s)
260 of 354 parts downloaded [73%] (1s | 8.4Mb/s)
270 of 354 parts downloaded [76%] (1s | 8.5Mb/s)
280 of 354 parts downloaded [79%] (0s | 8.5Mb/s)
290 of 354 parts downloaded [82%] (0s | 8.5Mb/s)
300 of 354 parts downloaded [85%] (0s | 8.5Mb/s)
310 of 354 parts downloaded [88%] (0s | 8.6Mb/s)
320 of 354 parts downloaded [90%] (0s | 8.6Mb/s)
330 of 354 parts downloaded [93%] (0s | 8.6Mb/s)
340 of 354 parts downloaded [96%] (0s | 8.6Mb/s)
350 of 354 parts downloaded [99%] (0s | 8.7Mb/s)
354 of 354 parts downloaded [100%] (0s | 8.5Mb/s)
Decryption Needed, attempting to decrypt
Started decrypting video

> "mp4decrypt" --show-progress --key 5e3aa31087873e55b0ea4eb8788443c2:8056320e332e0245df4c6dbc535d9fbd "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-G6Q4QXJPR.video.enc.m4s" "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-G6Q4QXJPR.video.m4s"
Decryption done for video
Started decrypting audio

> "mp4decrypt" --show-progress --key 5e3aa31087873e55b0ea4eb8788443c2:8056320e332e0245df4c6dbc535d9fbd "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-G6Q4QXJPR.audio.enc.m4s" "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-G6Q4QXJPR.audio.m4s"
Decryption done for audio
[WARN] Can't find urls for subtitles!
Requesting: [GR8VX8D1R] Ghost Stories (Subbed) - 4 - Requiem from the Dead — Elise
[ERROR] 403: Forbidden
[ERROR] <?xml version="1.0" encoding="UTF-8"?>
<Error>AccessDenied<Message>Access Denied</Message><RequestId>3EGSEHDS0TYPXM7P</RequestId><HostId>DvX/MzldNC8mKLh4b6jgFuYswbWwUO9kcnYEcFjVMnHo6VUkYOl5CQ66GDmnY7gFw7Qv0+ViNGE=</HostId></Error>
[WARN] Chapter request failed, attempting old API
Old Chapter request successful
Selecting raw stream
Full stream found! (✓1: adaptive_chrome_dash_drm/jpn/- )
Downloading video...
Playlists URL: https://cr-play-service.prd.crunchyrollsvc.com/v1/manifest/GR8VX8D1R/evs3/52e1798898344f61a84162536b2aaf2b/assets/p/h4rm1bfji3sqmf5_,2412471.mp4,2412469.mp4,2412467.mp4,.urlset/manifest.mpd?playbackGuid=11-17d48db9-0f83-4b40-8468-dc9ef2eb2e29&accountid=3211cb96-01fc-590e-a9e1-20e819a293c6 (adaptive_chrome_dash_drm/jpn/-)
Servers available:
        a-vrv.akamaized.net
Available Video Qualities:
        [1] 320x240 (485KiB/s)
        [2] 480x360 (969KiB/s)
        [3] 640x480 (1932KiB/s)
Available Audio Qualities:
        [1] 64kB/s
        [2] 96kB/s
Selected quality:
        Video: 640x480 (1932KiB/s)
        Audio: 96kB/s
        Server: a-vrv.akamaized.net
Stream URL: https://a-vrv.akamaized.net/evs3/52e1798898344f61a84162536b2aaf2b/assets/p/h4rm1bfji3sqmf5_,2412471.mp4,2412469.mp4,2412467.mp4
Total parts in video stream: 354 (36*10)
Saving stream to «D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-GR8VX8D1R.video.enc.m4s»...
Download and save init part...
Init part downloaded.
10 of 354 parts downloaded [3%] (45s | 5.9Mb/s)
20 of 354 parts downloaded [6%] (43s | 8.4Mb/s)
30 of 354 parts downloaded [8%] (43s | 8.7Mb/s)
40 of 354 parts downloaded [11%] (42s | 8.5Mb/s)
50 of 354 parts downloaded [14%] (38s | 8.8Mb/s)
60 of 354 parts downloaded [17%] (33s | 9.2Mb/s)
70 of 354 parts downloaded [20%] (31s | 9.4Mb/s)
80 of 354 parts downloaded [23%] (30s | 9.2Mb/s)
90 of 354 parts downloaded [25%] (27s | 9.3Mb/s)
100 of 354 parts downloaded [28%] (26s | 9.4Mb/s)
110 of 354 parts downloaded [31%] (26s | 9.4Mb/s)
120 of 354 parts downloaded [34%] (25s | 9.3Mb/s)
130 of 354 parts downloaded [37%] (24s | 9.1Mb/s)
140 of 354 parts downloaded [40%] (22s | 9.3Mb/s)
150 of 354 parts downloaded [42%] (21s | 9.3Mb/s)
160 of 354 parts downloaded [45%] (20s | 9.3Mb/s)
170 of 354 parts downloaded [48%] (19s | 9.4Mb/s)
180 of 354 parts downloaded [51%] (17s | 9.4Mb/s)
190 of 354 parts downloaded [54%] (17s | 9.4Mb/s)
200 of 354 parts downloaded [56%] (15s | 9.4Mb/s)
210 of 354 parts downloaded [59%] (14s | 9.4Mb/s)
220 of 354 parts downloaded [62%] (13s | 9.4Mb/s)
230 of 354 parts downloaded [65%] (12s | 9.4Mb/s)
240 of 354 parts downloaded [68%] (12s | 9.4Mb/s)
250 of 354 parts downloaded [71%] (11s | 9.4Mb/s)
260 of 354 parts downloaded [73%] (10s | 9.4Mb/s)
270 of 354 parts downloaded [76%] (8s | 9.4Mb/s)
280 of 354 parts downloaded [79%] (7s | 9.4Mb/s)
290 of 354 parts downloaded [82%] (6s | 9.4Mb/s)
300 of 354 parts downloaded [85%] (5s | 9.5Mb/s)
310 of 354 parts downloaded [88%] (4s | 9.5Mb/s)
320 of 354 parts downloaded [90%] (3s | 9.4Mb/s)
330 of 354 parts downloaded [93%] (2s | 9.5Mb/s)
340 of 354 parts downloaded [96%] (1s | 9.5Mb/s)
350 of 354 parts downloaded [99%] (0s | 9.6Mb/s)
354 of 354 parts downloaded [100%] (0s | 9.5Mb/s)
Total parts in audio stream: 354 (36*10)
Saving stream to «D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-GR8VX8D1R.audio.enc.m4s»...
Download and save init part...
Init part downloaded.
10 of 354 parts downloaded [3%] (16s | 1.1Mb/s)
20 of 354 parts downloaded [6%] (15s | 1.2Mb/s)
30 of 354 parts downloaded [8%] (14s | 1.2Mb/s)
40 of 354 parts downloaded [11%] (12s | 1.3Mb/s)
50 of 354 parts downloaded [14%] (11s | 1.4Mb/s)
60 of 354 parts downloaded [17%] (11s | 1.4Mb/s)
70 of 354 parts downloaded [20%] (10s | 1.4Mb/s)
80 of 354 parts downloaded [23%] (10s | 1.4Mb/s)
90 of 354 parts downloaded [25%] (10s | 1.3Mb/s)
100 of 354 parts downloaded [28%] (11s | 1.2Mb/s)
110 of 354 parts downloaded [31%] (10s | 1.2Mb/s)
120 of 354 parts downloaded [34%] (10s | 1.2Mb/s)
130 of 354 parts downloaded [37%] (9s | 1.3Mb/s)
140 of 354 parts downloaded [40%] (9s | 1.3Mb/s)
150 of 354 parts downloaded [42%] (8s | 1.3Mb/s)
160 of 354 parts downloaded [45%] (8s | 1.3Mb/s)
170 of 354 parts downloaded [48%] (7s | 1.3Mb/s)
180 of 354 parts downloaded [51%] (7s | 1.3Mb/s)
190 of 354 parts downloaded [54%] (7s | 1.3Mb/s)
200 of 354 parts downloaded [56%] (6s | 1.3Mb/s)
210 of 354 parts downloaded [59%] (6s | 1.3Mb/s)
220 of 354 parts downloaded [62%] (5s | 1.3Mb/s)
230 of 354 parts downloaded [65%] (5s | 1.3Mb/s)
240 of 354 parts downloaded [68%] (4s | 1.3Mb/s)
250 of 354 parts downloaded [71%] (4s | 1.3Mb/s)
260 of 354 parts downloaded [73%] (4s | 1.3Mb/s)
270 of 354 parts downloaded [76%] (3s | 1.3Mb/s)
280 of 354 parts downloaded [79%] (3s | 1.3Mb/s)
290 of 354 parts downloaded [82%] (3s | 1.3Mb/s)
300 of 354 parts downloaded [85%] (2s | 1.3Mb/s)
310 of 354 parts downloaded [88%] (2s | 1.3Mb/s)
320 of 354 parts downloaded [90%] (1s | 1.3Mb/s)
330 of 354 parts downloaded [93%] (1s | 1.3Mb/s)
340 of 354 parts downloaded [96%] (1s | 1.3Mb/s)
350 of 354 parts downloaded [99%] (0s | 1.3Mb/s)
354 of 354 parts downloaded [100%] (0s | 1.3Mb/s)
Decryption Needed, attempting to decrypt
Started decrypting video

> "mp4decrypt" --show-progress --key fa74e606a05137e4ae9b81ddd70b386a:200af3e3e946ab0115dff97981eca211 "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-GR8VX8D1R.video.enc.m4s" "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-GR8VX8D1R.video.m4s"
Decryption done for video
Started decrypting audio

> "mp4decrypt" --show-progress --key fa74e606a05137e4ae9b81ddd70b386a:200af3e3e946ab0115dff97981eca211 "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-GR8VX8D1R.audio.enc.m4s" "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\temp-GR8VX8D1R.audio.m4s"
Decryption done for audio
[WARN] Can't find urls for subtitles!
[mkvmerge] Started merging

> "mkvmerge" -o "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.mkv" --no-date --disable-track-statistics-tags --engage no_variable_data --video-tracks 0 --no-audio --track-name 0:"English [Uncut]" --language 0:eng "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.video.m4s" --track-name 0:"English" --language 0:eng --no-video --audio-tracks 0 --default-track 0:0 "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.audio.m4s" --track-name 0:"Japanese" --language 0:jpn --no-video --audio-tracks 0 --default-track 0 "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.Japanese.audio.m4s" --no-subtitles --no-attachments --chapters "D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.txt"
mkvmerge v86.0 ('Winter') 64-bit
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.video.m4s': Using the demultiplexer for the format 'QuickTime/MP4'.
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.audio.m4s': Using the demultiplexer for the format 'QuickTime/MP4'.
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.Japanese.audio.m4s': Using the demultiplexer for the format 'QuickTime/MP4'.
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.video.m4s' track 0: Using the output module for the format 'AVC/H.264'.
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.audio.m4s' track 0: Using the output module for the format 'AAC'.
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.Japanese.audio.m4s' track 0: Using the output module for the format 'AAC'.
The file 'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.mkv' has been opened for writing.
'D:\networking\multi-downloader-nx\multi-downloader-nx-windows-x64-gui\videos\Ghost Stories (Subbed) - S01E04 - Requiem from the Dead — Elise.English.video.m4s' track 0: Extracted the aspect ratio information from the MPEG-4 layer 10 (AVC) video data and set the display dimensions to 640/485.
The cue entries (the index) are being written...
Multiplexing took 1 second.
[mkvmerge Done]

Additional Information
When trying to download the episode, it seems to gives 403: Forbidden error
[ERROR] 403: Forbidden
[ERROR] <?xml version="1.0" encoding="UTF-8"?>
<Error>AccessDenied<Message>Access Denied</Message><RequestId>PFSDK0EQAPKGGKV7</RequestId><HostId>lImXcxKs4n+/2q3WWmmU2TXys1whtyF+P7bdd42uzyahrHxjYpmSRfjrCr48Gf/7WeQaSjRGpv4=</HostId></Error>
[WARN] Chapter request failed, attempting old API

The video seems fine, but doesn't have subtitle.

The error (403 forbidden) refers to the chapter. Unfortunately there are no normal english subs, only cc subs and they are in vtt format.
And since the last API change from crunchy there are apparently problems downloading the vtt subs with anidl.
So there is no jap dub and eng sub for Ghost Stories and for the eng dub there are only the cc subs, if you click on the normal ones it doesn't load any subs in the browser.

Ah, I see. Thanks for the response.


[Main] Check in yarn.lock file in its current state

Description
This PR removes yarn.lock from gitignore, and checks in the current version of the generated yarn.lock file, after running yarn install. Yarn.lock provides a few benefits:

Provides reproducibility, by being a source of truth for which versions of each dependency were downloaded. This also allows for debugging - if package.json specifies ^1.2.0, you could be using 1.2.0, 1.2.1, etc.
Includes integrity hash to make sure you are downloading from a trusted source

Category
[Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation]
Why these changes are required?
What is the old behavior before changes and new behavior after changes?
Issues Resolved
[List any issues this PR will resolve (Is this a backport? If so, please add backport PR # and/or commits #)]
Testing
[Please provide details of testing done: unit testing, integration testing and manual testing]
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Updating branch due to the merge of https://github.com/opensearch-project/security-dashboards-plugin/pull/1579

Hi @derek-ho, I just pushed another commit to include the latest Cypress 13 and update the yarn file.


LICENSE file absent from pip based install

Should be a simple fix by adding a "package_data" configuration option to setup.py

Interesting, we should definitely add that.


No redirect

When attempting to access a non-existent page.

Thank you for your attention. I fix it already please check it not bug again.


ReqMgr2: OutputDatasets is not updated during assignment

in the reqmgr_workload_cache doc. It seems to keep the same OutputDatasets built during request creation and that's it. See example:
https://cmsweb-testbed.cern.ch/couchdb/_utils/document.html?reqmgr_workload_cache/pdmvserv_HIG-RunIISummer15GS-01331_00274_v2__160308_111346_296

Does more output dataset is created after initial creation of request (i.e. When workflow is assigned)?
Could you point the code if you know on top of your head?

The output datasets are really defined when we assign a workflow, though we could predict the output dataset names during request creation (in case we have enough information or default cases).
However, this info must be updated during assignment, so it should be around this method:
https://github.com/dmwm/WMCore/blob/a95a5667070358ec5fd54804d52a987fc4235a4e/src/python/WMCore/ReqMgr/Service/Request.py#L389


adding a folder on another linux host without mounting?

I'm running Polaris on my fileserver, where my music collection is stored. But there's no desktop environment on the server so I have to access Polaris from my laptop. How do I ensure that a music folder on my server is scanned, not one on my laptop?

Closing with comment: this was working all along, but for some reason took a long time to scan the music folder.


feat(nexus): add file upload progress and make completion callback a list of callbacks

Description

Fixes WB-NNNNN
Fixes #NNNN

What does the PR do?

🤖 Generated by Copilot at 266dd3e
This pull request refactors the upload task and manager types to support multiple callbacks for each upload task. This enables more flexible and concurrent handling of upload tasks and their results in the ArtifactSaver type and the file counts channel. The changes affect the files saver.go, upload_manager.go, and upload_task.go.
Testing
How was this PR tested?

🤖 Generated by Copilot at 266dd3e

To upload files with more flair
The UploadManager had to share
It added more callbacks to each task
And moved some logic to a new cask
Now the ArtifactSaver can handle them with care

Need to fix done, why does it exit early?


wlr_output_layout_get_box: handle empty layout

Fixes swaywm/sway#3734
If there were no outputs in the output layout,
wlr_output_layout_get_box would return the box:
{
  .x = INT_MIN,
  .y = INT_MIN,
  .width = INT_MIN - INT_MAX,
  .height = INT_MIN - INT_MAX
}

which results in an integer underflow for both the width and height.
This changes the logic to have the box be all zeroes, since an empty
output layout does not have a width or height and the location of
something without a size is irrelevant so this just uses the origin.

Thanks!


Content-Type parsing in juniper-actix is too strict

In juniper-actix, there's a piece of code that extracts the request Content-Type before deserializing the request body:
    let content_type_header = req
        .headers()
        .get(CONTENT_TYPE)
        .and_then(|hv| hv.to_str().ok());
    let req = match content_type_header {
        Some("application/json") => {
            let body = String::from_request(&req, &mut payload.into_inner()).await?;
            serde_json::from_str::<GraphQLBatchRequest<S>>(&body).map_err(ErrorBadRequest)
        }
        Some("application/graphql") => {
            let body = String::from_request(&req, &mut payload.into_inner()).await?;
            Ok(GraphQLBatchRequest::Single(GraphQLRequest::new(
                body, None, None,
            )))
        }
        _ => Err(ErrorUnsupportedMediaType(
            "GraphQL requests should have content type `application/json` or `application/graphql`",
        )),
    }?;

This piece of matching code breaks for any Content-Type that has a charset parameter (e.g. text/html; charset=utf-8).
I think the fix is to just use the content_type() method on the req object, which automatically splits by ';' and extracts the sub-string this code wants to match on.
Something like this:
    let content_type_header = req.content_type();

Would love a PR!

Would love a PR!


Library not working with most of oneplus devices

Summary

Code to reproduce

Android version

Impacted devices

Installation method

SDK version

Other information

solved after updating library


[zh] Sync SVG contents to concepts/architecture/_index.md

PR Summary

EN upstream: content/en/docs/concepts/architecture/_index.md
Sync to: content/zh-cn/docs/concepts/architecture/_index.md

Sync check by scripts/lsync.sh on main branch:
$ scripts/lsync.sh content/zh-cn/docs/concepts/architecture/_index.md
diff --git a/content/en/docs/concepts/architecture/_index.md b/content/en/docs/concepts/architecture/_index.md
index 61fb48e714..7c9a45c71e 100644
--- a/content/en/docs/concepts/architecture/_index.md
+++ b/content/en/docs/concepts/architecture/_index.md
@@ -5,3 +5,4 @@ description: >
   The architectural concepts behind Kubernetes.
 ---

+{{< figure src="/images/docs/kubernetes-cluster-architecture.svg" alt="Components of Kubernetes" caption="Kubernetes cluster architecture" class="diagram-large" >}}

Sync check byscripts/lsync.sh on sync/architecture_index_20231022 branch:
$ scripts/lsync.sh content/zh-cn/docs/concepts/architecture/_index.md
content/zh-cn/docs/concepts/architecture/_index.md is still in sync

Thanks to the reviewers in advance.
Best Regards,
Kivinsae Fang

/lgtm
/approve


Map is empty for specific points

If I select some points wich one of them is POINT (766456911 13.5639657) empty map is displayed

Looks like a duplicate of #11021 :)


stats: Tag extraction doesn't work if resource name contains dots

Title: Tag extraction doesn't work if resource name contains dots
Description:
I've tested this for CLUSTER_NAME, but I assume it holds for others with similar regexes like listener, vhost, etc.
If you have a cluster name like foo.bar (this is every single cluster for us, since they are namespaced), then you'll have stats like cluster.foo.bar.upstream_cx_active. The tag extraction regex, however will match only foo and then use bar.upstream_cx_active for the stat name.
A reasonable workaround is to not use dots. However, if we take this approach, we should warn about this in the docs.
A fix will be a bit trickier. How do we possibly differentiate the resource name from the metric name when they both contain dots?
This issue might be considered an extension of #2687 and solving that in a general way would likely solve this. But it's a bit distinct in that there isn't a collision per se (in that two different stats don't resolve to the same thing), but it does have to do with tag extraction. Further, it isn't going to be solved with testing like in some of the other issues since it is dependent on the configuration. Either way, let me know if you think this is a dupe and I can move it to a comment on that issue.
Repro steps:
This repros on master with a trivial unit test change:
diff --git a/test/common/stats/tag_extractor_test.cc b/test/common/stats/tag_extractor_test.cc
index ac1c89c..7ca5dda 100644
--- a/test/common/stats/tag_extractor_test.cc
+++ b/test/common/stats/tag_extractor_test.cc
@@ -138,9 +138,9 @@ TEST(TagExtractorTest, DefaultTagExtractors) {
   // Cluster name
   Tag cluster_tag;
   cluster_tag.name_ = tag_names.CLUSTER_NAME;
-  cluster_tag.value_ = "ratelimit";
+  cluster_tag.value_ = "foo.bar";

-  regex_tester.testRegex("cluster.ratelimit.upstream_rq_timeout", "cluster.upstream_rq_timeout",
+  regex_tester.testRegex("cluster.foo.bar.upstream_rq_timeout", "cluster.upstream_rq_timeout",
                          {cluster_tag});

   // Listener SSL
@@ -163,7 +163,7 @@ TEST(TagExtractorTest, DefaultTagExtractors) {
   cipher_suite.name_ = tag_names.SSL_CIPHER_SUITE;
   cipher_suite.value_ = "ECDHE-RSA-AES128-GCM-SHA256";

-  regex_tester.testRegex("cluster.ratelimit.ssl.ciphers.ECDHE-RSA-AES128-GCM-SHA256",
+  regex_tester.testRegex("cluster.foo.bar.ssl.ciphers.ECDHE-RSA-AES128-GCM-SHA256",
                          "cluster.ssl.ciphers", {cluster_tag, cipher_suite});

   // ipv6 non-loopback (for alphabetical chars)

@ambuc could you investigate, since you're now de facto owner of this subsystem? :)

I think the right answer here is that cluster names can't contain dots. Should we add a check at runtime to reject configs which violate this (and update the docs) ? Like you said, this can't be detected with tests, since it has to fail on a specific "invalid" config.

Honestly, this seems like getting to the point where using a flat-name for the stat, computed from a format string, and then trying to unflatten it later, is breaking down. Should we consider storing/referencing stats by the non-flat name? They would be stored more like prometheus metrics then, where there's a basename and some attached labels.
I want to be able to store an fqdn (or similar) in the name of the cluster. Yes, I could encode the '.' in some way and decode it later, but it just feels like this is getting harder than it should be.

+1 to what @ggreenway said.. We use dots extensively in cluster names in Istio.. Heck, we also use '|'s as well. A non-flat name would work much better. Or how about using / as a separator between the components of a stat? WE could have a configurable option in bootstrap/CLI that allows the user to specify the stat separator.

It sounds to me like everyone is saying we want the stat key to be a proto :)

This is affecting gRPC stats in my usage too.
I have gRPC services defined named with the project name, service name, and service version, e.g.:
syntax = "proto3";

package myapp.widgets.v0;

service Widgets {
  ...
}

In DataDog I am seeing stats like envoy.cluster.grpc.widgets.v0.Widgets.failure with a tag {envoy.grpc_bridge_service:myapp}
It seems like I will be able to extract the remaining service name by defining a custom TagSpecifier regex.