HtmlWeb Load() method returns empty HtmlDocument when using cache

Hi,
I'm using HtmlAgilityPack 1.5.0-beta6 in .NETCoreAPP 1.1 console application.
Following code outputs empty string. Once I set UsingCache = false it will output page content, as expected.
var web = new HtmlAgilityPack.HtmlWeb()
{
	CachePath = @"C:\Users\Anton\Documents\Cache",
	UsingCache = true
};

var url = "https://github.com/zzzprojects/html-agility-pack";
var doc = web.Load(url);
Console.WriteLine(doc.DocumentNode.InnerText);

Hello @AntonKedrov ,
Thank you for reporting. We will investigate this issue over the weekend and provide you more information very soon.
Best Regards,
Jonathan

Unfortunately, it still doesn't work. web.Load(url) still returns uninitialized document.
BTW, I think there is a mistake in the unit test. I think there should be Assert.AreEqual instead of Console.WriteLine. :)
[Test]
public void TestLoadWithCache()
{
	...

	var docLoad = new HtmlAgilityPack.HtmlWeb().Load(url);
	Console.WriteLine(docLoad.DocumentNode.OuterHtml, docCache.DocumentNode.OuterHtml);
}

Best regards,
Anton

... omg!
We will look at it again during the weekend.
Best Regards,
Jonathan

Hello @AntonKedrov ,
We fixed the test. However, the code is still working fine for us with the v1.5.0-beta8.
Could you verify on your side to make sure this is not an issue related to an assembly caching and still using the old version? By example, by making sure to clean the solution first.
Best Regards,
Jonathan

Hello @AntonKedrov ,
We still have no received any update from your part concerning this issue.
Best Regards,
Jonathan

Hi,
I'm really sorry for delayed response.

To ensure that I have latest package I have removed C:\Users\Anton\.nuget\packages folder
Removed my cache folder C:\Users\Anton\Documents\Cache
Then created a new .NET Core Console App project (.NET Framework 4.7)
Installed HtmlAgilityPack 1.5.0-beta92
Copy-pasted code from the first post

In result C:\Users\Anton\Documents\Cache\github.com\zzzprojects directory created with two files: html-agility-pack and html-agility-pack.h.xml. But HtmlWeb.Load(url) still return empty document. (i.e. doc.DocumentNode.FirstChild is null, doc.DocumentNode.InnerText is empty string, etc.)
And once I set HtmlWeb.UsingCache to false, it works as expected.
I'm using following workaround. I'm checking if page is cached (using HtmlWeb.GetCachePath(uri) to get the path and then File.Exists(cachedPagePath)) and if it is cached, then I set HtmlWeb.CacheOnly to true before calling HtmlWeb.Load(url). In this case cached document will be loaded correctly.

Hello @AntonKedrov ,
Unfortunately, we are not able to reproduce it.
This feature works very well for my developer and me.
We are currently not sure how to reproduce it since we don't have the error whatever we try ;(
I believe the only solution currently is on your side downloading the source and trying to figure out what is not working.
Best Regards,
Jonathan

Sure, I'll try to help.
I cloned the repo and it seems that problem is in NETSTANDARD version of Get(Uri uri, string method, string path, HtmlDocument doc, IWebProxy proxy, ICredentials creds) method. Note that it is present in two projects: HtmlAgilityPack and HtmlAgilityPack.Shared.
Non-NETSTANDARD code looks like this:
if (UsingCache)
{
    // NOTE: LastModified does not contain milliseconds, so we remove them to the file
    SaveStream(s, cachePath, RemoveMilliseconds(resp.LastModified), _streamBufferSize);

    // save headers
    SaveCacheHeaders(req.RequestUri, resp);

    if (path != null)
    {
        // copy and touch the file
        IOLibrary.CopyAlways(cachePath, path);
        File.SetLastWriteTime(path, File.GetLastWriteTime(cachePath));
    }

    if (_usingCacheAndLoad)
    {
        doc.Load(cachePath);
    }
}

And NETSTANDARD code is like this:
if (UsingCache)
{
	// NOTE: LastModified does not contain milliseconds, so we remove them to the file
	SaveStream(s, cachePath, RemoveMilliseconds(response.Content.Headers.LastModified), _streamBufferSize);

	// save headers
	SaveCacheHeaders(request.RequestUri, response);

	if (path != null)
	{
	    // copy and touch the file
	    IOLibrary.CopyAlways(cachePath, path);
	    File.SetLastWriteTime(path, File.GetLastWriteTime(cachePath));
	}
}

I think that doc.Load(cachePath) code is missing here.
if (_usingCacheAndLoad)
{
    doc.Load(cachePath);
}

Best regards,
Anton

Thank you,
We will fix it today and check at the same time why our .NET Core project didn't raised this error.
Best Regards,
Jonathan

Hello @AntonKedrov ,
We have fixed our .NET Core project and we can now successfully reproduce this issue.
The V1.5.2-beta2 has been released.
It should now correctly work with your scenario.
Best Regards,
Jonathan

I can confirm that it works now. Thank you! :)
Best regards,
Anton

Closing Comment: Fix confirmed


Installation difficulties post retrospective. A request to update the tutorials documentation for Ubuntu.

Mission Statement
Before anything else, I've found a fix to my nokogiri installation problems. I just wanted to submit this issue to update the documentation to help others who might run into this specific installation problem.
I'm running a fresh install of Ubuntu 18.04 and was in the process of setting up my dev enviroment. When I tried to gem install nokogiri it failed, even after following everything in the installation tutorials docs. The solution that ended up working for me was having install libxml2 and libxslt by hand. In your docs, it says that nokogiri includes its own version of these, which didn't seem to be the case for me as I had to run
sudo apt-get install libxml2-dev
and
sudo apt-get install libxslt-dev
in order to get nokogiri to install on my computer.
The documentation doesn't really mention anything along these lines, so I was wondering if the documentation could be updated to include something along these lines?
Thank you!
Have you read and followed the installation tutorial at http://www.nokogiri.org/tutorials/installing_nokogiri.html?
Yes. I followed all the extra suggestions at the end and nokogiri still failed to install.  I'm requesting an update of the installation documentation to include this possible fix for my problem.
What is the complete output of gem install?
As this is a post issue retrospective, I am unable to post the contents of the gem install during the failure. It had something to do with being unable to create the Makefile and providing a huge list of configuration options.
Please make sure to escape the file contents with triple-backticks.
What are the complete contents of the mkmf.log file generated during the failed installation?
"gcc -o conftest -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0/x86_64-linux -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0/ruby/
  ...backward -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0 -I.    -O3 -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -
  ...Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-
  ...noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-
  ...packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-
  ...attribute=format -Wsuggest-attribute=noreturn -Wunused-variable  -fPIC  conftest.c  -L. -L/usr/share/rvm/rubies/ruby-2.7.0/lib -Wl,-rpath,/usr/
  ...share/rvm/rubies/ruby-2.7.0/lib -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic     -Wl,-rpath,'/../lib' -Wl,-rpath,'/../lib' -
  ...lruby  -lm   -lc "
  checked program was:
  /* begin */
  1: #include "ruby.h"
  2:
  3: int main(int argc, char **argv)
  4: {
  5:   return !!argv[argc];
  6: }
  /* end */

  "gcc -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0/x86_64-linux -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0/ruby/backward -I/usr/
  ...share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0 -I.    -O3 -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-function-
  ...declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-
  ...function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -
  ...Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-
  ...attribute=noreturn -Wunused-variable  -fPIC    -Werror -c conftest.c"
  checked program was:
  /* begin */
  1: #include "ruby.h"
  2:
  3: int main() {return 0;}
  /* end */

  "pkg-config --exists libxml-2.0"
  package configuration for libxml-2.0 is not found
  have_package: checking for libxml-2.0... -------------------- no

  --------------------

  "pkg-config --exists libxslt"
  package configuration for libxslt is not found
  have_package: checking for libxslt... -------------------- no

  --------------------

  "pkg-config --exists libexslt"
  package configuration for libexslt is not found
  have_package: checking for libexslt... -------------------- no

  --------------------

  "gcc -E -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0/x86_64-linux -I/usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0/ruby/backward -I/
  ...usr/share/rvm/rubies/ruby-2.7.0/include/ruby-2.7.0 -I.    -O3 -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-
  ...function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-
  ...cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-
  ...compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -
  ...Wsuggest-attribute=noreturn -Wunused-variable  -fPIC  -g -DXP_UNIX -O3 -Wall -Wcast-qual -Wwrite-strings -Wmissing-noreturn -Winline  conftest.
  ...c -o conftest.i"
  conftest.c:3:10: fatal error: libxml/xmlversion.h: No such file or directory
   #include <libxml/xmlversion.h>
            ^~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  checked program was:
  /* begin */
  1: #include "ruby.h"
  2:
  3: #include <libxml/xmlversion.h>
  /* end */

Please make sure to escape the file contents with triple-backticks.
What operating system are you using?
Ubuntu 18.04.4 LTS
Are you using Linux, MacOS, Windows, a BSD variant, or something else entirely?
Linux nyango 5.3.0-46-generic #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.4 LTS
Release:	18.04
Codename:	bionic

If Linux, please include the output of uname -a and the contents of /etc/lsb-release.

@DariaNico Sorry you had problems installing nokogiri.
The code path you're seeing in your mkmf.log file (using pkg-config) should only be invoked if you've set options asking Nokogiri to compile and link against system libraries. So I'm a little confused.
Can you post the output of gem env to see if, perhaps, you've got your gem environment set up to use system libraries?
If you're using bundler, can you post the output of bundler env for the same reason?

Installation docs have been rewritten since this post, and the OP hasn't responded. Closing.


millet: 0.10.0 -> 0.10.1

Diff: https://github.com/azdavis/millet/compare/v0.10.0...v0.10.1
Changelog: https://github.com/azdavis/millet/raw/v0.10.1/docs/CHANGELOG.md
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg eval


feat(aws): add MemoryDB service

Context
Add MemoryDB Service
Description
Added MemoryDB Service, Client, and Service Test
This can be used to build a full suite of checks for the MemoryDB service
Checklist

Are there new checks included in this PR? No

If so, do we need to update permissions for the provider? Please review this carefully. SecurityAudit contains memorydb:DescribeClusters

[X] Review if the code is being covered by tests.
[X] Review if code is being documented following this specification https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings
[X] Review if backport is needed.

License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hi @sansns, can you add a check to the service? If not, Prowler will not even execute this code. Thanks!

@MrCloudSec Added the first check for MemoryDB.


Modelica.Mechanics.MultiBody.Joints.Prismatic - s_offset - constant vs parameter

Is there any reason that s_offset is a constant rather than a parameter?  Thanks!

This lines
https://github.com/modelica/Modelica/blob/1de3f959ddaff0c106a9e0fcb56af61ff54b25ac/Modelica/Mechanics/MultiBody/Joints.mo#L22-L24

By the way the Evaluate Annotation is only specified for a component declared with the prefix parameter. See Spec. 18.3 Evaluate.

It looks like a bug introduced by c025e058b032a9542ceba9ca0335a6a59ed6da00. Note that s_offset is a proper parameter (without Evaluate annotation) in the other aggregate joints with prismatic degree of freedom.

No, this is not a bug, but a feature. The reason is that I wanted to get rid of offset, to make joints simpler. For backwards compatibility reasons, "offset" was not removed but kept. Via "constant" it was, however, removed from the parameter menu. Note, a constant "phi_offset" is also present in the Revolute joint (for the same reasons).

Still it would be wise to remove the Evaluate annotation for MSL 3.2.3.
Please file a new ticket with milestone MSL_next-MINOR-version to remove the superfluous s_offset constant then.

Another option is to let s_offset be a parameter with annotation (Dialog(enable=false)). This enables a user to still see there is such a parameter with some value which influences the model behavior.

@beutlich

... new ticket with milestone MSL_next-MINOR-version ...

Your rather mean MSL_next-MAYOR-version?

Your rather mean MSL_next-MAYOR-version?

Yes, edited MSL_next-MAJOR-version above. Thanks.


Add project dockerization

Add a Dockerfile with a multi-stage build (using rust:slim as builder and debian:buster-slim as the final image).

Sample/reference on how to create a Dockerfile https://github.com/LNP-BP/docker


Memory leaks in the use of SSR

Hi, we have a system in which we set up our websites using SSR, with a flow over 2000 rpm in production environment. Actually, we are facing memory leaks issues by rendering react component throught SSR. We are looking for alternatives for handling SSR efficiently by avoiding possible memory leaks generated by the react compoenents to render. We are evaluating the use of Next.js as framework for SSR. Have you face similar memory leaks issues before? Does Next.js have mechanism to avoid those memory leaks? Thank you so much for your support.

We haven't had reports of memory leaks so far, the main thing to keep in mind when doing React SSR is that you have to explicitly set NODE_ENV=production on bootup otherwise React's server renderer will boot up in development mode.
As far as prevention goes, there's nothing specific we do in this area.

Feel free to post any other questions on https://spectrum.chat/next-js as that's where the whole community can see your question.


Errors when running python3 demo.py

I have run python3 demo.py code
and I got this
/usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88   return f(*args, **kwds) Using TensorFlow backend. /usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88   return f(*args, **kwds) /usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88   return f(*args, **kwds) Downloading data from https://github.com/yu4u/age-gender-estimation/releases/download/v0.5/weights.18-4.06.hdf5 195813376/195810464 [==============================] - 56s 0us/step 2018-07-24 16:57:16.697715: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA VIDEOIO ERROR: V4L: can't open camera by index 0 Traceback (most recent call last):   File "demo.py", line 121, in <module>     main()   File "demo.py", line 82, in main     for img in yield_images():   File "demo.py", line 60, in yield_images     yield img   File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__     self.gen.throw(type, value, traceback)   File "demo.py", line 42, in video_capture     yield cap   File "demo.py", line 58, in yield_images     raise RuntimeError("Failed to capture image") RuntimeError: Failed to capture image
NOTE: I'm on Ubuntu 16.04 installed in VMware and I have a webcam in my laptop, I think I've installed all the dependencies correctly

Failed in capturing an image from the webcam.
https://github.com/yu4u/age-gender-estimation/blob/master/demo.py#L57
The webcam may not be recognized from the virtual machine.
Did you allow the virtual machine to used the webcam (as a USB device?) ?

I don’t think so, but my webcam is built-in. Do you know how can I check if the webcam is used as a USB device or how can I allow the VM to use it as USB device

As I could not reproduce your environment, I have no idea.
I found many related article by searching with "vmware built in webcam".

I have managed to make it work, but after the webcam started do I have to stop it manually ? I mean stop the program demo.py ?

Usually there is no need to start/stop a webcam after VM recognized a webcam but I'm not sure there is any VM-specific problem or not because I have never used a webcam from VMs.

I mean after running this python3 demo.py, the webcam started and it get my age and gender changing then what next ? should I close the window or wait ?

demo.py run in infinite while loop.
Pressing ESC key would finish the program.

for img in yield_images():
Instead of the above line in demo.py, you can read an image from file:
img = cv2.imread(image_path)
or you can loop:
from pathlib import Path

for image_path in Path(path_to_img_dir).glob("*.*"):
    img = cv2.imread(str(image_path))

So you mean I should commented out
for img in yield_images():
and put this code instead
img = cv2.imread(image_path)
image_path means the path of the image and its format like JPG, TIF, PNG etc. ?
and If I want to analyze my own dataset of 5000 images ? how can I do it and is it possible to do it without labeling them ?
I'm asking those questions coz I'm doing a research on this topic
Thnx

and If I want to analyze my own dataset of 5000 images ?

As described above, you can iterate over all files in path_to_img_dir directory by:
from pathlib import Path

for image_path in Path(path_to_img_dir).glob("*.*"):
    img = cv2.imread(str(image_path))

 File "demo.py", line 83     img = cv2.imread('/home/nasser/2.jpeg')	     ^ IndentationError: unexpected indent
I tried to analyze this picture with no avail, what do you think is wrong ?
I did like what you said check:
` #load model and weights
img_size = 64
model = WideResNet(img_size, depth=depth, k=k)()
model.load_weights(weight_file)
##for img in yield_images():
 img = cv2.imread('/home/nasser/2.jpeg')
    input_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    img_h, img_w, _ = np.shape(input_img)`

I tried to do 1 image but I get this error
'line 83 img = cv2.imread('/home/nasser/2.jpeg') ^ IndentationError: unexpected indent`
when I tried to iterate a directory of images:
 File "demo.py", line 84     for image_path in Path(“/home/nasser/google-images-download/Family/dataset_3000”).glob("*.*"):     ^ IndentationError: unexpected indent
check the screeenshot:

Both errors clearly indicate what's wrong; indent.

Thnx I managed to fix it,
but after like 300-400 pictures it stopped and gave me this error:
File "demo.py", line 124, in
main()
File "demo.py", line 86, in main
input_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(3.4.2) /io/opencv/modules/imgproc/src/color.hpp:253: error: (-215:Assertion failed) VScn::contains(scn) && VDcn::contains(dcn) && VDepth::contains(depth) in function 'CvtHelper'

Would you kindly solve by yourself problems not specific to this project.
In this case, the following modification may solve the problem...
img = cv2.imread(str(image_path), 1)

Hello @yu4u I have a question related to this project, when change add the code you mention above to analyze one picture, I recognize that ages range from 3-14 in pictures are not detected, for example if I put a baby face and his age range from 3-5 the demo.py can’t detect the face and didn’t give results, I tried many pictures and age ranges from 3-14.
Do you have any suggestion ?
Thnx a lot

It depends on the accuracy of face detector.
Please try dlib.cnn_face_detection_model_v1 detector instead of current dlib.get_frontal_face_detector().

I have change the detector you provided instead
# for face detection
detector = dlib.cnn_face_detection_model_v1
# load model and weights
img_size = 64
model = WideResNet(img_size, depth=depth, k=k)()
model.load_weights(weight_file)
then I get this error
Traceback (most recent call last):
File "demo.py", line 124, in <module>
main()
File "demo.py", line 90, in main
detected = detector(input_img, 1)
TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
1. dlib.cnn_face_detection_model_v1(filename: str)
Invoked with: array([[[180, 163, 137],
 [179, 162, 136],
[178, 161, 135],
...,
[129, 118, 100],
[129, 118, 100],
[129, 118, 100]],
[[179, 162, 136],
[179, 162, 136],
[179, 162, 136],
...,
[129, 118, 100],
[129, 118, 100],
[129, 118, 100]],
[[178, 161, 135],
[179, 162, 136],
[179, 162, 136],
...,
[129, 118, 100],
[129, 118, 100],
[129, 118, 100]],
...,
[[155, 166, 184],
[153, 164, 182],
[152, 163, 181],
...,
[119, 130, 148],
[120, 131, 149],
[121, 132, 150]],
[[155, 166, 184],
[152, 163, 181],
[152, 163, 181],
...,
[118, 129, 147],
[118, 129, 147],
[118, 129, 147]],
[[153, 164, 182],
[151, 162, 180],
[150, 161, 179],
...,
[119, 130, 148],
[117, 128, 146],
[117, 128, 146]]], dtype=uint8), 1>
I am sorry but I think this problem related to the project as well, is there something missing ?

It's simply the problem of how to use dlib CNN detector (dlib.cnn_face_detection_model_v1).
Please refer to dlib documents.

I have run python3 demo.py code
and I got this
File "demo.py", line 138, in
main()
File "demo.py", line 87, in main
file_hash=modhash, cache_dir=Path(file).resolve().parent)
File "/home/anmol/.local/lib/python3.5/site-packages/keras/utils/data_utils.py", line 177, in get_file
datadir_base = os.path.expanduser(cache_dir)
File "/usr/lib/python3.5/posixpath.py", line 228, in expanduser
if not path.startswith(tilde):
AttributeError: 'PosixPath' object has no attribute 'startswith'

Yes, It worked...thanks


Syntax Error with templates inside script tags.

Hi! First of all thank you for this project!
Prettier formatting for jinja itself seems neat, but now I'm able to use prettier plugins such as the tailwind plugin which is what does make a huge difference for me!
When working my project I've got a syntax error with code like this:
{% block scripts %}
<script>
	var STATE = {{ state | tojson }};
</script>
{% endblock %}

[error] templates/setup.html: SyntaxError: Unexpected token (2:15)
[error]   1 |
[error] > 2 |   var STATE = {{ state | tojson }};
[error]     |                ^
[error]   3 |

This does not happen when I'm running prettier without the jinja plugin. I'm unsure how is the plugin treating script blocks, but the issue seems to be specific to them, I'm also unable to ignore this code with prettier-ignore in both html and js.

This plugin relies on errors by "embedded" languages to bubble up in contrast to the default html plugin which ignores these errors.
The easiest way to ignore these kinds of errors is by adding a range ignore around the script block:
<!-- prettier-ignore-start -->
  {% block scripts %}
    <script>
      var STATE = {{ state | tojson }};
    </script>
  {% endblock %}
<!-- prettier-ignore-end -->


how can i try an example using python locally

when i try to run .py stored in cgi-bin/cgi_exe.py, the result (generated in cgi-bin/images/out) is only filled with purple. What is the problem?
I am currently running it on Window system.
I download the unet_128_standard & unet_512_standard and saved in cgi-bin/models.
and the input .png is stored in cgi-bin/images/line and cgi-bin/images/ref (with little color on it).
The input .png is taken from the website of PaintsChainer.
Thank you.

hi friend, how to download, the web seems not work?, i cant download from it?

Hi, @marco54654! Can you please share the downloaded pretrained models (unet_128_standard & unet_512_standard)? I cannot download them since the provided link in the repo does not function properly :(


Issue219

Angular deprecate the use of .success and .error methods and i have changed to the .then. I only changed this at the ng-token-auth.js file without creating a min version. Also, i dont test the code, if anyone can do it will help me a lot!

Unfortunately, the JS code is just auto-generated, so this really needs to be sourced from the CoffeeScript file. There are notes about the environment setup + unit testing steps on the README.

this issue should be closed

looks like we're still using the old syntax. still needs the fix.


instance method do not work on functional component... i.e Can't use vue-i18n inside

What problem does this feature solve?
currently I can't use vue-i18n in functional components...

What does the proposed API look like?
there's no API for that, it just have to work out-of-the-box

Use parent.$t

@darkylmnx
Sorry vue-i18n poor documentation. 🙇
I will add about it later.

@kazupon no prob, thanks

@yyx990803 I'm not sure if I should open another issue for this so I'm asking here :
Since functional components have no instance, is it possibile to use local components or we should absolutely defined components globally with Vue.component - this isn't said in the docs either.


Module resolution

Following the tweet by Marvin - and the subsequent article we should see how we can take that feedback into account and see how we can improve the performances around module resolution

stacktrace / throwIfNoEntry in fs.stat
cache when accessing file
file extension

FYI: It looks like only fs.statSync and fs.lstatSync support the throwIfNoEntry option. Can't find such a thing for the async variants.

It's also mentioned that fs.existsSync can be used, although it now still does an try-catch because we have a funny behavior that is pending to be deprecated: https://github.com/nodejs/node/blob/762a3a8ad925d56a12b43e0e7f7c811e93097784/lib/fs.js#L272-L277 technically I think we can still avoid that even without breaking it if we return false instead of just throwing in the validation helpers.

I'll take a look at it in a couple of days

In the meeting we mentioned it might be possible to implement the sync stat functions with fast calls. The stats are assembled from a typed array, which can be updated just fine in fast calls without triggering allocations.

I attempted something similar here https://github.com/nodejs/node/pull/46345
I think this could be made slightly more faster using internalBinding('stat') but havent found any consensus yet on wether such a API is useful

On hold, until off-thread loader pull request is merged.

stacktrace / throwIfNoEntry in fs.stat

FWIW I've seen the particular "fs.stat being slow due to captureLargerStackTrace" issue before. For the particular case I was looking at, it was actually caused by the deoptimizer (which is needed for the stack unwinding) forcing some garbage collection step to complete (v8 bug).
I'm not sure to what extent that was the cause of the issues that @marvinhagemeister saw, but at least in some cases it should be better soon.

@kvakil I saw it in the context of jest and eslint. Common configuration of these two projects involve adding their own compilation layer which means they do module resolution themselves to support tsconfig.json paths and similar things. It's these custom resolutions that are usually node tuned for perf. While looking at that I noticed that it's way better in node as is, but still with room for improvement. The scenarios I looked at loaded a very high number of files, mostly coming from a plethora of npm dependencies.


32-bit support, WASM bindings, and remove

:tada: This PR is included in version 0.22.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[hpx::execution] Added forwarding_scheduler_query

Signed-off-by: Shreyas Atre<EMAIL_ADDRESS>Proposed Changes

[X] forwarding_scheduler_query
[x] get_forward_progress_guarantee
[x] this_thread::execute_may_block_caller

Checklist

[X] Unit test - forwarding_scheduler_query
[x] Unit test - get_forward_progress_guarantee
[x] Unit test - execute_may_block_caller

Please ignore the clang 14 builder problems, those are unrelated

Performance test reportHPX PerformanceComparisonBENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTORFor Each(=)=+InfoPropertyBeforeAfterHPX Commita68196882e3e093ba44367f6675f538d613c9aa3eeb48ba1eb23a03879a82b48d1fd767aa8e1857cHPX Datetime2022-02-09T15:45:06+00:002022-05-01T13:42:33+00:00Hostnamenid00729nid01157EnvfileClusternamedaintdaintCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0Datetime2022-02-09T17:03:21.440240+01:002022-05-01T15:54:52.264264+02:00ComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch+InfoPropertyBeforeAfterHPX Commit96a2e4bd1c76207dea8d5302784962c927882406eeb48ba1eb23a03879a82b48d1fd767aa8e1857cHPX Datetime2021-11-11T08:14:57+00:002022-05-01T13:42:33+00:00Hostnamenid00006nid01157EnvfileClusternamedaintdaintCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0Datetime2021-11-11T09:28:13.071121+01:002022-05-01T15:55:08.919098+02:00ComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add(=)-(=)Stream Benchmark - Scale(=)(=)(=)Stream Benchmark - Triad(=)(=)(=)Stream Benchmark - Copy(=)(=)(=)InfoPropertyBeforeAfterHPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01aeeb48ba1eb23a03879a82b48d1fd767aa8e1857cHPX Datetime2021-11-10T19:14:21+00:002022-05-01T13:42:33+00:00Hostnamenid00120nid01157EnvfileClusternamedaintdaintCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0Datetime2021-11-10T20:28:18.266961+01:002022-05-01T15:55:24.137602+02:00Explanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

@SAtacker could you please fix the issues reported by inspect:
/libs/core/execution/include/hpx/execution/queries/get_scheduler.hpp: Line length limit: 64
/libs/core/execution_base/include/hpx/execution_base/this_thread.hpp: Line length limit: 75, 84, 90
?

@SAtacker could you please fix the issues reported by inspect:
/libs/core/execution/include/hpx/execution/queries/get_scheduler.hpp: Line length limit: 64 /libs/core/execution_base/include/hpx/execution_base/this_thread.hpp: Line length limit: 75, 84, 90
?

Thanks. This should be done in clang-format right? I have it as a commit hook as well as I did it manually but it doesn't change.

Thanks. This should be done in clang-format right? I have it as a commit hook as well as I did it manually but it doesn't change.

clang-format doesn't touch comments (see: https://github.com/STEllAR-GROUP/hpx/blob/master/.clang-format#L107), you will have to handle this manually.

@msimberg all of the CSCS builders seem to fail. Has something changed in the setup?

@msimberg all of the CSCS builders seem to fail. Has something changed in the setup?

Nothing that I'm aware of. The logs seem to say that the jobs timed out, which smells a bit like a transient slurm problem. Other jobs that were launched around the same time (and later) seem to have been started and run just fine.

retest cscs

The failure that prevented bors from merging is a known one. Merging manually...

The failure that prevented bors from merging is a known one. Merging manually...

Thanks for reviewing.


New version: HurdleDMR v1.1.2

Registering package: HurdleDMR
Repository: https://github.com/AsafManela/HurdleDMR.jl
Created by: @AsafManela
Version: v1.1.2
Commit: be39648ca613ee3304dacd9782d304aef301b57a
Reviewed by: @AsafManela
Reference: https://github.com/AsafManela/HurdleDMR.jl/commit/be39648ca613ee3304dacd9782d304aef301b57a#commitcomment-36148668

This is a patch release for an older version that does not rely on the newer StatsModels v0.6
[noblock]

Yea its a bug.


Introduce FetchDetails for Fetch Instance/Binding APIs

Introduce FetchDetails for Fetch Instance/Binding APIs
This pull request introduces FetchDetails for the OSB Fetch Service Instance & Fetch Service Binding requests.
As specified in the spec:

Fetch Service Instance: https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#parameters-3
Fetch Service Binding: https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#parameters-7

Both handlers should be able to accept service_id and plan_id parameters. Currently the brokerapi library doesn't expose that to library clients, which in some situations is vital information (even if OSB spec doesn't mandate these properties).
With this PR the intention is to allow library clients to have access to these request details.
Most notably the GetInstance handler now takes an additional argument FetchDetails like so:
//   GET /v2/service_instances/{instance_id}
GetInstance(ctx context.Context, instanceID string, details FetchDetails) (GetInstanceDetailsSpec, error)

Similarly with the GetBinding handler:
//   GET /v2/service_instances/{instance_id}/service_bindings/{binding_id}
GetBinding(ctx context.Context, instanceID, bindingID string, details FetchDetails) (GetBindingSpec, error)

Tests for the new handler arguments are included as well.

What's the status of this? Would be great to have this merged in :+1:

What's the status of this? Would be great to have this merged in :+1:

Thanks for the contribution @NickyMateev!
I have created #147 to do some tidying up and will close this PR in favour of that one.


[ENH] Extra telemetry/data feeds available

Proposed new feature or change:
While developing MultiViewer for F1 I've discovered a few endpoints that are currently not used, and wanted to share in case you want to implement them in your library.

DriverScore the driver scores for G-forces, throttle, steering and braking (more about it here: https://www.formula1.com/en/latest/features/2015/4/race-performance-ratings-explained---aggression.html)
LapSeries (maybe not as useful as this is already data that is available elsewhere)
PitLaneTimeCollection Not super precise, but single-decimal precision pit stop durations

... and more
You may also be able to use https://livetiming.formula1.com/static/2022/Index.json instead of relying on "guessing" the path etc., and there's also /Index.json endpoints for all seasons (/static/Index.json) and for each session /static/:season/:meeting/:session/Index.json.
Hope this helps!

Full list of data feeds
{
  "Feeds": {
    "SessionInfo": {
      "KeyFramePath": "SessionInfo.json",
      "StreamPath": "SessionInfo.jsonStream"
    },
    "ArchiveStatus": {
      "KeyFramePath": "ArchiveStatus.json",
      "StreamPath": "ArchiveStatus.jsonStream"
    },
    "TrackStatus": {
      "KeyFramePath": "TrackStatus.json",
      "StreamPath": "TrackStatus.jsonStream"
    },
    "SessionData": {
      "KeyFramePath": "SessionData.json",
      "StreamPath": "SessionData.jsonStream"
    },
    "ContentStreams": {
      "KeyFramePath": "ContentStreams.json",
      "StreamPath": "ContentStreams.jsonStream"
    },
    "ChampionshipPrediction": {
      "KeyFramePath": "ChampionshipPrediction.json",
      "StreamPath": "ChampionshipPrediction.jsonStream"
    },
    "AudioStreams": {
      "KeyFramePath": "AudioStreams.json",
      "StreamPath": "AudioStreams.jsonStream"
    },
    "DriverList": {
      "KeyFramePath": "DriverList.json",
      "StreamPath": "DriverList.jsonStream"
    },
    "TimingDataF1": {
      "KeyFramePath": "TimingDataF1.json",
      "StreamPath": "TimingDataF1.jsonStream"
    },
    "SPFeed": {
      "KeyFramePath": "SPFeed.json",
      "StreamPath": "SPFeed.jsonStream"
    },
    "LapSeries": {
      "KeyFramePath": "LapSeries.json",
      "StreamPath": "LapSeries.jsonStream"
    },
    "TimingAppData": {
      "KeyFramePath": "TimingAppData.json",
      "StreamPath": "TimingAppData.jsonStream"
    },
    "TimingData": {
      "KeyFramePath": "TimingData.json",
      "StreamPath": "TimingData.jsonStream"
    },
    "TopThree": {
      "KeyFramePath": "TopThree.json",
      "StreamPath": "TopThree.jsonStream"
    },
    "TimingStats": {
      "KeyFramePath": "TimingStats.json",
      "StreamPath": "TimingStats.jsonStream"
    },
    "ExtrapolatedClock": {
      "KeyFramePath": "ExtrapolatedClock.json",
      "StreamPath": "ExtrapolatedClock.jsonStream"
    },
    "CarData.z": {
      "KeyFramePath": "CarData.z.json",
      "StreamPath": "CarData.z.jsonStream"
    },
    "Position.z": {
      "KeyFramePath": "Position.z.json",
      "StreamPath": "Position.z.jsonStream"
    },
    "SessionStatus": {
      "KeyFramePath": "SessionStatus.json",
      "StreamPath": "SessionStatus.jsonStream"
    },
    "TyreStintSeries": {
      "KeyFramePath": "TyreStintSeries.json",
      "StreamPath": "TyreStintSeries.jsonStream"
    },
    "DriverRaceInfo": {
      "KeyFramePath": "DriverRaceInfo.json",
      "StreamPath": "DriverRaceInfo.jsonStream"
    },
    "LapCount": {
      "KeyFramePath": "LapCount.json",
      "StreamPath": "LapCount.jsonStream"
    },
    "Heartbeat": {
      "KeyFramePath": "Heartbeat.json",
      "StreamPath": "Heartbeat.jsonStream"
    },
    "WeatherData": {
      "KeyFramePath": "WeatherData.json",
      "StreamPath": "WeatherData.jsonStream"
    },
    "WeatherDataSeries": {
      "KeyFramePath": "WeatherDataSeries.json",
      "StreamPath": "WeatherDataSeries.jsonStream"
    },
    "TeamRadio": {
      "KeyFramePath": "TeamRadio.json",
      "StreamPath": "TeamRadio.jsonStream"
    },
    "TlaRcm": {
      "KeyFramePath": "TlaRcm.json",
      "StreamPath": "TlaRcm.jsonStream"
    },
    "RaceControlMessages": {
      "KeyFramePath": "RaceControlMessages.json",
      "StreamPath": "RaceControlMessages.jsonStream"
    },
    "CurrentTyres": {
      "KeyFramePath": "CurrentTyres.json",
      "StreamPath": "CurrentTyres.jsonStream"
    },
    "DriverScore": {
      "KeyFramePath": "DriverScore.json",
      "StreamPath": "DriverScore.jsonStream"
    },
    "PitLaneTimeCollection": {
      "KeyFramePath": "PitLaneTimeCollection.json",
      "StreamPath": "PitLaneTimeCollection.jsonStream"
    }
  }
}

It's awesome to know these many more. Will help in my future applications.

Nice! I would say that the PitLaneTimeCollection is not 100% new since you can gather that data by substracting the PitOutTime and PitInTime, from Laps.
However, what I found really interesting is the GForces (Lateral and Longitudinal) that provide SPfeed and DriverScore endpoints. The bad thing is that it gets updated just every 15 sec...

oh yes you are right. So, there is no raw data of the actual G-forces?

oh yes you are right. So, there is no raw data of the actual G-forces?

unfortunately, not that I've come across, but if I do, I'll share. I wish they did, because it shows up on the broadcast sometimes, and even has a directional element, too. It would be quite cool to show in MultiViewer (but also for y'all to analyse)

@f1multiviewer thank you very much for this information! This is awesome! I didn't know about these Index.json overview endpoints before. I wasn't quite "guessing" api paths, but somehow I missed these so far. This would have made some things so much easier. I need to take some time and look at what information there is and how I can best add this to FastF1.

@f1multiviewer Could you please give an example of how to access, for instance, the Heartbeet stream?

You can use the index files to find the season/session you need, then build the URL like so:
https://livetiming.formula1.com/static/:season/:meeting_handle/:session_handle/:topic.(json|jsonStream)
Example:
https://livetiming.formula1.com/static/2022/2022-07-31_Hungarian_Grand_Prix/2022-07-30_Qualifying/Heartbeat.jsonStream
Though, this is not new, that's already something Fast-F1 does.

Yeah, I had searched in the code for where the requests are sent to but :meeting in the URL template of this issue confused me at the start, thanks anyways!

You may also be able to use https://livetiming.formula1.com/static/2022/Index.json instead of relying on "guessing" the path etc., and there's also /Index.json endpoints for all seasons (/static/Index.json) and for each session /static/:season/:meeting/:session/Index.json.

I want to correct a slight bit of error here. It is not exactly true about each session having the Index.json. Index.json has been provided from 2018 US Grand Prix FP1 onwards. Any session before that (Start of the season to Japan GP Race) didn't have an Index.json with their respective paths.

Link to Index.json for 2018 US GP FP1: https://livetiming.formula1.com/static/2018/2018-10-21_United_States_Grand_Prix/2018-10-19_Practice_1/Index.json
Link to Index.json for 2018 Japan GP Race (404 Not Found): https://livetiming.formula1.com/static/2018/2018-10-07_Japanese_Grand_Prix/2018-10-07_Race/Index.json

Also, note that pre-season testing session information is not present Index.json, path for those still need to be "guessed".

@eXhumer thank you for that information. I guess some kind of combination of the index endpoints and the current schedule backend will be necessary to support all sessions.

@theOehrly I also have corner position/length (although I'm not sure how precise length is) available if this is useful. I have an API that I keep up to date, it currently has (or should have, if one is missing I'd love to hear) corner annotations for all circuits. I also have annotations for FIA marshal lights (and marshalling sectors are being added as we speak, some already have them). I've seen a previous issue https://github.com/theOehrly/Fast-F1/issues/19 but I'm not sure what the status is.
If this would be useful, lmk, I'll write some documentation (I can either send you the current data, or you can implement the API and get up-to-date information) on how to retrieve/interpret it.

@f1multiviewer That sounds very interesting and is something I'd like to add over the winter. I'd prefer using your API to get up-to-date information instead of working with an image of the data, if that's cool with you.
#19 hasn't made any further progress, altough @ukaoskid has added some more tracks to his data set. @f1multiviewer where do you get your data from? Do you create it all manually? If that's the case, maybe you can work together with @ukaoskid, as you say you're not sure how precise length is in your data. Maybe it is possible to create one fairly accurate database for this information?

@f1multiviewer That sounds very interesting and is something I'd like to add over the winter. I'd prefer using your API to get up-to-date information instead of working with an image of the data, if that's cool with you. #19 hasn't made any further progress, altough @ukaoskid has added some more tracks to his data set. @f1multiviewer where do you get your data from? Do you create it all manually? If that's the case, maybe you can work together with @ukaoskid, as you say you're not sure how precise length is in your data. Maybe it is possible to create one fairly accurate database for this information?

Data is manually added by overlapping the official FIA maps with a custom map editor and matching the rotation & size. I'm sure it's correct enough for "marking" it. The length is calculated by measuring the SVG path's length, which seems to match well with distance travelled. I'd say for corner numbers, this is fine, for marshalling sectors, I'd add a big fat asterisks.

@f1multiviewer that sounds like quite a bit of manual work. It'd sure be nice to be able to use that data in FastF1 as well.

@f1multiviewer that sounds like quite a bit of manual work. It'd sure be nice to be able to use that data in FastF1 as well.

Not nearly as much as some of the other stuff I've had to do to get MultiViewer where it is now haha, but I'll send over documentation later 💚

@f1multiviewer that sounds like quite a bit of manual work. It'd sure be nice to be able to use that data in FastF1 as well.

Not nearly as much as some of the other stuff I've had to do to get MultiViewer where it is now haha, but I'll send over documentation later 💚

Thank you. No need to hurry on your side. I will not get around to implementing that immediately. But I'd love to take a look at it, for sure.
(And I have to say, MultiViewer looks amazing! Now, if I could only get F1TV in Germany then I could even use it...)

(And I have to say, MultiViewer looks amazing! Now, if I could only get F1TV in Germany then I could even use it...)

Check r/F1TV, I can't really help with it, but there's plenty of people there who have access from regions where it's unavailable.

Hey everyone,
I did not abandon the project, it’s taking time to make it as much exact as
possible, and also I am slightly changing the dataset to allow multiple
versions of the same track (like layout changes over the time or different
versions of the same track, see Bahrein dual layout).
Any help would be appreciated. I will add a contribution file as soon as
today for better PRs
El El dom, 9 oct 2022 a las 13:13, Philipp Schaefer <
@.***> escribió:

@f1multiviewer https://github.com/f1multiviewer That sounds very
interesting and is something I'd like to add over the winter. I'd prefer
using your API to get up-to-date information instead of working with an
image of the data, if that's cool with you.
#19 https://github.com/theOehrly/Fast-F1/issues/19 hasn't made any
further progress, altough @ukaoskid https://github.com/ukaoskid has
added some more tracks to his data set. @f1multiviewer
https://github.com/f1multiviewer where do you get your data from? Do
you create it all manually? If that's the case, maybe you can work together
with @ukaoskid https://github.com/ukaoskid, as you say you're not sure
how precise length is in your data. Maybe it is possible to create one
fairly accurate database for this information?
—
Reply to this email directly, view it on GitHub
https://github.com/theOehrly/Fast-F1/issues/248#issuecomment-1272518616,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACVMZSG3WE4ITOTXIHI345TWCKSEVANCNFSM6AAAAAAQAO3E3Y
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hey all,
@theOehrly @f1multiviewer I have added a CONTRIBUTE.md file if you guys want to give your contribution.
Thanks!

Sorry for the delay, here's a little explanation about the API, how to use it, the data and limitations:
Basics
So to call the API for a circuit, you need the CircuitKey, you can find this for each session in SessionInfo.Meeting.Circuit.Key. There's already one exception here, where one session at Mugello is incorrectly miscategorised as 149, but that's actually the circuit key for Jeddah. I've handled this in MultiViewer like so:
// NOTE: Mugello is miscategorised as 149 (which is Jeddah) in the API
const circuitKey =
  sessionInfo?.Meeting?.Circuit?.ShortName === "Mugello"
    ? "146"
    : sessionInfo?.Meeting?.Circuit?.Key?.toString(10);

Once you have the correct CircuitKey, you can call the API to get the circuit metadata like so:
GET /api/v1/circuits/2/2022 HTTP/1.1
Host: api.f1mv.com

{
  "corners": [
    {
      "angle": 98.4899241531774,
      "length": 4523.666100769043,
      "number": 1,
      "trackPosition": {
        "x": 1192.50830078125,
        "y": 4503.826171875
      }
    },
    // ...
  ],
  "marshalLights": [
    {
      "angle":<PHONE_NUMBER>1329444,
      "length": 4.564748840332031,
      "number": 1,
      "trackPosition": {
        "x": -1754.1448974609375,
        "y": 1212.111572265625
      }
    },
    // ...
  ],
  "marshalSectors": [
    {
      "angle":<PHONE_NUMBER>729058,
      "length": 0,
      "number": 1,
      "trackPosition": {
        "x": -1756,
        "y": 1208
      }
    },
    // ...
  ],
  "circuitKey": 2,
  "circuitName": "Silverstone",
  "countryIocCode": "GBR",
  "countryKey": 2,
  "meetingKey": "1114",
  "rotation": 92,
  "year": 2022,
  // ... (omitted fields are not guaranteed to be available or may be removed in the future)
}

The corners array contains the corner data, which is the most important part of the API. The marshalLights and marshalSectors arrays contain the marshal data, which is used to display the marshal lights and marshal sectors on the map. The rotation field is the rotation of the circuit in degrees, which is used to rotate the map to the correct orientation (matching what FOM uses on the broadcast).
Corners, marshal lights and marshal sectors are all "track markers", each track marker follows the following type specification:
export interface ITrackMarker {
  /** The number of the marker */
  number: number;

  /**
   * If this marker number is referenced multiple times, the letter
   * is used to differentiate them, such as corner 2A in Hungary.
   */
  letter?: string;

  /**
   * The coordinate of the marker on the track map, resolution is
   * usually 10cm, but matches the resolution of the car position
   * from the live timing telemetry.
   */
  trackPosition: DOMPoint | { x: number; y: number };

  /**
   * This is the angle in degrees, used to offset the marker's
   * placement on a track map in a logical direction.
   */
  angle: number;

  /**
   * The approximate absolute length from the start of the track
   * to this marker, measured in SVG path length from all previous
   * track points.
   */
  length: number;
}

Limitations

Length may be incorrect: The length is calculated from the SVG path length, which is not always accurate. Sometimes, GPS position isn't accurate and will go forward and then backward before going forward again. The differences should be minimal, but your mileage may vary. To get the most precise length, I'd advise using the trackPosition value and calculating the length to the marker yourself. Given these are only meant to be used as visual markers for MultiViewer, this isn't a problem for me, but it may be for you.
Old track data: I'm still working on a script to gather track data for all circuits myself. I'm currently borrowing data from another source to get the x, y data returned by the API (and some of the other data as well, but not the corner/marshal data). I'm not sure how precise this data is, but it works well for visual purposes. In the future, this data (and the object keys) may change. I'll communicate any changes here and on the Discord server, and will try to make sure there are no backwards compatible/breaking changes, those should be published under /v2 if there's a need for this.
API comes without warranty: Basically the data is available free of charge, but without any warranties.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

User agent: MultiViewer identifies itself when sending requests to the API. I'd appreciate if FastF1 does the same. Format can be something like: fastf1/{version} platform/{platform}, if you can't or don't want to send the platform, that's fine as well, you can omit it. If you expect that this API will be called frequently by a single user of FastF1 during normal use, I'd appreciate if you can add some caching system.

Details

I've manually mapped (using the circuit data also returned in the API, and circuit/media maps provided by the FIA, and a custom map editor) all corners, marshal lights and marshalling sectors.
While I've paid great care to make sure the data is correct, there may be some errors. If you find any, please let me know and I'll fix them as soon as possible. I've also fixed some errors with respect to FIA maps, as for instance, in Singapore, the FIA map doesn't correctly show the marshal sectors (they are wrongly offset by 1). I'll continue to fix these inconsistencies as I find them.
There's a minor bug in the angles for track markers, where it may show a negative number instead of a positive number. This may be fixed in the future.
If you want, I can also provide you with the JSON files which contain the track markers. But you said before you prefer to implement the API, so that's fine with me as well.
If you want, you can credit MultiViewer for F1 as a data source, but it's not a requirement

Happy to answer any questions if anything is unclear! And if you think there's improvements I could make, let me know as well. I'm happy to help out where I can.
Finally @ukaoskid I'd love to contribute, but I'm busier than ever. I'd be happy to provide the data that I've gathered so far, but you'd have to check/convert it yourself. Let me know if you're interested :)

@f1multiviewer wow, that is a lot of information. Thank you very much. I'll see when I'll get around to trying out your API. In any case, this will only get implemented for the next season.
Regarding some of your notes about the interaction with the API:

Sending a user agent fastf1/{version} is no problem at all. This is already done with Ergast right now. I'll see about maybe adding a platform specifier.
FastF1 already uses caching for all API requests. Does your API server provide an HTTP cache-control header? If so, the specified max-age will be respected. That way, you would control how often the data is accessed by individual users. Else, I can manually specify how long the cache is valid.

I suppose more questions might come up when I work with the API.
And I would of course credit MultiViewer as the data source if I implement your API.

Hey @theOehrly! The API does expose cache-control headers, so it's good to know FastF1 respects them. I'm here in case you have any questions (or if you want, you can join the Discord server, link is on the website, if you prefer to chat)

I updated the domain name used for the API in the comment above, the old domain will continue working for the foreseeable future but best to use api.multiviewer.app

@f1multiviewer you made me aware of the season index API endpoints, so I assume you are likely making use of them?
When I use https://livetiming.formula1.com/static/2023/Index.json to get the schedule for the current season, all I get is information about events that have already taken place. That means, I cannot create a full event schedule for the current season from this endpoint. Am I missing something here, or does the API not provide information about future events?
Also, I'm really sorry that you took the time for creating such an extensive write-up of your API, but I still haven't got around to implementing it. Fingers crossed I'll get to it somewhere in the first half of this year.

Hey @theOehrly, no worries about that last part, I know how maintaining open-source projects can be, take all the time you need! 💚
Regarding the schedule for the current season, the index files are no use for that. They will contain the weekend schedule after the first session, but that's not really useful. Instead, you can use another F1 API for this:
# Get all events for a season
curl "https://api.formula1.com/v1/editorial-eventlisting/events?season=2023" \
     -H 'apikey: t3DrvCuXvjDX8nIvPpcSNTbB9kae1DPs' \
     -H 'locale: en'

# Get meeting data by meetingKey
curl "https://api.formula1.com/v1/event-tracker/meeting/1142" \
     -H 'apikey: t3DrvCuXvjDX8nIvPpcSNTbB9kae1DPs' \
     -H 'locale: en'

Where the first is used to retrieve basic information (including meetingKey) and the latter contains sessionLinkSets.replayLinks to get the replay paths. The API key used here is hard-coded in the iOS app.
Aside from that, I had one more question for you. Let's say I have a full URL to a session (not the livetiming.formula1.com hostname), is there any way to tell FastF1 to fetch it from there?

@f1multiviewer thanks for that information. I only knew about that API partially. I'll see what I can do with it.

Aside from that, I had one more question for you. Let's say I have a full URL to a session (not the livetiming.formula1.com hostname), is there any way to tell FastF1 to fetch it from there?

Do you mean something like fastf1.get_session("2023/2023-03-19_Saudi_Arabian_Grand_Prix/2023-03-19_Race")? I'm not sure, I really understood your question. If yes, should this work only for sessions that are listed in FastF1s event schedule anyway? Or should this work for any other (special) session that is currently not directly accessible through FastF1?
(Also sent you a DM on Reddit regarding another question)

@f1multiviewer you made me aware of the season index API endpoints, so I assume you are likely making use of them?
When I use https://livetiming.formula1.com/static/2023/Index.json to get the schedule for the current season, all I get is information about events that have already taken place. That means, I cannot create a full event schedule for the current season from this endpoint. Am I missing something here, or does the API not provide information about future events?
Also, I'm really sorry that you took the time for creating such an extensive write-up of your API, but I still haven't got around to implementing it. Fingers crossed I'll get to it somewhere in the first half of this year.

Can you explain why exactly is retrieving future events be needed for Fast F1? The year Index.json provides event information for events that are currently going on and have already happened in the past. There is no session or timing data to fetch for future events until the first session for the future event is live, so not sure why it would be needed for.

Can you explain why exactly is retrieving future events be needed for Fast F1? The year Index.json provides event information for events that are currently going on and have already happened in the past. There is no session or timing data to fetch for future events until the first session for the future event is live, so not sure why it would be needed for.

I consider the complete EventSchedule for the current seasons as a feature and not just as a tool for getting timing data for specific sessions. There are use cases where you would want to know information about upcoming events.

https://f1tv.formula1.com/2.0/R/ENG/WEB_DASH/ALL/PAGE/SEARCH/VOD/F1_TV_Pro_Annual/2?orderBy=meeting_End_Date&sortOrder=asc&filter_objectSubtype=Meeting&filter_season=2023&filter_orderByFom=Y&maxResults=100
From 2018 onwards, calendar events data from a season can be retrieved using this F1TV API.
There are other things that can be retrieved with the SEARCH/VOD endpoint, but that is out of the scope of this project.

@eXhumer from what I can tell, this API endpoint doesn't have starting times for the individual sessions. So that means that I would need to make an additional request for each event. This makes more than 20 requests to build the schedule, which is not really feasible.
The fastf1 backend is basically just doing that, aggregating the data from many requests every few hours so that FastF1 can access it with one request. I maybe should change how this data is served, i.e. not use Github 😅

I guess PitLaneTimeCollection is removed now. Not seeing in any of the Index.json files this season

That's correct, they've ceased to emit this feed/topic.

That's correct, they've ceased to emit this feed/topic.

Is there any alternative?

@mithileshchellappan FYI I think it's back again, might've been removed temporarily? I don't know for sure.

Hi,
Looks like you are into the F1 API. Excuse me when I'm in the wrong place, but I have one short question.
I'm looking for a way only getting the flag status during a session. This must be live and as soon as possible.
Is there such way getting this?
I want to program some lights to the color of the flag. Imation when there is a yellow flag all lights in my living room are also yellow :)
Looking for a while now without any success.
Hope finding a good start here. Thanks.
Bert
The Netherlands

@CreatoX that's actually an integration for MultiViewer, which sync your Hue lights (or even an LED panel) to the track status: https://f1mvli.jstt.me/docs/

@CreatoX I recommend you use what @f1multiviewer propose. FastF1 currently has no support for working with the data live.

Thanks for commenting @f1multiviewer @theOehrly . I will take look at MultiViewer.
Looks promising.


Passing data to nested models

Initial Checks

[X] I have searched Google & GitHub for similar requests and couldn't find anything
[X] I have read and followed the docs and still think this feature is missing

Problem
I need to validate nested models based on the parent model fields. For example, I want to choose a single value from all list-like fields (field2 and field3 in this case) based on the ending of name. Each field can be integer-like string or list-like string.
data = {
    "name": "item_2",
    "some_fields": {
        "field1": "1",
        "field2": "2 13 5",
        "field3": "4 6 1",
    }
}

I came up with 2 solutions, but each of them has its drawbacks
Solution 1
The most obvious solution is to use field_validator and ValidationInfo data for some_fields
class SomeFields(BaseModel):
    # big list of optional fields:
    field1: int | None = None
    field2: int | None = None
    field3: int | None = None

class Item(BaseModel):
    name: str
    some_fields: SomeFields

    @field_validator("some_fields", mode='before')
    def choose_single_value(cls, fields: dict, info: ValidationInfo) -> dict:
        """Choose single value from space separated list based on item name"""
        name = info.data.get("name")
        index_from_name = int(name[-1])
        for k, v in fields.items():
            if " " in v:
                fields[k] = v.split()[index_from_name]
        return fields

validated = Item.model_validate(data)
# > name='item_2' some_fields=SomeFields(field1=1, field2=5, field3=1)

But this solution have some performance drawbacks. We use loops both for preparing data (fields_validator mode=before) and for internal pydantic validation. In case of hundreds optional fields inside of some_fields it can be noticeable. Besides that, I expect the values of the nested model to be validated in the nested model, looks more "clean" to me
Solution 2
Better idea IMO is to use field_validator for all nested fields and somehow pass name to nested model. I achieved it via info.context.update in parent model:
class SomeFields(BaseModel):
    # big list of optional fields:
    field1: int | None = None
    field2: int | None = None
    field3: int | None = None

    @field_validator("*", mode="before")
    def choose_single_value(cls, value: str, info: ValidationInfo):
        name = info.context.get("name")
        index_from_name = int(name[-1])
        if " " in value:
            value = value.split()[index_from_name]
        return value

class Item(BaseModel):
    name: str
    some_fields: SomeFields

    @field_validator("name", mode='after')
    def add_context(cls, v: str, info: ValidationInfo):
        info.context.update({"name": v})
        return v

validated = Item.model_validate(data, context={})
# > name='item_2' some_fields=SomeFields(field1=1, field2=5, field3=1)

The problem in this case is the need to initialize context property via model_validate, which is inconvenient and sometimes limiting
Request

This is a common validation use case, so there should be some documentation about best ways to validate nested models based on parent data
Would be nice to have some way to pass data/context to nested models (via Field, _parent.data, etc). I'm not familiar with the implementation of the library, so I don't know what's possible to achieve. At least I want ConfigDict for default context initialization (init_context=true) instead of passing it explicitly in model_validate for every model creation.

Maybe I'm missing something and there are better ways to solve my problem.
Affected Components

[ ] Compatibility between releases
[X] Data validation/parsing
[ ] Data serialization - .model_dump() and .model_dump_json()
[ ] JSON Schema
[ ] Dataclasses
[X] Model Config
[ ] Field Types - adding or changing a particular data type
[ ] Function validation decorator
[ ] Generic Models
[ ] Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
[ ] Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

Hi!
At the moment, I think that the two approaches you've suggested are the best that we're going to be able to offer. The custom validator / validation context approaches are both good ways to solve this problem.
I think a good place to start would be with better docs here. Feel free to open a PR with some good example additions to the docs! Thanks for bringing this to our attention!

Hey Sydney - I'd be interested in picking this one up!

Re specifics:
I think we could extend the custom validators part of the Examples tab on the docs to include some of these.
We could also extend the Validators section of the Concepts tab to talk more about passing context to validators, and how powerful that can be!


[plot] Image API must provide a way to access to the displayed pixmap

The plot Image item provides a custom set/getPixmap.
On the previous API getPixmap was providing the displayed image, anyway a pixmap was defined.
It is not the case anymore.
The image item must provide an API to reach the displayed image (the pixmap or the data transformed with the colormap).

To be correct, the previous API didn't provide a way to retrieve the displayed image, only the alternative pixmap provided to addImage.
After discussions, I think the pixmap name is misleading. One might even expects getPixmap to return a QPixmap...
I would propose to find other names both for the alternative image representation and the displayed image.
And consider splitting the Image object into DataImage and RgbaImage...

Current Image API:
    def setData(self, data, pixmap=None, copy=True):
        """Set the image data"""

    def getData(self, copy=True):
        """Returns the image data"""

    def getPixmap(self, copy=True):
        """Get the optional pixmap that is displayed instead of the data"""

Proposed API common to RgbaImage and DataImage:
    def getData(self, copy=True):
        """Returns the image data as provided to :meth:`setData`"""

    def getRgbaImageData(self, copy=True):
         """Get the displayed RGB(A) image

        :returns: numpy.ndarray of uint8 of shape (height, width, 4)
        """

DataImage API:
    def setData(self, data, alternative=None, copy=True):
        """"Set the image data and optionally an alternative RGB(A) representation

        :param numpy.ndarray data: Data array with 2 dimensions (h, w)
        :param alternative: RGB(A) image to display instead of data, shape: (h, w, 3 or 4)
        :type alternative: None or numpy.ndarray
        """

    def getAlternativeImageData(self, copy=True):
         """Get the optional RGBA image that is displayed instead of the data

        :returns: None or numpy.ndarray
        """

RgbaImage API:
    def setData(self, data, copy=True):
        """"Set the RGB(A) image data

        :param data: Data array with 3 dimensions (h, w, 3 or 4)
        :type data: numpy.ndarray of uint8
        """

    def getGrayscaleImageData(self, copy=True):  # This can also be in the common Image API
         """Get the displayed image converted to grayscale

        :returns: numpy.ndarray of uint8 of shape (height, width)
        """


Please provide package repository's for major distributions

Hi,
please provide package repository's for your supported linux distributions. This will make it much easier
to keep our systems on track with the releases.

The agent supports updates from the service / web UI.  Go to admin pool web UI and click update pool.  The agents will download tar gz from service and update itself.  It's also the same code paths for windows, osx and all distros of linux.  This is why it's not a package repository in linux.

Yes but this is hard to automate and to monitor for new versions.
For Example we use only rpm packages on our systems and so we can use a nagios plugin that is called check_updates to check for new updates for all software.


add timetable

The PR should be against your own repo and master branch so you can keep your merged changes.


MRIQC process hangs during singularity run

Dear MRIQC Team,
When I tried to run mriqc using singularity interactively on my computing cluster's terminal, the process hangs for as long as I let it. When I turned on debug mode, the debug messages stopped at DEBUG   [U=32291,P=41324]  PostStartProcess()            Post start process. I have checked and can confirm that all inputs are valid. Could you please help shed some light on the issue?

Last few line of debugging messages:

Any suggestion is greatly appreciated!!! -Mengjia

Hey Team,
So I figured out why the process hung -- there were thousands of subjects in my working directory which would take pybids hours to index. After deleting the unnecessary subjects, the process ran smoothly without issues. -Mengjia


Websocket: Export Websocket to Proxyman Log and HAR

Description
Websocket data should be exported to ProxymanLog and HAR files too
⚠ Ping @NghiaTranUIT to get the HAR Model for Websocket and ProxymanLog.
Acceptance Criteria
ProxymanLog

Able to export WebSocket messages to ProxymanLog on Windows
Open this ProxymanLog on Proxyman for macOS/Windows -> Verify you can open it and see the content

Able to export WebSocket messages to ProxymanLog on macOS
Open this ProxymanLog on Proxyman for macOS/Windows -> Verify you can open it and see the content

HAR

Same step with ProxymanLog
Open https://www.piesocket.com/websocket-tester on Google Chrome / Firefox -> Make sure WSS request -> Open Google Dev Tool -> Export WS as HAR
Open it on Proxyman for Windows

Unit Tests
Proxyman Log
1. Import from Windows

Given: A ProxymanLog (contains WS and WSS), which is exported from Windows
Expected: Able to load by Proxyman Windows -> Assert the data

2. Import from macOS

Given: A ProxymanLog (contains WS and WSS), which is exported from macOS
Expected: Able to load by Proxyman Windows -> Assert the data

3. Export on Windows

Given: Write Unit Test to: Start the shell script (reference), capture a request in the FlowPool -> Export it by code
Expected: Import the file by code again -> Assert this data

HAR
1. Import from Windows

Given: A HAR (contains WS and WSS), which is exported from Windows
Expected: Able to load by Proxyman Windows -> Assert the data

2. Import from macOS

Given: A HAR (contains WS and WSS), which is exported from macOS
Expected: Able to load by Proxyman Windows -> Assert the data

3. Export on Windows

Given: Write Unit Test to: Start the shell script (reference), capture a request in the FlowPool -> Export it by code
Expected: Import the file by code again -> Assert this data


Linux support

Do you ever have plans to release a Linux version?
Thanks :)

There are currently no plans to support Linux, you could try running it with wine and maybe it will work
The main problem is that this project is based on WPF, which is windows only

ohh ok then, thanks :)


hotswap discards non-table values

https://github.com/rxi/lume/blob/da0d1bb/lume.lua#L694-L695
    local newmod = require(modname)
    if type(oldmod) == "table" then update(oldmod, newmod) end

currently, if the returned value of a module isn't a table, it (newmod) is discarded.
in my case, my module returns a function, and i expect package.loaded to contain the new value. i am using lurker.lua to trigger hotswapping.
local some_state = 'blah'
local function exec_stuff(name)
  local f = require('stuff_'..name) -- bug: never returns new function value
  some_state = f(some_state)
end

this dirty hack seems to work:
-    if type(oldmod) == "table" then update(oldmod, newmod) end
+    if type(oldmod) == "table" then update(oldmod, newmod) else oldmod = newmod end

The problem here is that there's no way to modify existing functions: if you've required and stored the same function somewhere else in the past, that version doesn't get updated with your patch, which means hotswap doesn't actually hotswap anything in that case.
A workaround could be to use __call like so:
return setmetatable({}, {__call = drop_arg(1, my_function)})

This is a normal, modifiable table object, so it should work with hotswap, but you can call it as if it were a normal function. That said, if the current behavior is fine with you you can do the same thing manually, without modifying hotswap:
package.loaded[my_mod] = nil
local f = require(my_mod)

right, afaik you can only emulate updating existing references either by utilizing a wrapper table or by replacing all of an existing table's values. that's why, in my example code, i'm running require each time a function is needed instead of caching it locally. i was expecting this.
what i'm really trying to describe is that i figured hotswap would update non-table return values in package.loaded. this includes returned numeric and userdata values as well.
nulling out the key in package.loaded would work, but that would need to be part of a lurker.lua hook to prevent reloading an untouched file, which might be expensive if the value is needed frequently.


RFC: Remove BUILDPLATFORM

I'm pretty split on this one. It should be largely inconsequential for the general case where an engineer is running Apple Silicon locally and building for a Graviton environment, but it has some implications when these platforms are not aligned.
https://docs.docker.com/build/guide/multi-platform/#platform-build-arguments
https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

This is my understanding:
Preferring BUILDPLATFORM:

Quicker to build when local platform differs from deployment target
Local behaviour is more likely differ from CI
Locally built image may not be deployable during disaster recovery

Preferring to hardcode amd64 or arm64:

Slower to build when local platform differs from deployment target
Local behaviour is less likely to differ from CI
Locally built image should be deployable during disaster recovery
More explicit in the Dockerfile

Preferring no --platform / TARGETPLATFORM:

Slower to build when local platform differs from deployment target
Local behaviour is less likely to differ from CI
Locally built image should be deployable during disaster recovery
Somewhat implicit and depends on tooling support for the --platform build option

Came back around to this and landed on the same conclusion, so I'll get this merged once I write up a changeset, then look at improving guidance in https://seek-oss.github.io/skuba/docs/deep-dives/arm64.html for existing x86 projects.


AttributeError: 'NoneType' object has no attribute 'rule'

(I use python 3.5 and flask.) I made update the raven package today. From this time I had below errors:
File "raven/contrib/flask.py", line 235, in after_request
    self.client.transaction.pop(request.url_rule.rule)
AttributeError: 'NoneType' object has no attribute 'rule'

and
raven/contrib/flask.py", line 220, in before_request                │
self.client.transaction.push(request.url_rule.rule)
AttributeError: 'NoneType' object has no attribute 'rule'

This looks to be a duplicate of https://github.com/getsentry/raven-python/issues/863


BasicAer qasm_simulator not detecting that sample measurement can be used

Informations

Qiskit Terra version: 0.7.0
Python version: 3.6.6
Operating system: Ubuntu

What is the current behavior?
Currently, it can detect that it can sample if there is only measurments at the end, but sometimes the measurment instruction is put further up, and it's not detected even when there is no more operations applied to the qubit.
Steps to reproduce the problem
If you look at what gets called with the following:
from qiskit import QuantumCircuit, ClassicalRegister, QuantumRegister
from qiskit import execute, BasicAer

qubit_reg = QuantumRegister(4)
clbit_reg = ClassicalRegister(4)

qc1 = QuantumCircuit(qubit_reg, clbit_reg)
qc1.h(qubit_reg[0])
qc1.cx(qubit_reg[0], qubit_reg[1])
qc1.measure(qubit_reg, clbit_reg)

job_sim = execute(qc1, BasicAer.get_backend('qasm_simulator'))
sim_result = job_sim.result()

The function _add_sample_measure is never called.
What is the expected behavior?
It should just sample measure, which would be quicker.
Suggested solutions
Change the _validate_measure_sampling function in qiskit/providers/builtinsimulators/qasm_simulator.py to be something like:
    def _validate_measure_sampling(self, experiment):
        """Determine if sampling can be used for an experiment

        Args:
            experiment (QobjExperiment): a qobj experiment
        """
        measure_flags = {}
        if hasattr(experiment.config, 'allows_measure_sampling'):
            self._sample_measure = experiment.config.allows_measure_sampling
        else:

            for instruction in experiment.instructions:
                if instruction.name == "reset":
                    measure_flags[instruction.qubits[0]] = False
                    self._sample_measure = False
                    return

                if measure_flags.get(instruction.qubits[0], False):
                    if instruction.name not in ["measure", "barrier", "id", "u0"]:
                        for qubit in instruction.qubits:
                            measure_flags[qubit] = False
                            return
                elif instruction.name == "measure":
                     for qubit in instruction.qubits:
                            measure_flags[qubit] = True

        self._sample_measure = True

        for key, value in measure_flags.items():
            if value == False:
                self._sample_measure = False
                return

I can make a pull request if needed.


[pages] Clarify Remix Function Generation Order

Closes #8357
Clarifies that when creating a Remix project, the functions directory will only be generated after interacting with Remix at least once(dev and/or build).

Force-push applied to clarify that the file that will be generated will have the extension .js rather than .ts as it doesn't require Type-checking after being built.


add initial spec

This spec is exported from the following notion document:
https://www.notion.so/statechannels/Copy-of-Dispute-game-design-2eee37cd8cc943759405a9ef97885411
I made a copy of Mike's version, and made the description more explicit.
The goal here is to describe DisputeGame in a way that facilitates the security analysis. This is important to Optimism; it is also important for us, as we plan to write a formal verification of the dispute protocol.
The explicit description also hopefully serves as an implementation guide.
We should choose to review this spec in notion or in GitHub, but not both. @lalexgap and @kerzhner should coordinate how to review this.
Resolves #37.
Resolves #10.

Per slack, we're going to use notion to house the design doc for now. Closing this.


Fix deprecated strict warning in phpunit.

Why did this one fail? Its exactly like the other ones.

Don't worry. I'll sort that out.

Thank you @vinkla.

Great! Thanks.


Allow the codeql-action to be run locally

This change allows the codeql-action to be run locally through
act.
In order to run the action locally, you need to do two things:

Add the CODEQL_LOCAL_RUN: true environment variable. The only way
I could figure out how to do this was to add it directly in the
workflow file in an env block. It should be possible to add it
through a .env file and pass it to act, but I couldn't get it
working.
Run this command act -j codeql -s GITHUB_TOKEN=<MY_PAT>

Setting the CODEQL_LOCAL_RUN env var will fill in missing env vars
that the action needs, but isn't set by act. It will also avoid
making api calls to github that would fail locally.
Merge / deployment checklist

[x] Confirm this change is backwards compatible with existing workflows.
[x] Confirm the readme has been updated if necessary.

Setting variables in a file named .env seems to work for me. Note that the flag is just a filename, which is resolved against the current working directory.

Hmmm...The .env did not work for me. I wonder if I'm using an older version than you. Also, are you on Linux?

Yes,  on linux.

When generating the status reports, we access the github API to get the workflow ID. This would also need to be dealt with somehow. Probably best to audit all uses of the getApiClient method.

@robertbrignull I believe that is already handled here: https://github.com/github/codeql-action/pull/117/files#diff-8cd4968b81985f0efc2053eabc37db6dR263
Under local mode, no status messages are sent.

I believe that is already handled here: https://github.com/github/codeql-action/pull/117/files#diff-8cd4968b81985f0efc2053eabc37db6dR263

That stops the reports from being sent, but the call I was thinking of is when we are constructing the data to send, which happens before that. However now I look closely it is already covered by https://github.com/github/codeql-action/pull/117/files#diff-8cd4968b81985f0efc2053eabc37db6dR102 as this is the method I was thinking of.

There is at least one call to apiClient that we need https://github.com/github/codeql-action/blob/main/src/config-utils.ts#L404. Though, I could probably still explicitly fail other requests if they are invoked from local mode.

I haven't used act before. Presumably it still runs in the context of some repository? In which case I expect that call should work as normal. I do wonder if it would make for a safer situation if we took the stance of denying all HTTP requests in local mode, and allowing only the ones we know to be safe, instead of the current situation which is the opposite.

Yes, that is what I was proposing indirectly above. I could do something like this:
export const getApiClient = function(allowLocalRun = false) {
  if (isLocalRun() && !allowLocalRun) {
    throw new Error('Invalid API call in local run');
  }
  ...

And then for calls we know to be safe, we pass the true flag. And for calls that aren't, we will fail-fast and then need to handle them differently.

I'm confused why I'm getting errors on the Code scanning action. This is coming from code that I don't think I have changed. It looks like this check was not run for #116, though it was run for #115 (and it failed, too, but for different checks).
Is it possible the codeql action is not being run for local pull requests?

Even more confusing is that this failing check is not showing up in the actions tab, but previous calls to the codeql actions were invoked on this branch and were successful. Eg- https://github.com/github/codeql-action/actions/runs/176329168

The Code Scanning check is the one that reports changes in alerts. So when it is marked as failed it's because that PR introduces alerts. Things might be a little confused right now because we had to revert some changes to the main branch so this reintroduced some alerts to that branch that had been fixed.

Thanks for the clarification.

@robertbrignull is this a worthwhile thing to add to the action?

@sampart I added a section to the readme. Should I move that to contributing?

Actually, moving it sounds like a great idea, thanks.  Please don't fully replace what's there, as I think it's helpful that we also tell people about the option of pushing a branch (that's simpler than using act), but please do add the info in.  I don't think we need it in the README if it's in the Contributing guide.  Thanks!

Right...this is a little different from the Running the action section. The instructions are for running from the main repository, but locally through act. So, to get this working, you need to run this from a repository different from this one. It's a subtle difference that I didn't call out in the README. I'll make that explicit. The main use case is to debug why a specific repository's codeql action is failing.

Oh, I totally hadn't realised that distinction, thanks for mentioning.  Would an end-user of the action use this to troubleshoot their own workflows that use the action?  If so, README would be the right place for this, after all.

@sampart Probably not useful for end users. So, I think contributing is the right place for it. When I discussed this earlier, the use case would be for maintainers or contributors to determine why a particular repository is failing by reproducing locally.

@aeisenberg a bunch of js files have got into the src directory somehow. They should be in lib, so I'm not sure how that happened.

Darn. Let me fix that. Not sure what I did.

I'm out tomorrow and Monday. Please merge if you feel like this is good. Otherwise, I'll look at it when I get back.


fix: add The Unlicense license SDPX in package.json

closes #230
This is required by our infra tooling to validate the license compliance and allows us to consumed this package.

@fearthecowboy is it possible to get this in?

I don't have access on this repo, so can't help you I'm afraid.
I'll a little surprised your infra tool doesn't access the inclusion of a LICENSE file in the package?

@G-Rath yeah we use d2l-license-checker and it lacks couple of features, but have not found any other good library do you any suggestion? I was surprised that it cannot scan the repo for the LICENSE file as well. there is definitely some improvement to be done there!

@G-Rath I've sent you an invite.


Fails on Mac - dotnet new --install Microsoft.AspNetCore.SpaTemplates::*

Trying to install SPA templates on dotnet cli - but seems to fail to complete on my MAC
MacOS - 10.12.5
Jsinhs-MacPro:~ jsinh$ dotnet new --install Microsoft.AspNetCore.SpaTemplates::*
  Restoring packages for /Users/jsinh/.templateengine/dotnetcli/v1.0.4/scratch/restore.csproj...
  Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/system.io.compression.zipfile/index.json'.
  Too many open files
  Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/system.io.compression/index.json'.
  Too many open files
  Failed to download package 'System.Threading.Timer.4.0.1' from 'https://api.nuget.org/v3-flatcontainer/system.threading.timer/4.0.1/system.threading.timer.4.0.1.nupkg'.
  An error occurred while sending the request.
    Couldn't connect to server
  Failed to download package 'System.Threading.Timer.4.0.1' from 'https://api.nuget.org/v3-flatcontainer/system.threading.timer/4.0.1/system.threading.timer.4.0.1.nupkg'.
  An error occurred while sending the request.
    Couldn't connect to server
/usr/local/share/dotnet/sdk/1.0.4/NuGet.targets(97,5): error : Failed to download package 'System.Threading.Timer.4.0.1' from 'https://api.nuget.org/v3-flatcontainer/system.threading.timer/4.0.1/system.threading.timer.4.0.1.nupkg'. [/Users/jsinh/.templateengine/dotnetcli/v1.0.4/scratch/restore.csproj]
/usr/local/share/dotnet/sdk/1.0.4/NuGet.targets(97,5): error : An error occurred while sending the request. [/Users/jsinh/.templateengine/dotnetcli/v1.0.4/scratch/restore.csproj]
/usr/local/share/dotnet/sdk/1.0.4/NuGet.targets(97,5): error :   Couldn't connect to server [/Users/jsinh/.templateengine/dotnetcli/v1.0.4/scratch/restore.csproj]
/usr/local/share/dotnet/sdk/1.0.4/NuGet.targets(97,5): error : Unable to load package 'System.Threading.Timer'. [/Users/jsinh/.templateengine/dotnetcli/v1.0.4/scratch/restore.csproj]

The installation of SPA templates did work though via yo when I tried.
Is this my machine only issue or this is not intended to work on Mac / non-windows environment?
Thanks in advance

Based on the log entries like this:
Failed to download package 'System.Threading.Timer.4.0.1' from 'https://api.nuget.org/v3-flatcontainer/system.threading.timer/4.0.1/system.threading.timer.4.0.1.nupkg'.
  An error occurred while sending the request.
    Couldn't connect to server

... it looks like a networking issue. You might find it works if you try again. If not, could you raise this with NuGet? It's not something we can address in this repo, as that's an unrelated .NET package.


Mention in Readme that - API level 29 onward's plugin will not work for reading files outside project directory.

Feature Request
Motivation Behind Feature

The readme does not mention that this plugin will not work if we use Api level greater than 29. Please mention that so the users will get a clear understanding of the app.
Feature Description

Google Play Store now accepts new apps with API levels greater than 29 only. So adding clear details to the readme will give a clear understanding.
Alternatives or Workarounds

You can still read files outside the project directory (which I assume you mean the application directory...)
But there are many directories outside of the project directory that are simply no longer accessible, or have limited access with Scoped Storage, which was introduced in API 29 and completely enforced in API 30.
You should be able to read files inside the cordova.file.external* paths for example, all of which are outside of your application directory, but you can only read files in which your app has created. And you may need the READ_EXTERNAL_STORAGE permission granted for reading files that wasn't created by your app.
Creating new directories in the external storage is also no longer permissible with the new scoped storage rules.
I agree though, documentation will likely need to be updated. I'm sure there is also some conflicting information in our previous documentation.

Noted at https://github.com/apache/cordova-plugin-file#androids-external-storage-quirks


Add cache strategy with cacheFirst and networkFirst

This change is to always get the latest GitHub user info when online. This will fix the stale user info.

@annsa9 Thanks for the PR 👍


Support for NER models

Discussed in https://github.com/cdpierse/transformers-interpret/discussions/83

Originally posted by pabvald May  1, 2022
Hi!
I have tried to generate explanations for NER models but I haven't found any description of how to implement this, so I have tried to extrapolate what is been already done for other tasks, such as multi-class classification.
I have followed the approach described in the Captum tutorial to interpret BERT in which a custom forward function that allows to access a particular position of the prediction using the position input argument is used and the attributions are computed with respect to the BertEmbeddings using the captum.attr.LayerIntegratedGradients class.
I have generated two types of visualizations:

Attributions considering each token (position) and its corresponding predicted NER label: this allows to see how the rest of the sentence influenced each token to be classified into a particular NER class.
Attributions considering a single token and all the possible NER classes: this allows to see how the rest of the sentence influenced a certain token to be classified into a class and not into any of the others.

A notebook with these two visualizations can be found in this repo. I would like to know if my approach makes sense, or, if not,  what I should do to correctly visualize explanations for NER models. Once I am sure my approach is right, I would like to contribute to Transformers Interpret implementing the support for NER models.
Thank you in advance for the help!

This issue has been solved with Release #91


Make "Name" useful

Goals

[ ] Consistent behavior across formatters
[ ] Requirement composition with namespacing

Tasks

[ ] Implement Push-Function [-Namespace] <string> [-Requirements] <Requirements[]> for composing arrays of requirements, automatically prefixing the Name property of the Requirements with Namespace.
[ ] Consistent use of Name vs Description in formatters
[ ] Deprecate nested Requirements
[ ] Update README with modified patterns

Not sure how to handle namespace delimiters

#46


FIX Issue #4

To delete a product, the path has to be /products/:id

Thanks for the fix, we'll merge it ASAP 👍


Bokeh model does not work

I'm having a trouble making bokeh model work. The corresponding script for this model (/codes/config/latent_bokeh/test.py) doesn't seem to be correct - there are multiple places where it fails, can anybody confirm that it should work?
The following output was obtained using these parameters:

src_lens: 160
tgt_lens: 14
disparity: 50

Gradio produces wrong picture with the config codes/config/latent_bokeh/options/bokeh/test/refusion.yml:

Hi, in test config file, you should change the network settings to:
network_G:
  which_model: ConditionalNAFNet
  setting:
    img_channel: 4
    width: 64
    enc_blk_nums: [2, 2, 4, 8]
    middle_blk_num: 12
    dec_blk_nums: [2, 2, 2, 2]

Moreover, since the bokeh effect transform task is quit different with other tasks, you should also change some functions (with **kwargs) in the sde_utils.py as:

    def score_fn(self, x, t, **kwargs):
        # need to pre-set mu and score_model
        noise = self.model(x, self.mu, t, **kwargs)
        return self.get_score_from_noise(noise, t)

    def noise_fn(self, x, t, **kwargs):
        # need to pre-set mu and score_model
        return self.model(x, self.mu, t, **kwargs)

    def reverse_sde(self, xt, T=-1, save_states=False, save_dir='sde_state', **kwargs):
        T = self.sample_T if T < 0 else T

        x = xt.clone()
        for t in tqdm(reversed(range(1, T + 1))):
            score = self.score_fn(x, t, **kwargs)
            x = self.reverse_sde_step(x, score, t)
            # x = self.reverse_sde_step_mean(x, score, t)

            if save_states: # only consider to save 100 images
                interval = self.T // 100
                if t % interval == 0:
                    idx = t // interval
                    os.makedirs(save_dir, exist_ok=True)
                    x_L, x_R = x.chunk(2, dim=1)
                    tvutils.save_image(torch.cat([x_L, x_R], dim=3).data, f'{save_dir}/state_{idx}.png', normalize=False)

        return x

Sorry, the problem persists. I applied all your changes and tried the image from dataset:
metadata: 00000, Sony50mmf1.8BS, Sony50mmf16.0BS, 35

Hi, can you also provide the refusion.yml file?

name: latent-reffusion-bokeh
suffix: ~  # add suffix to saved images
model: latent_denoising
distortion: bokeh
gpu_ids: [0]

sde:
  max_sigma: 50
  T: 100
  schedule: cosine # linear, cosine
  eps: 0.005

degradation:
  # for denoising
  sigma: 25
  noise_type: G # Gaussian noise: G

  # for super-resolution
  scale: 4

datasets:
  test1:
    name: Test
    mode: BokehLQ
    dataroot_LQ: /home/x_ziwlu/datasets/ntire2023/bokeh/ntire_val/src
    dataroot_meta: /home/x_ziwlu/datasets/ntire2023/bokeh/ntire_val/meta.txt

#### network structures
network_G:
  which_model: ConditionalNAFNet
  setting:
    img_channel: 4
    width: 64
    enc_blk_nums: [2, 2, 4, 8]
    middle_blk_num: 12
    dec_blk_nums: [2, 2, 2, 2]

network_L:
  which_model: UNet
  setting:
    in_ch: 3
    out_ch: 3
    ch: 64
    ch_mult: [1, 2, 4]
    embed_dim: 4

#### path
path:
  pretrain_model_G: pretrained_models/latent-reffusion-bokeh.pth
  pretrain_model_L: pretrained_models/latent-bokeh.pth

Hi, here is my results based on your provided image:

I have also uploaded the app.py to this repo (actually I have already updated several files last time). You may want to download the updated code and run it again.
Best.

Thank you! I replicated the test result!
Then I tried it on my own images (which are outpaintings done by Stable Diffusion) and here I see that diffusion process seems not ending well (noise left) or the blurry parts of the image are untouched.
My test images:

Some results:
upscale_4x_t_100_sigma_50_cosine:

upscale_4x_t_200_sigma_100_cosine:

Hi, our Latent Reffusion always produces noisy images if the testing image is out of the distribution of the training dataset. To fine-tune the model, you can add some small noise to the latent when pretraining the latent-UNet model. And since all training images are synthetically generated, the model is hard to deal with other types of images (out of the distribution). This problem also happened in other CNN bokeh models, which you can find in the challenge report.

It would be better if you have your own dataset to retrain the model.

Okay, understood, thanks. I guess this issue can be closed then!