UnboundLocalError on Arm64 M3 Mac with blablador backend

Whenever I submit a prompt to blablador, I get the following error message:
UnboundLocalError: cannot access local variable 'instructions' where it is not associated with a value---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
Cell In[2], line 1
----> 1 get_ipython().run_cell_magic('bob', '', 'load blobs.tif and show it\n')
File /opt/homebrew/Caskroom/mambaforge/base/envs/bob-env/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2541, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
2539 with self.builtin_trap:
2540     args = (magic_arg_s, cell)
-> 2541     result = fn(*args, **kwargs)
2543 # The code below prevents the output from being displayed
2544 # when using magics with decorator @output_can_be_silenced
2545 # when the last Python token in the expression is a ';'.
2546 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):
File /opt/homebrew/Caskroom/mambaforge/base/envs/bob-env/lib/python3.11/site-packages/bia_bob/_machinery.py:140, in bob(line, cell)
138         text = f"The file has been saved as {filename}. You can open it using:\n\n    jupyter lab {filename}\n\n"
139 else:
--> 140     code, text = generate_response_to_user(Context.model, user_input, image)
142     if image is not None:
143         # we need to add this information to the history.
144         Context.chat.append({
145             "role": "user",
146             "content": f"Assume there is an image stored in variable {image_name}. The image can be described like this: {response}. Just confirm this with 'ok'."
147         })
File /opt/homebrew/Caskroom/mambaforge/base/envs/bob-env/lib/python3.11/site-packages/bia_bob/_utilities.py:27, in generate_response_to_user(model, user_prompt, image, additional_system_prompt, max_number_attempts, system_prompt)
25 for attempt in range(1, max_number_attempts + 1):
26     if system_prompt is None:
---> 27         system_prompt = create_system_prompt()
28     if additional_system_prompt is not None:
29         system_prompt += "\n" + additional_system_prompt
File /opt/homebrew/Caskroom/mambaforge/base/envs/bob-env/lib/python3.11/site-packages/bia_bob/_utilities.py:231, in create_system_prompt(reusable_variables_block)
228 else:
229     warnings.warn("Deprecated use of create_system_prompt with reusable_variables_block. Do not pass this parameter to make your code work mid/long-term.")
--> 231 snippets, additional_snippets = generate_code_samples()
232 libraries = Context.libraries
234 system_prompt = f"""
235 You are a extremely talented bioimage analyst and you use Python to solve your tasks unless stated otherwise.
236 If there are several ways to solve the task, chose the option with the least amount of code.
(...)
278 Make sure to keep your answer concise and to the point. Make sure the code you write is correct and can be executed.
279 """
File /opt/homebrew/Caskroom/mambaforge/base/envs/bob-env/lib/python3.11/site-packages/bia_bob/_utilities.py:211, in generate_code_samples()
209         pass
210     # special treatment for code snippets from stackview, as it won't work with the custom kernel
--> 211     if "stackview" not in instructions or "stackview" in Context.libraries:
212         additional_instructions.append(instructions)
214 additional_snippets = "\n".join(additional_instructions)
UnboundLocalError: cannot access local variable 'instructions' where it is not associated with a value
minimal notebook:
biabob.ipynb.zip
steps to reproduce:
mamba env create -f https://github.com/haesleinhuepf/bia-bob/raw/main/environment.yml
mamba activate bob-env
mamba env config vars set BLABLADOR_API_KEY=<my API key>
mamba activate bob-env #to enable the BLABLADOR_API_KEY env variable
jupyter lab

Hi @thawn ,
thanks for the report. Unfortunately, this works on my PC:

As a hardware-related issue is unlikely, I presume it has something to do with your conda environment. Could you share the output of conda list?
Thanks again!

Ah, I presume this is related to #147 and the related fix. I'll do a little modification in a minute...

You know what, I think git-bob can do this.
git-bob think about this!

Fantastic. git-bob implement exactly this!

Hm, let's add the statement before the try: statement.
git-bob solve

Ok, git-bob didn't manage, but I released bia-bob 0.21.2 Would you mind checking if it works for you @thawn ?
pip install bia-bob==0.21.2

@haesleinhuepf thanks a lot for the lightning-fast fix! I just tested it on my machine and it works perfectly.
FYI, I had to switch to the model 'alias-large' to get a useful response.

Awesome, thanks for the feedback!
I must admit, I haven't used blablador for a while (happy that it still works) and get the most ouf of bia-bob using claude-3.5-sonnet these days. The new gpt4o-2024-08-06 has the longest output (16k tokens) btw in case you're generating really long code snippets ;-)
Thanks again!


[Issue] Cannot login on laptop with latest azd

[X] Make sure you've installed the latest version using instructions in the wiki

Output from azd version
azd version 0.5.0-beta.1 (commit 57cba771d598398f30acfef18a272d1906a8e4f1)
Output from az version
{
  "azure-cli": "2.40.0",
  "azure-cli-core": "2.40.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "containerapp": "0.3.11",
    "containerapp-compose": "0.2.2"
  }
}

Describe the bug
When I run azd login, I do a successful flow in the browser, but then see an error:
% azd login

ERROR: logging in: failed saving configuration. failed writing configuration data

To Reproduce
I do not have this issue in Github Codespaces, so it's possibly specific to my laptop. Notably, my laptop is an M1. I can replicate both inside VS Code and in Terminal.
Expected behavior
I expected successful login.
Environment
VS Code version dump:
Version: 1.74.3 (Universal)
Commit: 97dec172d3256f8ca4bfb2143f3f76b503ca0534
Date: 2023-01-09T17:07:18.579Z (2 days ago)
Electron: 19.1.8
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 22.2.0
Sandboxed: No

@v-xuto can you try to reproduce this issue and update?

@rajeshkamal5050 Sure. We tested on a Macbook Pro, executed the command Azd login successfully, we did not reproduce this issue.

I ran it in debug mode, here are the full logs:
(.venv) fastapi-azure-function-apim % azd login --debug
2023/01/12 07:41:12 main.go:169: using cached latest version: 0.5.0-beta.1 (expires on: 2023-01-13T00:40:02Z)
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-config-list-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-config-get-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-config-set-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-config-unset-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-config-reset-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-env-set-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-env-set-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-env-select-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-env-new-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-env-new-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-env-list-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-env-refresh-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-env-refresh-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-env-get-values-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-env-get-values-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-infra-create-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-infra-create-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-infra-delete-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-infra-delete-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-pipeline-config-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-pipeline-config-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-telemetry-upload-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-template-list-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-template-show-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-auth-token-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-auth-token-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-version-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-version-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-show-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-show-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-restore-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-restore-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-login-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-login-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-logout-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-monitor-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-monitor-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-down-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-down-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-init-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-init-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-up-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-up-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-provision-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-provision-action'
2023/01/12 07:41:12 cobra_builder.go:170: registering flags for action 'azd-deploy-action'
2023/01/12 07:41:12 cobra_builder.go:190: registering resolver for action 'azd-deploy-action'
2023/01/12 07:41:12 cobra_builder.go:131: Resolved action 'azd-login-action'
2023/01/12 07:41:12 middleware.go:106: running middleware 'debug'
2023/01/12 07:41:12 debug.go:38: failed converting AZD_DEBUG to boolean: strconv.ParseBool: parsing "": invalid syntax
2023/01/12 07:41:12 middleware.go:106: running middleware 'telemetry'

ERROR: logging in: failed saving configuration. failed writing configuration data

There's an error reported from debug.go but that might only happen in debug mode, so I don't know if that's an actual error. The actual error happens after it tries to run telemetry middleware.

Maybe the actual err could be reported on this line, to help with debugging?
https://github.com/Azure/azure-dev/blob/fa3bfef5402d135eb3ed39eb7cef3db8d160c9e9/cli/azd/pkg/config/manager.go#L60

@pamelafox While the build finishes, you could try rerunning with sudo and see if that fixes it. And if so, try deleting ~/.azdfolder andazd` will recreate it: https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/troubleshoot#azd-init-requires-sudo

That was indeed the issue, so I think this issue can be closed as an old issue. The PR is still a helpful PR however, leaving that open.


editor.snippetSuggestions not being honored

Default value for editor.snippetSuggestions is bottom
Type log in a typescript file
Stable build:

Insiders:

Create a css snippet with prefix bac
In the stable build, this snippet appears at the bottom.
In the Insiders, it appears after the items with prefix bac

Yeah, that the consequence of now sorting by score. So as soon as you type something, snippets will be ranked and sorted as all other completions. Unsure if that is desirable, esp since there is inline-sorting as well...


Equinix update

Various updates on EquinixMetal compute driver
Description
Handle exceptions where necessary, pass explicitly disk and disk_size instead of kwargs on create_node, add provisionable_on field on images and regions field on sizes.
Status

done, ready for review

Checklist (tick everything that applies)

[x] Code linting (required, can be done after the PR checks)
[ ] Documentation
[ ] Tests
[x] ICLA (required for bigger changes)

Codecov Report

Merging #1548 (d85509a) into trunk (1749a7f) will decrease coverage by 0.00%.
The diff coverage is 42.85%.

@@            Coverage Diff             @@
##            trunk    #1548      +/-   ##
==========================================
- Coverage   83.08%   83.08%   -0.01%
==========================================
  Files         394      394
  Lines       84795    84808      +13
  Branches     9004     9006       +2
==========================================
+ Hits        70456    70460       +4
- Misses      11274    11282       +8
- Partials     3065     3066       +1

Impacted Files
Coverage Δ

libcloud/compute/drivers/equinixmetal.py
64.28% <40.74%> (-0.98%)
:arrow_down:

libcloud/test/compute/test_equinixmetal.py
90.12% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1749a7f...d85509a. Read the comment docs.

Codecov Report

Merging #1548 (d85509a) into trunk (1749a7f) will decrease coverage by 0.00%.
The diff coverage is 42.85%.

@@            Coverage Diff             @@
##            trunk    #1548      +/-   ##
==========================================
- Coverage   83.08%   83.08%   -0.01%
==========================================
  Files         394      394
  Lines       84795    84808      +13
  Branches     9004     9006       +2
==========================================
+ Hits        70456    70460       +4
- Misses      11274    11282       +8
- Partials     3065     3066       +1

Impacted Files
Coverage Δ

libcloud/compute/drivers/equinixmetal.py
64.28% <40.74%> (-0.98%)
:arrow_down:

libcloud/test/compute/test_equinixmetal.py
90.12% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1749a7f...d85509a. Read the comment docs.


Range filter for duration

Can use Slider from MUI
Reference: https://mui.com/material-ui/react-slider/#custom-marks
need start and end so use https://mui.com/material-ui/react-slider/#range-slider

@pranshugupta54 can I work on this?

Hey @shresritik, assigned it.
We'll be releasing v2 soon, so you can use the version2 branch instead of main.
Contact us on Discord for help with local setup

Hello @pranshugupta54
I want to know where the filter range to be placed
Also, Discord link is displaying unable to accept invite

https://discord.gg/bsbBytBqBc

Fixed in #113


About Zipkin Monitor

When I use istio to deploy the application, how can my Zipkin monitor the call chain of MySQL At present, my zipkin  can only monitor my springboot web application.  I want it to monitor MySQL too, and show it in zipkin's  call chain graphy, thanks for your answer.

when  use kiali ,  it can monitot mysql  as above,  is this function exist in zipkin? how to open this function?

Tracing is only for HTTP traffic, which utilizes HTTP headers to carry tracing context and does not apply to MySql protocol. See https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/observability/tracing for some background knowledge. What you probably should do is writing application code with zipkin client library in your ts-food-service to create spans for the MySQL queries with trace context extracted from HTTP header and send them directly from your application to zipkin server.
Close since this works as expected. Let me if you have further questions. Thanks!


fix: Update metadata_utils.py

fix:
added watermark_type string conversion to lower to make sure that it will equate to 'high_watermark' string
https://github.com/amundsen-io/amundsen/issues/848
Summary of Changes

Tests
verfiy "/api/metadata/v0/table" response for table metadata to make sure that it is able to equate high_watermark string keyword. I have attached the test case gist
https://gist.github.com/NiravLangaliya/9fd2b5d243e92c924381d7d1509d2105
and also a screenshot which I ran.

Documentation

CheckList
Make sure you have checked all steps below to ensure a timely review.

[x] PR title addresses the issue accurately and concisely. Example: "Fix Bug report https://github.com/amundsen-io/amundsen/issues/848"

In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

[x] PR includes a summary of changes, including screenshots of any UI changes.
[x] PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
[x] In case of new functionality, my PR adds documentation that describes how to use it.

All the public python functions and the classes in the PR contain docstrings that explain what it does

[x] PR passes all tests documented in the [developer guide]

@NiravLangaliya can you re-push to fix the DCO issue? Then we can merge

ping @NiravLangaliya

ping @NiravLangaliya

Please fix the DCO issue and re-open -- I'll merge


Negative LoRA values overly change image - How to make bipolar LoRA?

Hello, I am having some issues when training a bipolar LoRA slider - a LoRA that supports negative and positive values. My goal is for negative values to yield to opposite of positive values, i.e. for a height slider, negative values make the subject shorter while positive values make the subject taller.
Training the slider, the positive values work well and as expected, but using negative values will desaturate and change the style of the image quite a bit, and often times it doesn't seem to encapsulate the desired LoRA effect well. I am training on the base v6 model of PonyXL.
Below is the configuration for a height slider.
Config-xl.yml:
pretrained_model:
  name_or_path: "ponyDiffusionV6XL.safetensors" # you can also use .ckpt or .safetensors models
  v2: false # true if model is v2.x
  v_pred: false # true if model uses v-prediction
network:
  type: "c3lier" # or "c3lier" or "lierla"
  rank: 4
  alpha: 1
  training_method: "noxattn"
train:
  precision: "bfloat16"
  noise_scheduler: "ddpm" # or "ddpm", "lms", "euler_a"
  iterations: 1000
  lr: 0.0002
  optimizer: "AdamW"
  lr_scheduler: "constant"
  max_denoising_steps: 60
save:
  name: "temp"
  path: "./models"
  per_steps: 250
  precision: "bfloat16"
logging:
  use_wandb: false
  verbose: false
other:
  use_xformers: true

Example prompts file:
- target: "1boy" # what word for erasing the positive concept from
  positive: "tall" # concept to erase
  unconditional: "short" # word to take the difference from the positive concept
  neutral: "1boy" # starting point for conditioning the target
  action: "enhance" # erase or enhance
  guidance_scale: 4
  rank: 4
  resolution: 512
  dynamic_resolution: false
  batch_size: 1
- target: "1boy" # what word for erasing the positive concept from
  positive: "very tall" # concept to erase
  unconditional: "very short" # word to take the difference from the positive concept
  neutral: "1boy" # starting point for conditioning the target
  action: "enhance" # erase or enhance
  guidance_scale: 4
  rank: 4
  resolution: 512
  dynamic_resolution: false
  batch_size: 1
- target: "1girl" # what word for erasing the positive concept from
  positive: "very tall" # concept to erase
  unconditional: "very short" # word to take the difference from the positive concept
  neutral: "1girl" # starting point for conditioning the target
  action: "enhance" # erase or enhance
  guidance_scale: 4
  rank: 4
  resolution: 512
  dynamic_resolution: false
  batch_size: 1

Command-line argument to start training:
python trainscripts/textsliders/train_lora_xl.py --attributes "1girl, 1boy, height" --name 'height-slider-v10' --rank 4 --alpha 1 --config_file "trainscripts/textsliders/data/config-xl.yaml"

I have played around quite a bit with the prompting to no avail. Is it better to train two slider LoRAs, and then carefully combine them to make a bipolar LoRA slider? Am I just messing something up with the config, or is training on non-base SDXL models not supported? Thanks.

hey @ItsAuver
As you tell, the positive seems to work. So I am guessing it's not an issue with the model checkpoint. Some suggestions that I can give from first glance:

You should use --attributes parameter in your training command for attributes you DO NOT want to change. For instance the race or gender of a person. Given you explicitly mentioned the 1boy and 1girl in your prompts, you can either remove it or add attributes that you do not want to change like race attributes or background attributes (eg. "colorful background, bokeh background, white, black, asian")

in your prompts file I would make a slight change:

- target: "1boy" # what word for erasing the positive concept from
  positive: "1boy, very tall" # concept to erase
  unconditional: "1boy, very short" # word to take the difference from the positive concept
  neutral: "1boy" # starting point for conditioning the target
  action: "enhance" # erase or enhance
  guidance_scale: 4
  rank: 4
  resolution: 512
  dynamic_resolution: false
  batch_size: 1

I would add the target prompt to the positive and unconditional too.
Let me know if these hacks work. All the best!


Update README.md

spelling mistake

spelling mistake

you have not upload full project what we do with this incomplete project


Way to disable tracing on specific routes?

I currently want tracing to be enabled globally except for some very specific routes that are extremely noisy. If I was to do this given the current configuration option, I would have to enumerate all routes and configure this plugin specifically for each route except for the one route that I want zipkin disabled on. This ends up being a ton of configuration which is hard to manage.
Instead, I would like to configure zipkin globally, but add in a small config list of matched routes to disable tracing on.
Example with db-less config:
services:
  - name: foo
    url: foo.com
    routes:
      - /foo
  - name: bar
    url: bar.com
    routes:
      - /bar
...etc...
plugins:
  - name: zipkin
    config:
      http_endpoint: http://zipkin:9411/api/v2/spans
      sample_ratio: 1
      ignore_routes:
        - path: /foo/health
          method: GET

Dear contributor,
We're closing this issue as there hasn't been any update to it for a long time. If the issue is still relevant in the latest version, please feel free to reopen it. We're more than happy to revisit it again.
Please have a look at our pledge to the community for more information.
Sincerely,
Kong Gateway Team


Prevent removing unexpected .a files in case of re-using the Dockerfile to build custom image

Hello, I'd like to propose a slight change for cleaning up *.a which is defined here, so that it doesn't delete unexpected files.
https://github.com/docker-library/python/blob/0047f00c0967161e731c9bab7d50fd95c7c09d46/3.10/slim-buster/Dockerfile#L92
Here is my background:
The current Dockerfile for the dockerhub Python image itself works pretty fine; it doesn't remove anything unexpected, even without my proposal.
However, since this official Dockerfile is so useful, clean and general, we often re-use (borrow) its content to make our own images on top of another image, typically by just rewriting the FROM section to something other.
For example, which is my actual case, we build a Python-ready image on top of nvidia/cuda:11.6.0-xxx. The issue here is that removing all the *.a under /usr/local as written in the current Dockerfile also destroys CUDA libraries. This combination is just an example but I think the same thing can happen in other patterns.
Looking at the actual files cleaned up here while building Python 3.10 images for slim-buster and alpine3.15, I found out /usr/local/lib/python3.10/config-3.10-x86_64-linux-gnu/libpython3.10.a is the only *.a file removed. So I think modifying the rule for the find command from '*.a' to 'libpython*.a' can prevent unexpectedly removing .a files of other libraries in case modifying FROM section, while it doesn't change anything on the image for dockerhub.

Interesting!  I'm not opposed but I'd definitely like to do a more thorough check of other versions and variants just to be sure. :sweat_smile: :see_no_evil:
(It'd be even cooler if we could find some way to convince Python not to build or install this file in the first place, but I'm not going to wish too hard for that one :smile:)

Thanks for the comment!

more thorough check of other versions and variants just to be sure

Sure! Then maybe I can proceed to run update.sh so you and anyone can test it.

Nice, that was more successful than I expected. :sweat_smile: :metal:


Doesn't Work in an Angular ngRepeat

It appears that if you attempt to use the switch inside an ngRepeat, it fails to toggle.  I have a Plunker of a very basic setup that recreates the issue:
http://plnkr.co/edit/5s3ZYWAQyih0tWKyXPTZ/
In the Plunker, if you remove the ng-repeat="Data in TestData from line 27, it all works as expected.

If you initialize bootstrap-switch on ready instead of immediately, it also appears to work properly.
    $(function(){
      $(".switch").bootstrapSwitch();
    });


Fix vector_field for Lindbladian calculation

Not sure whether this type of optimization is taken care of by the JIT. Better safe than sorry.

@gautierronan FYI this was not JIT-optimized, comparing cat CNOT simulations with each modes truncated at 40 (timing hand-made over multiple runs):

before: 12.8s
after: 10.7s

Great low-hanging fruit!! 🚀


Nice to meet you and here is the first feedback

Hello Triet @anticafe,
Nice to e-meet you 👋 and thanks for your effort on this submission. My name is Lucas - an Android Developer at Nimble and I would like to conduct the feedback via Issue tickets regarding your performance here.
Keep in mind, this is a bidirectional process, so we would love to hear back from you as well, so please don’t hesitate to raise your question/correction in case we missed anything.
If we’re aligned on any issue here and you’d like to correct them, please go ahead and open a Pull Request per fix, merge it when you feel confident and we will follow up those fixes 😇
I hope you find the process enjoyable. Good luck and happy coding 🤘
Here is our first feedback for the app:

List item reset after went back to home screen from the detail screen
Scroll to the last page got No result found for the first time and Unknown error for next times (scroll back 1 item and next again)? They were not consistent and should be revised.
Suppress the warning without proper configuration: android:allowBackup (https://medium.com/mindorks/auto-backup-in-android-dont-do-it-unless-you-know-it-cc9fe2f9176b)
No proguard

Hello Lucas @luongvo, nice to meet you.
Thanks for your feedback. Here is my comment:

List item reset after went back to home screen from the detail screen

I try to reproduce it by: load the first page, then scroll to end of the first page so that the second page to load more. After that, touch on "Take the survey" button to open the detail screen. Then press BACK key to back to home screen. However, I still see the list item there (with 2 pages data as well as previous indicator index).
I think there should be a special step to reproduce that issue - could you please give me more detail?

And about AllowBackup

Suppress the warning without proper configuration: android:allowBackup

You're right. I should disable autoBackup instead. Fixed in #25

@anticafe here it is, commit id 4c79d8fc64347199a6648c19d576425d2ab8b694

Well I see. According to @luongvo 's comment, you take the revision based on submission date, thus this issue still there. However, after that I make a commit 3fc0a93706 in PR #16 to fix it. So, this issue is now fixed.

Proguard rule is added via commit aec095d.

All issues has been fixed. Please help me to verify. Your comments are welcome 👍 .

Proguard rule is added via commit aec095d.

Proguard rule is added via commit aec095d.

So, we have to apply proguard for debug build type too? That doesn't make sense lol

@luongvo Proguard is only applied when minifyEnabled true. In my case, it is set to false for debug build instead, so I believe it doesn't take effect.

https://github.com/anticafe/survey_app/blob/2707bcfa29258f8f1a5e38d0611a464260a7c0d9/app/build.gradle#L60

In Github, there are many repositories has minifyEnabled false for debug build too. I guess they have the same reason as mine: they want to test proguard rule but doesn't have release configuration (keystore) available yet, thus use debug build temporarily.

@anticafe but you're pointing to the same proguard config for both build type. Nvm, it doesn't effect at least. 👍


test: use registry.k8s.io/e2e-test-images/busybox:1.29-4 for e2e

Signed-off-by: Anish Ramasekar<EMAIL_ADDRESS>
Reason for Change:

use registry.k8s.io/e2e-test-images/busybox:1.29-4 for e2e

k8s.gcr.io/e2e-test-images/busybox:1.29 doesn't work with Windows 2022

Requirements

[ ] squashed commits
[ ] included documentation
[ ] added unit tests and e2e tests (if applicable).

Issue Fixed:

Does this change contain code from or inspired by another project?

[ ] Yes
[ ] No

If "Yes," did you notify that project's maintainers and provide attribution?
Special Notes for Reviewers:

/azp run pr-e2e-azure


feat: as a user I want to auto register my new repositories

When a new repository is created, it should be possible to have it be automatically added to mediator without the need to run minder repo register
This also relates to https://github.com/stacklok/mediator/issues/575

Closing in favor of the epic.


lint: use flake8-quotes plugin

See https://github.com/zheller/flake8-quotes/

Codecov Report

Merging #64 (b08bee9) into master (049201b) will not change coverage.
The diff coverage is 50.00%.

@@           Coverage Diff           @@
##           master      #64   +/-   ##
=======================================
  Coverage   32.23%   32.23%
=======================================
  Files          67       67
  Lines        4191     4191
=======================================
  Hits         1351     1351
  Misses       2840     2840

Impacted Files
Coverage Δ

lona/server.py
20.77% <0.00%> (ø)

lona/html/attribute_dict.py
55.03% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 049201b...b08bee9. Read the comment docs.

@maratori: Great idea! :)


Fix Example with-firebase-hosting-and-typescript

Try to deploy this example, I have https://github.com/zeit/next.js/issues/5912 .
ref: https://github.com/zeit/next.js/pull/6331

Stats from current PR

Default Server Mode

General

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

buildDuration
11.5s
11.4s
-82ms

nodeModulesSize
52.6 MB
52.6 MB
✓

Client Bundles (main, webpack, commons)

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

main-HASH.js gzip
5.13 kB
5.13 kB
✓

webpack-HASH.js gzip
746 B
746 B
✓

4952ddcd88e7..54d3.js gzip
4.68 kB
4.68 kB
✓

commons.HASH.js gzip
4.06 kB
4.06 kB
✓

de003c3a9d30..e781.js gzip
13.8 kB
13.8 kB
✓

framework.HASH.js gzip
39.1 kB
39.1 kB
✓

Overall change
67.5 kB
67.5 kB
✓

Client Bundles (main, webpack, commons) Modern

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

main-HASH.module.js gzip
4.13 kB
4.13 kB
✓

webpack-HASH..dule.js gzip
746 B
746 B
✓

4952ddcd88e7..dule.js gzip
5.56 kB
5.56 kB
✓

de003c3a9d30..dule.js gzip
12.5 kB
12.5 kB
✓

framework.HA..dule.js gzip
39.1 kB
39.1 kB
✓

Overall change
62.1 kB
62.1 kB
✓

Legacy Client Bundles (polyfills)

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

polyfills-HASH.js gzip
4.76 kB
4.76 kB
✓

Overall change
4.76 kB
4.76 kB
✓

Client Pages

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_app.js gzip
1.15 kB
1.15 kB
✓

_error.js gzip
4.07 kB
4.07 kB
✓

hooks.js gzip
779 B
779 B
✓

index.js gzip
222 B
222 B
✓

link.js gzip
2.89 kB
2.89 kB
✓

routerDirect.js gzip
283 B
283 B
✓

withRouter.js gzip
282 B
282 B
✓

Overall change
9.68 kB
9.68 kB
✓

Client Pages Modern

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_app.module.js gzip
576 B
576 B
✓

_error.module.js gzip
3.06 kB
3.06 kB
✓

hooks.module.js gzip
371 B
371 B
✓

index.module.js gzip
212 B
212 B
✓

link.module.js gzip
2.46 kB
2.46 kB
✓

routerDirect..dule.js gzip
273 B
273 B
✓

withRouter.m..dule.js gzip
272 B
272 B
✓

Overall change
7.22 kB
7.22 kB
✓

Client Build Manifests

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_buildManifest.js gzip
61 B
61 B
✓

_buildManife..dule.js gzip
61 B
61 B
✓

Overall change
122 B
122 B
✓

Rendered Page Sizes

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

index.html gzip
1.03 kB
1.03 kB
✓

link.html gzip
1.04 kB
1.04 kB
✓

withRouter.html gzip
1.03 kB
1.03 kB
✓

Overall change
3.11 kB
3.11 kB
✓

Serverless Mode

General

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

buildDuration
12.2s
12.1s
-28ms

nodeModulesSize
52.6 MB
52.6 MB
✓

Client Bundles (main, webpack, commons)

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

main-HASH.js gzip
5.13 kB
5.13 kB
✓

webpack-HASH.js gzip
746 B
746 B
✓

4952ddcd88e7..54d3.js gzip
4.68 kB
4.68 kB
✓

commons.HASH.js gzip
4.06 kB
4.06 kB
✓

de003c3a9d30..e781.js gzip
13.8 kB
13.8 kB
✓

framework.HASH.js gzip
39.1 kB
39.1 kB
✓

Overall change
67.5 kB
67.5 kB
✓

Client Bundles (main, webpack, commons) Modern

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

main-HASH.module.js gzip
4.13 kB
4.13 kB
✓

webpack-HASH..dule.js gzip
746 B
746 B
✓

4952ddcd88e7..dule.js gzip
5.56 kB
5.56 kB
✓

de003c3a9d30..dule.js gzip
12.5 kB
12.5 kB
✓

framework.HA..dule.js gzip
39.1 kB
39.1 kB
✓

Overall change
62.1 kB
62.1 kB
✓

Legacy Client Bundles (polyfills)

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

polyfills-HASH.js gzip
4.76 kB
4.76 kB
✓

Overall change
4.76 kB
4.76 kB
✓

Client Pages

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_app.js gzip
1.15 kB
1.15 kB
✓

_error.js gzip
4.07 kB
4.07 kB
✓

hooks.js gzip
779 B
779 B
✓

index.js gzip
222 B
222 B
✓

link.js gzip
2.89 kB
2.89 kB
✓

routerDirect.js gzip
283 B
283 B
✓

withRouter.js gzip
282 B
282 B
✓

Overall change
9.68 kB
9.68 kB
✓

Client Pages Modern

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_app.module.js gzip
576 B
576 B
✓

_error.module.js gzip
3.06 kB
3.06 kB
✓

hooks.module.js gzip
371 B
371 B
✓

index.module.js gzip
212 B
212 B
✓

link.module.js gzip
2.46 kB
2.46 kB
✓

routerDirect..dule.js gzip
273 B
273 B
✓

withRouter.m..dule.js gzip
272 B
272 B
✓

Overall change
7.22 kB
7.22 kB
✓

Client Build Manifests

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_buildManifest.js gzip
61 B
61 B
✓

_buildManife..dule.js gzip
61 B
61 B
✓

Overall change
122 B
122 B
✓

Serverless bundles

zeit/next.js canary
fnwiya/next.js fix-example-with-firebase-hosting-and-typescript
Change

_error.js gzip
46.6 kB
46.6 kB
✓

404.html gzip
1.45 kB
1.45 kB
✓

hooks.html gzip
1.08 kB
1.08 kB
✓

index.js gzip
46.7 kB
46.7 kB
✓

link.js gzip
72.6 kB
72.6 kB
✓

routerDirect.js gzip
70.8 kB
70.8 kB
✓

withRouter.js gzip
70.6 kB
70.6 kB
✓

Overall change
310 kB
310 kB
✓

Commit: 92ca46644ae916598e9bdbc5f34769c69aac0dfd

@afuggini @oBusk Can you review this change 🙏

While the change I suggested back then still looks sensible, there has been more changes to the example since I've looked at it, and npm run deploy won't work for me, with or without this change.
There's seems to be linting added but the code is not following that linting... but even after fixing that slight lint-issue. I'm getting;
Error: > Couldn't find a `pages` directory. Please create one under the project root
    at findPagesDir (C:\Users\oBusk\Desktop\temp\with-firebase-hosting-and-typescript-app\dist\functions\node_modules\next\dist\lib\find-pages-dir.js:3:170)
    at new DevServer (C:\Users\oBusk\Desktop\temp\with-firebase-hosting-and-typescript-app\dist\functions\node_modules\next\dist\server\next-dev-server.js:1:3748)
    at Object.createServer [as default] (C:\Users\oBusk\Desktop\temp\with-firebase-hosting-and-typescript-app\dist\functions\node_modules\next\dist\server\next.js:2:105)
    at Object.<anonymous> (C:\Users\oBusk\Desktop\temp\with-firebase-hosting-and-typescript-app\dist\functions\index.js:16:27)
    at Module._compile (internal/modules/cjs/loader.js:1151:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14)
    at Module.require (internal/modules/cjs/loader.js:1040:19)
    at require (internal/modules/cjs/helpers.js:72:18)

Getting same issue both with master and this PR
@fnwiya Is this the only change needed for your npm run deploy to go through?

Closing as another user confirmed this does not fix the example for them.
Thanks for the PR!

I just encountered this issue and had to run with the environment valueNODE_ENV=production firebase deploy. Both this PR and the merged fix in with-firebase-hosting( #6331) can fix #5912


chore(main): release 1.252.0

When this PR is merged, a new tagged release will be created with the notes below.
1.252.0 (2024-02-12)
Features

add types mappings for esm distribution (#887) (811ed41)
align browser reported uncaught syntax errors (#881) (d4a0f30)
Capture Internal Metrics for Session Replay Configurations (#879) (f60e7f1)
Create more granular metrics about stylesheet fix success (#882) (697f13e)
Report config changes away from default state for UX improvement (#885) (aa19a9c)

Bug Fixes

Add safe logic to snapshots (#884) (1fcdd8d)
Fix adblock memory leak (#877) (695415b)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/newrelic/newrelic-browser-agent/releases/tag/v1.252.0 :sunflower:


GH-40428: [Python][CI] Fix dataset partition filter tests with pandas nightly

Rationale for this change
From debugging the failure, it seems this is due to pandas changing a filter operation to sometimes preserve a RangeIndex now instead of returning an Integer64Index. And the conversion to Arrow changes based on that (RangeIndex is metadata only by default, integer index becomes a column)
Therefore making the tests more robust to ensure there is always at least one non-partition column in the DataFrame, so it doesn't depend on the index whether the result is empty or not.

GitHub Issue: #40428

@github-actions crossbow submit test-conda-python-3.10-pandas-nightly

@github-actions crossbow submit test-conda-python-3.10-pandas-nightly


Feature/bliss 625/update sample app

Changelog

Added Support for latest react native
Added new brandings

:tada: This PR is included in version 0.12.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Any one wrapped this as a Home Assistant Add-On?

Has any wrapped this up as a docker file that can be used with Home Assistant?  I might have a go at trying to do this, but don't have much experience, however if someone has already done it, then great!!!

or an add-on via HACS for some that are running HA OS

Workaround: Using withings-sync with AppDaemon in Home Assistant
I put together this workaround to integrate withings-sync with Home Assistant using AppDaemon. It's the result of trial and error, so it might not be perfect. Here's how to set it up:

Steps:

Install AppDaemon Add-On:

Install the AppDaemon add-on from the Home Assistant Add-on Store.
Enable "Start on boot" and start the add-on.

Add Required Python Packages:

In the AppDaemon configuration, add these to the python_packages section:

withings-sync
pexpect

Save and restart AppDaemon.

Create the Script:

In your AppDaemon apps folder (e.g., data/addon_configs/a0d7b954_appdaemon/apps/), create a file withings_sync_runner.py with this code:

import appdaemon.plugins.hass.hassapi as hass
import subprocess
import time
import pexpect
import os

def read_and_clear_file(filepath):
    """Reads and clears the content of the file, returns the content."""
    with open(filepath, 'r', encoding='utf-8') as file:
        content = file.read().strip()
    if content:
        # Clear the file content after reading
        with open(filepath, 'w', encoding='utf-8') as file:
            file.write('')
    return content

class WithingsSyncRunner(hass.Hass):

    def initialize(self):
        self.listen_event(self.run_sync_event, "WITHINGS_SYNC_START")
        self.listen_event(self.run_test, "WITHINGS_SYNC_TEST")

    def ha_notify(self, msg):
        self.call_service("notify/pavel_notification_group", title="Withings Sync", message=msg)

    def run_test(self, event_name, data, kwargs):
        self.ha_notify("test")

    def log_error(self, message):
        """Logs error messages using `ascii_encode=False`."""
        self.log(message, ascii_encode=False)
        self.ha_notify(message)

    def run_sync_event(self, event_name, data, kwargs):
        os.environ["WITHINGS_USER"] = "/config/apps/withings_user.json"
        os.environ["GARMIN_SESSION"] = "/config/apps/garmin_session"
        self.log("Starting sync...")
        child = pexpect.spawn('withings-sync --garmin-username=USERNAME --garmin-password=PASSWORD')

        prompts = ['Enter MFA code', 'Token :', 'MFA code:']

        try:
            while True:
                index = child.expect(prompts + [pexpect.EOF, '\n'], timeout=-1)
                if index < len(prompts):  # Prompt detected
                    token_msg = f"Prompt detected: {prompts[index]}"
                    self.ha_notify(token_msg)
                    self.log(token_msg, ascii_encode=False)
                    token = None
                    waiting_time = 0
                    max_waiting_time = 300  # 5 minutes
                    self.log("Waiting for code...", ascii_encode=False)
                    while not token and waiting_time < max_waiting_time:
                        token = read_and_clear_file('/config/apps/input.txt')
                        time.sleep(5)  # Waiting for the file to be updated
                        waiting_time += 5
                    if not token:
                        self.log_error("Time to enter the code has expired.")
                        break
                    self.log(f"Code received: {token}", ascii_encode=False)
                    child.sendline(token)
                elif index == len(prompts):  # EOF - End of file (process finished)
                    self.log("Process finished", ascii_encode=False)
                    break
                else:
                    # Output the response if no specific prompt is detected
                    print(child.before.decode('utf-8', errors='ignore').strip())
        except pexpect.exceptions.TIMEOUT:
            self.log_error("Timeout occurred.")
            self.log(f"Output before timeout: {child.before.decode('utf-8', errors='ignore').strip()}", ascii_encode=False)
        except pexpect.exceptions.EOF:
            self.log_error("Process ended prematurely.")
            self.log(f"Output before EOF: {child.before.decode('utf-8', errors='ignore').strip()}", ascii_encode=False)
        except Exception as e:
            self.log_error(f"Unexpected error: {e}")

Replace USERNAME and PASSWORD with your Garmin credentials.
Replace notify/pavel_notification_group with your device/group for MFA and error notifications.

Trigger Sync:

Use a Home Assistant automation or script to fire the WITHINGS_SYNC_START event.
Example for automation actions:
   actions:
     - event: WITHINGS_SYNC_START
       event_data: {}

Notes:

MFA Handling: When prompted, enter the MFA code into input.txt (you may need to create the file first). The script will pick it up automatically.
Security: Credentials are stored in plain text, so use with caution.

(Disclaimer: I've never worked with Python before, so this is just a workaround I managed to put together. Any suggestions or improvements are more than welcome!)


Fixed Recursive References

Users have been mentioning that recursive references are still a problem.
Setting cell a1 to the value '=A1' will crash the application.
This fixes that, by scanning for recursive references prior to attempting a render.

This already works. Demo isn't updated.

It's not working if you follow the dev process
git clone https://github.com/myliang/x-spreadsheet.git
cd x-spreadsheet
npm install
npm run dev
Should it be working on local dev servers?

This already works. Demo isn't updated.

Kkk

@Levirclark thx, good code


Add basic CI configuration for Linux, macOS and Android

Adds a basic CI configuration using GitHub Actions which builds sjpeg:

on Linux using CMake and GCC 10
on macOS using CMake and AppleClang 12
for Android using CMake and the Android NDK

This configuration uses GitHub Actions, which needs to be activated here. An example build from my fork can be seen here.
For Windows there is already an AppVeyor configuration, which was added in #110.

@jzern @madebr @skal65535 I'm doing some cleanup of old, open PRs. If you would like to get these changes in, please leave a reply or review.
In two weeks I will close this PR if no reaction received.

Thanks for pinging this, I guess it got missed.


BigQuery documentation is misleading

Describe the bug
Our documentation currently says:
name: my_bigquery_project
connection:
    type: bigquery
    account_info: <PATH TO YOUR BIGQUERY ACCOUNT INFO JSON FILE>
    dataset: sodasql
...

but our code expects

account_info_json, and
that property should have JSON object, not the location of it

OS:
Python Version: *
Soda SQL Version: main
Warehouse Type: BigQuery

closed via https://github.com/sodadata/soda-sql/commit/e1e390373139bf66dcda08bdd96fa4be66d1ca4c

@Antoninj this helps but it doesn't actually solve the issue's problem. Your diff should look like this
name: my_bigquery_project
connection:
    type: bigquery
    - account_info: <PATH TO YOUR BIGQUERY ACCOUNT INFO JSON FILE>
    + account_info_json: <YOUR BIGQUERY SERVICE ACCOUNT INFO JSON FILE>
    dataset: sodasql

the key needs to be renamed, right now when I run with a warehouse.yml that uses the account_info key instead of account_info_json I get this rather unhelpful error messages:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sodasql/cli/cli.py", line 181, in analyze
    warehouse_yml_parser = WarehouseYmlParser(warehouse_yml_dict, warehouse_file)
  File "/usr/local/lib/python3.8/site-packages/sodasql/scan/warehouse_yml_parser.py", line 64, in __init__
    self.warehouse_yml.dialect = Dialect.create(self)
  File "/usr/local/lib/python3.8/site-packages/sodasql/scan/dialect.py", line 61, in create
    return BigQueryDialect(parser)
  File "/usr/local/lib/python3.8/site-packages/sodasql/dialects/bigquery_dialect.py", line 34, in __init__
    self.account_info_dict = self.__parse_json_credential('account_info_json', parser)
  File "/usr/local/lib/python3.8/site-packages/sodasql/dialects/bigquery_dialect.py", line 92, in __parse_json_credential
    return json.loads(parser.get_credential(credential_name))
  File "/usr/local/lib/python3.8/json/__init__.py", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

FWIW I the wording "YOUR BIGQUERY SERVICE ACCOUNT INFO JSON FILE" is a bit ambiguous as to if its the inline json or a path to a file. Maybe an example would be better:
name: my_bigquery_project
connection:
    type: bigquery
    # YOUR BIGQUERY SERVICE ACCOUNT INFO JSON FILE
    account_info_json: >
      {
        "type": "service_account",
        "project_id": "...",
        "private_key_id": "...",
        "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
        "client_email"<EMAIL_ADDRESS>        "client_id": "...",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/..."
      }
    dataset: dataset

@abuckenheimer there is somewhere another issue for refactoring the file references with include and read.  Would be good to link the 2 issues.  If someone has time to find it and make the link...

Thanks @abuckenheimer for pointing this out! I just updated the documentation following your suggestion.

closed via https://github.com/sodadata/soda-sql/commit/33cab9b551110a905157bad31223c4e0222bad0e

Following the documentation still does not work for me. I have tried the following:
name: soda_sql_tutorial
connection:
    type: bigquery
    account_info_json: env_var(BIG_QUERY_ACCESS)
    auth_scopes:
    - https://www.googleapis.com/auth/bigquery
    dataset: dbt_customer_360

soda_account:
  host: cloud.soda.io
  api_key_id: env_var(API_PUBLIC)
  api_key_secret: env_var(API_PRIVATE)

Which throws the following exception:
 2.1.0b22
  | Analyzing warehouse.yml ...
  | Dialect initiated from the create command, cred is None.
  | Exception: Account_info_json is not provided
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/sodasql/cli/cli.py", line 248, in analyze
    warehouse = Warehouse(warehouse_yml_parser.warehouse_yml)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/sodasql/scan/warehouse.py", line 26, in __init__
    self.connection = self.dialect.create_connection()
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/sodasql/dialects/bigquery_dialect.py", line 74, in create_connection
    self.try_to_raise_soda_sql_exception(e)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/sodasql/scan/dialect.py", line 473, in try_to_raise_soda_sql_exception
    raise exception
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/sodasql/dialects/bigquery_dialect.py", line 66, in create_connection
    raise Exception("Account_info_json is not provided")
Exception: Account_info_json is not provided
  | If you think this is a bug in Soda SQL, please open an issue athttps://github.com/sodadata/soda-sql/issues/new/choose
  | Starting new HTTPS connection (1): collect.dev.sodadata.io:443
  | https://collect.dev.sodadata.io:443 "POST /v1/traces HTTP/1.1" 200 0

I get the same exception when proving a path to the json file. The only thing that works is when including the credentials directly in the scan.yml file.

I think if you change:
   account_info_json: env_var(BIG_QUERY_ACCESS)
to:
   account_info_path: env_var(BIG_QUERY_ACCESS)
it might work. See - (Bigquery Datasource Configurations)[https://docs.soda.io/soda/warehouse_types.html#gcp-big-query]


Please invite me to the GitHub Community Organization

Name
Mars Bryant
Discord Username (if applicable)
Alina1120#5512
Additional Context
want join your team.

It's great having you contribute to this project
If you would like to continue contributing to open source and would like to do it with an awesome inclusive community, you should join our Discord Server and our GitHub Organisation - we help and encourage each other to contribute to open source little and often 🤓 . Any questions let us know.

Invitation sent for the GitHub Organisation. Welcome to the community 🎉#
Don't forget after accepting to make it public so it appears on your GitHub profile for everyone else to see, you can do this by finding your name in the GitHub organisation list and change the dropdown to public https://github.com/orgs/jaidevstudio/people
Like this

Tips for practising:
Remember contributing to open source is not just about code, its about collaboration, communication and adding value
I hope that helps

Please check your email or organisation page 🙏


Allow CubitPy to run with Cubit Coreform

@m-frey can you try if this works on your Linux machine together with Cubit Coreform? Some tests will still fail, I get the following pytest result:
============================ short test summary info =============================
FAILED testing.py::test_element_types_tet - assert False
FAILED testing.py::test_extrude_mesh_function - assert 0.6917559630511103 == 0.6934429579015018 ± 6.9e-11
FAILED testing.py::test_extrude_mesh_function_average_normals_for_cylinder_and_sphere - assert 0.02668549643643842 == 0.02675360258727785 ± 2.7e-12
FAILED testing.py::test_fluid_functionality - assert False
FAILED testing.py::test_point_coupling - assert False
FAILED testing.py::test_get_id_functions - assert [2] == [1, 2]
FAILED testing.py::test_serialize_nested_lists - TypeError: 'NoneType' object is not callable
FAILED testing.py::test_display_in_cubit - assert 'open "/tmp/c...e On\ndisplay' == 'open "/tmp/c...e On\ndisplay'
FAILED testing.py::test_extrude_artery_of_aneurysm - assert 13.570135865871498 == 13.614146346307278 ± 1.4e-04
FAILED testing_tutorial.py::TestTutorial::test_tutorial - assert False
=================== 10 failed, 27 passed, 1 warning in 38.40s ====================

Can you also try if the display_in_cubit method works?

../../miniforge3/envs/meshpy/lib/python3.12/subprocess.py:1955: FileNotFoundError
================================================================================================================================================== short test summary info ==================================================================================================================================================
FAILED testing.py::test_element_types_tet - FileNotFoundError: [Errno 2] No such file or directory: 'code'
FAILED testing.py::test_extrude_mesh_function - assert 0.6917559630511103 == 0.6934429579015018 ± 6.9e-11
FAILED testing.py::test_extrude_mesh_function_average_normals_for_cylinder_and_sphere - assert 0.02668549643643842 == 0.026753602587277842 ± 2.7e-12
FAILED testing.py::test_fluid_functionality - FileNotFoundError: [Errno 2] No such file or directory: 'code'
FAILED testing.py::test_point_coupling - FileNotFoundError: [Errno 2] No such file or directory: 'code'
FAILED testing.py::test_get_id_functions - assert [2] == [1, 2]
FAILED testing.py::test_serialize_nested_lists - TypeError: 'NoneType' object is not callable
FAILED testing.py::test_display_in_cubit - assert 'open "/tmp/c...e On\ndisplay' == 'open "/tmp/c...e On\ndisplay'
FAILED testing.py::test_extrude_artery_of_aneurysm - assert 13.570135865871498 == 13.613844185308892 ± 1.4e-04
FAILED testing_tutorial.py::TestTutorial::test_tutorial - FileNotFoundError: [Errno 2] No such file or directory: 'code'
============================================================================================================================================== 10 failed, 27 passed in 21.31s ===============================================================================================================================================
(meshpy) a11bmofr@mergen ~/cubitpy/tests ((HEAD detached at ivo-fork/coreform))$

Thanks for the feedback @m-frey. Let's merge this, then CubitPy is ready for Coreform Cubit, we can have a look at the failing tests under Coreform Cubit in the future.


Config file preview doesn't contain longer strings

As per the screenshot, there's an X-axis overflow, but the background doesn't seem to support that.
I think either have a hard-wrap, or make the background extend (the latter probably being a slightly nicer experience).

Nice spot - have updated to add a wrap. Now deployed


az storage blob directory delete - directory is not recognized

Command used:
az storage blob directory delete -h
Received output:
'directory' is misspelled or not recognized by the system.

TRY THIS:
https://aka.ms/cli_ref
Read more about the command in reference docs

Expected output
Help for the specified command.
Environment:
azure-cli                         2.27.2 *
core                              2.27.2 *
telemetry                          1.0.6
Extensions:
datafactory                        0.5.0
Python: 3.8.9.
OS: Windows 10, Also tried on ubuntu 18 machine.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 978a095c-3ead-282f-304f-fb627dad53a4
Version Independent ID: 43fb2b4e-3f87-f3b8-2e85-ed5812a69ffc
Content: az storage blob directory
Content Source: latest/docs-ref-autogen/storage/blob/directory.yml
Service: storage
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

@romanzdk Please help with the link of document on which above document feedback request is created.
Under this repository we want to make sure any issue is linked to a doc so we can help others that may be having the same issue.

@romanzdk Are you following this doc? Could you try with complete command and see if it works?

az storage blob directory delete -c MyContainer -d MyDirectoryPath --account-name MyStorageAccount

.

@romanzdk Are you following this doc? Could you try with complete command and see if it works?

az storage blob directory delete -c MyContainer -d MyDirectoryPath --account-name MyStorageAccount

Same result

@romanzdk I would suggest you upgrade CLI to latest version and try the command.
I just tried the command and it is showing the desired results. Please see below.

It shows that the command is implicitly deprecated. You need to use 'az storage fs directory'

This command is implicitly deprecated because command group 'storage blob directory' is  deprecated and will be removed in a future release. Use 'az storage fs directory'  instead.

@rloutlaw Could you please look into this feedback and update the doc appropriately with new command list.

@romanzdk  Do let us know if you are still blocked.

I just tried to install az CLI on other machine and when running the command for the first time it asks for storage-preview extension and then it works. I guess that might be the reason it does not work on the first machine as it is an ubuntu docker container without access to the internet. Will try to include the extension to the image itself and try again. Will let you know.

azure-cli                         2.31.0
core                              2.31.0
telemetry                          1.0.6

Python location '/opt/az/bin/python3'
Extensions directory '/home/profile-name/.azure/cliextensions'

Python (Linux) 3.6.10 (default, Dec  3 2021, 05:08:37)
[GCC 7.5.0]

OS: Ubuntu 18.04
command: az storage blob directory delete -c containerName-d folder-1/folder2 --account-name "accountName" --auth-mode login --recursive
output: ERROR: 'directory' is misspelled or not recognized by the system.

Adding @SumanthMarigowda for further insights.


fix(deps): Phoenix 1.5 compatibility

I think this isn't required for Phoenix 1.5 compatibility? I'm running Phoenix 1.5.7, MasterProxy 0.1.3 and Plug.Cowboy 2.4.1, seemingly without dependency or runtime issues.

I think this isn't required for Phoenix 1.5 compatibility? I'm running Phoenix 1.5.7, MasterProxy 0.1.3 and Plug.Cowboy 2.4.1, seemingly without dependency or runtime issues.

Closing because this is stale and isn't needed for Phoenix 1.5 compatibility.


error TS2304: Cannot find name 'module'.

Hi, I download the latest template from the master branch and when I run
npm run build:prod

I get
><EMAIL_ADDRESS>build:webpack C:\Users\rodolfo.ortiz\Documents\blitz-registration\tmp\aspnetcore-angular2-universal-master
> webpack --progress --color "--env.prod"

[0] Hash: db0ea034fb8a47ea29fa
Version: webpack 4.20.2
Child
    Hash: db0ea034fb8a47ea29fa
    Time: 6336ms
    Built at: 2018-09-27 08:38:33
     1 asset
    Entrypoint main-client = main-client.js

    ERROR in ClientApp/boot.browser.ts(8,5): error TS2304: Cannot find name 'module'.
    ClientApp/boot.browser.ts(9,3): error TS2304: Cannot find name 'module'.
    ClientApp/boot.browser.ts(10,3): error TS2304: Cannot find name 'module'.

npm run build:prod works fine. I've tried several different ways to fix this but none seem to work.

I forgot to add, one thing I did do was updating webpack-cli to 3.1.1 because I was getting an error cannot read property 'properties' of undefined. Upgrading to 3.1.1 got rid of the undefined error.

Can you take a look and see if
https://github.com/MarkPieszak/aspnetcore-angular2-universal/pull/686
fixes this issue for you?

That fixed, it, thank you so much.


alliance: 1.0.5-0 in 'indigo/distribution.yaml' [bloom]

Increasing version of package(s) in repository alliance to 1.0.5-0:

upstream repository: https://github.com/adrianohrl/alliance.git
release repository: https://github.com/adrianohrl/alliance-release.git
distro file: indigo/distribution.yaml
bloom version: 0.6.1
previous version for package: null

alliance
* removed rqt_mrta run depend
* Contributors: adrianohrl

alliance_msgs

No changes

rqt_alliance

No changes

Thanks for your attention, @dirk-thomas, and sorry for any inconvenience.

sorry for any inconvenience.

No problem :+1:


syscall: SysProcAttr{ NoInheritHandles: true } broken in 1.17 on Windows

What version of Go are you using (go version)?
go version go1.17 windows/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\knutz\AppData\Local\go-build
set GOENV=C:\Users\knutz\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\knutz\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\knutz\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.17
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\code\fd\agent_svc\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\knutz\AppData\Local\Temp\go-build1048178528=/tmp/go-build -gno-record-gcc-switches

What did you do?

package main

import (
	"log"
	"os/exec"
	"syscall"
)

func main() {
	cmd := exec.Command("notepad.exe")
	cmd.SysProcAttr = &syscall.SysProcAttr{
		NoInheritHandles: true,
	}
	err := cmd.Start()
	log.Printf("err: %v", err)
}

What did you expect to see?
2021/08/28 11:59:56 err: <nil>
What did you see instead?
2021/08/28 12:00:02 err: fork/exec C:\Windows\system32\notepad.exe: The parameter is incorrect.
The problem
NoInheritHandles was added in 1.16:
https://github.com/golang/go/commit/1e3b535b6eb7f13eb6d903f7998c384a36e9bba8#diff-ec673c10a75fe2d2faa9c788e03823294b263c68cc3de50f4a0865a53e28f3a3
Between that commit and 1.17 there was a series of commits by @zx2c4:
https://github.com/golang/go/commits/master/src/syscall/exec_windows.go
I don't fully understand the reasoning behind the changes but they are documented here: https://github.com/golang/go/commit/2d760816ff30bea82f54682f3049cfb6c6027da7#diff-ec673c10a75fe2d2faa9c788e03823294b263c68cc3de50f4a0865a53e28f3a3
These changes are not compatible with NoInheritHandles because they seem to add handles explicitly on this line:
https://github.com/golang/go/blob/f4cd001b57d91d24843325578a7bbd24dfc628fc/src/syscall/exec_windows.go#L395
Windows doesn't accept adding handles + NoInheritHandles which I guess makes sense.
How to solve this?
I'm not sure what the new commits do by @zx2c4 or how they are expected to act when NoInheritHandles is true. I would assume that no manipulation of handles/adding handles should happen. You can toggle between a non-working/working state by changing this:
// Do not accidentally inherit more than these handles.
if len(fd) > 0 {
	err = updateProcThreadAttribute(si.ProcThreadAttributeList, 0, _PROC_THREAD_ATTRIBUTE_HANDLE_LIST, unsafe.Pointer(&fd[0]), uintptr(len(fd))*unsafe.Sizeof(fd[0]), nil, nil)
	if err != nil {
		return 0, 0, err
	}
}

to
// Do not accidentally inherit more than these handles.
if len(fd) > 0 && !sys.NoInheritHandles {
	err = updateProcThreadAttribute(si.ProcThreadAttributeList, 0, _PROC_THREAD_ATTRIBUTE_HANDLE_LIST, unsafe.Pointer(&fd[0]), uintptr(len(fd))*unsafe.Sizeof(fd[0]), nil, nil)
	if err != nil {
		return 0, 0, err
	}
}

in this file:
https://github.com/golang/go/blob/f4cd001b57d91d24843325578a7bbd24dfc628fc/src/syscall/exec_windows.go#L393
@zx2c4 - What do you think? Would it be possible for you to fix this? Otherwise I can submit a PR with the above quick fix but I'm not sure how that influences the rest.

I'll be back from travels in about a week and will fix it. Sorry for the regression.

@zx2c4 Could be, len(fd) seems never to be 0

@gopherbot, please backport to Go 1.17. This is a regression from Go 1.16.


Github API access token scopes

Hi,
what are the scopes one has to activate for a github API token to work with nixpkgs-review?
I get an error message with the scopes I picked:

urllib.error.HTTPError: HTTP Error 403: Forbidden

Good questions. I would say something like repo -> public_repo? I have not tried it thoroughly though.
Currently nixpkgs-review posts pull requests comments, approves pull requests and merge them. It also uses the token to get information from public repositories.

For posting pull request comments, public_repo is enough.

Does this answers the question?

It makes sense to add this information to the readme. Could you test if public_repo is enough for the approve and merge functionality? I can create a PR for the documentation, but I cannot test the merge part on nixpkgs.

According to the documentation, here is what the public_repo scope allows:

Limits access to  public repositories. That includes read/write access to code, commit  statuses, repository projects, collaborators, and deployment statuses  for public repositories and organizations. Also required for starring  public repositories.

My interpretation of "includes read/write access to code" is that it implies merging PR.
(although, I could not find anywhere -in their doc or the internet- any confirmation of that...)

I can confirm that repo:public_repo  is enough to post comment.
It would be good to have this information in the readme.

I can confirm that repo:public_repo  is enough to post comment.
It would be good to have this information in the readme.


error message when >16 threads available

When I run "mps" from the command line, I get the error message below. My Mac Pro reports 24 processors (12 cores + hyperthreading), which appears to be more than what mplayer will allow. Would you have MPlayerShell check the number of processors and make sure it is set to at most 16 threads?
The threads option must be <= 16: 24
Error parsing option on the command line: -lavdopts
MPlayer 1.1-4.2.1 (C) 2000-2012 MPlayer Team

any updates on this?
The threads option must be <= 16: 24
Error parsing option on the command line: -lavdopts
MPlayer 1.2-4.2.1 (C) 2000-2015 MPlayer Team


Reduce number of tries if wayback machine is overloaded

The wayback machine is currently overloaded.
Getting a The capture will start in ~3 hours, 40 minutes because our service is currently overloaded. You may close your browser window and the page will still be saved. message.
So captures will not trigger until ~4 hours from time of capture.
Unfortunately spn.sh just keeps trying and pausing and trying and pausing for quite a long time(until Job Failed), just waiting for a captured URL(that won't come), without moving on to the next URLs on the list.
Could you add a way for the script to not wait if this particular message is generated upon save by the wayback machine?
So instead of saying a Job Completed it would say Wayback Machine overloaded. Job Scheduled to be Completed in ~3 hours, 42 minutes or something, while the script for that job sleeps in the background for aforementioned time and moves onto the next URL?
You probably know of a better way to resolve this error.
Thanks for the script @overcast07 . It's simply excellent!

As an aside, they now seem to have reduced the number of parallel capture jobs from 8 to 6, so 429 errors galore.
curl -s -X GET -H 'Accept: application/json' -H 'Authorization: LOW S3accesskey:secret' https://web.archive.org/save/status/user

I'm not sure if this is a normal situation that the script needs to handle. Jason Scott has stated on Twitter,

Internet Archive is under attack. Someone is posting tens of thousands of posts, comments and other vectors and is crushing the system. We're working on it, sorry for the inconvenience.
While it would be possible to wait for 3 hours for all of the requests to be processed, I don't think the script would be particularly usable in this situation. anyway

Currently the script limits the number of child processes that can be spawned, both due to the number that you could realistically use under normal conditions, and inherent limits of how many processes your computer could run at the same time. To make the script make sense under these conditions, either you would be limited to about 60 captures every 4 hours or something ridiculous like that, or you would have to store job IDs and their corresponding wait times in text files, so that the API can be checked hours later when the job actually starts.
It would be theoretically possible to implement a solution where individual job IDs are stored in separate text files, but I would rather wait and see what happens, since I would really hope that this situation is temporary. If everyone's already waiting for 4 hours, making it easier to submit even more URLs to the backlog would just make the situation worse.

I've checked just now and the situation appears to have gone back to normal for the time being, since it didn't tell me the service was overloaded and the captures went through immediately.

@overcast07 I still see similar message when I try to capture frequently-visited websites, like GitHub and the Microsoft doc pages.
Sometimes the capture gets delayed by just 10 minutes, sometimes more.
I'd say if you get a message that the capture is delayed, just don't retry, it will only make things worse.
Could you do that?

One of the problems with SPN, especially when using it without authentication, is that the script has to get information by pattern matching the HTML source code of the response. Unfortunately, while there was previously support for handling delays, the format of the error message was changed without me realizing, and the matching doesn't work any more. So, before anything else gets fixed, I need to update that part of the script so that it correctly processes the amount of time to wait.

Furthermore, because of the the capture rate now being limited server-side, the -p parameter no longer makes sense to set to a low value like 8, since if the website isn't extremely fragile then what the server does will be enough to avoid issues. That part of the script will also have to be changed.

It can be done, it's not the same as the overload issue.
There's actually 3 different issues.

overloaded in general
too many requests about a website (by any number of users)
too many requests by the user

I know it can be done, but I need to know if the format of the message is similar to the format of the other messages, which are

The capture will start in ~[n] seconds because we are doing too many captures of [domain] right now. You may close your browser window and the page will still be saved.

The capture will start in ~[m] hours, [n] minutes because our service is currently overloaded. You may close your browser window and the page will still be saved.

Matching for "capture will start in ..." works for both of these cases, but I haven't been able to get the other type of error message yet. I'm adding a sleep command so that the script actually waits for however long the delay is before starting the captures, which it previously didn't do.

I've just managed to get this variant. I wonder how this happened lol

The capture will start in ~ because our service is currently overloaded. You may close your browser window and the page will still be saved.

From the https://web.archive.org/save page, as an anonymous user, I got this one.

Sorry
You have already reached the limit of active Save Page Now sessions. Please wait for a minute and then try again.

That's perfectly normal and not a problem. The script will just retry the submitted URL until it works.
lock.txt exists specifically in order to handle this particular message: the file is created by one child process, other child processes don't attempt to retry URL submissions once the file is created (a file is necessary because otherwise the child processes can't pass data between each other), and then once the first child process submits its URL successfully, the file is removed, and only then are the remaining URLs retried.

Cool. Does it wait for the specified amount of time, or does it just retry in n seconds?

It waits for 2 seconds between requests. The request itself takes several seconds, so the effect is that it's retried about every 5 seconds, but it depends on the latency of your internet connection.

You have already reached the limit of active Save Page Now sessions. Please wait for a minute and then try again.

This doesn't literally mean that you have to wait for exactly one minute; if they meant that they would probably have written "wait for 1 minute". I think the intended meaning is "wait for a short time", since (I think) how it works is that once your number of concurrent active processes is below a certain number, it will let you start submitting more URLs.

I'm adding a sleep command so that the script actually waits for however long the delay is before starting the captures, which it previously didn't do.

This has been implemented now. I've been testing it myself and it appears to work correctly.
In the edge case that was originally reported, the new default behavior would be to submit 30 captures, spend 3 hours and 40 minutes waiting, and then check the SPN status API for the results. I don't think this would be ideal behavior in that situation, but at least it would (probably) work. Hopefully that sort of thing remains extremely rare.

If you can keep a list of:

how many downloads have been completed in the last 60 seconds
how many downloads are still being done

You could use it to avoid making the script hit the download limit.

The script does not need to handle the limit of captures in the last 60 seconds by itself, as far as I can tell. SPN does not prevent you from submitting new URLs until there are multiple URLs actually being crawled by IA, so it kind of sorts itself out. I'm probably not going to adjust the script's behavior in this context.
The script already keeps track of how many capture jobs are active ("how many downloads are still being done"). However, this is not done in the same way that SPN keeps track of them, because the script also counts URLs that haven't been submitted yet; URLs that have been submitted but which are not being crawled yet; and URLs that have finished being crawled that the script doesn't know about yet. As such, there is no reason for the script's default number of maximum parallel capture jobs to correspond to the server-side limit.
We could use https://web.archive.org/save/status/user to get the number of active processes, but there is a limit to the overall rate of web requests that the script can send to web.archive.org. If you go over about 1 request per second, the site serves you a 429 error and you can't send any web requests for a minute. The script can already tell if the limit has been reached by submitting new URLs, which it already has to do anyway in order to work, so it would be counterproductive to also use a completely separate way of checking if the limit has been reached.
As far as I can tell, there is no time penalty associated with getting the "You have already reached the limit of active Save Page Now sessions" error message. It would not be meaningfully more efficient to switch to a different method, even if we didn't need to worry about getting 429 errors.

I hope this explanation makes sense. I'm not trying to avoid working on it, but just trying to sort out the logic of whether or not it would actually be beneficial to make particular changes to the script.

how many downloads have been completed in the last 60 seconds

This isn't possible to calculate directly, because the script doesn't actually know when each capture job was completed. By default, if -p is set to 30, then the script also waits 30 seconds in between checking the status API for each URL, so if it gets the response that a capture was completed, then it could have been at any time in the past 30 seconds.
In the API response we do get the time it took to complete the job (e.g. "duration_sec":12.8) and the time that the main URL was captured (e.g. "timestamp":"20230226190000"), but I don't think this would take into account how long it took for the first URL to load, so it wouldn't be quite exact either.

The explanation makes sense overall. However, there's a couple of things I don't understand.
The API say when the capture was complete, so it would be possible to make a time difference and figure out how many downloads were completed within the last n seconds.
Add the number of downloads we are still waiting to complete, and you have the number to tell if we are within the limit.
However, I don't understand this part

if -p is set to 30, then the script also waits 30 seconds in between checking the status API for each URL

If 30 downloads all start together, and they all complete within a minute, don't we hit the limit of 12 downloads per minute?
I'm surprised it works.

If 30 downloads all start together, and they all complete within a minute, don't we hit the limit of 12 downloads per minute?

The first ~4 are started together, but then after that the script waits 2.5 seconds between starting captures. It's specifically 2.5 seconds because of the need to avoid 429 errors. I haven't changed this behavior recently.
The limit of 12 captures per minute doesn't seem to be something that we need to be concerned about in the context of the script's code, because the server seems to take care of it. The delays that are introduced appear to effectively reduce the rate to 12 captures per minute even if you submit URLs more frequently than that.

The messages are actually a little bit misleading. If it says

The capture will start in ~1 minute because our service is currently overloaded.

this actually just means that you've hit the limit of 12 or 4 URLs per minute. In other words, it's saying you're the one overloading the service. Of course, any individual user can't make much of a dent in their overall traffic, so it's not entirely true.
The difference in the length of the delays appears to be solely dependent on whether or not you're authenticated, and thereby whether your limit is 12 or 4.

Thanks for the explanation.
If needed, the field status_ext  does have a clearer error code, see page 5 and 6 of the API doc

error:user-session-limit
User has reached the limit of concurrent active capture sessions.

By the way, looking at the error code, some are not recoverable, for example:

error:invalid-url-syntax
Target URL syntax is not valid.
error:invalid-host-resolution
Couldn’t resolve the target host.

You might want to treat them separately.

These error codes could only be produced when initially submitting the URLs, but you can't get them if you aren't authenticated, because they aren't included in the HTML. But whether or not you're authenticated, it gives you the plain English error messages, so those strings can be checked for in both cases.
The script only checks error codes when dealing with the status API endpoint, which always returns JSON. You can't get these errors from that API, since if you're checking it in the first place then the relevant URL has already been submitted successfully.
The script only needs to check whether the error is one that is known to be recoverable. If the error isn't one of them then the capture job is immediately considered to be failed, so it doesn't need to have a list of all of the non-recoverable error strings.

error:invalid-url-syntax
It's pretty much impossible to get this error from the API anyway. If you do try to archive invalid URLs for some reason, then it usually just gives you a 400 Bad Request error instead of a normal JSON or HTML response.

I have a text file containing 1000s of URLs.
I begin archiving them using spn.sh -p 10 -nq -a 'S3KEY:S3PASS' -d 'skip_first_archive=1&capture_outlinks=1&capture_screenshot=1&if_not_archived_within=10000d&delay_wb_availability=1' list.txt
For the first few URLs, I get
The capture will start in ~4 seconds
The capture will start in ~5 seconds
The capture will start in ~6 seconds

And then out of nowhere,
The capture will start in ~10 hours 40 minutes

This capture is now going to eat 1 p job for the next ~10 hours.
And then back again to
The capture will start in ~5 seconds
The capture will start in ~6 seconds

As spn.sh works its way down the list, more and more super long capture jobs slowly end up saturating all 10 available ps , eventually, completely stalling the archival process. All 10 parallel jobs are now sleeping.
Proposal - could you please add an additional flag, where the user may choose to have the job immediately fail(without any retries) if the "capture will start in time" message returned by the wayback machine, exceeds a certain time specified by the user in Nh Nm Ns ?
I would rather spn.sh skip URLs on the list and move on to the rest of the list, than sleep for 10 hours. I could get back to the links I didn't capture later.
Thanks!

Do you mean that the delayed captures should be treated as invalid or as failed?
If the capture is going to start in 10 hours then it will still start eventually, so to me it would make sense to mark them as failed, so that the script eventually ends up recording the capture without too much disruption to other captures.
I think it would make sense for the default maximum wait time to be an hour, under the assumption that if the amount of time is less than an hour then there would probably be a good reason for it. To be consistent with the other parts of the script the time would probably be stored in seconds.

I have faced this delay several times when manually requesting a capture from their UI. This often happens when snapshotting GitHub. It also seems like the pending capture still counts towards your max concurrent capture limit.
If you want to implement this, please do not make it a default.

Do you mean that the delayed captures should be treated as invalid or as failed?

Treat as invalid if time in the "capture will start in" message exceeds user specified time in a new flag.
I don't know why I didn't use the word "invalid" in my previous comment. Even though you asked a very simple question. Sorry for the long-windedness of my previous comment @overcast07 .


Tried to pay: https://ludvigart.com/product/test/

Tried to pay: https://ludvigart.com/product/test/

See our FAQ. node 033ce... is asking for a huge 1000 blocks cltv delay which makes it really hard to route through it. In the short term there is not much you can do now except open a direct channel to the node that you want to pay.

Ok.


Fixed bug in xSPWebApplication

Fixed issues as discussed in issue #268
Thanks for submitting a Pull Request to this project.
Please make sure you have read the Contribution Guidelines.
To aid community reviewers in reviewing and merging your PR, please take the time to run through this checklist:

[X] Change details added to Unreleased section of readme.md?
[X] Readme.md updated with details of new/changed resource?
[X] Added/updated documentation and descriptions in .schema.mof files where appropriate?
[X] Examples updated for both the single server and small farm templates in the examples folder?
[X] New/changed code adheres to [Style Guidelines]?(https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md)?
[X] PSScriptAnalyzer run?
[X] Unit and Integration tests created/updated where possible?

This change is

Reviewed 3 of 3 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


How to create your own data set

I have a face picture, and I want to generate the corresponding .pt file from it, which is the same as the data set provided by the demo. How do I generate a 3D model of my own picture?
Can you give me a hint? thank

Hi, you need to first perform GAN inversion to get the latent code of the images. To do this you may use code from https://github.com/rosinality/stylegan2-pytorch/blob/master/projector.py or use our code here https://drive.google.com/file/d/1pCfnDiHZNnRoEVZ4RhcLfZyPrJgUWEYk/view?usp=sharing.
Next, you can run this GAN2Shape code to get the 3D shape. You need to revise the config file accordingly by changing the input image path and latent code path.


fix(website): #244 fix fetch request

Closes: #244

@Han2-Ro The commit message could use some better detail (something like "Fix sorting"). In general the commit message should be more about the "Why"/Reasons of the change. The "What" can be inferred from the code changes / diff :)