MMT 3.0 training - when will it stop ?

Hi
I am doing a training since a week. I notice that logs are completely different from MMT 2.5 (no more perplexity...)
Can you please let me know when it will stop. Can we stop it, evaluate and resume training if not satisfactory?
thanks
currently last lines of my log are :
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
2018-08-27 19:42:37.505728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-27 19:42:37.505763: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-27 19:42:37.505769: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0
2018-08-27 19:42:37.505774: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N
2018-08-27 19:42:37.505876: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7707 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from /home/ubuntu/mmt/runtime/frnl/tmp/training/neural_model/model.ckpt-1144000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 1144001 into /home/ubuntu/mmt/runtime/frnl/tmp/training/neural_model/model.ckpt.
INFO:tensorflow:loss = 0.70003885, step = 1144000
INFO:tensorflow:global_step/sec: 2.67082
INFO:tensorflow:loss = 0.6209078, step = 1144100 (37.442 sec)
INFO:tensorflow:global_step/sec: 3.11626
INFO:tensorflow:loss = 0.6314922, step = 1144200 (32.090 sec)
INFO:tensorflow:global_step/sec: 3.12672
INFO:tensorflow:loss = 0.71939456, step = 1144300 (31.983 sec)
INFO:tensorflow:global_step/sec: 3.09593
INFO:tensorflow:loss = 0.77452344, step = 1144400 (32.300 sec)

In MMT 3.0 the only stopping condition is the number of "train-steps" which is unlimited by default, i.e. the training continues forever.
You can monitor the trend of the training by greping lines containing the string "Saving dict for global step", which reports several metrics including perplexity and loss on the  validation set. By default the validation is computed every 5000 steps.

thanks.
and which values should i look for, which are considered good enough to stop please? can i resume if stopped and results are not good?
i got these last lines from grep:
INFO:tensorflow:Saving dict for global step 1140000: global_step = 1140000, loss = 0.8813926, metrics-translate_mmt/targets/accuracy = 0.79710984, metrics-translate_mmt/targets/accuracy_per_sequence = 0.109, metrics-translate_mmt/targets/accuracy_top5 = 0.9362138, metrics-translate_mmt/targets/approx_bleu_score = 0.5400281, metrics-translate_mmt/targets/neg_log_perplexity = -0.9533308, metrics-translate_mmt/targets/rouge_2_fscore = 0.6141006, metrics-translate_mmt/targets/rouge_L_fscore = 0.7205059
INFO:tensorflow:Saving dict for global step 1141000: global_step = 1141000, loss = 0.88628525, metrics-translate_mmt/targets/accuracy = 0.7972115, metrics-translate_mmt/targets/accuracy_per_sequence = 0.112, metrics-translate_mmt/targets/accuracy_top5 = 0.93622833, metrics-translate_mmt/targets/approx_bleu_score = 0.54020756, metrics-translate_mmt/targets/neg_log_perplexity = -0.9531698, metrics-translate_mmt/targets/rouge_2_fscore = 0.6145789, metrics-translate_mmt/targets/rouge_L_fscore = 0.72062117
INFO:tensorflow:Saving dict for global step 1142000: global_step = 1142000, loss = 0.8893292, metrics-translate_mmt/targets/accuracy = 0.7969937, metrics-translate_mmt/targets/accuracy_per_sequence = 0.111, metrics-translate_mmt/targets/accuracy_top5 = 0.9354586, metrics-translate_mmt/targets/approx_bleu_score = 0.5395844, metrics-translate_mmt/targets/neg_log_perplexity = -0.9572926, metrics-translate_mmt/targets/rouge_2_fscore = 0.6141151, metrics-translate_mmt/targets/rouge_L_fscore = 0.72059435
INFO:tensorflow:Saving dict for global step 1143000: global_step = 1143000, loss = 0.88409007, metrics-translate_mmt/targets/accuracy = 0.7977634, metrics-translate_mmt/targets/accuracy_per_sequence = 0.11, metrics-translate_mmt/targets/accuracy_top5 = 0.9356619, metrics-translate_mmt/targets/approx_bleu_score = 0.53940755, metrics-translate_mmt/targets/neg_log_perplexity = -0.95406073, metrics-translate_mmt/targets/rouge_2_fscore = 0.61364555, metrics-translate_mmt/targets/rouge_L_fscore = 0.719994
INFO:tensorflow:Saving dict for global step 1144000: global_step = 1144000, loss = 0.8826237, metrics-translate_mmt/targets/accuracy = 0.7968485, metrics-translate_mmt/targets/accuracy_per_sequence = 0.109, metrics-translate_mmt/targets/accuracy_top5 = 0.93557477, metrics-translate_mmt/targets/approx_bleu_score = 0.5409114, metrics-translate_mmt/targets/neg_log_perplexity = -0.95345736, metrics-translate_mmt/targets/rouge_2_fscore = 0.61535597, metrics-translate_mmt/targets/rouge_L_fscore = 0.7214207

It is difficult to answer to your question, because it depends on the size and type of your training data and setting as well as your hardware architecture (number and type of GPUs) and, last but not least, money and time you can use for the purpose.
By the way, the "loss" is a reasonable trigger to understand the trend of the training; it can fluctuate a lot, but its moving average over several points does no increase anymore (or enough), you can consider the convergence has been reached.
In our training on 1B words, we usually run for about 1 week on a 4-gpu machine, which correspond to about 800K steps.
I am pretty sure that your training has reached the convergence.
For big/huge training I would suggest to set "--eval-steps 5000"  in order to reduce the evaluation steps. The default, which you are using, is 1000 (not 5000 as I wrongly said in my previous post).
If you are interested, you can have a visualization of the status of the training by using Tensorboard
https://www.tensorflow.org/guide/summaries_and_tensorboard.
Please refer to its documentation to learn how to use it.

I'm afraid that pointing Tensorboard to the MMT directory doesn't visualize anything. See my other thread about this.

Actually I just tried (with a toy example) and I succeeded to visualize plots as expected and as you can see from picture attached.
basically I run tensorboard as follows
tensorboard --logdir=./runtime/default/tmp/training/neural_model
and visualize via browser accessing the page
http://URL:6006/
where URL is the IP of the machine where tensorboard is running

I will test this again when retraining a new model using ./runtime instead of ./mmt/runtine. As for now you can close this!

The last part of the thread is related to issue #394,
which has been closed just now.

thanks Nicola for the details to check in the log, will try .About  tensorboardi could install and run, but http://localhost:6006/ does not work, i guess because i have to forward the port to the containing running the training (that i dont want to stop now)...

you have to use the "numeric" IP (probably <IP_ADDRESS>) instead of "localhost"
this assume that you can access the machine from the browser
if both browser and tensorboard run on the same machine no problem should occur.
Nicola

I was able to solve this issue by uninstalling and reinstalling Tensorboard via pip. This seems to have been an issue in older versions.

Anyway, it would be a great feature to have ModernMT automatically detect when it's time to halt because the perplexity is not improving so much anymore.
Also it would be helpful to start automatically Tensorboard during training. We will focus on these features for the next release!

We will advertise all our users, when we will introduce any new features proposed by @davidecaroselli .
At the moment, we close this issue.

In your help pages, I read:
By default this process will run forever, until a Ctrl+C command is sent or the corresponding command kill -2 <process-pid> is run in a shell.
Training is step 5/6. I think, with a CTRL+C, step 6 won't run. What does this step 6 ? (pack_model, in the -h infos ?)
Will the obtained model be ready to work ?

If you wait for a while, the program will continue to step 6. Once that step is finished, you can use your build.

Loek is right.
CTRL-C does not kill the full training process; It just sends a signal which interrupts step 5 only and let step 6 continue until the end.
The engine created is ready-to-use.

First, I searched for the process id:
lm-dev8@dev8:~/MMT32$ ps aux|grep mmt
lm-dev8   9034  179  7.0 42964340 4620492 pts/12 Sl+ oct.25 45452:12 /usr/bin/python /usr/local/bin/t2t-trainer --t2t_usr_dir /home/lm-dev8/MMT32/build/lib/t2t --data_dir=/home/lm-dev8/MMT32/runtime/FREN_New/tmp/training/neural_train_data/data --problem=translate_mmt --model=transformer --hparams_set=transformer_base --output_dir=/home/lm-dev8/MMT32/runtime/FREN_New/tmp/training/neural_model --local_eval_frequency=50000 --train_steps=100000000 --worker_gpu=1 --hparams batch_size=3072

As said, I killed it:
lm-dev8@dev8:~/MMT32$ kill -2 9034

The process stopped with this error, without running step 6:
INFO: (1 of 6) Corpora cleaning...                                     DONE (in 47m 8s)
INFO: (2 of 6) Corpora pre-processing...                               DONE (in 22m 57s)
INFO: (3 of 6) Aligner training...                                     DONE (in 2h 7m 44s)
INFO: (4 of 6) Preparing data...                                       DONE (in 1h 59m 26s)
INFO: (5 of 6) Training model...
ERROR Unexpected exception: None

Traceback (most recent call last):
  File "./mmt", line 610, in <module>
    main()
  File "./mmt", line 556, in main
    actions[command](args)
  File "./mmt", line 143, in main_create
    builder.build()
  File "/home/lm-dev8/MMT32/cli/mmt/engine.py", line 556, in build
    self._build(resume=False)
  File "/home/lm-dev8/MMT32/cli/mmt/engine.py", line 629, in _build
    method(self, args, skip=skip, log=log_stream, delete_on_exit=self._delete_on_exit)
  File "/home/lm-dev8/MMT32/cli/mmt/engine.py", line 428, in __call__
    self._f(*args, **kwargs)
  File "/home/lm-dev8/MMT32/cli/mmt/engine.py", line 768, in _train_model
    n_train_steps=self._n_train_steps, n_eval_steps=self._n_eval_steps)
  File "/home/lm-dev8/MMT32/cli/mmt/engine.py", line 189, in train_model
    raise ShellError(' '.join(command), return_code, None)
cli.libs.osutils.ShellError: Command 't2t-trainer --t2t_usr_dir /home/lm-dev8/MMT32/build/lib/t2t --data_dir=/home/lm-dev8/MMT32/runtime/FREN_New/tmp/training/neural_train_data/data --problem=translate_mmt --model=transformer --hparams_set=transformer_base --output_dir=/home/lm-dev8/MMT32/runtime/FREN_New/tmp/training/neural_model --local_eval_frequency=50000 --train_steps=100000000 --worker_gpu=1 --hparams batch_size=3072' failed with exit code 1

What should I do now ?

you should hit CTRL-C to stop it, then it runs step 6 properly... (and not kill the process)

@EtienneAb3d
if the directory runtime/ENGINE/tmp/training is still present and not empty, you have not lost your training.
You can run the following command to "finalize" the training
mmt create -e ENGINE --steps pack_model

This is not fully tested, so please backup your training directory runtime/ENGINE/ before running the command above
If you prefer I can make this test, but I need to know exactly, the script you run and the way you kill the process.

In your command, it lacks the languages and corpus path that seem to be mandatory.
There is then a "already exists" question, a bit suspicious. Answering yes, it seems to really launch the good step.. but fail with an error.
Checking the data, the runtime was effectively reset. I have now to restore from the backup.
Is this a bug to be fixed, or is ther something lacking in the command ? The "resume" option ?

Etienne you are right: you have to use the --resume option, not the "steps" one. Just type the full original command plus the --resume flag.

@EtienneAb3d
probably it was my fault, and the "--resume" option is mandatory
Let me test the exact procedure.
I will report with the correct solution asap.

Using the exact original "create" command, and adding "--resume --steps pack_model" at the end seems to work.

I summarise this discussion.
Suppose you are training a system, by running the following command:
mmt create en it data
In order to correctly stop the engine and get a working (finalised) system before it stops autonomously, you have to:

CTRL-C the process "mmt create" ( neither t2t-trainer or a shell script which runs "mmt create")
kill -2 proc_id (equivalent to CTRL-C) the process "mmt create" (with process id is proc_id)

If for any reason, the training failed or the training process is killed in a wrong way, you can resume the training by running
mmt create en it data --resume
which will stop autonomously when the maximum number of training steps is achieved.
If you do not want to resume the training, but simply "finalise" the engine without the maximum number of training steps is achieved, you have to run
mmt create en it data --resume --steps pack_model
Note that, you can finalise an engine only if at least one checkpoint is stored.

In my case, a simple "--resume" didn't work. I got an error. But, not sure if this is because of my special conditions. I will check this again later, because I would like to test this procedure:

train to a special point, with a quite short delay, and stop the training
duplicate all data, in order to be able to finalize a first prod model
resume to continue the training, in order to possibly get a better prod model later

I imagine you got a message that the engine is already existing, right?
Actually, you should find your engine in engines/default.
In this case your engine should be formally fine; you could try to start it and test it.
The performance depends on how much it was trained.
Assuming that the engine is formally fine, but you would like to continue training. This is possible if you run the first command  mmt create  with the --debug  flag.
Otherwise, there is no easy way to resume. It is still possible, but in an approximated way and with several manual actions. Hope you do not need it.

Thanks @nicolabertoldi ! I hope this would work and I won't lose almost 4 days of training. By the way, I see saved checkpoints in the tmp directory and the last one was saved later today. Can I resume the training from the last saved checkpoint, then kill it properly to build the engine?

hi @mzeidhassan
You can resume training as you have this checkpoint.
To be extremely sure, I would suggest to backup the whole runtime/default/tmp/ directory.
Could you please look at runtime/default/tmp/training/checkpoint.json, which represent the training steps already performed?
The content should be the following:
["clean_tms", "preprocess", "train_aligner", "write_config", "prepare_data", "train_model"]

If you see also an additional step pack_model, your training is completed and the engine is ready to use.
In this case the --resume does not work (because the training is considered finished); to actually resume the training you have to edit the file and remove the step pack_model.

Thanks @nicolabertoldi ! It ended all well and the training was finished. Thanks for providing more info.
Any plans for automatic termination of training once engine reaches convergence? Cannot you use the same script used in 2.5?

Hi @mzeidhassan,
good to know!
Regarding the automatic termination policy,  we decided to not implement it in the transformer-based. However, we can re-think about this in the future.
I open a new "feature-request" issue. Please follow it to keep update
Please feel free this issue if you do not have any further questions.

Done, thanks @mzeidhassan ! :)


decidable.rec stuck on simple example

Hi,
I cannot seem to figure out why the following evaluation gets stuck.
    import data.set
    open bool set

    lemma decidable_univ [instance] {T : Type} [T_dec : decidable_eq T] (x : T) :
      decidable (x ∈ univ) := decidable.inl trivial

    eval if tt ∈ univ then tt else ff

yields
decidable.rec (λ (Hc : true), tt) (λ (Hnc : true → false), ff) (decidable_univ tt)

Evaluation works fine if the definition of univ is made reducible and the lemma is removed. Any clarification would be appreciated.
Thanks,
Daniel

This is because the proof of theorems (and hence lemmas) are elaborated in parallel. Therefore, the value of decidable_univ is not available for the remainder of the file, by default. If you want to have the value available, you can either make it a definition, or use the reveal command.
import data.set
open bool set

lemma decidable_univ [instance] {T : Type} [T_dec : decidable_eq T] (x : T) :
  decidable (x ∈ univ) := decidable.inl trivial

reveal decidable_univ
eval if tt ∈ univ then tt else ff

For discussion about this, see this issue: #576.

I believe all the instances of decidable in the standard library are declared as definitions. This seems to me to be natural; the instance is a decision procedure which returns a piece of data. Even though thoerems and the like are no longer opaque, I think we should still reserve "theorem", "lemma", etc. in the standard library for things that return elements of Prop.

Thanks. I usually make instances of decidable definitions instead of lemmas, and this was just a careless mistake on my part. I agree that "theorem" and "lemma" should be reserved for things that return elements of Prop, especially since one could accomplish the same thing by using "xyzw definition". Perhaps Lean could even give an error/warning when this is not the case.


Fix mask -> tag terminology

Addresses https://github.com/rust-lang/rustc-guide/pull/282#discussion_r265550258.

 I will look into it this weekend.

@scalexm Do you know why all of the chalk links broke?

If I had to guess, it's because chalk got moved out of the nursery into @rust-lang/chalk.
It looks like that project should have been under a Pin<&'links>.

I opened #288 to fix the links, but I'm pretty confident that this fix is ok without travis :P


First in last out for buy and sell order

System information

Have I written custom code (as opposed to using zenbot vanilla): I can understand and modify basic codes, like change the output from "vs. buy hold" to "bot vs. buy hold"

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.3 LTS
Release:	16.04
Codename:	xenial

Zenbot version (commit ref, or version): current master

Zenbot branch: master

NodeJS version: v8.9.4

Python version (when using a python script): not using python script

Exact command to reproduce (include everything): NA

Did I make any changes to conf-sample.js?: I changed the conf.js to my desired settings

Describe the problem
I have made changes to the engine.js file to enable it to make multiple buys without waiting for a sell to happen by reducing "held_pct" to 0.
When the sell side is triggered, it only looks at the last buy price instead of performing a first-in last-out of the buy prices. I'm guessing a counter needs to be added to the buy price for the sell price to reference, such as buy_price_1, and buy_price_2, and buy_price_3.. or buy_price_x where x is incremented for each buy and when it triggers a sell it will look at the most recent buy_price_x , for example buy_price_3, and makes sure to reference buy_price_3, and then nulls buy_price_3 and moves on to the buy_price_2 if another sell is triggered to sell by referencing buy price_2. It correctly works with the last buy price but I would like it to work for all the previous buys until only the first buy is left to be referenced as the last buy price.
basically the scenario will play out as follows:
first buy trigger:
buy_price_1
sell trigger
sell references buy_price_1
buy trigger:
buy_price_1
buy trigger:
buy_price_1 exist
create buy_price_2
sell trigger
sell references buy_price_2
nulled/deletes buy_price_2
buy trigger:
buy_price_1 exist
create buy_price_2
buy trigger:
buy_price_1 exist
buy_price_2 exist
create buy_price_3
sell trigger
sell references buy_price_3
nulled/deletes buy_price_3
sell trigger
sell references buy_price_2
nulled/deletes buy_price_2
buy trigger:
buy_price_1 exist
create buy_price_2
sell trigger
sell references buy_price_2
nulled/deletes buy_price_2
sell trigger
sell references buy_price_1
sell trigger
sell references buy_price_1
That's in. First-in Last-out for buy_price when a sell triggered.
Source code / Error logs
Feature request

I suppose this is in reference to #1389
held_pct was added in b0b7c9205e5ca587aacbe80bb5c3d76108664a84 which unfortunately doesn't outline why it was added in the first place.
Now considering held_pct blocks the bot from doing consecutive buys i'd say we should look into figuring out why and simply fix just that.
Regarding this report, will i get the idea behind FILO i don't see how this is supposed to help improve the bots buy/sell habit as we absolutely do want to always use the last buy/sell price for sell/loss protection.
Lets assume the bot does a buy - sell - sell - buy - sell we want to use the first buy price during the first and second sell, followed by the second sell price during the second buy and finally the second buy price during the third sell to trigger buy/sell loss.

After thinking about this the idea isn't to bad at all.
Assuming the previous example using buy - sell - sell - buy - sell the bot would make a possible loss on the second buy signal if the second sell price was above the first.
sell low ... sell high, buy low = loss
sell high ... sell low, buy low = win

Will we currently ensure the buy price is below the last sell price it could still be (way) above the first sell price, causing a loss.
Same for a sell on two (or more) previous consecutive buys if the last buy price is below the previous.
buy high ... buy low, sell high = loss
buy low ... buy high, sell high = win

Without looking into it i suppose held_pct is supposed to prevent just that, only for buy signals of course, as we don't have any such "protection" for sells right now. Which would also explain why the bot refuses to do consecutive buys but not sells.
I'll look into it once if get the chance.

If you're able to implement a FILO on referencing the buy_price when it makes a sell, every sell will be higher than it's buy in price if sell_loss protection is enabled. I was looking into stack, but can't figure out how to make it happen.

Frankly this shouldn't be to hard to implement if we simplify it by always aiming for a profit, no matter what.

Let's do it. :)

I'm just so happy with your usage.


implement full docker build

Full docker based build which could be used in github actions
The build step is split in two parts and to make the final image as small as possible. I'm not that familiar with ruby so there might be a possibility to build the gem with all dependencies in the build step so there would be no need to add more dependencies in the final image.
I've tested the build with native docker build . (single arch image) run and with docker buildx build --platform linux/amd64,linux/arm64  . (multi-arch image)
Question was raised in https://github.com/sj26/mailcatcher/issues/497

Do you also need/want a docker github action workflow?

Hiya! I don't think this is building the frontend assets required. And it won't be using the signed gem. Is there a benefit to moving away from the simple gem install?
I would love a github workflow that builds a docker image for new github releases, using the released gem binary.

maybe have a look at https://github.com/webdevops/azure-metrics-exporter/blob/main/.github/workflows/release-docker.yml
this is a pipeline to build multi-arch images

hm.. i'm thinking about the signed package but not sure it this is a real benefit when shipping docker images.
it would make more sense to sign the docker image at all, not just a part of it which you cannot check it when running the image.
singing the package makes more sense if you install it from via package system, but here we're going to ship even the runtime and the gem is just a small part of it. signing the docker image is a different topic and a lot of people doesn't check these signatures.
it would enable you to build a docker image for every commit in main so people can easily try them and give feedback.


HIDIVE Authentication

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.05.18. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.05.18

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Description of your issue, suggested solution and other information
After checking the hidive.py file i noticed it doesn't log the user in, which makes the whole script useless, since all content is locked for people that aren't logged in. So i would like to propose the idea of adding an authentication functionality to that script.
If Account Credentials are needed let me know how i can send them to you privately and i will do so!
Thanks in advance

@remitamine Hey, i downloaded the source to test if it is working, but running this "python -m youtube_dl https://www.hidive.com/stream/real-girl/s01e008 -u usernamehere -p passwordhere" gives me this output:
[HiDive] None: Downloading webpage ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)> (caused by URLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)'),))
Am I doing something wrong or is the code broken?

post the full verbose log and make sure that you're using an updated python version(using python 3.6.x is recommended).

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['https://www.hidive.com/stream/real-girl/s01e008', '-u', 'PRIVATE', '-p', 'PRIVATE', '-v'] [debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252 [debug] youtube-dl version 2018.05.18 [debug] Python version 3.3.0 (CPython) - Windows-post2008Server-6.2.9200 [debug] exe versions: none [debug] Proxy map: {} [HiDive] None: Downloading webpage ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)> (caused by URLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)'),))   File ".\youtube_dl\extractor\common.py", line 579, in _request_webpage     return self._downloader.urlopen(url_or_request)   File ".\youtube_dl\YoutubeDL.py", line 2211, in urlopen     return self._opener.open(req, timeout=self._socket_timeout)   File "C:\Python33\lib\urllib\request.py", line 473, in open     response = self._open(req, data)   File "C:\Python33\lib\urllib\request.py", line 491, in _open     '_open', req)   File "C:\Python33\lib\urllib\request.py", line 451, in _call_chain     result = func(*args)   File ".\youtube_dl\utils.py", line 1101, in https_open     req, **kwargs)   File "C:\Python33\lib\urllib\request.py", line 1255, in do_open     raise URLError(err)
Using the latest python version (3.6.5)


New package: QuantizedArrays v0.1.0

Registering package: QuantizedArrays
Repository: https://github.com/zgornel/QuantizedArrays.jl
Created by: @zgornel
Version: v0.1.0
Commit: 174cf0cad50f837e8a5232730278dbd0971d3352
Reviewed by: @zgornel
Reference: https://github.com/zgornel/QuantizedArrays.jl/issues/1

You can probably have at least Test as a test-only dependency.

Good point. The next release will have it.


Bug: console.info accidentally included in released package<EMAIL_ADDRESS>

Lexical version: 0.13.0

View it on npm registry: https://www.npmjs.com/package/@lexical/react?activeTab=code

@acywatson @fantactuka @zurfyx URGENT, production codes are also polluted

@yf-yang - Sorry about that, we'll release 0.13.1 today

Released v0.13.1


SVG export

there is no option to export the current cell as an image.
[wish] I would use nothing else if there were the option to export to tikzit, even tikz.

We could have separate buttons for "Export to TikZ" and "Export to SVG" in the "Export" dialog (#525).


"No such file - ["config/database.yml"]" (not a rails engine)

I have a gem which isn't actually a Rails::Engine (it has no Rails::Engine in it), which could be the root of the problem. My gem doesn't need any of the features that come with Rails::Engine, but it is meant for use with Rails (and does depend on ActiveRecord), and I want to have some integration tests in rspec to verify it works as expected in the context of Rails.  So combustion seems very useful.
Not sure if not being a Rails::Engine is part of the problem or not.  I have a bunch of specs that were already running fine, including many ActiveRecord. Now I want to add some integration tests.
My spec_helper.rb, trying to get the simplest thing that might work working, has:
require 'bundler'
Bundler.require :default, :development
Combustion.initialize! :active_record

From the Combustion.initialize! line, I get an exception raised:
RuntimeError: Cannot load `Rails.application.database_configuration`:
Could not load database configuration. No such file - ["config/database.yml"]

full stack trace
I have run bundle exec combust, and do have the files at spec/internal, including spec/internal/config/database.yml
Any ideas or hints of what might be going on, or where I should go next? Thanks!

Aha. I had standalone_migrations in my app, from before I considered using combustion.
They are incompatible in some way. Removing standalone-migrations, the error goes away, and my tests are running (although oddly having some failures they didn't have before, even before actually adding the integration test!  Due to some differences in environment between combustion vs just require 'activerecord' I guess. But this I can explore!)
So nevermind, closing! Good to know it is incompatible with standalone_migrations (and indeed there's probably no good reason to use both).

Great to know you've got things working :) Certainly if you hit other problems you're unsure about, feel free to open further issues here!

@pat thanks! I've got it working, but did have some confusions and things I'm still not certain about. Would it make sense for me to open an issue for each individually?


How I can set parent element?

I want to set parent element for tooltipster's div, how I can do it?

@louisameline
Hi,
V3.3 is awesome, Thanks!
But inability to set parent is the issue on my site which uses rochal slimscroll. Which is causing the tooltip to be fixed when the element scrolls.
Realized that you are providing parent support and refactoring the positioning system.
When can i expect V4 stable release?
Thanks again!

Hi and thanks for your message ! I'm crazy busy with my company but I hope to get the time this summer to finalize this.

Hey @louisameline
Have you been able to make any progress on this enhancement? We need this feature for one of the projects we are working on at my company, and we would be happy to assist you in anyway possible.

Hello,
to people who want to set a parent other than body just to achieve fixed positioning, you don't need to. Tooltipster v4 already gives a fixed position when needed.
Having the tooltip inside another parent may cause overflow issues (see #463), I'm not sure what you'd expect. If the parent you want to append to is actually positioned at (0, 0) window coordinates, I think that shouldn't be an issue though. Could you please tell me what your usecase is ? Thanks
Note: the Tooltipster v4 branch should work pretty well but is not stable yet, some options/methods may still be modified.

We are trying to use tooltipster with Magnific Popup (http://dimsemenov.com/plugins/magnific-popup/) and found two issues.

The tooltips doens't properly display next to the trigger element.
It's causing the modal background to scroll to the top.

By being able to set the parent element, that should take care of these two issues we face.

Ok, I think that probably neither of these issues should require to change the parent element. Please try v4 and see if your issues are solved.

The position appears to be more accurate with v4, but the tooltip doesn't function well when the modal scrolls vertically, as tooltipster listens for scrolls on the window and not the parent container. The code you provided in this issue (https://github.com/iamceege/tooltipster/issues/174) appears to fix this issue though.
As for issue 2, I still haven't found a good solid solution for that yet. The main issue is that magnific-popup is applying overflow:hidden to the html node, and tooltipster is applying overflow-x: hidden to the body node. In addition, our css is setting the html height to 100%, and the body height to 100%.
What I found is that when you set the html and body height to 100%, scroll down the page, and then apply overflow: hidden to the html node and overflow-x: hidden to the body node, the browser resets your scrollY to 0. Setting body: min-height to 100% instead of height to 100% appears to eliminate the scroll issue, but I'm still determining how viable of a solution that is. It feels that setting the parent element's overflow-x to hidden instead of the body would prevent this scroll issue from happening.

I'm not sure you really used Tooltipster v4.0.0rc14 as we do listen to all parents' scroll events, and do not affect overflow-x anymore.

Option to set parent element is indeed needed. For me it's mostly styling. I don't want to use another theme or add my own CSS file for it, I simply want to add a couple style definitions to main site's CSS file. Now I have to do it with "body" declaration to override Tooltipster styles. Would be much better to add, well... less global parent.

I don't understand what you mean and how this would be of help. Could you give a sample of the code you think you have to write and of what you wish you could write? Thank you.

Well, it's a simple CSS definition inheritance. You can't override, for example, .tooltipster-default definition without putting parent before it. The only parent you can put in v3 is "body". But I'd prefer to put another parent - for example, ".toolbar", if I'm using Tooltipster for editor toolbar buttons only.

I see. In Tooltipster v4, the theme system is better, you can create a theme that will override only a couple properties. Let me quote v4's documentation:
/* This is how you would create a custom secondary theme on top of tooltipster-noir: */

.tooltipster-noir.tooltipster-noir-customized .tooltipster-box {
	background: grey;
	border: 3px solid red;
	border-radius: 6px;
	box-shadow: 5px 5px 2px 0 rgba(0,0,0,0.4);
}

.tooltipster-noir.tooltipster-noir-customized .tooltipster-content {
	color: blue;
	padding: 8px;
}

And where we can get that v4? I wasn't able to find any links or mentioning of it except some of your replies.

It's on the v4 branch of the project. I'm still working on the plugin system so it's not fully stable, however there are no bugs as far as I know, so it's pretty safe to use it I think. It includes an updated documentation, take a look if you stumble into the breaking changes.

Thanks for pointing out, pal. GitHub is So confusing to me, I can barely find anything here. Most confusing part is pull request system. I absolutely fail to understand it, despite that I've tried several times and am a developer myself :-)

No problem ;)

Tooltipster v4 is out and I think will address most issues that make people want to choose a different parent.
But I don't think we'll ever support something else than the body tag as a parent for the tooltip... except if the parent itself is located at [0,0] in the body, in which case you can use the undocumented parent option at your own risk.
Thanks
What's new in v4: #565
Migration guide from v3 to v4: #566


Would like to ask the recent release of nuxt about how long?

Would like to ask the recent release of nuxt about how long?
Really looking forward to be faster, very grateful

Team is working hard, I'm even not sure does @pi0 sleep at all :smile:

https://github.com/nuxt/nuxt.js/releases/tag/1.0.0-rc3


Refactor code to generate the Paradigm Table

Client did not like the code quality for the generation of the paradigm tables. It needs to be refactored so it may be used with multiple languages and paradigm formats.

Code has been refactored to the clients standards


Use Pageable.getOffset instead of calculating the start record in JestElasticsearchTemplate

Request
Could the following line of code in JestElasticsearchTemplate be changed to use Pageable.getOffset instead?
https://github.com/VanRoy/spring-data-jest/blob/97428c3bf7893a61b1b7b18f5ce2369505a474f4/spring-data-jest/src/main/java/com/github/vanroy/springdata/jest/JestElasticsearchTemplate.java#L1153
Details
In the referenced line of code, the start position is calculated using Pageable.getPageSize * Pageable.getPageNumber.  The same result is also available through the Pageable.getOffset method.
I know the main difference is Pageable.getOffset returns a long and the startRecord assignment is an int so there will be some Integer overflow issues to worry about (see Potential Concerns).
Reasoning
The reasoning for asking is we are using a custom Pageable that works with limits and offsets (similar to https://stackoverflow.com/a/32763885).  Unfortunately there does not seem to be an easy way to alter/override the referenced JestElasticsearchTemplate line without making our own version of the class.
Potential Concerns
My main concerns with the change are:

It would impact current users of the library that use a custom Pageable that doesn't calculate the offset [correctly].  That means this change will have to be communicated clearly in the the release notes.
Potential integer overflow issue during the downcast from long to int.  Though, this would be an issue in the current code if pageSize * pageNumber was high enough.  Plus that would be well into the Elasticsearch max_result_window territory.

Way Ahead
If you agree with this change, I'd be happy to submit a PR for approval.

Hi @jbogan , It's really seem's to be the right way to implement this.
I will be very glad if we propose a PR.
Thanks a lot for your feedback.
Julien.

Hi @jbogan , Thanks a lot for your help.
You PR is merged and released in 3.2.5.RELEASE version.


Frontend: Add account statement export button

Purpose
Expose account statement export (#15) as a button in the frontend.
Changes
Add an "Export" button in the account export widget.
Before:

After:

Checklist

[x] My code follows the style of this project.
[x] The code compiles without warnings.
[x] I have performed a self-review of the changes.
[x] I have documented my code, in particular the intent of the
hard-to-understand areas.

Approved from a design perspective.

Couldn't sucessfully export an account statement. I assume this doesn't have anything to do with these changes, but shouldn't this work on stagenet currently?

There is an extra graphql in the URL that shouldn't be there (note the .. in ${apiUrl}/../rest/export/statement?accountAddress=${props.account.address.asString}). Is this the URL that the frontend invokes or just a test?

There is an extra graphql in the URL that shouldn't be there (note the .. in ${apiUrl}/../rest/export/statement?accountAddress=${props.account.address.asString}). Is this the URL that the frontend invokes or just a test?

I just clicked the export button 😄

There is an extra graphql in the URL that shouldn't be there (note the .. in ${apiUrl}/../rest/export/statement?accountAddress=${props.account.address.asString}). Is this the URL that the frontend invokes or just a test?

I just clicked the export button smile
It it a vue thing that it can resolve a url like that, or is it native anchor href stuff? Never used it before. Alternatively you could pull in something to resolve a constructed path (like nodejs path.resolve(apiUrl, '..', 'rest/export/statement'), which I think should work.

Christ. It happened because there was an extra / on the URLs of the non-dev envs. Will try and use such a path resolver if I can do it without adding a new dependency.

Christ. It happened because there was an extra / on the URLs of the non-dev envs. Fixed them. Will try and use such a path resolver if I can do it without adding a new dependency.

I would be surprised if you could 😀

Regarding if it's a vue thing, no it's just standard string interpolation.

I meant the ../ part of the url. Was just surprised that it could be a part of a valid URL used in a href attribute 😉

Christ. It happened because there was an extra / on the URLs of the non-dev envs. Fixed them. Will try and use such a path resolver if I can do it without adding a new dependency.

I would be surprised if you could grinning

It actually turns out that you can :exploding_head:

Regarding if it's a vue thing, no it's just standard string interpolation.

I meant the ../ part of the url. Was just surprised that it could be a part of a valid URL used in a href attribute wink

Ah yeah that's just like file paths (which is what URL paths originally was). Except that //../ is then handled differently than (at least Linux) does it :palm_tree:


Set Command doc string from the callback so Sphinx docs look nicer

Signed-off-by: Ray Lehtiniemi<EMAIL_ADDRESS>

The effort should rather go into a sphinx extension that is able to deal with commands properly (providing autocommand directives or something like that). Documenting CLI commands as classes or functions might be a good quick hack, but as @Almad said, you're missing the arguments with that.
Also, I don't think it's a semantically correct thing to set the docstring on this function, because click.command is not a "pure" decorator: The resulting object has no API compatibility with the wrapped function.
I'll leave this open because it might be useful for others though.

I'm somewhat okay with that, so I'm going to merge it.  That said, @untitaker is right about that this probably is better solved with an extension.


Ubuntu repositories unable to verify Release.gpg

Describe the issue
When trying to mirror the prod repositories with debmirror I get a errors from GPG, that Release.gpg can't be verified.
gpgv: Signature made Tue Aug 27 19:39:44 2024 UTC
gpgv:                using RSA key EB3E94ADBE1229CF
gpgv: BAD signature from "Microsoft (Release signing<EMAIL_ADDRESS>
When did the issue occur?
2024-09-06 (6th of September).
If applicable, what package did you attempt to install, and from which repo?
I'm mirroring these three repositories, and they all give the same errors:
https://packages.microsoft.com/ubuntu/20.04/prod/
https://packages.microsoft.com/ubuntu/22.04/prod/
https://packages.microsoft.com/ubuntu/24.04/prod/
Steps to Reproduce

Download debmirror with apt install debmirror
Run this debmirror script for Ubuntu 24.04 repository:

TMP_PATH="/tmp/debmirror"
KEYRING_NAME="trustedkeys.kbx"
KEYRING_FILE="$TMP_PATH/$KEYRING_NAME"
SLEEP_TIME=5

mkdir -p $TMP_PATH
if ! gpg --no-default-keyring --keyring $KEYRING_FILE --fingerprint; then
    echo "Error: Failed to create keyring"
    exit 1
fi
export GNUPGHOME=$TMP_PATH

if ! gpg --no-default-keyring --keyring $KEYRING_FILE --import /srv/gpg-keys/microsoft.asc; then
    echo "Error: Failed to import GPG key"
    exit 1
fi

arch="amd64"
section="main"
release="noble"
server="packages.microsoft.com"
inPath="/ubuntu/24.04/prod"
proto="https"
outPath="/srv/repositories/daily/apt/microsoft-packages-noble"
timeout="120"

debmirror -a $arch \
	  -s $section \
	  -h $server \
	  -d $release \
	  -r $inPath \
	  -t $timeout \
	  --method=$proto \
	  --progress \
	  --rsync-extra=none \
	  --getcontents \
	  --nosource \
	  $outPath

debmirror_status=$?
if [[ ! $debmirror_status -eq 0 ]]; then
    echo "Error: Failed to mirror repository with debmirror"
    exit 1
fi

echo "Success: Mirrored repository with debmirror"
exit 0

Actual Result
debmirror can't verify the signature on the Release.gpg file.
Mirroring to /srv/repositories/daily/apt/microsoft-packages-focal from<EMAIL_ADDRESS>Arches: amd64
Dists: focal
Sections: main
Pdiff mode: use
Will clean up after mirroring.
Attempting to get lock ...
Getting meta files ...
** GET https://packages.microsoft.com/ubuntu/20.04/prod/dists/focal/Release ==> 200 OK (2s)
[  0%] Getting: dists/focal/Release... 	 #ok
** GET https://packages.microsoft.com/ubuntu/20.04/prod/dists/focal/InRelease ==> 200 OK
[  0%] Getting: dists/focal/InRelease... 	 #ok
** GET https://packages.microsoft.com/ubuntu/20.04/prod/dists/focal/Release.gpg ==> 200 OK
[  0%] Getting: dists/focal/Release.gpg... 	 #ok
[GNUPG:] NEWSIG
[GNUPG:] KEY_CONSIDERED BC528686B50D79E339D3721CEB3E94ADBE1229CF 0
[GNUPG:] KEY_CONSIDERED BC528686B50D79E339D3721CEB3E94ADBE1229CF 0
[GNUPG:] BADSIG EB3E94ADBE1229CF Microsoft (Release signing<EMAIL_ADDRESS>[GNUPG:] VERIFICATION_COMPLIANCE_MODE 23
gpgv: can't allocate lock for '/tmp/debmirror/trustedkeys.kbx'
gpgv: Signature made Tue Aug 27 19:39:44 2024 UTC
gpgv:                using RSA key EB3E94ADBE1229CF
gpgv: BAD signature from "Microsoft (Release signing<EMAIL_ADDRESS>.temp/.tmp/dists/focal/Release.gpg signature does not verify.
Errors:
 .temp/.tmp/dists/focal/Release.gpg signature does not verify
Failed to download some Release, Release.gpg or InRelease files!
WARNING: releasing 1 pending lock...
Error: Failed to mirror repository with debmirror

Expected Result
The Release.gpg signature should be able to verify against the microsoft.asc GPG key, and the repository should be mirrored.
This has worked up until last 2024-09-06, so I expect something is wrong with the GPG signature.
Additional context
I can't verify Release.gpg with GPG itself, either.
$ wget https://packages.microsoft.com/ubuntu/24.04/prod/dists/noble/Release
$ wget https://packages.microsoft.com/ubuntu/24.04/prod/dists/noble/Release.gpg
$ wget https://packages.microsoft.com/keys/microsoft.asc
$ gpg --import microsoft.asc
$ gpg --verify Release.gpg Release
gpg: Signature made Mon 26 Aug 2024 12:48:15 PM CEST
gpg:                using RSA key EB3E94ADBE1229CF
gpg: BAD signature from "Microsoft (Release signing<EMAIL_ADDRESS>[unknown]

I get the same error.

Thanks for reporting this, we apparently have an incorrect cache lifetime set on Release.gpg (so not a security issue, just an infrastructure issue). We'll work to get this fixed, ETA ~24 hours.

Good to see that it's not a security issue and that it will be fixed soon. Thank you.👍

The problem should be fixed now. @trv-gunnar @A4M5 can you confirm?

The problem should be fixed now. @trv-gunnar @A4M5 can you confirm?

@daviddavis
The problem is solved on my side, I get the packages back fine.
Thanks a lot !


Adding "How to" to execute "iamge_to_c" in terminal

Hi. I found your library AnimatedGIF very useful for my project. And of course, I want to try with my own GIFs. I  understand the purpose of this repository to convert GIFs to C array, but I am a little bit confused about how to use that.
I assume that I need to run image_to_c script in the terminal sending image that I want to convert the path as param, but I don't know how exactly to do it. I am using Mac OS.
I think it's would be very useful for others to have that short description on main page of the repository.
Thanks.

The image_to_c program (not a script) tells you how to use it if you run it with no parameters.

Hi bitbank2
may I ask you please to help me a little. Below the result of using your tool for converting a gif to  C array.
Unfortunately it is doing something but not creating a file with content. What is it what I am doing wrong?
When I enter following in the Mac terminal.

ast login: Sun Dec 12 14:48:36 on ttys001
You have new mail.
<EMAIL_ADDRESS>~ % open /Users/...../Desktop/...../Programmierung/image_to_c-master/dist/MacOS/image_to_c ./flame.gif > outtest.h
<EMAIL_ADDRESS>~ %
ast login: Sun Dec 12 14:50:14 on ttys000
You have new mail.
/Users/..../Desktop/..../Programmierung/image_to_c-master/dist/MacOS/image_to_c ; exit;
<EMAIL_ADDRESS>~ % /Users/......./Desktop/..../Programmierung/image_to_c-master/dist/MacOS/image_to_c ; exit;
image_to_c Copyright (c) 2020 BitBank Software, Inc.
Written by Larry Bank
Usage: image_to_c
output is written to stdout
example:
image_to_c ./test.jpg > test.h
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
[Prozess beendet]

It is creating a file but without content.

Hi @AlonsoMinas,
I just tested the latest code on my Mac (which I use for 99.99% of my work) and it works fine. I'm really not sure why the stdout generated by my program wouldn't get captured properly by a ">" redirect. Does it print the output to the terminal if you don't specify an output file?

Hi @AlonsoMinas,
I just tested the latest code on my Mac (which I use for 99.99% of my work) and it works fine. I'm really not sure why the stdout generated by my program wouldn't get captured properly by a ">" redirect. Does it print the output to the terminal if you don't specify an output file?

Hi Larry,
thank you for your answer and efforts with me. I have tried it like this:
<EMAIL_ADDRESS>~ % open /Users/.../Desktop/.../Programmierung/image_to_c-master/dist/MacOS/image_to_c ./flame.gif
<EMAIL_ADDRESS>~ %
Terminal is showing:
Last login: Sun Dec 12 18:44:00 on ttys001
You have new mail.
/Users/.../Desktop/.../Programmierung/image_to_c-master/dist/MacOS/image_to_c ; exit;
<EMAIL_ADDRESS>~ % /Users/.../Desktop/.../Programmierung/image_to_c-master/dist/MacOS/image_to_c ; exit;
image_to_c Copyright (c) 2020 BitBank Software, Inc.
Written by Larry Bank
Usage: image_to_c
output is written to stdout
example:
image_to_c ./test.jpg > test.h
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
[Prozess beendet]
When I search for     test.h      on my Mac there is no file found with this name. There must be something I am doing wrong?
But I have created attached two files with another tool in internet (https://lvgl.io/tools/imageconverter) may you check if this will work with your AnimatedGIF on a ESP32 ( I don't have it yet, is ordered otherwise I would try it my self).
If we cannot solve the problem and my files are not usable I will try it with a Windows computer.
Archiv.zip

Why are you using "open" to run the program? Just run it like any other command line program:
./image_to_c

Great worked thank you very much.


Nuget upgrade to 1.1.2

Hi! Can somebody build nuget package for new akka 1.1.2. ?
I'm getting this exception
Method 'ReplayMessagesAsync' in type 'Akka.Persistence.Cassandra.Journal.CassandraJournal' from assembly 'Akka.Persistence.Cassandra, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null' does not have an implementation." [akka://ClusterManager/user/sharding/my-actorCoordinator/singleton/coordinator#720169034]: Akka.Actor.ActorInitializationException: Exception during creation ---> System.TypeLoadException: Method 'ReplayMessagesAsync' in type 'Akka.Persistence.Cassandra.Journal.CassandraJournal' from assembly 'Akka.Persistence.Cassandra, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null' does not have an implementation.

Found pull request for 1.1.1


added options for absolute position (to body), and aling dropdown menu

Аbsolute positioning is necessary for situations in which the Dropdown exceeds the size of the container of the parental control
Align for more advanced customization
Аdding is compatible with the current version of the script.

this fix problem #26

May you consider a refactoring of this PR using popper.js as described in https://github.com/FezVrasta/dropdown.js/issues/26#issuecomment-212508046 ?


Flakey tests

https://github.com/microsoft/vscode-jupyter/runs/6886093523?check_suite_focus=true

Tests (raw, nonConda, 3.9, ubuntu-latest Error: FAILED TEST NAME: Interactive Window Debugging Debugging with VSCodePythonDebugger Debug a cell from a python file https://github.com/microsoft/vscode-jupyter/runs/6886093523?check_suite_focus=true
Tests (raw, conda, 3.9, ubuntu-latest Error: FAILED TEST NAME: DataScience - VSCode Notebook Kernel Error Handling - (Execution) (slow) Raw Kernels Display error about overriding builtin modules (without Python daemon https://github.com/microsoft/vscode-jupyter/runs/6886093523?check_suite_focus=true

https://github.com/microsoft/vscode-jupyter/blob/main/src/test/datascience/notebook/kernelCrashes.vscode.test.ts#L388-L389 when increasing the timeout to 50s it then passes

Tests (raw, nonConda, 3.9, ubuntu-latest FAILED TEST NAME: DataScience - VSCode Notebook - (Execution) (slow) Cell failures should not get cached https://github.com/microsoft/vscode-jupyter/runs/6885901069?check_suite_focus=true
`

They are now being tracked separately.


Requests 2.5.1 doesn't recognize unicode filenames for uploads

After merge of https://github.com/kennethreitz/requests/pull/2379, to allow filenames to be int types, unicode filenames are no longer recognized under Python 2.
This checks that the filename is a builtin str, which has different behaviour on Python 2 and Python 3:
requests/utils.py:118:    if name and isinstance(name, builtin_str) and name[0] != '<' and name[-1] != '>':
In requests/compat.py, builtin_str is defines as str, which is non-unicode bytes in Python 2 and unicode in Python 3. Perhaps the check should be against basestring, or is this change in behaviour intended?

In fact, I have not tried this yet, but I think it will make writing code compatible with Python 2 and Python 3 much more difficult. In working around the Python 2 issue by encoding the unicode filename, it is likely that a Python 3 issue will be introduced. This is going to require version specific encoding handling to ensure a unicode filename under Python 3 and a non-unicode filename under Python 2.

@sjagoe I have a question: if you provide us a unicode filename, what encoding should we render that with when transmitting that over the wire?

@Lukasa Admittedly, that is an issue. There is no way for you to know. I guess in the past is just encoded to ascii?
In which case, this issue is still two things:

v2.5.1 broke compatibility with v2.5.0.
I have just verified that under Python 3, if the filename is encoded beforehand, then the original issue I reported under Python 2 rears its head. That is, requests does not send the filename and my server sees the filename simply as 'file', instead of the original.

Agreed.
What we need, I think, is for guess_filename to check on basestring and then call to_native_str.

I would even be okay with it if a unicode filename was rejected outright with an exception, requiring the user to explicitly encode, but I don't know what the policy of requests is wrt user-annoying strictness ;-)

Of course, in the interest of compatibility, using to_native_str for any 2.5.x fixes would be the best way forward in any case.

So the reality is that we need to unbreak this until 3.0 at which point we can redefine the requests interface for this. I have an idea for a solution and am working on a pull request.

I'm seeing this issue now in 2.6.0 with the following code:
    url = 'https://slack.com/api/files.upload'
    with open('File β.txt', 'rb') as file:
        r = requests.post(url, files={'file': file}, params={
            'token': api_token,
            'channels': channel
        })

I get a response that no data was found. I'm using Python 3.

@abbeycode Can you run the following snippet for me please?
from requests.utils import guess_filename

with open('File β.txt', 'rb') as file:
    print guess_filename(file)
    print type(guess_filename(file))

Luckily Slack is very responsive about these sorts of things and you should kindly inform them that they're not following 10+ year old specifications and they should (namely RFC 2231). I suspect this is more of a duplicate of #2117 than this issue.

Yeah, suspect you're right @sigmavirus24. Note that this is a fairly widely-deployed standard, and the HTTPBis is working on bringing it up to date in the draft for RFC 5987bis, so servers and frameworks that don't support it really should.

@Lukasa it seems that 5987bis is expired =( (Expires Jan 3, 2015)

Work is still ongoing, WG is aiming for last call by IETF Prague.

@Lukasa this is what I get:
File β.txt
<class 'str'>

Yup, so this means your server doesn't support the RFC that's used to automatically encode binary data. I recommend you encode the name of your file before opening it, or provide the filename yourself:
url = 'https://slack.com/api/files.upload'
with open('File β.txt', 'rb') as file:
    r = requests.post(url, files={'file': ['File β.txt'.encode('utf-8'), file]}, params={
        'token': api_token,
        'channels': channel
    })

@Lukasa thanks for all the help, but I'm getting this:

TypeError: Type str doesn't support the buffer API

on this line:
if not any(ch in value for ch in '"\\\r\n'):

in urllib3/fields.py (line 34). This seems like a Python 3 issue, perhaps?

Hm. Let me look at that

Oh wait, you mean from using @Lukasa's example using encode? Yeah. I think you're better bet is to annoy Slack until it works. I'll have to look into your existing problem first though.

I sent Slack an email describing what's happening, so hopefully they'll respond and fix the issues. Thanks @Lukasa and @sigmavirus24 for all the help!


Alarm Clock

Is your feature request related to a problem? Please describe
An alarm clock in python
Describe the solution you'd like
A simple alarm clock in python

Hey, can you assign this to me?


Add liboauth missing feaures

Added full windows and theoretically mac compatibility, though some issues may remain. Still missing proper error handling on requests and some other less important features.

Compatibility is done, closing issue


Use REMIX_DEV_ORIGIN for the live reload hostname

Discussed with @pcattori on Discord.
For live reloading, we currently use the protocol and port from the REMIX_DEV_ORIGIN variable, but use the hostname from window.location. This PR also uses the hostname from REMIX_DEV_ORIGIN (when provided).
This also means that we can support use cases where the development server is on a different hostname from the static assets. This is the case, for example, with GitHub Codespaces, which proxies separate ports through HTTPS on different subdomains. For example:

https://<CODESPACE_NAME>-3000.app.github.dev for the static assets
https://<CODESPACE_NAME>-3001.app.github.dev for the websocket

(If for whatever reason you’re a Codespaces user seeing this, I set REMIX_DEV_ORIGIN to https://${process.env.CODESPACE_NAME}-3001.${process.env.GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN]} when I launch the development server)

@huw can you locally rebase your branch onto dev and the force push that onto here? this needs to target dev not main since its a code change (not just docs or templates changes).

@pcattori Done! Cheers ^_^


Make a macro to add saving/loading in rust engines

Rejected because PyO3 pymethod macro does not allow for declarative macro


feat: add new line

Add support for \n on notes and simple signals.
You can test it here: https://gateway.pinata.cloud/ipfs/QmZeMC4dFDWK7gJpFPBdd77bNb6eo8Vwgvf2eRcutn2Go9/

(Would be great to upload it to IPFS tho 🚀 )

❤❤💕💕 thank you!!


canFireEvent: returns false if event.sourceStates is nil

Currently, - [TKStateMachine canFireEvent] only checks whether event.sourceStates contains the state machine's currentState.  However, the event will also fire if event.sourceStates is nil.

Merged


New-EventLog did not work

Steps to reproduce
 New-EventLog -Source "SupperSocketService" -LogName "TestLog" -MessageResourceFile "C:\Test\TestApp.dll"

Expected behavior
wirte an event log

Actual behavior
New-EventLog : The 'New-EventLog' command was found in the module 'Microsoft.PowerShell.Management', but the module could not be loaded. For more information, run 'Import-Module Microsoft.PowerShell.Management'.
At line:1 char:1
+ New-EventLog -Source "SupperSocketService" -LogName "TestLog" -Messag ...
+ ~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (New-EventLog:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoloadMatchingModule

By the way, I have already tried Import-Module Microsoft.PowerShell.Management, but it still failed with same error message.
Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0
PSEdition                      Core
GitCommitId                    6.1.0
OS                             Microsoft Windows 10.0.16299
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Yes, the problem is that New-EventLog exists in Microsoft.PowerShell.Management in Windows PowerShell, but not in PSCore6.  Use of the WIndowsCompatibility module is the way to use it.

@SteveL-MSFT  installed failed, maybe I need close this issue and submit another new issue in https://github.com/PowerShell/WindowsCompatibility/issues
~\Desktop> Install-Module -Name WindowsCompatibility
PackageManagement\Install-Package : No match was found for the specified search criteria and module name 'WindowsCompatibility'. Try Get-PSRepository to see all available registered module repositories.
At C:\program files\powershell\6\Modules\PowerShellGet\PSModule.psm1:9491 char:21
+ ...          $null = PackageManagement\Install-Package @PSBoundParameters
+                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (Microsoft.Power....InstallPackage:InstallPackage) [Install-Package], Exception
+ FullyQualifiedErrorId : NoMatchFoundForCriteria,Microsoft.PowerShell.PackageManagement.Cmdlets.InstallPackage

~\Desktop> Get-PSRepository
WARNING: Unable to find module repositories.

@chucklu
PS C:\> Find-Module WindowsCompatibility

Version              Name                                Repository
-------              ----                                ----------
0.0.1                WindowsCompatibility                PSGallery

@alexandair  I will close this issue cause I have submit another issue in https://github.com/PowerShell/WindowsCompatibility

@SteveL-MSFT  bad news, the New-EventLog still can't work after I install the WindowsCompatibility.
~\Desktop> New-EventLog -Source "SupperSocketService" -LogName "TestLog" -MessageResourceFile "C:\Test\TestApp.dll"
New-EventLog : The 'New-EventLog' command was found in the module 'Microsoft.PowerShell.Management', but the module could not be loaded. For more information, run 'Import-Module Microsoft.PowerShell.Management'.
At line:1 char:1
+ New-EventLog -Source "SupperSocketService" -LogName "TestLog" -Messag ...
+ ~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (New-EventLog:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoloadMatchingModule

~\Desktop> Get-Module

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     0.0        build                               {Clear-PSRepo, Compress-TestContent, Convert-TxtResourceToXml, ConvertFrom-PesterLog...}
Script     0.0        chuck                               Reset-FolderTime
Manifest   <IP_ADDRESS>    Microsoft.PowerShell.Management     {Add-Content, Clear-Content, Clear-Item, Clear-ItemProperty...}
Manifest   <IP_ADDRESS>    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}
Script     <IP_ADDRESS>    PackageManagement                   {Find-Package, Find-PackageProvider, Get-Package, Get-PackageProvider...}
Script     <IP_ADDRESS>    posh-git                            {Add-PoshGitToProfile, Expand-GitCommand, Format-GitBranchName, Get-GitBranchStatusColor...}
Script     1.6.7      PowerShellGet                       {Find-Command, Find-DscResource, Find-Module, Find-RoleCapability...}
Script     2.0.0      PSReadLine                          {Get-PSReadLineKeyHandler, Get-PSReadLineOption, Remove-PSReadLineKeyHandler, Set-PSReadLineKeyHandler...}

~\Desktop> (Get-Module Microsoft.PowerShell.Management).Path
C:\program files\powershell\6\Modules\Microsoft.PowerShell.Management\Microsoft.PowerShell.Management.psd1

It's weirdm, there is no module named as WindowsCompatibility.
How can I check if I have installed the WindowsCompatibility successfully?

@chucklu I think you should ask in WindowsCompatibility repo.

Finally I execute the command successfully
Write-EventLog -Source "SupperSocketService" -LogName "TestLog" -Message "test on 20180928" -EventId 3001


updating from 0.8.1 to 0.9.0: concat not supported in sqlite

The concat function (https://www.sqlite.org/lang_corefunc.html#concat) was introduced in sqlite v3.44.0 (https://www.sqlite.org/changes.html).
It seems that when upgrading the sqlite version used changed to an older version that does not support concat as I am now getting:

I guess I will have to rewrite all the concatting functions.

Hey @barbalex
I'm not sure why your wa-sqlite dependancy was downgraded when you upgraded to 0.9.0. wa-sqlite is a peer dependancy and if you install at least 0.9.9 (https://github.com/rhashimoto/wa-sqlite/releases) you will get sqlite v3.44.0.
We have the peer dependancy set to ^0.9.8 and so maybe if you removed your node_modules and lock file it fetched that instead?
You should only have to do: npm install rhashimoto/wa-sqlite#semver:^0.9.9 to get it working (I think)

Yeah, that solves it.
Thanks!


Forge verify does not parse etherscan section of foundry.toml

Component
Forge
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.2.0 (df8ab09 2023-03-14T10:00:26.197329Z)
What command(s) is the bug in?
forge verify-contract
Operating System
macOS (Apple Silicon)
Describe the bug
Calling this command
forge verify-contract --chain polygon <address> <name>
Gives
Error:
ETHERSCAN_API_KEY must be set

after probably this commit. It doesnt work even this way
forge verify-contract --chain polygon --etherscan-api-key <key> <address> <name>
getting the same error

cc @DaniPopes
@debych can you please post your config file.

@mattsse
[profile.default]
test = 'test/solidity'
solc_version = '0.8.17'
optimizer = true
optimizer_runs = 10000
sender = '<sender>'
tx_origin = '<tx_origin>'
out = 'out'
fs_permissions = [
  { access = "read", path = "./config/" },
  { access = "read-write", path = "./test/logs/" },
]
ffi = true
libs = ["node_modules", "lib"]

[rpc_endpoints]
polygon = "${ETH_NODE_URI_POLYGON}"

[etherscan]
polygon = { key = "${POLYGON_ETHERSCAN_API_KEY}", url = "https://api.polygonscan.com/api" }

Is the POLYGON_ETHERSCAN_API_KEY environment variable set? I just ran this:
forge init test
cd test
forge build

printf '\n[etherscan]\npolygon = { key = "${POLYGON_ETHERSCAN_API_KEY}" }' >> foundry.toml

# error: ... not set
unset POLYGON_ETHERSCAN_API_KEY
forge v $(cast address-zero) Counter --chain polygon

# error (after making the request to etherscan): invalid api key
export POLYGON_ETHERSCAN_API_KEY="abcd"
forge v $(cast address-zero) Counter --chain polygon

Yes it is set in the .env file like so
POLYGON_ETHERSCAN_API_KEY=<key>

@debych could you please link the repo or prepare a repro for this?


Build and install Linux and Apple Silicon-targeted versions of kwt

see vmware-tanzu/carvel#218
Do it like we do for ytt.

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

brew install vmware-tanzu/carvel/kwt
kwt: The x86_64 architecture is required for this software.
Error: kwt: An unsatisfied requirement failed this build.

Waiting too long, I do it myself
https://github.com/carvel-dev/kwt/pull/35
Before PR is merged in, can use my formula to install
brew tap vangie/formula
brew install vangie/formula/kwt

Binaries available and homebrew updated


Is the tutorial up-to-date?

Is the showed tutorial (readme.md) up to date? I was trying to follow along with the files-to-be-downloaded etc. and it seems to be not working.

@bernardo-cruz A new V2 version of Watson Discovery was just released on IBM Cloud and is incompatible with this code pattern. There are plans to update this material, but no firm date on that.


User needs to check if they're eligible for a bus pass using the checker but couldn't do so so

Age checker doesn't actually work (maybe it's not supposed to in the prototype so happy to close the issue if that's the case) and always says you're eligible even when you're born after 1952

@irinapencheva Yeah, we don't need this page to actually work in the prototype

@sanjaypoyzer - cool stuff


Is there any way to add outline for text?

AFAIK, there is https://docs.rs/bevy/latest/bevy/prelude/struct.Outline.html
But all my attempts to make it work failed

Outlines seems to be added in 0.12: https://bevyengine.org/news/bevy-0-12/#ui-node-outlines. Looking at Bevy source code, I am not able to find texts with outlines, so I don't think it is possible unless it is implemented in a custom way. You sure that it can be used with texts? I can add it in 0.15 update if it is already something handled by Bevy itself.

Actually I couldn't find any info about texts with outlines. Probably should test it with real ui text

Please let me know if you can find anything, I'll look into it while checking 0.15 PR, and if it is something Bevy already handles, will try to incorporate it.


Appointment command can maybe accept dates with more flexibility?

I was thinking instead of users being 'forced' to type 04/01/2025, for example, maybe they could be given the flexibility to type 4/1/2025 instead? If I am not wrong it is just changing the DateTimeFormatter from DD/MM/YYYY to D/M/YYYY so now it would accept both 04 and 4 as valid inputs
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("d/M/yyyy");
// This will now accept both "04/01/2025" and "4/1/2025".

Oh I see, thanks for pointing out this enhancement, I will make this fix (but maybe on a lower priority after looking at all the critical changes)

i think can close this
is an enhancement now

priority low + probably cannot implement in v1.6


Use MUnit in examples

A final migration on MUnit in tests.
N. B.: some test suites fail in armeria-fs2grpc on the previous scalatest based tests and brand-new MUnit based tests. I've tried to figure out what's wrong - and not found out, unfortunately.

Thanks again! 🙇♂ @danicheg

some test suites fail in armeria-fs2grpc on the previous scalatest based tests and brand-new MUnit based tests.

Is it still produced? Let me take a look soon.

some test suites fail in armeria-fs2grpc on the previous scalatest based tests and brand-new MUnit based tests.

Is it still produced? Let me take a look soon.

Yes, it is.


Generate Scaffolding for Bindings Features

closes: https://github.com/ngrok-private/ngrok/issues/32542
What
Add the scaffolding for new bindings features
See: https://github.com/ngrok-private/ngrok/issues/32542
How
Use kubebuilder CLI
Breaking Changes
Are there any breaking changes in this PR?

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#427  👈
#430
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @hjkatz and the rest of your teammates on  Graphite

Merge activity

Sep 17, 3:11 PM EDT: Graphite rebased this pull request as part of a merge.


Upgrade to Flutter 3.22

Targeting Flutter 3.22.0.
https://github.com/flutter-tizen/flutter-tizen/issues/545

rebased


Bug fix 3.5/ui view editor ro

Same as: https://github.com/arangodb/arangodb/pull/11747

Cannot recreate your behavior. Browser? Os?

Cannot recreate your behavior. Browser? Os?

Google Chrome for Windows. Version 83.0.4103.61 (Official Build) (64-bit).


Broken link in docs

The following link is broken:
https://github.com/ros-controls/gazebo_ros2_control/blob/6458276ca1eaf3ee646afa8e0833386f9785e585/doc/index.rst?plain=1#L289C77-L289C148
Is this section up-to-date? Does one really need Foxy? Can we link to the gazebo_ros2_control_demos at master branch?

you can remove this, the repository doesn't exists https://github.com/ros-controls/gazebo_ros_demos

But where is this launch file located? I couldn't find any tutorial in the moveit2 docs
  ros2 launch rrbot_moveit_demo_nodes rrbot_demo.launch.py

@ahcorde what do you think? or should I remove this section completely?

I would say remove


Fix wildcard with embbeded registry test

Proposed Changes
Thanks @vitorsavian
Fix test broken by combined changes from https://github.com/k3s-io/k3s/pull/9599 and https://github.com/k3s-io/k3s/pull/9556
One PR added tests, another PR changed the logic; the combination of the two was not tested until both were merged to master.
Types of Changes
test
Verification
check CI
Testing
yes
Linked Issues

User-Facing Change

NONE

Further Comments

e2e flaked; merging to fix integration tests


[Ubuntu] Cleanup /root/.cache directory

Description
Some files are left after the image generation in the /root/.cache and can be safely deleted.
Related issue:
n\a
Check list

[ ] Related issue / work item is attached
[ ] Tests are written (if applicable)
[ ] Documentation is updated (if applicable)
[ ] Changes are tested and related VM images are successfully generated

/azp run ubuntu1804, ubuntu2004


Lack of actual CLI implementation

Should support RPC endpoint continuous and the special script mode for qualified users with the extended functionality. It is necessary to provide a standard way of distribution through cargo - cargo install emerald.

@dulanov can you please clarify these points using examples.
for example the "special script mode" would it look like this:
emerald -script "<script>"

What do you want this DSL to do?
Is it not just better to expose the emerald functionality via CLI and programmers can just compose emerald features in bash instead of passing in scripts to emerald to parse?
Or when you use the word "continuous" do you mean there is a background emerald process keeping RPC connections alive and then a short lived CLI process which'll connect to the background process send the action then immediately die.
I'd imagine the latter would be a better way to go?

@sjm first version without script (DSL), we should release something simple from begining.


ddpb366b try out squid proxy

Purpose
Briefly describe the purpose of the change, and/or link to the JIRA ticket for context
Fixes DDPB-####
Approach
Explain how your code addresses the purpose of the change
Learning
Any tips and tricks, blog posts or tools which helped you. Plus anything notable you've discovered about DigiDeps
Checklist

[ ] I have performed a self-review of my own code
[ ] I have updated documentation (Confluence/ADR/tech debt doc) where relevant
[ ] I have added tests to prove my work
[ ] The product team have approved these changes

Frontend

[ ] I have run an in-browser accessibility test (e.g. WAVE, Lighthouse)
[ ] There are no deprecated CSS classes noted in the profiler
[ ] Translations are used and the profiler doesn't identify any missing
[ ] Any links or buttons added are screen reader friendly and contextually complete

Closing as this PR is ancient - we can resurrect if we want to play around with 🦑  in the future.


[Ironsworn] Move vows to their own page

This avoids cluterring the Summary page, and this improves the readability when working with vows, which don't need anythign related to the Summary page.
This would be as in Starforged.
The page Vows would be on the right of the page Summary.

@idispatch75 this was already done but was reverted because of community pushback


Sleigh instruction with conditional prefix

I'm looking to create a set of instructions with the condition prefix "v". (vadd, vsub, vmul etc) I setup a table called VPREFIX and placed it in front of the mnemonic surrounded in quotes and separated by ^. ex : VPREFIX^"add" Unfortunately this causes sleigh to think that every instruction has the same mnemonic and eventually runs out of heap space during compilation. Is there a way I may do this without having to define vadd and add separately when they do the same thing but have a different mnemonic depending on context flags?

try :^VPREFIX^"add"

try :^VPREFIX^"add"

That was the first thing I tried. I keep running out of heap space during compilation which I think is because sleigh is dumping them all into one table and trying to determine which one to use from the patter fields. Basically the mnemonic for all those instructions becomes "".
I believe the correct was is to define them without the prefix and use | in the pattern then do the following.
:VPREFIX^instruction is context {};
:instruction is epsilon {};

When I tried adding both conditions for the instruction (with and without the context register set) and also ran out of heap space. I think I just have an instruction causing trouble somewhere that I need to track down. I'll have to come back to this.

try :^VPREFIX^"add"

Either that or the above is likely correct. It appears there is either something wrong with my definitions or there is a problem rooted in sleigh somewhere.


If third-party cookies are disabled in browser, Safe Apps localstorage is unaccessible and can lead to errors

Description
https://www.chromium.org/for-testers/bug-reporting-guidelines/uncaught-securityerror-failed-to-read-the-localstorage-property-from-window-access-is-denied-for-this-document
There's a restriction in modern browsers that don't allow usage of localStorage in frames if third-party cookies are disabled (example). We should show a warning or create a help center article
Environment

Browser: Chromium-based, Firefox, maybe more

Steps to reproduce

Block third-party cookies in the browser
Open curve.fi safe App

Obtained result
The app shows a blank screen

Can't we just mock localStorage in memory for incognito users?

mmm but this is a problem with the localStorage used in the iframe for third party apps

Ah, my bad.

Adding a doc exposing the problem and possible solutions
https://docs.google.com/document/d/1saWzdAtnfkoGXsu-5B9z6QuK-dVVYeWsLBYXdheySYk/edit?usp=sharing

We have a design proposal based in one of the possible solutions
https://www.figma.com/file/CSMM622zKmfeMs2DUPOILh/Safe-Apps-issue-with-third-party-cookies?node-id=2%3A2

Added a comment in the design over the banner. not sure if you received the email.
What's expected when you press the Edit Cookies link?

@liliiaorlenko @dasanra

What's expected when you press the Edit Cookies link?

I would assume it opens the cookies banner again.

How? But the message is in the cookies banner. I would say some help context or trying to open chome://settings (Not possible from JS)

Sorry, I misunderstood. I guess we could put together a help article to link to?

Yeah I was thinking in detecting browser and redirecting to the help section as I'm not able to open settings in the browser (Seems a Security protection). But wanted Liliia to confirm

If we can change cookie settings, I updated designs with a lead to help article https://www.figma.com/file/CSMM622zKmfeMs2DUPOILh/Safe-Apps-issue-with-third-party-cookies?node-id=19%3A57
Also deleted close banner button, to encourage user deal with a problem if he wants it gone.

ok nice. I had added a 10 seconds timeout for auto remove the banner each time the user access  an app. Should I remove it and show it always?

proposed another behavior

Ok!! What's the place to write the help article @dasanra ? https://help.gnosis-safe.io?
If that's the case. How can I publish there?


Fix truffle console commands output and tab completion

Was just going to ask what this solved, but then ran into issues which this fixed. Thanks!


Fix typo: Minimun -> Minimum

@schlotter,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

@schlotter, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.
Thanks, Microsoft Pull Request Bot


[e2e]: generic schedule main multi-uses slsa3

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/8446490551
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.schedule.main.multi-uses.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.schedule.main.multi-uses.slsa3.yml
Trigger: schedule
Branch: main
Date: Wed Mar 27 04:10:38 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/8477039911
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.schedule.main.multi-uses.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.schedule.main.multi-uses.slsa3.yml
Trigger: schedule
Branch: main
Date: Fri Mar 29 04:10:10 UTC 2024
Tests are passing now. Closing this issue.


Presentation API 2.1 : Adding "rendering" property to IIIF resources

I tried to follow the same conventions as you, such as the javadoc comming from the IIIF spec.
I've also added a "rendering" example in the manifest.json used for test to assert that it does not break the mapper (although it does not assert that the content has been deserialized).
Is it fine ?

Related issue : #8

Codecov Report

Merging #7 into master will decrease coverage by 0.21%.
The diff coverage is 45.45%.

@@             Coverage Diff              @@
##             master       #7      +/-   ##
============================================
- Coverage     54.45%   54.24%   -0.22%
- Complexity      235      240       +5
============================================
  Files            65       67       +2
  Lines           909      931      +22
  Branches         45       45
============================================
+ Hits            495      505      +10
- Misses          387      399      +12
  Partials         27       27

Impacted Files
Coverage Δ
Complexity Δ

...on/model/impl/jackson/mixin/v2/RenderingMixIn.java
0% <0%> (ø)
0 <0> (?)

...pl/jackson/v2/IiifPresentationApiObjectMapper.java
100% <100%> (ø)
3 <0> (ø)
:arrow_down:

...tation/model/impl/v2/AbstractIiifResourceImpl.java
87.09% <100%> (+1.38%)
19 <2> (+2)
:arrow_up:

...iiif/presentation/model/impl/v2/RenderingImpl.java
35.29% <35.29%> (ø)
3 <3> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d06e009...f11edb2. Read the comment docs.

Thank you! And sorry this took so long!


Xamarin.Android.Support.Design is not used anymore with AndroidX

Package to work need to have xamarin.android.support.design library installed. With androidX this library is not used anymore. Is possible to recompile project with androidX support? thanks

@whitedevils  please try 3.0.1

3.0.1 need NETStandard 2.1. If solution works on UWP/IOS/Android it's not possible use NETStandard 2.1. Could you make work on NETStandard 2.0?


Review mobile layout for widget examples

Ping!

Just saw this issues… I'm about to merge widgets branch to v4, you'd need to merge v4 with your WIP branch to be able to finish this one I think, but I suspect that the changes should be done in https://github.com/CartoDB/deep-insights.js instead.
Let me know when you're done @piensaenpixel and I'll help you move the changes if necessary. Sorry for the inconvenience.


Allow moving the side-panel next to the explorer panel

First of all I love the new side panel, thank you for that. I'm really amazed by it's capabilities so far and the execution was well done, especially that I can even put things from the bottom panel into it.
Right now the new side-panel has to be on the opposite side of the explorer panel, but I would like to put the side-panel and explorer next to each other, so I end up with 3 columns like so:
+------------------------------+-------------+-------------+
|                              |             |             |
|                              |             |             |
|                              |             |             |
|        Main Context          | Side Panel  |  Explorer   |
|                              |             |             |
|                              |             |             |
|                              |             |             |
+------------------------------+-------------+-------------+

It would be nice if the side-panel could be moved to the left or right in a similar fashion like the explorer panel or bottom panel.
I would welcome a context menu entry "move side panel to left/right" when right-clicking it:

This would really be helpful to use the full potential of the new side-panel. Some more background info why I want both panels next to each other:
I use an extension to auto-collapse the explorer panel and I use the side-panel for opened editors and outline. I hided tabs and use VSCode in portrait aspect ratio. When I open new files I expand the explorer panel or use a command to open a file and only the side-panel is seen permanently to de-clutter my session, I just keep all relevant files to my session open for quicker access. I wish to have both panels on the right side to make my work-flow simpler.
Thank you!

Looks like duplicate of #142742 that was created from comment on the same issue - #141349

/duplicate of #142742

Yes, you are right, thank you for merging and copying my comment to the relevant issue. Really appreciate it! :)


Overwork Tests

Instead of [Fact], [Theory] may be better…

Most tests have 100 % coverage. But yet not all Classes have a test.


windows defender of windows 10 reported virus

What version of Go are you using (go version)?

$ go version 1.17

Does this issue reproduce with the latest release?
What operating system and processor architecture are you using (go env)?
windows 10 64bit

What did you do?

i build example: go build ./main.go
// You can edit this code!
// Click here and start typing.
package main
import "fmt"
func main() {
fmt.Println("Hello, 世界")
}
and windows defender of windows 10 reported virus and kill it.
Why does the golang program contain viruses?, please fix it.

this is the result when i scan on virustotal.com
https://www.virustotal.com/gui/file/356e4ac39a9ec0946dcecf7df5e184b4615333595ec11e53f3d63b2a98a04d2f/detection

See https://golang.org/doc/faq#virus.


Add !linuxgames command

Add a !linuxgames to get a count of the number of linux games available on steam either dynamically via the storefront API or by perhaps using SteamLinux..

@LeonB has one implementation in PR #15.


multipleOf does not work consistently for negative numbers

Hi,
thanks for the great library!
I tried to use the multipleOf keyword to validate negative fractional numbers.
While this works for some numbers
jsonschema.validate(-0.001, {'type': 'number', 'minimum': -1, 'maximum': 1, 'multipleOf': 0.001})
it fails for others:
jsonschema.validate(-0.059, {'type': 'number', 'minimum': -1, 'maximum': 1, 'multipleOf': 0.001})
ValidationError: -0.059 is not a multiple of 0.001

Failed validating 'multipleOf' in schema:
    {'maximum': 1, 'minimum': -1, 'multipleOf': 0.001, 'type': 'number'}

On instance:
    -0.059

The reason could be that -0.059 / 0.001 = -58.9999 in python but i am not sure.

Hi. Thanks for the kind words!
Yes, this is a property of floating point numbers, nothing specific neither to this library nor even Python.
If you want precise division, you may want to use decimal.Decimal to load your JSON (or in your validation in general), which this library supports.


Process hangs with a specific sequence of add/removes on a set

Description
Compiling a function that should perform a specific sequence of add/remove operations on a set hangs the process. CTRL-C is not enough to abort it, it must be killed via terminal.
To reproduce
The problematic sequence was originally longer and the example below is the shortest I could get that still hangs.
from numba import njit

@njit
def bug():
    tmp = set()

    tmp.add(0)
    tmp.remove(0)
    tmp.add(1)
    tmp.add(2)
    tmp.remove(1)
    tmp.add(3)
    tmp.add(4)
    tmp.add(5)
    tmp.add(6)
    tmp.remove(4)
    tmp.add(7)
    tmp.remove(3)
    tmp.add(8)
    tmp.remove(7)
    tmp.add(9)
    tmp.remove(8)
    tmp.remove(9)
    tmp.add(10)
    tmp.add(11)
    tmp.add(12)
    tmp.add(13)
    tmp.remove(5)
    tmp.add(14)
    tmp.remove(13)
    tmp.add(15)

    return 0 in tmp

print(bug())

Expected behaviour
Regular python version does not hang (in this case it should print out False). Other sequences also generally do not hang.
Additional context

The original sequence had different (larger) numbers. When the process still hanged after replacing them with smaller numbers, I thought that the numbers do not need to be specific, that only their symbolic order is important. However, now replacing e.g. all instances of 15 with 150 in the above example, I cannot reproduce the bug - the process behaves normally.
Swapping the last line in the function for return 0 in tmp.copy() works - the process behaves normally.
I was able to reproduce the bug on a different Linux machine with python 3.9 and the same numba version number.

System/versions
OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Libc version: glibc-2.31

Python version: 3.8.12 (default, Oct 12 2021, 13:49:34)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.0-53-generic-x86_64-with-glibc2.17

# Name                    Version                   Build
numba                     0.56.3           py38h417a72b_0
python                    3.8.12               h12debd9_0

IIRC this is known and there are both issues and PRs about it already, but alas it remains unsolved. Would recommend to check the issue tracker for duplicates.

I was unable to find any issues or solutions regarding this behaviour with sets. My current workaround is to simulate a set by using a typed dictionary, where values are arbitrary, e.g. all 0, since I only need its keys.

I was unable to find any issues or solutions regarding this behaviour with sets. My current workaround is to simulate a set by using a typed dictionary, where values are arbitrary, e.g. all 0, since I only need its keys.

I just had another look, I think this may be a duplicate, or potentially related to : https://github.com/numba/numba/issues/6543

Thank you for finding that issue, I read through it and can confirm that this is a duplicate. Interestingly, my encounter with the bug also came from implementing A* and the last comment is referring to the same solution that I am using.
Since the issue is still open, I think this behaviour should be more known, perhaps noted in the official documentation. Running into a random series of add/removes that causes problems is not something one would expect to happen and a heads-up would help to avoid them.

Closing this issue as it is a duplicate of #6543. A potential fix for this issue is being developed in https://github.com/numba/numba/pull/7875


Upgrade pyqt5 to 5.12 which has support for python 3.8

This will hopefully enable us to run qt related tests on python 3.8 and eventually build the docs too.
Pyqt 5.13 and 5.14 have been released but are not yet supported by spyder

This does mean that the pyqt version used in conda and pip builds will diverge as conda still ships 5.9

This does mean that the pyqt version used in conda and pip builds will diverge as conda still ships 5.9

shall we wait for conda? or do we prioritize availability on python 3.8?

L

This does mean that the pyqt version used in conda and pip builds will diverge as conda still ships 5.9

shall we wait for conda? or do we prioritize availability on python 3.8?

Lets see if this actually works. I am getting tired of waiting on conda. This seems to be very low priority for them

Well this segfaulted with all 3 python versions so ...

Well this segfaulted with all 3 python versions so ...

i'm not sure with what smiley to reply :)

It seems like we are seeing segfaults with pyqt5.12/13/14 for a simple matplotlib plot. It's not clear if this is a qt bug or specifically related to travis and the xvfb setup that we use


Styleparser

Working on the style parser that can parse jsons based on the maplibre style specification https://maplibre.org/maplibre-gl-js-docs/style-spec/ and https://github.com/maplibre/maplibre-gl-js/blob/main/src/style-spec/reference/v8.json.

Added properties from the style spec
Added 'extra' capturing properties that weren't captured in the style spec
Added tests to run against actual style jsons
You don't have to include the work from the map-styles, original, and tests directory

💻 Examples

🚨 Test instructions
cargo test --package maplibre --lib -- style::style::tests shows that 30 tests are passing, 2 are failing. I need to work on getting those two tests to pass - currently the two styles that fail have an array with a mix of integers and floats, they need to be parsed in a way that can accept both types
✔ PR Todo

@kimpham54 I removed the proprietary files and removed the tests. We can create those tests in some other repository hosted outside of maplibre.
For that we need to expose the style parsing functionality. Either through a new maplibre-style crate or by exporting the functions from the existing maplibre crate.
Reviewing now :)

Lots of amazing work on this PR, thanks @kimpham54 !  I wonder if we can point this work to https://github.com/maplibre/maplibre-gl-js/tree/main/test/integration/render/tests -- it has a lot of MapLibre style files that could be used for testing?  Also, any let me know if I can help moving this along.

I think this is a great step forward!
The way I imagined it was that all the parsing and validation of the style spec would be in the maplibre-style-spec repo, creating a packages for every platform.
This way all the style.json files that needs to be tested are there as well.
Later on, if we decide for example to change the underlying architecture to use json scheme it will all be in the same place.
This is just an initial thought though...

@kimpham54 Hi! are you still interested in pushing this forward? :)


Feature request - Pass Request default options

Hi @simskij
Related to #81 , I'm looking into way to set the "headers" globally to overwrite the default" x-www-form-urlencoded".
It seems that it is not (yet) supported https://github.com/k6io/k6/issues/761#issuecomment-782031960 .
I might have a feature request, but before I try to figure how to code it as a PR, I wanted to receive your input, if you think it is valuable.
Currently the package support "options" for setting Postman execution related data & variables (https://github.com/k6io/postman-to-k6#options)
It would help to be able to pass the  Request "options", that are being used in the generated script.js file
import "../../libs/shim/urijs.js";

postman[Symbol.for("define")]({
  name: "Get all list configurations",
  id: "efc9d5ad-c0f1-4f41-ae17-e903fa56f82b",
  method: "GET",
  address: "{{baseUrl}}/lists/v2?$top=100&$skip=0&$count=false",
  headers: {
    "X-ApiKey": "{{username}}:{{password}}"
  },
  post(response) {
    // Validate status 2xx
    pm.test("[GET] /lists/v2 - Status code is 2xx", function() {
      pm.response.to.be.success;
    });
  }
});

The feature request would be pass a Request options object that will extend/overwrite the Postman Request methods like the "headers".
Idea:
--request-options ./request-headers.json
which would contain custom options like { headers: { 'Content-Type': 'application/json' } }
Which would result in the Postman request, that gets extended by the passed on request headers.
This would be "default" options that you pass, which would apply for all requests.
import "../../libs/shim/urijs.js";

postman[Symbol.for("define")]({
  name: "Get all list configurations",
  id: "efc9d5ad-c0f1-4f41-ae17-e903fa56f82b",
  method: "GET",
  address: "{{baseUrl}}/lists/v2?$top=100&$skip=0&$count=false",
  headers: {
    "X-ApiKey": "{{username}}:{{password}}",
    'Content-Type": "application/json"
  },
  post(response) {
    // Validate status 2xx
    pm.test("[GET] /lists/v2 - Status code is 2xx", function() {
      pm.response.to.be.success;
    });
  }
});

Where the "headers" section got extended from
headers: {
    "X-ApiKey": "{{username}}:{{password}}"
  }

to
headers: {
    "X-ApiKey": "{{username}}:{{password}}",
    'Content-Type": "application/json"
  }

Or another would be to provide support in the Postman request for passing that are globally defined.
https://k6.io/docs/javascript-api/k6-http/params/
Example
export default function () {
  let params = {
    cookies: { my_cookie: 'value' },
    headers: { 'X-MyHeader': 'k6test' },
    redirects: 5,
    tags: { k6test: 'yes' },
  };
  let res = http.get('https://k6.io', params);
}

I'm not sure if the Postman request function uses all the "options"?
I tried this, but it does not seem to work, or am I missing something?
Example script.js file
// Auto-generated by the Load Impact converter

import "./libs/shim/core.js";
import { group } from "k6";
import "./requests/Lists/Get-all-list-configurations.js";

export let options = { maxRedirects: 4 , headers: { 'Content-Type': 'application/json' } };

const Request = Symbol.for("request");
postman[Symbol.for("initial")]({
  options,
  collection: {
    baseUrl: "https://example.com/api"
  },
  environment: {
    baseUrl: "https://example.com/api"
  }
});

...

Hi,
Got it! Looks really good. I'll try to process the PRs as soon as possible, although I'm quite swamped at the moment so I can't promise I'll get to it this week. Thanks again for your excellent contributions! 🙇🏼
Best,
Simme

@simskij Take your time. Don't need to rush it. I have made a merged branch with which I am using the PRs combined, so I can use it until you have tested it.
Thank you for the excellent package.