Suggestion: Weblate for Translations

Feature/enhancement description:

Hello,
I'd like to suggest using Weblate for translations, makes it much easier for the translators.
I would then add Swedish. :D

Hey! Yeah, my custom solution doesn't scale very well i got to admit 😁. I'm looking into it!

Hey! Yeah, my custom solution doesn't scale very well i got to admit grin. I'm looking into it!

Also has the following advantages:

keep track of what's missing
automatically push translations to git
simple for translators: Here's a string, now what is that in $language? And then the translator just types it in
keep track of the translators for the credits page (forgetting a translator will get them frustrated and you lose them)

I'm using Weblate to contribute to:

usageDirect (https://codeberg.org/fynngodau/usageDirect)
Traewelling (https://github.com/Traewelling/traewelling)
SchildiChat (https://github.com/SchildiChat/SchildiChat-android)

Works great! Originally convinced Traewelling, they had a similar solution as yours. It was so painful I did not complete my translation. Using Weblate I then did it in a day, IIRC.


Use npm-conf

When https://github.com/kevva/npm-conf/pull/2 is merged you could probably save some LOC by using it.

Yeah maybe. We can consider it when https://github.com/kevva/npm-conf/pull/2 has landed and the module has had some time to mature. I spent a lot of time and manual (cross-platform) testing to get this logic right and I don't have time to do the same rigorous testing again right now. I also have to be careful about performance as the logic is executed during startup and not when imported.

Sure, I agree :). I was curious about the performance though so I set up some benchmarks and it seems that using npm-conf was about 65% faster.
get prefix
128,942,283 op/s » globalDirs (working directory)
76,580,703 op/s » globalDirs (upstream/master)

That is very curious. I don't see how my implementation could actually be any faster, so how can npm-conf be. This module is already doing just a few fs calls. Maybe the ini parser is slow. That's the only thing I can think of. The performance is more than good enough for both though.

Yeah, I'm also somewhat surprised. Your implementation should return at the first fs call on my system too so it's kinda weird.

Going to pass on this. The module works pretty well now, and it's unknown how much problems it's going to cause to switch. This module also handles Yarn, which npm-conf doesn't.


Update contribution.rst with information about labels

this could be added to ivadomed's contribution guideline: https://github.com/neuropoly/spinalcordtoolbox/blob/master/CONTRIBUTING.rst#pr-labels

@joshuacwnewton do you mind adding the information, or, if appropriate, redirect to the centralized contributions in our wiki? thanks!


fixfix some nereids bugs

Proposed changes
Issue Number: close #xxx
Problem summary

add json_unquote and json_extract functions
remove mv releated code in visitPhysicalOlapScan
forbid bitmap and hll type for topn node's sort exprs
HashDistributionInfo of olap scan node should use the slots from output not the full schema
SelectMaterializedIndexWithoutAggregate should use the filter node's output together with the predicate to get the correct mv
forbid SimplifyArithmeticRule for decimal type
make DecimalLiteral's type and value consistent with each other if the value is decimalv2
json_array need support empty argument

Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.1 seconds
stream load tsv:          439 seconds loaded<PHONE_NUMBER>9 Bytes, about 162 MB/s
stream load json:         24 seconds loaded<PHONE_NUMBER> Bytes, about 93 MB/s
stream load orc:          61 seconds loaded<PHONE_NUMBER> Bytes, about 17 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230516132042_clickbench_pr_144709.html

run buildall

run buildall


Transition animation not working on android and WP

The animation "ios-transition" causes the returning page to occupy half of the screen when the back button is pressed. This only happens on Android and WP.
Steps to reproduce:
When I open another page with the code below, the animations works as expected from right to left on IOS, Android and WP, but when the back button is pressed on the opened page, then the parent’ content is rendered only the half.
        this.navController.push(SearchPage, null, { animate: true, animation: "ios-transition" });

I am using Ionic 2.0.0-beta.10
Cordova CLI: 6.1.1
Gulp version:  CLI version 3.9.1
Gulp local:
Ionic CLI Version: 2.0.0-beta.32
Ionic App Lib Version: 2.0.0-beta.18
OS:
Node Version: v4.4.6

Confirmed this is a issue in our e2e tests:

Hi, was this fixed? because I'm running beta.11 on ios 9.3.2 and still getting this issue

Still getting the issue as well. Didn't see it happen on Android though, but on iOS.

@schankam @gadld should be fixed thanks to: #8272


i lost all my container when i restart my rancher

Rancher versions:
rancher/server: 1.6.10
rancher/agent: 1.2.6
Infrastructure Stack versions:
healthcheck: v0.3.3
ipsec:
network-services: 0.9.4
scheduler: 0.8.2
kubernetes (if applicable):
Docker version: (docker version,docker info preferred)
17.09.0-ce
Operating system and kernel: (cat /etc/os-release, uname -r preferred)
Ubuntu 16.04.3 LTS
Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
Bare-metal OVH
Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
single node rancher
Environment Template: (Cattle/Kubernetes/Swarm/Mesos)
Cattle
Steps to Reproduce:
Results:
Hi,
When i restart my bare-metal serveur and restart my rancher server container, i lost my agent and all my stack... how to restart a single-node and never lost all services ?

All state is, by default, stored in the embedded MySQL database, which stores it's data on a VOLUME defined in the Dockerfile (https://github.com/rancher/rancher/blob/v1.6.10/server/Dockerfile#L73). This volume is stored in /var/lib/docker (by default). If you restart the server (reboot) or restart the container (docker restart container_id_or_name), they created volume is still associated with the container and will be used. If /var/lib/docker is cleared, or if you start a new container (docker run), a new volume will be created and there will be no state to be re-used.
If you want to be more flexible, you can bind-mount a host directory to the MySQL directory (http://rancher.com/docs/rancher/v1.6/en/installing-rancher/installing-server/#single-container-bind-mount) or use an external database (http://rancher.com/docs/rancher/v1.6/en/installing-rancher/installing-server/#single-container-external-database).
Closing as I don't think this is a bug, let me know if you have any other questions.

Same thing happened to me while running Rancher on a RancherOS machine.
I was using binding, which can be verified in the output of docker inspect:
...
        "HostConfig": {
            "Binds": [
                "/var/lib/mysql/:/var/lib/mysql"
            ],
...

I was trying to solve issue #11142 , and someone suggested I restarted rancher.
I first did a docker restart to the rancher container, which didn't solve my problem (but didn't erase my configurations either).
Then I tried to reboot the server (RancherOS on DigitalOcean).
When I accessed Rancher again, it was a fresh install, and I realized I lost all my staging and production environments.

@jonathascarrijo The default RancherOS console is ephemeral, and only specific directories are persisted and survive a reboot.


fix: Add pgcrypto for gen_random_uuid

Hey, I ran into an issue with "ERROR: function gen_random_uuid() does not exist."
Added pgcrypto to the database/setup.sh script to sort it out. Could be useful for others. Thoughts?

Good catch, thanks for adding it to the setup!


Display phone number in profiles of contributors with incomplete onboarding

This displays the phone number used for Signal onboarding in the contributors profile, so editors can contact them to assist with finishing the onboarding process.

Fixes #1349

The .incomplete.text is the same for Signal and Telegram besides the channel name. It might be nice to unify them. For example, why is it useful to know the number of someone who has an incomplete onboarding in Signal, but not from Telegram?
I guess we don't have the phone number for our Telegram contributors just their ids so we can't display them...

Yes, exactly!


Not working for recent versions of macOS?

I've tried this on macOS Ventura, but I can see no preview. Does the code need some update, maybe?

Also, maybe you have an idea if after an update it would also help Spotlight see and index Jupyter notebook files? See https://github.com/jupyter/notebook/discussions/6862.

The actual plugin here works for me on Ventura / arm64. Could you please try with nbviewer-app instead? It bundles this project and requires no manual installation of the Quick Look generator.

I've tried nbviewer-app on Ventura / arm64. It does not render all of my notebooks. I also used to use nteract.io and the JupyterLab app on macOS before, and while they give better overview of entire notebook files they are not as quick as a Quick Look plugin called with Command-Space in the Finder.app to quickly check only the first few paragraphs.
So if there is a reasonable way to update the code for this plugin I at least would definitely appreciate!

@deeplook all you need is: xattr -cr ~/Library/QuickLook/ipynb-quicklook.qlgenerator


Minor: Bumped swc to 0.287.

Works on Linux but fails on Windows.
notfiy mentioned .... hmmm where have I seen that before ( #274 )
error[E0308]: mismatched types
    --> C:\Users\runneradmin/.cargo\registry\src\index.crates.io-6f17d22bba15001f\notify-4.0.17\src\windows.rs:282:13
     |
280  |         let ret = winbase::ReadDirectoryChangesW(
     |                   ------------------------------ arguments to this function are incorrect
281  |             handle,
282  |             req_buf,
     |             ^^^^^^^ expected `winapi::ctypes::c_void`, found `libc::c_void`
     |
     = note: `libc::c_void` and `winapi::ctypes::c_void` have similar names, but are actually distinct types

Putting a not here that this is failing because of winapi, which needs a feature flag it doesn't seem to have. Not sure where it's doing this


Question: how best to set the epoch dependent parameters

For a new max epoch run, is there any recommendation for setting epoch related values like: flat_epoch, no_aug_epoch, policy epoch, mixup epoch and stop epoch?
## Our LR-Scheduler
flat_epoch: 29 # 29    # 4 + epoch // 2, e.g., 40 = 4 + 72 / 2
no_aug_epoch: 8 #25 # 8

train_dataloader:
  dataset:
    transforms:
      policy:
        epoch: [4, 29, 50] #[15, 76, 142] #[4, 29, 50]   # list

  collate_fn:
    mixup_epochs: [4, 29] #[15, 76] #[4, 29]
    stop_epoch: 50

Thank you very much for your interest and attention to our work. If you like what we’ve done, please consider giving this repo a star.
The issue you mentioned is actually quite simple. Assuming the total training epochs are 50:
The first 4 epochs are for data augmentation warmup.
Half of the epochs are used for dense one-to-one (o2o), i.e., 50/2 = 25.
The flat epoch is the sum of data augmentation warmup and dense o2o, i.e., 25 + 4 = 29.
In D-FINE, 8 epochs are used for searching EMA, so we follow this setting. As a result, no_aug_epoch = 8, and stop_epoch = 50.
The policy is epochs@[4, 29, 50].
I hope this helps you better understand our work!


Update index.md

Added release note for the release of  Stories Beta

@Adam-Dupaski This is for 13 December release.
There is another PR (https://github.com/mendix/docs/pull/2167) which has some documentation.
I can deal with this when I get in if you have other priorities.


GPU usage with the Transformer model

Description
I've created a custom translation Problem following the example of
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/translate_enmk.py
Everything from data generation on my own data to interactive decoding went fine and I'm very happy with the results! However, while reviewing TensorBoard I noticed that I've got quite low global_step/sec ratio (around 0.26 global_step/sec). My setup is a double GTX 1080 Ti with CUDA and cuDNN. I'm using around 3 million training examples.
nvidia-smi reports low GPU usage in both memory usage as the GPU-Util. In fact, while continuing a training session my nvida-smi reports 0% GPU usage.
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.48                 Driver Version: 390.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:17:00.0 Off |                  N/A |
| 12%   45C    P8    19W / 260W |      2MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  Off  | 00000000:65:00.0  On |                  N/A |
| 19%   47C    P8    14W / 260W |    254MiB / 11175MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    1      1267      G   /usr/lib/xorg/Xorg                            16MiB |
|    1      1322      G   /usr/bin/gnome-shell                          49MiB |
|    1      2063      G   /usr/lib/xorg/Xorg                            86MiB |
|    1      2208      G   /usr/bin/gnome-shell                          99MiB |
+-----------------------------------------------------------------------------+

The only hook I've got is that while supplying --worker_gpu=2 to the trainer I do see
      INFO:tensorflow:worker_gpu=2
      INFO:tensorflow:sync=False
      INFO:tensorflow:datashard_devices: ['gpu:0', 'gpu:1']
      INFO:tensorflow:caching_devices: None
      INFO:tensorflow:ps_devices: ['gpu:0', 'gpu:1']

What is going on here?
P.S. In #390 someone suggested looking at the pcie mode and look at GT/s. I've run
sudo lspci -vv | grep -P "[0-9a-f]{2}:[0-9a-f]{2}.[0-9a-f]|LnkSta:
and saw the following GPU settings, suggesting they do not use the x16 mode? Sorry If this is wrong, I'm not an expert in GPU configs:
65:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1) (prog-if 00 [VGA controller])
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
65:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1)
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

Excuse me for a lot of info and thank you very much for the amazing tensor2tensor lib!
Training commando
t2t-trainer     --data_dir=$DATA_DIR     --t2t_usr_dir=$CUSTOM_DIR     --problem=$CUSTOM_PROBLEM    --model=transformer    --hparams_set=transformer_base_single_gpu     --output_dir=$TRAIN_DIR   --train_steps=20000     --worker_gpu=2

Environment information
OS: Ubuntu 17.10

$ pip freeze | grep tensor
tensor2tensor==1.7.0
tensorboard==1.10.0
tensorflow==1.10.0
tensorflow-gpu==1.5.0
tensorflow-tensorboard==1.5.1

$ python -V
Python 3.6.4 :: Anaconda, Inc.

you may want to delete tensorflow? or put tensorflow and tensorflow-gpu into 2 virtual-env.
I suppose the system is using tensorflow cpu instead of tensorflow-gpu.

@colmantse Thank you! You pointed me in the right direction: as mentioned in
https://github.com/tensorflow/tensorflow/issues/12388
I needed to uninstall tensorflow and tensorflow-gpu and downgrade it to 1.8.0. Now both GPU's are used intensively and the step/sec drastically decreased!
The issue with my old configuration (tf version 1.10.0 and tf-gpu 1.5.0) stays, however. Maybe it's a CUDA issue but I'm not exactly sure.

i find for linux, the best way to install tensorflow gpu is from source, it is extremely easy and guarantee always better performance than with pip.


excluding tokens from authorizable packages. fixes #538

Marked as undocumented to update the "Updated" status in docs


Add activate token support in Identity service

Child of #414
Support user creation with activation token like tempus do, in HAF identity service

Changes are pushed on branch 414_HAF_Authentication_token


Fdroid or Github versions without google restrictions

Until 2019 (I think), background playing for Youtube used to work in Kiwi browser. Then it got specifically restricted due to Google Play store policy / TOS. Since Kiwi is now fully open-source, it should have unrestricted versions available outside (in addition to) play store. Compiled apks can be made available through Github or F-Droid.
A browser should be neutral towards all websites and no artificial restriction should be there for a specific website. If youtube can run in the background and picture-in-picture mode on desktop browsers, then there shouldn't be any browser-imposed restriction in Kiwi. This is also unfair for other video streaming websites like Vimeo, Dailymotion, etc, just because they don't have arm-twisting power like google. This is just my opinion and I believe a unique project like Kiwi should remain independent.

.

@mutageneral wrote:

I do not have the play store on my phone. Can you please put this app on F-Droid? Even if it has the pointless restrictions I don't care, at least I will still be able to actually try out what's seems like a great web browser. I know I could sideload the APK, but a web browser is something important to keep up to date and I would clearly forget to update it

If in principle you do not mind installing nonfree software (and Kiwi Browser, as distributed by Geometry OU, is of course nonfree, see its EULA [0]) with questionable background [1] from Google Play, all that stops you is the absense of a client, that would keep it up to date, then Aurora Store [2] is always to your service.
[0] https://privacy.kiwibrowser.org/home/terms-of-service
[1] https://github.com/kiwibrowser/android/issues/30#issuecomment-583223430
[2] https://f-droid.org/packages/com.aurora.store

@d-g No. You'd be surprised that it's very different.
In the interval, I actually had checked if Kiwi Classic was eligible for integration, and actually it was quite ahead of everything you see in the Chromium world.
Even at the time this GitHub issue you are referring to was opened, Kiwi was one of the only (if not the only one, except Chromium itself) Chromium-based browser that had reproducible, public (directly on GitHub servers) and transparent builds, from the build to the Play Store.
Try building /src (Kiwi Classic) and you'll see that it's one of the rare browser that offered end-to-end build on GitHub on a GitHub Actions, and that even all the file hashes matches 1:1 (except the final signature of course, that is done by Google).
From that perspective, it was really exemplary, way better than even of supposedly open-source browsers (who provide partial patches, etc).
I've spent hours and hours to open-source and share my work, and give the code, and even other browsers took from the code of Kiwi (often without properly or just partially crediting) so please don't say it's non-free because it's very wrong.
https://github.com/kiwibrowser/src/runs/3329899976?check_suite_focus=true
/src was way enough to be included in the actual F-Droid repository (even the dependencies are mirrored!)
/src.next could be built on GitHub directly if to spend a non-trivial amount of work adapting all the flow (or just create a 3rd party F-Droid repo like other do)
Why not, but just not super urgent I think, as the number of users who don't have Google Play where it's enforced on most devices, don't have Aurora, don't have GitHub, don't have Samsung Store, don't have Huawei store, is a bit of a niche (but the work to reach it is almost certain).

About non-free, I was thinking about one thing, maybe we don't refer to the same thing ? Technically Chromium uses compiler binaries that are precompiled by Google, if this is what you meant ?
I think this is a problem for all Chromium browsers, I remember https://gitlab.com/thermatk/Unobtainium tried to solve it, but it seems extremely difficult to do anything more open-source than Chromium itself (and Kiwi is really on-par with that in the /src repo) considering all the dependencies and dependencies of dependencies, or side-tooling that may come with it

I really appreciate how fast you have left a detailed reply (not to mention your magnificent work on providing *droid users with a decent web-browser), but, alas, cannot say the same about how substantive it is.

No. You'd be surprised that it's very different.

What ‘it’ is very different?  I believed that all I said was rather trivial, namely:

Kiwi Browser, that can be downloaded from Google Play, is nonfree software.
In past, geometry.ee was unfortunate to take several decisions on this topic, that attracted public questions on its good motives.
If one wants to install software from Google Play without using its official client, he can use Aurora Store.

What of those is arguable?  Hardly the third.
The second?  Basically, what you are saying is that those questions from the public were invalid.  Okay...

Even at the time this GitHub issue you are referring to was opened, Kiwi was one of the only (if not the only one, except Chromium itself) Chromium-based browser that had reproducible, public (directly on GitHub servers) and transparent builds

...despite that chronology contradicts even your own version of KBʼs history told in a pinned post on Reddit [1], I am not going into dive into that.  Let historians of the future or anyone else, who thinks these nuances have any importance, to dig it.
It does not amend the fact that a particular attitude towards the issue had emerged.  (And thatʼs not to say about some nearly-hysterical posts about Kiwi Browser being a ‘spyware’ I wonʼt even link here.)
[1] https://old.reddit.com/r/kiwibrowser/comments/g3kc1n/kiwi_browser_is_now_opensource_including_chromium/a
Finally, the first?

About non-free, I was thinking about one thing, maybe we don't refer to the same thing ? Technically Chromium uses compiler binaries that are precompiled by Google, if this is what you meant ?

No, while I am aware of that, of course (and, fwiw, actually believe, that anyone aiming to provide more freedom for users who rejects to deal with Chromium at all due to that is making a tactical mistake),
I meant exactly what I said: that the Kiwi Browser itself — the ‘app’, that can be downloaded from play.google.com, is nonfree software.  And I am really surprised that it evoked any objections.
Have your read your own EULA (I left a link above)?  Even if your donʼt (srsly, I am not joking, developers are not always aware of what really happens with their beloved project when it finally gets itself in hands of sales dept. managers ;-), does not the very word (E-U-L-A) suggest that that is a licence for nonfree software?
And there is nothing unusual in fact, that a pile of free code is turned out into nonfree ‘app’ in Google Play Store somewhere in process of that (ostensibly purely technical) transformation.  This is the same approach that your upstream — that is Google — employs for its browser, after all, is not it?

Mhhh you got some valid points.
First I appreciate you taking the time to explain the reasoning.
Yes, there can be critics about the timeline, overall the project changes over time, plus there can be contradictions.
Kiwi is far away from being a masterminded plan and this explains a lot; my views tend to evolve as I get convinced by other people or when I understand why some features are useful to others (e.g. for the source-code, it was originally not planned).
I agree with you, the EULA is a confusing, it was created way very early in Kiwi when the original plan of Kiwi was to launch a search engine (exactly like Brave Search) which was based on top of the typical Bing/Yahoo feeds.
Later on, Kiwi actually got accepted into the program to officially promote Bing so it didn't even make sense to launch Find dot Kiwi, but all the text remained.
Back to the topic of F-Droid, I'll try to figure out a solution and I'll do that once I clarified a bit more the roadmap for 2022 in general.

Hey @kiwibrowser
Really appreciate your efforts for such a wonderful free app. Was wondering if you were actively trying to help put this on f-droid?

@kiwibrowser Did you consider making your own F-Droid repo for kiwibrowser?
No need to get added to the main repo, just create a repo and put the repo URL on the GitHub and also a QR code and people will be able to add the repo and download the app and have updates. See these links on how to make a repo: https://www.f-droid.org/en/tutorials/create-repo https://f-droid.org/en/docs/Setup_an_F-Droid_App_Repo
If you don't want to make your own, there is also the IzzyOnDroid repo, which has many apps that could not get into the main F-Droid repo because of their strict inclusion criteria. You can ask to get added in here: https://gitlab.com/IzzyOnDroid/repo/

No, the point in adding it to the official fdroid repo is that is build fully from source, without any proprietary components.

Nov 30, 2021 4:57:41 AM mutageneral - notifications at github.com @.***>:

@kiwibrowser[https://github.com/kiwibrowser] Did you consider making your own F-Droid repo for kiwibrowser?
No need to get added to the main repo, just create a repo and put the repo URL on the GitHub and also a QR code and people will be able to add the repo and download the app and have updates. See these links on how to make a repo: https://www.f-droid.org/en/tutorials/create-repo https://f-droid.org/en/docs/Setup_an_F-Droid_App_Repo
If you don't want to make your own, there is also the IzzyOnDroid repo, which has many apps that could not get into the main F-Droid repo because of their strict inclusion criteria. You can ask to get added in here: https://gitlab.com/IzzyOnDroid/repo/
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub[https://github.com/kiwibrowser/src.next/issues/91#issuecomment-982127475], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AVRPVUW4TMO77NPU3OILHLLUOQD6TANCNFSM5A3XSI6A].
Triage notifications on the go with GitHub Mobile for iOS[https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675] or Android[https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub].

can we make a checklist for this to happen? thanks!

can we make a checklist for this to happen? thanks!

I, personally, don't believe that FDroid would be an option at this point in time. I do like FDroid, but this could be something that might have to wait until a better time to venture into.
~Ibuprophen

@IzzySoft can you give them a hand please ? normally i would ask you to add it to your repo , but the apks seem to be higher than what you accept

@kiwibrowser, the official repository accepts software that contains advertisement, so http://github.com/kiwibrowser/src.next/issues/91#issuecomment-1067076965 is nonsensical.

@kiwibrowser, the official repository accepts software that contains advertisement, so http://github.com/kiwibrowser/src.next/issues/91#issuecomment-1067076965 is nonsensical.

maybe the library he uses for ads is proprietary? idk

The main revenue source is ofcourse the default search engine Yahoo, & Bing search engine query sharing. We can't really going to see it in fdroid due to possible abide of FOSS guidelines.

I don't remember what was the default web browser engine in Fennec F-Droid, but I know for sure it wasn't an instance of one of the 3 FOSS search engines that exist: whoogle, LibreX and searX. So having a proprietary search engine by default is not an issue for getting in F-Droid.

Could always add to IzzyOnDroid afaik

@FieryMewtwo afraid not: APK exceeds the size limit of 30M per app almost 4 times :wink:

I believe that this issue should be separated into separate parts, because the nonsensical explanation that @kiwibrowser provided would at least be irrelevant regarding addition of entire OSS compilations to GitHub, and that would provide the ability to perform reproducible builds from F-Droid, so submission to F-Droid would be 70% complete.


The variable x will be assigned to int type wrongly

https://github.com/miguelvr/dropblock/blob/0a1f2abc1cb34b9f6c7d3eaf7f232d038691bb58/dropblock/dropblock.py#L50

Can you elaborate?

            if any([x <= 0 for x in mask_sizes]):
                raise ValueError('Input of shape {} is too small for block_size {}'
                                 .format(tuple(x.shape), self.block_size))

            # get gamma value
            gamma = self._compute_gamma(x, mask_sizes)

if any([x <= 0 for x in mask_sizes]): makes the variable x be assigned to int type, which results in mistakes in following code gamma = self._compute_gamma(x, mask_sizes)

I'm pretty sure that the variable x in the list comprehension is limited to its own scope, so it won't affect the input variable x

thanks a lot， maybe the reason is that my python version is 2.7.

ah, yes... I didn't test with python 2.7, only with python 3.6, so I won't guarantee anything with that version.
I'll add a note to the README.


Rio JSON LD Jackson core/databind dependency problem

Current Behavior
Here, I'm depending on rdf4j-rio:rdf4j-rio-jsonld (version 5.0.0-M2, but I think this problem affects 4.3.9 as well; haven't tested).
My RdfTest fails like this:
java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
        at org.eclipse.rdf4j.rio.jsonld.JSONLDWriter.endRDF(JSONLDWriter.java:163)
        at org.eclipse.rdf4j.rio.Rio.write(Rio.java:653)
        at org.eclipse.rdf4j.rio.Rio.write(Rio.java:629)
        at org.eclipse.rdf4j.rio.Rio.write(Rio.java:549)
        at dev.enola.rdf.RdfWriterConverter.convertInto(RdfWriterConverter.java:45)
        at dev.enola.rdf.RdfTest.check(RdfTest.java:89)
        at dev.enola.rdf.RdfTest.testRDF(RdfTest.java:83)
        ... 23 trimmed
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ObjectMapper
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
        ... 31 more

The workaround is to manually add com.fasterxml.jackson.core:jackson-databind:2.13.5 to one's project.
Expected Behavior
It should work without having to manually fiddle with dependencies...
Steps To Reproduce
Clone my project and remove the com.fasterxml.jackson.core:jackson-databind:2.13.5 and you can reproduce it.
Version
5.0.0-M2
Are you interested in contributing a solution yourself?
Perhaps?
Anything else?
My project uses https://bazel.build instead and it's theoretically possible Bazel is interpreting some  dependency differently than Maven or Gradle.

My project uses https://bazel.build/ instead and it's theoretically possible Bazel is interpreting some dependency differently than Maven or Gradle.

Actually, strike that - I doubt that's the cause. Because rdf4j-rio-jsonld only declares a dependency on jackson-core, not jackson-databind, so this does seem odd, no?
It may will be some dependency mess bug ultimately more rooted in the "upstream" https://github.com/jsonld-java/jsonld-java/tree/master than here, of course - but I thought I would still make a start here, first.

I think this might be due to the jsonld module importing the query module as a test dependency, which contains databind.
Btw. RDF4J 5.0.0 will hopefully use Titanium JSON-LD, which doesn't use jackson.

@vorburger I'm trying to track down the cause of this issue and I see that jsonld-java depends on jackson-databind, so rdf4j-rio-jsonld should include this transitively. Any chance you can provide a minimal example project that reproduces the bug?

Btw. RDF4J 5.0.0 will hopefully use Titanium JSON-LD, which doesn't use jackson.

Good to know! FYI I'm on 5.0.0-M2, and happy to upgrade to try it.

I'm trying to track down the cause of this issue and I see that jsonld-java depends on jackson-databind, so rdf4j-rio-jsonld should include this transitively. Any chance you can provide a minimal example project that reproduces the bug?

Thank You! But as my project uses https://bazel.build/ instead and it's theoretically possible Bazel is interpreting some dependency differently than Maven or Gradle. I doubt you have any interest in tracking down a potentially Bazel specific problem. As I've been able to locally solve the problem, unless you want to further dig into this for other reason, let's perhaps just close as not planned? Fine for me!


"Go to public dev portal" icon text is not appropriate

Description:
Text in "Go to public dev portal" icon in dev portal is not appropriate. When user clicks on this button, he is directed to a page where he can select any other dev portal. (not to a public dev portal)

This is something like a public hub where user can select and go to another tenant's dev portal.

So the text "Go to public dev portal" is not appropriate.
The suggested text is "Switch Dev Portal"
Steps to reproduce:
Affected Product Version:

Environment details (with versions):

OS:
Client:
Env (Docker/K8s):

Optional Fields
Related Issues:

Suggested Labels:

Suggested Assignees:

Fixed.


adding configurable http client timeout

What does this PR do?
I'm adding the capability to configure the 5s timeout currently used.
I'm passing around the timeout variable from the user configuration to the place where it is used.
In a following PR, https://github.com/DataDog/datadog-lambda-go/pull/64 I'm adding also a circuit breaker so in case of hitting repeatedly the timeout,
your endpoint won't get called for a while.
Motivation
We observed high latency at certain times and we would like to reduce the time and resources
it takes to resolve requests. This is related to support case 441237.
Testing Guidelines
I changed the default timeout from 5s to a low number and observed the timeouts on the logs as errors.
Additional Notes
This will be followed by a circuit breaker PR, to reduce the calls instead of the high latency situation happens again.
Types of changes

[ ] Bug fix
[x] New feature
[ ] Breaking change
[ ] Misc (docs, refactoring, dependency upgrade, etc.)

Checklist

[ ] This PR's description is comprehensive
[ ] This PR contains breaking changes that are documented in the description
[ ] This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
[ ] This PR impacts documentation, and it has been updated (or a ticket has been logged)
[ ] This PR's changes are covered by the automated tests
[ ] This PR collects user input/sensitive content into Datadog

@lhelman thanks for your PRs! Since #64 contains the commit from this PR, mind me closing this one and use #64 instead?


[MXNET-1181] Added command line alternative to IntelliJ in install instructions

Description
Added Maven package setup instructions as an alternative to IntelliJ. Also removed a duplicate Java setup/install markdown file.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)

@lanking520 @andrewfayres

@mxnet-label-bot add [pr-awaiting-merge]


trigger lifecycle methods on paged viewcontrollers

I want viewWillAppear/Disappear to trigger whenever user switches to relevant segment view - how can I achieve that?
introducing MXSegementedPager to my app caused lots of breakage with good existing code...

Pages are child view controller, viewWillAppear/viewWillDisappear are triggered when the are added/removed from the parent, not on every user switch.
You can subclass MXSegmentedPagerController and trigger these methods manually.


python310Packages.aiortm: 0.6.0 -> 0.6.2

Diff: https://github.com/MartinHjelmare/aiortm/compare/v0.6.0...v0.6.2
Changelog: https://github.com/MartinHjelmare/aiortm/blob/v0.6.2/CHANGELOG.md
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 221560 run on x86_64-linux 1

2 packages built:

python310Packages.aiortm
python311Packages.aiortm


How do I support webpack alias?

postcss.config.js
var nodesass = require('postcss-node-sass');

module.exports = ctx => ({
    syntax: 'postcss-scss',
    map: ctx.options.map,
    plugins: [
        nodesass(),
    ]
})

webpack.config.js, note that alias
const path = require('path');
const MiniCssExtractPlugin = require("mini-css-extract-plugin");

module.exports = {
    mode: 'development',
    entry: './css/index.js',
    output: {
        filename: './bundle.js',
    },
    module: {
        rules: [{
            test: /\.scss$/,
            use: [
                MiniCssExtractPlugin.loader,
                { loader: 'css-loader', options: { importLoaders: 2 } },
                'postcss-loader'
            ]
        }]
    },
    plugins: [
        new MiniCssExtractPlugin({
            filename: "[name].css",
        })
    ],
    resolve: {
        alias: {
            testalias: path.resolve(__dirname, 'css/test-alias'),
        }
    }
};

index.scss
@import "testalias/aaa.scss";

test-alias/aaa.scss
body {
    color: green;
}

error: Error: Failed to find 'testalias/aaa.scss'

Bad news, but thanks, have a plan to support webpack alias?

@yanhaijing alias for what?

Aliasing url and @import in css do in css-loader, alias for @import in sass/scss do in sass-loader. What you need aliases in postcss-loader? We just wrapper around postcss

I see what you mean. my idea is not use sass-loader，only use postcss-loader+postcss-node-sass

@yanhaijing aliases works using css-loader

Look at what I gave you demo, using css-loader + postcss-loader+postcss-node-sass, but don't work

@yanhaijing Please read documentation how write aliases https://webpack.js.org/configuration/resolve/


Atomic operations in PSRAM (esp. with 3rd party libraries)

AFAIK the current atomic operations on the S3 uses native atomics. However this creates a problem: a 3rd party library could be using atomics on their data structure placed on the SPI PSRAM.
In the IDF, they handled it by first checking for PSRAM address, then try acquire the external RAM lock, then do CAS: https://github.com/espressif/esp-idf/blob/c9df77efbf871d4c3ae9fb828778ff8c4ab36804/components/esp_hw_support/cpu.c#L199
I wonder if a similar thing should be done for the S3 (for example, a PR to portable-atomic that adds a feature for a similar implementation?
Based on discussions with @ivmarkov

I don't see a simple way out of this without introducing our own, "PSRAM-safe" atomics. Which will be a usability and interoperability nightmare, because library developers will not likely use it, or even know about it.
Turning portable-atomic into a shim of sorts for us is at least technically possible, but still doesn't solve every issue - for MCUs that can use the core atomic types, those ones can't really be shimmed for us to provide our own implementation.
I don't think we can extend portable-atomic to know about our particular memory mappings.

Do we know if an exception is raised whilst accessing atomics in psram, or does it just silently fail? If we can detect it, that at the very least opens up some options for us.

Do we know if an exception is raised whilst accessing atomics in psram, or does it just silently fail? If we can detect it, that at the very least opens up some options for us.

I doubt any exception is raised. From the APP/PRO CPU POV, accessing PSRAM is like accessing regular internal RAM. And it ends up accessing regular internal RAM after all - thanks to the caching MMU.
With that said, I'm not in the position to discuss a possible fix, as I do not understand the operation of the MMUs to a deeper-enough detail. All I know is that ESP-IDF provides a special xchg atomic implemented in software, which is to be used when PSRAM is enabled, as pointed to by @ProfFan .
With that said, whoever wrote and maintains this is probably in that position. (@bjoernQ - is that you?).
I would be happy to know though why hardware atomics cannot work with PSRAM:

Is it because the two CPUs might have their own separate dram caches for the same psram; why would we even want such a setup?
Or is it because an atomic might span across two cache-lines (the end of one, and the beginning of the other) and as such the writing might actually not be atomic at all?

Anyway, I guess me and @ProfFan are just raising the flag that there is an issue with atomics and PSRAM, and somebody needs to look into it. For both ESP-IDF and baremetal by the way, as the ESP-IDF multicore targets are also happily using only hardware atomics ATM; and once we know of a fix, we should probably change the definition of the upstream multicore STD targets from just using hardware atomics to emitting calls to the atomic intrinsic functions, so that the user code can use whatever ESP-IDF had put in the definition of these intrinsic functions, depending on user settings).
Ah, one more question, couldn't resist:

Would hardware atomics work correctly, if the programmer is ensuring that the relevant PSRAM memory region is accessed only by one of the two CPUs? As in, do we even have a workaround for the current situation...

With that said, whoever wrote and maintains this is probably in that position. (@bjoernQ - is that you?).

Yes, I ported the code and no I don't know. I also doubt an exception is raised there but we can just give it a try
I'm also curious about the "why doesn't it just work"

I'll try and find those answers and get back to you.
In the meantime, the only truly bullet-proof way to solve this right now is to have a target variant where we disable hardware atomics, falling back to portable-atomic or something else to ensure atomicity (note: this is exactly what esp-idf does, when the work around is enabled). I'd really like to avoid doing this though if we can help it.

I'll try and find those answers and get back to you.
In the meantime, the only truly bullet-proof way to solve this right now is to have a target variant where we disable hardware atomics, falling back to portable-atomic or something else to ensure atomicity (note: this is exactly what esp-idf does, when the work around is enabled). I'd really like to avoid doing this though if we can help it.

The question is, is portable-atomic even having a feature that would enable atomics (at least on bare-metal, for ESP-IDF we have other options) to operate correctly.
The critical-section one is tempting, but do you remember why our support for u64 on multicore riscv got kicked out from upstream STD? Because it essentially used disable-all-interrupts-critical-sections-plus-spinlocks and they said using spinloocks for atomic operations defeats their very purpose in the first place.
Also if you notice, the software PSRAM-safe xchg impl in the ESP-IDF does something different... yet similar. They seem to implement the "fallible" xchg variant only, which can sporadically fail. But then the question is, what would the user do when presented with such an xchg impl, other than re-trying in a loop, which suspiciously looks like a cousin to spinlocks anyway...

Would hardware atomics work correctly, if the programmer is ensuring that the relevant PSRAM memory region is accessed only by one of the two CPUs?

I think the answer currently for the S3 is most likely true but I am not sure. It's probably good to open an issue in esp-idf and tag @igrr for help 🍭

for MCUs that can use the core atomic types, those ones can't really be shimmed for us to provide our own implementation.

If we have a custom target (for example xtensa-none-elf-esp32s3) this can be put as a part in core right? As we build core with the application anyway.

The critical-section one is tempting, but do you remember why our support for u64 on multicore riscv got kicked out from upstream STD? Because it essentially used disable-all-interrupts-critical-sections-plus-spinlocks and they said using spinloocks for atomic operations defeats their very purpose in the first place.

I wonder what they did for Cortex-M0/M0+ and such? See https://doc.rust-lang.org/std/sync/atomic/index.html#portability
thumbv6m does not support CAS at all.

I wonder what they did for Cortex-M0/M0+ and such? See https://doc.rust-lang.org/std/sync/atomic/index.html#portability thumbv6m does not support CAS at all.

If Cortex-M0/M0+ is always single-core (is it?) then disable-all-interrupts-one-way-or-another as a strategy does work fine and since it is a single core, you don't have the "is a spinlock a lock or just waiting" issue.
By the way... by reading the opening paragraph in the "Portability section"... namely:
"
All atomic types in this module are guaranteed to be lock-free if they’re available. This means they don’t internally acquire a global mutex. Atomic types and operations are not guaranteed to be wait-free. This means that operations like fetch_or may be implemented with a compare-and-swap loop.
"
I'm now even more puzzled as to why they kicked out our u64 multicore support...
I mean, if we talk about "spinlocks" (that we used in that implementation...)

Are these "locks" in the terminology of that section? <-- I DOUBT this now
Or are these "waits" in the terminology of that section <-- I now believe this is the case, and if so, our spinlocking impl should've not been kicked out in the first place...

If Cortex-M0/M0+ is always single-core (is it?)

No, see RP2040

If Cortex-M0/M0+ is always single-core (is it?)

No, see RP2040

Ah so you mean for rp2040 the use a spinlock+disable-all-interrupts software impl? If yes, then this only reinforces my suspicion, that our u64 atomic impl Shall Be Re-Instated...
I guess at the time (and now still) I was not understanding the subject matter well enough to object their decision on technical ground...

...and if spinlocking+disable-all-interrupts is fine as a strategy, then the critical-section impl of portable-atomic is good enough of a workaround for multicore.

No, see RP2040

Beaten me in 5 secs lol

Ah so you mean for rp2040 they use a spinlock+disable-all-interrupts software impl? If yes, then this only reinforces my suspicion, that our u64 atomic impl Shall Be Re-Instated...

Probably no, see https://github.com/rust-lang/rust/issues/99668

...and if spinlocking+disable-all-interrupts is fine as a strategy, then the critical-section impl of portable-atomic is good enough of a workaround for multicore.

For real-time applications like what we use ESPs for I think this is worse than the IDF way of "best effort" weak CAS, but can live without if it's too complicated to make something similar in Rust.

No, see RP2040

Beaten me in 5 secs lol

Ah so you mean for rp2040 they use a spinlock+disable-all-interrupts software impl? If yes, then this only reinforces my suspicion, that our u64 atomic impl Shall Be Re-Instated...

Probably no, see rust-lang/rust#99668

TL;DR :(
If you can summarize for me,  what does anybody use for CAS in multicore M0?
Because I don't think - whatever is used - can avoid using a loop and a loop is as close to a spinlock as you can get? As both are essentially just wait-and-check OR wait-and-retry which is essentially the same?

...and if spinlocking+disable-all-interrupts is fine as a strategy, then the critical-section impl of portable-atomic is good enough of a workaround for multicore.

For real-time applications like what we use ESPs for I think this is worse than the IDF way of "best effort" weak CAS, but can live without if it's too complicated to make something similar in Rust.

Because the weak cas esp idf impl does not disable interrupts (which is... slow?) or if not why then?
Because a strong cas impl on top of a weak cas would also require a loop anyway...

I would be happy to know though why hardware atomics cannot work with PSRAM

Whilst it's related to the cache/MMU setup, the reason it doesn't work isn't PSRAM is treated too much like flash cache. Unfortunately, the appropriate signals to provide the required locking (which aren't needed when caching flash) were also not implemented for caching PSRAM, meaning the CPU will happily do atomic modifications on PSRAM memory without actually ensuring it's an atomic operation.
As you mentioned, PSRAM is treated just like DRAM in the esp32 and esp32s3, and because of this, we can't use the ATOMCTL register to control what the CPU does when  it executes s32c1l on external memories.

Would hardware atomics work correctly, if the programmer is ensuring that the relevant PSRAM memory region is accessed only by one of the two CPUs? As in, do we even have a workaround for the current situation

Hmm, I'm not sure if this would work, consider the case of running one core but sharing an atomic variable between an interrupt. I believe these would be two different accesses, but I suppose the CPU (being a single core in this case) can only do one thing at a time so access would be sequential?
So in summary, we have no way of detecting this, we need to be proactive when using PSRAM. Essentially for every atomic access, we need to

Check if the variable is located in PSRAM
If it is, also acquire a global spinlock to ensure atomicity

I guess the question is where we do this. I already suggested a different target as a nuclear option but we can probably do better. Maybe we can do better, perhaps with something like -Ctarget-feature=-feature to disable atomics and fall back to libcalls that have the work around in place when psram is being used in the program?

From our discussion in the meeting, we need to determine whether enabling portable-atomic/critical-section even on targets with native atomics will use a critical section. If it does, we at least have a nice-ish work around for the issue as we can detect PSRAM use and enable this feature.
This is not ideal though as it incurs a cost to all atomics, even ones stored in internal memory. Longer term it would be better to have a solution like esp-idf's where it checks where the atomic is before deciding whether to guard access with a spinlock.

As we're using atomic compare_exchange in our multi-core critical section implementation, can we enable portable-atomic/critical-section without things blowing up?

As we're using atomic compare_exchange in our multi-core critical section implementation, can we enable portable-atomic/critical-section without things blowing up?

Ah I forgot about that, good catch. Fortunately, I think we can control this by using core::sync::atomic::* in that impl, and ensuring that the global spinlock gets placed in internal memory via a linker attribute.

we need to determine whether enabling portable-atomic/critical-section even on targets with native atomics will use a critical section.

portable-atomic/critical-section is ignored if the target has atomic (i.e., cfg!(target_has_atomic = "ptr") is true).
I'm open to adding a new cfg to adjust the behavior here, but do not intend to change the default or add a new Cargo feature because:

Due to bad behavior of the cargo v1 resolver, other platform-specific dependencies enable this feature, causing build failures or silent performance degradation.
Some people unconditionally enable the portable-atomic/critical-section feature without reading documentation, e.g., https://github.com/matklad/once_cell/issues/264.
Not officially documented, but in the current portable-atomic, if cfg!(target_has_atomic = "N") is true, portable_atomic::Atomic{I,U}N is lock-free[^1], but changing this behavior risks introducing issues into the code that assumes it.

[^1]:  Except in the case of a compiler bug, such as https://github.com/rust-lang/rust/issues/117305, https://github.com/rust-lang/rust/issues/117306

Thanks @taiki-e :heart:!

I'm open to adding a new cfg to adjust the behavior here, but do not intend to change the default

I think this entirely reasonable, I have a couple of questions:

Do you have something in mind for the impl, or would you like us to have a go and PR to portable-atomic?
What do you think the name of the feature should be?
Do you think there is any scope for a conditional check? For instance we don't need to use the critical section if the atomic is placed in internal memory, so it would be nice to run a small address range check prior before committing to acquiring the lock.

I think there are two options on what implementation to use:

It sounds like using the critical-section impl would be more straight forward I think, but if you have some of the scaffolding ready for the global sharded lock, then it might be worth trying that. Either of these approaches would be acceptable to us.

And even two options on when to use it:

For our specific use case, the latter is more attractive to me. Note that we only need to do this on four targets, xtensa-esp32-none-elf, xtensa-esp32-esp-idf, xtensa-esp32s3-none-elf, xtensa-esp32s3-esp-idf. Are you suggesting special casing some address ranges for these targets, or do you think a more general solution of somehow supplying an address range to portable-atomic?

Is it safe to call interrupt disable instruction from user code in xtensa-*-esp-idf? For example, RISC-V and Arm usually restrict the use of such instructions in user mode. If the same is true for Xtensa esp-idf, there will be no option to use anything other than critical-section.

@ivmarkov Could you clarify this, the details are a bit fuzzy for me.

I think this depends on the actual spec and guarantees. If it is something that is clearly guaranteed in the spec, then I guess you can hard code it, but if it is something like “vendor may change it in the future” or “different for each chip or firmware configuration” then we can't hard code the value into the portable-atomic code.

The silicon has a hardcoded address space for PSRAM, it will never change. It is dependant on the chip configuration (at time of production) this means, the esp32 and esp32-s3 have different address spaces for PSRAM so we'll need a solution that accounts for that (I believe we can achieve this with cfg's). Firmware configuration has no impact.


[UX] Use of AutomaticParameterWarning and "auto" parameters

As things have developed, there's been a few instances where we can set some default in an automatic fashion, however it may not be clear to the user that we are making a hueristic choice, and it would be better if they specified this.
While it was not declared anywhere, I think it makes sense to make some rule set that essentially:

If you are making some "auto" choice for a user, raise a warning indicating what was done and how to silence it.
These should inherit from the AutomaticParameterWarning introduced in #244.

# amltk.exceptions
class AutomaticParameterWarning(UserWarning): ...

class AutomaticBlubWarning(AutomaticParameterWarning):
    """We can deduce that we often want to 'blub' but this can backfire silently when x, y, z."""

# amltk.blub
from amltk.exceptions import AutomaticBlubWarning

def function_that_blubs(is_blub: bool | Literal["auto"] = "auto") -> BlubBatch:
	if is_blub == "auto":
        warnings.warn(
        	"blub blub blub. To silence this warning, please specify"
        	" function_that_blubs(is_blub=...)",
            AutomaticBlubWarning,
            stacklevel=2,
        )

This should remain an issue until:

The UserWarning and automatic parameters in AMLTK have been unified

UserWarning derived from the use of "auto" should be replaced be a unique warning type.

Functions relying on the f(x: T | None = None) that make heuristic choices should be converted to use f(x: T | Literal["auto"] = "auto"). It's important to distinguish the absence of a value with the activation of an automatic heuristic. It might be good to give examples here:

# Here the `None` indicates the absence of a value and some default directory/name is fine.
def f(working_dir: Path | None = None):
	"""Uses timestamp if left as None"""
	...

# Here the `None` is indicating the absence of a choice.
# This applies a heuristic to make a choice and should be indicated as such
def run(display: bool | None = None):
	"""Display the progress. By default, this is true in a notebook, otherwise not."""
	...

# Prefer instead:
def run(display: bool | Literal["auto"] = "auto"): ...

@LennartPurucker Please let me know what you think of this when you have time. It's come up a few times now

I am actually not sure about the first case either:
# Here the `None` indicates the absence of a value and some default directory/name is fine.
def f(working_dir: Path | None = None):
	"""Uses timestamp if left as None"""
	...

If this is a private function, None is good. But if this is a public function, then I would also set this to "auto" or the option "timestamp". Right now, from the function definition, while using the function in an editor, I could not tell what happens by default without reading the docstring.
Sidenote: IMO, the typical behavior of defaulting to timestamp from a None input is very unintuitive sometimes and actually not fully thread/multi-processing safe! Thus, I would rather know about this as a user. If the function prints the paths anyway, an additional warning might not be needed.

Otherwise, this is a very good thing. also, I am not yet fully sure about the custom warnings. I guess custom warnings are better code quality and allow for catching/ignoring them, right?

Otherwise, this is a very good thing. Also, I am not yet fully sure about the code overhead of custom warnings. I guess custom warnings are better code quality and allow for catching/ignoring them, right?

I would consider it low overhead on the development side and yes, it allows for distuinguishing them more easily. Anything that wants to interact with that warning can do so by type rather than by content. This allows us to for example, change the wording in the warning without breaking any system that relies on the type for functionality.
This also applies to errors:
# Needed to make sure a certain exception occurs
# Will break if someone decides to make it a proper sentence and capitalize "Blub"
with pytest.raises(ValueError, match="blub should be provided"):
    ...

# Only breaks if the underlying exception type changes
with pytest.raises(BlueError):
    ...

You can see this in effect in one of the doc hooks, which disables warnings from the running examples in the doc strings:
https://github.com/automl/amltk/blob/0af8f7fe77cebb87e5524ad1934de22a74f0b338/docs/hooks/cleanup_log_output.py#L21-L31

From our in-person discussion:
Cases:

Optional (e.g., sample weights)
Defaults (e.g., working dir)
Autos (e.g., visualization)


Merge branch deployed on elizabeth's branch

Report downloads with analysis from a report

Sorry about all the arbitrary commits - I was trying to get it deployed on Heroku and had to try so many times to figure out what was going on 😢

The image analysis may not be working well. We need to make sure that the paths are correct.
We also need to delete old datasets. I will add these issues to the project so we can work on them after submission.


Why do I get a 404 for the documentation???

I signed into the editor via GitHub. The editor works fine, but when I click on documentation I get a 404 error
https://bootstrapemail.com/docs/introduction
404
There isn't a GitHub Pages site here.
If you're trying to publish one, read the full documentation to learn how to set up GitHub Pages for your repository, organization, or user account.

Working on migrating to a different docs repo as we speak. Bear with me 🙏  should be back up shortly

Should be back up, thanks for your patience 👍


[oatpp-sqlite] port added

Other oatpp modules hav already been included and this sqlite one left and now here it is

After seeing the build failure on UWP , I reported to the main author and he replied. Here is the link:
https://github.com/oatpp/oatpp-sqlite/issues/3#issuecomment-712106236
As you see, build fail at him too. Please investigate the problem at vcpkg.

The failures on x64-uwp and arm-uwp:
5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42196,56): error C2065: 'AreFileApisANSI': undeclared identifier [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42196,56): warning C4312: 'type cast': conversion from 'int' to 'sqlite3_syscall_ptr' of greater size [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42226,52): error C2065: 'CreateFileA': undeclared identifier [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42226,52): warning C4312: 'type cast': conversion from 'int' to 'sqlite3_syscall_ptr' of greater size [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42235,52): error C2065: 'CreateFileW': undeclared identifier [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42235,52): warning C4312: 'type cast': conversion from 'int' to 'sqlite3_syscall_ptr' of greater size [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42385,52): error C2065: 'GetFileSize': undeclared identifier [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42385,52): warning C4312: 'type cast': conversion from 'int' to 'sqlite3_syscall_ptr' of greater size [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42531,53): error C2065: 'HeapValidate': undeclared identifier [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42531,53): warning C4312: 'type cast': conversion from 'int' to 'sqlite3_syscall_ptr' of greater size [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42548,53): error C2065: 'LoadLibraryA': undeclared identifier [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42548,53): warning C4312: 'type cast': conversion from 'int' to 'sqlite3_syscall_ptr' of greater size [D:\buildtrees\oatpp-sqlite\x64-uwp-dbg\src\sqlite.vcxproj]
     5>D:\buildtrees\oatpp-sqlite\src\d2e5962be1-146fcc8dbb.clean\src\sqlite\sqlite3.c(42557,53): error C2065: 'LoadLibraryW': undeclared identifier

The error occurred when building sqlite3.c file.
Since sqlite3 has been provided in vcpkg now.
We should depend on sqlite3 in vcpkg to build oatpp-sqlite instead of building the source codes embedded in oatpp-sqlite here https://github.com/oatpp/oatpp-sqlite/tree/master/src/sqlite.
So could you please update this to rebuild this again?

fixed but failed. For the failure, is there any informative log that need to shared here ?

CMake Error at D:/downloads/tools/cmake-3.18.4-windows/cmake-3.18.4-win32-x86/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find SQLite3 (missing: SQLite3_INCLUDE_DIR SQLite3_LIBRARY)
Call Stack (most recent call first):
  D:/downloads/tools/cmake-3.18.4-windows/cmake-3.18.4-win32-x86/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  D:/downloads/tools/cmake-3.18.4-windows/cmake-3.18.4-win32-x86/share/cmake-3.18/Modules/FindSQLite3.cmake:52 (find_package_handle_standard_args)
  C:/a/1/s/scripts/buildsystems/vcpkg.cmake:536 (_find_package)
  CMakeLists.txt:108 (find_package)

Please check the log here
https://dev.azure.com/vcpkg/public/_build/results?buildId=44522&view=artifacts&type=publishedArtifacts

Fixed and builds are in success now

1.2.0 is only version number main author gives us. No bump version we get to display

1.2.0 is only version number main author gives us. No bump version we get to display

You're right, the version number is from upstream. But we also need to bump CONTROL file. In this case, we can add Port-Version: 1. Since there are changes in these ports.
Please see this doc https://github.com/microsoft/vcpkg/blob/master/docs/maintainers/control-files.md#port-version

Port-Version: 2 to oatpp and other oatpp modules except oatpp-sqlite , right ?

Port-Version: 2 to oatpp and other oatpp modules except oatpp-sqlite , right ?

All Port-Version: should be 1, except oatpp-sqlite.

Thanks again!


Heroku discontinuing free plans

Hey Scott, I just came across this news that Heroku is ending all of their free offerings: https://techcrunch.com/2022/08/25/heroku-announces-plans-to-eliminate-free-plans-blaming-fraud-and-abuse
I'm guessing this impacts anyone who has this hosted there. Is there an alternative offering to host the app you'd recommend? Cheers!

Unfortunately I don't have any recommendations for free hosting. There are plenty of ways to do it, but all require a lot more work to do.
My recommendation if you want it to be as cheap as you can without managing servers etc is to just use YNAB Direct Import, and if you really want webhooks, then use Sync for YNAB.


Onboarding updates

Disclaimer moved from the first-page load to first trade
Updated MetaMask label
On SignUp flow, show Buy DAI + Test Bet modals
Help Center added to Mobile Menu

I assume this is still in development and disabled.


upgrade rules using updated script

#837 and all the comments broke the GitHub web UI...
Repeating the work now here using a script and fixing things locally - for the most part.

See https://github.com/mandiant/capa-rules/pull/839/commits/409da0b775bd467d70826e5149974f3b2fffdbc3 for some first updates to rules with subscopes. Is that what you had in mind @williballenthin @yelhamer?

If you've looked at a dozen or so changes from each commit (as they roughly group changes) I'm fairly confident in these updates and don't think we need a detailed review of each rule.

ah, that's a great way to group changes. i'll do this review tomorrow. thanks!

[ ] ensure there are no more TODOs associated with scopes

@mr-tz please see dbefb55

Is VMCI_SOCKETS_GET_AF_VALUE a valid argument to socket?
The other changes look good, thank you!

via here:  https://github.com/mandiant/capa/blob/210a13d94ea40114e01b40927c77c1c74047780f/capa/rules/init.py#L544
an instruction subscope acts implicitly like an and: so we can save a line/indentation by not adding those. same with call. maybe we can do a cleanup of all the rules in another PR.

Is VMCI_SOCKETS_GET_AF_VALUE a valid argument to socket?

ah, you're right, this is only passed to DeviceIoControl/ioctl. maybe the logic should look like:
      - and:
        - os: windows
        - api: DeviceIoControl
        - number: 0x81032068 = VMCI_SOCKETS_GET_AF_VALUE
        - optional:
          - api: socket
      - and:
        - os: linux
        - api: ioctl
        - number: 0x7B8 = VMCI_SOCKETS_GET_AF_VALUE
        - optional:
          - api: socket

For call scope the optional socket would never match and the scopes should still be thread (and function) IMO.

via here: https://github.com/mandiant/capa/blob/210a13d94ea40114e01b40927c77c1c74047780f/capa/rules/init.py#L544
an instruction subscope acts implicitly like an and: so we can save a line/indentation by not adding those. same with call. maybe we can do a cleanup of all the rules in another PR.

I don't see that implemented for call, but think we should add and simplify the rules.

rebased to master and:

changed scopes again for VMCI socket: 5e2dae155ff67edc7489dc779c6f8159285aecb9
updated scopes for new rules:<PHONE_NUMBER>7bf3c43e41067b0f986f1be55449ee
graduated rule: 305adfd16bab08b2526596e56b50b638d8f4bf43


Include tags from other branches

Describe the bug
Tags which are on release branches and not on the default branch are ignored and the commits are marked as Unreleased. We are migrating to release branches at the moment and therefore there are tags on the main branch.
To Reproduce
Steps to reproduce the behavior:

Create repository
Create some commits and tags on the main branch
Create release branch
Do some changes on the release branch
Set the tag on the release branch
Execute git tags --list with the pattern used in the config file

Expected behavior
The git tags are included as versions in the changelog. The diff for the commits is done by comparing 2.3.4...6.5.4
System:

OS Information: Ubuntu WSL2
Project Version: 0.5.0

Hi! If you want to include tags in a specific branch in your changelog, you can checkout to that branch. If this isn't what you want, can you clarify your use case with examples?

6. Execute git tags --list with the pattern used in the config file

Why this step is required for reproducing this problem?

The diff for the commits is done by comparing 2.3.4...6.5.4

I didn't get what you mean. What versions are you talking about?

I just wanted to tell that I am preparing an example to underline what I mean.  Section B is now combined with A"
git branch -d feature/ab

# Create release for 2.0.0
git checkout -b release/v2.0
git tag --annotate v2.0.0 --message "Release of version 2.0.0"

user@machine:/projects/git/some-repository$ git cliff
# Changelog

All notable changes to this project will be documented in this file.

## [2.0.0] - 2022-02-23

### Bug Fixes

- Add footer

### Features

- [**breaking**] Section B is now combined with A

## [1.0.0] - 2022-02-23

### Features

- Section A in README
- Section B in README

### Miscellaneous Tasks

- Init git cliff

<!-- generated by git-cliff -->

As you can see the tag v1.0.1 is not listed here.
user@machine:/projects/git/some-repository$ git log v1.0.0...v1.0.1
commit 3b4a83ebeafb8 (tag: v1.0.1, release/v1.0)
Author: me<EMAIL_ADDRESS>Date:   Wed Feb 23 18:17:08 2022 +0100

    fix: add footer

    (cherry picked from commit b713025b0b15)
user@machine:/projects/git/some-repository$ git log v1.0.1...v2.0.0
commit 6a5d3d6ad541 (HEAD -> release/v2.0, tag: v2.0.0, main)
Author: me<EMAIL_ADDRESS>Date:   Wed Feb 23 18:17:11 2022 +0100

    feat!:  So in order to include tags from another branch, you should simply have them in the branch that you are working on. It works like the git log --simplify-by-decoration command while parsing the commit history:

--simplify-by-decoration: Commits that are referred by some branch or tag are selected.

When you run git-cliff on the main branch, only the commits that are in the main branch are used to create a common Release object. During this process, each tag is being evaluated for associating it with a commit. However, since your v1.0.1 tag is not on the main branch, it cannot be associated with a commit and it is simply ignored.
* 07da03a (HEAD -> release/v2.0, tag: v2.0.0, main) feat!: Section B is now combined with A
* 5a09ecf fix: add footer
| * a814f87 (tag: v1.0.1, release/v1.0) fix: add footer ---------------> tag is here
|/
* 920934b (tag: v1.0.0) feat: Section B in README
* 59e6734 feat: Section A in README
* 65cd0fb chore: Init git cliff
* 7f21e6e Initial commit

In this case, I recommend creating a tag after you checkout to the main branch.
* 07da03a (HEAD -> release/v2.0, tag: v2.0.0, main) feat!: Section B is now combined with A
* 5a09ecf (tag: v1.0.1) fix: add footer ------------------------------> tag should be created here
| * a814f87 (release/v1.0) fix: add footer
|/
* 920934b (tag: v1.0.0) feat: Section B in README
* 59e6734 feat: Section A in README
* 65cd0fb chore: Init git cliff
* 7f21e6e Initial commit

A possible question is, why can't git-cliff do this? Well, including tags from different branches mean including commits as well which will further complicate the usage and might have side effects due to combining n different histories. Maybe this could be done after #13 since that issue also requires combining histories.

@orhun  I completely agree, including other unvisible branchs from the main branch when going through the log, is pretty darn complicated. It will boil down to basically parsing the main branch then parsing all other refs to see if they merge into the main branch, and then include these sub-changelogs into the main one. Its not easy, but should if implemented be done in such a way. Its a hard and non-trivial task.
I came also across these issues with https://github.com/release-lab/whatchanged which does something, but actually not really correct. Meaning the above description.

You can already do this now, but not in an optimal way.
Assuming you are maintaining a product with some form of support contract where you have to backport bug and security fixes for the supported branch and maintain that for a specific time period, then what happens is that you at one point branch off the support branch from the main branch and you keep the main branch as the development branch, backport only fixes to the new branch, and eventually stop supporting it (or some variant of this).
Assuming these branches are named v1 and v2 (most likely v2 would be named "main", "master" or "development").
Then you can generate the release notes for v1 with git cliff v1... If you deleted the v1 branch you can use the latest tag from v1 instead.
Then you can generate the release notes for v2 with git cliff $(git merge-base v1 v2)..v2, which will specify the range since v2 branched off from v1 (or the other way around).
If you also have a v3 branch you can keep doing this git cliff $(git merge-base v2 v3)..v3
You can append/prepend these to the same output, but they will be individually sorted by date, so all the v2 releases before all the v1 releases :(
I would love if if we could support multiple commit ranges, like we now support multiple repositories, ex: git cliff v1.. $(git merge-base v1 v2)..v2 $(git merge-base v2 v3)..v3. Or better yet if git cliff could handle it for us git cliff --branches "v1,v2,v3"


Addition of the OAuth2 Refresh Token flow

Following the guidelines of https://dev.twitch.tv/docs/authentication#refreshing-access-tokens

Thanks!


Is need text in training corpus or only wav files?

Thanks you!

Both text and audio are required.

Is can mix languages? or English only is supported?

Cross-lingual conversion will be possible if it's trained with a dataset of multiple languages. This repo does not directly support such training strategy, but you'll be able to try that by changing some parts of the code!

I have five chinese speakers, make training is OK. How long each voice should have? How many speaker should have?


README for EV3D4

This is for issue #239
The commits are a little screwed up right now, I'll work on cleaning this up.  I had a commit I had made to the changelog but never did a pull request for it so in an attempt to 'git rebase -i' to drop that commit I screwed something up so that it is now showing 48 commits.

Yeah, looks like you rewrote a fair bit of history :laughing: I suspect that a git rebase origin/master would fix the issue. If not, there's always the git diff > tmp.patch && git reset --hard master && git apply tmp.patch option. Don't count on my impromptu git-ing there, though.

https://xkcd.com/1597/

Maybe @dlech's 1337 skills can get us out of this mess :laughing:

Am going with the nuke option...will re-issue this in a second


Relocate Smartthings multi-sensor to gate

Relocate Smartthings multi-sensor to the garden gate. This is because these sensors are much better for security and perimeter doors/windows due to the accelerometer where it can act as a tamper sensor. Use an Aqara door contact sensor to replace wherever it is taken from.

Sensor swapped. Might need to add some form of weatherproofing to this sensor as it is fully exposed right now.

Consider including this in #189.


Use chained exception context when raising NamingException

This can make it easier to debug the underlying issue.
NamingException is an older exception and the API is a bit different compared to other exceptions.

If you have a reason for suppressing the underlying exception, feel free to close this.

Good hint. Coming soon with 0.18.0.


Add Default Client TimeOut.

Add connectRequestTimeout，connectTimeout，readTimeout。

@wzwllu Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@wzwllu Thank you for signing the Contributor License Agreement!


How to extend mix?

Any clue on how to extend Mix like it's possible with elixir?
const elixir = require('laravel-elixir');
const gulp = require('gulp');
const svgmin = require('gulp-svgmin');

elixir.extend('svgmin', (filename, options) => {
    new elixir.Task('svgmin', () => {
        gulp.src('resources/assets/svg/' + filename)
            .pipe(svgmin(options))
            .pipe(gulp.dest('public/svg/'))
    })
})

Mix doesn't have any plugin ecosystem. If you need to extend it, copy Mix's webpack.config.js file to your project root, and then modify it directly like any other project that uses Webpack.

I found it relatively easy to extend over the CustomTasksPlugin.
Checkout laravel-mix-pug to get the idea on how to do it.
Great work @JeffreyWay !


Update kine to v0.8.1

Proposed Changes
bump kine to v0.8.1
Types of Changes
version bump
Verification
normal CI / QA tests
Linked Issues

#3660

User-Facing Change

K3s will now ensure that the write-ahead log for the embedded sqlite datastore does not grow excessively large.

Further Comments

Codecov Report

Merging #4569 (24defe3) into master (0c1f816) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4569   +/-   ##
=======================================
  Coverage   12.03%   12.03%
=======================================
  Files         135      135
  Lines        9179     9179
=======================================
  Hits         1105     1105
  Misses       7838     7838
  Partials      236      236

Flag
Coverage Δ

inttests
0.76% <ø> (ø)

unittests
11.60% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c1f816...24defe3. Read the comment docs.


PEP 498: unparenthesized expressions

PEP 498 expresses that the f-string expr contents will be parsed as ( + body + ). The primary reason for this is omitting the leading whitespace (like (   1)) which already done using the trim() here, but one side effect of this usage is, it permits using things like def foo(): f'{yield}' which is unfortunately not available on RustPython. https://www.python.org/dev/peps/pep-0498/#id7

Some usages from real world: https://github.com/taolluo/simpy/blob/2973dbe7b702dc416e0b7f6bb1ed941a325f862f/tests/test_resources.py#L637-L640


Refactor code to use one entrypoint for addons and operators and unify logic

unify user input verification
unify action execution

/verified


Don't run with last release of yt-dlp-exe

It is not possible to initiate a download with the latest version of yt-dlp.exe (2<IP_ADDRESS>).
On the other hand, it works with the 2023 version (2<IP_ADDRESS>).
I have not tested other versions of yt-dlp.exe.

Maybe this is related?
#145

Hi,
It's not exactly the same thing.
When I use the latest version of  yt-dlp.exe, if I start analyzing the file given in a link, the software runs idle for an indefinite time without downloading anything.
The software uses between 0 and 1.5% CPU.
If we press the "Cancel" button, the software crashes. I use the release 2023-03-28 of the gui.

I fixed it by deleting youtube-nsig and youtube-sigfuncs folders from downloads location


Simpler syntax for reg-sub

I've been using a macro that allows me to write subscriptions like this:
(r-sub :result [] [output [:get-in [:output]]
                   words [:get-in [:words]]
                   hanja [:get-in [:hanja]]]
       (query-result words hanja output))

Arguments are:

subscription keyword
vector of direct subscription arguments, equivalent to fn argument vector
vector of input subscriptions, syntax equivalent to let, direct subscription arguments can be used
computation body

Here's the macro output for the above example:
(re-frame.core/reg-sub
  :result
  (fn [[_] _] [(re-frame.core/subscribe [:get-in [:output]])
               (re-frame.core/subscribe [:get-in [:words]])
               (re-frame.core/subscribe [:get-in [:hanja]])])
  (fn [[output words hanja] [_] _] (query-result words hanja output)))

Here's another example:
(r-sub :char-result-group [i] [char-results [:char-results]]
       (get char-results i))

; macro output
(re-frame.core/reg-sub
  :char-result-group
  (fn [[_ i] _] [(re-frame.core/subscribe [:char-results])])
  (fn [[char-results] [_ i] _] (get char-results i)))

You can also use the direct subscription arguments as arguments to subscription inputs, something that is afaik not possible with the sugared syntax of reg-sub:
(r-sub :num-char-results-in-group [i] [char-result-group [:char-result-group i]]
       (count char-result-group))

; macro output
(re-frame.core/reg-sub
  :num-char-results-in-group
  (fn [[_ i] _] [(re-frame.core/subscribe [:char-result-group i])])
  (fn [[char-result-group] [_ i] _] (count char-result-group)))

If you need access to the whole app-db, leave the input subscriptions vector empty and access db:
(r-sub :get-in [path] []
       (get-in db path))

; macro output
(re-frame.core/reg-sub :get-in (fn [db [_ path] _] (get-in db path)))

I find this syntax much less alien than reg-sub. It also looks and works a lot like fn / let, which most users will be familiar with.
Here's the macro:
(defmacro r-sub
  [sub-name args inputs body]
  (let [query-v (into ['_] args)
        partitioned-inputs (partition 2 inputs)
        input-query-vectors (mapv second partitioned-inputs)
        input-names* (mapv first partitioned-inputs)
        input-names (if (empty? input-names*) 'db input-names*)
        input-fn (list 'fn [query-v '_]
                    (mapv #(list 're-frame.core/subscribe %)
                           input-query-vectors))
        value-fn (list 'fn [input-names query-v '_]
                    body)]
    (if (empty? input-query-vectors)
      (list 're-frame.core/reg-sub
        sub-name
        value-fn)
      (list 're-frame.core/reg-sub
        sub-name
        input-fn
        value-fn))))

What do you think?
One concern might be that you can't plug in a pure function in stead of the body. I'm still thinking whether there's a valid use case for something like this:
(r-sub :foo [a b] [c [:c] d [:d a]] some-function)

With the above macro this subscription would just return whatever some-function evaluates to. However, the syntax could be extended such that if the body is not a list but a symbol, it's treated as a function, and the subscription effectively calls this function with all arguments as specified to calculate the subscription value. Here's an adapted macro that does that and the macro output for the above example:
(defmacro r-sub
  [sub-name args inputs body]
  (let [query-v (into ['_] args)
        partitioned-inputs (partition 2 inputs)
        input-query-vectors (mapv second partitioned-inputs)
        input-names* (mapv first partitioned-inputs)
        input-names (if (empty? input-names*) 'db input-names*)
        input-fn (list 'fn [query-v '_]
                    (mapv #(list 're-frame.core/subscribe %)
                           input-query-vectors))
        body* (if (list? body)
                body
                (concat (list body) args input-names*))
        value-fn (list 'fn [input-names query-v '_]
                       body*)]
    (if (empty? input-query-vectors)
      (list 're-frame.core/reg-sub
        sub-name
        value-fn)
      (list 're-frame.core/reg-sub
        sub-name
        input-fn
        value-fn))))

; macro output for the above example
(re-frame.core/reg-sub
  :foo
  (fn [[_ a b] _] [(re-frame.core/subscribe [:c])
                   (re-frame.core/subscribe [:d a])])
  (fn [[c d] [_ a b] _] (some-function a b c d)))

Hi @mbertheau this definitely looks interesting. Would you be able to update your post to show the equivalent reg-sub that would be written (where it is possible) so we can compare and contrast?

One concern might be that you can't plug in a pure function in stead of the body. I'm still thinking whether there's a valid use case for something like this:

Yep, people have cases where we want to unit test a function, so we need to be able to separate it from the reg-sub.

@danielcompton Thanks for your feedback. I updated the post as you suggested.
Re: passing a function instead of a body to the macro: I added a version of the macro that supports that, however I think that actually subscribing and testing subscription results as demonstrated in https://github.com/Day8/re-frame-test would neatly cover unit testing.

@mbertheau  thanks for this suggestion.
What you propose is a fairly thin layer over the top of reg-sub.  And, in the process it does introduce the cost of a further micro DSL.
As a result, I'm not sure I can see the cost/benefit working.  I've actively resisted all macros to this point.
So I'm inclined to not include this in core re-frame.   But please don't be put off trying to help.  I appreciate the effort.

@mike-thompson-day8 Of course, no problem. Nothing stops me from using re-sub in my own projects :)
Could you elaborate a bit on the cost of a further micro DSL?

Every macro introduces a DSL.   To program it, you have to understand the syntax and associated symantecs involved in the macro.  Because it doesn't follow normal clojure rules, you can't understand it without docs.
So that's a cost you have to weight against the benefits the DSL/macro brings.

Ok, good to know I didn't miss any of the cost aspects. For me the cost/benefit works out well, but for re-frame that's not my call obviously :-)


[wip] Even more crop

So far what I've done here is to make it so that cropping only occurs if the original SRS is WGS84. This gets us out of traps where the original SRS doesn't have a cartesian "extent" that we can project [-180, -85.05113, 180, 85.05113] into (e.g. a lambert conformal conic projection). Apologies, I'm having trouble really articulating what this problem is...
I also added more tests to compare the outputs from node-wmshp against outputs from ogr2ogr. All the tests pass except for the one where the input is Antarctica in a polar stereoscopic projection. What this case represents is one where the source data falls outside the EPSG:3857 boundaries, but the source SRS is not WGS84. In this case we fail with:
Error: Full reprojection failed, but partial is possible if you define OGR_ENABLE_PARTIAL_REPROJECTION configuration option to TRUE

I did this to represent the consequence of only cropping when the source is WGS84. The next step is to determine how to handle this scenario.
cc @BergWerkGIS @GretaCB

I would suggest the following:
Either check what the most common uploaded coordinate systems are, that fail and prepare clipping for them.
Or wait till one fails and then create the clipping procedure for it.
Clipping procedure meaning:

reproject the WGS84 bbox to that other needed coordinate system
use the reprojected bbox for clipping before reprojecting the data to WebMercator

Illustration how that bbox woud look like in different coordinate systems:
WGS84 BBOX (+-180/+-85) displayed in WGS84

bbox displayed in WebMercator

bbox displayed in Van der Grinten World 1:

WGS84 with more vertices along the borders in Van der Grinten:

Right, so you're suggesting a whitelist of reference systems that we're comfortable clipping. If we do this, do we set OGR_ENABLE_PARTIAL_REPROJECTION and allow shapefiles with non-whitelisted SRS to get through?

Clipping procedure meaning:

reproject the WGS84 bbox to that other needed coordinate system
use the reprojected bbox for clipping before reprojecting the data to WebMercator

This is what I was doing before, and what I found was that when reprojecting the WGS84 bbox into some other systems, I got bad results -- the example I was tinkering with was http://spatialreference.org/ref/esri/102004/. Do you have any good explanation of what the problem was I am encountering?

@BergWerkGIS this most recent commit:

bounds.js takes the coordinates representing the world bbox in WGS84 and presents them in WKT
index.js uses this WKT to create a gdal.Geometry
that geometry is projected into the shapefile's SRS, the output is validated, and simplified with zero tolerance to remove any duplicate coords. If, after that step, the geometry is still invalid, we give up on clipping.
individual features are clipped using this geometry

At this point all the tests pass which compare the node-wmshp output to an ogr2ogr output (same extent and feature count), except for the case of Antarctica in a stereoscopic projection. In this scenario, the clipping step completely eliminates the geometry, and the node-wmshp result has no features in it.

https://github.com/mapbox/node-wmshp/commit/4d54b4d4d945e2eb79eb5ff70c62f14ede83bff6 checks if the input geometry is valid (this antarctica case is not !!) and runs geom.simplify(0) on it. This gets me a step closer to passing the test -- the feature is no longer dropped out entirely. Instead the feature's envelope is a little bit off from what I got using ogr2ogr. Close!

The problem with reprojecting a cropped, stereoscopic representation of Antarctica is that it crosses the dateline. What is a single polygon originally actually needs to be split into a multipolygon after reprojection.

Not splitting the polygons means that we end up with a single self-intersecting polygon that looks kind of "inside-out" when rendered (arrows show the order of vertices)

My takeaway is that we've solved the cropping problem here, but in the meantime we've uncovered a dateline issue.

Hm, strange, for me the clipped and reprojected Antarctica looks like this:

ogr2ogr has the flag wrapdateline which splits geometries crossing the dateline meridian.
Maybe node-gdal implements a similiar function.

I suspect you and I are clipping the polygon at different latitudes, which would explain why your example has lost the "tail" seen in the stereoscopic image.
I took stab at splitting along the dateline and, well... it is non-trivial

you have to take a different approach for lines than you do for polygons
reprojection of coordinates that were clipped at, and hence fall on or very near the dateline can result in rounding discrepancies where the majority of coordinates in a geometry are, say, +180 and then a single outlier ends up at -180.


docs: document how to mimic --verbose with environment variable RUST_LOG

The doc was unclear to me and I had to dig in the code to understand that RUST_LOG could do the same as adding --verbose

#8783

@NameNoQuality is that the wrong link?


Broken inverse_of

Adding unscoped to association seems to break inverse_of and caching of parent model instances.
class RedemptionOffer < ActiveRecord::Base
  has_one :voucher_template, inverse_of: :redemption_offer, dependent: :destroy
  acts_as_paranoid
end

class VoucherTemplate < ActiveRecord::Base
  belongs_to :redemption_offer, inverse_of: :voucher_template#, unscoped: true
  acts_as_paranoid
end

jruby-1.7.24 :033 > r = RedemptionOffer.new
jruby-1.7.24 :034 > r.build_voucher_template
jruby-1.7.24 :035 > r.voucher_template.redemption_offer.class
 => RedemptionOffer(columns)

After uncommenting the #unscoped: true from VoucherTemplate model:
jruby-1.7.24 :033 > r = RedemptionOffer.new
jruby-1.7.24 :034 > r.build_voucher_template
jruby-1.7.24 :035 > r.voucher_template.redemption_offer.class
 => NilClass

Tested on Rails 3.2.21. I might try to fix that later if I'll find the time. Overall nice gem, 10/10 would use again

Hi @MmKolodziej, thanks for that!
I really didn't use the inverse_of option in the apps I have with this gem, so I never found this crash 😅 . There are no tests related, so yes, it can probably break something. To be honest, this part of Rails itself is kind of "fragile" and obscure, or at least for me. They made some related changes, since Rails 4, to allow to use blocks on associations, but last time I tried, didn't work fully ...
I'll try to take a deeper look asap and see if I can make some patch to fix this case. If you finally got some time to work on it, would be really helpful.
About the caching problem, I think it's also related to https://github.com/markets/unscoped_associations/pull/14.
Thanks!

Hi @MmKolodziej, did you manage to find some time to take a look? I just pushed a new version (0.7.0) with some caching stuff to avoid fetch from DB on each "getter" call and some code clean up. Let me know if this version works for your case!


Lambda script for data load

Description

PR contains the changes to load the release information for Automate and Habitat in DynamoDB. This Lambda contains two sets of operations:

Load the latest release information from S3 to DynamoDB (Applicable for both Automate and Hab)

For this the Lambda will be directly triggered by the S3 bucket when the latest data is added.

Load the old release information from S3 to DynamoDB (Applicable for Hab only)

For this the Lambda should be triggered manually using "Test" feature and provide the below mentioned "Event Request"
{ "manualTrigger": "true", "s3Bucket": "chef-automate-artifacts", }

Related Issue

https://chefio.atlassian.net/browse/CHEF-4526
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] I have run the pre-merge tests locally and they pass.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[ ] All commits have been signed-off for the Developer Certificate of Origin.

Video: https://progresssoftware.sharepoint.com/:v:/s/ChefCoreC/EWX7t2gGubJBqhn-GM2AcEoBZSUHh8cWyalbi5Dqmkg3FA?e=J1gKxE

Did we run this? considered timeout scenarios or retry mechanism?

Yes, the Lambda needs to be configured and the timeout should be changed from 3 seconds to 1 min.


Ubuntu 18 error

msg: 'An unhandled exception occurred while running the lookup plugin ''password''. Error was a <class ''ansible.errors.AnsibleError''>, original message: Unable to create local directories(/private/etc/nextcloud): [Errno 13] Permission denied: b''/private/etc/nextcloud''

@UAnton did you run the playbook with sudo?

@ReinerNippes I'm run from my mac. I try with sudo and without. Now have one more error -
fatal: [<IP_ADDRESS>]: FAILED! => {
"msg": "'nc_web_port' is undefined"
}

i never tested the playbook on a remote server. the permission error should be fixed now.
https://github.com/ReinerNippes/nextcloud/commit/3c3b172884c8bb99d69667818c67dc577fcd78c9
the variable nc_web_port is defined in the inventory. did you change it?
https://github.com/ReinerNippes/nextcloud/blob/3c3b172884c8bb99d69667818c67dc577fcd78c9/inventory#L28