https://balena.io/engine/install.sh still installs 17.12 version of balena-engine

When using https://balena.io/engine/install.sh | sh to install the engine, one gets this engine:
greg@debianengine:/mnt/balena$ sudo balena-engine version
Client:
 Version:	v17.12.0
 API version:	1.35
 Go version:	go1.10.4
 Git commit:	3fdfd0d
 Built:	Wed Oct 17 15:42:29 2018
 OS/Arch:	linux/amd64
 Experimental:	false
 Orchestrator:	swarm

Server:
 Engine:
  Version:	v17.12.0
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.10.4
  Git commit:	3fdfd0d
  Built:	Wed Oct 17 15:42:29 2018
  OS/Arch:	linux/amd64
  Experimental:	false

The latest one is v18.9.9, I believe, so there's a lot we could bump, if it works, obviously.

The install.sh script still points to 17.12 but I don't think changing it there will update the marketing site as that is not automatically deployed.
I think moving to balenaCI #172 is one blocker for that... Then we could also change it to always pull the latest tagged release.

The install.sh dowloaded from the site is actually here.
We can update the version there for now (as a quick and dirty fix).

@roman-mazur good to know! Is the data on the gh-pages branch used at all right now, are there plans to use it eventualy?
cc @dimitrisnl

The guys are working on landr which will allow to create websites directly from the main repo - that when it's ready, we'll have engine site resources directly in this repo.
And for now, everything accessible at balena.io is is in that repo. Merging to master triggers a deploy there.

#!/bin/sh

tag="v17.12.0"
tag=$(echo "$tag" | sed 's|+|.|g')

machine=$(uname -m)

case "$machine" in
	"armv5"*)
		arch="armv5"
		;;
	"armv6"*)
		arch="armv6"
		;;
	"armv7"*)
		arch="armv7"
		;;
	"armv8"*)
		arch="aarch64"
		;;
	"aarch64"*)
		arch="aarch64"
		;;
	"i386")
		arch="i386"
		;;
	"i686")
		arch="i386"
		;;
	"x86_64")
		arch="x86_64"
		;;
	*)
		echo "Unknown machine type: $machine"
		exit 1
esac

url="https://github.com/balena-os/balena-engine/releases/download/${tag}/balena-engine-${tag}-${arch}.tar.gz"

curl -sL "$url" | sudo tar xzv -C /usr/local/bin --strip-components=1

cat <<EOF

   Installation successful!
 _           _
| |__   __ _| | ___ _ __   __ _
| '_ \\ / _\` | |/ _ \\ '_ \ / _\` |
| |_) | (_| | |  __/ | | | (_| |
|_.__/ \__,_|_|\___|_| |_|\__,_|

the container engine for the IoT
EOF

This is the script, Should just updating the tag work?

I will PR an update to the current script and then send one that updates the website.

I don't like that bumping the script will increase the current balena-engine version... Personally I would like to include this change in the balenaCI PR #126 because after that we can have it automatically increase with every merge (because we will have builds for every tagged version)
I'm not sure if we should make the change to v18.9.7 (which is the latest release with binaries) just in the website repo and not here though.
cc @roman-mazur @dimitrisnl

@robertgzr to update the site, only the resin-site PR is needed
And a quick suggestion: once the automatic releases creation is implemented, we could change the installation script to accept the version as an input paramter. Then the website app can make an API call to github to get the latest version and generate instruction
install.sh <latest from github>

@robertgzr does updating the version on the site fix the issue for now?

@iamsolankiamit the problem is that we currently don't build binaries for each release as part of the engine ci pipeline. we decided that we would rather move to transformers, than put more effort into the current system. until that is done this is stuck...


Unable to parse seemingly valid AFP file

Thank you for open-sourcing this library. I'm currently working on an Apache Tika parser for AFP files that wraps afplib.
However, none of the documents in my corpus can be parsed. The parser looks for the 5A magic byte and fails to find it before the threshold constant. If I fast-forward the stream to the first 5A, then it parses the structured field as an unknown structure, then errors out before the end of the stream.
Opening the file in AFPWorx renders nothing - 0 pages and 0 resources. Nothing is logged in the error log.
The file appears to be valid. I can see the magic bytes D3A8 at the beginning and D3A9 at the end.
Is this some other variant of the format which is unsupported?
I've attached one of the files in question, encrypted using your public key as it's semi-confidential. You'll first have to unzip it, then decrypt it.
D1-106.afp.gpg.zip

@mattcg,
it looks like there are structured fields in your file - just there are no 5a magic bytes in front of them. All AFP I have seen so far have 5a bytes in front of each sf introducer:
5a <2bytes length><3 bytes id> ....
Some host append an additional length field:
<2 or 4 bytes of length> 5a <2bytes length> ....
Yours looks like this:
<2bytes length><3bytes id>...
So again I have never seen this before. I also ran your file through PSF (IBMs print service facility) and it errors. So not quite sure what to do here ... what is the source of your data?

Thanks for checking it with the PSF! Can you view it using IBM's viewer plugin? It isn't available for Mac so I can't do so myself.

I will check later this week when I am back home - I only have my macbook with me :) But I am pretty certain this won't show anything.

Well I stand corrected - this is a valid AFP file - I could print it. So I need to change the code to not require 5a magic bytes

Interesting. How are you printing it on a Mac?

I have a virtual machine running psf and one that is running an IPDS virtual printer

latest commit can parse your afp

one more thing: I noticed your afp contains only images (BIM) - who render ok but you probably won't get any useful information out for tiki - unless it can do OCR ...

Having said that I think your project is a very interesting one - i'd like to see an AFP tiki module - would be very useful!

Success! And yes, the Tika standard way to handle images would be to extract the stream, then to delegate parsing of each one to Tika, which would invoke the TesseractOCRParser if the image format is supported.

That's interesting. Will your module be open source and do you need help writing the image extraction part?


consumer-group: do not cancel sessions context during rebalance

https://github.com/Shopify/sarama/pull/2110 was merged to avoid stopping heatbeats on rebalance.
However, the change is not as effective as it could be since session context is canceled on rebalance (which results in the recreation of the session in the end).
This patch allows session consumer to continue its life if heartbeat loop detected rebalance event.

@pavius 👋 can you comment on this PR?

after re-reading this area of code, I think the change makes sense — lets merge and we can roll out main into some applications to test the functionality longer term


HWKAGENT-32 Cannot collect platform resources due to a NPE in PlatformLocationRe

@jmazzitelli this applies the same logic in PlatformLocationResolver we already have in DMRLocationResolver thus allowing for null base paths and basically interpreting them as root.
I was not able to reproduce the issue quickly enough and therefore I am also not able to validate the fix. But given that we already have the same logic in place in DMR, there is a good chance that the fix will work. I must turn to other tasks now.


Uncaught TypeError: Cannot read property 'querySelector' of null

Uncaught TypeError: Cannot read property 'querySelector' of null
Atom Version: 0.162.0
System: linux 3.16.0-4-amd64
Thrown From: color-picker package, v1.4.4
Steps To Reproduce

...
...

Stack Trace
At /home/alphyon/.atom/packages/color-picker/lib/ColorPicker-view.coffee:101

TypeError: Cannot read property 'querySelector' of null
  at ColorPickerView.module.exports.ColorPickerView.open (/home/alphyon/.atom/packages/color-picker/lib/ColorPicker-view.coffee:101:66)
  at Object.module.exports.open (/home/alphyon/.atom/packages/color-picker/lib/ColorPicker.coffee:108:23)
  at atom-text-editor.atom.commands.add.color-picker:open (/home/alphyon/.atom/packages/color-picker/lib/ColorPicker.coffee:20:46)
  at CommandRegistry.module.exports.CommandRegistry.handleCommandEvent (/usr/share/atom/resources/app/src/command-registry.js:243:29)
  at /usr/share/atom/resources/app/src/command-registry.js:3:61
  at KeymapManager.module.exports.KeymapManager.dispatchCommandEvent (/usr/share/atom/resources/app/node_modules/atom-keymap/lib/keymap-manager.js:549:16)
  at KeymapManager.module.exports.KeymapManager.handleKeyboardEvent (/usr/share/atom/resources/app/node_modules/atom-keymap/lib/keymap-manager.js:391:22)
  at HTMLDocument.module.exports.WindowEventHandler.onKeydown (/usr/share/atom/resources/app/src/window-event-handler.js:167:20)

/cc @atom/core

Hi. Please update Atom and try again. Thanks!

Thanks it is solved my problem


Unable to access native kubeless api via REST

I want to submit a function ( via http-trigger ) from a NodeJS process to kubeless but I do not want to use the js equivalent of
curl --data '{"term":"Albemarle"}' localhost:8080/api/v1/proxy/namespaces/default/services/bikesearch/ --header "Content-Type:application/json"

because that needs me to know the actual IP address of the service running the function. I want to be able to access the kubeless api that gives me the level of indirection by just knowing the name of the function
kubeless function call bikesearch --data '{"term":"Albemarle"}'

Is there anyway to access the above ( function call ) api via node or python?

Thanks @arjunrao87 for the question. Currently we haven't supported API for calling function yet. That's already in our plan. Please stay tuned.
Meanwhile, we support you to create a route for function with kubeless ingress command. If you're familiar with Kubernetes Ingress, that's it. Here below is steps to go:

Make sure to have an ingress controller running in your cluster. If you're in Minikube, there is already an addon you can enable by running minikube addons enable ingress. If you're in GKE, please check this doc. Alternatively, we provide a manifest to deploy a nginx-ingress-controller manually at here

Once having the ingress controller running, you can just run this command to add a route to your function:

$ kubeless ingress create --function bikesearch

Then checking the created ingress, you will see the URL to connect to your function at HOSTS field.
$ kubectl get ing

If you have a real DNS name point to your k8s cluster, we provide --hostname flag to override the default generated hostname.

I'd love to see actual API support as well, but so far @ngtuna that is a great solution. 😄

Thanks @ngtuna. Ill mark the issue closed for now since you already have it in your roadmap. You can reference that issue/PR here if you'd like :100:


Help command is broken

In the latest release of the mod (1.10.2-0.0.3), the /help is broken.  I'm unsure if this was fixed already along with the dump command (#9), as there's been no builds since then.
I thought I'd mention the issue just in case it wasn't known though.

This was fixed in cb5b9d1cc3d8fa88d96fbb445b97c7f261b588db.


Would you like to move this repo into the alexa-js organization?

In theory SSML is not tied to Alexa in any way, but it's the current main use case. I also think we should probably provide better SSML support in the alexa-app library.
Would you like to move this repo into the newly created alexa-js organization, https://github.com/alexa-js?
This would reduce bus factor as currently only @mandnyc can add contributors to this project and make it more discoverable. By moving org admins can at least add new people, currently that's @matt-kruse, @mreinstein and myself.
There's no other purpose. There will be no changes "forced" on you or any other additional help should be expected from "the organization", so nothing would change other than the above.
If you do want to do this, because Github requires one be admin of both source and destination, transfer the repo to me (@dblock) and I'll immediately move it to the org.

actually to be honest I don't think this repo belongs in alexa-js as an organization. SSML is a standardized language that is not specific to just Alexa.

if this module truly is just servicing the alexa subset of SSML, it would be good to rename the repo to alexa-ssml-builder to make that more clear.

I agree @mreinstein, I don't feel (too) strongly about this.

@dblock I'm not saying don't merge it in, I guess my question is, is this alexa specific? if it truly is then sure it would be great to throw it under the alexa-js maintenance umbrella.

Totally, I am completely agreeing with you - lets leave it to @mandnyc to decide - if this repo is only about Alexa, then it should be renamed and we can move it. But as is I vote to leave it alone.

Just to add my two cents, I've been making basic use of this module for both an Alexa skill and a Google Assistant Action with a great deal of success.

@dblock @mreinstein - The real purpose of SSML-Builder is to focus on building SSML. Although it was originally designed to work with Alexa, it can also be used for Actions on Google and other APIs.
@Tribex - What do you think it is missing from the project? Can you clarify the issues you are running into? You mentioned about the internationalization. Can you give me more details on that?

@mandnyc Nothing is missing or causing trouble in particular. I actually love the module.
The reason I'm migrating to SSML strings is that many of the things I am having read out have multiple sentences and it's just much easier to store the version with SSML in the locale files. Pauses may need to be in different places depending on the language, and an internationalized multiple-sentence string doesn't map very well to a chainable API. :/


Added Evothings Studio to Cordova Tools listing

Request to add info about Evothings Studio to the Cordova Tools section of the Cordova home page.

Have you had the time to review this pull request for the Tools section of the Cordova home page? Is the format ok?

@mikaelkindborg thank you for submitting your app to the showcase! Sorry that it took us so long to address your PR. The Cordova developer community had a discussion that you can view here, and it was decided that the app showcase ought to be closed for submissions. Please however feel free to email<EMAIL_ADDRESS>if you have any questions or concerns regarding this decision.

@dblotsky Hi, I emailed the dev list as you suggested, but got no response as far as I can tell. Note that this pull request is not the app showcase, but for the tools section. Are you also rejecting additions to the tools section?

We are still discussing it, we'll let you know when something is decided.

@jcesarmobile Thanks!

@jcesarmobile Updated pull request with clarified description of Evothings Studio. What do you think?

@dblotsky @jcesarmobile Hi, did you reach a decision regarding merging this pull request?

@mikaelkindborg I've merged the PR! Please feel free to close it. Somehow my merging script didn't append the "this closes..." part.

@dblotsky Many thanks indeed for merging!


Rate limit

Today at around 18 CET, there was already a rate limit for the 10k tests per day and the error message said we have to wait midnight PT, which is 9 CET (so the app is unusable more than half a day).
It would be nice if users cannot easily abuse the rate limit and affect other users, by allowing around 5 requests a day per IP address.

Yes I agree but that needs a database for the api server and I don't have the time to make that now..

If we just need to store IP addresses with the number of requests, I guess it would be fine to use plain text instead of a more complex database.
For example, create a text file for each IP address using the app (with the IP address as file name) and store the number of requests in the text file. Then we just need to read / write the text file and compare the number of requests stored to the one allowed per IP address. And every day, the files can be removed to reset the counter, with a cron job for example.

but there is no server, the api is serverless hosted on vercel.com (it's free that's why i use it)

I saw you use a Node.js server on https://github.com/1nikolas/play-integrity-checker-server so I thought my solution would be possible.

I use vercel for hosting which has database/kv storage but on the free plan has lots of limits so it's not an option. For the client side I don't think many people would not think to clear data and reset it. After all most people who use this app have magisk and know these basic stuff.
Maybe I can port the server into Cloudflare Workers and use D1 which has a generous free plan


Roll Engine from f3847febb247 to d7229e95d24a (10 revisions)

https://github.com/flutter/engine/compare/f3847febb247...d7229e95d24a
2021-05-12<EMAIL_ADDRESS>Roll Skia from 433d25c947e4 to 0270bf5d10be (5 revisions) (flutter/engine#26100)
2021-05-12<EMAIL_ADDRESS>Sped up the objc standard message codec (flutter/engine#25998)
2021-05-12<EMAIL_ADDRESS>Roll Skia from 827bb729a84d to 433d25c947e4 (3 revisions) (flutter/engine#26097)
2021-05-12<EMAIL_ADDRESS>Roll Skia from d9a7c5953df3 to 827bb729a84d (2 revisions) (flutter/engine#26096)
2021-05-12<EMAIL_ADDRESS>Remove "unnecessary" imports. (flutter/engine#25736)
2021-05-12<EMAIL_ADDRESS>Roll Skia from d9a7c5953df3 to 827bb729a84d (2 revisions) (flutter/engine#26093)
2021-05-12<EMAIL_ADDRESS>Roll Fuchsia Linux SDK from 1PbnXEErn... to MoY7UVVro... (flutter/engine#26091)
2021-05-12<EMAIL_ADDRESS>Roll Dart SDK from d59cb89f73fe to 934cc986926d (1 revision) (flutter/engine#26090)
2021-05-12<EMAIL_ADDRESS>Get licenses check deps from gclient rather than pub (flutter/engine#26077)
2021-05-12<EMAIL_ADDRESS>Roll Skia from c1b6b6c615a7 to d9a7c5953df3 (2 revisions) (flutter/engine#26089)
Also rolling transitive DEPS:
fuchsia/sdk/core/linux-amd64 from 1PbnXEErnpjU to MoY7UVVro7ip
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/master/autoroll/README.md

Commit queue failed; closing this roll.


Any method that provides URL and Filename safe type base64 encoding scheme.

Hi Team,
I am trying to find out some alternate method for base64.urlsafe_b64decode(available in python library)
I am actually looking for file and URL safe encoding and decoding scheme.
Could you please tell which method shall I use to achieve this?
Best Regards
Sethi Shivam

Base64.decode() decodes both flavors of Base64 as SYNOPSIS says.  Maybe I should add .decodeURI() as an alias of .decode()…

Hi dankogai , Thanks for the response.
I am looking for the encoding method as well so that I can encode and decode both for URL safe.
Best Regards
Sethi Shivam

Base64.encodeURI() is exactly what you are looking for.

Yes .... shall I use the existing encode method ?  or the functionality doesn't exist yet ?

READ [SYNOPIS] NOT JUST THREAD!  THE FUNCTIONALITY ALREADY EXISTS!
Base64.encode('小飼弾');    // 5bCP6aO85by+
Base64.encodeURI('小飼弾'); // 5bCP6aO85by-
Base64.decode('5bCP6aO85by+');  // 小飼弾
// note .decodeURI() is unnecessary since it accepts both flavors
Base64.decode('5bCP6aO85by-');  // 小飼弾

case dismissed.


[BACKLOG-39559] JDK17 - Build issue for https://github.com/pentaho/pe…

…ntaho-cassandra-plugin.git

@pentaho-whartman Copyright header date changes are completed.


Make a script to submit the simulation as a job to gadi

See here for how Navid setup a submission script for an Oceananigans.jl model on Gadi.

Hi @navidcy, I am trying to write a shell script to submit an Oceananigans.jl model to gadi. I used what you did in the ASC-idealised model as a guide but my output file does not seem to be saving where I have set it to save (or at all) as I get this error
 [ Info: Oceananigans will use 12 threads
 [ Info: Model setup
 [ Info: Setting initial conditions
 [ Info: Building DNS
 [ Info: Setting two layer initial conditions
 [ Info: Setting up simulation
 ERROR: LoadError: NetCDF error: Opening path /g/data/e14/jb2381/CabbelingExperiments/outputs/cabbeling_stepchange_nothing_5min.nc: No such file or directory (NetCDF error code: 2)
 Stacktrace:
 [1] nc_open(path::String, mode::UInt16)
 @ NCDatasets ~/.julia/packages/NCDatasets/st9Jz/src/netcdf_c.jl:274
 [2] NCDatasets.NCDataset(filename::String, mode::String; format::Symbol, share::Bool, diskless::Bool, persist::Bool, memory::Nothing, attrib::Dict{Any, Any})
 @ NCDatasets ~/.julia/packages/NCDatasets/st9Jz/src/dataset.jl:217
 [3] NCDataset
 @ ~/.julia/packages/NCDatasets/st9Jz/src/dataset.jl:177 [inlined]

When I run the same experiment setup locally all works fine and similarly if I run the experiment in an interactive session using the VDI desktop on ARE. Have you come across this error when submitting julia jobs to gadi or have I got things in my submission script wrong? Any help would be great but only if you have time!

strange...
not sure if something changed with NCDatasets.jl or something? I wasn't doing anything more special back then... perhaps the situation calls for a Zoom to debug?
do you enforce all versions of the packages to be fixed by using a Manifest.toml?
(cc @taimoorsohail)

Yes I have compat entries in TwoLayerDirectNumericalShenanigans.jl. For NCDatasets.jl it is
NCDatasets = "0.12"

I have not changed to 0.13 yet because I remember seeing something about a bug 0.13.
It is a little strange because the outputs folder is correctly created but then no file to save to is there. I will try to submit a job now just in case something magic happens but if not I think you might be right about a zoom for debugging..

OK, except magic happens, then try to submit a job when you only do something like:
using NCDatasets
# This creates a new NetCDF file /tmp/test.nc.
# The mode "c" stands for creating a new file (clobber)
ds = Dataset("/tmp/test.nc","c")

# Define the dimension "lon" and "lat" with the size 100 and 110 resp.
defDim(ds,"lon",100)
defDim(ds,"lat",110)

# Define a global attribute
ds.attrib["title"] = "this is a test file"

# Define the variables temperature with the attribute units
v = defVar(ds,"temperature",Float32,("lon","lat"), attrib = OrderedDict(
    "units" => "degree Celsius"))

# add additional attributes
v.attrib["comments"] = "this is a string attribute with Unicode Ω ∈ ∑ ∫ f(x) dx"

# Generate some example data
data = [Float32(i+j) for i = 1:100, j = 1:110]

# write a single column
v[:,1] = data[:,1]

# write a the complete data set
v[:,:] = data

close(ds)

[Taken from NCDatasets.jl README; removed the using DataStructures which is unnecessary I think... try it locally first! :)]

Also ensure you have the right storage flags in your qsub script for where you are trying to save...?

Yes I have compat entries in TwoLayerDirectNumericalShenanigans.jl. For NCDatasets.jl it is
NCDatasets = "0.12"

I have not changed to 0.13 yet because I remember seeing something about a bug 0.13.

NCDatasets 0.13.1 should be OK!

No luck will try the example job you suggest above.

Also ensure you have the right storage flags in your qsub script for where you are trying to save...?

I think so.. @taimoorsohail took a look and said it looked correct.

NCDatasets 0.13.1 should be OK!

Great will set a new compat!

Another thing to experiment is to use CPU() and see if it makes a difference? It shouldn't ... but who knows...
https://github.com/jbisits/CabbelingExperiments/blob/2f8bd86240400f87a824c3f706edef72894fc807/cabbeling_gpu_salinitynoise.jl#L3

Running run_nc_test.sh, which uses the example above, works. Correctly saves "test.nc" in outputs with the same PBS job settings..

but your package does not..?

Yep that is right, the error is that the file that is not being created
  7 ERROR: LoadError: NetCDF error: Opening path /g/data/e14/jb2381/CabbelingExperiments/outputs/cabbeling_stepchange_nothing_5min.nc: No such file or direct
    ory (NetCDF error code: 2)
  8 Stacktrace:
  9  [1] nc_open(path::String, mode::UInt16)
 10    @ NCDatasets ~/.julia/packages/NCDatasets/st9Jz/src/netcdf_c.jl:274
 11  [2] NCDatasets.NCDataset(filename::String, mode::String; format::Symbol, share::Bool, diskless::Bool, persist::Bool, memory::Nothing, attrib::Dict{Any,
    Any})
 12    @ NCDatasets ~/.julia/packages/NCDatasets/st9Jz/src/dataset.jl:217
 13  [3] NCDataset
 14    @ ~/.julia/packages/NCDatasets/st9Jz/src/dataset.jl:177 [inlined]

My suspicion is that it is something I have done so I am going to go back and check everything locally, then try an interactive session (where everything had been working) and see if I can find out what might be going on.

Does the outputs exist? Not sure if NCDatasets can create the directory. Try having an empty outputs directory?

Does the outputs exist? Not sure if NCDatasets can create the directory.

Yep, the function creates the outputs directory separately (which it did on the first run but does not need to on subsequent runs).

Try having an empty outputs directory?

It has been on all attempts except my most recent one where the only other file in it was the test.nc (created from the submission script).
It is doing everything as it should in terms of naming as well as this is what the filename should be and where it should be being saved
/g/data/e14/jb2381/CabbelingExperiments/outputs/cabbeling_stepchange_nothing_5min.nc

I have been having some troubles with the netcdf saving locally, including seeing the same error as is occuring in the gadi submission script. I am not sure in which commit these started appearing as everything was working fine locally so will have to do some debugging. I will let you know how I get on but if once I have fixed these problems I am still not able to save on gadi I will get in touch!

ok!

Fixed with TwoLayerDirectNumericalShenanigans.jl v0.5.0

I don't really understand what the fix was here. Link to a PR?

I'm happy that this is resolved but I don't really understand what the fix was here. Link to a PR?

Sorry I forgot to link the issue to the PR.
The issue had to do with overwrite_existing behaviour in the NetcdfOutputWriter and JLD2OutputWriter. In previous versions of TwoLayerDirectNumericalShenanigans.jl (TLDNS) I set overwrite_existing to true by default so a file was always created or overwritten if it already exists. In TLDNS v0.4.5 I changed this default behaviour to false so a file is only appended to not created. So when the simulation setup was run some of my custom outputs would try to save to simulation.output_writers[:outputs].filepath but it had not been created. The fix was to set overwrite_existing or more specifically the keyword argument overwrite_saved_output = nothing in TLDNS_simulation_setup
function TLDNS_simulation_setup(tldns::TwoLayerDNS, Δt::Number,
                                stop_time::Number, save_schedule::Number;
                                save_file = :netcdf,
                                output_path = SIMULATION_PATH,
                                checkpointer_time_interval = nothing,
                                cfl = 0.75,
                                diffusive_cfl = 0.75,
                                max_change = 1.2,
                                max_Δt = 1e-1,
                                density_reference_gp_height = 0,
                                overwrite_saved_output = nothing,
                                save_velocities = false)

to match behaviour of the source code
function NetCDFOutputWriter(model, outputs; filename, schedule,
                                          dir = ".",
                                   array_type = Array{Float64},
                                      indices = (:, :, :),
                                   with_halos = false,
                            global_attributes = Dict(),
                            output_attributes = Dict(),
                                   dimensions = Dict(),
                           overwrite_existing = nothing,
                                 deflatelevel = 0,
                                      verbose = false)

I have however run into a different issue when saving output all of a sudden..


Aliasing GPIO names at the board level still available for after v2 Refactor?

One thing I did notice during my migration here: #363  is whether or not now there's a way to alias the names of the pins for specific boards. Is that still possible or with the refactor that won't work anymore?

The define_pins macro is still available in v2 https://github.com/atsamd-rs/atsamd/blob/f36882efa0708126f59e995530c6420e43bb3da6/hal/src/common/gpio/v2/pin.rs#L1419

Thanks @sajattack
Maybe it's my lack of understanding. Looks like that macro is being used here: https://github.com/atsamd-rs/atsamd/blob/f36882efa0708126f59e995530c6420e43bb3da6/hal/src/common/gpio/v2/pin.rs#L1447
Am I wrong?
I can access them this way.
        let mut pins = Pins::new(peripherals.PORT);
        let led_pass = pins.pa23.into_push_pull_output();

I asked @bradleyharden about this and he said he hasn't implemented a solution for BSP aliases yet but he's going to work on it.

Cool. Thanks @sajattack. I'll close this out unless you guys want to keep this open for tracking purposes.

Might as well keep it open until it's implemented.

@jaredwolff, the v2 define_pins macro is actually completely unrelated to the v1 define_pins macro. That's just an unfortunate naming coincidence. The v2 version is intended only for internal use.
I'm looking at the v1 define_pins macro now, and I think it would make sense to define something equivalent for v2. I'll try to put something together that satisfies the same purpose and perhaps improves on it, as discussed in #363.

@jaredwolff, the v2 define_pins macro is actually completely unrelated to the v1 define_pins macro. That's just an unfortunate naming coincidence. The v2 version is intended only for internal use.
I'm looking at the v1 define_pins macro now, and I think it would make sense to define something equivalent for v2. I'll try to put something together that satisfies the same purpose and perhaps improves on it, as discussed in #363.

Does #371 satisfy this issue? If it is merged, should this issue be closed?

Does #371 satisfy this issue? If it is merged, should this issue be closed?

Correcto. We can close this out once that is merged.

Correcto. We can close this out once that is merged.


How does bigcache handle hash collisions

https://github.com/allegro/bigcache/blob/main/shard.go#L120
I find out bigcache  cannot handle hash collisions?

BigCache does not handle collisions


Cudnn conflicts

Hi, I'm trying to compile and install tensorflow 2.1 with cuda toolkit 10.1 and cudnn 7.6.5. But I have matlab installed which comes shipped with cudnn7.5. Thus the  line
 export TF_CUDNN_VERSION="$(find /opt /usr -name 'libcudnn.so.*' | tail -n1 | sed -r 's/^.*\.so\.//')"

Returns the value 7.5 and not the value 7.6 expected.
find /opt /usr -name 'libcudnn.so.*'

returns :
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/MATLAB/R2019b/bin/glnxa64/libcudnn.so.7
/usr/local/MATLAB/R2019b/bin/glnxa64/libcudnn.so.7.5.0
my quick fix is to limit the search to:
/usr/local/{CUDAVERSION}/

As, if my memory is correct, is where nvidia installation guides suggests to put the cudnn header an lib.
Same for NCCL
Btw, it could be nice to add support for the number of jobs
make -j 4

calls bazel with all available threads, and make my laptop crash

Hi,

regarding the cudnn path - unfortunately, cuda is not installed to /usr/local/{CUDAVERSION} in all cases (e.g., on Arch, not sure about Ubuntu). But I could change the find to also require cuda in path, e.g.,find /opt /usr -name 'libcudnn.so.*' -path '*/cuda*'

Would that work for you?

In this case, make -j4 does not affect the number of jobs for bazel. Extra parameters for bazel can be passed here. But if your system is sensitive to the number of threads, you may want to edit your .bazelrc adding something like:build --jobs 2 --ram_utilization_factor 50

Hi,

regarding the cudnn path - unfortunately, cuda is not installed to /usr/local/{CUDAVERSION} in all cases (e.g., on Arch, not sure about Ubuntu). But I could change the find to also require cuda in path, e.g.,find /opt /usr -name 'libcudnn.so.*' -path '*/cuda*'

Would that work for you?

In this case, make -j4 does not affect the number of jobs for bazel. Extra parameters for bazel can be passed here. But if your system is sensitive to the number of threads, you may want to edit your .bazelrc adding something like:build --jobs 2 --ram_utilization_factor 50

Hi,
The fix should work or at least it rejects Matlab paths! I'm not aware of any software other than Matlab that will have its own version of cudnn. Thanks for the help!
Closing the issue

Hi,
The fix should work or at least it rejects Matlab paths! I'm not aware of any software other than Matlab that will have its own version of cudnn. Thanks for the help!
Closing the issue


(Open) Adding three stream heat exchanger temperature profiles

Add the test cases from this paper:
Sekulic 1995 - Three-Fluid Heat Exchanger Effectiveness - Revisited.pdf


COMP: ITK_THREAD_RETURN_DEFAULT_VALUE with WASI toolchain

To address:
In file included from /work/wasi-build/_deps/elx-src/Common/ImageSamplers/itkImageToVectorContainerFilter.h:197:
/work/wasi-build/_deps/elx-src/Common/ImageSamplers/itkImageToVectorContainerFilter.hxx:284:46: error: expected unqualified-id
  return itk::ITK_THREAD_RETURN_DEFAULT_VALUE;

For the record, this commit was cherry-picked from pull request #920 "WebAssembly Support", by Matt McCormick (@thewtex)


[PID/(Q)EAA Data Model - PID Claims] missing element

Within PID claims, Link  we need to add a reference to the Schema used by the PID.

I agree that a ref to ARF where the PID data set schema is provided


tryLock in redis cluster working not properly

Expected behavior
tryLock of RLock can not return true when the lock key still exist in cluster redis.
Actual behavior
Sometimes tryLock can acquire the lock key although it still exists. Moreover, the TTL of the lock key also is reset.
Steps to reproduce or test case
Invoke boolean res = lock.tryLock(100, 3600, TimeUnit.SECONDS) => res = true. Then after 10s, TTL of the key is 3590, in another thread, invoke boolean res = lock.tryLock(100, 3600, TimeUnit.SECONDS) => res = true. TTL of the key is 3600
Redis version
5.0.7
Redisson version
3.12.5

Which Redis vendor do you use?

Unable to reproduce.
Here is my test code:
        RLock lock = redisson.getLock("lock1");
        Assert.assertTrue(lock.tryLock(100, 3600, TimeUnit.SECONDS));

        Thread.sleep(10000);

        // next row  for test renew expiration tisk.
        //Thread.currentThread().sleep(TimeUnit.SECONDS.toMillis(RedissonLock.LOCK_EXPIRATION_INTERVAL_SECONDS*2));
        Thread thread1 = new Thread() {
            @Override
            public void run() {
                RLock lock1 = redisson.getLock("lock1");
                try {
                    Assert.assertFalse(lock1.tryLock(100, 3600, TimeUnit.SECONDS));
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        };
        thread1.start();
        thread1.join();

        lock.unlock();

Unable to reproduce.
Here is my test code:
        RLock lock = redisson.getLock("lock1");
        Assert.assertTrue(lock.tryLock(100, 3600, TimeUnit.SECONDS));

        Thread.sleep(10000);

        // next row  for test renew expiration tisk.
        //Thread.currentThread().sleep(TimeUnit.SECONDS.toMillis(RedissonLock.LOCK_EXPIRATION_INTERVAL_SECONDS*2));
        Thread thread1 = new Thread() {
            @Override
            public void run() {
                RLock lock1 = redisson.getLock("lock1");
                try {
                    Assert.assertFalse(lock1.tryLock(100, 3600, TimeUnit.SECONDS));
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        };
        thread1.start();
        thread1.join();

        lock.unlock();

It sometimes occurs, not always. Could you run it multiple times?

Sorry and thank you so much. I found at the tryLock method is invoke by the same thread. The reason is the thread is mock-up from the thread pool.


TEst

Description

Motivation and Context

How Has This Been Tested?

Checklist

[ ] I have updated the documentation accordingly.
[ ] All new and existing tests passed.

Codecov Report
Attention: 1346 lines in your changes are missing coverage. Please review.

Comparison is base (b384285) 89.90% compared to head (dba634a) 58.70%.

Files
Patch %
Lines

api/controller/loan.py
9.75%
222 Missing :warning:

api/controller/opds_feed.py
17.79%
134 Missing :warning:

api/controller/work.py
17.88%
101 Missing :warning:

api/controller/circulation_manager.py
25.66%
84 Missing :warning:

api/admin/controller/collection_settings.py
0.00%
68 Missing :warning:

api/admin/dashboard_stats.py
0.00%
65 Missing :warning:

api/admin/controller/self_tests.py
0.00%
63 Missing :warning:

api/admin/controller/integration_settings.py
0.00%
62 Missing :warning:

api/controller/marc.py
39.56%
55 Missing :warning:

api/admin/controller/quicksight.py
0.00%
54 Missing :warning:

... and 45 more

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #378       +/-   ##
===========================================
- Coverage   89.90%   58.70%   -31.20%
===========================================
  Files         212      244       +32
  Lines       28646    28991      +345
  Branches     6538     6520       -18
===========================================
- Hits        25754    17019     -8735
- Misses       1886    11276     +9390
+ Partials     1006      696      -310

Flag
Coverage Δ

migration
26.70% <19.92%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Jest test are failing

Below is issue am getting when I try to run the jest command.

jest --config jest.config.json

FAIL  test\Home.spec.js
? Test suite failed to run
SyntaxError: Unexpected reserved word  at transformAndBuildScript (node_modules\jest\node_modules\jest-cli\node_modules\jest-runtime\build\transform.js:320:12)

Below is my configuration. Package.json with webpack, jest versions
{
  "name": "testjest",
  "version": "1.0.0",
  "description": "Test React and Redux using Jest",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "dev": "webpack-dev-server --content-base --inline --hot",
    "test": "jest --config jest.config.json"
  },
  "dependencies": {
    "babel-preset-es2015": "^6.18.0",
    "babel-preset-react": "^6.16.0",
    "babel-preset-stage-0": "^6.16.0",
    "react": "^15.4.1",
    "react-dom": "^15.4.1",
    "react-redux": "^5.0.1",
    "redux": "^3.6.0"
  },
  "devDependencies": {
    "babel-core": "^6.26.0",
    "babel-jest": "^21.2.0",
    "babel-loader": "^6.2.10",
    "babel-plugin-transform-decorators-legacy": "^1.3.4",
    "babel-plugin-transform-es2015-modules-commonjs": "^6.26.0",
    "core-js": "^2.4.0",
    "enzyme": "^2.7.0",
    "jest": "^18.1.0",
    "json-stable-stringify": "^1.0.1",
    "react-addons-test-utils": "^15.4.1",
    "react-test-renderer": "^15.4.1",
    "redux-mock-store": "^1.2.1",
    "regenerator-runtime": "^0.11.0",
    "regenerator-transform": "^0.10.0",
    "identity-obj-proxy": "3.0.0",
    "webpack": "^1.14.0",
    "webpack-dev-server": "^1.16.2"
  }
}

Here is my json config file.
{
    "moduleFileExtensions": [
        "js",
        "jsx"
    ],
    "moduleDirectories": ["node_modules", "bower_components", "shared"],
   "moduleNameMapper": {
      "\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$":
        "<rootDir>/__mocks__/fileMock.js",
      "\\.(css|less)$": "<rootDir>/__mocks__/styleMock.js"
    },
    "testPathIgnorePatterns": [
      "<rootDir>/node_modules/",
      "<rootDir>/legacy_notes_and_workbook/",
      "<rootDir>/src/stylesheets/"
    ],
    "transform": {},
    "transformIgnorePatterns": [ "<roodDir>/node_modules/(?!)" , "node_modules/(?!(jest-)?react-native|react-navigation)" ],
    "testRegex": "(/__tests__/.*|\\.?(test|spec|Test|Spec))\\.(ts|tsx|js|jsx)$"
}

Node versions installed on my machine are:
node -v
v4.2.2
npm -v
1.4.28
Please let me know if anybody has any solution on above issue. Thanks in advance.

Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions. Thank you :)


Revert "Simplified and updated go.mod"

Reverts kudobuilder/kudo#1325 because it re-introduces the issue fixed in https://github.com/kudobuilder/kudo/pull/1244
cc @armandgrillet @nfnt

How does the issue in #1244 manifest. It isn't detected by the CI, which makes it hard to determine which dependencies can be bumped. How about adding a replace directive in go.mod instead? This would make it clear that this specific version is intended to be used. Not sure if comments can be used in go.mod, but having an additional comment here would be great as well.

Sorry for reintroducing a bug, I was unaware of the mentioned issue. I've just tried bumping k8s.io/code-generator on the master branch to v0.18.0-alpha.1 (then v0.18.0-alpha.2) but it then introduces other issues, will have to keep working on it.

How does the issue in #1244 manifest?

Code generation script barfs when running with GOPATH unset or GOBIN set to something else than $GOPATH/bin, IIRC.

It isn't detected by the CI, which makes it hard to determine which dependencies can be bumped.

Right. I don't think we invoke code generation in the CI runs at all, do we?

How about adding a replace directive in go.mod instead? This would make it clear that this specific version is intended to be used.

When reading the documentation for go.mod it did not seem to me that this is what replace is to be used for? Also, I'm not sure that:
require k8s.io/code-generator v0.16.6
replace k8s.io/code-generator v0.16.6 => k8s.io/code-generator v0.18.whatever

would qualify as making the intent clear :-)

Not sure if comments can be used in go.mod, but having an additional comment here would be great as well.

I think it allows comments. However I don't think that would scale. Even if we commented that a particular version of a dependency is necessary for some reason X, we would have no idea whether that ceases to be true if X is no longer a valid reason (as there can be other reasons).
This whole file is a delicate mesh of subtleties and it's not possible to document what would fail in case one of these lines are changed. The only assertion we can make is that "it worked like this for whoever touched it last, possibly only after lots of sweat and tears". I think that increasing test coverage to also invoke the less often used tooling is the only way to go...

@porridge Would backporting https://github.com/kubernetes/code-generator/commit/6b257a9d6f461b5e15dc2f0d13e29731a5b5255a to https://github.com/kubernetes/code-generator/tree/release-1.16 and then using it in KUDO make sense?

@porridge Would backporting kubernetes/code-generator@6b257a9 to https://github.com/kubernetes/code-generator/tree/release-1.16 and then using it in KUDO make sense?

In theory yes, but in practice it took over a month to get that change merged, and this time we'd need to wait for another k8s point release after that.
But perhaps we could replace k8s.io/code-generator v0.16.6 => github.com/kudobuilder/code-generator v0.16.6-withpatch
WDYT about creating a temporary fork under kudobuilder @gerred ?

If code-generator is only used as a tool (not runtime dependency) then I would consider making tools a module in it's own right in a subdirectory (/tools/) in this repo. Sorry if I've missed context!

@jimmidyson you're not wrong. Check out tools.go - that's how and why we're getting these into go.mod, based on the recommendation here: https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
It's a little clunky, but it's at least recognizable as a pattern in a few projects now. @porridge I'm OK with that patch. Do we have a path forward to not rely on that long term and get these dependencies up to date? Or at least, a diff of things that prevent us from using the most up to date code generator now? If so, we can in parallel start working on stamping out those issues in the KUDO codebase proper.

@gerred My point is that by adding all the tools dependencies into your module's go mod you're adding dependencies which aren't actually source dependencies, but purely build dependencies. By moving the tools dependencies into their own submodule in <REPO_ROOT>/tools then you separate out build time dependencies from source dependencies, which has saved me time on other projects when updating conflicting dependencies, as well as benefitting downstream projects by  limiting transitive dependencies, again to save time fighting conflicting dependencies.

Superseded by #1359


Why is the pluck deprecated?

I saw this commit and I'm wondering why pluck is deprecated? I couldn't find any issue where this is discussed.
By the logic behind the explanation about using optional chaining, this means that you can easily deprecate lot's of other simple operators like startWith, endWith, timestamp and so on, because it's always possible to use pipe + something in case of these. I just wonder what's the reason behind deprecating this operator.

pluck('prop') is a shorthand for map(x => x.prop), the main advantage to using pluck was the path traversal safety, meaning you can do
pluck('foo', 'bar', 'baz')

and this would be fine even if the emitted value didn't have a foo property. Contrast this with
map(x => x.foo.bar.baz) which will error if the emitted value doesn't have a foo property
With optional chaining this advantage is no longer true since you can now do
map(x => x?.foo?.bar?.baz)

And it will be as safe as the pluck usage. Note that having proper TypeScript typing for pluck is quite complex and not as robust as the map usage which is a major reason to prefer the map method

FWIW, deprecated parts of the API won't necessarily be removed in the very next version of the library. ATM, one of the things on which we are working is improving the deprecation messages to make them easier to understand and to include links to the documentation to explain what is being deprecated and why. IMO, pluck won't be removed from the API until version 8 or later.

Thanks for the clarifications @kolodny, @cartant. I like this operator and I use it a lot because it's easier (and more clean IMO) to write pluck than map.

Just dropping my 2 cents here.
As much as the code is equivalent and the usage of optional chaining is preferred, with the transition to map you are losing a huge "readability" advantage for the person reading the code.
It's mentally a lot easier to recognize a pluck than to identify the map call is doing the same. Maybe an alias function that still uses the underlying code but allows quicker recognition?

And it will be as safe as the pluck usage. Note that having proper TypeScript typing for pluck is quite complex and not as robust as the map usage which is a major reason to prefer the map method

Late to the party, but, I just wanted to echo this part ☝ of @kolodny's argument for preferring map.  We got bit by an unintended action payload shape change, which was hidden from TS via pluck. The path was no longer valid, which meant that the output of pluck was typed as unknown, but with some bad luck in exactly how that was consumed downstream (straight comparisons and string concatenation), there was no TS error. If we'd used map instead from the beginning, we would have immediately noticed the erroneous payload shape, instead of shipping broken code..


Moving the image with a gesture

Moving the image with two fingers to the left will result the application to close.
Moving to the right works fine.

I actually cannot test this rigorously as I don't have any touch device, however it shouldn't really be a problem since MCS doesn't manually handle/catch any touch related input.
I tried simulating touch input using Windows Simulator (1, 2) but I can't reproduce the issue. Moving two fingers to any direction works normally, can you try with another device?

Ah sorry it's not a touchscreen but a touchpad.
I'm using ASUS Smart Gesture.

I do use a asus laptop (with asus smart gesture v<IP_ADDRESS>) too but I can't reproduce the issue.
All the gesture gets converted by smart gesture as a mouse input to the program. In this case, two fingers to the left should result in a shift + mouse wheel. Try using mouse and scroll using mouse wheel (+ shift for horizontal scroll) to see if the problem exist on mouse too. If it doesn't the problem probably lies on the touchpad converter, however other program like browser shouldn't work too which is kinda impossible.

I've same version as yours.
Scrolling with Shift+Mouse Wheel works fine to left and right.
But swipping with two fingers to make the image move to the left will resulting the application to close.
Even when there's no horizontal image so there's no horizontal scrollbar but doing the swipping will get the same result.

After a little bit debugging, this relates to arithmetic overflow on WpfMouseWheelLib.dll which has been fixed. However the project doesn't create a newer release and the current dll we're using is the first release from it's codeproject.
Newly built dll will be shipped in the next release.


No connected devices found. Returning best error VRInitError_Init_HmdNotFound

The vrserver log reports that the driver inits correctly. However, Steam VR reports that no headset is present:
virtual_display: Headset connected to Intel(R) HD Graphics 530.
virtual_display: Using NVIDIA GeForce GTX 1070 as primary graphics adapter.
virtual_display: Launching "E:\SteamVR Virtual Display\bin\win32\virtual_display.exe" 1920 0 2160 1200 90 1
virtual_display: Adding Virtual device
...........................................
No connected devices found. Returning best error VRInitError_Init_HmdNotFound
I am able to work around the issue by also adding an HMD device on Init, TrackedDeviceClass_HMD.
Am I missing something or TrackedDeviceClass_DisplayRedirect cannot be used on it own?

It is not intended to be used on its own.  Instead, it is intended to be used in conjunction with a "proper" driver (which provides tracking, etc.) like driver_lighthouse.  Those drivers must also be DisplayRedirect aware - such that they don't fail if they do not find the hmd display directly attached if a DisplayRedirect driver has been activated.  This, for instance, prevents this being used with driver_oculus.

The Oculus part is kind of a bummer. Thanks for your reply!


sync_argocd_apps.sh has an unbound variable: _out_of_sync_apps

Checks

[X] I have searched the existing issues.
[X] This issue is NOT specific to the CLI. (If so, please open an issue on the CLI repo)

deployKF Version
0.1.4
Kubernetes Version
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.2", GitCommit:"5835544ca568b757a8ecae5c153f317e5736700e", GitTreeState:"clean", BuildDate:"2022-09-21T14:33:49Z", GoVersion:"go1.19.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.2", GitCommit:"5835544ca568b757a8ecae5c153f317e5736700e", GitTreeState:"clean", BuildDate:"2022-09-21T14:27:13Z", GoVersion:"go1.19.1", Compiler:"gc", Platform:"linux/amd64"}

Description
when variable _out_of_sync_apps is an empty array, it always comes with an error:

I found a post from reddit that might cause the error.
Relevant Logs
No response
deployKF Values (Optional)
No response

@quloos can you confirm how you're running this? I I assume you're running on WSL?
Also, can you try the newest version of the sync script to confirm it still happens (get the latest version from the main branch).

@quloos can you confirm how you're running this? I I assume you're running on WSL?
Also, can you try the newest version of the sync script to confirm it still happens (get the latest version from the main branch).

I am running on CentOS7.
And the newest version of sync script won't help.

At the first time I run the sync script, there's no error, because for every sync_wave, variable _out_of_sync_apps is not empty. When I try to resync, the argocd app deploykf-app-of-apps has a Synced status, variable _out_of_sync_apps is an empty array, bash script fails.

See this
Upgrade bash should fix this.

@quloos I must have mistakenly put the minimum version of bash as 4.2 when it actually should be 4.4.
Would this be a problem for you? Is it possible to update the version of bash on your server?

all good, thx

@quloos thanks for the report, I have updated the minimum bash version check to 4.4+ in https://github.com/deployKF/deployKF/pull/126


Renaming is now in private preview

Needs to be updated as per https://docs.microsoft.com/en-us/sharepoint/tenant-rename

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7fff5ef6-b4cc-5a18-10a8-7b5182e2deae
Version Independent ID: 8176e436-b25b-e455-09ac-be08bea2bfcb
Content: Change your SharePoint domain name - SharePoint in Microsoft 365
Content Source: SharePoint/SharePointOnline/change-your-sharepoint-domain-name.md
Service: sharepoint-online
GitHub Login: @kaarins
Microsoft Alias: kaarins

Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

Please see https://github.com/MicrosoftDocs/OfficeDocs-SharePoint/pull/2841#issuecomment-896331049
@scanum please close thanks


Check required fields in update event type form

Currently the form for updating an event type would happily allow you to omit some fields which are marked as required in the form itself, e.g. "Owning application" and "Audience".  Then the API responds with 400 Bad Request which is sometimes not really straightforward to parse.
In case of omitting the owning application, no error occurs at all.  So this is contradictory to the marking of the field as required.
Preferably, the HTML form in UI would highlight the required and missing fields with red color when the user tries to submit it, avoiding the API call in the first place.

Fixed


Don't validate field when it's target gets filled.

Is your feature request related to a problem? Please describe.
Hi,
I have a custom validator that uses a target.
For example I got age field with rule required.
And second field that is select box and uses age as target for custom rule.
Lets say for example Confirm age with rule required|confirm:age
Now when i change the first age field that only has required rule. It also runs a validation for Confirm age field and marks this invalid. When there is big form with many fields and many refs the form starts to get very red even when you haven't filled half of it.
So on my example it runs required validation for Confirm age field. And prints error This field is required or something.
Describe the solution you'd like
I would like to disable the validation for ref.
Describe alternatives you've considered
I was thinking if i can edit my custom rule. But it wont be triggered.
Other alternative would be to create custom event that removes an error when second field does not have a value.

I see the use case here. I would argue that instead of not validating when the target is changed, only validate the field if it has been manipulated by the user.
Validating the field again is necessary for the whole premise around the target-based or cross-field validation to work.
Slightly tweaking this to only validate the targets when they have been validated before (either manual by calling validate() or by user interaction) would make it acceptable for your use-case without risking incorrect validation state.


Wildcard or "route not found" handling?

Hi,
is there a way to handle wildcard requests? Or catch unknown routes?
greetngs.

Hi, shelf_plus uses the "official" router engine from shelf_router. From what I've seen, it allows you to have wildcard also with regex matching (last example in link). Maybe this helps you:
https://pub.dev/documentation/shelf_router/latest/shelf_router/Router-class.html

You can also add a default not found handler to Router() constructor: https://pub.dev/documentation/shelf_router/latest/shelf_router/Router/Router.html
And then calling the .plus extension.


pin_project cannot be used with structs that hold mutable references

Hello,
I just tried using pin-project, as it looks like a really nice helper!
However, while doing so, I hit a wall: with pin-project = "0.4.17",
#[pin_project::pin_project]
pub struct Foo<'a, R> {
    buf: &'a mut [u8],

    #[pin]
    stuff: R,
}

impl<'a, R> Foo<'a, R> {
    fn foo(self: std::pin::Pin<&mut Self>) {
        let this = self.project();
        *this.buf = &mut this.buf[1..];
    }
}

fn main() {
}

fails to compile with this error message:
   Compiling foo v0.1.0 (/tmp/foo)
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'pin in function call due to conflicting requirements
  --> src/main.rs:11:25
   |
11 |         let this = self.project();
   |                         ^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 10:5...
  --> src/main.rs:10:5
   |
10 | /     fn foo(self: std::pin::Pin<&mut Self>) {
11 | |         let this = self.project();
12 | |         *this.buf = &mut this.buf[1..];
13 | |     }
   | |_____^
note: ...but the lifetime must also be valid for the lifetime `'a` as defined on the impl at 9:6...
  --> src/main.rs:9:6
   |
9  | impl<'a, R> Foo<'a, R> {
   |      ^^
note: ...so that the types are compatible
  --> src/main.rs:11:25
   |
11 |         let this = self.project();
   |                         ^^^^^^^
   = note: expected  `std::pin::Pin<&mut Foo<'_, R>>`
              found  `std::pin::Pin<&mut Foo<'a, R>>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0495`.
error: could not compile `foo`.

Weirdly enough, replacing &'a mut by &'a works, so I'd guess something has been implemented specifically for &'a but hasn't been implemented for &'a mut?
Do you think it'd be possible to handle this in pin-project, or is it something that can't be fixed and I should just not use it? :)

Hmm... I could reproduce this without pin-project (playground):
pub struct Foo<'a> {
    buf: &'a mut [u8],
}

impl<'a> Foo<'a> {
    fn foo(self: std::pin::Pin<&mut Self>) {
        self.buf = &mut self.buf[1..];
    }
}

I guess this related to https://github.com/rust-lang/rust/issues/54934, but I'll look into this more.

Ok, this looks like https://github.com/rust-lang/rust/issues/72337 and can be fixed by adding lifetime 'a to std::pin::Pin<&mut Self>:
playground
  impl<'a, R> Foo<'a, R> {
-     fn foo(self: std::pin::Pin<&mut Self>) {
+     fn foo(self: std::pin::Pin<&'a mut Self>) {
          let this = self.project();
          *this.buf = &mut this.buf[1..];
      }
  }

Oh. You're right, this isn't related to pin-project at all, sorry for having bothered you with it!
This being said, your solution unfortunately requires the Pin to be borrowed for the lifetime of the whole struct, which isn't the API I'm looking for, but I'm going to raise that point with the compiler directly.
Thank you! :D

I also commented on https://github.com/rust-lang/rust/issues/72477, but this way should fix it completely:
playground
#[pin_project::pin_project]
pub struct A<'a, R> {
    buf: &'a mut [u8],

    #[pin]
    stuff: R,
}

impl<'a> Foo<'a> {
    fn foo(mut self: std::pin::Pin<&mut Self>) {
        let this = self.project();
        let (_, rest) = std::mem::replace(&mut *this.buf, &mut []).split_at_mut(1);
        *this.buf = rest;
    }
}


Reduce lib/config access points for easier testing

Today there are a bunch of areas in the codebase that directly require lib/config. This makes it challenging to write test cases that depend on different configuration values.
To aid in the testability of the project, config should become part of the app express object where possible, potentially exposed off the req object.
Other areas of the application may need to behave in a similar way (db for example may need to created via some sort of factor function that takes a config object).
Definitely open to other suggestions if anyone has them.

I'm going to consider this mostly addressed as of #540, #541, #542, and #543


Should the API response include units?

As soon as the API responses include physical quantities that have units, we need to decide whether to include these units as part of the response.

pros:

including units (from a standardized set) as part of the response makes it possible to avoid (some) unit conversions.
Numbers may be more human-readable by using units adapted to the data in the corresponding entry

cons:

clients will need to implement unit conversion
the schema will be slightly more complicated
the size of the response increases

We (Fawzi, Markus, Andrius, Leopold) propose, for the moment, to do the simple thing and not include units as part of the response.
Standard units will be defined for all physical quantities. This information will be present in the JSON schema, preferably in machine-readable form.

I agree that for now we decide sensible units and define them in the specification. E.g. in #23 we use angstrom for atom coordinates and cell vectors. Ideally we should always use the same units across the API when sensible

Final task: Make sure that all values that have units in the specs now specify clearly the units, then we can close this issue

Done. Closing the issue.

@merkys does this mean that you checked and all units were already specified?

@merkys does this mean that you checked and all units were already specified?

@giovannipizzi exactly, all seem complete


How can I read a byte without writing anything?

Hello, is there a was I can read a byte over the i2c bus using this node?
I need something that does something like this:
$ i2cget -y 1 0x20
0xff

I tried leaving the command option blank in nodered but i get message saying 'command value is missing'
Thanks

thanks to @exlicite this issue should now been fixed.


Quiet mode for the RPA module? - to learn more and hear other users' inputs

In the TagUI documentation, there is a flag, -q, for running without output to a command prompt. Link to the docs:
https://tagui.readthedocs.io/en/latest/reference.html#run-options
I'd like to ask that quiet mode be added to the next release. In my local RPA-Python repository, I was able to enable this functionality by modifying the tagui.py file. I added a quiet_mode keyword argument to the init() function definition:
def init(visual_automation = False, chrome_browser = True, headless_mode = False, turbo_mode = False, quiet_mode=False)

and then modified the code at line 531 as follows:
try:
    # Do not launch a subprocess shell if quiet_mode is turned on
    if quiet_mode:
        shell_option = False
    else:
        shell_option = True

    # launch tagui using subprocess
    _process = subprocess.Popen(
            tagui_cmd, shell=shell_option,
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE
    )

Thank you for the awesome RPA module!

Hi @jesuisme thanks for raising this issue and sharing your modification! I'm happy that you enjoy using rpa package :)
Can you tell me more what is the observable difference to users, between running shell=True or shell=False?
By design, the process runs in the background, so it is not visible to user, the background command prompt process.

@kensoh
In Windows 11, shell=True causes a Command Prompt to open with the title "Chrome Engine".  The window is a console for the TagUI process, outputing responses to each of the RPA interface functions. For example, the output shown for r.init():
[tagui] START  - listening for inputs

[tagui] INPUT  -
[1] {"id":1,"method":"Page.setDownloadBehavior","params":
{"behavior":"allow","downloadPath":"C:\\Users\\Jessie\\repos\\rpa"}}
[tagui] OUTPUT -
[1] {"id":1,"result":{}}

No Command Prompt opens when shell=False, which is the default behavior for subprocess.Popen():
https://docs.python.org/3.8/library/subprocess.html#subprocess.Popen
I'm running Python 3.8.3 and using the PyCharm IDE. I'm also executing r.init() and r.url() in a wxPython GUI application loop, launched within a multiprocessing Process.  Docs:
https://docs.python.org/3.8/library/multiprocessing.html#multiprocessing.Process

Hi @jesuisme thanks for sharing this! I get your intention now. Can I check with you, other than the command prompt not showing up, is there any other differences when you use the rpa package with shell = False compared to shell = True?
I should be keeping this issue open for some time to see if any other users have this use case. I'll prefer not to add options which are not used by many users, because it will quickly make the API more complex and harder to use. Also, for the time being, I don't have a Windows laptop to validate and do extensive testing.

@kensoh I haven't noticed any other differences. But, I'm not doing much beyond what's in the examples.
I totally understand regarding the API. Thanks again for the work on the module!

Until I get access to a Windows laptop, @jesuisme, could you try to run the following on your modified package without the command prompt windows opening, and let me know the output? And also what is in rpa_python.log file?
import rpa as r
r.debug(True)
r.init()
r.url('https://ca.yahoo.com')
r.type('ybar-sbq', 'github')
print(r.read('ybar-sbq'))
r.click('ybar-search')
r.wait(6.6)
r.snap('page', 'results.png')
r.snap('logo', 'logo.png')
r.close()

Previously when I tried to hide the command prompt on the TagUI engine on Windows, all the data in the transactional windows get routed to the main TagUI command prompt and messes up the output. Not sure if doing it from Python process will lead to similar side effects.

Adding some dev notes -
Running on Mac / Linux with shell=True requires tagui_cmd.split() in the subprocess.Popen() function call, otherwise the invocation of TagUI will fail. Current implementation has double quotes enclosure to support folder paths with spaces.
Trying without double quotes and sending in tagui_cmd as a command file path and parameters list, seems to be unable to launch the process successfully for folder paths with spaces. At least for Mac, not sure of Linux.

In the meantime, I was able to briefly access a Windows laptop. To run with shell=False on Windows, for rpa v1.47, I have to make the following changes. 1. remove the double quotes in the tagui_cmd, 2. add .cmd, 3. add tagui_cmd.split() and 4. set shell=False. However, after adding all these, there is still the Chrome Engine command prompt window on execution.
tagui.py
    # entry shell command to invoke tagui process
    tagui_cmd = '' + tagui_executable + '.cmd' + ' rpa_python ' + browser_option

    # run tagui end processes script to flush dead processes
    # for eg execution ended with ctrl+c or forget to close()
    os.system('"' + end_processes_executable + '"')

    try:
        # launch tagui using subprocess
        _process = subprocess.Popen(
            tagui_cmd.split(), shell=False,
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE
        )

Thus I'm leaning to keeping shell=True unchanged, given the complications with Mac / Linux to run properly, and even on Windows (tested on Windows 10), running with shell=False still result in having the Chrome Engine command prompt window. On your environment it works, but on the Windows I tested on it doesn't. Probably means that the setting has sensitivity to environments, which would make it hard to generalise to work correctly on various environments.

@kensoh I think you're right that environments/paths may be playing a role; I didn't have to make any of those changes. It may also make a difference that I'm running on Windows 11 and that I'm starting the rpa instance in a separate process, not a Pipe.
When I run the automation code from above, everything seems to work as expected and the command prompt does not appear.  The logo.png file contains the Yahoo! logo and results.png contains the search results for GitHub. Contents of the rpa.log:
START - automation started - Sun May 01 2022 17:23:50 GMT-0500 (Central Daylight Time)

LIVE MODE - type done to quit
 [RPA][STARTED]
[RPA][0] - listening for inputs
[RPA][1] - https://ca.yahoo.com
[RPA][1] - listening for inputs
[RPA][2] - exist_result = exist('ybar-sbq').toString()
[RPA][2] - listening for inputs
[RPA][3] - dump exist_result to rpa_python.txt
[RPA][3] - listening for inputs
[RPA][4] - type ybar-sbq as github
[RPA][4] - listening for inputs
[RPA][5] - exist_result = exist('ybar-sbq').toString()
[RPA][5] - listening for inputs
[RPA][6] - dump exist_result to rpa_python.txt
[RPA][6] - listening for inputs
[RPA][7] - read ybar-sbq to read_result
[RPA][7] - listening for inputs
[RPA][8] - dump read_result to rpa_python.txt
[RPA][8] - listening for inputs
[RPA][9] - exist_result = exist('ybar-search').toString()
[RPA][9] - listening for inputs
[RPA][10] - dump exist_result to rpa_python.txt
[RPA][10] - listening for inputs
[RPA][11] - click ybar-search
[RPA][11] - listening for inputs
[RPA][12] - snap page to results.png
[RPA][12] - listening for inputs
[RPA][13] - exist_result = exist('logo').toString()
[RPA][13] - listening for inputs
[RPA][14] - dump exist_result to rpa_python.txt
[RPA][14] - listening for inputs
[RPA][15] - snap logo to logo.png
[RPA][15] - listening for inputs
[RPA][FINISHED]

https://ca.search.yahoo.com/search?p=github&fr=yfp-t&fp=1&toggle=1&cop=mss&ei=UTF-8 - github - Yahoo Canada Search Results
FINISH - automation finished - 18.6s

Thanks for investigating -- probably time to close ;-)

Hi @jesuisme, Larry, thanks for the suggestion nevertheless! I'll close the issue, but will watch out for future requests by other users. It does sound that it works on Windows 11 but not Windows 10. Shall archive this change for now.
In the meantime, I've made 2 commits in your name and rolled back. To register your contribution and idea. Your name should show up shortly in the contributors history of the package -
https://github.com/tebelorg/RPA-Python/graphs/contributors


Pull in campaigns automatically

Pull in campaigns to the takeover screen automatically. As a user signs up for campaigns, they are removed and replaced with others.

The campaigns are pull in based on type, issue, source of traffic or some other category

Or some combination i.e a few collection campaigns, a few based on issue

Only campaigns the users is not signed up for
They should show up in random order.

dosomething_campaign_preprocess_action_page
  $campaign_id = '123';
  $campaign_blurb = "blah blah";

  drupal_add_js(
    array('dosomethingCampaign' =>
      array(
        'recommended_campaigns' => array(
          'campaign_id' => $campaign_id,
          'blub' => $campaign_blurb,
        ),
      ),
    ),
    'setting'
  );

Closed with commit 6344a5f.
Uses existing dosomething_campaign_get_recommended_campaign_nids function to dynamically add campaigns. Only thing that is missing that is listed above is that campaigns are not replaced with others after a user signs up for campaign (instead, "SIGNUP" button just changes text to "Signed up!").


Merge with upstream

I resolved conflicts in src/coreclr/inc/crsttypes.h and src/coreclr/vm/gcheaputilities.cpp.

@joncham: It looks like we have a successful test run after syncing (and resolving a few conflicts this week). I think that we should take this upstream merge.


Stuck  when running on Android  devices

✓  Built build/app/outputs/flutter-apk/app-debug.apk.
Installing build/app/outputs/flutter-apk/app.apk...
D/FlutterLocationService(12143): Creating service.
D/FlutterLocationService(12143): Binding to location service.
i used A50 samsung  Android 11

Hi @abdalazeez1 , please provide a completed and minimal reproducible code sample so that we may verify this. Also, please provide the output of flutter doctor -v as well.
Thank you!


add promtail audit log config link

What does this PR do?
Add the missing link for Promtail audit log configuration example.
What is needed from the reviewers?
Is this link fine or should we instead link directly to the concerned file at https://github.com/giantswarm/promtail-app/blob/master/examples/values-kubernetes-audit-logs.yaml ?

I think it's fine ;)


Update URL viladate, compatible with RFC that is not URL standard

Description
The following validation should not be fail,
This PR fixes #
Notes for Reviewers

Signed commits

[x] Yes, I signed my commits.

Codecov Report

Merging #5287 (a19a201) into master (e60a879) will not change coverage.
The diff coverage is 0.00%.

@@          Coverage Diff           @@
##           master   #5287   +/-   ##
======================================
  Coverage    6.00%   6.00%
======================================
  Files         107     107
  Lines        9824    9824
======================================
  Hits          590     590
  Misses       9074    9074
  Partials      160     160

Flag
Coverage Δ

e2etests
?

gointegrationtests
6.00% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

handlers/oam_handler.go
0.00% <ø> (ø)

mesheryctl/pkg/utils/helpers.go
0.00% <0.00%> (ø)

models/pattern/stages/validate.go
0.00% <0.00%> (ø)

router/server.go
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f4d451...a19a201. Read the comment docs.

@m9rco Great! Thank you for your first PR. It makes sense to me, but I've no idea about why checks failed. I think above golang checks is not related to this PR
cc @leecalcote

Good stuff, @m9rco 💪

@gyohuangxin, right on. These golang lint issues are a separate concern.
@alphaX86, almost completed their fixes?

@leecalcote lint issues are fixed, we just need to rerun the workflow now

@alphaX86 I re-run the jobs, the golangci-lint still seems to fail.

@alphaX86 haha! 😆

@m9rco Yay! 🎈 Congratulations. 😄


Login Issues

when I entered my email and password first time it worked fine and downloaded with good speed. But after that when again i tried login its showing username/password incorrect. I am entering correct credentials.
can you help me out on this.

1.5.1 was working with username and password... but now again showing "Invalid username/password"

I have published v1.5.2. Download and let me know if the problem has been resolved.

Thanks Faisal, its working now !!

Thanks Faisal,
This 1.5.2 version is now working fine.
Thanks.
On Tue, May 22, 2018, 8:36 PM Faisal Umair<EMAIL_ADDRESS>wrote:

I have published v1.5.2. Download and let me know if the problem has been
resolved.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FaisalUmair/udemy-downloader-gui/issues/98#issuecomment-391025540,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYWH7QPL2HyD6KuKXKd0EsjlN71M54Dgks5t1ClZgaJpZM4UHFXj
.


Add fingerprints of TOTEM Door Lock for SmartThingsEdgeDriver

The fingerprints of TOTEM Door Lock have already existed in the DTH(SmartThings Public).
I tried to manage to migrate the fingerprints from DTH to Edge Driver.
Zigbee Device ID: "TOTEM H60/H90" & "TOTEM P30"

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hello, at this time we are only accepting new fingerprint submissions as part of the WWST Works With SmartThings Program. If you work for a device company and would like your device WWST Certified please reach out to<EMAIL_ADDRESS>or check out more information  here.

Hello, I'm now an intern working at SRC-N. The TOTEM fingerprints already exist in the DTH(SmartThings Public).  My leader wants me to migrate the fingerprints from DTH to Edge Driver.

Hey @FrankSpringfield Can you provide the JIRA ticket that your working from so we can have more discussion internally?

Hi, the JIRA ticket is here: https://smartthings.atlassian.net/browse/WWST-8009.

Hi, @lelandblue.
Considering that the 'base-lock' profile has one additional capability, 'lockCodes', compared to the 'lock-battery' profile, in this commit I have changed the deviceProfileName from 'lock-battery' to 'base-lock'.
After testing, we confirmed that the TOTEM H60 can be onboarded correctly through SmartThings after updating the edge driver.


insert API

Allows for

Specific start and end dateTime
Has to be in ISO8601 format
allows setting repeating parameters
set description

https://developers.google.com/calendar/v3/reference/events/insert

Thanks!


Debugger fails with following message

Hi
When i'm trying to debug, it fails with the following message and after that i need to restart Electron, if i try it again it will fail at same point, self it's undefined and if i go step by step it happens
self.options.loading--;

 Fatal error in y:\jenkins\workspace\libchromiumcontent-win\vendor\chromium\src\v8\src\debug\debug.cc, line 511
 Check failed: !it.done() && (it.frame()->fp() == thread_local_.last_fp_).

I think it is a bug of V8, it would be great if you can work out a test case that can reproduce the problem in Chrome browser.
Refs https://github.com/nodejs/node/issues/3253.

Closing since it is a bug of V8 and there is nothing we can do with it in Electron.


fix: avoid using scrollHeight/scrollWidth when the position is fixed and when the offsetParent is the window

Fixes https://github.com/popperjs/popper-core/issues/1323
Note: as master is broken at the moment, it isn't up to date with the latest master, but instead is based on top of f4a6550

Master should be good now, could you please rebase your branch on top of it?


Not supported by NeoOnnx: operator Transpose

Hello.
I'm trying to load the onnx-model after the tf2onnx.convert converter, the program is interrupted with an error:
"Not supported by NeoOnnx: operator Transpose".
I use the ONNX opset version 9. Same error when trying to load any model from https://github.com/onnx/models
How do I solve this?

Hello.
The support of this operator (and some others) will be added soon. I'll comment here when it's done.

Hello again.
As a result we've decided to heavily refactor NeoOnnx.
Now refactoring is finished (#421) and NeoOnnx supports more operators (including Transpose).


[youtube:tab] "Incomplete data received" on "Redownloading playlist API JSON with unavailable videos"

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.09.24 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
US
Provide a description that is worded well enough to be understood
When listing (--flat-playlist) or downloading a YT playlist, a series of warnings and an error about Incomplete data received will appear, before starting download the videos normally.
Also, the count of the video items seems wrong, it will say:
[youtube:tab] Playlist {playlist name} Downloading n+1 items of n
[download] Downloading item 1 of n+1
...
[download] Downloading item n of n+1

Instead of:
[youtube:tab] Playlist {playlist name} Downloading n items of n
[download] Downloading item 1 of n
...
[download] Downloading item n of n

I can still download all the available playlist videos without problem.
I can confirm this still happens when --compat-options no-youtube-unavailable-videos is added.
Something is wrong in a call to the _extract_response() function without the note func arg.

Related code

Redownloading playlist API JSON with unavailable videos:
https://github.com/yt-dlp/yt-dlp/blob/b095fd3fa9d58a65dc9b830bd63b9d909422aa86/yt_dlp/extractor/youtube.py#L5262

Incomplete data received:
https://github.com/yt-dlp/yt-dlp/blob/b095fd3fa9d58a65dc9b830bd63b9d909422aa86/yt_dlp/extractor/youtube.py#L997

[youtube:tab] Playlist {playlist name} Downloading n+1 items of n:
https://github.com/yt-dlp/yt-dlp/blob/b095fd3fa9d58a65dc9b830bd63b9d909422aa86/yt_dlp/YoutubeDL.py#L1960

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', '--flat-playlist', 'https://www.youtube.com/playlist?list=PL4F761764995C5F15']
[debug] Portable config "D:\Programs\Media\yt-dlp\yt-dlp.conf": ['--proxy', 'http://<IP_ADDRESS>:7890/', '--downloader', 'aria2c', '-N', '4', '-r', '128M', '--progress', '--console-title']
[debug] Encodings: locale cp936, fs utf-8, pref cp936, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[088add956] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg N-109420-g262e7439c6-20221214 (setts), ffprobe N-109420-g262e7439c6-20221214, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.35.5, websockets-11.0.3
[debug] Proxy map: {'all': 'http://<IP_ADDRESS>:7890/'}
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 125465ac4416bcd4bf1632b899630ceecc08b341e83fdf10bd34700c58ea5152
yt-dlp is up to date<EMAIL_ADDRESS>[youtube:tab] Extracting URL: https://www.youtube.com/playlist?list=PL4F761764995C5F15
[youtube:tab] PL4F761764995C5F15: Downloading webpage
[youtube:tab] PL4F761764995C5F15: Redownloading playlist API JSON with unavailable videos
[download] Downloading playlist: Madness Combat Series
[youtube:tab] PL4F761764995C5F15 page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Retrying (1/3)...
[youtube:tab] PL4F761764995C5F15 page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Retrying (2/3)...
[youtube:tab] PL4F761764995C5F15 page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Retrying (3/3)...
[youtube:tab] PL4F761764995C5F15 page 1: Downloading API JSON
ERROR: Incomplete data received
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1567, in wrapper
  File "yt_dlp\utils\_utils.py", line 2427, in <lambda>
  File "yt_dlp\utils\_utils.py", line 2195, in __getitem__
  File "yt_dlp\extractor\youtube.py", line 4933, in _entries
  File "yt_dlp\extractor\youtube.py", line 944, in _extract_response
  File "yt_dlp\utils\_utils.py", line 5112, in __iter__
  File "yt_dlp\extractor\common.py", line 3760, in _error_or_warning
  File "yt_dlp\utils\_utils.py", line 5120, in report_retry
yt_dlp.utils.ExtractorError: Incomplete data received

[youtube:tab] Playlist Madness Combat Series: Downloading 18 items of 17
[debug] The information of all playlist entries will be held in memory
[download] Downloading item 1 of 18
[download] Downloading item 2 of 18
[download] Downloading item 3 of 18
[download] Downloading item 4 of 18
[download] Downloading item 5 of 18
[download] Downloading item 6 of 18
[download] Downloading item 7 of 18
[download] Downloading item 8 of 18
[download] Downloading item 9 of 18
[download] Downloading item 10 of 18
[download] Downloading item 11 of 18
[download] Downloading item 12 of 18
[download] Downloading item 13 of 18
[download] Downloading item 14 of 18
[download] Downloading item 15 of 18
[download] Downloading item 16 of 18
[download] Downloading item 17 of 18
[download] Finished downloading playlist: Madness Combat Series

Maybe also see: https://github.com/yt-dlp/yt-dlp/issues/6289

fixed in master/nightly

This is causing test YoutubePlaylist_2 to fail, even after the fix for #8206.
The test needs to ignore the retry warnings and the former "Giving up" error that (I gather) is now a warning:
--- old/yt_dlp/extractor/youtube.py
+++ new/yt_dlp/extractor/youtube.py
@@ -6687,7 +6687,7 @@ class YoutubePlaylistIE(InfoExtractor):
             'uploader_url': 'https://www.youtube.com/@milan5503',
             'availability': 'public',
         },
-        'expected_warnings': [r'[Uu]navailable videos? (is|are|will be) hidden'],
+        'expected_warnings': [r'[Uu]navailable videos? (is|are|will be) hidden', 'Retrying', 'Giving up'],
     }, {
         'url': 'http://www.youtube.com/embed/_xDOZElKyNU?list=PLsyOSbh5bs16vubvKePAQ1x3PhKavfBIl',
         'playlist_mincount': 455,

Below log shows today's master, but similar behaviour occurs with 2023.06.22 once the consent issue is fixed (it also reported "Downloading 5 of 4 items" but this isn't happening now).
The log shows the failing test and than fetching the test URL directly.
Presumably the fix for
As CI download tests are all being skipped, I couldn't find examples in the workflow log.
yt-dl doesn't (yet) know how to _reload_with_unavailable_videos(), which is being tested here, and now returns an empty playlist.
$ python3.9 test/test_download.py TestDownload.test_YoutubePlaylist_2
[debug] Loaded 1890 extractors
[youtube:playlist] Extracting URL: https://www.youtube.com/embed/videoseries?list=PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu
[youtube:tab] Extracting URL: https://www.youtube.com/playlist?list=PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu: Downloading webpage
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu: Redownloading playlist API JSON with unavailable videos
[download] Downloading playlist: JODA15
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu page 1: Downloading API JSON
ERROR: [youtube:tab] Incomplete data received. Retrying (1/3)...; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
Traceback (most recent call last):
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1567, in wrapper
return func(self, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2427, in
return type(self.ydl)._handle_extraction_exceptions(lambda _, i: self._entries[i])(self.ydl, i)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2195, in getitem
self._cache.extend(itertools.islice(self._iterable, n))
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/youtube.py", line 4955, in _entries
response = self._extract_response(
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/youtube.py", line 1001, in _extract_response
should_retry = next(icd_retries, None)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 5114, in iter
self.error_callback(self.error, self.attempt, self.retries)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/common.py", line 3760, in _error_or_warning
RetryManager.report_retry(
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 5128, in report_retry
warn(f'{e}. Retrying{format_field(suffix, None, " %s")} ({count}/{retries})...')
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/common.py", line 1132, in report_warning
self._downloader.report_warning(msg, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/test/helper.py", line 315, in _report_warning
real_warning(w, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/test/test_download.py", line 50, in report_warning
raise ExtractorError(message)
yt_dlp.utils.ExtractorError: [youtube:tab] Incomplete data received. Retrying (1/3)...; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
E
ERROR: test_YoutubePlaylist_2 (main.TestDownload):
Traceback (most recent call last):
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1567, in wrapper
return func(self, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2427, in
return type(self.ydl)._handle_extraction_exceptions(lambda _, i: self._entries[i])(self.ydl, i)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2195, in getitem
self._cache.extend(itertools.islice(self._iterable, n))
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/youtube.py", line 4955, in _entries
response = self._extract_response(
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/youtube.py", line 1001, in _extract_response
should_retry = next(icd_retries, None)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 5114, in iter
self.error_callback(self.error, self.attempt, self.retries)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/common.py", line 3760, in _error_or_warning
RetryManager.report_retry(
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 5128, in report_retry
warn(f'{e}. Retrying{format_field(suffix, None, " %s")} ({count}/{retries})...')
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/extractor/common.py", line 1132, in report_warning
self._downloader.report_warning(msg, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/test/helper.py", line 315, in _report_warning
real_warning(w, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/test/test_download.py", line 50, in report_warning
raise ExtractorError(message)
yt_dlp.utils.ExtractorError: [youtube:tab] Incomplete data received. Retrying (1/3)...; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/df/Documents/src/yt-dlp-new/test/test_download.py", line 172, in test_template
res_dict = ydl.extract_info(
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1556, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1567, in wrapper
return func(self, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1723, in __extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1802, in process_ie_result
return self.extract_info(
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1556, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1567, in wrapper
return func(self, *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1723, in __extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1852, in process_ie_result
return self.__process_playlist(ie_result, download)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1927, in __process_playlist
entries = resolved_entries = list(entries)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 738, in _iter
for x in iterable:
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2392, in get_requested_items
for i, entry in self[index]:
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2454, in getitem
entry = self._getter(i)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/utils/_utils.py", line 2427, in get_entry
return type(self.ydl)._handle_extraction_exceptions(lambda _, i: self._entries[i])(self.ydl, i)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1585, in wrapper
self.report_error(str(e), e.format_traceback())
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 1045, in report_error
self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
File "/home/df/Documents/src/yt-dlp-new/yt_dlp/YoutubeDL.py", line 984, in trouble
raise DownloadError(message, exc_info)
yt_dlp.utils.DownloadError: ERROR: [youtube:tab] Incomplete data received. Retrying (1/3)...; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

Ran 1 test in 6.050s
FAILED (errors=1)
$ # Fetching the same playlist directly
$ python3.9 -m yt_dlp -v --flat-playlist 'https://www.youtube.com/embed/videoseries?list=PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu'
[debug] Command-line config: ['-v', '--flat-playlist', 'https://www.youtube.com/embed/videoseries?list=PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[b634ba742] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: 4a601c9ef
[debug] Python 3.9.18 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-glibc2.23 (OpenSSL 1.1.1w  11 Sep 2023, glibc 2.23)
[debug] exe versions: ffmpeg 4.3, ffprobe 4.3
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2019.11.28, requests-2.25.1, secretstorage-3.2.0, sqlite3-3.37.2, urllib3-1.25.11
[debug] Proxy map: {}
[debug] Request Handlers: urllib
[debug] Loaded 1890 extractors
[youtube:playlist] Extracting URL: https://www.youtube.com/embed/videoseries?list=PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu
[youtube:tab] Extracting URL: https://www.youtube.com/playlist?list=PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu: Downloading webpage
WARNING: [youtube:tab] YouTube said: INFO - 4 unavailable videos are hidden
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu: Redownloading playlist API JSON with unavailable videos
[download] Downloading playlist: JODA15
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Retrying (1/3)...
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Retrying (2/3)...
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Retrying (3/3)...
[youtube:tab] PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu page 1: Downloading API JSON
WARNING: [youtube:tab] Incomplete data received. Giving up after 3 retries
[youtube:tab] Playlist JODA15: Downloading 4 items of 4
[debug] The information of all playlist entries will be held in memory
[download] Downloading item 1 of 4
[download] Downloading item 2 of 4
[download] Downloading item 3 of 4
[download] Downloading item 4 of 4
[download] Finished downloading playlist: JODA15
$


HBASE-22654 apache-rat complains on branch-1

License check fails after build with errorProne profile activated. The hbase-error-prone module is added to the root pom only when the profile is active so running RAT check does not consider hbase-error-prone/target as a build directory.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
1634
Docker mode activated.

_ Prechecks _

+1
@author
0
The patch does not contain any @author tags.

-0
test4tests
0
The patch doesn't appear to include any new or modified tests.  Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ branch-1 Compile Tests _

+1
mvninstall
152
branch-1 passed

+1
compile
93
branch-1 passed

-1
shadedjars
22
branch has 10 errors when building our shaded downstream artifacts.

+1
javadoc
171
branch-1 passed

_ Patch Compile Tests _

+1
mvninstall
99
the patch passed

+1
compile
194
the patch passed

+1
javac
194
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
xml
4
The patch has no ill-formed XML file.

-1
shadedjars
49
patch has 10 errors when building our shaded downstream artifacts.

+1
hadoopcheck
439
Patch does not cause any errors with Hadoop 2.8.5 2.9.2.

+1
javadoc
122
the patch passed

_ Other Tests _

-1
unit
9488
root in the patch failed.

+1
asflicense
27
The patch does not generate ASF License warnings.

12712

Reason
Tests

Failed junit tests
hadoop.hbase.util.TestHBaseFsck

Subsystem
Report/Notes

Docker
Client=18.09.5 Server=18.09.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-357/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/357

Optional Tests
dupname  asflicense  javac  javadoc  unit  shadedjars  hadoopcheck  xml  compile

uname
Linux 91737288b6fc 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
/testptch/patchprocess/precommit/personality/provided.sh

git revision
branch-1 / 1d93622

maven
version: Apache Maven 3.0.5

Default Java
1.8.0_212

shadedjars
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-357/1/artifact/out/branch-shadedjars.txt

shadedjars
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-357/1/artifact/out/patch-shadedjars.txt

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-357/1/artifact/out/patch-unit-root.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-357/1/testReport/

Max. process+thread count
3951 (vs. ulimit of 10000)

modules
C: . U: .

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-357/1/console

Powered by
Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.