v2.2.1-alpha quirks

Hello,
I wanted to report a few things I noticed and ask for quick guidance!
I found that when copying  ALVRFreePIE.dll to C:\Program Files (x86)\FreePIE\plugins, the file NEEDS to be unblocked in its properties, otherwise Freepie will run as a process (it will show in Task Manager), but it will not draw any GUI (so it practically borks FreePie). Maybe there's a way to unblock it before packaging it or else it should be noted that this change in the properties needs to occur for anything to work!
I noticed the new "Launch only SteamVR without Steam", but for me, every single previous release has been already doing that, in fact, if anything I would have expected the opposite option "Launch Steam AS WELL". So now I'm not sure if what's been happening on my machine is in fact an anomalous behavior, or what?!
When starting the server (regardless of whether you select to start with or without Steam), if SteamVR crashes, or you X it out (quit), the process vrserver.exe doesn't get killed, leaving ALVR status to "Server is Alive", making it impossible to restart SteamVR from ALVR, unless you kill vrserver.exe manually. Restarting ALVR doesn't cure this, in fact it makes things even worse by showing that's "alive" and connected to the client (if you previously closed ALVR while being connected to the client) while the client is truly not receiving anything (as it displays the message to "click connect"). The server though thinks it's connected so you are forced to disconnect/reconnect the client manually.
Ideally, exiting ALVR should kill every and all SteamVR related process (especially if started by it in the first place) or at least not get in the way of SteamVR "quit routine". It was convenient though to be able to quit SteamVR and return the to the "Server is Down" status without needing any taskkill operations.
Lastly, as I already mentioned in #79, I was hoping to make the positional tracking generated by  PSMoveSteamVRBridge+PSMoveFreepieBridge work with ALVR. While I am pumped that you integrated FreePie into the project, I couldn't for the life of me figure out how to adapt the script provided in  PSMoveFreepieBridge to the new variables you created.
Here is the "original" script I'm trying to work with (which allows Riftcat to use the data in their stream):
def update():
freeTrack.pitch = -freePieIO[0].pitch
freeTrack.roll =  freePieIO[0].roll
freeTrack.yaw = freePieIO[0].yaw
#Multiply by 10 for SteamVR use
freeTrack.x = (freePieIO[0].x * 10)
freeTrack.y = (freePieIO[0].y * 10)
freeTrack.z = (freePieIO[0].z * 10)

if starting:
freePieIO[0].update += update
I "get" it should get mashed-up with the following, but not fully getting how:
global offset
if starting:
offset = [0.0, 0.0, 0.0]
alvr.override_head_position = True
alvr.head_position[0] = alvr.input_head_position[0] + offset[0]
alvr.head_position[1] = alvr.input_head_position[1] + offset[1]
alvr.head_position[2] = alvr.input_head_position[2] + offset[2]
I only want to use the x.y.z from the "original" script (I'm using a ping pong ball lightbulb tracker, and obviously prefer to use the GearVR IMU for the head orientation), and I'm assuming that freePieIO[0].x=alvr.input_head_position[0] (are they the same?). So, yeah, I'm stumped!
Hope you will shine some wisdom this way!
ありがと

I can confirm the vrserver.exe not getting killed properly. this is a bad problem, something that plagues NOLO still. It gets annoying very quickly as you have to manually kill it in task manager before you can use steamVR again, even if using it with a vive.

I can confirm the vrserver.exe not getting killed properly. this is a bad problem, something that plagues NOLO still. It gets annoying very quickly as you have to manually kill it in task manager before you can use steamVR again, even if using it with a vive.

OUCH! I'm sure this is a regression though, not a new feature. Every other previous build was shutting down SteamVR properly. To mitigate this, we can probably create a .bat that kills everything, to be executed on disconnect (although then you break the autoconnect if you manually disconnect). I'm confident @polygraphene will sort things out (and hopefully, magically make the VirtualHMD ping pong ball positional tracking work!)

Thank you for detailed report!

NEEDS to be unblocked

We need proper installer. #74 will resolve it.

"Launch only SteamVR without Steam" ... already doing that

Yes. I forgot to mention. Default behaviour was changed!
Previous version launch only SteamVR. And new version launch Steam and you can rollback behaviour by enabling that option.
As you said, we need more robust mechanisms to manage SteamVR lifecycle. We can put "kill steamvr" button to GUI, but it is not "clean" solution.
related: https://github.com/ValveSoftware/openvr/issues/473 https://github.com/ValveSoftware/openvr/issues/779

@vrokolos You can use "DebugCaptureOutput" on debug tab. It create "capture.h264" (even when you use H265) file on driver folder. I think your issue is caused by jitter of network. The jitter cause client to drop frames. As we disscussed on #42, we need buffering mechanism on client side to reduce frame drop.

Great. I just want to make sure it's client side. I'll test it out thanks

@vrokolos The file is raw h265 stream. You can play with ffplay, or transcode it with ffmpeg.

Yeah got it thanks

@VRtinker I have an automated setup of tasker and eventghost that will wake up my pc and run alvr server and bigscreen whenever i run alvr client and will kill all vr services when i remove my gear vr. Can give you more info if you need it

@polygraphene I noticed some client crashes also. Is there a crash log on the client or should I use adb to get it?

@vrokolos  Client has no crash log. You need to use adb. Thank you as always!
@VRtinker @Lightsteed Oh! I missed the point. That was a bug of newer version. It will be fixed on next alpha release.

I didn't try, but it should look something like the following.
alvr.override_head_position = True

alvr.head_position[0] = freePieIO[0].x * 10
alvr.head_position[1] = freePieIO[0].y * 10
alvr.head_position[2] = freePieIO[0].z * 10

alvr.head_position[0],[1],[2] mean x,y,z coordinate respectively.

@vrokolos Thanks for your offer, though I already have a system in place. I use Remote Launcher Free on the phone and its relative server side applet. I then have a NFC tag configured to initiate the sequence and one to terminate it, so that I can just bang one of the controller to the phone to start everything and the other to kill everything. I use an NFC tag attached to a daydream controller to retool the Gear VR into a Daydream View (that's where I got the idea).
@polygraphene

alvr.override_head_position = True
alvr.head_position[0] = freePieIO[0].x * 10
alvr.head_position[1] = freePieIO[0].y * 10
alvr.head_position[2] = freePieIO[0].z * 10

YAY! It works! except I had to change from *10 to /100, so here is how the script looks like now:
alvr.override_head_position = True
alvr.head_position[0] = freePieIO[0].x /100
alvr.head_position[1] = freePieIO[0].y /100
alvr.head_position[2] = freePieIO[0].z /100
It was just a cursory test but seems pretty spot on. As soon as I turn on the ping pong ball, it immediately tracks in space, and as I turn it off, it throws me right back in the initial position. When multiplying "*10" it would throw me up high, waaaaaay up high, so high you wouldn't even see the guidelines, just the default gray background. Using "/10" keeps me "grounded", but it's waaaaay too sensitive (heavy breathing would be enough to skew a few meters to the sides). "/100" felt about right, but maybe there's a way to calculate this number exactly instead of through guesstimation?!?
During the next 24hrs I will put the setup under a more proper stress test to see how it holds up.
Thanks a lot, again, for your outstanding work!

Great!
SteamVR accepts coordinates in meters. Put alvr.head_position[*] in meters.
And Your PSMove may report in centi-meter.

So I had a chance to test a little more and arrived to the conclusion that an offset needs to be added in the script in order to attempt proper positional tracking. By using the diagnostic.watch function I determined that y and z both need to be adjusted with an offset that should look like this: [0.0, 0.1, -0.1]. I tried to following:
global offset
if starting:
offset = [0.0, 0.1, -0.1]
alvr.override_head_position = True
alvr.head_position[0] = alvr.input_head_position[0] + offset[0]
alvr.head_position[1] = alvr.input_head_position[1] + offset[1]
alvr.head_position[2] = alvr.input_head_position[2] + offset[2]
alvr.head_position[0] = freePieIO[0].x /100
alvr.head_position[1] = freePieIO[0].y /100
alvr.head_position[2] = freePieIO[0].z /100
diagnostics.watch(alvr.input_head_position[0])
diagnostics.watch(alvr.input_head_position[1])
diagnostics.watch(alvr.input_head_position[2])
diagnostics.watch(alvr.head_position[0])
diagnostics.watch(alvr.head_position[1])
diagnostics.watch(alvr.head_position[2])
but it seems to have no effect. I tried to exaggerate the values to make the changes appear more obvious, but still no luck, so no doubt, the syntax is wrong. I'm confused about whether alvr.head_position already incorporates the offset or not. In other words,  does "alvr.head_position[0] = alvr.input_head_position[0] + offset[0]" need to be specified, or is it always the case? if it is implicit, than just specifying some offset values at the top of the script would be enough. Any wisdom thrown in this direction on this matter will be appreciated!
The reason why an offset is need is in the first place is because the tracker sits on the top-center of my head (on the top strap) while the system probably assumes its position to be where my eyes are. So when looking down, the position shifts forward.
So, how do you correctly apply an offset?
Cheers!

alvr.head_position[0] = alvr.input_head_position[0] + offset[0]
alvr.head_position[1] = alvr.input_head_position[1] + offset[1]
alvr.head_position[2] = alvr.input_head_position[2] + offset[2]
alvr.head_position[0] = freePieIO[0].x /100
alvr.head_position[1] = freePieIO[0].y /100
alvr.head_position[2] = freePieIO[0].z /100

You overridden head_position.
This may work.
alvr.head_position[0] = alvr.input_head_position[0] + offset[0] + freePieIO[0].x /100
alvr.head_position[1] = alvr.input_head_position[1] + offset[1] + freePieIO[0].y /100
alvr.head_position[2] = alvr.input_head_position[2] + offset[2] + freePieIO[0].z /100

What are you using to get positional tracking? Driver4vr?

No, as I said in the first post here, and in #79, I'm using PSMoveSteamVRBridge+PSMoveFreepieBridge.
Now off to figure out this offset ordeal!

Ah i see.. yes this is going to be a common issue for other drivers interacting with ALVR it seems. I tried using openvr input emulator to change the position of the oculus go hmd when using alvr and it does not work at all. Im thinking it will be best for the albr dev to try and make the software compatible with other such drivers so that they can take more control or position and orientation and not lose any fidelity on the image. It seems to work for vridge, but they do not use ATW, so there are probably challenges to overcome there.

@polygraphene
Thanks for your pointer!
While your suggestion works (yay!) it does require to declare the offset variable and its value at the beginning of the script, as in:
global offset
if starting:
offset = [0.0, 0.1, -0.1]
alvr.override_head_position = True
alvr.head_position[0] = alvr.input_head_position[0] + offset[0] + freePieIO[0].x /100
alvr.head_position[1] = alvr.input_head_position[1] + offset[1] + freePieIO[0].y /100
alvr.head_position[2] = alvr.input_head_position[2] + offset[2] + freePieIO[0].z /100
#optional, for diagnostic purposes
diagnostics.watch(alvr.input_head_position[0])
diagnostics.watch(alvr.input_head_position[1])
diagnostics.watch(alvr.input_head_position[2])
diagnostics.watch(alvr.head_position[0])
diagnostics.watch(alvr.head_position[1])
diagnostics.watch(alvr.head_position[2])
I found I can instead use this formula to get the same result without the mouthful :-P
alvr.override_head_position = True
alvr.head_position[0] = freePieIO[0].x /100
alvr.head_position[1] = (freePieIO[0].y /100) + .1
alvr.head_position[2] = (freePieIO[0].z /100) - 0.1
#optional, for diagnostic purpose
diagnostics.watch(alvr.input_head_position[0])
diagnostics.watch(alvr.input_head_position[1])
diagnostics.watch(alvr.input_head_position[2])
diagnostics.watch(alvr.head_position[0])
diagnostics.watch(alvr.head_position[1])
diagnostics.watch(alvr.head_position[2])
Maybe the first script is cleaner since should you ever need to change the offset's value, you can do it all at once on the declaration line, but on the other hand, it's not like you need to ever change the offset again once you find the correct values, so unless you see an adverse effect when using the second script, I will probably stick with that!
I'm still getting a bit of "bouncy" feeling when looking down, or moving in general, but, for the moment, I'm writing it off to the fact that I'm using only 2 PS3 cam for tracking (it's kinda like a smoothing feel, hard to describe, maybe I'll make a video if you are interested - even though I'm not sure it would even appear evident on a 2D screen) . Since I actually bough 6 of them (for a whooping $11.94 total!), I'll try to hook up one or two more to see if that improves the overall experience.
I'll be back when I gather more data!
@Lightsteed
Not sure what's the "common issue" you are referring to, but tracking works perfectly fine without using driver4vr. It sure can use some tuning, but at least the method to add an offset is now clear.

@VRtinker
I'm using PSMoveSteamVRBridge and PSMoveFreepieBridge as well, but after running the following script in FreePie I can't get positional tracking in ALVR.I wonder what's the problem.
global offset
if starting:
offset = [0.0, 0.1, -0.1]
alvr.override_head_position = True
alvr.head_position[0] = freePieIO[0].x /100
alvr.head_position[1] = (freePieIO[0].y /100) + .1
alvr.head_position[2] = (freePieIO[0].z /100) - 0.1
#optional, for diagnostic purpose
diagnostics.watch(alvr.input_head_position[0])
diagnostics.watch(alvr.input_head_position[1])
diagnostics.watch(alvr.input_head_position[2])
diagnostics.watch(alvr.head_position[0])
diagnostics.watch(alvr.head_position[1])
diagnostics.watch(alvr.head_position[2])
I use LED ping pong ball for head tracking in PSMoveService and it tests good, the software I am using:
PSMoveSteamVRBridge_1.5.0
PSMoveFreepieBridge-Release15
ALVR-portable-v2.3.1
Any help is appreciated.

Problem sloved after updating PSMoveFreepieBridge to release 16 by HipsterSloth. Finally got my GO 6dof tracking!


Unaligned loads in decoder fail on asm.js.

Asm.js doesn't support unaligned loads, but WASM does.   These are both 32-bit compiles.  I've noted that for most test inputs, the lzfse reference decoder fails to decode in asm.js but works in wasm as a result.  It looks like there were attempts to avoid unaligned loads, but that they weren't all fixed.  Compression libraries often have a flag to enable/disable unaligned load usage.
Here's where the first failure occurs and this halts the decode.
err: -3 lzfse_decode_base.c 224
Decode failed with size 0 into buffer 2706.
    //  Error if D is out of range, so that we avoid passing through
    //  uninitialized data or accesssing memory out of the destination
    //  buffer.
    if ((uint32_t)D > dst + L - s->dst_begin)  <- this is a compile warning already
      return LZFSE_STATUS_ERROR;

Running "Undefined Behavior Sanitizer" on the xcode 64-bit build turns up these.  There are also about 20 unsigned/signed mismatch warnings throughout, and many teams like our have "warnings as errors" enabled.  These may not be indicative of the unaligned load issue, but seem worth investigation.
lzfse_decode_base.c:297:20: runtime error: addition of unsigned offset to 0x00011480a46f overflowed to 0x00011480a36a
lzfse_decode_base.c:243:20: runtime error: addition of unsigned offset to 0x00011480b106 overflowed to 0x00011480b105
The decode works for very small data sets, but anything above a given byte count fails.
Also the decoder example seems to treat 0 the same as returning dest length in the growth loop.  That seems to only be returned for the error case.

@alecazam do you have any example code using the wasm to decode a buffer?

I don’t have access to that code anymore.  I basically took the reference implementation, and was deciding the clipboard data from Sketch.  I had to skip the first 12 bytes or so, and then the decode worked in wasm.  It was asm.js that failed.  But for large content both failed IIRC.


Add diagnostics method to TracePosterior class

Addresses #1560.
This refactors the mcmc module as follows:

Holds chain id information in TracePosterior, and ensures that we are able to deterministically sample from the TracePosterior object in the presence of multiple chains. This allows us to remove the gnarly code that I introduced in #1506 for this purpose.
Adds a diagnostics method to an MCMC specific EmpiricalMarginal which returns stats from @fehiepsi's ops.stats (currently effective sample size and gelman rubin statistic), but more will be added. I initially tried to share the code between the MCMC and Importance EmpiricalMarginal but it turned out to be quite convoluted getting existing oed and importance code to work given that the new implementation does not override Empirical and has slightly different interface. As such, I have tried to not refactor the importance and search code. There is more code sharing that could be done, but the benefits are questionable.

The baseball example is modified to use this and is a minor refactoring over @fehiepsi's previous PR - #1545.
TODO:

[x] Ensure that deterministic sampling order is preserved despite samples being stored out of order in EmpiricalMarginal.
[x] Fix mean, variance in Empirical in the presence of multiple chains.
[ ] Test diagnostics is correctly computed for different latent shapes.
[ ] Test diagnostics is correctly computed when run on CUDA.

Thanks @neerajprad ! All the changes make sense to me. I guess you would also need to adapt svi _traces to this change? When you finish, I'll go for another pass.

@fehiepsi - I think it will be better to move this into the empirical distribution. That way anyone should be able to access the tensor directly and be able to see samples for each chain (e.g. if we would like to have something like traceplot). I will make that change in this PR.

@neerajprad - Agree! As we discussed on Monday, a restriction is EmpiricalMarginal requires samples have a same shape. I am waiting to see your idea. Here is an api I have in mind
mcmc = MCMC(...).run()
marginal = EmpiricalMarginal(mcmc, sites=["a", "b", "c"])  # sites a, b, c does not need to have a same shape; hence empirical marginal is just a container of Empirical for each site
marginal.effective_sample_size()  # return a dict of `n_eff` with keys "a", "b", "c"
marginal.gelman_rubin()  # return a dict of `r_hat` with keys "a", "b", "c"
marginal.sites["a"]  # return a empirical distribution for "a"
marginal.sites["a"].effective_sample_size()  # return n_eff with keys "a"

Basically, I think that all the computation should go under empirical distribution class. And EmpiricalMarginal is just a convenient wrapper for a list of sites.

I think that's a good idea! In that case the restriction on the empirical distribution does not matter, we can have a separate empirical distribution for each of the marginals.

Since that will be a breaking change, I created #1557 to gather more opinions / suggestions.

Sure ;)

I think the interfaces are fine and I'd be happy to merge this as-is, but I wonder if it makes sense to factorize the internals slightly differently:

Have a basic vectorized Empirical distribution that would be constructed from already-_finalized sample and weight tensors, whose batch_shape is the same as a Categorical constructed from the weights and whose event_shape is determined by aligning the batch_shape and the sample tensor shape on the left.
Move all of the _finalize logic into _Marginal.populate_traces, so that the full sample and weight tensors are constructed there; for each site, construct a single sample/weight tensor pair whose leftmost dimension is the chain dimension and use that to construct an Empirical marginal for the site
Move the diagnostic logic into _Marginal (or into an MCMCMarginal distribution that inherits from Empirical that is constructed for each site and knows which dimension is the chain dimension)

@neerajprad you may have already tried something like this and found problems with it, WDYT?

@neerajprad you may have already tried something like this and found problems with it, WDYT?

I think it's a good idea to implement (1) and (2), and have consistent semantics for vectorizing! For (3), I think at that point we might as well just put that logic in EmpiricalMarginal, and each site's marginal can be an instance of that. `Let me try to refactor, and get back on this.

This should be good for review again, with the design changed as per @eb8680's suggestions.


Change virtualbox ip address

Is it possible to change the Virtualbox Docker Machine's IP address? I tried it and it lost association with Kitematic.

By default on a VM, the Docker-Machine created will have the IP address of <IP_ADDRESS> - I don't believe there are any commands to change it, unless you re-write the driver for it.

Can this be added as enhancement then?

If you're so incline, you could just pull the source code from github and just set it to whatever you wanted:
https://github.com/docker/machine/blob/c10962320729a8d436eb09ce9aaae1f4bdd7d0ad/drivers/virtualbox/virtualbox.go

I'll add it as a feature request for people to vote on.
@TeckniX's suggestion is sound if you're inclined to do so.

where is the location for the feature request vote on this issue? Would be a huge bonus for me/my testing to be able to assign/adjust IP addresses. Thanks!

@TechJournalist Sorry, I meant this as a place where people can comment and help submit a PR.

ok - and how can i help submit a PR or get this changed?

k i'm getting stuck trying to figure out how to make this work / but happy to try and contribute. Does the -defaultHostOnlyCIDR  - variable just need to be set to FOO or?

Would love to see this in the GUI so normal users can get this done easily.

Hi @TechJournalist , to do a pull request on Kitematic please feel free to fork Kitematic's code, make the necessary changes on your repository, and then make a pull request to us. That is when we will review the code with the team.
I believe this might be a feature for Docker machine - adding @nathanleclaire from Machine team for technical help. Also cc'd @jeffdm
Thanks for helping!

@TechJournalist a current workaround, assuming you've got the Toolbox installed:

Close Kitematic
Delete the current default VM: docker-machine rm -f default
Create a new default machine docker-machine create -d virtualbox --virtualbox-hostonly-cidr "<IP_ADDRESS>/24"
Open Kitematic and it should be using this VM.

As for making this doable via the GUI, I think eventually we'll want KM to be configurable to point to any host but this is work in progress. Check out https://github.com/kitematic/kitematic/pull/876 for a cool example of a team at the DockerCon 2015 hackathon (see #3, Lightning Kite) https://blog.docker.com/2015/07/highlights-dockercon-2015-hackathon/

I believe most of these to be fixed in the latest Docker for Mac/Windows+Kitematic.


Can the caching solution be uised to avoid refetching a document during a single resolver call ?

In many use cases, I want to check if a document exists, and if it does, I want to cache it in my query's context so I may access it later without having to hit the database once again.
Similarly I often require some document in my DB hooks that would benefit from the cache (e.g. : when updating an invoice row document, I need the Invoice document, and it is likely I've updated several rows in the same mutation, meaning I'd highly benefit from fetching the invoice only once and there sharing it.)
For now, I've build my own basic mechanism for that kind of caching, but the documentation regarding Caching at GQL level seems to only provide a way to avoid refetching a query result if the same variables query with the same variables have been called a moment before.

Bluelibs have an internal caching executor , think that would help
  Query: [
    [],
    {
      UsersFind: [X.Cache([X.ToNova(UsersCollection)])],
      UsersCount: [X.ToCollectionCount(UsersCollection)],
    },
  ];

For more config options check the caching paragraph in the documentation x-framework >server >core

HI @y-moufakkr , Yes, 'that's actaully what I'm talking about :)
Is this cahe mechanism only purpose to cache and resend a query response if it's called with the same varaibles twice in short timespan, or does it make the cached documents accessible dinside the resolvers and collections events ?

Hi @Fen747 , well I ve worked on it just for queries responses, but its flexible on resolvers, but not so sure on collection events, but in all cases, you can always  ignore the X.cache executor  to use the CacheService methodes get ,set  and generateCacheKey to build ur own logic

I think @Fen747 wants a X.CheckDocumentExists({ cache: { body, etc}); and then a mechanism to take that document from the cache. @Fen747 do you have an API proposal?

Basically, what I did is a simple set of set/setup/get functions that read a Symbol in the provided object (meant to the the context object), and I made an executor that replaces X.CheckDocumentExists that caches the result and takes a option to throw if the document is missing.
Here it is


IronPython is mishandling the bytes literal

In https://docs.python.org/2/reference/lexical_analysis.html#string-literals it states:
A prefix of 'b' or 'B' is ignored in Python 2; it indicates that the literal should become a bytes literal in Python 3 (e.g. when code is automatically converted with 2to3)
If I run this:
print repr(b'2433333')
I see:
b'2433333'
This impacts on libraries like shapely

In Python 2 bytes == str which is why the prefix is ignored. However, in IronPython 2 str == unicode so the b prefix cannot be ignored.


match+gym not automatically setting up provisioning profiles

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description

I'm trying to set up my pipeline to use fastlane match+gym to build my app, but consistently receive an error stating
error: "<TARGET NAME REDACTED>" requires a provisioning profile with the App Groups feature. Select a provisioning profile in the Signing & Capabilities editor. (in target '<TARGET NAME REDACTED>' from project '<PROJECT NAME REDACTED>')

I was expecting match to automatically set all of that up. When I go into my xcode project and manually set the provisioning profile for my targets to the profiles installed by match, and then run gym, it works as expected and successfully builds and archives the app. If I don't manually do that, it fails with the message above.
Based on http://docs.fastlane.tools/codesigning/xcode-project/, it seems like this might be intended behaviour. Would you be open to me creating a pull request to go through the provisioning profiles from match and update the targets via fastlane before running gym? I think it would be a much cleaner experience to be able to just say
lane :appstore-build do
  match(type: 'appstore')
  gym(export_method: 'app-store')
end
lane :enterprise-build do
  match(type: 'enterprise')
  gym(export_method: 'enterprise')
end

and have that work without needing to manually update provisioning profiles in xcode between running the two lanes.
I think I have pretty much everything I need to get that working and create a PR, except for the ability to get the certificate name (in order to update the codesigning identity) from match. Would appreciate any support on figuring out how to get that value from match, but if we can do that, then I would be happy to update the actions and create a PR to set that all up automatically without needing any intervention.

I know that tagging you is a bit rude, so apologies in advance, but @joshdholtz you've been very helpful with my PRs in the past so would love to hear if this is something you would be interested in getting added to fastlane, and if you could help me with getting the certificate name. Thanks!

I am also facing exact same issue. Is your issue resloved? any recommendation how to solve this issue.? Can any one please help?

This works really well for us, thank you!	Facing same issue here. Help is appreciated.
 MyCoverageInfo.com

When I quicklook the profiles I see this, and the value next to "Name" under certificates is the value that I want.

FastlaneCore::ProvisioningProfile doesn't seem to populate this value. I do see DeveloperCertificates as a key, but that references an array of StringIOs. When I run lines = profile['DeveloperCertificates'][0].readlines I get what looks like some encoded strings but haven't been able to decrypt those. I do see "iPhone Distribution: ." in lines[4], but no clue how to reliably parse that out :/
If you know what format those are and if there's a way to reliably parse those it'd be super helpful

Hi @tejassharma96 this is a really nice idea - having a logic to automatically setup profile in the project based on match result would probably make it easier to use fastlane.
As you noticed though, leaving a decision about code signing for the users looks like an intended behaviour by design. I have a feeling that such a functionality should be opt-in, and I'm thinking if it shouldn't be turned off by default and enabled with an option switch in gym. Otherwise it could potentially be a breaking change and cause issues for the current users after update to newer version of fastlane. What do you think?
And as a side note, currently code signing settings in the project can be either handled manually in Xcode or changed with action http://docs.fastlane.tools/actions/update_project_provisioning/#update_project_provisioning
just between match and gym call. Match adds environment variable with path to provisioning profile and it could be used in update_project_provisioning to set the profile. If you also need to change team, you could use update_project_team action.

right I was thinking I could add a update_targets_automatically flag to match that defaults to false. I wrote a custom action that achieves this and was hoping we could just get it into fastlane instead, since it definitely feels like something fastlane could/should support. I'd argue that it should be in match over gym since just running match will automatically set up code signing for your xcode project/workspace and then you can build in xcode too vs just via gym. I'd also love to get https://github.com/fastlane/fastlane/pull/20187 merged in to support this. I'd be happy to create a draft PR of how it might look directly integrated into match once that gets merged.
I can't link to my action since it's in an internal repo, but I've copied it below:
module Fastlane
  module Actions
    class EnhancedMatchAction < Action
      INFOPLIST_FILE_KEY = "INFOPLIST_FILE"
      BUNDLE_ID_KEY = "CFBundleIdentifier"
      TEAM_ID_KEY = "DEVELOPMENT_TEAM"
      CODE_SIGN_IDENTITY_KEY = "CODE_SIGN_IDENTITY"
      PROFILE_NAME_KEY = "PROVISIONING_PROFILE_SPECIFIER"
      CODE_SIGN_STYLE_KEY = "CODE_SIGN_STYLE"
      CODE_SIGN_STYLE_VALUE = "Manual"
      CERTIFICATE_DATA_KEY = "DeveloperCertificates"
      CERTIFICATE_NAME_KEY = "CN"

      def self.run(params)
        other_action.match(self.match_options(params).dup)

        if params[:update_targets_automatically]
          mapping = Actions.lane_context[SharedValues::MATCH_PROVISIONING_PROFILE_MAPPING]
          unless mapping
            UI.user_error!("Could not load the provisioning profile mapping from match")
            return
          end
          self.update_targets_automatically(params, mapping)
        end
      end

      def self.description
        "Enhanced match"
      end

      def self.authors
        ["Tejas Sharma"]
      end

      def self.details
        "Runs fastlane match and uses the provisioning profile mapping to automatically update the targets "\
        "in your xcodeproj files. You must provide the team id and code sign identity manually"\
      end

      def self.available_options
        Fastlane::Actions::MatchAction.available_options + [
          FastlaneCore::ConfigItem.new(key: :build_configurations,
                                       env_name: "ENHANCED_MATCH_BUILD_CONFIGURATIONS",
                                       optional: true,
                                       type: Array,
                                       description: "Specify build_configurations you want to update signing for. (default to all configurations)"),
          FastlaneCore::ConfigItem.new(key: :code_sign_identity,
                                       env_name: "ENHANCED_MATCH_CODE_SIGN_IDENTITY",
                                       type: String,
                                       description: "Code signing identity type (iPhone Developer, iPhone Distribution). If you don't provide a value for this, "\
                                                    "the action will attempt to parse the value from your provisioning profile",
                                       optional: true),
          FastlaneCore::ConfigItem.new(key: :update_targets_automatically,
                                       env_name: "ENHANCED_MATCH_UPDATE_TARGETS",
                                       description: "Specify whether the action should automatically update provisioning profiles for your targets."\
                                                    "You must pass in values for team_id and code_sign_identity when setting this to true",
                                       default_value: false,
                                       type: Boolean,
                                       optional: false)
        ]
      end

      def self.is_supported?(platform)
        [:ios, :mac].include?(platform)
      end

      def self.match_options(params)
        Fastlane::Actions::MatchAction.available_options.map { |i| [i.key, params[i.key]] }.to_h.compact
      end

      def self.update_targets_automatically(params, mapping)
        Helper::XcodeprojHelper.get_all_xcodeprojects.each do |proj|
          update_signing_for_targets(params, proj, mapping)
        end
      end

      # rubocop:disable Metrics/PerceivedComplexity
      def self.update_signing_for_targets(params, project, mapping)
        project.targets.each do |target|
          target.build_configurations.each do |config|
            if params[:build_configurations] && !params[:build_configurations].include?(config.name)
              UI.verbose("Skipping #{config.name} not selected (#{params[:build_configurations].join(',')})")
              next
            end

            bundle_id = self.get_bundle_id(config)
            unless bundle_id
              UI.verbose("Skipping #{config.name} for #{target.name}, no bundle id")
              next
            end
            unless mapping.key?(bundle_id)
              UI.verbose("Skipping #{config.name} for #{target.name}, bundle id #{bundle_id} not included in match mapping")
              next
            end
            team_id = params[:team_id] || get_team_id(params, bundle_id)
            unless team_id
              UI.user_error!("No value provided for :team_id, and we were unable to automatically detect a value based on the provisioning profile.")
              return nil
            end
            code_sign_identity = params[:code_sign_identity] || self.get_code_sign_identity(params, bundle_id)
            unless code_sign_identity
              UI.user_error!("No value provided for :code_sign_identity, and we were unable to automatically detect a value based on the provisioning profile.")
              return nil
            end

            set_build_setting(config, CODE_SIGN_STYLE_KEY, CODE_SIGN_STYLE_VALUE)
            UI.verbose("Set Code Sign Style to: Manual for target: #{target.name} for build configuration: #{config.name}")

            set_build_setting(config, CODE_SIGN_IDENTITY_KEY, code_sign_identity)
            UI.verbose("Set Code Sign Identity to: #{code_sign_identity} for target: #{target.name} for build configuration: #{config.name}")

            set_build_setting(config, TEAM_ID_KEY, team_id)
            UI.verbose("Set Team ID to: #{params[:team_id]} for target: #{target.name} for build configuration: #{config.name}")

            set_build_setting(config, PROFILE_NAME_KEY, mapping[bundle_id])
            UI.verbose("Set Provisioning Profile Name to: #{mapping[bundle_id]} for target: #{target.name} for build configuration: #{config.name}")

            UI.important("Updated code signing values for target: #{target.name} for build configuration: #{config.name}")
          end
        end
        project.save
      end
      # rubocop:enable Metrics/PerceivedComplexity

      def self.set_build_setting(configuration, name, value)
        # Iterate over any keys that start with this name
        # This will also set keys that have filtering like [sdk=iphoneos*]
        keys = configuration.build_settings.keys.select { |key| key.to_s.match(/#{name}.*/) }
        keys.each do |key|
          configuration.build_settings[key] = value
        end

        # Explicitly set the key with value if keys don't exist
        configuration.build_settings[name] = value
      end

      def self.get_bundle_id(build_configuration)
        infoplist_path = build_configuration.build_settings[INFOPLIST_FILE_KEY]
        unless infoplist_path && File.exist?(infoplist_path)
          return nil
        end

        plist_data = Xcodeproj::Plist.read_from_path(infoplist_path)
        return plist_data[BUNDLE_ID_KEY]
      end

      def self.get_team_id(params, bundle_id)
        require 'match'
        env_variable_name = Match::Utils.environment_variable_name_profile_name(app_identifier: bundle_id,
                                                                                type: Match.profile_type_sym(params[:type]),
                                                                                platform: params[:platform])
        return ENV[env_variable_name]
      end

      def self.get_code_sign_identity(params, bundle_id)
        require 'match'
        require 'openssl'
        env_variable_name = Match::Utils.environment_variable_name_profile_path(app_identifier: bundle_id,
                                                                                type: Match.profile_type_sym(params[:type]),
                                                                                platform: params[:platform])
        path = ENV[env_variable_name]
        unless path && File.exist?(path)
          return nil
        end

        profile = FastlaneCore::ProvisioningProfile.parse(path)
        cert = OpenSSL::X509::Certificate.new(profile[CERTIFICATE_DATA_KEY].first.string)
        cert_info = cert.subject.to_s.gsub(/\s*subject=\s*/, "").tr("/", "\n")
        values = cert_info.split("\n")
                          .map { |x| x.split(/=+/) if x.include?("=") }
                          .compact
                          .to_h
        return values[CERTIFICATE_NAME_KEY]
      end
    end
  end
end

getting eyes on https://github.com/fastlane/fastlane/pull/20187 as an intermediary (so I can update get_code_sign_identity above) would also be great

Hi @tejassharma96, I really like the idea.
@joshdholtz, could you take a look at the discussion and the code in the comment above and say what you think about such an actiont?

pls no close fastlane bot. @joshdholtz have you had a chance to take a look at this?

Ah sorry! This got lost in my notifications ☹
I love this idea and I think it would be a great action to have! I did something similar in a few of my projects in the past. I'm totally happy to accept a PR for this 🙌

awesome! glad to hear it :) I'll see if I can get something up this weekend!

sshhhhhh bot, we have a pull request


Improve type support for model related methods

Description
Enhanced type support by adding docblock annotations with class-string<Model> and, in some cases, adding the native type Model & Authenticatable to ensure the object extends the Model class and implements the Authenticatable interface.
Addressed an edge case for SpatieTagInput by checking if the getTagClassName method exists before calling it. This was necessary because there isn't a docblock annotation to describe an instance of a class using a specific trait.
Visual changes

Functional changes

[x] Code style has been fixed by running the composer cs command.
[x] Changes have been tested to not break existing functionality.
[ ] Documentation is up-to-date.

Thanks


patchelfUnstable: unstable-2023-09-19 -> unstable-2023-09-27

Description of changes
Pick up https://github.com/NixOS/patchelf/pull/521 for Firefox.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 257971 run on x86_64-linux 1

1 package built:

patchelfUnstable


Document the app versioning

The current app versioning needs to be documented.

Proposal for documentation in https://docs.google.com/document/d/13sFmADf-KqJrH76OIaYDONzS9jhqaTQnroxsYH3nd_s/edit

@wmontwe @cketti above is a first draft, which might be missing a few things. I would appreciate if you could take a look.


Community: link to SubmittingPatches and MyFirstContribution on the website

Two small changes to the Community page :

Since we now build and host "SubmittingPatches", link to it instead of the raw document on GitHub
Since we now build and host "MyFirstContribution", make it easier to discover by mentioning it there also.

Another awesome contribution, thanks for noticing and fixing it @phil-blain . Will merge tomorrow to give some time for other reviewers


Speech to text recognition

For speech recognition you can use google cloud speech to text API (60 minutes of speech is free for each month) https://github.com/googleapis/nodejs-speech
Or you can try to implement SpeechRecognition() in page evaluation https://youtu.be/0mJC0A72Fnw

The biggest problem right now is to obtain the audio file from the recaptcha v2 in the first place.
By merely navigating via puppeteer to the audio link like I do, recaptcha tends to block me.
I think you need to automate the browser by clicking, dragging and navigating with the mouse like in https://github.com/ecthros/uncaptcha2 .
But it's strange. I mean audio captchas are mainly there for visually impaired people. Maybe recaptcha v2 looks for behavior linked to visually impairment and rejects if it does not detect it.
Any idea?


Prow Internal Server Error -- Cannot view test results

Example: https://k8s-gubernator.appspot.com/build/istio-prow/pr-logs/pull/istio_istio/13849/istio-integ-k8s-tests/7860
Traceback (most recent call last):
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7679791231c143f9/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7679791231c143f9/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7679791231c143f9/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7679791231c143f9/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/apps/s~k8s-gubernator/v20190412-f1ffa5e9b.417445111776570698/view_base.py", line 85, in dispatch
    webapp2.RequestHandler.dispatch(self)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7679791231c143f9/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7679791231c143f9/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~k8s-gubernator/v20190412-f1ffa5e9b.417445111776570698/view_build.py", line 233, in get
    testgrid_query = testgrid.path_to_query(job_dir)
  File "/base/data/home/apps/s~k8s-gubernator/v20190412-f1ffa5e9b.417445111776570698/testgrid.py", line 116, in path_to_query
    tabs = dashboard['dashboard_tab']
KeyError: 'dashboard_tab'

Seems to apply to every PR

The error is in testgrid's config. We are sharing/using k8s' testgrid and k8s engineers have been informed and fix is on the way.
The report page should come back like before once testgrid's config is fixed.

The result are back. Closing the issue.


Add maintainer list.

Hi guys~
These developers have contributed lots of code and proposals to this project. I would like to invite them as project maintainers:

invite @estebanreyl as a reviewer.
invite @yuchen0cc as a committer

Needs explicit LGTM from @estebanreyl @yuchen0cc and 1/3 of the 'acelerated-container-image' Committers.
according to https://github.com/containerd/project/blob/main/GOVERNANCE.md :

After a candidate has been informally proposed in the maintainers forum, the
existing maintainers are given seven days to discuss the candidate, raise
objections and show their support. Formal voting takes place on a pull request
that adds the contributor to the MAINTAINERS file. Candidates must be approved
by 2/3 of the current committers by adding their approval or LGTM to the pull
request. The reviewer role has the same process but only requires 1/3 of current
committers.
If a candidate is approved, they will be invited to add their own LGTM or
approval to the pull request to acknowledge their agreement. A committer will
verify the numbers of votes that have been received and the allotted seven days
have passed, then merge the pull request and invite the contributor to the
organization.
For non-core sub-projects, only committers of the repository that the candidate
is proposed for are given votes.

[x] @BigVan
[ ] @fuweid
[x] @liulanzheng

I'd also like to get a few LGTMs from the Core Committers too. (not necessary)

yuchen.cc is the main developer of userspace image convertor.
northtyphoon and estebanreyl provided many suggestions for release specifications and practices in Azure.

@BigVan please invite the candidates to check this~ we need them LGTM as well.

LGTM

Invites sent


Docker Hub pull rate limits

N.B. This issue description will be updated as new information becomes available.
As of 2020-08-13, Docker have updated their terms of service and pricing page, indicating that:

unauthenticated pulls will be rate limited to 100 per 6h
authenticated pulls will be rate limited to 200 per 6h

The page on rate limits clarifies that this applies ~to layers being pulled, not~ images. ~Since most images comprise multiple layers, the effective image pull rate limit will be very low.~
The rate limit page is inconsistent about the number, but also states that these rate limits are being introduced gradually.
Based on a recent build it does not appear that these rate limits are being enforced yet. However, it seems as though they will be in effect by 2020-11-01, so we need to understand the implications for various groups.
Implications

For environments where images are persisted between builds, the rate limit may not be hit. For ephemeral CI environments where images are discarded between builds, it seems more likely that the rate limit will be hit.

In general, all user groups may need to either:

use an authenticated Docker Hub account for builds. This page describes how this can be set up.

testcontainers-java has good support for authenticated image pulls. Other language forks will need work done.
Docker Hub's Free/Pro/Team plans will allow 300/unlimited/unlimited layer pulls per 6h.

OR, users may wish to set up their own registry to mirror/copy images which they require. This is already possible, and work is underway within Testcontainers to continue to make this easier.

user groups with low image pulling requirements (e.g. few images or a low rate of builds) will presumably not be impacted.

For open source projects using Testcontainers within their builds (including Testcontainers itself)

per https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/, an open source project plan will be available with free unlimited pulls, and Docker have an application form here: forms.gle/vvKURDTYwok7Pc4r5
For projects using GitHub Actions: GitHub Actions will have mitigation in place  to prevent rate limits from affecting builds.
For projects not using GitHub Actions: For projects that set up authenticated pulls in their CI infrastructure, it is not clear what approach should be followed for repository forks. Using build secrets to hold Docker Hub auth tokens means it will not be available to forks.

For companies using Testcontainers within their builds

If Hub's rate limiting is based on IP address, and if developers share an IP address, it seems possible that rate limits could apply to developers as well as CI infrastructure.
For projects that set up authenticated pulls in their CI infrastructure, it is also not clear what approach should be followed for repository forks. Using build secrets to hold Docker Hub auth tokens means it will not be available to forks, although in a company environment this may be more manageable.
Companies may be more likely to have private docker registries (e.g. ECR, GCR) which they can copy required images to. Testcontainers will implement an image substitution mechanism to allow private registry image names to be used as substitutes for public Docker Hub images (see below).

Actions for users

Be mindful that rate limits may apply soon, and some preparation will be needed. Consider the above options and implement.
Please help clarify open questions if you have additional information.

Actions for Testcontainers team

[ ] Set up authenticated pulls for Testcontainers' own builds (N.B. this refers to adding docker login to our CI scripts. Authenticated pull support is already implemented in testcontainers-java)

[x] GitHub Actions - skip, not needed
[ ] CircleCI, Azure Pipelines for Linux, Travis - switch to master branch only and use secrets
[ ] Azure Pipelines for Windows (triggered by maintainer PR comments) - use secrets

[x] Implement transparent 'image substitution' (whereby Testcontainers will consult a user-provided piece of code to obtain an alias for an image, which may reside on a private registry). PR: #3102

[ ] Investigate the possibility of providing an implementation of image substitution that treats GitHub Packages as a cache.

[ ] Retrofit to language forks: authenticated image pulls and image substitution

[x] Improve documentation around authenticated image pulls and options for mirroring registries / copying images to non-Hub registries

Open questions

[x] When will rate limits noticeably kick in? Unclear: nominally 1st November, but some Docker/Testcontainers users have reported rate limiting already

Docker have said that IPs pulling the highest volume will be experiencing the rate limit earlier. All other signs point to 1st November being the start.

[x] Are CI platforms going to attempt to mitigate this?

GitHub Actions: YES
Others, no information yet

[x] Is there a separate Free plan for Open Source projects that allows unlimited pulls? Yes, see https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/

[ ] How should repo forks manage the need to provide Hub credentials?

Aside from GitHub Actions providing a different approach to mitigation, it appears there is no solution to this problem for other CI platforms.
teams using their own CI executors may find other mitigations, such as configuring a local docker registry server to act as an authenticated mirror.

One idea: for fork builds on GitHub Actions, we could look at using GitHub Packages as a cache for required images. Images pushed to GitHub Packages should be readable on forks, so the main (automatable) work would be:

to ensure that an OSS project's docker dependencies are published to GitHub Packages
to ensure that image substitution is performed to rewrite references to the original image name with a corresponding GitHub Package URI.

Both could be accomplished using the pluggable image substitution mechanism that we have in mind.
I'll add an action above for us to investigate.

AFAIK, GitHub registry requires API token even for image pulls from public repositories, so I'm not sure forks would be readable on forks.

@vlsi you might be right, so I'd like to do some investigation before putting a lot of effort in. My understanding of the docs was that forks can access a GITHUB_API_TOKEN secret which is a different, read-only, token. It's quite possible that I'm wrong.

@vlsi, @rnorth is correct the github token provided by github action has read only access on forked repos.
See https://docs.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token#permissions-for-the-github_token

just fyi, this started happening for us today in aws codebuild in our pipeline, tests fail because limit exceeded on docker hub, so some sort of rates are being enforced already (not waiting until november)

@fongie that's alarming and disappointing - my understanding was that this was not being applied yet.
Based on a test branch (#3098) I get the impression that GitHub Actions and CircleCI might not be getting rate limited yet.
The advice would remain the same - to use an authenticated account for pulling or to use a mirror/copy in another registry. I'm saddened that we've not had time to release our image substitution yet, though.

Updated above with link to https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/ which clarifies:

that pulls will be based on image pulls, not layers.
an open source project plan will be available and Docker have an application form here: https://forms.gle/vvKURDTYwok7Pc4r5

From my perspective, two major topics remain unresolved:

for open source projects, how can Docker Hub auth credentials be shared with forks of repos for PR builds? Will CI platform vendors step in with a solution?
we want to ship transparent image substitution in Testcontainers. If we have until November I'd be confident in this; if rate limiting is being applied much sooner then we do not have time.

@fongie I've reached out to a contact at Docker to see if we can get clarification on when the rate limiting is supposed to be taking effect.

Great! Sorry for not being more specific last post, I was busy trying to get our release out hehe
This is the error message we got:
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not pull image: error pulling image configuration: toomanyrequests: Too Many Requests. Please see https://docs.docker.com/docker-hub/download-rate-limit/
in AWS CodeBuild, trying to pull 'postgres:10.7'
It worked on the third attempt simply by retrying. I understand that there is the option to set up an authenticated account, I just dont have the time for that right now, but I would guess this is happening to more people and you might get more questions about this issue, so it might be nice to prepare thorough instructions for how to do it. If I log in to another docker registry in my codebuild environment, will testcontainers automatically use that to pull from when I do new PostgreSQLContainer("postgres:10.7"), or do I need to instruct testcontainers on which registry to use?

@fongie sorry for the slow response. If you log in to Docker Hub then Testcontainers will be able to use that hub account for pulling.
If you log in to another docker registry (e.g. ECR) then that does not help directly, but it could be made to work. ECR does not function as a pull-through cache (AFAIK), so you'd have to copy the required image into ECR and use new PostgreSQLContainer("your.ecr.registry.amazonaws.com/somepath/postgres:10.7") to create the container. This is a bit of a pain, which is why I'm working on #3102 to at least help with the second aspect.
Logging in to Docker Hub requires fewer changes.

For GitHub Actions users, this issue looks worth following: https://github.com/actions/virtual-environments/issues/1445

Really good news for GitHub Actions users: https://github.com/actions/virtual-environments/issues/1445#issuecomment-713861495

Would it not make sense to configure a registry mirror the same way native docker does?
We have Nexus setup to mirror the official DockerHub registry, and we have configured our Jenkins slaves to point towards this mirror..
# docker info
....
 Registry Mirrors:
  https://<internal_company_mirror_here>:5000/
...

Although it's nice that there will be a piece of callback code that we can hook into to modify the image name before it's fetched, would it not be easier to allow a registry mirror to be configured in ~/.testcontainers.properties that would act as the default host to fetch images from that do not explicitly specify a host in the image name?
This seems like what 99% of people would want, and it would stop everyone implementing the same code to do it.

@aaronjwhiteside registry mirror is the easiest way to "fix" the rate limits issue, yes. There is no need to implement any code for it and it can be set in Docker's settings already today.
Also, there is #3413 that implements a default, prefixing substitutor, for those who don't have access to Docker's settings.

@bsideup Interesting, we have a registry mirror set but I still see rate limit errors while fetching images, I'll have to dig into our build system and try and figure out what is going on.
#3413 looks promising!

I'm seeing the same behavior as @aaronjwhiteside on our internal network with registry mirrors setup.

@brianwyka see https://www.testcontainers.org/supported_docker_environment/image_registry_rate_limiting/

@bsideup, That circularly brought me back here 😆 .  We are on testcontainers 1.14.x.  Maybe we need to update to 1.15.0 ??

@brianwyka oh, yes, definitely. 1.14.x is from pre-ratepocalypse era :D

@brianwyka @bsideup
I think what is happening is that when docker gets an error while pulling from the configured registry mirror is falls back to going directly to dockerhub, I checked our nexus logs and found it was receiving the rate limit error too..
I'm not sure if this is documented behaviour, I haven't checked, though a little unexpected it kinda makes sense.

we solved the issue by forcing docker-java (which is used under the hood) library to point to the right registry
src/test/resources/docker-java.properties
registry.url=your.artifactory.domain/some-path/

That's really interesting. I wonder if we could/should obtain the daemon's registry URL from the info endpoint and use that automatically. That may not be universally applicable though...
At the very least we should document this.

@poznas, were using that docker-java.properties configuration in addition to testcontainers.properties.  I ran into the rate limit problem with just the docker-java.properties configuration present...

@brianwyka, for one of our repos this solution also did not work.
Tired of trying to track down the cause, I just replaced testcontainers away 😁
I replaced it with a slightly more manual approach: gradle-docker-plugin

@poznas IMHO moving away from testcontainers to gradle docker plugin & maven docker plugin is 2 steps back. It decreases significantly development experience.

Are there any plans to publish test containers to the ECR public gallery: https://aws.amazon.com/about-aws/whats-new/2020/12/announcing-amazon-ecr-public-and-amazon-ecr-public-gallery/
This could be a good solution for many who have their CI on AWS environments as it provides free data transfer if you're on AWS.

@Fraserhardy testcontainers/* is exempt from rate limits on Docker Hub already.

@bsideup now that testcontainer images are whitelisted in Docker Hub, can this issue be considered as closed? do you still recommend to update to 1.15.1 and use the "image name prefix" solution to point to an internal registry? Thanks

@bsideup now that testcontainer images are whitelisted in Docker Hub, can this issue be considered as closed? do you still recommend to update to 1.15.1 and use the "image name prefix" solution to point to an internal registry? Thanks

@bsideup We are getting rate limit issues in aws codeBuild. If testcontainers/* are exempt from rate-limit, I don’t understand why we are getting this error. Below are the logs.
we are using testcontainers 1.15.1
2021-01-19T09:57:18,596 546 [main] WARN o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
2021-01-19T09:57:18,634 584 [main] INFO o.t.d.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/root/.goenv/shims, /root/.goenv/bin, /go/bin, /root/.phpenv/shims, /root/.phpenv/bin, /root/.pyenv/shims, /root/.pyenv/bin, /root/.rbenv/shims, /usr/local/rbenv/bin, /usr/local/rbenv/shims, /root/.dotnet/, /root/.dotnet/tools/, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin, /opt/tools, /usr/local/android-sdk-linux/tools, /usr/local/android-sdk-linux/tools/bin, /usr/local/android-sdk-linux/platform-tools, /codebuild/user/bin])
2021-01-19T09:57:19,445 1395 [main] INFO o.t.d.DockerClientProviderStrategy - Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
2021-01-19T09:57:19,492 1442 [main] INFO o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
2021-01-19T09:57:20,429 2379 [docker-java-stream-1239536715] ERROR c.g.d.a.a.ResultCallbackTemplate - Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"}
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
at java.base/java.lang.Thread.run(Thread.java:834)

should we copy the testcontainer images to our internal registry and use the 'image name prefix' solution?

@bsideup We are getting rate limit issues in aws codeBuild. If testcontainers/* are exempt from rate-limit, I don’t understand why we are getting this error. Below are the logs.
we are using testcontainers 1.15.1
2021-01-19T09:57:18,596 546 [main] WARN o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
2021-01-19T09:57:18,634 584 [main] INFO o.t.d.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/root/.goenv/shims, /root/.goenv/bin, /go/bin, /root/.phpenv/shims, /root/.phpenv/bin, /root/.pyenv/shims, /root/.pyenv/bin, /root/.rbenv/shims, /usr/local/rbenv/bin, /usr/local/rbenv/shims, /root/.dotnet/, /root/.dotnet/tools/, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin, /opt/tools, /usr/local/android-sdk-linux/tools, /usr/local/android-sdk-linux/tools/bin, /usr/local/android-sdk-linux/platform-tools, /codebuild/user/bin])
2021-01-19T09:57:19,445 1395 [main] INFO o.t.d.DockerClientProviderStrategy - Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
2021-01-19T09:57:19,492 1442 [main] INFO o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
2021-01-19T09:57:20,429 2379 [docker-java-stream-1239536715] ERROR c.g.d.a.a.ResultCallbackTemplate - Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"}
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
at java.base/java.lang.Thread.run(Thread.java:834)

should we copy the testcontainer images to our internal registry and use the 'image name prefix' solution?

@Fraserhardy testcontainers/* is exempt from rate limits on Docker Hub already.

@bsideup That's good news, but many internally used images are not from that repo, like alpine or localstack.
I think a single property in testcontainers.properties would be a handy way to set a registry for all the internal images listed in org.testcontainers.utility.TestcontainersConfiguration.
Because now I have to do this one-by-one in testcontainers.properties, like:
ambassador.container.image=my-registry.com/richnorth/ambassador
socat.container.image=my-registry.com/alpine/socat
vncrecorder.container.image=my-registry.com/testcontainers/vnc-recorder
...etc...

which is cumbersome and fragile for changes.
Something similar to the docker-java solution, but I don't want to directly depend on testcontainers' third-parties either.

we solved the issue by forcing docker-java (which is used under the hood) library to point to the right registry
src/test/resources/docker-java.properties
registry.url=your.artifactory.domain/some-path/

@Fraserhardy testcontainers/* is exempt from rate limits on Docker Hub already.

@bsideup That's good news, but many internally used images are not from that repo, like alpine or localstack.
I think a single property in testcontainers.properties would be a handy way to set a registry for all the internal images listed in org.testcontainers.utility.TestcontainersConfiguration.
Because now I have to do this one-by-one in testcontainers.properties, like:
ambassador.container.image=my-registry.com/richnorth/ambassador
socat.container.image=my-registry.com/alpine/socat
vncrecorder.container.image=my-registry.com/testcontainers/vnc-recorder
...etc...

which is cumbersome and fragile for changes.
Something similar to the docker-java solution, but I don't want to directly depend on testcontainers' third-parties either.

we solved the issue by forcing docker-java (which is used under the hood) library to point to the right registry
src/test/resources/docker-java.properties
registry.url=your.artifactory.domain/some-path/

@adambriny see https://www.testcontainers.org/features/image_name_substitution/#automatically-modifying-docker-hub-image-names

@adambriny see https://www.testcontainers.org/features/image_name_substitution/#automatically-modifying-docker-hub-image-names

Oh, thank you @bsideup!
It's my fault, I was looking for the rate limit keywords only.. 🤦♂
Maybe it would be easier to find if it was mentioned in https://www.testcontainers.org/supported_docker_environment/image_registry_rate_limiting/ as a possible solution...

Oh, thank you @bsideup!
It's my fault, I was looking for the rate limit keywords only.. 🤦♂
Maybe it would be easier to find if it was mentioned in https://www.testcontainers.org/supported_docker_environment/image_registry_rate_limiting/ as a possible solution...

@adambriny good idea!
P.S. contributions are more than welcome 😊

@adambriny good idea!
P.S. contributions are more than welcome 😊

How about KISS registryMirrors soution?
https://github.com/GoogleContainerTools/jib/tree/master/jib-gradle-plugin#properties

I just tried to use ECR as the testcontainers prefix (TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX=public.ecr.aws/), but unfortunately the images aren't publically available there. I found some random public repositories in ECR such as TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX=public.ecr.aws/bigeye/ which host a mirror of the testcontainer images, but since those aren't official, I can't use them at work... is there any chance in the future testcontainers could setup a public repo in ECR as well?

Hi @perrin4869
The testcontainers org is in Docker's open source program, so is supposed to be exempt from rate limits... Are you seeing rate limiting occurring?

(Just to clarify, this means that images like testcontainers/ryuk should be exempt. Other images may still have rate limits)

@rnorth As far as I can see, the ryuk image is not part of the open source program. The open source program label is missing on the docker hub page:
Comparing:
https://hub.docker.com/r/testcontainers/ryuk is missing that label
https://hub.docker.com/r/fluent/fluent-bit has that label


Displays out-of-screen areas

@GanZhiXiong you can try this package i made https://pub.dev/packages/custom_tooltip on pub dev


Make scrollIntoView work for Safari and SVGElement

The polyfill for scrollIntoView doesn't work if you want to use it on an SVGElement (g, circle polygon, etc.) on Safari.
On Safari SVGElement does not inherit from HtmlElement || Element.
To make it work,
after:
import smoothscroll from 'smoothscroll-polyfill' smoothscroll.polyfill()
add:
window.SVGElement.prototype.scrollIntoView = (window.HTMLElement || window.Element).prototype.scrollIntoView

If you don't use {behavior: 'smooth'} in your arguments it won't work either.

I'd recommend using https://github.com/magic-akari/seamless-scroll-polyfill 👍 instead of this plugin.

I'd recommend using https://github.com/magic-akari/seamless-scroll-polyfill 👍 instead of this plugin.

Thank you fixed my issue! 👍


v4.3.8 release

See https://github.com/codeigniter4/CodeIgniter4/blob/develop/admin/RELEASE.md

[x] #7848
[x] #7849

It seems there is no epub userguide v4.3.8:
https://github.com/codeigniter4/userguide
There is no version in the commit message:
https://github.com/codeigniter4/userguide/commit/368723d53171b7e9e0e8e377eb1eea1eff8632f6
and CodeIgniter.epub was created.
Manually renamed for now.
https://github.com/codeigniter4/userguide/commit/9852f1c35d8371cffd64baffd873ddd2e4a607fb

Run ./source/.github/scripts/deploy-userguide ${GITHUB_WORKSPACE}/source ${GITHUB_WORKSPACE}/userguide ${GITHUB_REF##*/}
  ./source/.github/scripts/deploy-userguide ${GITHUB_WORKSPACE}/source ${GITHUB_WORKSPACE}/userguide ${GITHUB_REF##*/}
  shell: /usr/bin/bash -e {0}
Preparing for version
Merging files from /home/runner/work/CodeIgniter4/CodeIgniter4/source to /home/runner/work/CodeIgniter4/CodeIgniter4/userguide
error: pathspec 'master' did not match any file(s) known to git
Running Sphinx v4.3.2

https://github.com/codeigniter4/CodeIgniter4/actions/runs/5970889910/job/16199163159
The version number is missing.
Output:
Preparing for version
Merging files from /home/runner/work/CodeIgniter4/CodeIgniter4/source to /home/runner/work/CodeIgniter4/CodeIgniter4/userguide

Shell script:
SOURCE=$1
TARGET=$2
RELEASE=$3
VERSION=`echo "$RELEASE" | cut -c 2-`

echo "Preparing for version $3"
echo "Merging files from $1 to $2"

Does anyone know why the script couldn't get the version number?

${GITHUB_REF##*/} does not work anymore?

No, when v4.4.0 was released, it worked.
https://github.com/codeigniter4/userguide

But it was manually renamed, right?

No, I manually renamed v4.3.8 epub. v4.4.0 epub is created automatically.


Feature for certificate authentification

HI,
for etcds connection does vip-manager support cert-authentification  ?
The aim is to avoid to store user's password in the configuration file, because of security issues.
Thank you

If you had taken the time to search for the word "cert" on the main page of the repo or using the search function, you could have answered this question yourself. 🙂
It is already supported, you'll just have to pass the relevant environment variables, flags, or parameters in the yaml config file:
https://github.com/cybertec-postgresql/vip-manager#Configuration


Unable to provide env variables to container

Trying to pass environment to a container:
resources:
  containers:
    - container: mysql
      image: mysql:5.7
      ports:
        - 13306:3306
      env:
        - MYSQL_USER: test_user
        - MYSQL_PASSWORD: test_password
        - MYSQL_DATABASE: test_database

But I'm not able to find the right way to get this done. Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7d1c794c-d676-fc7e-758e-3464f16a1554
Version Independent ID: 941de04a-2d34-ec82-8612-00e4070de673
Content: Service Containers - Azure Pipelines & TFS
Content Source: docs/pipelines/process/service-containers.md
Product: devops
GitHub Login: @steved0x
Microsoft Alias: dakale

Hi @jrjohnson
You can specify the container environment with:
- container: mycontainer
    image: myimage
    env:
      FOO: bar

and reference the container as a service. Or, if you define your service inline:
services:
    nginx:
      image: nginx
      ports:
      - <IP_ADDRESS>:8080:80/tcp
      env:
        SOME_ENV: value


Update the Quantum Neural Networks tutorial with QNNCircuit

What should we add?
With the recent introduction of QNNCircuit it is important to show how to use it in the QML tutorials. The most suitable opportunity is to update the 01_neural_networks.ipynb and add a simplified way of instantiating quantum neural networks. Other tutorials can be updated as well.

@adekusar-drl great idea 👍 !
I'd love to pick up this one 😃 .

Hey @adekusar-drl ! Would it be possible for me also to be assigned to this issue? I'd love to help improve the QML tutorials.

@david-alber are there any updates?

@adekusar-drl  I am still working on this issue. We can split the work though: I will update the 01_neuronal_networks.ipynb tutorial. @athanzxyt  if you find any other tutorials where the QNNCircuit can be applied, please go ahead and do the updates 😀.

@athanzxyt You can think where QNNCircuit can be applied else. Thanks.

@adekusar-drl Can you reference the API documentation for QNNCircuit? I could not find anything related to QNNCircuit.

The documentation is not available since it is not released yet.

Ahh, alright!


Remove deprecated --initialize-at-build-time= option from native image

This task is automatically imported from the old Task Issue Board and it was originally created by Pavel Marek.
Original issue is here.

native-image reports that --initialize-at-build-time= (with no arguments) is deprecated, and should be only used in conjunction with --diagnostics-mode. Providing empty arguments to this option means that all the classes that are not in --initialize-at-run-time should be initialized during build time.
Proper implementation should provide a list of packages/classes.
The involved sources are in build.sbt and project/NativeImage.scala.

Linked to #4714


Implement support for serial interfaces

Urgently needed feature to use this node in wider range of applications

Serial Communication feature implemented an working

Serial Communication feature implemented an working


Wait For Navigation fails: 'str' object has no attribute 'name'

Describe the bug
Promise to wait for navigation fails due to library error:
'str' object has no attribute 'name'

To Reproduce
Steps to reproduce the behavior:
Click "${locator}" And Wait For Navigation To "${target}" Page Until "${event}"
    ${target_url} =    Resolve Full Url    ${target}
    ${page_navigation} =    Promise To    Wait For Navigation    url=${target_url}    wait_until=${event}
    Click    ${selector}
    Wait For    ${page_navigation}

wait_until has value load during test.
Expected behavior
Promise to wait for navigation should be successful.
Traceback
Traceback (most recent call last):
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/playwright.py", line 149, in grpc_channel
    yield playwright_pb2_grpc.PlaywrightStub(self._channel)
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/keywords/network.py", line 245, in wait_for_navigation
    waitUntil=wait_until.name,
AttributeError: 'str' object has no attribute 'name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/browser.py", line 957, in run_keyword
    raise e
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/browser.py", line 947, in run_keyword
    return DynamicCore.run_keyword(self, name, args, kwargs)
  File "/home/robot/testenv/lib/python3.10/site-packages/robotlibcore.py", line 94, in run_keyword
    return self.keywords[name](*args, **(kwargs or {}))
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/keywords/promises.py", line 166, in wait_for
    return promises[0].result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/keywords/network.py", line 238, in wait_for_navigation
    with self.playwright.grpc_channel() as stub:
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/robot/testenv/lib/python3.10/site-packages/Browser/playwright.py", line 156, in grpc_channel
    raise AssertionError(str(error))
AssertionError: 'str' object has no attribute 'name'

Additional context

Tested working until robotframework-browser v12.1.0
Tested breaking since robotframework-browser v12.2.0

@aaltat Please have a look at PR 2100

@ealap PR 2100 fiexes the issue

Done in https://github.com/MarketSquare/robotframework-browser/commit/c0929586f984e136bd4acdd57c34318301a65cc8


[PSM Interop] Add unittests CI with github actions

Add Github Action to conditionally run PSM Interop unit tests:

Only run when changes are detected in tools/run_tests/xds_k8s_test_driver or any of the proto files used by the driver
Only run against PRs and pushes to master, v1.*.* branches
Runs using python3.9 and python3.10
Ready to be added to the list of required GitHub checks

Add tools/run_tests/xds_k8s_test_driver/tests/unit/__main__.py test loader that recursively discovers all unit tests in tools/run_tests/xds_k8s_test_driver/tests/unit
Add basic coverage for XdsTestClient and XdsTestServer to verify the test loader picks up all folders

FYI @ejona86 @ctiller

Test job run example already available in the PR checks: https://github.com/grpc/grpc/actions/runs/5933744203/job/16089599399?pr=34125


Fix potential nullref in embedBuilder value setter

Collapse into a string extension method instead of repeating the same logic


Initial components

You are probably trying to make a HAT for the Pi, right?

We do not want to create a conformant HAT per se, but just a board that is plug-able onto the Raspberry Pi. But it wont hurt to follow the guidelines and I wasn't aware of them until now, so thanks :+1: .

If we focus on HATs, then we can also create separate packages for the HAT and uHAT specifications and include the mechanical dimensions on the documentation layer of the packages.

I was thinking about that as well, but I'd expect a RaspberryPi library to actually provide both: Just the connector, for a minimal adapter board and one with all the mechanical information (mounting holes, cut-outs, copper mask, ...)

Ok, then I'd call the connector-only package "Raspberry Pi GPIO 40-Pin", since it's not tied to any specific model.
The devices could be called "Raspberry Pi 2", "Raspberry Pi 3" or "Raspberry Pi HAT".

If you want to add a logo: https://tmp.dbrgn.ch/raspi.png

@dbrgn Could you take a look at the updated version?

@dbrgn @rnestler @ubruhin
I added all the changes. Could you check it again?

@mwit-a I hope you're not totally stressed out about the change requests yet :joy: But for the official libraries it's important that we get them right from the start, and that all elements of the standard library have consistent look & feel.
Thanks for your patience :slightly_smiling_face:

@mwit-a I would remove the "SPI_" part in the symbol pin name for the CE0 and CE1 pins, since we don't include that string for the other SPI pins either. Also, the component needs to be updated too with the same name change :slightly_smiling_face:

@mwit-a I would remove the "SPI_" part in the symbol pin name for the CE0 and CE1 pins, since we don't include that string for the other SPI pins either. Also, the component needs to be updated too with the same name change slightly_smiling_face

Yeah we had some trouble locally exactly because of that and didn't realize that we need to adjust the names in the component as well :see_no_evil: We then pushed our work in progress and checked it out again in a fresh workspace to debug the problem :wink:

@dbrgn @rnestler
Ok, the new version should include all the changes.
No problem, that's what code reviews are made for ;)


Rename cask gpgtools to gpg-suite

General troubleshooting steps

[x] I have checked the instructions for reporting bugs (or making requests) before opening the issue.

[x] None of the templates was appropriate for my issue, or I’m not sure.

[x] I understand that if I ignore these instructions, my issue may be closed without review.

Description of issue
I propose that the cask gpgtools be renamed to gpg-suite, with gpgtools as an alias. I would also like to make a similar request for gpgtools-beta and gpgtools-nightly.
"GPGTools" seems to refer to the organization, while "GPG Suite" refers to the software bundle. Also see release notes.
Website: https://gpgtools.org/#gpgsuite
GPG Suite includes:

GPGMail
GPG Keychain
GPG Services
MacGPG
GPG Suite Preferences (formerly GPGPreferences) System Preferences pane
Libmacgpg framework.

References to name change:

Commit https://github.com/GPGTools/GPGTools_Installer/commit/7a5db70a885cda14a06e958520a054cd7f62cd84
Commit https://github.com/GPGTools/GPGPreferences/commit/0b899b7ed2b45257547d122ff6ae214470efb344
Commit message https://github.com/GPGTools/GPGPreferences/commit/7004e82fc30262a0155c2a69abb4e3d6896cbf33

with gpgtools as an alias.

We don’t support aliases.
But I’m fine with renaming to gpg-suite.

https://github.com/caskroom/homebrew-cask/pull/39865
https://github.com/caskroom/homebrew-versions/pull/4627

Just to note that this caused issues for me:
$ brew cask outdated
Error: Cask 'gpgtools' is unavailable: No Cask with this name exists.
$ brew cask list
Error: Cask 'gpgtools' is unavailable: No Cask with this name exists.

The output of brew bundle dump was also missing all casks as a side-effect.
I worked around this with mv /usr/local/Caskroom/gpgtools /usr/local/Caskroom/gpg-suite, which fixed the problem.

What is the update path then? Or: how do I get from cask "gpgtools" to "gpg-suite" without losing settings or keeping stale files?

What I did was renamed "/usr/local/Caskroom/gpgtools" to "/usr/local/Caskroom/gpg-suite" and then did brew cask reinstall gpg-suite. That seemed to have worked. Would there have been a better way?


Update dependencies to resolve security scans

Dependency update to address security scans
Need to make sure bumping Ray version here is fine, should be fine if tests pass
Changed over the storage-initializer to use UBI-based images. Need to make sure tests pass with it

Testing instructions should just be does this storage-initializer work, which I think the tests will cover

Need to figure out how to make E2Es more stable.
Retesting, although I think this PR should go against release-v0.11 branch.
/test e2e-slow e2e-fast

/test e2e-slow e2e-fast

/retest

/retest

Forcing to :green_circle: CI jobs with known issues
/override ci/prow/pr-image-mirror-kserve-agent ci/prow/pr-image-mirror-kserve-controller ci/prow/pr-image-mirror-kserve-router ci/prow/pr-image-mirror-kserve-storage-initializer

Cycle PR to force GHActions to run.
/reopen

/retest

/test e2e-fast

/test e2e-fast

Since we're not using 3.11, and it looks like the test failure is independent, going to merge so we have security scans resolved


Add functions to test properties of default Lua metamethods

Lua has a number of default metamethods (Lua 5.3 Manual -> 2.4 – Metatables and Metamethods) that defines an object behaviour for example for default Lua operators. Most of these metamethods can be covered by general properties. For example if a and b have metamethod __eq then (a == b) == (b == a) and (a == b) ~= (a ~= b), for metamethod __lt - (a > b) == (b < a) and so on. I believe it would simplify testing a bit if lua-quickcheck will discover available metamethods and them with such properties.
What do you think, @luc-tielen? I'm ready to spend some time implementing this.
General methods

__index - none?
__newindex - none?
__mode - none?
__call - none?
__metatable - none?
__tostring

type(tostring(a)) == 'string'

__len

type(#a) == 'number'

__pairs - none?
__ipairs - none?
__gc - none?
__name

type(a.__name() == 'string'

__close

Mathematic Operators

__unm

-(-a) == a

__add

a + b == b + a
a + 0 == a

__sub - none?
__mul

a * 2 == a + a
a * 1 == a * 1 * 1

__div - none?
__idiv - none?
__mod

a % b == a - math.floor(a / b) * b

__pow
__concat - none?

Bitwise Operators

__band

(a & (a - 1)) == 0, if a is x^2

__bor

a | 0 == a

__bxor

a && b == b && a

__bnot

~(~a) == a

__shl

a << 0 == a

__shr

a >> 0 == a

Equivalence Comparison Operators

__eq

(a == b) == (b == a)
(a == b) ~= (a ~= b)

__lt

(a > b) == (b < a)

__le

(a >= b) == (b <= a)

Hi @ligurio , thanks for the suggestion.
I'm not so sure about the general methods, but I think the rest could be interesting to provide.
I wouldn't automatically do it though, since most of these operations might not make sense, and how would users fix it then?
But maybe we could provide some functions that take X generators, and then set up a default property that is often used?
For example: property_commutative_addition(generator1, generator2)?


Deploy public-facing packages to pypi

We can't make users pip install -e git+https://...
todo: Need to decide what our organisation's name on pypi should be - and which email address should we sign up under?
@canwaf knows all about how to name mailboxes, so he seems like the most suitable person to push this issue forwards. @JasonHowell may be required to support this too.

Are we there on getting them to the public pypi instance yet?

Cannot publish to pypi without making some modifications, which I did on the test-pypi publish. Your fork of pydantic, @robons, cannot be referenced directly via url to a GitHub repo.
HTTPError: 400 Client Error: Invalid value for requires_dist. Error: Can't have direct dependency: [name]

I have changed the existing pydantic reference in csvcubed to pydantic proper, and published 0.1.0-rc1 on test-pypi for all four packages. I am holding off until we talk about it on the proper pypi.

All published now:

https://pypi.org/project/csvcubed/
https://pypi.org/project/csvcubed-devtools/
https://pypi.org/project/csvcubed-models/
https://pypi.org/project/csvcubed-pmd/
https://pypi.org/project/csvcubed-pydantic/

With release procedure (login required): https://collaborate2.ons.gov.uk/confluence/display/PHEM/Release+Procedure


start-stop-daemon: Don't segfault if --exec was given a non-existing file name

Starting program: /sbin/start-stop-daemon --start --exec i-dont-exist
Program received signal SIGSEGV, Segmentation fault.
0x0000555555559053 in main (argc=1, argv=0x7fffffffdc20)
at start-stop-daemon.c:631
631                         *exec_file ? exec_file : exec);
X-Gentoo-Bug: 755197
X-Gentoo-Bug-URL: https://bugs.gentoo.org/755197

Can you please add a test?


Explore process calculi

Explore process calculi

Process calculi (process algebras) is a diverse family of approaches to formally model and reason about concurrent systems. The central notions in process calculi are processes (actors) and named communication channels. A process is a structure exhibiting certain behavior, whereas channels are the means of interaction between processes. Process calculi usually rely on small collections of primitives and operators for defining processes. Based on a set of axioms and algebraic laws, one can formally reason about behavior of processes and their equivalence.
The basic primitives in most process calculi are channel names, which represent means of communication, and the corresponding actions, which represent the act of interaction between processes. Actions usually come in dual polarities for each channel name (subject of interaction): input (positive) actions and output (negative) actions. Actions can be seen as offering or demanding communication. Interaction between processes is considered as communication, which typically happens atomically (i.e. synchronously) between a pair of opposite actions on the same subject channel and results in substituting or identifying (fusing) the corresponding object names on both ends.
Operators allow combining the primitives in order to define processes of complex behavior. Some common operators in various process calculi are:

Parallel composition, combining processes that can operate concurrently and may interact if they have compatible actions;
Choice (summation), combining alternative processes, so that only one of them can operate (chosen by the environment or non-deterministically);
Prefixing, combining an action and a follow-up process, so that the process can only operate after the action is resolved in interaction;
Restriction (encapsulation), limiting the visibility scope of names;
Replication / Recursion, allowing finite descriptions of indefinite behaviors by defining repeated processes.

There are different approaches to define the semantics of processes in process calculi. Traditionally, it was the transition semantics where the semantics was defined based on the labeled transition system describing the possible operation of the process. Alternatively, the semantics can be defined based on the unlabeled transition system, similar to term rewriting systems (reduction systems), yielding the reduction semantics. In some cases, the reduction semantics may be more simple than than transition semantics.
One of the central notions for equational reasoning in process calculi is the congruence relation, which relates processes that exhibit identical behavior in all contexts. The rules for describing semantics and equational reasoning are usually split into two groups: structural congruence and operational transitions/reductions. Structural congruence equates processes that will be never distinguished for any semantic reason; it usually includes such rules as alpha-conversion, associativity, commutativity, and identity. Transition/reduction rules deal with interaction between processes.
One of the fundamental methods for defining and verifying behavioral equivalence is the concept of bisimulation (bisimilarity). Intuitively, processes are bisimilar iff every behavior of one process is matched by the behavior of the other and vice versa. There are different forms of bisimilarity yielding different versions of congruence, such as strong congruence that requires every action be matched exactly in all contexts and weak (observational) congruence that requires only observable actions be matched in all contexts.
In most process calculi there is an expansion law that provides a rule for expressing the parallel composition of processes in terms of sequential choice. The expansion law is valuable in reasoning about complex process interactions, as it provides a way to systematically "expand" parallel compositions into a sum of simpler terms.
There is a number of branches of process calculi with numerous variants and extensions. In the scope is this issue were explored:

Calculus of Communicating Systems (CCS),
Communicating Sequential Processes (CSP),
Algebra of Communicating Processes (ACP),
π-calculus,
Asynchronous π-calculus,
Fusion Calculus,
D-Fusion (Distinctive Fusion) Calculus,
Fusion Calculus of Solos and  Solo Diagrams.

CCS, CSP, and ACP are the major branches of process calculi. Historically CCS was the first followed by CSP; they share a lot of similarity and mostly agree on the fundamental operations such as sequencing, nondeterministic choice, and parallel composition, though CCS uses a smaller set of operations. ACP uses more abstract communication scheme and is more focused on the algebraic aspects and axiomatization.
The central idea in pi-calculus was to enable process mobility by allowing processes create new communication channels and share them over already established communication channels, thus dynamically establishing connections between processes. This turned out to be a fundamental change that made pi-calculus a self-contained model of concurrent computation where everything can be expressed as a process accessed through named channels, similar to the actor model and lambda calculus.
Asynchronous pi-calculus is a variant of pi-calculus where output actions cannot act as prefixes for continuation processes. This essentially removes synchronization effects of communication. Despite the limitations, it was shown that this process calculus is quite expressive, e.g. it can even express the construct of choice guarded by input or output actions using parallel composition, though it is not capable of representing choice guarded by mixed input and output actions.
Fusion calculus was a generalization of pi-calculus with symmetry between input and output actions and a single unified operator for introducing new names and restricting their scope. The effect of interaction in fusion calculus is identification (fusion) of the object names rather than combined binding and substitution of object names in the input action. D-Fusion calculus is a variant of fusion calculus with another operator for introducing new names that must be unique, i.e. never fused together by interaction actions. Fusion calculus of solos is a surprisingly expressive asynchronous variant of fusion calculus where continuations are removed altogether, i.e. there is no prefix operation. There is a simple graphical representation for Solos calculus, called solo diagrams.


White screen appears after labeling a few images and the running speed is slow？

Hello, I encountered a white screen issue while labeling image defects and the running speed was slow

This seems to be a problem with label studio, I'll see if there is a solution for you.


Switch layout from tray icon

Description of the new feature / enhancement
Change the fancy zone layout from the tray icon. Right-click on Power Toys app, and use an option that permits selecting another different layout.
Scenario when this would be used?
To change the layout of fancy zones, it's necessary to open the de Power Toys application, then select the fancy zones option, open de layout editor and click over the new layout.
I have different situations working on my PC, when I develop, when browsing the internet, watching movies, etc. I would like to use the de context menu in the tray icon of Power Toys to switch between layouts.
Supporting information
No response

Sounds like issue #6751

You can also assign a hotkey to a specific layout. It's in the settings for each layout.

Yes, now I use the hotkey option, but I would like to have a UI option in the tray icon.


Parse cache hit/miss into Prometheus metric

Closes #944
Reading from cache data from summary packet, specifically rd.hits and rd.miss, I got 0 for both values even if requested multiple files through my cache-origin federation (with EnableFallbackRead set to false). Skimming through xrootd source code, it seems that the two values are only updated for rmc type cache (read? memory cache), but I only got pfc type cache in the monitoring packets.
@bbockelm Is this value the one we want to parse? Or it's something else?

Close this PR as the implementation is not what we want. Will open a separate PR once that's done.


v1.8.5 doesn't seem to work for me

This version should support the running yt version 11.04.56 on my phone, but for some reason it doesn't.
I am not sure what the prerequisites are (didn't find them here), for instance, my phone (android 5.1.1. S6 Edge Plus) is not rooted.
I tried couple times to reinstall and rebooting the device again but nothing seems to work.
As soon as I press home button, yt stops streaming
Thanks guys!

Xposed Framework is required to run the module.
Installation of Xposed may be NOT trivial for beginner users (especially on  Samsung devices which require special version) and it requires ingerention in the system, including flashing custom recovery.
It may also result in tripping Knox flag (= warranty void).
Please search on xda for details.

many thanks esgie for your reply


Add a token limit to Absinthe Parser

📖 Description
We currently do not have any limit regarding the parsing of a GraphQL query. That means that a huge query could takes minutes being parsed. To prevent this, we add a token_limit that will short-circuit the parsing if it goes over the maximum.
🦀 Dispatch

#dispatch/elixir

🦀 Requesting reviewers for this pull request:

@gcauchon (contributor with 0 commits in the last 90 days and 49 commits overall)
@lucbernard (reviewer for the elixir stack)