TS_USE_MMAP: mmap+memcopy instead pread+pwrite

apt install libaio-dev
./configure --enable-mmap --enable-experimental-linux-native-aio
./configure --enable-mmap

@cukiernik Is this supposed to be faster than pread/pwrite?  Do you have any benchmarks.

The improvement is not significant, single percentages. Speed up is due to reduce a system calls, but at the cost of increased page faults.
The reading from the file is done only on the missing page (instead of a system call each time). Memory writes to a file when the system is low on free pages. So this performance of IO thread's is very close to that of a swap file.
Further acceleration will be achieved by optimizing the agg_copy. Mmap allows aggregation into target memory, which will eliminate memcpy.

Are you currently at Intel, can you talk about the applications you all are using ATS for?

@ywkaras : Yes, I ran a tests, but I don't know what's on your mind when you write all?

To run all the Au tests, you cd to the root directory of your clone of the trafficserver repo, build it with your branch, install it, and then do:
cd tests
./autests.sh --ats-bin <install-root-directory>/bin

It will take 20 - 40 minutes to run.

@cukiernik , A few percent sounds significant already 👍 this could be very useful. You have run it without RAM cache?
I think that for a write into a memory mapped file it has to page the data into memory first (a read), so that adds some overhead. We may be able to get some more performance here by writing with pwrite instead of memcpy.
This also keeps write behaviour predictable, which is better if ATS crashes (which it never does of course).
With memory mapped files it is not guaranteed that a valid index will have been written after a crash. A msync could solve this, but the order in which the msync writes the dirty pages is not guaranteed, so a bit more orchestration would be needed for msyncing the index + data. Worst case a lot of data will be written at a very inconvenient moment.
Linux provides a consistent view between pwrites and memory mapped files, for other systems we'd have to check the behaviour.
You might want to take a look at this:
https://github.com/apache/trafficserver/blob/3012ca0f94cf7202e75422757d7498bc7e4f9295/iocore/aio/AIO.cc#L598
I think we're hitting sync I/O here if the pages are not in memory, blocking the thread instead of using the original async nature of the function. I don't know if passing the read to a number of specialized threads would yield better performance, I suspect it will all depend on the workload.
Mincore and msync can check whether pages are in memory, so that could be used to set up a fast (serve from memory) path, but the drawback would be the overhead of these functions.

Just checked whether ink_aio_read and ink_aio_write are used by anything else. In the C API we have TSAIOWrite + TSAIORead. This feature will break plugins which expect to be able to write past the end of the file with TSAIOWrite.
As the mmap stuff is experimental it I guess it is not a problem now?

Hello, after my long holidays.
@ywkaras: I didn't run the au test, because pip and PyPi seem to endanger my system security.
@keesspoelstra: I was also wondering whether to force msync after each modyfication, using MS_ASYNC, a add dedicated thread to MS_SYNC. but I couldn't decide on any of the solutions, I will propose a solution without it. It is not needet to a correcty working. As I wrote above, I check it with a gigantic amount of RAM, and after modifying the whole cache, msync MS_SYNC takes more than a few minutes.
In my task, I don't need to store content cache after shutdown. In this case, MAP_ANONYMOUS works faster. It is also suggested @SolidWallOfCode, but with MAP_SHARED it's easier to trace and debug for me.

@moonchen is going to review this

I'm still reading through all the changes.  One thing I've noticed is that there doesn't seem to be bounds check for the memory mapped I/O.  This may introduce crashes or even security issues.  Interested in hearing how others feel about this.

[approve ci autest]

@cukiernik do we still need this?  If so, I can ask about it in the weekly meeting.


Ephemeris defaults to DE440

When reading a par file with load_pulsar, there is an option to set ephem which defaults to DE440. This is problematic when the EPHEM is set in the par file already, because then the wrong ephemeris is used.
Physically one could argue that using the latest ephemeris is always best, especially when simulating data, but this default overwriting the value in the par file violates the principle of least surprise.

Yes, that's a good point. The two places I see where we actually use ephem is pint.toa.get_TOAs when reading in the pulsar and for setting up and enterprise Pulsar object for saving it. Are those smart enough that if we call them without an ephem specified they will just use what's in the par file?
Maybe the default should be None in which case we could call these without ephem specified and print out a note that we are using whatever's in the par file. But if the user specifies a specific ephem we can use that.


Make it possible to add flow settings to the configuration with a GUI

Is your feature request related to a problem? Please describe.
People can only add flow settings by hand in the XML.
Describe the solution you'd like
Add a GUI for it.
There could be a button that writes the user settings to the config.
Describe alternatives you've considered
A new modal with a form for all options.

extra settings modal voor de canvas.

:tada: This issue has been resolved in version 2.12.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Errors in currently selected datapack

Describe the Bug
Minecraft thinks there's an error in my datapacks I have selected depsite the fact I have none yet. [Expect for the default one for fabric mods] it's preventing me from creating or entering a world.
This happened when I installed the new patch of Create Fabric.
Reproduction Steps

Install Create Fabric Patch 5
Load a world or Create a new world
You'll see the error screen shortly after that.
...

Expected Result
I expected my world would have just loaded perfectly without any issues.
Screenshots and Videos

Crash Report or Log
No response
Operating System
Windows 11
Mod Version
0.4.1
Minecraft Version
1.18.2
Other Mods
No response
Additional Context
No response

provide a log for more info.
this issue usually take place when you delete some mod.

Here's a log, same issue on my end:
https://pastebin.com/GmhdcUXd

Try to remove Farmer's Delight and Debugify.

It worked when I removed Farmers Delight

I can confirm, patch 2 works with Farmer's Delight and patch 5 doesn't

For me, disabling both Farmer's Delight and Debugify still didn't fix it.


Proper way to clean up FrameProcessor in Android?

Hello there,
I'm wondering how to clean up FrameProcessor in Android.
I took a look into basic/MainActivity.java in the example directory, but it seems it closes only an external converter instance and leaves a FrameProcessor instance alive. Is this an expected way?
I tried to close by calling FrameProcessor.close() but I found it sometimes hangs regardless of calling FrameProcessor.getGraph().closeAllInputStreams().
I'm asking this question because I think I need to re-create a FrameProcessor when my input side packet gets changes. (My understanding is input side packet is a kind of constant and I cannot change it as soon as the graph starts. I searched from previous questions and there was a recommendation to use input stream instead of input side packet, but my case wouldn't work because I need to feed it to FaceGeometryPipelineCalculator which takes camera environment as a side packet.
Thank you.

I found @sureshdagooglecom tagged solution:face_geometry. Actually, I'm not using the solutions. Instead, I built my aar according to the manual and succeeded to run a graph using a face geometry. My initial question was what's a code sequence to shutdown processor cleanly. I tried a code like below, but got an error.
processor.getGraph().closeAllInputStreams();
processor.getGraph().cancelGraph();
processor.close();

Hi @dkmoon ,
Could you share latest error so that we can investigate further on this.


Update doc to reflect use of AppDeployer property keys

Fixes https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry/issues/216

LGTM, merging

Rebased and merged as<PHONE_NUMBER>2e7ac0c18b27f0488177883da99af0


Add install block

Prevents users from guessing if the name in the readme is the module name, or checking the package.json.

@homu r+

:pushpin: Commit 1146385 has been approved by poteto

:zap: Test exempted - status


Update etcd parms to limit allowable IPs

So I think including localhost in advertise client urls is not actually correct - although in our STs there won't be any side effect of it.
Also it's good to close down the available IPs clients can access the server with.
With this change, it is possible to connect using local host still.
As per etcd docs:

etcd listens on listen-client-urls to accept client traffic. etcd member advertises the URLs specified in advertise-client-urls to other members, proxies, clients. Please make sure the advertise-client-urls are reachable from intended clients. A common mistake is setting advertise-client-urls to localhost or leave it as default when you want the remote clients to reach etcd.

Will I still be able to use etcdctl ls with no additional options after this change? If so, LGTM.

Yup etcdctl still works using the default localhost


FWIW: Naming conventions and styles

Hi there,
I know that within PX4 astyle already enforces a lot of the code formatting, etc., but I hoped it might be useful if I posted a few bullet points for naming/indenting/etc that has helped my teams/projects align our styles during projects in the past.  Again, I hope this is useful to anyone coming across this post!
Coding standards:

ISO standard C++ only

C++11

Use Doxygen style comments (with @ for commands) for every:

Class
Data member
Function

@brief command for every function
@param command (if not void) for every parameter
@return command (if not void)

File

@copyright command

Allman (BSD) indentation style

Naming Convention:

Use CamelCase for everything: classes, variables, filenames.
Classes start with a capitalized letter and are nouns.
Variables start with a lowercase letter and are nouns.
Routines start with a lowercase letter and begin with verbs. To name a function, use a description of the return value. To name a procedure, use a strong verb followed by an object. See Code Complete 2 §7.6 for the difference between a function and a procedure verb.
Filenames are the name of the class or namespace within.  One class per file.
Constants are named identically to variables.
Globals are named identically to variables.
Member variables are named identically to variables.

Indentation:

Indentation shall be 4 space characters, not tabs.
Braces shall begin and end on the indentation level.
Namespaces are NOT indented.
Case statements are NOT indented.
Class visibility statements are NOT indented (public, protected, private)
One statement per line.  This includes variable declarations.
Do not put short if() ...; statements on one line.
If the constructor initializers don't fit on a single line, put constructor initializer list items one per line, starting with the comma and aligned with the colon separator.  For example:
Class::Class()
: var1(1)
, var2(2)
{
...
}

(The purpose of this indentation policy, which can feel "incorrect" at times is to ensure that changes are isolated to the minimum number of lines.  Our tools: compilers, editors, diff viewers, and source code repository (git), all operate on a line-by-line basis.  When someone makes a change that affects a portion anywhere in the line, the tools consider the entire line changed.  This can lead to nasty issues like complex merge conflicts, or worse obscure the developer activity.)
Include Headers
A good practice is to include headers in the order most local to least local and alphabetized to whatever degree possible to avoid duplications. The purpose for this is to ensure that a proper dependency chain is maintained, because as projects grow, dependency driven compilation failures sometimes can be difficult to identify and resolve.
This means that header files have includes (alphabetized) in the order:
project includes
project dependency includes
system includes
Source files have the includes (alphabetized) in the order:
definition include
project includes
project dependency includes
system includes
I hope this might be useful in the future to anyone reading this!
-Mark

@hamishwillee and anyone else, please feel free to close this issue anytime you feel it has served its' purpose!


feat: added links to tutorials.

Description
This PR adds two new tutorial links to the tutorials page:

"Building a Dark-themed Design System with React, SCSS, TypeScript, and CVA"
"Creating a Dark-themed Design System with SolidJS, CSS Variables, SCSS, TypeScript, and CVA"

Additional context

The changes in this PR include updates to the tutorials.mdx file to include the new tutorial links. The tutorials are written by me and are relevant to cva.

What is the purpose of this pull request?

[ ] Bug fix
[ ] New Feature
[x] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[x] Read the Contributing Guidelines.
[x] Follow the Style Guide.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).

@joe-bell can we add the above links to the tutorials section. Thanks.

Hey @yaldram sorry for dropping the ball on this! Just a heads up your tutorials are being added in https://github.com/joe-bell/cva/pull/214 (for beta.cva.style) 🙏


AssertionError: Attempting to delete a global file that is in use by another job

http://jenkins.cgcloud.info/job/toil-branches/branch/master/181/testReport/junit/src.toil.test.src.jobFileStoreTest/JobFileStoreTest/testJobFileStoreWithMediumCacheAndBadWorker/

http://jenkins.cgcloud.info/job/toil-pull-requests/1446/testReport/junit/src.toil.test.utils.utilsTest/UtilsTest/testUtilsStatsSort/

This was resolved in #974. Closing

https://github.com/BD2KGenomics/toil/pull/974/commits/3b3a8e0ed81ac2a32555908d4be6b3791d640dc9


Quick question: Time it takes to train?

Quick question: Time it takes to train on a GPU (If could mention the respective hardware used, that would be awesome).
Thanks!

I works on a p4 gpu. Train model 1 it takes about two minutes a epoch and model2 more than half a minute.

@happyhugo Your answer is useful for me, Thx!


rustpython-vm compile failed with cursive

Is it possible to raise the chrono version to 0.4.7?

What reasons might stop us from raising it?

I ran the tests with crono set to 0.4.7 and they pass.

Sorry for not looking into this issue earlier; you should be able to bump the chrono version. Feel free to make a PR for it if you want, otherwise I could do it in a bit.

This should be closed @coolreader18.


Ensure retention service removes shards locally

Required for all non-trivial PRs

[x] Rebased/mergable
[ ] Tests pass
[x] CHANGELOG.md updated

Fixes #8819.
As well as the main fix, this PR adds a generic tsdb.Store mock to the internal package, which can be used in other parts of the codebase. It also adds some initial testing to the Retention Service, as well as a repro test for #8819.
Previously, the process of dropping expired shards according to the retention policy duration, was managed by two independent goroutines in the retention policy service. This behaviour was introduced in #2776, at a time when there were both data and meta nodes in the OSS codebase.
The idea was that only the leader meta node would run the meta data deletions in the first goroutine, and all other nodes would run the local deletions in the second goroutine.
InfluxDB no longer operates in that way, and so we ended up with two independent goroutines that were carrying out an action that was in reality dependent on each other.
If the second goroutine (local deleter) runs before the first (meta deleter), then it may not have yet seen the meta data changes indicating which shards should be deleted. Therefore, it won't
delete any of those shards locally. Shortly after this, the first goroutine will run and remove the meta data for the shard groups.
This results in a situation where it looks like the shards have gone, e.g., according to SHOW SHARDS or SHOW SERIES, but in fact the shards remain on disk (and importantly, their series remains within the index) until the next time the second goroutine runs. By default that's another 30 minutes.
In the case where the shards to be removed would have removed the last occurrences of some series, then it's possible that if the database was already at its maximum series limit (or tag limit for that matter), no further new series can be inserted.

Oh yeah I forgot to add in the description: prior to this PR it was impossible to disabled the retention service via the config 😂


Add more gl functions

Each commit is standalone.

I will stop adding commits here as all remaining stuff that is missing for is less trivial to add.

@sagudev is this ready to review?

Yes, all my PRs that are not marked as draft are ready to land.


Add reroute nodes

Same as UE4 blueprint reroute nodes. Double-clicking on a connection should create a reroute node for better organisation.

Done in https://github.com/RobinKa/netprints/commit/617cbd24ef71c6ed2c4174d6d65da529cc5bf287


X1 Extreme Gen2 - Lenovo BIOS 1.37

Hi!,
Recently lenovo released BIOS update 1.37 for X1 Extreme Gen2, would like to know if any of you tested @erpalma's fix with this BIOS version on the machine and to tell any obvious changes in behaviour.
Thanks.

No issues and two BIOS upgrades, so, i'm closing the issue.


Possible bug fadeIn images with Firefox

I think, there is an issue about showing fadeIn images this the class lazyloaded with Firefox.
When I scroll quickly the page, Image are fade In correctly when the class change but disappears just after load for appear again with scall  (same effect when the plugin is desabled).
I test on the demo page of the plugin https://afarkas.github.io/lazysizes/. There is the same issue.
browser firefox version : 42.0
my CSS
.lazyload, .lazyloading {
    opacity: 0;
 }

.lazyloaded {
     opacity: 1;
    -webkit-transition: opacity 500ms;
	   -moz-transition: opacity 500ms;
	     -o-transition: opacity 500ms;
	        transition: opacity 500ms;
}

Not sure, whether I was able to replicate your bug. Through my testing I came a cross the following problem, that appears, if you have the site in cache already.
What to do:

Open the page: https://jsfiddle.net/tzutp2np/embedded/result/
Load it fully.
Hit reload button.

What you get:
On subsequent load you first see the image from your previous visit. Then it suddenly switches to the new one.

What you expect:
On subsequent load you should only see one image without switch and flicker.
Now, in case the images are the same, you would see something like this:

The image loads.
Disappear.
Loads again.

Is this what you mean?

I can't recreate my issue on your exemple. I'm not sur it's a cache problem because I clear the cache of my browser before testing again and the issue is the same.
Can I send you a

Yes, you can.


Material Design Support?

Do you have plans for material design support?

@alivelimeli I've just imported Material bootstrap then use this plugin its already using Material design


remove sp-std dependency

Asfaict all imports are either core or alloc, so there is no need to proxy them through sp-std. This way this crate could be easier used in envs that don’t use sp-std

I agree, and I think we can do this for much of, maybe all of, our sp-* library ecosystem


[C++] Add missing C++ Cookbook link in CONTRIBUTING.md

Looks good to me, given it's a fairly small change I'll just directly merge it


Get24HrStats does not work

According to the API this should work and it does. The test explorer fails until this edit is made.
///
/// Get 24 hour stats for a trading pair
///
/// Trading pair
/// TradingPairStats object
public async Task Get24HrStats(string pair)
{
var endpoint = $"/api/v1/market/stats?symbol={pair}";
        return await Get<TradingPairStats>(endpoint, false);
    }

Sorry for the late response, not sure what the issue is. It is working for me both on sandbox and live api


vswhere doesn't find MSBuild on Windows 11 ARM64

Windows build: 22000.100 (Windows 11 Beta)
I have the Visual Studio 2019 Build Tools installed on my Surface Pro X (Windows ARM64). I just upgraded to Windows 11 beta (build 22000.100) and now tools like vswhere can't find MSBuild.exe anymore. I tried reinstalling the 2019 Build Tools to no avail.
MSBuild is installed at C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\MSBuild.exe and is found correctly by tools like CMake.
The following command doesn't output anything:
PS C:\Program Files (x86)\Microsoft Visual Studio\Installer> .\vswhere -latest -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe

.\vswhere -all also isn't particularly helpful 😅
PS C:\Program Files (x86)\Microsoft Visual Studio\Installer> .\vswhere -all
Visual Studio Locator version 2.7.1+180c706d56 [query version 2.7.3111.17308]
Copyright (C) Microsoft Corporation. All rights reserved.

Since I'm pretty sure this broke after the Windows 11 Beta upgrade, I consider this a bug for now. Please let me know if that's not the case or if I can provide additional logs.
Here's the packages I have installed:
{
  "version": "1.0",
  "components": [
    "Microsoft.VisualStudio.Component.Roslyn.Compiler",
    "Microsoft.Component.MSBuild",
    "Microsoft.VisualStudio.Component.CoreBuildTools",
    "Microsoft.VisualStudio.Workload.MSBuildTools",
    "Microsoft.VisualStudio.Component.Windows10SDK",
    "Microsoft.VisualStudio.Component.VC.CoreBuildTools",
    "Microsoft.VisualStudio.Component.VC.Tools.x86.x64",
    "Microsoft.VisualStudio.Component.VC.Redist.14.Latest",
    "Microsoft.VisualStudio.Component.Windows10SDK.19041",
    "Microsoft.VisualStudio.Component.VC.CMake.Project",
    "Microsoft.VisualStudio.Component.TestTools.BuildTools",
    "Microsoft.VisualStudio.Component.VC.ASAN",
    "Microsoft.VisualStudio.Component.TextTemplating",
    "Microsoft.VisualStudio.Component.VC.CoreIde",
    "Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core",
    "Microsoft.VisualStudio.Workload.VCTools",
    "Microsoft.VisualStudio.Component.VC.Tools.ARM64",
    "Microsoft.VisualStudio.Component.VC.ATL.ARM64",
    "Microsoft.VisualStudio.Component.VC.MFC.ARM64"
  ]
}

Thanks in advance! 😊

Just found out this has to do with PowerShell x64 not finding a class that reports whether VS is installed:
PS C:\repos\node-gyp> powershell -ExecutionPolicy Unrestricted -Command "Add-Type -Path .\lib\Find-VisualStudio.cs; [VisualStudioConfiguration.Main]::PrintJson()"
Exception calling "PrintJson" with "0" argument(s): "Retrieving the COM class factory for component with CLSID {177F0C4A-1CD3-4DE7-A32C-71DBBB9FA36D} failed due to the following error: 80040154 Klasse is niet geregistreerd (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG))."
At line:1 char:44
+ ... b\Find-VisualStudio.cs; [VisualStudioConfiguration.Main]::PrintJson()
+                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : COMException

This works correctly in PowerShell x86 on Windows ARM64. Reported the issue to the VS team: https://developercommunity.visualstudio.com/myfeedback?port=1025&fsid=ede5705d-c496-4469-af0b-ca8275cbb656&entry=myfeedback
So I'm closing the issue here 👍🏼

Build Tools is not included by default. You have to pass -products * (see #8). The usage -? text says only Community, Professional, and Enterprise are included by default.
vswhere.exe is 32-bit only, so it can load the COM server just fine. Loading the COM server yourself in an ARM64 host is a known limitation being considered currently, to either compile the COM server for ARM64 or create a surrogate. These two issues are separate.
If you want to find Build Tools in addition to any Visual Studio SKU, just pass -products * along with any other command lines. See -? for more information.

Thanks for the info, great to know. I got distracted by the -all switch and expected that to simply return all installed products. It indeed works with .\vswhere -products * -format json -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe - thank you!


Gate_value is all 0 in the ViT-T5-base repleased model

Hi, Dear Authors:
Not sure if it is my issues. I downloaded your ViT-T5-base model from your github and test it with eval script.
But when I check the value of the gate value from layer 12 to layer 23. All of them are zero, which means the new features (question-aware) is totally ingored by the QA-transformer, which make it almost no difference from villina clip transformer model weights.
Could you double check with this? If you can see some meaningful values from your end, maybe it is my problem.
Thanks for your time! Keep building!

For people who wants to watch the weights of gate_value from layer12~layer23 of base version, I just print them here.

-0.21439486742019653 (layer index 12
0.2283252328634262
0.2884502112865448
-0.2394029051065445
0.3377605080604553
0.3446634113788605
0.37189891934394836
-0.36175721883773804
-0.37525999546051025
-0.3238457441329956
0.29202741384506226
0.07214009016752243 (layer index 23

So the mean absolute value is around 0.2~0.3, which means the original features are still important for representing the image.
Fun to see that the last layer's weight is very low, which may suggest something but i dont fully figure it out (hope authors could give some insights here :)


Allow the the api spec to specify the version

The guys here at Movio have asked if it would be possible to optionally include the version of the api inside the apidoc.json
So the suggestion would be to allow:
{
  "name": "my api",
  "version: "0.1.0-SNAPSHOT"
  ...
}

If a version is included in the json file it would override the version specified in the UI. If no version specified then use the version entered in the UI.
If you're happy with this, I can code it up.
Thoughts?

+1 I like this

If a version is included in the json file it would override the version
specified in the UI.

I would recommend that we do not do this and instead if two versions are
provided (one in the path and one in the api.json), then we return a
validation error (i.e. neither version trumps the other).
Folks that want to use version in api.json will then end up using endpoints
that simply don't have a version number (unnecessary) and will never have a
conflict.
People who use the tool the way they do today continue w/out change.
And we never end up leaving a user questioning why the version number
wasn't what they expected.
On Wed, Mar 2, 2016 at 11:26 PM, Kalman<EMAIL_ADDRESS>wrote:

The guys here at Movio have asked if it would be possible to optionally
include the version of the api inside the apidoc.json
So the suggestion would be to allow:
{
"name": "my api",
"version: "0.1.0-SNAPSHOT"
...
}
If a version is included in the json file it would override the version
specified in the UI. If no version specified then use the version entered
in the UI.
If you're happy with this, I can code it up.
Thoughts?
—
Reply to this email directly or view it on GitHub
https://github.com/mbryzek/apidoc/issues/417.

Yeah, good call. I'll throw something together.

Looking at this - we'd like to define an endpoint that we can send a spec to, it would return a version if one was specified in the spec. It would be useful to include validation errors if the json was invalid. We'd use this in the UI, when someone selects a file to upload. We'll send it to the endpoint, if we get a version back we'll disable the version input field, if no version then the user must enter it. Obviously it would make sense to display validation errors at the same point.
I've had a look at the validation endpoint and thought of a couple of options. Copy it and include an additional optional field in the response version. Alternatively I could add an optional field version to the validation endpoint. Which would you prefer?

One other alternative: In the webapp, take the json document and validate
as a Map - and if successul just look for the the "version" key directly.
The pros of this approach are no network call (and thus no additional
endpoint to manage in the api). The cons if no reusability and duplication
of knowing about the 'version' key - but in this case I think it is fine as
this is the way the www app determines which API to use.
On Sun, Mar 6, 2016 at 2:36 AM, Kalman<EMAIL_ADDRESS>wrote:

Looking at this - we'd like to define an endpoint that we can send a spec
to, it would return a version if one was specified in the spec. It would be
useful to include validation errors if the json was invalid. We'd use this
in the UI, when someone selects a file to upload. We'll send it to the
endpoint, if we get a version back we'll disable the version input field,
if no version then the user must enter it. Obviously it would make sense to
display validation errors at the sam�e point.
I've had a look at the validation endpoint and thought of a couple of
options. Copy it and include an additional optional field in the response
version. Alternatively I could add an optional field version to the
validation endpoint. Which would you prefer?
—
Reply to this email directly or view it on GitHub
https://github.com/mbryzek/apidoc/issues/417#issuecomment-192821350.

Yep, didn't think of that. Nice and simple to start with.


parse_voice uses PyValueError while all other functions use Exception

Is there a reason why parse_voice has a PyValueError::new_err(format!("{e}")) here, while all the other functions have Exception::new_err(format!("{e}")) (see)
I also noticed that all function have this code block in common:
        let mmap = match create_mmap(self.path.clone()) {
            Ok(mmap) => mmap,
            Err(e) => {
                return Err(Exception::new_err(format!(
                    "{}. File name: {}",
                    e,
                    self.path.clone()
                )))
            }
        };
        let settings = ParserInputs {
            wanted_players: vec![],
            wanted_player_props: vec![],
            wanted_other_props: vec![],
            wanted_events: vec![],
            wanted_ticks: vec![],
            real_name_to_og_name: AHashMap::default(),
            parse_ents: false,
            parse_projectiles: false,
            only_header: false,
            count_props: false,
            only_convars: false,
            huffman_lookup_table: &vec![],
            order_by_steamid: false,
        };
        let mut parser = Parser::new(settings, false);
        let output = match parser.parse_demo(&mmap) {
            Ok(output) => output,
            Err(e) => return Err(PyValueError::new_err(format!("{e}"))),
        };

Would it make sense to move this to the constructor? Then one could also more easily allow different way to pass the file, like directly passing a file handle or something like that (see discord)

Ah yes of course :+1:. Don't know why it is like that.

The second part seems to not be completely trivial.
The part from here: let settings = ParserInputs { can be moved into the constructor because some of the methods allow passing different thins to the ParserInputs.
There are also some very smal differences in the huffman_lookup_table inputs. One has this line let arc_huf = Arc::new(huf); before the huf is passed there. Although most of the other functions also call it arc_huf despite it not being arc.

Also, from the commented out code. It seems that this part used to be in the constructor. Not sure why it isnt anymore.


Need list of all wallet addresses that still have locked tokens

Talked to @robvanmieghem he'll write a script for that

@sabrinasadik do you also need the unlocktimes?

https://github.com/threefoldfoundation/tft-stellar/blob/master/scripts/timelocked.py
output is csv (owner account, TFT, escrow account)
in the readme next to the script is explained how to set up your virtual env to run the script


Error: ./~/request/~/tough-cookie/package.json

stream.js:74
          throw er; // Unhandled stream error in pipe.
          ^
    Error: ./~/request/~/tough-cookie/package.json
    Module parse failed: /Users/DongYao/Develop/proj-1/weicang_c/node_modules/request/node_modules/tough-cookie/package.json Line 2: Unexpected token :
    You may need an appropriate loader to handle this file type.
    | {
    |   "author": {
    |     "name": "Jeremy Stashewsky",
    |     "email"<EMAIL_ADDRESS>     @ ./~/request/~/tough-cookie/lib/cookie.js 38:14-40./~/request/~/mime-types/~/mime-db/db.json
    Module parse failed: /Users/DongYao/Develop/proj-1/weicang_c/node_modules/request/node_modules/mime-types/node_modules/mime-db/db.json Line 2: Unexpected token :
    You may need an appropriate loader to handle this file type.
    | {
    |   "application/1d-interleaved-parityfec": {
    |     "source": "iana"
    |   },

I am getting this too, any help will be greatly appreciated! :+1:


[CT-695] [Feature] Disable custom tests from dbt_project.yml

Is this your first time opening an issue?

[X] I have read the expectations for open source contributors

Describe the Feature
Missing feature
In our project, we sometimes want to avoid running certain tests depending on the context (target.name) and/or the location of the test (e.g. in an installed package). To do that, we expected dbt_project.yml to help us conditionally disable tests (enbaled = false).
This feature seems to be available for models, but I could not disable tests in the same fashion.
I have unsuccessfully tried the following syntax:
tests:
    dbt_utils:
      +enabled: |
        {%- if  target.name == "prod" -%} true {%- else -%} false {%- endif -%}

This triggered the following error
There are 1 unused configuration paths:
- tests.dbt_utils

Oddly enough, the documentation seems to assert this feature is available, but I haven't been able to use this feature.
I documented my qui research in the following slack thread.
Why do we sometimes need to disable certain tests
In our CI tests, we build and test models that have been changed along with their dependencies everytime a published pull requests receives a new commit. Since we have a lot of models in our project, this might mean we need to rebuild hundreds of models everytime. To prevent expansive builds, we have decided to add a limit 100 to our model builds. This means we do not test our models with a full volumetry, but dbt's syntax and SQL are tested.
Yet some of the tests we want to run out of the CI (in production) require full volumetry, such as dbt_utils.recency. In the CI, we would like to disable tests that will fail without full volumetry.
Describe alternatives you've considered
I tried adding tags to tests when they were assigned to models. Indeed those tags can be excluded from CI workflow commands.
Example:
      tests:
          - dbt_utils.unique_combination_of_columns:
              combination_of_columns:
                - id
                - name
              config:
                tags: ['packages']

Can be excluded with the following command:
dbt test --exclude tag:packages

I also considered disabling dbt data testing from the CI workflow given that tests run tests that are mostly or totally incomplete. The result of those tests are therefore almost always irrelevant to the actual results we will get in production.
Who will this benefit?
I think disabling any node from the dbt_project can be useful for other applications. It seems natural to me that we should be able to manage any node's config from dbt_project.yml.
Are you interested in contributing this feature?
I am not familiar with dbt-core's source code at all, but I would like to review this feature's code for sure!
Anything else?
I tested this both on my professional project and in a dummy project to make sure this feature was not disabled by our config. I hope I am not mistaken!
More generally I feel like it is difficult for dbt users to explore nodes and figure out which resource is linked to which resources-path. Maybe there is a command that could let us output the resource-path of a node in the console?

Hey @LoicChesneau! Thanks for the clear write-up. I think the confusion here might be around the difference between disabling specific test resources, which are often instances of generic tests, versus the generic test definitions (macros) that are (re)used to define each of those resource. This is a subtle distinction, and I'll do my best to explain it below.
Specific instances of generic tests — actual test resources — are defined in specific yaml files. Those files are included in the test resources' fully qualified names, allowing them to be configured from dbt_project.yml. So if I have a dbt_utils.equality test defined in a file named models/subfolder/some_tests.yml:
# models/subfolder/some_tests.yml
version: 2
models:
  - name: some_model
    tests:
      - expression_is_true:
          expression: "1=1"

I can conditionally the tests defined in that file via:
tests:
  my_project:
    subfolder:
      +enabled: "{{ target.name == 'prod' | as_bool }}"

Generic test definitions are templates/functions, which can be reused to define specific tests. They can define default configuration for all tests within their functional definition. For example, I could redefine the equality test from dbt_utils to change its default configuration, while cleverly avoiding the need to copy-paste all its logic (very similar to the discussion in https://github.com/dbt-labs/dbt-core/issues/3841):
{% test expression_is_true(model, expression, column_name=None, condition='1=1') %}
    {{ config(enabled = (target.name == "prod")) }}
    {{ dbt_utils.default__test_expression_is_true(model, expression, column_name=None, condition='1=1') }}
{% endtest %}

Then, all specific instances of expression_is_true tests in my project will be disabled by default. This configuration will take precedence over configuration in dbt_project.yml, but it will be superseded by configuration on each specific test in its .yml file (https://docs.getdbt.com/reference/test-configs):
# models/subfolder/some_tests.yml
version: 2
models:
  - name: some_model
    tests:
      - expression_is_true:
          expression: "1=1"
          config:
            enabled: true

Does that help answer your question? I think the thing you want already exists—it may just be trickier to disentangle than it should be, and we could come up with clearer ways to expose or document the functionality.

Hi Jeremy,
Got it, that's exactly what I was looking for!
I redefined recency as a project test with the following syntax:
{% test recency(model, datepart, field, interval) %}
    {{ config(enabled = (target.name == "prod")) }}
    {{ dbt_utils.default__test_recency(model, field, datepart, interval) }}
{% endtest %}

And used the function in my tests just like this:
  - name: my_tested_model
    description: some description
    tests:
      - recency:
          datepart: day
          field: created_at
          interval: 2
          severity: warn

Under target prod, the test was run, under target dev, it was not.
Thank you! 👏

@LoicChesneau Excellent! I think it would be valuable to add this as an example in the docs. In the meantime, I'm going to close the issue as resolved.


Create reversify

As mentioned on IRC, should make your own repo and I don't accept others scripts here.


Support Windows

What?
Provide package support for Windows
Why?
YAML Cpp is supported in Windows
How?
Use AppVeyor to build the project

Solved by #3


add ToLower to all domain fields

You post your configuration file, so merge the pr at  commit https://github.com/kenshinx/godns/commit/4275900fb2b9139aeb97fbc1e5963e8d90d5eb6d .

Ok , it's fine .


fix: load Foundation after jQuery

Description

This loads Foundation after jQuery

Closes

Types of changes

[ ] Documentation
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (anything that would change an existing functionality)
[ ] Maintenance (refactor, code cleaning, development tools...)

Checklist

[x] I have read and follow the CONTRIBUTING.md document.
[x] The pull request title and template are correctly filled.
[x] The pull request targets the right branch (develop or develop-v...).
[x] My commits are correctly titled and contain all relevant information.
[ ] I have updated the documentation accordingly to my changes (if relevant).
[ ] I have added tests to cover my changes (if relevant).

@joeworkman please don't rush releases =)


Your requirements could not be resolved to an installable set of packages

There is not a stable package for Yii2 requirements.
https://asset-packagist.org/package/bower-asset/inputmask

The first stable version starts from V4.0.0
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - yiisoft/yii2 2.0.x-dev requires bower-asset/inputmask ~3.2.2 | ~3.3.5 -> no matching package found.
    - yiisoft/yii2 2.0.29 requires bower-asset/inputmask ~3.2.2 | ~3.3.5 -> no matching package found.
    - Installation request for yiisoft/yii2 ~2.0.29 -> satisfiable by yiisoft/yii2[2.0.29, 2.0.x-dev].

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
 - It's a private package and you forgot to add a custom repository to find it

A temporary solution is: "bower-asset/inputmask": "4.0.0 as 3.3.5"

Was fixed. Now all versions of "bower-asset/inputmask" available: https://asset-packagist.org/package/bower-asset/inputmask

Yes, works as should

I confirm that it currently cannot be reproduced.


Bug Fix: Passing clientOptions to HttpPipelineBuilder in buildPipeline

resolves #28783

API change check
API changes are not detected in this pull request.


❓ [Question] Can you convert model that operates on custom classes?

❓ Question
I have a torch module that creates objects of custom classes that have tensors as fields. It can be torch.jit.scripted but torch.jit.trace can be problematic. When I torch.jit.script module and then torch_tensorrt.compile it I get the following error: Unable to get schema for Node %317 : __torch__.src.MyClass = prim::CreateObject() (conversion.VerifyCoverterSupportForBlock)
What you have already tried
torch.jit.trace avoids the problem but introduces problems with loops in module.
Environment

Build information about Torch-TensorRT can be found by turning on debug messages

PyTorch Version (e.g., 1.0): 1.10.2
CPU Architecture: intel
OS (e.g., Linux): linux
How you installed PyTorch (conda, pip, libtorch, source): pip
Build command you used (if compiling from source):
Are you using local sources or building from archives: from archives
Python version: 3.8
CUDA version: 11.3
GPU models and configuration:  rtx 3090
Any other relevant information:

You should be able to use the method in this issue to compile your model https://github.com/NVIDIA/Torch-TensorRT/issues/876

Also make sure your operation is registered through torchbind https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html


Ability to provide a idp_hint parameter

I've configured a custom Idp in fusion auth that I would like to use for login in my app.
I do not want to see the Fusion Auth hosted login page and need the user redirected to the Idp for him to put his credentials.
It looks like the Identity Provider hints is what I want to use as described here : https://fusionauth.io/docs/lifecycle/authenticate-users/identity-providers/#hints
So from what I understand I would need to append a query parameter like &idp_hint=44449786-3dff-42a6-aac6-1f1ceecb6c46 when I get redirected to my login page.
The problem is that this package login options do not contain that option. I can only see the following :
export interface IFusionAuthContext {
    login: (state?: string) => Promise<void>;  <------------- the login function called from my React UI only takes a state param
    logout: () => Promise<void>;
    register: (state?: string) => Promise<void>;
    user: Record<string, any>;
    isLoading: boolean;
    isAuthenticated: boolean;
    refreshToken: () => Promise<void>;
}
...
export interface FusionAuthConfig extends PropsWithChildren {.  <-------------- No other params to specify additional query params for the login URL
    clientID: string;
    serverUrl: string;
    redirectUri: string;
    onRedirectSuccess?: RedirectSuccess;
    onRedirectFail?: RedirectFail;
    scope?: string;
    accessTokenExpireWindow?: number;
    loginPath?: string;
    logoutPath?: string;
    registerPath?: string;
    tokenRefreshPath?: string;
    mePath?: string;
}

So is there another way for me to add such a parameter ? Or is it a case where I need to provide a custom backend myself ?
Thanks

Thanks @hgrubst ! This would be an enhancement to the react SDK. In particular, you'd want to add a configuration or run time option to set the idp hint and have the login code updated to read it: https://github.com/FusionAuth/fusionauth-react-sdk/blob/main/src/providers/FusionAuthProvider.tsx#L81
We gladly accept and review PRs.

Thanks @hgrubst ! This would be an enhancement to the react SDK. In particular, you'd want to add a configuration or run time option to set the idp hint and have the login code updated to read it: https://github.com/FusionAuth/fusionauth-react-sdk/blob/main/src/providers/FusionAuthProvider.tsx#L81
We gladly accept and review PRs.

Right on, this would be an enhancement to the react SDK
Indeed, accepting & reviewing PRs over there.


add myself to approvers and reviewers

Changes

add myself as a reviewer and approver in owners for the async component

/assign @maximilien

Closing this, adding myself to async-approvers in comunity instead.


Spin-out builder

Description
As indicated by our roadmap, we want the builder to be pulled out of the Engine. The end goal is that the daemon should know nothing about the Dockerfile.
The lack of ability to copy files to a container makes it impossible to fully implement the builder using exclusively the API. We have ongoing PR for this, that would be a prerequisite for this PR:

#13171: docker cp to and from containers
#14058: more docker cp

Apparently @jlhawn also has work on his fork for a client-side builder.
How to

[x] Help review and merge the docker cp PR mentioned above
[ ] Help review and see what pieces of @jlhawn can be integrated in the Engine

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

@docker/maintainers I've just rebased the PRs referenced above so they are back in a merge-able state after 2 weeks of falling behind the master branch.
So far, my client-side builder work has been happening in a separate project named dockramp but it's more of an proof-of-concept for a client-side builder as well as new Dockerfile syntax. We definitely want to keep backwards compatibility if we spin out the builder to be client-side in Docker 1.x so we should continue to use the current Dockerfile parser and consider dockramp to be a potential Dockerfile v2.

:+1:

Good news: #13171 is in. Let's try and work through the rest for 1.9.0.

Awesome. This means that dockramp will work with Docker 1.8 if people want to use it.

is there a way to add dockeramp as an experimental feature?
thinking; perhaps we should have a plugin system for the client after all, so that users can add custom builders; docker build --builder=dockramp -f Dockerfile.dockramp?

@jlhawn Can you please explain how are you running commands inside a container during build time and how do you implement caching for a client side builder?

@jlhawn  @thedrow  the way I think about it, is that we start a temporary container, do the build, commit the container into an image.

@muayyad-alsadi Then how do you get the caching capabilities of the normal builder?
If you start a temporary docker container and build it using docker exec you won't get any caching.

@thedrow @muayyad-alsadi COPY and RUN creates a new container. For COPY files are copied into the container via the new API endpoint and the container is immediately committed having never been started. For RUN the container is started using the command specified by the RUN, then committed. A commit returns a new image ID.
Caching is done by the client storing a hash of the image ID + the build command(s) that would lead to the committed image. This hash maps to the committed image ID. The client stores this mapping in a local file. For every step of the build file, the builder does something like this:
I'm starting this build step with image ID `x` and about to execute a command that does `y`.
Is there a local mapping of `sha256(x+y)` -> image ID `z`?
yes? does it exist on the daemon?
yes? that's a cache hit and the build step can be skipped.
continue to the next build step with image ID `z`.

I see this in the 1.10 roadmap, but with little recent activity. Eyes are on this for better artifact management in combination with nested build steps for greater CI implementation flexibility. This is kind of the missing part to make things perfect at the far end of the CI Toolchain. /fyi @kurkop

@blaggacao It's definitely being actively worked on.
Will it make 1.10? I'm not sure. Code freeze is coming

A lot of work was achieved to decouple the current builder subsystems from other parts of the Engine (most notably from the daemon). As of Docker Engine 1.10, this sub-package could now be reused to provide out-of-daemon build capabilities using the API.
We won't go further on this effort ourselves, as backward compatibility constraint forces us to preserve a daemon-side implementation of the Builder anyway. I hope some people will catch up with this effort and propose interesting things outside of the daemon.

So this won't be released separately?

So as a docker amateur I ran into the issue of not being able to set the correct user:group when doing a COPY. After spending quite of time trying to reverse engineer what happened on this issue I found #9934 by @thaJeztah et al that addresses this issue back in Jan 2015 (!). After quite some discussions it was decided to fix this with options which were actually implemented and merged May 2015 to master as part of #10775 @duglin @icecrime et al. But so far the COPY uid issue is still not fixed, but all preparations are done. Then May 2015 @kostickm tried to actually do the implementation that was prepared for and decided on in #13600. This was blocked Jul 2015 because Jun 2015 @icecrime started this effort to move builder logic to client side. End result is that after more then a year the COPY uid issue is still open (or better said: #9934 and #13600 are actually closed, but the actual problem is not resolved). So what is the plan on this? Are we now in a right state to reattempt #13600 ?


Pipfile.lock and dev transitive dependencies

Issue description
Pipfile.lock "development" section seems appears to include transitive dependencies which are redundant, yet often have different versions than "default" dependencies.
I've confirmed that these redundant dependencies are not needed.  (E.g. if a package is only listed as dependency in "default' of Pipfile.lock, and you do a pipenv sync --dev, it will get installed.)
Expected result
Given that --dev implies default + dev dependencies, I don't expect there to be dependencies duplicated between the default and dev sections of Pipfile.lock.  Moreover it's confusing to see contradicting versions specified for the same package within the Pipfile.lock.
Actual result
"development" section of Pipfile.lock contains transitive dependencies which are redundant and often different than "default' section.

Yes, yet the sections are different and they therefore contain different content.  This is because you can specify different requirements in each section and they are resolved separately, and in theory (although due to an implementation detail, not in practice just yet), they can be manipulated separately.
In reality this functionality has been built out but the desired behaviors are not well defined.  The former constraints were around merging the sections together during installation, so we always prioritize any appearance of a package in packages as opposed to dev-packages.  This is a lossy approach as extras are dropped during this process (among other things).  In a future release we will be able to merge this information properly so that extras specified in the dev section are appropriately merged into any extras from the main section, etc.
So, while the metadata underlying the transitive dependencies may on occasion be overridden or ignored, that definitely does not mean the section is not necessary.  If you were to read the code underlying the implementation and propose an approach via a PEEP (pipenv enhancement proposal) to alter the way we are handling dependency resolution, explaining how we can remove our implementation and why you feel it is not necessary, I am guessing we would review it.  The current implementation is not an accident however so you would have to address the reasons behind its design and an alternative way to do the things we are planning to do with it.
As this issue is kind of a more general comment about the current situation being somewhat unclear/unfortunate, I'm going to close it for now.  As an FYI I'd be happy to review/comment on/work through a PEEP that just outlines the existing plan if that helps -- time is the biggest constraint on our end which is why none of this is documented clearly
thanks for the issue and hopefully this at least somewhat clarifies things


Support for Spring ThreadPoolTaskExecutor

It would be nice to support other task executors besides ScheduledExecutorService and Scheduler.
Since I'm using Spring with AsyncTaskExecutor I cannot take full advantage of failsafe.
Suppose I have a bean:
@Bean
public AsyncTaskExecutor asyncExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(5);
    executor.setMaxPoolSize(10);
    executor.setQueueCapacity(25);
    executor.setThreadNamePrefix("myExecutor-");

    return executor;
}

It would be nice to do something like:
Failsafe.with(retryPolicy)
                        .with(asyncExecutor())
                        .onFailedAttempt((o, throwable, context) ->
                                log.warn("Retrying, Message: {}, Attemp: {}", throwable.getMessage(), context.getExecutions())
                        )
                        .onFailure(throwable -> log.error("Failure, {}", throwable.getMessage()))
                        .withFallback(() -> log.info("Recovering"))
                        .run(() -> myService.doStuff());

Instead of:
asyncExecutor().execute(() ->
                Failsafe.with(retryPolicy)
                        .onFailedAttempt((o, throwable, context) ->
                                log.warn("Retrying, Message: {}, Attemp: {}", throwable.getMessage(), context.getExecutions())
                        )
                        .onFailure(throwable -> log.error("Failure, {}", throwable.getMessage()))
                        .withFallback(() -> log.info("Recovering"))
                        .run(() -> myService.doStuff())
        );

Hi @mguilherme
I currently have no plans to support ThreadPoolTaskExecutor since this would require a dependency on Spring. The idea behind Scheduler is that it can be used to adapt other library's schedulers to be used in Failsafe. To make this a bit easier, Failsafe included a DefaultScheduledFuture. Have a look at this Vert.x example that includes a Scheduler implementation for Vert.x.

Spring's ThreadPoolTaskExecutor is just using Java's normal executors/schedulers under the hood. I'd just try to not use Spring's "abstraction" and use the traditional interfaces directly.


[Feature Request] Regex-based beartype.vale.IsMatch validator API

This feature request – originally courtesy @Saphyel at issue #32 and then reiterated by @xerz-one in a helpful necrobump – politely begs @beartype to provide a new beartype.vale.IsMatch validator.
As its name suggests, beartype.vale.IsMatch is a space- and time-efficient validator matching arbitrary strings in callable parameters and returns against the caller-defined regular expression subscripting (i.e., indexing) that validator: e.g.,
from beartype import beartype
from beartype.typing import Annotated
from beartype.vale import IsMatch

StringUUID = Annotated[str, IsMatch[
    r'[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$']]
'''
PEP-compliant type hint matching a 128-bit
universally unique identifier (UUID).
'''

@beartype
def make_uuid() -> StringUUID:
    '''
    Generate a random UUID using a classic Linux trick.

    Please don't actually do this, everyone.
    '''

    # ...kek, kek, kek.
    with open('/proc/sys/kernel/random/uuid', 'r') as random_uuid:
        return random_uuid.read()

Ominious Questions Arise

Does @beartype internally compile and cache each caller-defined regex {regex} subscripting IsMatch[{regex}]? I say, "Gods, yes!" @beartype promises speed. I intend to make good on that promise. But then...
Does @beartype cache an infinite number of compiled regexes? I say... "Maybe?" How large is the average compiled regex, anyway? How many IsMatch[...] validators would the average app stack require, anyway? If space is a concern, should @beartype LRU this? If so, at most how many compiled regexes should @beartype internally cache into that regex LRU?

All these questions and more, I expect someone who is not me to answer. I'm currently hip-deep in automagical import hooks for our upcoming beartype 0.11.0 release. So, my mental bandwidth for dark regex magic is scant at best.
Let the typing rabble voice their educated opinions, I say.

@leycec I'm interested in helping contribute to this, though it feels like a stretch at this moment, having only discovered beartype a few minutes ago.
I asked a StackOverflow question related to this five years ago, so it would be a cool feat to help make it come true.
In addition to diving into beartype's current codebase, do you have any suggestions for how to get started on this feature?

@leycec I'm interested in helping contribute to this

OMG. Yes. So much yes! Take all the good karma I've accrued over a brief unpaid career of balding open-source volunteerism, because it's all I have to give. If I had hair, I'd give that too. Sadly, this emoji resembles me. :disappointed:

...having only discovered beartype a few minutes ago.

You're in luck yet again! You just received all my good karma. ...see above
More importantly, this is actually a fantastic issue for first submitters. @tlambert03, who recently authored our Decidedly Object-Oriented Runtime-checking (DOOR) API via a series of wizardly pull requests (PRs), can verify that I am a permissive maintainer and will happily merge literally anything.
Please, @tlambert03. Say the things I'm desperately begging you to say.
Just Tell Me What to Do, You!
Okay. So, this is indeed a fun issue that should be mostly fun and only slightly painful.
The @beartype validator API resides in our beartype.vale subpackage. @beartype validator factories (e.g., Is[...], IsAttr[...], IsEqual[...]) are currently defined in individual submodules of our private beartype.vale._is subpackage. So far, so good.
The existing beartype.vale._is._valeisoper submodule seems like an appropriate landing place for a new IsMatch[...] validator. That submodule contains validators that validate by performing a unary or binary operation on the thing to be validated. Equality testing is an obvious example, but so is string matching. Right? Right. Let's imagine that I speak the truth.
Specifically, let's define a new _IsMatchFactory subclass somewhere in beartype.vale._is._valeisoper. Initially, don't sweat the rough stuff like docstrings. I mean, you can if you'd like; that'd be great. But a minimal length implementation is totally fine, too. Something like:
from re import compile, match

class _IsMatchFactory(_BeartypeValidatorFactoryABC):
    @callable_cached
    def __getitem__(self, regex: TypeOrTupleTypes) -> BeartypeValidator:

        #FIXME: Generalize this factory to accept subscriptions by two or
        #more regexes, maybe? For now, one is definitely fine. KISS, yo!
        # If this factory was subscripted by either no arguments *OR* two or
        # more arguments, raise an exception.
        if isinstance(regex, tuple):
            raise BeartypeValeSubscriptionException(
                f'{self._getitem_exception_prefix}empty tuple or two or more arguments.')
        # Else, this factory was subscripted by exactly one argument.

        #FIXME: Validate "regex" to be a non-empty string here, please. It'd be great
        #if we could also validate "regex" to be a valid regular expression? Can we do
        #that? Probably. The standard "re" module probably raises various exceptions
        #when passed strings that are invalid as regular expressions (e.g., "{").

        # Compiled regex, subsequently cached to improve time efficiency.
        regex_compiled = compile(regex)

        #NOTE: We internally refer to the current object being validated as the "pith."
        #Why? Just because we can! \o/
        # Callable inefficiently validating against this object.
        is_valid = lambda pith: match(regex_compiled, pith)  # <-- pretty sure that works?

        # Dictionary mapping from the name to value of each local attribute
        # referenced in the "is_valid_code" snippet defined below.
        is_valid_code_locals: CallableScope = {}

        # Name of a new parameter added to the signature of wrapper functions
        # whose value is this compiled regex, enabling this regex to be tested in
        # those functions *WITHOUT* additional stack frames.
        param_name_regex_compiled = add_func_scope_attr(
            attr=regex_compiled, func_scope=is_valid_code_locals)

        #FIXME: Right. So, we also need to implement the
        #"VALE_CODE_CHECK_ISMATCH_TEST" string global referenced
        #below. This is the Python code snippet performing this match that
        #will be embedded in type-checking wrapper functions generated by
        #the @beartype decorator. Should be mostly easy based on similar
        #existing snippets, but please ping me if you hit any deep snags!
        # Code snippet efficiently validating against this object.
        is_valid_code = VALE_CODE_CHECK_ISMATCH_TEST_format(
            param_name_regex_compiled=param_name_regex_compiled)

        # Create and return this subscription.
        return BeartypeValidator(
            is_valid=is_valid,
            is_valid_code=is_valid_code,
            is_valid_code_locals=is_valid_code_locals,
            get_repr=lambda: f'{self._basename}[{repr(obj)}]',
        )

Phew! I'm exhausted. So are you. Let's start there, if you'd like. Exposing that private factory to end users entails a bit more – but not much, honestly. Tests and documentation would also be swell at some point – but sorta out-of-scope for a first PR unless you have a humongous excess of free time.
And... I'm a quietly pass out here. :sleeping:


Conformance Tests: TLSRoute Passthrough

This issue is intended to track conformance test development for TLSRoute Passthrough. Comment below if you're interested in working on covering any of these areas.
Core Capabilities:

[x] Valid TLSRoute with 1 backendRef/rule
[ ] Valid TLSRoute with 1 backendRef/Rule and matching SNI hostname
[ ] Valid TLSRoute with 1 backendRef/Rule and matching SNI hostname with wildcard prefix
[ ] Valid TLSRoute with 1 backendRef/Rule and matching SNI hostname for both Listener and TLSRoute with intersecting hostname
- [ ] Invalid TLSRoute  - no backendRef/rule doesn't become Ready
- [ ] Invalid TLSRoute - invalid backendRef/rule doesn't become Ready (some or all from sublist?)
- [ ] Missing Rule
- [ ] Missing Port
- [ ] Kind, if not nil, is also not Service
- [ ] Multi ParentRef sections names not set
- [ ] Multi ParentRef sections names not unique
- [ ] SNI hostname invalid - IP address

[ ] TLSRoute must not be Accepted for any invalid hostnames; ‘Accepted’ condition False in RouteParentStatus
- [ ] SNI hostname invalid - not RFC1123, not wildcard prefix
[ ] Matching SNI hostname for both Listener and TLSRoute without intersecting hostname
[ ] TLSRoute hostname that doesn’t match the Listener hostname is ignored
- [ ] More than 16 hostnames
- [ ] More than 16 rules
- [ ] More than 16 backendRef
-[ ] Invalid TLSRoute with 1 backendRef/Rule that doesn’t exist performs no default forwarding

[ ] Invalid TLSRoute with invalid BackendObjectReference performs no default forwarding (some or all from sublist?)
- [ ] Name not set
- [ ] Kind not  set

[ ] Namespace (of backend) not set
[ ] Namespace (of backend) missing ReferenceGrant
- [ ] Port missing when Kind is Service

[ ] Invalid TLSRoute with filters that provide response must REJECT connection or return 500 status (some or all from sublist?)

[ ] Rejections must observe weight, e.g. if 50% might have been delivered, then 50% should be rejected instead

[ ] For a Listener setting mode: "terminate", attempting to set TLSRoute in AllowedRoutes.kinds would yield a ListenerConditionType Accepted with status: false and ListenerConditionReason InvalidRouteKinds

[ ] this could probably be validated by webhook too

[ ] For a Listener setting mode: "terminate", TLSRoute should not be present in ListenerStatus.SupportedKinds
[ ]  Attempting to attach a TLSRoute to a Listener setting mode: "terminate" should yield RouteConditionType Accepted with status: false and RouteConditionReason NotAllowedByListeners on the TLSRoute

Thanks @candita! I think it would be worth differentiating between tests of our API validation and tests of the actual underlying implementation. At least so far, I've been working with the assumption that conformance tests are only intended to cover implementation details, not our CRD or webhook validation. https://github.com/kubernetes-sigs/gateway-api/issues/1514 has a bit more context on that discussion though.

I agree that for now, if one of these cases is covered by CRD validation, webhook validation, or both, we can check it off, and then come back to testing it as part of sorting out #1514.

Yep, that looks great, thanks @candita!

Suggested additions to the list of conformance tests since we've discussed disallowing mode: "terminate" with TLSRoute:

[ ] for a Listener setting mode: "terminate", attempting to set TLSRoute in AllowedRoutes.kinds would yield a ListenerConditionType Accepted with status: false and ListenerConditionReason InvalidRouteKinds

[ ] this could probably be validated by webhook too

[ ] for a Listener setting mode: "terminate", TLSRoute should not be present in ListenerStatus.SupportedKinds
[ ] attempting to attach a TLSRoute to a Listener setting mode: "terminate" should yield RouteConditionType Accepted with status: false and RouteConditionReason NotAllowedByListeners on the TLSRoute

@mikemorris

Suggested additions to the list of conformance tests since we've discussed disallowing mode: "terminate" with TLSRoute:

[ ]  for a Listener setting mode: "terminate", attempting to set TLSRoute in AllowedRoutes.kinds would yield a ListenerConditionType Accepted with status: false and ListenerConditionReason InvalidRouteKinds

[ ]  this could probably be validated by webhook too

[ ]  for a Listener setting mode: "terminate", TLSRoute should not be present in ListenerStatus.SupportedKinds

[ ]  attempting to attach a TLSRoute to a Listener setting mode: "terminate" should yield RouteConditionType Accepted with status: false and RouteConditionReason NotAllowedByListeners on the TLSRoute

I added these to the description so they wouldn't get lost.  Can you provide more detail about  this could probably be validated by webhook too?  It's just a comment, not a test, right?

Added an issue here https://github.com/kubernetes-sigs/gateway-api/issues/2153 which is a follow up for https://github.com/kubernetes-sigs/gateway-api/pull/2076
Looks relevant to include as part of the checklist here

/remove-lifecycle rotten

This is a blocker for #2643.

@sunjayBhatia

Added an issue here #2153 which is a follow up for #2076
Looks relevant to include as part of the checklist here

That is this one, correct?

Invalid TLSRoute with invalid BackendObjectReference performs no default forwarding (some or all from sublist?)

Updating my prior comment about passthrough and terminate - there's not a conclusive determination yet on what the expected behavior should be for this, and we'll probably need to figure that out as part of encoding anything in conformance.
Refs Slack thread, #1474 and #2111

/remove-lifecycle-stale

/lifecycle frozen
/priority important-soon


support CIDRs for hosts config

Signed-off-by: tfwang<EMAIL_ADDRESS>We can use CIDR for docker's insecure registry config, but containerd doesn't support it
This PR tries to support CIDR in registry hosts configs.
For the CIDR in the format of 'x.x.x.x/x' need change to x.x.x.x_x as the directory name.
We can use * as the host placeholder in hosts.toml.
For example, if we want to config insecure registry for <IP_ADDRESS>/24, the config looks like this:
# <IP_ADDRESS>_24/hosts.toml
[host."http://*"]
  capabilities = ["pull", "resolve", "push"]
  skip_verify = true
[host."https://*"]
  capabilities = ["pull", "resolve", "push"]
  skip_verify = true

How does this work with IPv6?
IPv6 addresses contain : but : is prohibited as a dir name character on Windows

How does this work with IPv6?
IPv6 addresses contain : but : is prohibited as a dir name character on Windows

Thank you for point out it. @AkihiroSuda
Can you give any advice?
Maybe we can replace : to - ?

We can use CIDR for docker's insecure registry config, but containerd doesn't support it

Can you discuss the use case a little more? Our goal isn't just to reach config parity with Docker, but to support as broad a range of use cases as we can. This implementation seems very specific, would like to understand how and why registry namespaces are being used in this that it requires CIDR parsing and whether we might have other ways we could configure it. Replacing "*" with the namespace host doesn't really fit with the existing logic.

We can use CIDR for docker's insecure registry config, but containerd doesn't support it

Can you discuss the use case a little more? Our goal isn't just to reach config parity with Docker, but to support as broad a range of use cases as we can. This implementation seems very specific, would like to understand how and why registry namespaces are being used in this that it requires CIDR parsing and whether we might have other ways we could configure it. Replacing "*" with the namespace host doesn't really fit with the existing logic.

The background is we are migrating from docker to containerd.
One of our use case is to test harbor automatically.
We will create a harbor instance and pull images from it.
When using docker, we can set a CIDR such as "<IP_ADDRESS>/24 in the docker's config insecure_regitstries, so that we can pull images from the testing harbor.
But when using containerd, we have to set the specific IP address of the harbor in containerd's hosts config. The harbor is created dynamically, which means its IP address may changed. So we have to update the configs on each node, that is a burden when you have lots of nodes to deal.
In our use case, we don't need hosts.toml to replace registry host. The "*" is just a placeholder to stands for using the origin host.


⚠ Bofrost.at has degraded performance

In 84804ff, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2106 ms

Resolved: Bofrost.at performance has improved in c0e1b34.


Helpful Python logging

Description
As a TRE Administrator/Developer
I want to see useful logs when errors occur in the python apps - specifically the API
So that could understand and resolve bugs
The following pattern is widely used in the api app but it doesn't help too much.

It doesn't include stack trace, making it hard to understand where the error came from
Sometimes the e isn't converted to string correctly

try:
  something
except Exception as e:
  logging.error(f"Some error: {e}")
  maybe raise something

Acceptance criteria

[ ] TBD
[ ] ...

@martinpeck to flesh out issue requirements.

These two articles are well worth reviewing/reading and cover the details of how logging should be used in Python, and how exception blocks should be used:
https://guicommits.com/how-to-log-in-python-like-a-pro/
https://guicommits.com/handling-exceptions-in-python-like-a-pro/
In particular, we should be using logger.exception in exception blocks:
try:
    ...
except CustomException:
    logger.exception("custom message")

You don't have to log the exception object. The exception function of the logger is intended to be used as-is inside except blocks. It already handles the stack trace with execution info and displays which exception caused it with its message set to ERROR level!


IncompatibleBases as a static versus dynamic check, i.e. should Basis be isbits?

The deeper I dig into the basis system as part of #32, the more I seem to see reasons for an overhaul 😅. This is of course very related to #27 and #33 but different enough that I thought it helpful to have a dedicated issue. Currently it seems there's two methods at tension for checking whether bases are compatible: the samebases machinery versus the typesystem. This is easily illustrated with an example
using QuantumOpticsBase
k1 = basisstate(SpinBasis(1), 1)
k2 = basisstate(SpinBasis(3//2), 2)
k1 + k2

throws ERROR: QuantumInterface.IncompatibleBases()
meanwhile
using QuantumOpticsBase
k1 = basisstate(NLevelBasis(3), 1)
k2 = basisstate(NLevelBasis(4), 2)
k1 + k2

throws ERROR: DimensionMismatch: a has size (2,), b has size (3,), mismatch at dim 1
The relevant part isn't so much the error messages themselves, we can easily (and probably should) make the latter example throw IncompatibleBases, but in how these two situations are caught due to using the type system
+(a::Ket{B}, b::Ket{B}) where {B} = Ket(a.basis, a.data+b.data)
+(a::Ket, b::Ket) = throw(IncompatibleBases())

When differences in bases are encoded within their types as currently happens with SpinBasis when called created with an isbits type, then this pattern catches incompatible bases. However, if incompatible bases are not evident at the type level, for example when using BigInt which has isbits false, then a dynamic samebases check is required.
I think making Basis isbits could potentially to unlock performance benefits since this might allow StaticArrays to be used! Then downside is that this means one has to carefully subtype Basis so that equality of bases is fully encoded in type parameters for which isbits is true. I think this is certainly doable for all the bases collected or proposed in #33. Of course this would also be a major breaking change.
In either case, I think it makes sense to pick only one pattern, document it here, and try to stick to it in dependent packages.

"Breaking change" can mean two different things in the Julia ecosystem.
If it is breaking to some other libraries that rely on internal details of QuantumInterface, then that is not "technically/semver" breaking.
If it is breaking the public API, then it is "technically/semver" breaking.
I suspect much of what you are suggesting here is not "technically" breaking because it is about the internal layout of structs. We can still keep their old constructor API even after they are modified to be isbits. Can we do this in small increments? What is the smallest commit that can be made in this direction? Maybe making just one of the basis be isbits? Doing it in such small increments might be possible to do 90% of the work without any breaking.

Yes I've been keeping the constructors the same as I've been experimenting on this, that's easy. The question is whether changes in parametric typing is considered breaking? This is also the same question about changing from AbstractOperator{BL,BL}' to AbstractOperator{B<:OperatorBasis}` since anything that method dispatches based on the parametric types will break.

I have not checked too carefully, but I suspect changes in the type parameters is not semver/technically breaking.
Also, as we are going through these changes, especially through fixes, we should see whether we can simplify the dispatch signatures so that they are less susceptible to being broken by such internal changes.

I have been thinking about related issues for a while and I would like to offer my two cents here. There is a place where relying on the type system for basis-equality checks is, in my opinion, problematic: CompositeBasis.
A lot of functionality relies on type checking to exclude invalid basis combinations. This sometimes misses incompatibilities (as you describe above). It is also sometimes too strict (as in #27) and, in the case of CompositeBasis in particular, leads to a combinatorial explosion of possible basis types, because the concrete type of a CompositeBasis includes a tuple of the component basis: tensor(FockBasis(d), GenericBasis(2)) gives us a basis with a different type to tensor(GenericBasis(2), FockBasis(d)) for example.
Of course, this is by design since we rely on types to test basis equivalence, but it can lead to excessive compilation overhead. Consider the basis reduce(tensor, (GenericBasis(2) for _ in 1:N)) - we will end up compiling any high-level function for every value of N encountered.
This makes me consider whether we should rely much less on the type system for basis checking and have runtime (even runtime-dispatched) basis checking by default. We can choose to turn off these checks (e.g. via a global variable) for things like integration loops in time evolution. Interested to hear your thoughts @akirakyle and @Krastanov.
One strategy might be to introduce runtime checks for CompositeBasis in particular (and maybe transition CompositeBasis to use a vector of component bases, rather than a tuple).

I am also leaning towards using much more runtime tools. The incompatible basis checks are not needed in hot loops, so there is no engineering need to make them static.

In terms of pure programming style I think the using the type system for basis checks is much more elegant. As for the performance cost (https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-value-type), I'm not sure I necessarily agree that the overhead is unmanageable since it really depends on the usage patterns? Ideally both approaches could be benchmarked, but that requires a bit of work. I think one thing to keep in mind is that the number of elements in a CompositeBasis, at least for anything using explicit data types, is practically quite limited given that the total Hilbert dimension grows exponentially.

and maybe transition CompositeBasis to use a vector of component bases, rather than a tuple

I'm curious why you think a vector is preferable over a tuple? I would think that regardless of how checks are implemented, we should treat bases as immutable objects?

Alright I think I'm convinced that at least the potential explosion in types with making bases isbits isn't going to be worth it for how bad the performance pitfalls could be. As an experiment I tried
abstract type Basis end

struct GenericBasis{N} <: Basis
    GenericBasis(N) = new{N}()
end

struct CompositeBasis{T<:Tuple{Vararg{Basis}}} <: Basis
    bases
    CompositeBasis(x) = new{typeof(x)}(x)
end

bases(b::CompositeBasis) = b.bases
nsubsystems(b::Basis) = 1
nsubsystems(b::CompositeBasis) = length(bases(b))

function reduced(b::CompositeBasis, indices)
    if length(indices)==0
        throw(ArgumentError("At least one subsystem must be specified in reduced."))
    elseif length(indices)==1
        return bases(b)[indices[1]]
    else
        return CompositeBasis(bases(b)[indices])
    end
end

function ptrace(b::CompositeBasis, indices)
    J = [i for i in 1:nsubsystems(b) if i ∉ indices]
    length(J) > 0 || throw(ArgumentError("Tracing over all indices is not allowed in ptrace."))
    reduced(b, J)
end

function timeit(bs)
    @time r1 = [ptrace(b, [i for i=1:nsubsystems(b) if i%3==0]) for b in bs];
    @time r2 = [ptrace(b, [i for i=1:nsubsystems(b) if i%3==0]) for b in bs];
    return r1,r2
end

timeit([CompositeBasis(Tuple(GenericBasis(i+j) for j=1:40)) for i=1:100]);

giving
  2.129629 seconds (4.18 M allocations: 200.085 MiB, 1.23% gc time, 99.59% compilation time)
  0.349378 seconds (232.07 k allocations: 11.013 MiB, 99.29% compilation time)

If instead I replace the structs with
struct GenericBasis <: Basis
    N
end

struct CompositeBasis <: Basis
    bases
end

and run the benchmarks as
println("small tensors - tuple")
timeit([CompositeBasis(Tuple(GenericBasis(i+j) for j=1:40)) for i=1:1000]);
println("small tensors - vector")
timeit([CompositeBasis([GenericBasis(i+j) for j=1:40]) for i=1:1000]);

println("big tensors - tuple")
timeit([CompositeBasis(Tuple(GenericBasis(i+j) for j=1:100)) for i=1:1000]);
println("big tensors - vector")
timeit([CompositeBasis([GenericBasis(i+j) for j=1:100]) for i=1:1000]);

I get
small tensors - tuple
  0.080450 seconds (223.22 k allocations: 10.906 MiB, 96.35% compilation time)
  0.029728 seconds (96.21 k allocations: 4.518 MiB, 91.00% compilation time)
small tensors - vector
  0.038910 seconds (65.98 k allocations: 3.877 MiB, 32.06% gc time, 96.60% compilation time)
  0.001320 seconds (20.00 k allocations: 1.518 MiB)
big tensors - tuple
  0.013207 seconds (96.29 k allocations: 5.158 MiB, 58.38% compilation time)
  0.005447 seconds (89.00 k allocations: 4.799 MiB)
big tensors - vector
  0.002483 seconds (21.00 k allocations: 3.258 MiB)
  0.002457 seconds (21.00 k allocations: 3.258 MiB)

So I think just by not parametrically typing subtypes of Basis we'll avoid the issue with an explosion of compiling specalized functions. While the vector implementation is slightly faster, I would worry about the fact that they can be unintentionally mutated which could cause bugs when the mutated bases field no longer corresponds to whatever object it is attached to (i.e. Ket, Bra, Operator).

This is a really neat set of benchmarks and experiments, thank you!
You are right about all the worries about mutation and generally the fact that runtime checks are less elegant. This is a frequent seen tradeoff between perfection and practicality.
Julia usually lets people mutate private fields on the principle "we are all adults here". As long as these fields are not exposed as a public part of the API, I think the risk level of someone messing them up and mutating them is acceptable.

Julia usually lets people mutate private fields on the principle "we are all adults here". As long as these fields are not exposed as a public part of the API, I think the risk level of someone messing them up and mutating them is acceptable.

Got it, vectors it is then 😅
I think this further means that for StateVector, AbstractKet, AbstractBra, AbstractOperator, or AbstractSuperOperator (I'll call all these quantum objects) we should not give them a type parameter constrained to be a subtype of basis since we don't want functions which accept their subtypes to be specalized based on the subtype of Basis they're defined on. Rather we just require them to implement basis and fullbasis methods.
While this change will require a fair bit of updating in QuantumOpticsBase because there's a lot of methods which use the (a::Ket{B}, b::Ket{B}) where {B} pattern, it's definitely going to be good to have a well defined pattern for knowing when objects are compatible. That brings up the next issue which is what that interface should look like. Currently here we have

multiplicable(a,b) with docstring stating "Check if two objects are multiplicable." and defined on both subtypes of Basis and subtypes of quantum objects.
samebases with docstring stating "Test if one object has the same left and right bases or if two objects have the same bases" and defined for one and two arguments of subtypes of bases and one and two arguments of subtypes of quantum objects.
check_multiplicable and check_samebases which call their respective functions after first testing BASES_CHECK which can be disabled the @samebases macro.

As far as naming go, I personally find it quite confusing that the @samebases actually disables calls to samebases and multiplicable happening within that call stack. I also think allowing these functions to be called on both subtypes of basis and subtypes of quantum objects is a bit confusing, for example what does it mean to multiply two bases? I think practically there's several distinct patterns that need checks: if quantum objects can be added, or if quantum objects can be multiplied, and if operators or superoperators have the same left and right bases. Thus I might suggest clarifying the interface to something like

'multiplicable(a,b)' where a,b are subtypes of quantum objects and checks that they can be multiplied (right_basis(a) == left_basis(b))
addible(a,b) where a,b are subtypes of quantum objects and checks that they can be added (left_basis(a) == left_basis(b) && right_basis(a) == right_basis(b))
'square(a)' where a is a subtype of a quantum object and checks that it is square (left_basis(a) == right_basis(b))
check_* versions of the above with a @compatiblebases macro which disables them.
For checking if bases are the same, I think we should just explicitly use == rather than a samebases since I think that's clerarer anyways. Also I think this would allow a slower transition to the new interface as samebases could be kept the same but eventually deprecated once everything is using a "finer grained" basis checking interface.

Thoughts?


Circle mask on AnimatableView in IB requires layer.masksToBounds to be true

I couldn't get the circle mask working without layer.masksToBounds set to true in MaskDesignable. The other masks worked fine as they weren't using cornerRadius.
Am I missing something?

I checked the Clip Subviews box and it is now masked correctly (without masksToBounds being set to true). That seems to be what I was missing if that is the recommended approach.


WIP: (feat) Implement  with life-cycle.

Goal: Convert existing Polymer v1 behavior to a Polymer v2 data element, which follows idiomatic conventions. Have a smooth and seamless transition from behavior to data element.
Element:

<dolphin-controller>

Properties:

clientContext: Object, defaults to window.clientContext
auto: Boolean, defaults to false
name: String
state: String
_controller: Object

Methods:

create()
destroy()

Considerations:

Being able to use the element without <script> code (i.e. it will use the global window.clientContext, if available).
Being able to enable and disable updates from Dolphin to the model property, or even empty the model completely, if disabled.
Being able to pass <dolphin-controller>'s model property or its subproperties to elements defined in the host element via attributes.
Being able to use <dolphin-controller>'s model property or its subproperties within the host element, e.g. define a model property and observers on it in the host element.

Advantages:

It will give us a modular reusable data element, similar to <iron-ajax>.
It will give us an easy way to create and destroy the Dolphin controller within the element.
It will give us better support for documentation and linting, because we define a subclass of Polymer.Element, see: https://www.polymer-project.org/2.0/docs/tools/documentation.

@hastebrot this PR do not make sense to me since it is a PR that do not targets the master. Can you please create a PR that will update the master branch?

@hendrikebbers This is intentional to get rid of all the infrastructure and codestyle changes in #12.

@hastebrot ok, fine for me. But doing this as a PR do not make sense since no one without you does know the code of the 'polymer-2' branch. So what should be reviewed here?

Status of migration of tests to <dolphin-controller>-API:

[ ] (0/3 tests) simple binding with dolphin bean.
[ ] (0/7 tests) simple binding with dolphin bean array.
[ ] (0/9 tests) deep binding of bean within a bean.
[ ] (0/11 tests) deep binding of bean within an array.
[ ] (0/2 tests) dolphin commands.

I think getting a DP polymer element that can work as iron-ajax is a very good move


Lack of abs(number) xpath function

Hi.
I'm using pugi xml for a personal project (at least in this stage), which uses intensively xpath.
I have to say that performance of pugi's xpath implementation is more than satisfactory in general, but just today I've realized that fn:abs(number) is not implemented.  It doesn't seems to be a complex function to implement, and is part of the standard functions of xpath defined by W3Schools' specification.
Is it possible to add as a new feature?
Best regards.

i think he only implements xpath 1.0 specs. abs() is from XPath 2.0 (XSLT 2.0) specs.
it's easy but maybe he doesn't want to commit to implementing the whole 2.0 specs

Yeah this is intentional because abs isn't part of XPath 2.0. pugixml only implements XPath 1.0.
XPath 2.0 is way larger than XPath 1.0 and way more complex; implementing it is completely outside of the scope. While it's of course possible to add selected functions from 2.0 to pugixml, it's unclear how you draw the boundary between the subset that's reasonable vs full implementation.


reproduce wasmtime-hs segfault

Steps to reproduce:

Store this minimal WASM to a file:

$ echo -e "00000000  00 61 73 6d 01 00 00 00                           |.asm....|\n00000008" | xxd -r > e.wasm
$ hexdump -C e.wasm
00000000  00 61 73 6d 01 00 00 00                           |.asm....|
00000008

create an empty counter.did file
$ touch counter.did

and create a dfx.json file with the following content:
$ cat dfx.json
{
  "canisters": {
    "counter": {
      "type": "custom",
      "build": "",
      "wasm": "",
      "candid": "counter.did"
    }
  }
}

Run ic-ref:

$ cabal run ic-ref -- --listen-port 8080
Starting ic-ref...
Running at http://<IP_ADDRESS>:8080/

Create and install a canister with the WASM from step 1:

$ rm -rf .dfx && dfx canister create counter --no-wallet && dfx canister install counter --wasm e.wasm

then we see the following output from ic-ref:
done
Segmentation fault (core dumped)

Note: I don't observe a crash when running with valgrind. I still observe a crash when running with gdb though.

The issue has been fixed in wasmtime-hs: https://github.com/dfinity/wasmtime-hs/pull/31


Update pulsar kerberos document

Currently, kerberos document is lack of some description when using client to connect brokers with kerberos.

@jiazhai  PTAL

run java8 tests

run java8 tests

run java8 tests


JSON field behaving erratically, making it almost impossible to enter data properly

Link to reproduction
No response
Describe the Bug
I have a simple JSON field. I'm trying to enter an array of strings. It's almost impossible to do.
Entering [ shows [] for a split-second and then vanishes again. Trying to enter a new line within a {} doesn't work either, it reset to {} on the same line.
When I first create {["something"]} and then delete the {}, I do get an array. But then adding new entries is again almost impossible, with the cursor jumping to the end of the field every time I enter a character.
I also tried setting editorOptions: { formatOnPaste: false, formatOnType: false } but it didn't change anything.

To Reproduce
Create a JSON field. Try to enter an array.
Payload Version
2.14.2
Adapters and Plugins
db-mongodb, bundler-webpack

Thanks for reporting! @kendelljoseph something probably changed with our recent work on the json field. let’s look into it

I have been experiencing the same issues.
I opened a similar (or nearly identical) issue last fall (2023) that was resolved in a subsequent release. However, I noticed the same issue happening again a week or so ago (around May 30, 2024). Our application is currently on v2.18.3

Here is a link to the now closed issue I opened last December v2.3.0:
https://github.com/payloadcms/payload/issues/4403

Hi @wrobson-lllow .  I'm working on this today.
It looks like an problem with the way the Monaco editor is refreshing when data is updated.  I'll try to replicate the behavior now.

I was able to replicate the behavior locally.

This field should accept a valid JSON object, array, or literal, however the field visual data is indeed fluttering if you try using an array from a empty state.
~
I'm still investigating...

Using a default value populates the field properly. However the problem returns once the default value is deleted.

~
I'm looking for any interference between using jsonSchemas and this issue as well.
~
... still working on this.

I see that this issue is still open, so this Pull Request maybe unrelated. I just updated to v2.22.0 and I am still having an issue. Thanks for taking a look at this @kendelljoseph  it is much appreciated!
In our application we give all JSON fields a default value of an empty object.
https://github.com/payloadcms/payload/assets/132296416/984857e3-c3a3-47b8-a8c4-3ca91ee0f00b

I'm having this issue with v2.26.0 . One workaround that worked for me is to use a code field with language: "json". It's not strictly the same since, if I understand correctly, this won't save the json but just a string, but at least makes it usable.

@sideral Am I correct that a code field saves a normal string to the DB not a JSON string? If that's the case I don't think that will work for my use case.


Use v4 UUID instead of v1 for query cacheKey

I'm a co-author of the uuid npm module which is being used in some places within the code base of sqlpad.
I'm currently trying to understand real-world use cases of time-based UUIDs ("v1 UUIDs").
I found one occurrence where v1 UUIDs were being used instead of v4 UUIDs and I was wondering if this case really requires the semantically much more complex v1 UUIDs or whether we could live equally well with purely random v4 UUIDs?
I'd be really curious to understand the motivation for choosing v1 over v4 UUIDs in the first place, so any feedback on this would be highly appreciated!
@rickbergfalk, as far as I can tell from the git history you introduced this in https://github.com/rickbergfalk/sqlpad/commit/84aae812da4bfdcb30e5ce7103f2f8b9a2afd1af, I'd be really curious about your feedback!

Hi @ctavan! Thanks for your work on the uuid module and this PR!
The use here definitely doesn't require v1 UUIDs. v4 would be just fine.
As for my original motivation for using v1 here -- that was out of habit to be honest. I used to use a lot of SQL Server where clustered primary keys are sometimes recommended to use v1 UUIDs instead of v4 in order to better fill data pages (https://docs.microsoft.com/en-us/sql/t-sql/functions/newsequentialid-transact-sql?view=sql-server-2017)
So for a while my thought process was... do I need the pure randomness or is a time sequenced value okay? ... with v1 being a default out of consideration for the db index 😅
Since the embedded db SQLPad uses doesn't benefit from v1 at all, and it is generated in the browser and not on the db itself, v4 is the more appropriate UUID to use here.

As for my original motivation for using v1 here -- that was out of habit to be honest. I used to use a lot of SQL Server where clustered primary keys are sometimes recommended to use v1 UUIDs instead of v4 in order to better fill data pages (https://docs.microsoft.com/en-us/sql/t-sql/functions/newsequentialid-transact-sql?view=sql-server-2017)

This is very valuable and insightful feedback, thank you!
And also thanks a lot for the quick response and for accepting my patch, greatly appreciated!