[Feature request] Pairs

Would you accept Pair<K,V>, similar to https://docs.oracle.com/javase/9/docs/api/javafx/util/Pair.html and https://developer.android.com/reference/android/util/Pair in your codebase ? I use Pairs in my product and would be interested in PairIntInt and PairIntObject<V>.
If you'd accept such a PR, I would try to create it if I understand the language

So, are you sure there are not pairs of primitives around, say in Guava or similar libraries? It seems weird nobody ever tried to implement this. But yes, they could be a nice addition.

No, I've not checked all other libraries. A quick search online does not allow me to find anything, and I would suspect that it would be a nice place to put it, given that you already have tools to generate all types. I must admit that I do not know Guava at all, and I didn't check thoroughly through it.
I should not that even finding your library was complex, so i would not be surprised that I missed such a simple set of classes

@Arthur-Milchior @vigna why add a duplicated tuple class? I mean fastutil has already tuples.
new AbstractInt2DoubleMap.BasicEntry(0, 0.2D);
Int2DoubleMap.createTuple(int key, double value) might be a good addition though.

BasicEntry is neither immutable nor final.

Yeah, but that isn't a bad thing.
There is plenty of cases (especially if you want to have good garbage collection levels) where mutable is nessesary.
It's not a replacement, but I honestly don't know a reason outside of constants where to use Immutable Pairs.

Uh ... mutability makes GC (in case there are references) and general optimization by the compiler much more difficult.

No? I mean fastiterators in Maps are the best example. They recycle variables because of mutability.
Immutability can help if you implement caches but you can only cache so much, while with mutability you can pool actions.
A good example is my 3d vector for my UI library that is as a helper there to reduce object allocation for data generation for my UI.
I would generate thousands of pairs or triples if it was all immutable.
Yes Immutable has its place but no its not a ALL way solution. Same with Mutable. But Mutable a much broader range in that regard.

No? I mean fastiterators in Maps are the best example. They recycle variables because of mutability.

Even there it isn't clear that the compiler doesn't inline a lot of things. Admittedly, it probably is faster since Java wasn't built with ideas of immutability in mind (however, they are slowly catching up, the upcoming record types will improve a lot in that regard, with those I'd bet that immutable types are faster; see basically any functional language for how fast immutable paradigms can be) and primitive assignments are so simple (it is different with references, as I said). Don't forget that (re)assigning variables also has its cost (Java is quite eager with fencing if I recall correctly).

I would generate thousands of pairs or triples if it was all immutable.

So what? You can generate billions of objects per second. And, again, you don't know how the compiler reuses memory. And you are not forced to use immutable Pairs in such a specific context.

Yes Immutable has its place but no its not a ALL way solution.

It is for practically all applications of Pair. If you want mutable tuples you have BasicEntry anyway, right?

Edit: And why i would generate thousands of Immutable objects? I am doing matrix calculations on the CPU because otherwise i would have transfer a lot more data over for what i want out of my toolset. (9 floats to 25 floats per vertex)

So many questions. Why TF do you not use a dedicated library for that? And why do you not use primitive arrays? Why do you think that you need to transfer less memory when you use mutable types? Don't assume, measure. Compilers are one of the most complex black magic pieces of software. But again, this has nothing to to with the Pair discussion (I hope).
In any case, design wise, immutability is such a big plus that I'd gladly incur that performance penalty (if there even is one for most users).

So many questions. Why TF do you not use a dedicated library for that?

2 reasons: 1: No library in java supported what i wanted, and there aren't as many as you think and don't have heavy dependencies attached. 2: I wanted to learn how it works in the backend and write myself one.

Why do you think that you need to transfer less memory when you use mutable types?

You missunderstood me there. Less Memory allocation has to be done because of reusing a mutable variable. (Vec3f transferResult), which i would have to allocate every single time, for each vertex that is generated.

There are a ton of matrix-vector libraries, even with BLAS/CUDA backends. But, besides the point. Any, as word of advice, don't try to reinvent the wheel. Linear algebra is extremely difficult to do right.

You missunderstood me there. Less Memory allocation has to be done because of reusing a mutable variable. (Vec3f transferResult), which i would have to allocate every single time, for each vertex that is generated.

Well, again, you are assuming that everytime a new allocation is happening. I guess that Java does this, but I am not sure. Functional languages don't do that, memory is reused. It might well be that newer JVMs figure such things out, too.
In any case, this whole discussion is not contributing to the issue, so lets focus back on the topic, pairs.
I think that a good design choice probably is a) having an interface
XYPair {
 X left();
 Y right();

  static XYPair of(X left, Y right) {
    return new ImmutableXYPair(left, right);
  }

with the implementation
ImmutableXYPair implements XYPair {
  private final X left;
  private final Y right;
  // constructor + getters
}

and let BasicEntry implement XYPair.

@Arthur-Milchior @vigna why add a duplicated tuple class? I mean fastutil has already tuples.
new AbstractInt2DoubleMap.BasicEntry(0, 0.2D);

You're technically correct. This contains everything I need. However, as far as type system goes, it would be pretty strange to state that a function which returns two elements returns a basic entry of a map. Let's say those two elements are X and Y coordinate of a click, it has semantically nothing to do with map, and I would not want that someone joining the project tries to figure out which map this is coming from.
However, if we want to avoid code duplication, it may make sens to state that
AbstractInt2DoubleMap.BasicEntry is another name for ImmutableIntDoublePair (or whatever names you decide to choose).

Int2DoubleMap.createTuple(int key, double value) might be a good addition though.

Android's Pair library has a function to create pairs. I should admit that I do not know why createPair would be better than new Pair

Android's Pair library has a function to create pairs. I should admit that I do not know why createPair would be better than new Pair

Factory methods hide the implementation (you can return Pair instead of PairImplementation) and you can perform additional validation (however not applicable here). In particular, it solves the "naming" issue. If you let BasicEntry implement pair, you can simply return BasicEntry without anyone wondering :-)

AbstractInt2DoubleMap.BasicEntry is another name for ImmutableIntDoublePair (or whatever names you decide to choose).

BasicEntry would be mutable though, but jeah.

Edit: I know that it's sometime considered bad practice to return a pair instead of creating a small class specilazed for the return value of the function. I don't want to do it each time I return two values, it does not really seems worth the extra code added. But I might understand you don't want to encourage what is sometime considered to be a bad behaviour

Having generic Pairs is perfectly fine (especially in this particular type-specific case). Pairs / Tuples are such elementary concepts.

Well, again, you are assuming that everytime a new allocation is happening. I guess that Java does this, but I am not sure. Functional languages don't do that, memory is reused. It might well be that newer JVMs figure such things out, too.

Everything that starts with "new" allocates memory. If it is now stack memory or heap memory it is not know. The compiler decides that but it is sure that it will allocate memory. the java compiler actually does not change much code, it might resort it if there is not dependencies or synchronization but from what i have experienced especially with mc going immutable on their coordinate system is that garbage collection rises. In MCs example it was i think multiple gigabytes of allocation in a mass worldgneration test that took up 50% of all allocations in Java and that was 60 second JavaMission Control allocation recording.

Everything that starts with "new" allocates memory.

No, the GC does not immediately free unused memory back to the system, similarly new does not result in a memory allocation call. See for example here or here for some additional myths.
Also isn't minecraft running on some ancient Java version (8 oder something?). A lot has happened since then.

No, the GC does not immediately free unused memory back to the system, similarly new does not result in a memory allocation call. See for example here or here for some additional myths.
Also isn't minecraft running on some ancient Java version (8 oder something?). A lot has happened since then.

Its running on java8 yes. I haven't tried newer versions yet. And fastutil is also still running on java8 (actually 7 if we look at all the holes that we see) So my comparison is to that.
I doubt that java deallocates/recycles memory that efficently especially since in the example i saw it was creating gigabytes per "couple (2-3)"seconds. I am not sure how likely that is.
I know if you write code memory efficient it does not actually allocate memory while still recycling memory but in such a scale, don't trust the JVM/Compiler. (In my engine it say it is allocting 200kb a second yet the memory usage is not going up so it is recycing data quickly enough.
I know it is a selfinvalidating statement, but in my case i am pooling a lot in my engine and  try to avoid allocations or optimize them a lot, and in mc it is not doing that at all.
So this is where: Immutable vs mutable comes into play. I have made a lot of bad experience with Immutable.

Its running on java8 yes. I haven't tried newer versions yet. And fastutil is also still running on java8 (actually 7 if we look at all the holes that we see) So my comparison is to that.

Compile time version != runtime time version. Java uses JIT compilation.

I doubt that java deallocates/recycles memory that efficently especially since in the example i saw it was creating gigabytes per "couple (2-3)"seconds. I am not sure how likely that is.

That seems very slow. Object allocation is, for all intents and purposes, one of the fastest operations you can even have on JVM. It is even faster than on C++ for a lot of applications, since less system calls are needed. Also, let me quote:

Current GC algorithms are actually optimized for creating many many small objects that are short lived, that is basically the 99% heuristic for Java objects in every program. Attempts at Object Pooling will actually make the JVM perform worse in most cases.

By pooling, you are just throwing obstacles into the way of the GC. It only makes sense to pool if you throroughly tested your code and are sure that pooling helps (and not some other side effect).
Anyway, we are deviating, again. I think Pair and maybe MutablePair interfaces would be nice, with appropriate implementations + factory methods (see my above proposal).

@Speiger, at this point you are generating just an infinite amount of noise. Nobody is interested in optimizing everything for MC running on Java 8. I suggest that you restrict your comments to correct technical statements that have general validity. Otherwise, I'll have to block you. You are seriously impeding a sensible discussion on a number of issues.

Compile time version != runtime time version. Java uses JIT compilation.

Yes but the JIT runs in the JVM.

That seems very slow. Object allocation is, for all intents and purposes, one of the fastest operations you can even have on JVM. It is even faster than on C++ for a lot of applications, since less system calls are needed. Also, let me quote:

If you would see the code you would puke. It is not efficient at all.

By pooling, you are just throwing obstacles into the way of the GC. It only makes sense to pool if you throroughly tested your code and are sure that pooling helps (and not some other side effect).

Not really. When i started pooling objects GC actually got faster since references were still present and objects could be faster evaluated for removal. This is what I have experienced.

Anyway, we are deviating, again. I think Pair and maybe MutablePair interfaces would be nice, with appropriate implementations + factory methods (see my above proposal).

+1

@Speiger, at this point you are generating just an infinite amount of noise. Nobody is interested in optimizing everything for MC running on Java 8. I suggest that you restrict your comments to correct technical statements that have general validity. Otherwise, I'll have to block you. You are seriously impeding a sensible discussion on a number of issues.

Never said that I intendet optimizing mc, I am using that as a example why Immutability can cause issues if you are not careful.
Well incaseoftrouble i assume that we have now to stop our disscussions that I actually found very interesting.
last comment from my side then.

If you can (and you should, IMO) use Java Records, started at JDK14 and continued in JDK15 as a preview feature, it will be implement as a feature in JDK16.
With records, declaring a primitive ordered pair is achieved with one line:
public record OrderedIntPair(int left, int right) { }
also:
public record OrderedIntObjPair<T>(int left, T right) { }

I know,  but we're at java 8 here ;).

Just to add on that: Project Valhalla / JEP 218 (once implemented) will probably make this whole library obsolete. But its a long road till then.
For records to be used we'd at least have to wait until they are part of the current standard language release (not a preview feature)

Just to add on that: Project Valhalla / JEP 218 (once implemented) will probably make this whole library obsolete. But its a long road till then.

Mmmh. Not sure:

Non-Goals
It is not a goal of this effort to produce fully reified generics.

s/whole/a lot/g :-) We don't know what exactly will be implemented there, anyway. But I'd guess that if records and primitives can be used as generics, the reference collection implementations will fully support them.

I was a bit under pressure as I need this elsewhere, so I tried and the result is in c9c12c3cbf7.
We now have a pair interface, mutable pairs and immutable pairs. Entries from maps have been retrofitted to implement the suitable pair interface. Accessors obey three type of language: left/right (preferred), first/second, key/value. You just have to implement left/right. There are .of() methods in interfaces and in implementations. A completely generic Pair interface lives in it.unimi.dsi.fastutil.
I'd love to get comments on the design and the implementation!

I have also added in 026a1f173b5 a facility for sorted pairs. That can be useful if you have to represent unordered pairs by canonicalization. I need it for some WebGraph adapters I'm writing for JGraphT.

A random thought while reading: Maybe define a macro for the
	/** {@inheritDoc}
	 * @deprecated Please use the corresponding type-specific method instead.
	 */
	@SuppressWarnings("deprecation")
	@Deprecated
	@Override

block. Its the same everywhere :-) Otherwise fine; I guess that it makes sense to have "mutable by default" interfaces and specialize to immutable. But then the Pair should not default the setters I think, rather ImmutablePair should add these defaults. Objects implementing Pair and not ImmutablePair should be mutable or explicitly decide that they are not I think.

Thank you very much for taking care of it.
I must admit I'm slightly confused by the hierarchy.
drv/ImmutablePair.drv implements either drv/Pair.drv or src/it/unimi/dsi/fastutil/Pair.java and drv/Pair.drv implements src/it/unimi/dsi/fastutil/Pair.java
This means that ImmutablePairs have a setter method, which throws an exception.
I totally understand why you want inheritance. It's important for code that only use the getter to have a way to accept both mutable and imutable pairs. I also understand that using Pair.java allows to have once and for all the definition of first, and key using the definition of left
What I don't understand is why are the setters in Pair.
Is there a problem with creating an interface src/it/unimi/dsi/fastutil/MutablePair.java which inherits src/it/unimi/dsi/fastutil/Pair.java and contains all the setters, removing the setters from src/it/unimi/dsi/fastutil/Pair.java. Then
drv/MutablePair.drv could extends both drv/Pair.drv and src/it/unimi/dsi/fastutil/MutablePair.java. it seems that Java handle the diamond problem correctly and that this inheritance scheme would not create any trouble.
I strongly suspect that doing so will avoid a lot of bugs, due to people seeing that a setter function is available thanks to autocomplete and not checking whether it is actually implemented.
If we want the name "ImmutablePair", then we can always create an ImmutablePair that inherits from Pair and adds no other feature.

@Arthur-Milchior I agree with your thoughts, just for context: One point for the current design is that Java's Collection API works the same way. You have mutators on the base interface and throw exception if immutable.  A "solution" is given by explicitly returning "ImmutablePair" (as Guava recommends for ImmutableXY). I am not sure if consistency is worth the design flaw here though, too.

Well, Guava does the same with immutable classes (maybe for compatibility with the JDK?).
Can someone give an example of widely-used collection framework not using the same strategy? I perfectly understand the issue with the completion, but having a single Pair interface has the enormous advantage that we do not have to specify whether we return immutable or mutable pairs. In fact, the idea that a MutablePair inherits from ImmutablePair is, from the viewpoint of design-by-contract, quite crazy: you have an interface that specifies immutability, and then a derived interface contradicts the same idea...

BTW, nothing is set in stone!

BTW, we can deprecate the mutation methods in the immutable implementations. That should help with the completion.

In fact, the idea that a MutablePair inherits from ImmutablePair is, from the viewpoint of design-by-contract, quite crazy: you have an interface that specifies immutability, and then a derived interface contradicts the same idea...

I think the idea would be that you have a Pair base class which defines getters, an ImmutablePair extends Pair which signifies complete immutability, and MutablePair extends Pair which defines setters. Stuff like entries where one is mutable and one is not extend Pair and add setters on their own (which makes them unusable for methods taking MutablePair though ...). I guess the last thing is the main point of having setters on the base methods. Even if you have a pair which does not fully support mutation, you can still throw it at something mutating only, e.g., the left part of the pair. Its a bit like duck-typing / python style in that sense.

BTW, we can deprecate the mutation methods in the immutable implementations. That should help with the completion.

I think the main point is that the Pair type will often be used blindly as also supporting mutation. If you know you are getting an ImmutablePair its probably more obvious (but deprecation certainly helps!)

Another issue that makes me unconfortable is that this adds ~200 classes. No joke. There are Mutable/Immutable interfaces for Pairs and SortedPairs.

I must say that whenever I look immutability is always an implementation, not an interface.

I must say that whenever I look immutability is always an implementation issue, not an interface issue.

Jeah, its the Java way™ of handling immutability I fell like.

Another issue that makes me uncomfortable is that this adds ~200 classes. No joke. There are Mutable/Immutable interfaces for Pairs and SortedPairs.

Well you could in theory have a generic Immutable marker interface (yes marker interfaces are bad ...) that is applied to all immutable types and only have Pair base class + MutablePair extends Pair. But its still a lot of classes and by now Java people probably are just to this "mutability is optional" approach.

Also, I've programming in Java for >20y at this point and never, once if my life I got an exception because I tried to mutate something I shouldn't have. I cannot really imagine the situation in which that could happen.

Also, I've been programming in Java for >20y at this point and never once if my life I got an exception because I tried to mutate something I shouldn't have. I cannot really imagine the situation in which that could happen.

I've spent 6 years teaching java to first and second year student in my past jobs. I guess this lead to very different feelings about what can be expected from people programming.
It's also quite possible that I just don't know java enough to know what's done usually in java. I've worked with a single java project outside of university. So if you believe this approach is the more standard one as far as java goes, that's fine. I still stand by my remark, but I've no more argument in favour of it, and if it seems a bad idea to more experienced devs', I won't insist.
Thanks @incaseoftrouble for explaining my idea better than I did. Even if we don't go this way, I'm happy to know that the exact idea I tried to give was considered.
Concerning the number of class, happily, in AnkiDroid (where I expect to use it) removal of all unused classes is automated when we generate the app, so that would not be a problem for me. But I totally understand that it is not an universal rule.

Well, I taught C for 12 years and Java for 10 years both at first-year and graduate students, so I had my share of experience, believe me.
In any case, I should let you see my Eclipse trying to recompile 2400 classes 😂. It is not a matter of jar size—it's just a matter of maneuverability.
So, for a Java programmer used to the structure of the JDK, there's an interface with optional mutable methods, and then there are mutable and immutable implementations. It is important, in designing software, to adhere to the principle of minimum surprise—a completely different design might surprise someone experienced, and fastutil is definitely not for the faint-at-heart.

Speaking of least surprise: I was surprised to see setters without set prefix.
Thanks for taking the time to add pairs and considering my remarks.

Yeah, that's a vexata quaestio since ever. Even in the JDK depending on the programmer they used a different standard. For example, FileChannel.position(long)/FileChannel.position() use the "argument/no argument" standard, whereas Map.Entry uses the "get/set" standard. Personally I've always preferred the "argument/no argument" way and I'm happy when I find it.


Changed device name ARM -> CPU

@ilya-lavrenov should we change lib name to libcpuPlugin.so?

@ilya-lavrenov should we change lib name to libcpuPlugin.so?

I think this is not needed.

Remove from /home/apankrat/projects/openvino/inference-engine/plugins/arm/include/arm_plugin/arm_config.hpp:
DECLARE_ARM_CONFIG_KEY(THROUGHPUT_STREAMS);

smoke_BehaviorTests/CoreThreadingTests.smoke_GetMetric failed on master branch


Azure Load Balancer

Please note that the part concerning adding a Health Probe is missing. Load Balancing rules can not be setup without it.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ed6b8cfa-ab60-a8e1-aa44-4f60b018492d
Version Independent ID: f7e6a613-058e-3aca-1d0a-40c2421ee800
Content: Add an RD Connection Broker server to configure high availability in RDS
Content Source: WindowsServerDocs/remote/remote-desktop-services/rds-connection-broker-cluster.md
Service: unspecified
GitHub Login: @lizap
Microsoft Alias: elizapo

Hi - we do add a probe to see what servers are active. Are you suggesting we need another probe? Thanks, Liza

Closing due to inactivity.  Please reopen to pursue further.  Thanks!


when I use it for chatglm ，error like this

Traceback (most recent call last):
File "main_step1.py", line 310, in
main()
File "main_step1.py", line 306, in main
train(args)
File "main_step1.py", line 164, in train
model = get_peft_model(model, peft_config)
File "/app/anaconda3/lib/python3.8/site-packages/peft/mapping.py", line 142, in get_peft_model
peft_config = _prepare_lora_config(peft_config, model_config)
File "/app/anaconda3/lib/python3.8/site-packages/peft/mapping.py", line 117, in _prepare_lora_config
raise ValueError("Please specify target_modules in peft_config")
ValueError: Please specify target_modules in peft_config

Hello, please refer issue #221  or https://github.com/27182812/ChatGLM-LLaMA-chinese-insturct/tree/main. They have got PEFT working with ChatGLM! Please let us know if this resolves the issue.

set target_modules=["query_key_value"]


add win32 WM_SIZING event

There seems to be another Windows event to get continuous resizing. WM_SIZE alone seems to have a "debounce" behavior, that produces this issue if we redraw every resize:

Using only WM_SIZING produces the inverse, where it will redraw occasionally while dragging but not once you stop, leaving the same black border issue.
Combining the two seems to solve the problem, but do you want to provide a similar behavior as Windows where there's a final "resized" event and a separate continuous "resizing" event? I find the Windows method kind of strange personally, at least for drag based resizing, because they're both essentially continuous and it just depends on how quickly you're dragging the window.

For reference: https://learn.microsoft.com/en-us/windows/win32/winmsg/wm-sizing

There's something else going on here actually. WM_SIZING and WM_SIZE actually seem to be duplicate events, so it was something about calling the frame draw twice that was fixing my problem and not the handling of missed resize events.

Some follow up on this. WM_SIZE works okay calling onFrame, but I had to throttle that actual drawing to 14ms because the onResize events fire much faster than the frame rate (1-2ms) which was causing a lot of glitching. There's still a small issue in that the throttling causes the final WM_SIZE event to not draw, and the regular onFrame won't fire while the mouse button is held down, which results in a black border until the mouse button is released. I guess that can be avoided by maintaining some other frame pacing to force draws if the onFrame never fires.


Windows 7 installation issue

Hi, i am trying to install this app in Windows, but getting the following error :
D:\projects>go get github.com/yudai/gotty
# github.com/kr/pty
D:\go_workspace\src\github.com\kr\pty\ioctl.go:6: undefined: syscall.SYS_IOCTL
D:\go_workspace\src\github.com\kr\pty\ioctl.go:6: not enough arguments in call to syscall.Syscall
D:\go_workspace\src\github.com\kr\pty\run.go:21: unknown syscall.SysProcAttr field 'Setctty' in struct literal
D:\go_workspace\src\github.com\kr\pty\run.go:21: unknown syscall.SysProcAttr field 'Setsid' in struct literal
D:\go_workspace\src\github.com\kr\pty\util.go:26: undefined: syscall.SYS_IOCTL
D:\go_workspace\src\github.com\kr\pty\util.go:28: undefined: syscall.TIOCGWINSZ
D:\go_workspace\src\github.com\kr\pty\util.go:30: not enough arguments in call to syscall.Syscall

Is this a Windows specific issue?

Yes, currently Windows is not supported.

ok!

Unfortunately, it looks very hard to support Windows for now.  Closing.

ok, just mention it in the Readme

Why not mention it in Readme?? :disappointed:

Added a note in the development section of README.

@yudai do you think this project could help enabling Windows support for gotty?
Thanks.


Stock tickers not persisting

I'm starting to lose some quotes, as the ticker name does not persist.

I uninstalled and then installed again and it is back to working .... go figure


Remove version: Jackett.Jackett version 0.20.2593

404 - release deleted
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/Jackett/Jackett/releases/download/v0.20.2593/Jackett.Installer.Windows.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 918)


SpatialTransformer - DataTracker functionality

The new Salesforce data has a new project number field. I is now called: BT_Legacy_Project_ID__c
Please ensure that this field is used to gather the project numbers when --load is 'datatracker'
Note on handling the data in this field - It is possible that BT_Legacy_Project_ID__c is blank in some cases.

Change has been made in the commit: https://github.com/cat-cfs/twobilliontoolkit/commit/d7885c5293978dfb8c2064bc85227a701e3141fa
Changes:

The field that is used to gather the project numbers from the master data sheet when --load is set to 'datatracker' is now BT_Legacy_Project_ID__c instead of Project Number

@nesdolya I could not find a suitable master datasheet to test this on, so before we close this I would like to test even though the change is simple.

You could use this for testing: V:\2BT\04_Program_year\2023_planting\DataAnalyses\2BT_2021_2023_PlantingActivityData_MASTER_v103.xlsx


Support CacheAdapters

Summary of the change: Support CacheAdapters
Laravel recommends using CachedAdapter from league/flysystem-cached-adapter to cache driver meta data.
Prior to this commit it wasn't supported by this package and throws the following exception when attempting to upload a file:
Call to undefined method League\Flysystem\Cached\CachedAdapter::getPathPrefix() {"exception":"[object] (Symfony\\Component\\Debug\\Exception\\FatalThrowableError(code: 0): Call to undefined method League\\Flysystem\\Cached\\CachedAdapter::getPathPrefix() at /SNIP/vendor/unisharp/laravel-filemanager/src/LfmStorageRepository.php:29)
[stacktrace]
#0 /SNIP/vendor/unisharp/laravel-filemanager/src/LfmPath.php(77): UniSharp\\LaravelFilemanager\\LfmStorageRepository->rootPath()
#1 /SNIP/vendor/unisharp/laravel-filemanager/src/LfmPath.php(222): UniSharp\\LaravelFilemanager\\LfmPath->path('absolute')
#2 /SNIP/vendor/unisharp/laravel-filemanager/src/Controllers/UploadController.php(34): UniSharp\\LaravelFilemanager\\LfmPath->upload(Object(Illuminate\\Http\\UploadedFile))
#3 [internal function]: UniSharp\\LaravelFilemanager\\Controllers\\UploadController->upload()

The fix in this PR is similar to as used in laravel/framework at https://github.com/laravel/framework/blob/5.7/src/Illuminate/Filesystem/FilesystemAdapter.php#L376-L397 in that it unwraps the CachedAdapter.
It has made a significant improvement to my file listing performance with S3.

We notice there is some serious performance issue with S3 when integrating laravel file system, just cannot figure why. So glad to receive this PR! Thanks @mnightingale !


Split RenderingDevice into API-agnostic and RenderingDeviceDriver parts

A huge chunk of RenderingDeviceVulkan is API-agnostic bookkeeping, validation, etc. Such code is also shared with others, like RenderingDeviceD3D12. This PR keeps a single, non-abstract RenderingDevice with all those API-agnostic elements and moves all the API-specific parts to a thin wrapper around the driver, which is RenderingDeviceDriver for Vulkan. This makes much easier to write additional rendering drivers, by allowing one to focus on the API-specific stuff. In some cases that will require modifying the interface between RD and RDD, which is not exposed to the user.
Other relevant points:

A common context interface has been extracted from VulkanContext (APIContextRD) with the bare minimum set of services RenderingDevice and direct usage needs. (The specific RenderingDeviceDrivers have access to the specific context implementation, which open doors for them to a much richer, but non-standard, API.)
There are important notes on design choice in the code. Future maintainers, please follow or discuss, but don't break consistency.
Every struct involved now have default initializers, both in RenderingDevice and RenderingDeviceDriver.
Some of the wiring between context and RD has been enhanced so now you can instantiate a RenderingDevice even when Godot is running in compatibility mode (this will allow to have a RD-based lightmapper available in a GL compatibility project). This piece of code shows how to ensure you have a usable local RenderingDevice in any circumstances:RD *orig_rd_singleton = RD::get_singleton();

ApiContextRD *vc = nullptr;
if (GLOBAL_GET("rendering/renderer/rendering_method") == "gl_compatibility") {
#ifdef VULKAN_ENABLED
	if (GLOBAL_GET("rendering/rendering_device/driver") == "vulkan") {
		vc = memnew(VulkanContext);
 }
#endif
	CRASH_COND_MSG(!vc, "No RD driver available.");
	vc->initialize();
} else {
	vc = RD::get_singleton()->get_context();
}

RenderingDevice *rd = memnew(RenderingDevice);
if (GLOBAL_GET("rendering/renderer/rendering_method") == "gl_compatibility") {
	DEV_ASSERT(RD::get_singleton() == rd);
}
rd->initialize(vc, true);
// Do stuff with RD here.
memdelete(rd);

if (GLOBAL_GET("rendering/renderer/rendering_method") == "gl_compatibility") {
 	memdelete(vc);
}
DEV_ASSERT(RD::get_singleton() == orig_rd_singleton);

Deprecations: RD::texture_get_native_handle() (superfluous) and DRIVER_RESOURCE_VULKAN_* (replaced by API-agnostic).

~This includes a couple of merged (commits will go away once rebased) and unmerged (#82797) PRs.~
TODO:

[X] Finish debugging and fix a few known issues.
[X] Rebase on top of the lastest master.
[X] Fix ligthmapper (local device?).
[X] D3D12: Adapt to this new infrastucture. 2023-12-11: The PR now includes #70315, which is then adapted to the new model.
[ ] Debug.
[ ] Remove the interim change-tracking-assistance commit.
[ ] Ensure #86106 is fixed.

Looks fantastic!

I'll be leaving feedback in general here and on a per-line basis if necessary. Some of my comments are just criticizing a few design carry-overs from the current design and aren't necessarily related to this PR being approved or not, it just feels like the right time to bring these points up.
Type Safety
I already brought this up over DMs, but there's currently no type safety system for the IDs returned by RDD. Using the uint64_t's directly means we can mix and match arguments around and the compile won't indicate the error. We can solve this in a few ways.
A) Every type returned gets a struct that wraps around the uint64_t. A very minor change that guarantees the types can't be mixed together.
struct FramebufferID {
  uint64_t handle;
};

B) Every type returned is a pointer to an interface that the backend has to inherit. This requires more work but since these handles are already pointers (they're not references you can indefinitely hold) in disguise, there's not much of a downside to it. The major advantage is that debuggers can access these types natively and display their contents.
struct Framebuffer {
  // Empty interface.
};

virtual Framebuffer *framebuffer_create() = 0;
virtual void framebuffer_free(Framebuffer *p_framebuffer) = 0;

struct FramebufferVK : Framebuffer {
  VkFramebuffer fb = VK_NULL_HANDLE;
}

Another benefit to consider is const-correctness can be implemented into the pointer types and will therefore be directly indicated by the API whether the object's internals should be modifiable or not.
Vertex Formats
There seems to be currently a forced correlation between bindings and attributes, meaning that attributes that share the same buffer can't share the same binding (e.g. if 10 elements are bound to the same buffer, they each get their own binding instead of using the same one). I'm not familiar if this has potential performance implications, but it's worth investigating if this is correct.
This is not new to this PR, as it was pretty much directly copied as is from the current Vulkan RD.
Command Queues
Command queues are not directly exposed but managed by the backend, meaning RenderingDevice has no control whatsoever on where commands are executed, on what type of queue and when exactly. This could prove to be very useful in the feature as a lot of the logic around managing command queues should be common to most backends and it takes off some responsibility for the backend maintainer. I think we could address this in this PR and move most of the logic that Vulkan Context currently handles to RD instead.
I'll do a separate comment for concerns about D3D12 compatibility in the future.

These are not something that necessarily need to be solved by this PR but are probably worth considering before doing the D3D12 version of RDD. Some are really just points to consider when doing the D3D12 refactor.
Combined Image Samplers
As far as I know these are not supported by D3D12 and the backend would need to work around it in some ugly ways to support them. Are we relying on this feature a lot? What approach did the D3D12 PR take to solving this?
Compute dispatch base
I'm not aware D3D12 has anything about this. This is a new addition to the API by this PR. We might want to consider it more carefully. It's also a Vulkan 1.1 feature at minimum, so we need to query device support for it. I think we should just remove it and let the user handle it by adding the offset in a UBO/Push constant.
Barriers
D3D12 requires resource barriers if not using enhanced barriers. I think we should take the chance to make our barrier-API only take resources instead. I don't think we ever rely on memory barriers. As far as I'm aware the current D3D12 PR needs to work around this and I think it's hard to find use cases for full memory barriers with the upcoming acyclic render graph/barrier solver design.
Semantics vs Locations
What approach does the current D3D12 PR take for matching vertex location inputs to the semantics? Is that something that we'd need to address on this API or is it solved through some other means as part of the backend responsibility? If there's a possibility this API will take shaders in its native format, it sounds like we might need to allow for a type of configuration that allows input semantics.
Unordered Access State
What is the suggested solution for mapping this API to the unordered access state in D3D12? Using the image layout for general and the bit for shader writes?
Texel Buffers
These don't translate to a flag during creation in D3D12, so we'll just need to track that bit to enable using the format on the SRV/UAV when it's assigned.

You may want to check the D3D12 PR (#70315) to see how I mapped the RenderingDevice features to it.
Regarding barriers, the D3D12 driver using resource-level barriers would just ignore the detailed ones (that's exactly what the PR above is doing). If enhanced barriers are eventually implemented, well, it just have to obey them.
I'm addressing the type safey of ids in the next push.
Regarding other things, this is just the first iteration and will for sure need changes to fit other APIs or future RD features better.

Regarding type safety, I'm taking the first approach you discuss because most RDD ids aren't actually pointers to its view of resources, but just the Vulkan handles.

Re-pushed. Lightmapper is fixed (or so I think).

@sakrel, thank you for your feedback. I acknowledge that the new single RenderingDevice class in this PR is indeed tailored to the current supported, and about to be supported, Vulkan and D3D12. Therefore, it's indeed unsuitable for APIs following different models.
The best option I can think of to make WebGPU fit again is just making RenderingDevice more flexible, in collaboration with the driver. As such, RenderingDeviceDriver would have a new function to query about the model of the underlying API-platform. RenderingDevice would get extra logic to handle the different possibilities. RenderingDeviceDriver would, in turn, have to provide new functions (I think) for the pieces of functionality that need different mechanics (e.g., callback system for async), that would only be implemented for RenderingDeviceDriverWebGPU, and any other potentially needing similar characteristics in the future.
As I see it , the interface of RenderingDeviceDriver will be subject to change for a long time (potentially forever), to let it adapt to any needs that arise on the side of RenderingDevice, to support all its present and future features, as well as every graphics API.

After applying the fix I mentioned in my comment I tested this with https://github.com/BastiaanOlij/godot_openxr_demo and it now works for OpenXR.
Tested using the mobile renderer with:

Valve Index running in PCVR mode
Quest Pro running natively
Quest 3 running natively (surprising, works without the bypass implemented in #84244, did you do something to cause presentation to be skipped when there is nothing to present?)

Performance seems on-par with Godot 4.2 stable, not surprising as the mobile renderer likely has less to gain from these improvements.

Performance seems on-par with Godot 4.2 stable

Would this make a difference when multiple render passes is implemented? I feel like this would have a more impact once it comes to thatl

@RandomShaper sadly I found a more serious issue. Things go really bad when I enable MSAA.
On desktop it's rendering fine, but inside the Quest I'm getting all sorts of issues with polygons flickering. It very much looks like that it is either resolving the buffer before its finished rendering, or it may be an issue with a missing barrier before the results are handed off to the XR Compositor (seeing this happens outside of the control of the rendering device, it has likely no idea a barrier is needed).
Interestingly enough recording, while the issues remain visible in headset, are not present in the recording, which suggest Meta might introduce a barrier here.
I suspect that I didn't have trouble on PCVR simply because we finish rendering WAY before the XR compositor decides to present the image, I'm at 0.5ms render time so the GPU isn't even breaking a sweat.

Performance seems on-par with Godot 4.2 stable

Would this make a difference when multiple render passes is eventually implemented? I feel like this would have a more impact once it comes to that.

RDD isn't expected to have performance improvements on its own as far as I know since the barrier management is the same as what it was before.

Just an update from the production team. We're ready to merge this whenever it's ready, it's next in the line of big rendering changes. We would be okay merging it for dev1, which we plan to have early next week. But if you'd rather have it merged afterward dev1, that's also okay. The next public build will only be made in January, so that won't yield any broad user testing yet, but at least your burden with juggling branches will be easier.
With that said, this still needs a rebase from @RandomShaper to finalize the commits and remove the D3D12 prerequisites.
And this still needs some approvals and positive reviews to formally proclaim it as ready. So cc @clayjohn and @BastiaanOlij. And I'm ready when you all are. 🙃

I can attest I've been doing some very extensive testing of this branch as well as people who've tested the Acyclic Graph have been using this. I'll be doing a rebase when the final version of this PR is submitted to double-check everything still works as intended.

@YuriSizov, thanks for the heads-up. The problem is that,  now that D3D12 is merged, this PR is not complete until I'm done bug fixing the RDD incarnation of it. So, this can only be put in dev1 if I can do that in time. (Likely and desirable.)

@RandomShaper No pressure from us in that regard 🙂

@YuriSizov @RandomShaper my testing so far with this, it all looks pretty good.
My only issue I ran into was on Quest where we seem to have a sync or fence issue introduced. The Quest starts consuming the image before we finish rendering it. @DarioSamo and I were looking into that and I've still got some tests to complete.
However I think that shouldn't be a showstopper for this to go ahead esp as we're recommending everyone on Quest to use compatibility already. It's something we can fix up later.

Before jumping into code review. Just a few notes on the testing this branch has had (that I am aware of and apart from the testing that is mentioned above):

Used in https://github.com/godotengine/godot/pull/84976
Used by stuart carnie in developing the Metal backend
Used by The Forge as their base for Vulkan optimizations
Tested by me in multiple scenes of varying complexity
a. Test scene from https://www.youtube.com/watch?v=9kcjFJJxO6I&t=1484s (tons of assets, textures, materials, GI, heavy shaders)
b. TPS demo with different permutations of GI, fog, etc.
c. Simplfied TPS demo using mobile renderer, but with heavy use of animations, particles, and decals

At this point any problems that have been identified have been fixed and are included in the current state of things. In testing the current iteration I did not identify any issues (no graphical bugs, no crashes, no new errors, no new validation layer errors)

Because I saw @BastiaanOlij's comments about this not working on the Quest, I decided to try it myself, and, unfortunately, and this PR seems to completely break the Compatibility renderer. :-/ When I switch to the Vulkan Mobile renderer, it seems to work fine. So, kind of the reverse of what of what Bastiaan saw?
This is the error I'm getting with the Compatibility renderer (via adb logcat):
12-19 13:59:25.549  4418  4459 E godot   : USER ERROR: Parameter "native_window" is null.
12-19 13:59:25.549  4418  4459 E godot   :    at: DisplayServerAndroid (platform/android/display_server_android.cpp:572)
12-19 13:59:25.549  4418  4459 E godot   : USER ERROR: Unable to create DisplayServer, all display drivers failed.
12-19 13:59:25.549  4418  4459 E godot   :    at: setup2 (main/main.cpp:2466)
12-19 13:59:25.549  4418  4459 E godot   : USER ERROR: Condition "!_start_success" is true. Returning: false
12-19 13:59:25.549  4418  4459 E godot   :    at: start (main/main.cpp:2928)

And that last message is spammed over and over.
I suspect there's something slightly off in the Android changes?

I suspect there's something slightly off in the Android changes?

ANativeWindow *OS_Android::get_native_window() const {
#if defined(VULKAN_ENABLED)
	return native_window;
#else
	return nullptr;
#endif
}

So these lines should be inside Vulkan check.
		ANativeWindow *native_window = OS_Android::get_singleton()->get_native_window();
		ERR_FAIL_NULL(native_window);

I found a particular project that doesn't work with this branch in D3D12 mode, I'll see if I can get an MRP once I narrow down what the project does that is incompatible, but it seems it's straight up unable to create the graphics PSOs on this project.

Pushed removing many cases of runtime allocation (powered by PagedAllocator), plus a change to PagedAllocator to improve its compatibility (e.g., usable with HashMap), plus the fix discussed above.

I suspect there's something slightly off in the Android changes?

ANativeWindow *OS_Android::get_native_window() const {
#if defined(VULKAN_ENABLED)
	return native_window;
#else
	return nullptr;
#endif
}

So these lines should be inside Vulkan check.
		ANativeWindow *native_window = OS_Android::get_singleton()->get_native_window();
		ERR_FAIL_NULL(native_window);

I already fixed it locally, but hadn't pushed yet.

I've pushed with the requested fixes, plus a small change to use PagedAllocator for the multiple RDD-bookkept resources, instead of using plain memnew().
Regarding the SDFGI issue reported by @DarioSamo, it's a known issue the pre-RDD Direct3D 12 renderer already had. (See Deprecate Texture Aliasing in #70315.) It will be addressed separately.

News on the SDFGI thing: there was a silly bug initializing textures requiring format sharing. On Windows 11 you don't even need the Agility SDK for the feature to work without the hack, so I've pushed a fix to let it work. (The hacky version still works for me on W10 with no Agility SDK, on Nvidia. A proper fix for other GPUs will be provided in the future.)

If CI succeeds (it failed because of a C++ technicality in the just added VersatileResource helper for allocation avoidance).

Okay, so Android/Quest issues have been addressed, and if the CI passes we're ready to go. Correct? 🙂

Okay, so Android/Quest issues have been addressed, and if the CI passes we're ready to go. Correct? 🙂

I just reported an issue with the latest change so just a bit more.

Not the D3D12 one from above which is also present in master?

There's just an implementation error in the recent change that will cause the list to get filled incorrectly with the size of the array is > 1. Should be fixed easily.

It seems to be a GPU driver or OS version specific issue. Works as expected on some setups, fails on another. Maybe a driver bug. We are fine with merging as it is now. The code seems to be OK.

This is never ending, is it? 😅 Pushed with a fix for the memory leaks.

I think I'm going to force-push again...
Just kidding. 😈

Thanks! As often is the case with such achievements, this monumental effort is not the end, but a beginning for even more exciting new things to be added into the engine in near and far future. But tonight you can rest with a proud smile. Cheers to everyone who contributed and tested this PR!

Really happy to see this, just wanted to say thanks for the effort and can't wait for future improvements this will enable!

This does not compile on macOS, it fails on the linking stage:
[100%] Linking Program bin/godot.macos.editor.arm64 ...
Undefined symbols for architecture arm64:
  "_vkCmdDrawIndexedIndirectCount", referenced from:
      RenderingDeviceDriverVulkan::command_render_draw_indexed_indirect_count(RenderingDeviceDriver::CommandBufferID, RenderingDeviceDriver::BufferID, unsigned long long, RenderingDeviceDriver::BufferID, unsigned long long, unsigned int, unsigned int) in libdrivers.macos.editor.arm64.a(rendering_device_driver_vulkan.macos.editor.arm64.o)
  "_vkCmdDrawIndirectCount", referenced from:
      RenderingDeviceDriverVulkan::command_render_draw_indirect_count(RenderingDeviceDriver::CommandBufferID, RenderingDeviceDriver::BufferID, unsigned long long, RenderingDeviceDriver::BufferID, unsigned long long, unsigned int, unsigned int) in libdrivers.macos.editor.arm64.a(rendering_device_driver_vulkan.macos.editor.arm64.o)
ld: symbol(s) not found for architecture arm64

This does not compile on macOS, it fails on the linking stage (I tested both arm64 and x86_64):
[100%] Linking Program bin/godot.macos.editor.arm64 ...
Undefined symbols for architecture arm64:
  "_vkCmdDrawIndexedIndirectCount", referenced from:
      RenderingDeviceDriverVulkan::command_render_draw_indexed_indirect_count(RenderingDeviceDriver::CommandBufferID, RenderingDeviceDriver::BufferID, unsigned long long, RenderingDeviceDriver::BufferID, unsigned long long, unsigned int, unsigned int) in libdrivers.macos.editor.arm64.a(rendering_device_driver_vulkan.macos.editor.arm64.o)
  "_vkCmdDrawIndirectCount", referenced from:
      RenderingDeviceDriverVulkan::command_render_draw_indirect_count(RenderingDeviceDriver::CommandBufferID, RenderingDeviceDriver::BufferID, unsigned long long, RenderingDeviceDriver::BufferID, unsigned long long, unsigned int, unsigned int) in libdrivers.macos.editor.arm64.a(rendering_device_driver_vulkan.macos.editor.arm64.o)
ld: symbol(s) not found for architecture arm64

It compiles in the CI. Are you using any different parameters that would have an appreciable difference?
Molten VK appears to have support for both those functions, so it is odd that it's not linking

@clayjohn Fixed, the problem was that I was using an old version of MoltenVK. If anyone else is experiencing this issue, try updating the Vulkan SDK.


walks: re-title and update More Resources

Extracted from https://github.com/iterative/vscode-dvc/pull/1936#pullrequestreview-1019243063

Sanity checked this locally and it's good:

CI jobs tend to timeout. (re-running...)

Another timeout.
Not sure how to get this merged.


infercnvpy.tl.infercnv result matrix is of incorrect dimensionality

Report
The matrix added to my AnnData object by infercnvpy in anndata.obsm['X_cnv'] has an unexpected number of columns compared to the original input matrix (anndata.X); these values were 6671 and 6965 respectively.
I assumed that the matrix would be reduced to the number of windows over which CNV values were calculated. I used windowsize=100 and step=1, so I would have guessed I would have seen 6866 columns.
Is the anndata.obsm['X_cnv'] representing something other than the CNV scores for the windows? Also, is there a way to annotate the columns in this matrix with the corresponding chromosome?
Version information
No response

Hi @sean-at-tessera,
thanks for your questions.

The matrix added to my AnnData object by infercnvpy in anndata.obsm['X_cnv'] has an unexpected number of columns compared to the original input matrix (anndata.X); these values were 6671 and 6965 respectively.
I assumed that the matrix would be reduced to the number of windows over which CNV values were calculated. I used windowsize=100 and step=1, so I would have guessed I would have seen 6866 columns.

The running mean is computed using np.convolve with mode="same". That means for each chromosome the number of the columns in the CNV matrix is identical to the number of genes in that chromosome, independent of the window size (At the beginning and the end of the chromosome, the mean is computed on less genes than the window size).
The reason why you see the discrepancy is that X and Y chromosomes are excluded by default.

Also, is there a way to annotate the columns in this matrix with the corresponding chromosome?

There's a chr_pos field in adata.uns that gives you the starting index for each chromosome
>>> adata.uns['cnv']['chr_pos']
{'chr1': 0,
 'chr2': 508,
 'chr3': 894,
 'chr4': 1192,
 'chr5': 1434,
 'chr6': 1711,
 'chr7': 1999,
 'chr8': 2275,
 'chr9': 2501,
 'chr10': 2715,
 'chr11': 2932,
 'chr12': 3248,
 'chr13': 3534,
 'chr14': 3661,
 'chr15': 3857,
 'chr16': 4054,
 'chr17': 4292,
 'chr18': 4575,
 'chr19': 4690,
 'chr20': 4970,
 'chr21': 5106,
 'chr22': 5185}

Hope that helps!

Thank you @grst  for your clear and helpful answer!


Add go generate mention to CONTRIBUTING.md

a WIP for the improvement of CONTRIBUTING.md as detailed by #2579 , mentioning the existence of generated files and the fact that they should be added alongside the manual part of the commit.  We could probably add a few more things, but those will come along if suggestions do.

So, maybe should we put the information on generated files after line 58? moreso explaining the function of go generate when the concept is introduced. Or what ideas do you have regarding this?

@Parker77 - do you have any ideas about how these instructions in CONTRIBUTING.md can be improved?
For context, there was some confusion in #2577 and we are trying to minimize the friction that new contributors encounter when creating PRs for this repo.  Do you have any thoughts about this?

I guess in step 5 (maybe right after line 60), like @XaurDesu suggested, we could give a brief explanation of what’s happening when that command is run, along with the instruction that the resulting changes should be included in a PR, which could go something like:
The go generate ./... command will update or generate certain files, and the resulting changes should be included in your pull request.

Sorry for the absence guys, i've been a bit occupied in my personal life as of late, coming back to the topic at hand, i think that yes, i think @Parker77 's format seems like a good idea for them. I'll be adding that exact comment to README.md once i get home from work today, if you guys don't mind. Maybe i could convert this pull request into a draft until we have a specific plan for the whole thing.

Hey guys, do you think it could be a good idea to add a few links to the official Go documentation of the commands of interest in README.md? Also, i just realized that README.md mentions golint as a command that should be ran after fmt. but According to the golint repository,  golint is deprecated, and given that it reccomends running 'vet' instead of golint, and that is a command we use already, should we change that reference?

Adding links to official Go documentation in the README is fine by me.
And yes, updating the README from "golint" to "go vet" sounds like a good idea too.

Tried putting some comments in the style of @Parker77 's idea for them (including his description of go generate, actually) please check for any styling/conceptual problems in it, please.  Also updated the go lint mentions.

Just committed the changes requested.

Just commited both changes, it should be good now

Just corrected the dead link, i should probably have seen it before you honestly. Sorry for that.

Hmm, it's been a few days, are we sure @Parker77 is available for code review?

Thank you, @Parker77 !
Merging.


Update item in array

Hello everyone, how's it going?
I need to update an item in an array, how could I do?
The item would be active inside card
"gateway" : [
{
"iugu" : {
"default_payment_method_id" : "9360991C853C4FCFA90F28E45DDCE3BE",
"customer_id" : "030C8A08058A4CA7BD112D5A4F54C81C",
"card" : [
{
"brand" : "VISA",
"name" : "Anna Silva",
"number" : "XXXX-XXXX-XXXX-1881",
"month" : 1,
"year" : 20,
"active" : true
}
],
"subscription" : {
"id" : "C23E7E49880642A8A8BBC256C22C94AD",
"suspended" : false,
"value" : 2990,
"expires_at" : "2019-08-01",
"created_at" : "2019-07-25T16:12:27-03:00",
"updated_at" : "2019-07-25T16:12:27-03:00",
"customer_id" : "030C8A08058A4CA7BD112D5A4F54C81C",
"customer_name" : "Anna Silva",
"plan_name" : "Plano semanal",
"logs" : [
{
"id" : "803CD10FACA0427E8403850F84C1F9B8",
"description" : "Assinatura Criada",
"notes" : "Assinatura Criada ",
"created_at" : "2019-07-25T16:12:27-03:00"
}
]
},
"created_at" : "2019-07-25T16:12:25-03:00",
"updated_at" : "2019-07-25T16:12:25-03:00",
"active" : true
}
}
],

You can use push and pull
Or see example here


[BUG] Preview windows only support YUV420

Describe the bug
Trying to use libcamera-still with a usb camera. I understand that a preview can't be shown, but setting --nopreview doesn't help, and that error message is still displayed. Is this intended behavior?
Bug report
I don't know where to get camera-bug-report, it isn't part of a default lite os image. This is the output I'm getting from libcamera-still

pi@camera:~ $ libcamera-still --nopreview 1 -o test2.jpg
[0:29:13.224418338] [752]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
Stream configuration adjusted
[0:29:13.399755927] [752]  INFO Camera camera.cpp:937 configuring streams: (0) 640x360-MJPEG
terminate called after throwing an instance of 'std::runtime_error'
what():  Preview windows only support YUV420
Aborted

Additional context
I'm running libcamera-still as it is in apt repo:

pi@camera:~ $ libcamera-still --version
libcamera-apps build: 2a38ae93f143 21-10-2021 (14:49:17)
libcamera build: v0.0.0+3156-f4070274

Hi, it looks like your USB webcam is supplying an MJPEG stream, would that be correct? I'm afraid our libcamera implementation only supports embedded (raw Bayer) sensors, not webcams that produce fully processed images or MJPEG streams.
I agree that the error message that comes out seems to have got a bit confused, but I think the basic problem is that this sort of camera just isn't supported.
I think libcamera does have some code ("pipeline handlers" as they are known) to handle webcams, so it might be worth trying to compile that and see if you get any further. Alternatively you can presumably address your camera as a regular V4L2 sensor through /dev/video<n> (for some value of <n>).

Thanks @davidplowman. I take it then that it isn't just the preview being an issue, but that my camera doesn't even want to supply a raw image. It is a Microsoft Corp. LifeCam HD-3000.
Thank you,
Errol

It looks like you already have the libcamera pipeline handler for UVC there, as it identified the camera, and selected an MJPEG format. But the issue sounds like libcamera-still doesn't accept a camera that already produces a jpeg image stream.

So far as I know we only ask for the raspberrypi PH, but maybe something has happened to cause uvc to be dragged in. Nonetheless, handling non-raw cameras is well beyond the scope of what we're doing with these applications and unlikely to change for the foreseeable future - so I think we can probably close this one.


Final Windows fixes to complete compilation

The PR include different fixes:

Remove unneeded Qt support from GuiSystem
Visibility fixes
Disable TestSystem on Windows. Needs more inspection about an error with ign-plugin.

Nice! I see the build passing here:  :tada:

Nice! I see the build passing here:  :tada:

New test on Windows

New test on Windows

New test after fix visibility

New test after fix visibility


Pipenv can't find setup.py in dependency with subdirectory

Issue description
One of our projects has a dependency on a private git repo. The dependency is in a subdirectory of that repo, which is a monorepo. The dependency is specified in the Pipfile like this:
mypackage = {editable = true, subdirectory = "mypackage", git =<EMAIL_ADDRESS>
Expected result
The repo should be cloned, and pipenv should execute the setup.py in the subdirectory of the repo. I have confirmed this is the case in the following versions of pipenv:

2018.05.18
2018.7.1

Actual result
Pipenv tries to execute python setup.py ... in the root of the dependency, rather than in the subdirectory as expected. Installing the same dependency using the latest version of pip directly still works. I verified that this issue was introduced in pipenv==2018.10.9.
Steps to replicate

Create a git repo with a Python package in a subdirectory of the repo. The subdirectory should contain a setup.py, but the root of the repo should not.
Create a second Python package with a dependency on the package from step 1, as per the above Pipfile config.
Run pipenv install and note that the installation of the dependency fails because pipenv looks for setup.py in the root of the cloned repo, rather than in the subdirectory.

I can confirm, there's been a pull request to support this in the past: https://github.com/pypa/pipenv/pull/507
But it still does not appear to be working.
Example output:
# pipenv install -e 'git+https://github.com/WoLpH/redisrpc.git@asyncio#egg=redisrpc&subdirectory=python'
Installing -e git+https://github.com/WoLpH/redisrpc.git@asyncio#egg=redisrpc&subdirectory=python…
Obtaining redisrpc from git+https://github.com/WoLpH/redisrpc.git@asyncio#egg=redisrpc
  Updating .venv/src/redisrpc clone (to revision asyncio)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File ".venv/lib/python3.6/tokenize.py", line 452, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '.venv/src/redisrpc/setup.py'

    ----------------------------------------

Error:  An error occurred while installing -e git+https://github.com/WoLpH/redisrpc.git@asyncio#egg=redisrpc&subdirectory=python!
/bin/sh: -i: not found
Command "python setup.py egg_info" failed with error code 1 in .venv/src/redisrpc/

This is likely caused by a bug in redisrpc. Report this to its maintainers.

Note that installing it through pip works effortlessly.

I think I may have introduced this bug recently. @WoLpH can you tell me what shell you're using?

I'm using the z-shell
Not sure if it broke recently, this is the first time I've tried one of these with pipenv to be honest.

I'm pretty sure it broke reasonably, I've been relying on this feature for months and it only broke when 2018.10.9 was released

Thanks guys I think I have a fix, having a testable example was super helpful

Thanks!


About semantic consistency>0.7 in  Text-to-Video Fine-tuning?

Maybe it should be  the average LPIPS distance between each pair of frames in the video clip.
A smaller value indicates higher similarity, reflecting better semantic consistency.
But in Text-to-Video Fine-tuning stage, Semantic Consistency (LPIPS score)>0.07.   It's to get more complex video,  not for more stable video?

Yes.
As we want the model to generate videos with more dynamics at the SFT stage, we choose an LPIPS score > 0.07 to remove stable videos (especially those made of a single image with some simple transformations).


chore: update guild feature list

Summary
adds a few missing features and marks a few others as no more effect.
Checklist

[ ] If code changes were made, then they have been tested

[ ] I have updated the documentation to reflect the changes
[ ] I have formatted the code properly by running task lint
[ ] I have type-checked the code by running task pyright

[ ] This PR fixes an issue
[ ] This PR adds something new (e.g. new method or parameters)
[ ] This PR is a breaking change (e.g. methods or parameters removed/renamed)
[ ] This PR is not a code change (e.g. documentation, README, ...)

Still need to check on this list a bit more, I think a few others have been removed/deprecated on this list.

Added a few currently undocumented features to types/guild but did not document them in our own documentation.


WIP: Unify names of generic relations to media classes

Fixes

Fixes #4553 by @sarayourfriend
Description

This is still WIP because I need to test a bit more thoroughly. So far things appear to work with manual testing, but I haven't run unit tests, and I suspect there are edge cases in the model admins that I have missed.
Heads up through, @dhruvkb, if you want to take a look at the approach I've taken. It's a bit abstract, and I could definitely see wanting to use a more concrete, straightforward, declarative approach that just manually creates the back references onto the media classes.
The approach overrides the metaclass for Django models with one specifically for models related to a media type. The abstract models reference that metaclass, along with a related name, and then subclasses are processed to get assigned to the media class as {related_name}_class, and the related name from the abstract class is passed down to the Meta of the final models (which Django doesn't do on its own, only a small set of fields are inherited from Meta by default).
I initially tried this with __init_subclasses__ but you can kind of see what that results in with the admin models and forms... but a bit more unwieldy because of the related name. It ended up being a lot of specific, fiddly implementation details on each of the related abstract classes, which the metaclass was able to tidy up into a single implementation pretty nicely. Of course, it is a bit magical, so if we want to just, say, hand-maintain a nested dictionary of these relations, kind of like the media_type_config fixture in the tests (which could be replaced or reduced by some of this work), and prefer that for its straightforwardness (if being slightly more clunky and requiring more manual configuration of relation names) then I can probably be convinced.
My goal was to prevent any SQL migrations, and indeed, sqlmigrate shows the current migration as a complete noop.
Anyway, let me know what you think, Dhruv, if you get a chance to look at this while it's still a draft. Very curious to see what you'll think of this introspective/meta-programming based approach, and whether you can think of an easier way to do it or think a more concrete and less meta approach would be better.
Testing Instructions

TBD.
Checklist

[ ] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[ ] My pull request targets the default branch of the repository (main) or a parent feature branch.
[ ] My commit messages follow best practices.
[ ] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible errors.
[ ] I ran the DAG documentation generator (./ov just catalog/generate-docs for catalog
PRs) or the media properties generator (./ov just catalog/generate-docs media-props
for the catalog or ./ov just api/generate-docs for the API) where applicable.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Yeah, totally agree metaprogramming is a pretty intense approach for this. I suppose the motivation of using metaprogramming to solve this, as opposed to explicitly configuring the related names on each real model (and additional explicit __init_subclass__ or other type methods) was to make the wiring of these happen automatically, so new media related models (including other subclasses for new future media types) behave identically without needing to repeat the wiring everywhere.
The alternatives I considered were:

as mentioned, __init_subclass__ on each of the related media models. It is still metaprogramming, just maybe metaprogramming "lite", and required manually assigning related names on fields, because __init_subclass__ happens after the Django metaclass initialisation, and so after _meta is constructed, and critically after contribute_to_class is called on each of the fields... meaning __init_subclass__ cannot be the source of default_related_name on the subclasses' Meta (it doesn't happen early enough in the lifecycle). So ultimately this required a lot of manual configuration still, for every abstract related model and for each of the subclasses (which needed Meta.default_related_name defined individually or to manually inherit Meta from the parent class).
The same as above, but instead of __init_subclass__ on each related model, create a caching @property \ @classmethod on AbstractMedia for each of the related media models, which would just call <Related media model>.__subclassess__() and find the one for the media type of the abstract media subclass (and then cache it so the runtime cost of accessing the classes this way is amortised). As with __init_subclass__, this still requires manually configuring the related_names on the related models fields or Meta class.

What I like about the metaclass approach is that it fully centralises the configuration of the related name and the backreferences of the actual classes from the media models. so that subclasses of the related models don't have to do anything (other than define media_class, which most already did anyway). It removes the need to manually configure anything beyond the abstract classes, except for edge cases like the moderator reference, where the default related name clashed.
So, the metaclass approach looks pretty intense at face value, but it does, I think, ultimately simplify the configuration and make it less fragile or prone to random accidental variations (for example, when new related models are added).

(Here's an aside about our data model, inspired by thinking about the different approaches and where this complexit comes from).
To be honest though, I think it's worth us considering whether the per-media type segmentation of the models (and tables) is entirely useful, and whether there is a generic approach that would actually eliminate the need to repeat structures for each media type, especially when considering aspects of the data model like the complexities I touched on in #1290, where our current model is so focused on individual works, it doesn't allow for the actually complex relationships that sometimes exist. Audio having Audioset as a distinct audio-specific implementation is interesting, for example. We already have images in our catalogue that are parts of real, meaningful sets (which could, for example, be extremely useful for related media queries). Mostly these are scans of pages from the same books (Biodiversity Heritage Library being the source that immediately comes to mind), but there are other forms of images-in-series or collections... not to mention the idea of being able to create Openverse-user curated collections, which we've touched on frequently.
So where metaprogramming in this PR seems like it could be overly complex or abstract, it might also (or instead) point to a problem with our data model, which pre-encodes this level of abstraction and complexity that might neither be necessary or even real in a fundamental sense, and for which meaningful differences between media types could be accounted in less complex ways.
Notably, other systems for modelling the metadata of individual works all work under the assumption of linked data built off core metadata shared by most works regardless of the kind. For example DublinCore and the more complex RDF (which both came up in #4430) work from the same set of basic terms shared by all works, and enabling much more flexible modeling and metadata storage than we currently have, demonstrated by Fedora Repository[^notlinux] as an example. Our per-media-type approach is not necessarily the best way of doing things generally, and other systems that pre-date ours and which have been manually tested and applied to many millions of digital and analogue works do things a little differently.
Crucially broken in our system, for example, is how a multimedia work or collection of works would be described. We don't have any way of modelling audio that are related to images, for example, that wouldn't require creating a separate container for those works from containers of audio (which we have as audio set) and containers of images (which we do not have).
Anyway, that's not really important to the question of what the best way of solving this problem of generic relations between these Python classes... I just thought it was interesting to consider why these relations exist, whether they're necessary and whether they're even real (in the sense of, does this division actually exist when works are reduced down to an identifier pointing to a binary blob, with a few to many of potentially infinite number of data points associated). If we step back from the model we've implemented and especially if we look at how others have tried to solve this problem, our approach is rather distinct. Whether that's good, purposeful, or serves our use case in a meaningful way that is worth the trade-offs... good for us to think about!
[^notlinux]: Unrelated to the Linux distro.

This is looking quite interesting. Similar to Dhruv's opinion, I like how much code this saves, and the resulting one looks clean. I also worry about the metaprogramming part, it diverges quite a lot from our current approach. If we can reach an intermediate stage that would be wonderful. Perhaps with your second alternative:

The same as above, but instead of init_subclass on each related model, create a caching @property \ @classmethod on AbstractMedia for each of the related media models, which would just call .subclassess() and find the one for the media type of the abstract media subclass (and then cache it so the runtime cost of accessing the classes this way is amortised). As with init_subclass, this still requires manually configuring the related_names on the related models fields or Meta class.

So where metaprogramming in this PR seems like it could be overly complex or abstract, it might also (or instead) point to a problem with our data model, which pre-encodes this level of abstraction and complexity that might neither be necessary or even real in a fundamental sense, and for which meaningful differences between media types could be accounted in less complex ways.

Regarding data modeling, I find our current approach still simple, as we currently have a few central tables. Actually, most of the complexities I usually find come more from maintaining a sort of mirror with the catalog DB. Of course, I'd be interested in exploring new ways to store/represent the data, but I don't see this as a problem for our current use case but with expanding the use cases that we could potentially include.

Crucially broken in our system, for example, is how a multimedia work or collection of works would be described. We don't have any way of modelling audio that are related to images, for example, that wouldn't require creating a separate container for those works from containers of audio (which we have as audio set) and containers of images (which we do not have).

The key is that the current models weren't exactly planned for collections, so it's understandable that they cannot be fit currently. It's important to note this because relational models are designed with very specific requirements so they can work better. It's not set in stone, of course, but trying to change it one way or another has several implications.

If we step back from the model we've implemented and especially if we look at how others have tried to solve this problem, our approach is rather distinct.

I'd like to know more about these cases you mentioned so I can understand them better! And see how they relate to what we are trying to resolve. I agree we can reevaluate all of these, but we also need to define the requirements/constraints beforehand.


[FEATURE REQUEST] Align indentation system

What is the problem you're trying to solve?

Let's make sure we have the same indenting system for all files in the project.

What solution would you like to see?

Let's configure Eslint to use tabs for (t|j)s(x)

Solved in #16 with Prettier


Show the highlighted option when searching for "open files"/"project files" using the "goto file"/"goto everthing" menu

First of all, when the "goto file menu" is open nothing is shown (accept the history of recently open files), this should change:

either by showing all open (open editors) and project files directly beneath "recently open files" by default;
or by showing all open (open editors) and project files beneath "recently open files", after the user types a specific character like "*".

Note that the open files (open tabs) must show on top of other files, just beneath recently open files.

When a menu open is highlighted, the corresponding preview of the file content should be shown in a "temporary tab"

This is similar to when the user is a shown a preview of color themes from a similar menu";
The "temporary tab" behavior is similar to when the user clicks a file once (not double click it) from the project folder (a preview tab is shown but the file is not considered fully open).

Notes:

This requested feature is present in anther popular proprietary text editor (I don't know if I'm allowed to mention its name);
I may provide an animated images later to showcase what I'm talking about.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Why we tried ib_read_bw and ib_write_bw testings without FFO installed  but succeeded? And why we installed libibverbs but can't find drivers?

In Section 4.3 where one-sided operations are discussed, we see there are two problems to support one-sided operations, and the first is the local FFR does not know the corresponding s-mem on the other side. To solve this problem, FreeFlow builds a central key-value store in FFO for all FFRs  to learn the mapping between mem’s pointer in application’s virtual memory space and the corresponding s-mem’s pointer in FFR’s virtual memory space. However, our testings of ib_read_bw and ib_write_bw all succeeded without FFO installed, though we don't know how to install FFO.
It should be noted that all of our ib_send/read/write_bw testings are based on rdma_cm mode, because if we install libibverbs, we will encounter a warning of 'no userspace device-specific driver found'.

So we only install libmlx4 and librdmacm, and all testings are based on standard libibvers of rdma. Then if we test based on non rdma_cm mode, it will not go through router.
Did you met this problem before?  We tried to solve this problem, and found that the function try_driver in init.c fails to find dirvers when executing

Then we think it is caused by driver initialization, and locate to function mlx4_driver_init defined in mlx4.c in libmlx4. We also found in file mlx4.c, you cut many lines, that make us confused. The problem we finally located to is in the following code, it doesn't 'goto found', so 'return NULL' early.

But why? Why rdma_cm mode doesn't met this problem? But with libibvers installed, both modes are influenced?
Wish your answer!

Did you install Mellanox OFED driver outside the container, and mount the user space driver path into the container, like  -v /sys/class/:/sys/class/ ? You can find this in the README.md command line. Do you have /sys/class/infiniband_verbs/uverbs0 ?

Yes, we have installed Mellanox OFED driver both outside and inside the container. And we can make sure we have mounted the user space driver path into the container, because we used the command you provided to start the application container, without any modification. We also have

We have found out why it fails to find devices. Because the abi_version of Mellanox NICs we used is 1, not within 3 to 4, so it needs to match libmlx5, not libmlx4. However, we must use high version Mellanox NICs. Anyway, thank you for your attentions.

Sorry to bother you again. But we still want to know why we tried ib_read_bw and ib_write_bw testings without FFO installed but succeeded under rdma_cm mode? According to the analyses what have been discussed in your paper, we can see FFO is indispensable when executing one-sided operations, but how is it reflected in the open source environment.
Here is our test case of ib_read_bw.
In the server side,

In the client side,

And we got the output

The above testing was executed between two containers from different hosts, and all succeeded.
Maybe our testing method was wrong, but it really went through FFR.
Look forward to your reply.
Thanks.

We have found out why it fails to find devices. Because the abi_version of Mellanox NICs we used is 1, not within 3 to 4, so it needs to match libmlx5, not libmlx4. However, we must use high version Mellanox NICs. Anyway, thank you for your attentions.

Hi ling0329, we are also trying to run Freeflow with newer Mellanox NICs. Do you mind telling me what you had to change? We also say that driver initialization code failed since it cannot find a matching device in hca_table. Did you have to modify libmlx5 instead?

@ling0329 I think in this implementation, they hardcoded the one-sided mapping information in code.

the released implementation hard-codes the host IPs and virtual IP to host IP mapping in https://github.com/Microsoft/Freeflow/blob/master/ffrouter/ffrouter.cpp#L215 and https://github.com/Microsoft/Freeflow/blob/master/ffrouter/ffrouter.h#L76.

Also, it looks like you are also trying to run Freeflow with newer NICs. Do you get it to work successfully? And can you share what version of Ubuntu and OFED you are running? (both container and host OS)

We are still trying to solve this problem but failed. Actually, we are not ready to modify libmlx5, because there are much differences between libmlx4 and libmlx5.
This is our Ubuntu version on host OS

The Ubuntu version of container is the same

Our OFED is MLNX_OFED_LINUX-4.0-<IP_ADDRESS>-ubuntu14.04-x86_64. We use ConnectX-4 40G NICs, and you can see details here

MT27700 Family is not listed in libmlx4

and we want to try more newer NICs, like ConnectX-5 25G.

I see. Thanks for sharing your setup.
Yeah, porting these changes to libmlx5 is probably gonna take a lot of effort.
It seems that FreeFlow only works with ConnectX-3. I saw your workaround for the hca_table check and used that to get rdma_client/rdma_server working, albeit it still hangs 20% of the time.

The current architecture of Freeflow works only with libmlx4. It's possible to use the LD_PRELOAD trick to re-implement a cross-driver-version solution by intercepting relevant calls. However, it requires quite a bit efforts, and all the authors of this project are now busy with something else...


updated build instructions for RN Tester in Apple silicon based machines

Summary:
I was trying to build the rn-tester app on Apple Silicon Mac (M3 Pro) and encountered build failures with command PhaseScriptExecution failed with a nonzero exit code. Installing Rosetta 2 resolved this issue, indicating it's required for proper x86_64 command execution on Apple Silicon. This PR documents this requirement.
Changelog:
[INTERNAL] [FIXED] - updated build instructions for RN Tester in Apple silicon based machines
Test Plan:
Running xcworkspace in Xcode resulted in error, now the app builds successfully.

cc @huntie  @hoxyq

I was trying to build the rn-tester app on Apple Silicon Mac (M3 Pro) and encountered build failures with command PhaseScriptExecution failed with a nonzero exit code. Installing Rosetta 2 resolved this issue, indicating it's required for proper x86_64 command execution on Apple Silicon

That's actually not necessary. I've been building RNTester on a M1 Max without rosetta installed without problems. You probably had a different issue.
@cipolleschi can also tell you more about M1 support for building React Native.

Yeah i might be wrong but can we execute x84 commands? if rosetta is not installed by default, i was getting the hermes phase execution error. As i installed rosetta, it happened to make build successfully.


Fix fromInfinite error

In the implementation of fromInfinite, we pass an infinite list to Data.Primitive.Array.fromListN (bits + 1), but fromListN expects a list a list of length bits + 1, which results in a runtime error.
To fix this, we take the first bits + 1 of the infinite list before passing to fromListN.
Updated solution: To fix this, we add a check in go to break recursion and to ensure that the list that is passed into fromListN has length bits + 1.

Fixes issue https://github.com/Bodigrim/chimera/issues/40

Thanks a lot!

Released in https://hackage.haskell.org/package/chimera-<IP_ADDRESS>


The "more" button disappears when Bookstack is in smartphone mode

Describe the Bug
When I try to edit a page of my "books" on my smartphone, the "WYSIWYG" editor does not show the "more" button that has 3 dots.
This only happens when I try to edit a page on my smartphone or tablet. This doesn't happen when I'm on my laptop.
Steps to Reproduce

Open a Android browser like Vivaldi or Firefox
Try edit a page
Notice the lack of the "more" button.

Expected Behaviour
That Bookstack correctly displays the "more" button when I am editing a page from my smartphone.
Screenshots or Additional Context
Here is the view from my browser on my laptop:

And here is the view from my smartphone.

Browser Details
Vivaldi 6.7.3335.149 (Android)
Exact BookStack Version
v24.05.2

Could what I have described be a bug? Or is it a limitation of the WYSIWYG editor when in phone mode? Or is it done that way by design? Or does it just happen to me and doesn't happen to anyone else?
Bookstack is an amazing software. And it has the best smartphone-friendly design I've seen in a long time.
So I find it a little annoying that the button to insert code or media (like in the image below) doesn't appear when I use my phone.

I don't know if I have managed to explain myself correctly since English is not my first language.


Using the tool with non-cellpose/watershed segmentation algorithm

Hi @timwiggin ,
I am evaluating different segmentation algorithms and so far it looks like the best performance for my tissue comes from a different tool.
Is it possible to plug in a mask per tile and then continue from compile-tile-segmentation and how would I go about doing that?
Thank you!

Thanks for reaching out! Unfortunately, VPT doesn't accept segmentation masks as an input.
If you're able to convert the masks into cell boundary polygons, it may be possible to import the cell boundaries into parquet format using the convert-geometry tool:
https://vizgen.github.io/vizgen-postprocessing/command_line_interface/index.html#required-arguments
convert-geometry currently supports QuPath-formatted geojson files and Vizgen-formatted hdf5 cell boundary inputs.

Ok, I will give it a shot :)


Fixed GenericSerializerImpl to work with multiple topics.

writerSchemaString is used only when sending the schema to SchemaRegistry , so there is no need to be stored locally in collection.
Since subject is the only unique element it's used as a key to the collection of registered schemas, and schema id is added as part of this collection.
Then schema id is retrieved from registered schemas with the subject as key

It looks like @NikolayDekov hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot

[clabot:check]

@confluentinc It looks like @NikolayDekov just signed our Contributor License Agreement. :+1:
Always at your service,
clabot

Hi @mhowlett , i have updated the PR , the check if schema is already in collection is after the call to schema registry in order to register the subject with this schema.
Regards.

one more thing before merging ... instead of constructing the KafkaException with an ErrorCode, can you provide an Error with a custom explanation. something like new Error(isKey ? ErrorCode.Local_KeySerialization : ErrorCode.Local_ValueSerialization, $"Duplicate schema registration encountered: Schema ids {schemaIds[writerSchemaString]} and {newSchemaId} are associated with the same schema.")

beautiful.

note that the most convenient way to use this before the next release is probably to use the appveyor nuget feed (instructions in the README).


Post separator option and fix for #51

Please forgive me, it's my first pull request :)
Post separator only split on the first sentence, so this has an option to separate on a user defined string or regex (default is ).
Also made a change to parseContent method so the other style of markdown h2 headers can be used (issue #51). I tested this for all the cases I can think of in a new example post, but maybe not thoroughly enough.

I just learned pull requests update when I update my fork. Didn't intend to add the disqus and url changes to this. I'll work on a different branch in the future.

Thank you for the suggestion! Closed because of version 2.0 release. Feel free to resubmit under v2 if it's still relevant.


Question: How to handle dependency for static binary

I have a Rust project where I build a static binary which I want to include into the deb package.
depends = "$auto"

yields
warning: Failed to find dependency specification. (no auto deps for....

Question: How would I code the depends statement correctly?

Auto is only for executables. You need to list Debian deps manually.

My wording was bad. Instead of static binary I should have said static executable. In this case ldd yields not a dynamic executable.

I see. In that case you have no dependencies, so there's nothing to set.

Ok. Now tried
depends = ''

which indeed works fine.
Thank you.


No way to turn off automatic Intellisense popups.

I find automatic popups to be very distracting while writing code!
I have disabled automatic Quick Suggestions in settings.json and use Ctrl+Space whenever I DO want to see them.
But, there seems to be no way to keep Intellisense messages from automatically popping up without disabling them completely.
...Bob<EMAIL_ADDRESS>

use these two settings: editor.quickSuggestions and editor.suggestOnTriggerCharacters

jrieken, thank you so much.
That's been driving me crazy.
...Bob

Johannes,Thank you so much.
That's been driving me crazy.
...Bob
On Thu, Dec 10, 2020 at 12:21 AM Johannes Rieken<EMAIL_ADDRESS>wrote:

use these two settings: editor.quickSuggestions and
editor.suggestOnTriggerCharacters
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/112185#issuecomment-742362236,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQVEH3PZ3WKZOJ5UBZQBPCTSUCAJ3ANCNFSM4UUEVZQQ
.

Johannes,
Sorry to bother you again but I spoke too soon.
With the change you suggested, the popups do not, indeed, appear
automatically as I type BUT they DO appear automatically whenever my cursor
happens to land on some random function anywhere in my code.
Is there a way I can make that kind of popup behavior manually activated as
well?
...Bob
On Thu, Dec 10, 2020 at 10:08 AM Robert Yodlowski<EMAIL_ADDRESS>wrote:

Johannes,Thank you so much.
That's been driving me crazy.
...Bob
On Thu, Dec 10, 2020 at 12:21 AM Johannes Rieken<EMAIL_ADDRESS>wrote:

use these two settings: editor.quickSuggestions and
editor.suggestOnTriggerCharacters
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/112185#issuecomment-742362236,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQVEH3PZ3WKZOJ5UBZQBPCTSUCAJ3ANCNFSM4UUEVZQQ
.

@rbtyod this feature is called a "hover". Try searching settings for that term. Maybe set "editor.hover.enabled": false


Make nwc keys random

Thought about it and there's really no reason to have determinism with these nwc keys. As long as we save the random child key index, then it should be fine. When these are stored remotely then it should be fine as well.
This allows for backwards compat with existing keys that are based on index as before. So no need to recreate NWC profiles.

needs formatting


Failure to upload large files on MacOS

This is continuing the discussion in #2.
There, it was discovered that large files were not being uploaded correctly on MacOS.  It appears that the beginning of the file was full of \x00 bytes.  This kept us from figuring out the file type, but presumably this would also cause the file as uploaded to be broken.  The same files that cause this problem work fine on a Linux system, so it's something specific to the Mac, it seems.
A couple of possibilities:

MacOS is writing the file full of null bytes first.  Not sure why it would do this, nor why it only does this for large files.
Something is going wrong in llfuse, which is being used for MacOS.  But I can't reproduce it with llfuse on Linux.
Something is going wrong in MacFuse.

A bit of a shot in the dark, but if you're willing to try something, @AaronDavidSchneider:
Let's try setting stats.f_bsize = stats.f_frsize = 0 here: https://github.com/rschroll/rmfuse/blob/master/rmfuse/fuse.py#L486.  Another filesystem I saw specifically set these to zero for Macs, although I couldn't make out why.  We'll also have to adjust the next line to avoid a divide-by-zero error.  The right-hand side change be changed from 2**32 / stats.f_frsize to 2**32 / 4096, or any other large number, presumably.
I'm not too hopeful that this solves the problem, but I haven't had any better ideas, I'm afraid.

Hi,
I just tried it. Unfortunately nothing changed. Large files (like the flight plan still do not work)

Thanks for trying, anyway.  At this point, I may need to reach out to the
MacFUSE people to see if they have any ideas.  I'll tag you when I do, if
that's okay with you.
On Mon, Mar 15, 2021 at 12:12 AM Aaron David Schneider <
@.***> wrote:

Hi,
I just tried it. Unfortunately nothing changed. Large files (like the
flight plan still do not work)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rschroll/rmfuse/issues/18#issuecomment-799174745, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACZ2HZEZ7ZDSZ4YOMOPAGDTDWXN7ANCNFSM4YYFU3EQ
.

I am really curious about this bug... Did you ask the FUSE devs? Do you have a link to that discussion?


URL constructor doesn't allow credential

The code new URL("http://username:password@<IP_ADDRESS>:80/jsonrpc") throws A security problem occurred. on Edge (14.14372).
The standard never mentions about disallowing credential in URLs, and the code above runs successfully in Firefox (48) and Chrome (52) and returns username and password without problem.
I'm porting an extension from Chrome, it relies on this API to parse URL and make requests.

The URL class is defined by the Edge host. Marking as external. I'll see how we can route this bug.

Seems like a regression. Can you file create an issue at https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/ so we can track that? thanks.

OK, here it is: https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/8004284/

Thanks @CnSimonChan - closing as External.