[ Security issue ] Viruses detected after compile

Greeting,
Today I wanted to build LMMS on Windows using the latest MSVC and QT 6 to help with porting to the latest QT version released.
Before that, using VCPKG I tried to install all the needed dependencies, which include "lilv" library.
To my surprise it wouldn't build, at least not the release ( the debug builds ) and after hours of investigation, I have noticed that my antivirus spotted a danger with the generated DLL "lilv.dll" and another executable generated in "meson-private\tmpu5lnsej4\output.exe" in the build directory which is also spotted as a virus.
In the doubt of a false positive I decided to send the DLL to virustotal website to be tested by more antiviruses, and 10 of them spotted the DLL as infected.
VirusTotal result
I don't know since when the malicious code was pushed because last time I compiled LMMS on Windows was years ago. But it means that all linux distro shipping the lib lilv are at risk as of now.

I don't know since when the malicious code was pushed because last time I compiled LMMS on Windows was years ago. But it means that all linux distro shipping the lib lilv are at risk as of now.

Since it's all built on your machine from source code, that means it is easily vetted to determine whether that's a false positive... antivirus software is notorious for false positives if you don't pay someone money to sign your program...

The thing here is that it could potentialy fall under an eventual false positive if it was just one antivirus or two, but here we have    10 antiviruses detecting something, so there is definitely something fishy that needs attention.

Virus scanners do pattern recognition in binaries, and they share databases. They are not security scanners, and one actually should expect a false positive in one tool to show up in several others.  Not seeing that would be quite strange.
I can look into the idea that someone has snuck malware past me into lilv, but this is wildly unlikely for a whole host of reasons. Please tone down the fear mongering unless you have something more substantial to base it on.  All of this code passes through me personally (the overwhelming majority being written by me alone, contributions significant enough to be relevant here are very rare), and I'm a known person in this particular niche and have been for a long time.  As alluded to above, virus scanner companies have a vested interest in casting as wide a net as possible, and have many questionable practices based on that where they directly benefit financially from the issues (like this one) that causes.  Meanwhile, I certainly have no interest in destroying my reputation (and that of LV2 or free audio software in general) by doing something so silly as to put malware into lilv.

In particular, if it is claiming that meson-private\tmpu5lnsej4\output.exe is a virus, that is probably a compiled executable generated from a C one-liner as part of Meson's builtin support checks. It is... very easy to ascertain whether the input file in question would have a virus in it. The file meson-logs/meson-log.txt would show the source code (also likely still present in testfile.c in the same directory) and the command line for e.g. MSVC's cl.exe used to compile it.

Yeah, I'd like to see that too.  What pattern is being tripped by such a tiny thing might be interesting. Could be something so silly as the standard setup for a common(ish) system call... there's not much else there, really.

I just used the basic command given in the LMMS build on windows to retrieve//install package from VCPKG:
.\vcpkg install fftw3 libsamplerate libsndfile lilv lv2 sdl2 fluidsynth
Also I'm not trying to spread fear mongering nor point finger at anyone. But when someone just follow what should be a simple command to retrieve dependencies and get virus alert then proceed to get further feedback from more tools to check before posting concerns to be cleared by people who have knowledge about the concerned project, there is nothing wrong trying to clarify what is going on.

Well, if we know which bit of code exactly is tripping up the virus scanner, that might help figure out if there is something that can be done to fix it. ;)
So, that log file could be a good first step.

I'll look for it .
Also the issue can be reproduced by just installing the latest MSVC and latest QT then just running the VCPKG command I have shown, so if we get more people testing we could figure out faster aswell ;)

For the DLL no idea as of now, the only hint I have is that for the debug build of lilv there is no issue, it happens just for the release build.
Also this is the part in the log about meson that seems to have triggered the antivirus:

Checking if "printf supports the 'ls' directive" runs: YES
Running compile:
Working directory:  E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4
Command line:  C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe -INCREMENTAL:NO -IE:/Programming/Projects/vcpkg/installed/x64-windows/include E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c /FeE:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\output.exe /nologo /showIncludes /utf-8 /MD -nologo -DWIN32 -D_WINDOWS -W3 -utf-8 -MP -MD -O2 -Oi -Gy -DNDEBUG -Z7 /nologo /showIncludes /utf-8 /Od /Oi- /link -machine:x64 -DEBUG -OPT:REF -OPT:ICF /LIBPATH:E:/Programming/Projects/vcpkg/installed/x64-windows/lib
Code:
#include <stdio.h>
#include <string.h>
static char buf[10000];
int main ()
{
int result = 0;
buf[0] = '\0';
if (sprintf (buf, "%Lf %d", 1.75L, 33, 44, 55) < 0
|| strcmp (buf, "1.750000 33") != 0)
result |= 1;
buf[0] = '\0';
if (sprintf (buf, "%Le %d", 1.75L, 33, 44, 55) < 0
|| strcmp (buf, "1.750000e+00 33") != 0)
result |= 2;
buf[0] = '\0';
if (sprintf (buf, "%Lg %d", 1.75L, 33, 44, 55) < 0
|| strcmp (buf, "1.75 33") != 0)
result |= 4;
return result;
}
Compiler stdout:
testfile.c
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\stdio.h
Note: including file:  C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt.h
Note: including file:   C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vcruntime.h
Note: including file:    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\sal.h
Note: including file:     C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\concurrencysal.h
Note: including file:    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vadefs.h
Note: including file:  C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wstdio.h
Note: including file:   C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_stdio_config.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\string.h
Note: including file:  C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_memory.h
Note: including file:   C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_memcpy_s.h
Note: including file:    C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\errno.h
Note: including file:    C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vcruntime_string.h
Note: including file:  C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wstring.h
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(9): warning C4474: 'sprintf' : too many arguments passed for format string
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(9): note: placeholders and their parameters expect 2 variadic arguments, but 4 were provided
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(13): warning C4474: 'sprintf' : too many arguments passed for format string
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(13): note: placeholders and their parameters expect 2 variadic arguments, but 4 were provided
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(17): warning C4474: 'sprintf' : too many arguments passed for format string
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(17): note: placeholders and their parameters expect 2 variadic arguments, but 4 were provided
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(9): warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(13): warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\testfile.c(17): warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
Compiler stderr:
cl : Command line warning D9025 : overriding '/O2' with '/Od'
cl : Command line warning D9025 : overriding '/Oi' with '/Oi-'
Could not run: ['E:\Programming\Projects\vcpkg\buildtrees\glib\x64-windows-rel\meson-private\tmpu5lnsej4\output.exe'] (error: [WinError 5] Accès refusé)

Also I'm not trying to spread fear mongering nor point finger at anyone. But when someone just follow what should be a simple command to retrieve dependencies and get virus alert then proceed to get further feedback from more tools to check before posting concerns to be cleared by people who have knowledge about the concerned project, there is nothing wrong trying to clarify what is going on.

Siiiiiigh.  Fine:
I'm not saying I don't appreciate the feedback, I just asked you nicely to tone it down.  Given the bolding, "[ Security issue ]", "I don't know since when the malicious code was pushed because [...]. But it means that all linux distro shipping the lib lilv are at risk as of now" I think that's quite reasonable.  This is not a very nice thing to have to deal with in your inbox on an otherwise pleasant Sunday.  Now you're being defensive and trying to shift the subject to "trying to clarify what is going on here", which... well, no.  Asking for clarification would have at least one single question mark in it.  You outright stated in a confident and objective-sounding tone, right out of the gate, that this project contains malicious code and that all Linux distributions are compromised by it.
I've told you I don't appreciate that.  It's not a debate, and this is not a product/consumer relationship in which I'm obligated to be nice about it.  As it happens, I have a universal policy about that with respect to ticket etiquette: I'm nice exactly once.  You've spent your one nice token.  Thank you for the investigation, truly - however, please be advised that any more defensiveness around this tangent, or any future tickets with an accusatory or entitled or fear-mongering tone, will be met with immediate deletion and, after that, banning if necessary.  I really do not like being slandered in exchange for giving people countless thousands of hours of work for free (can you blame me?), and I am far too old and crusty to beat around the bush about it.

I have no idea how to narrow down what's triggering these in the lilv source code itself (the above check code comes entirely from meson, it's not a custom check), but given that the above pasted check is flagged for some reason, I'm pretty confident in saying that it's not really worth the time anyway.  Static code scanners for security and other issues should be taken very seriously with extreme scepticism that anything is actually a false positive, but virus scanners are a very different situation.
If someone else narrows it down, I can try to manipulate things to avoid it, but I'm not about to wade through a bunch of proprietary Windows virus scanners.  Last I checked, the built-in one in Windows 10 even flags meson itself, so I doubt this sort of thing is that unusual, although I don't use Windows much so I don't really know.
For the record, since sometimes people get confused by the ubiquitous use of URIs in the LV2 ecosystem: lilv doesn't do any network access whatsoever.  It also never writes to disk, with the exception of lilv_state_save for obvious reasons.  It's certainly not a hardened library in any sense, but there's very little opportunity for malice here.  Particularly since the whole domain is inherently about loading and executing arbitrary native code (plugins) anyway.
(The next major version of lilv will be pretty radically different internally, here's hoping it just goes away for free soon enough)


plugin ignores @formatter:off and  @formatter:on

the plugin is ignoring this subsequence... the result in my case is that some javadocs have become completely unreadable. I already turned the option on that IntelliJ should not ignore this sequence and if I disable eclipse code formatter IntelliJ does ignore it.
please fix...

what?

I mean the
// @formatter:off
...
// @formatter:on
notation is completely ignored by this plugin. This leads to problems in some of our projects. Would be great if support is added.

https://github.com/krasa/EclipseCodeFormatter/issues?utf8=✓&q=%40formatter%3Aoff


Make TileData of Tile in Tileset accessible

Hey I was just trying to get some properties/animation/collision data from the tiles in my tileset and I found that TileDatas visibility is pub(crate) so its not accessible right now and I don't see a way to do this either. There does not seem to be any plans of adding this yet.
Either we need functions to access the TileData properties from Tile or make TileData truely public. I don't know which approach makes more sense.

Tile derefs to TileData. You can access data from Tile directly, or if for some reason you need the data type itself (Not common) You can just *tile.

Thanks for the quick reply. Damn I tripple checked that I'm not overlooking something but dereferencing didn't even cross my mind. Tried it just now and it works. If you don't mind I would create a PR with an example showcasing this as I don't think its very obvious. Though maybe this is a very common approach and I just didn't do rust long enough yet?

I agree, the docs aren't very obvious on this regard. Perhaps adding a mention of this to the data/wrapper docs wouldn't hurt. An example showcasing the differences between the data and wrapper would be much appreciated too :)


trunk serve --open gives error: no storage space

I'm following the leptos getting started page,  (with a nix flake to provide a dev shell with cargo nightly + some other deps (essential leptos's development flake.nix.
Same deal with cargo nightly via rustup.
When I run trunk serve --open, I get the following:
Error: failed to watch "/home/til/projects/website" for file system changes

Caused by:
    no storage space

I definitely have storage space.
Version Info

trunk 0.16.0 (this is what cargo install trunk gave me
cargo 1.71.0-nightly (ac8401032 2023-05-02)
OS: NixOS 22.11.20230221.a3d745e (Raccoon) x86_64

I am getting the same error. I am using trunk 0.17.1 now and I also use nix for rust/cargo installation (though I just have Ubuntu 22.04 with nix installed).

Note: I verified that following snippet with just using notify crate (at version 4) gives me the same error (no storage space):
fn main() -> anyhow::Result<()> {
    use notify::Watcher;
    let (sync_tx, sync_rx) = std::sync::mpsc::channel();
    let mut watcher = notify::watcher(sync_tx, std::time::Duration::from_secs(1))?;
    watcher.watch("/home/myuser/code", notify::RecursiveMode::Recursive)?;
    Ok(())
}


External Drop on scheduler

Hi Mendix-DevTeam,
can we expect to see the external dragging of events on the scheduler-view anytime soon?
(See: https://fullcalendar.io/releases/fullcalendar-scheduler/1.9.4/demos/external-dragging.html)
Or is there a way to activate this is in the widget?
Best regards,
Alexander

No we will not continue adding feature to this widget, we will add new feature to https://appstore.home.mendix.com/link/app/107954/ but we dont expect to add external drag drop.


Support decimal casting in Parquet

Supports converting between ShortDecimals <-> LongDecimal when reading from Parquet Decimal column into Presto decimal column.
When Presto scale is < Parquet scale then value got rounded.
In every other case this code checks if converted value overflows Presto decimal type (if it does - exception is thrown) or got truncated on significant side (it it does - exception is thrown).

Merged, thanks!


Run missing debug command against 4.x cluster

What type of PR is this?
/kind bug
What does does this PR do / why we need it:
Specific debug command serial spec has never been run against 4.x cluster since beginning.
Which issue(s) this PR fixes:
Fixes NA, Sending the pr directly without any issue reference.
PR acceptance criteria:

[ ] Unit test : NA

[ ] Integration test : NA

[ ] Documentation : NA

[x] I have read the test guidelines

How to test changes / Special notes to the reviewer:
Missing test should run in prow CI

changes looks good to me
/lgtm

/test v4.5-integration-e2e

/test v4.5-integration-e2e

@zhengxiaomei123 You need to update System-PZ test script once this pr goes in. Please keep an eye on it.

@zhengxiaomei123 You need to update System-PZ test script once this pr goes in. Please keep an eye on it.

@amitkrout Thanks for reminding. Since there are issues for devfiles test ,I will test locally firstly and then update the script if it worked.

@amitkrout As per our 1-1 conversation over bjn. It clears my doubt for test execution time and failure analysis. My thoughts were leading into different direction. +1 for running the test with optimised approach.
It looks to me.
/lgtm

New changes are detected. LGTM label has been removed.

/lgtm

Probably a silly question but, since there's no issue related to the PR, what bug are we fixing through this PR? Looking at the code change, it feels more like cleanup/refactoring of test flow to me. Changing accordingly, feel free to revert.
/area testing
/kind cleanup
/remove-kind bug

https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_odo/3821/pull-ci-openshift-odo-master-v4.5-integration-e2e/1299756958319906816#1:build-log.txt%3A1109
/retest

/lgtm


Restricted attach in control_replication

I am hitting this in my checkpointed Regent Stencil code:
[0 - 7fde6c821840]    0.432527 {5}{runtime}: [error 604] LEGION ERROR: Attach operations in control replication context main (UID 6) requested a restriction. Restrictions are only permitted for attach operations in non-control-replicated contexts currently. (from file /scratch2/eslaught/resilience3/legion/runtime/legion/legion_context.cc:19020)

I can work around this, but I'm just wondering if this is an easy or hard fix to make.
(Relative to my other issues, this is low priority.)

Is this a singular attach operation or an index attach operation?

Also, are you sure you really want restricted coherence? Remember that will eagerly flush changes back to the instance(s) which could be expensive in the control replicated case.

It's a singular AttachLauncher.
The pattern I'm doing is:

Attach
Copy
Detach

So I think the restriction only helps me here (e.g., it helps the mapper be smarter about where to send the copy).
But the fact that I'm issuing individual attaches may indicate that my covering set algorithm is failing, somehow. I should probably look into this before going any further.
Like I said, please consider this low priority unless it's easy.

I'm going to wait to see if you really want this before working on it. I'm still pretty skeptical that anyone really wants restricted coherence in a control replicated setting like this.

It turns out that I only hit this if I checkpoint before running any timesteps of the main loop, because (surprise) the user has not written any partitions at that point. If I checkpoint after running at least one timestep I go down the IndexAttachLauncher path and everything works as intended.
I'm going to close this, because I don't believe it'll impact any of my applications at this point.


Events detectors

Includes:

Storage change events detector: emits a warning when storage is modified in any way and the contract does not emit an event.
Storage change events test cases.
Token Interface events detector: emits a warning when the token interface trait is implemented and the functions belonging to it that should emit events do not.
Token Interface events test cases.

sofi check the comments let us know whatever you thinkg about it..


Mapping ports without a listening application

I am trying to run a container, with a mapped port. Initially the application inside the container is not listening to the specified port, I need to start it later, when the container is up and running.
So I do this
toxiproxy = new GenericContainer("shopify/toxiproxy:latest").withExposedPorts(8474);  toxiproxy.setPortBindings(Collections.singletonList("11211:" + port));
I want to run another container, and then proxy its port with toxiproxy.
The problem is that this container refuses to start
org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32827, 11211] should be listening)

Hi @arashbi
By default GenericContainer will use a wait strategy that waits for all exposed ports to be listening. You can customise this though - please see the docs.
When trying out Toxiproxy myself recently, I've used new HttpWaitStrategy().forPath("/version").forPort(8474), for example.
Separately, is there a reason for manually setting port bindings? You should be able to avoid static port mapping, and we'd encourage you to do so - static port mappings make for troublesome port conflicts.
In case you're interested, I've put my proof of concept code for a Toxiproxy container class into a gist. With this code, I was able to implement Toxiproxy without any static port mappings.
HTH
Richard

@rnorth thanks a lot. It was very helpful. I ended up doing something similar.

Will close, as I think this is answered.


ASpanFormer (LofTR-like) support

Hi, would it be possible to create an example for ASpanFormer model, a loftr like model, that is mostly using the same code as loftr.
I tried to implement it using your loftr example, but first, got an error from onnx saying that the opset 12 doesn't support grid_sampler. error below:
Exception has occurred: UnsupportedOperatorError Exporting the operator 'aten::grid_sampler' to ONNX opset version 12 is not supported. Support for this operator was added in version 16, try exporting with this version.   File "convert_to_onnx_aspan.py", line 35, in main     torch.onnx.export(   File "convert_to_onnx_aspan.py", line 70, in <module>     main()
After setting the opset version to 16, the code successfully exported the onnx model, but when calling the inferenceSession as sess = onnxruntime.InferenceSession("aspanformer.onnx") got another error saying:
Exception has occurred: InvalidGraph [ONNXRuntimeError] : 10 : INVALID_GRAPH : Load model from aspanformer.onnx failed:This is an invalid model. Type Error: Type 'tensor(bool)' of input parameter (/coarse_matching/And_1_output_0) of operator (ReduceMax) in node (/coarse_matching/ReduceMax_2) is invalid.
The exported model even passes the onnx.checker.check_model validation.
Any help would be great, and thanks for the good work.

Hi. Here is my quick guess:
You need to convert bool type somewhere to int, somehow like this:
https://github.com/xmba15/LoFTR/commit/6cdf54498c3340dee0cc02d9406b60b2a0ecd541

Hi. Here is my quick guess:
You need to convert bool type somewhere to int, somehow like this:
https://github.com/xmba15/LoFTR/commit/6cdf54498c3340dee0cc02d9406b60b2a0ecd541

Still WIP but if you intend to use new onnxruntime version. You might need to take a look at this.
https://github.com/xmba15/onnx_runtime_cpp/commit/567a7b9bc2429154622e3b8f5bb201e058eec092

If it is possible, I am more than happy if you can create a PR to integrate your work on ASpanFormer into this repo.

@fatihkykc I could export the (new) outdoor loftr model too, but the performance was bad. Some other issues in loftr repo also reported the same thing with the (new) outdoor loftr model. If you can test and still get a good result, it would be great. I am curious to see the result.

@xmba15 Hi, can you provide some sample results, or at least a link to some sample images that outdoor loftr version doesnt work well, it may be a good start for comparing the results


How to evaluate DeepGlobe graph with APLS metric?

Hi, thank you for your response.
I wonder how to evaluate the inferred graph using APLS metric?
What data structure do you use to evaluate, and what's the key attribution that the data structure must have?
The reason why I ask so is that, according to https://github.com/CosmiQ/apls, the evaluation metric seems to be connected with lon/lat coordinate or utm coordinate.
Source codes may help more! Thx!

Hi @tansor, thanks for the query. I used the simple skeletonization of roads and convert to line-string formats. You can refer Spacenet-Road Challenge 2 winners source code. Evaluate the APLS metrics by APLS Visualizer tool with source code available at link, which is released as part of challenge.

Hi @anilbatra2185 , thanks for your answer. I'm still confused about how to obtain the DeepGlobe gt graph. Because the test method only allows gt graph to be networkx pkl or geoJson.
parser.add_argument('--test_method', default='gt_pkl_prop_pkl', type=str,
                        help="Method for creating ground truth and proposal "
                        + "graphs. Options:"
                        + " gt_pkl_prop_pkl   = ground truth pickle, proposal pickle"
                        + " gt_json_prop_pkl  = ground truth json, proposal pickle"
                        + "gt_json_prop_wkt  = ground truth json, proposal csv in wkt format"
                        + "gt_json_prop_json = ground truth json, proposal json"
                        )

Hi @tansor, I used the same steps of skeletonization and conversion to line-strings using the DeepGlobe segmentation GT mask. And now it will act as GT graph for deepglobe and you might need to do small changes in APLS visualizer to compare deepglobe GT and predicted graph.
STEPS for DG

Convert GT mask into skeleton mask.
Convert to GT graph
Modify APLS visualizer tool (specifically some paths to read images)
Evaluate APLS metric for DG using the predicted and GT graph.

Thank you for your patience!
You have solved my confusion. This issue can be closed now.

I also wonder how you evaluate DeepGlobe dataset with APLS metric. I tried to do that via code provided by "https://github.com/CosmiQ/apls", but I failed to use it on DeepGlobe dataset.
Please give me some details.. I read above talks, but I still didn't understand.

Hi, @hjlim0314.
The author did not use the code provided by "https://github.com/CosmiQ/apls", but a java implementation called APLS Visualizer tool in "https://drive.google.com/file/d/1rwbj_o-ELBfruPZuVkCnEQxAX2-Pz5DX/view?usp=sharing", which is released as part of the SpaceNet challenge.

@tansor Thank you for your tip! It will be very helpful

Hi @hjlim0314, @tansor I just made some modifications for DeepGlobe APLS metric measurement. Please use if needed, https://github.com/antran89/road_visualizer

Hi @hjlim0314, @tansor I just made some modifications for DeepGlobe APLS metric measurement. Please use if needed, https://github.com/antran89/road_visualizer

Thank you for your code. It works on my deepglobe data set very well.
I appreciate you notice me about that.


pdf.js partially fails to render text - bad x_height

Link to PDF file:
ud.pdf
Configuration:

Web browser and its version: all
Operating system and its version: Mac OS X 10.11.6
PDF.js version: most recent master

Steps to reproduce the problem:

Load PDF
Compare text areas on pages 9, 11 to preview or chrome's built in pdf viewer

What is the expected behavior? (add screenshot)sc

What went wrong? (add screenshot)

Getting this error multiple times: downloadable font: OS/2: bad x_height: -485 (font-family: "g_d0_f22" style:normal weight:normal stretch:normal src index:0)

It seems like the font cannot be sanitized properly, so I'm labeling it as such for now.

Works for me, despite the downloadable font: ... warnings being present in the console, using PDF.js version 1.5.437 on Windows 7. Could this be worse on MacOS?

@Snuffleupagus I didn't expect that but it appears to be true. It renders much better on Windows Chrome, Firefox, and IE.

Closing since this now renders fine. The sanitizer warnings are gone.


avoid debug-level logging from curl_httpclient

curl_httpclient logs way too much (all requests and responses) at debug-level.

Avoids setting log-level for curl lower than INFO, even if the rest of the app has debug logging enabled
Stops setting log-level to debug in the helm chart

Go for it @betatim


Fix pylint

Pre-Flight checklist

[x] Testing

[x] Code is tested
[x] Changes have been manually tested

What are the relevant tickets?
Closes #345
What's this PR do?
Modifies the github workflow command to run pylint so that it checks all python files.
Fixes lines of code that cause pylint validation errors.
How should this be manually tested?
Tests should pass

Codecov Report

Merging #348 (876a8d5) into master (9ce3077) will increase coverage by 4.89%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #348      +/-   ##
==========================================
+ Coverage   88.64%   93.54%   +4.89%
==========================================
  Files         123       60      -63
  Lines        3805     1378    -2427
  Branches      572      302     -270
==========================================
- Hits         3373     1289    -2084
+ Misses        392       88     -304
+ Partials       40        1      -39

Impacted Files
Coverage Δ

fixtures/common.py

news/urls.py

main/apps.py

main/admin.py

websites/apps.py

main/serializers.py

websites/__init__.py

websites/admin.py

main/urls.py

ocw_import/conftest.py

... and 51 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ce3077...876a8d5. Read the comment docs.

I'm still seeing a whole bunch of linting issues locally because apparently there's an issue with the ignore configuration in pylintrc: https://github.com/PyCQA/pylint/issues/2686
It seems that version 2.9 fixes this, but it's still pre-release, but I think we should probably upgrade when it's ready


Help: InsufficientQuotaError after following the quickstart guide

This is my code which I think is exactly what's in the quickstart guide
import { OpenAI } from "langchain/llms/openai";

const llm = new OpenAI({
  modelName: "gpt-3.5-turbo",
  temperature: 0.1,
});

const predict = async () => {
  const result = await llm.predict(
    "What would be a good company name for a company that makes colorful socks?"
  );
  console.log(result);
};

predict();

But I keep getting the error in the screenshot

I did exactly like the AI example and it never worked.  This will give you more information about your current usage and API limits/quota.

@kelvinsekx Have you looked at your OpenAI account usage? This will give you more information about your current usage and API limits/quota.

I recently just got to realize I need to subscribe to pay as you go plan of the API and OpenAI no longer have a free version.


feat: add TRY_PARSE_JSON

Pretty similar to -former- parse_json transform; wraps expression in TRY_CAST(<expr> as JSON).

Thank you!


List minimum specs for creating docker containers

even though in the docker machine commands and docker commands it is clear that min 8G is required for image should state that up front (would have told me I needed to run from 16G MBP rather than day to day 8G MBA)

@scanlonjts what types of issues were you seeing?
i run with 8 GB from time to time and, while not the snappiest, it's not dead in the water.
perhaps i've added something recently that is causing problems.  gimme some more deets - and i'll do some testing on my end.
there is definitely a slimmer profile that i can default to.  by default - and for demo purposes - i'm starting every service.  this is definitely not required, however.
we'll get it running for ya.

i was actually just doing the initial docker provisioning and my machine went into meltdown mode - MacBookAir - 8G all SSD - when VBox pushes too hard it goes into meltdown (I will setup on my other machine)

here's instructions on how to use a cloud instance like EC2:
https://github.com/fluxcapacitor/pipeline/wiki/Cloud-Provisioning
this should get you further, anyway.  :)


Recommended way for applying forces?

Is there a a good way to apply forces to a system? A use case is when you drag an element on a touch device and release it with a specific velocity. You want the element to being thrown into that direction.
Additionally an element could have another force that is being applied on it, so that it will eventually move to a specific end position.
Compare for example the photo gallery of the react-touch demo of Pete Hunt. When you flick with a higher velocity, multiple photos can be navigated at once.
Is there a recommended way for doing this with react-motion?

Yup coming up! @chenglou's fixing some important thingy inside the API so we're waiting on those before adding features.
It'll probably a prop like currVelocity={currVelocity => ...}

Great, thanks for the quick response!

I'm actually working on a photo gallery while exploring touch gestures and velocity. I'll ping you as soon as I get a gist up in the next couple of days.

Cool, thanks!
Btw. for inspiration for physics based animations the examples from Ralph Thomas are also very nice: http://iamralpht.github.io/constraints/ and http://iamralpht.github.io/physics/. If these can be rebuilt with react-motion then I don't know what else there could be in this lib. But he is actually lacking some features that react-motion is capable of, such as setting an end value.

Haha, I've talked to him, definitely knows what he's doing.
Hey @iamralpht, check out this new repo =).
My goal is to get as declarative as the constraint stuff, which I really appreciate. Unfortunately JS isn't expressive enough, and most people will be scared away by the constraint API so as a pragmatic reason I'll hold off. Now, if you could somehow express these constraints naturally using react's component hierarchy, I'm immediately deprecating this repo.
(Kidding, react-motion's awesome so far, but still.)

Any news on this? :)

Not yet; hopefully soon.

I'd love to see some way to do this too, just being able to set current state and velocity would cover my use case - I tried with setState but I'm guessing #88 prevents this working.

#252 might be a subset of this.

You can view his source to see his implementations of friction, gravity, etc.  They look fairly modular - composing them into react-motion might be a good first-step into moving beyond strictly springs.

I assume this has fallen off the map, but just as food for thought: what might this API look like? Is there any way to provide force/impulse in a declarative way?


TLSv1.3: Draft-20 implementation

This moves our TLSv1.3 implementation to draft-20. The main change is that all of the HKDF labels have been amended. I also updated the supported_versions version indicator to the draft-20 value.
As for draft-19 there are no official test vectors at the moment so tls13secretstest uses self-generated vectors (i.e. we are checking our own homework). It is entirely possible I messed up somewhere but the tests will still pass.
Each draft implementation is incompatible with previous versions. If nothing else this is because of the supported_versions number change, but in this case it is also because of the HKDF label change. In other words if anyone is using the draft-19 implementation then they will find it will no longer inter operate with this version.
Checklist

[ ] documentation is added or updated
[x] tests are added or updated

post to the tls@ietf list that master now does 20, once you merge?

The claim is that no support for the "post_handshake_auth" extension is needed at this time because we don't have an API for the server to send such CertificateRequests anyway?

The claim is that no support for the "post_handshake_auth" extension is needed at this time because we don't have an API for the server to send such CertificateRequests anyway?

Yes. Technically we were non-compliant before (we failed as a client if a CertificateRequest arrived post-handshake). Now we are compliant without having to do anything. :-)

Pushed. Thanks.


Make description of card ability stand out

[x] "On reveal" should be shown as "On Reveal"
[x] "Ongoing" should be shown as "Ongoing"
[x] "+3 power" should be shown as "+3 power" (+3 in green)
[x] "-1 power" should be shown as "-1 power" (-1 in red)
[x] "+1 energy" should be shown as "+1 energy"
[x] "+1 max energy" should be shown as "+1 max energy"
[x] All summon card names should be shown in capitalized and bold

[x] Widow's Bite
[x] Rock
[x] Rocks
[x] Doombot
[x] Squirrel
[x] Demon
[x] Drone
[x] Mjolnir
[x] Tiger

[x] "'limbo'" should be shown as "'Limbo'"

Done w/ #5 and f8672945000ce1d431da4af55363931e182307c8


Move to default branch

We shouldn't assume that master is the default branch. Cleaning up to remove this assumption.

This is not ready at all.

From: Joey Wunderlich<EMAIL_ADDRESS>Sent: Tuesday, August 4, 2020 12:57:57 PM
To: microsoft/pxt<EMAIL_ADDRESS>Cc: Peli de Halleux<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [microsoft/pxt] Move to default branch (#7231)
@jwunderl approved this pull request.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fpxt%2Fpull%2F7231%23pullrequestreview-461129008&data=02|01|jhalleux%40microsoft.com|86e8aaf6b588432258cf08d838b0af74|72f988bf86f141af91ab2d7cd011db47|1|0|637321678792583460&sdata=OX7YscysYT3%2BliNSPaKmzVyB%2FBMecd0bMh%2Fsmh7TK30%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAA73QKORF6E7KRM7LNEUAP3R7BR4LANCNFSM4OC5JGRQ&data=02|01|jhalleux%40microsoft.com|86e8aaf6b588432258cf08d838b0af74|72f988bf86f141af91ab2d7cd011db47|1|0|637321678792593455&sdata=ZMe%2BGR6vuBC%2B6XWIzJxYD0L9GackCFXimMbF2hL0moY%3D&reserved=0.

@pelikhan what is pending in this PR?
@aznhassan

Backend need to be able to pull the default branch when not specified.
Switch template to use "main" be default?
Otherwise lots of testing.


Add support for Alluxio Hive metastore

This PR adds a new Alluxio connector, which talks to Alluxio Catalog service which is available in recent Alluxio v2.1 release.
This connector is implemented based on Hive connector but using AlluxioHiveMetastore (subclass of HiveMetastore) to communicate with Alluxio catalog service. Here is a demo on using this connector in Presto.
This work is a collaboration contributed by Zac Blanco, David Zhu,  Haoyuan Li, Calvin Jia, Bin Fan

I haven't read the code yet, but why haven't you simply add new metastore to hive connector (like glue was added).

Also what are the key features of Alluxio metastore comparing to other metastore implementations?

I agree, this doesn’t seem to require being a separate plugin.
Can we add integration tests for this?

I haven't read the code yet, but why haven't you simply add new metastore to hive connector (like glue was added)?

This PR is mainly an implementation of the HMS interface. We took the same approach as Glue by implementing the HMS interface in the presto-hive module. There is a separate module with a POM and a single java file that we used for packaging the jar previously.

Can we add integration tests for this?

I agree - we'll get some added.

@apc999 Is Alluxio Catalog service simply a caching layer for Hive Metastore at this point or does it also bring extra features (e.g file stats caching)?

@electrum unit tests are added. For the integration tests, we are working on Alluxio repo to make something similar to hadoop minicluster work in external projects like Presto. We plan to add it back here in the future.

@apc999 What about docker image of Alluxio. That would be much better. We don't use Hadoop mini cluster in tests.

Thanks for updating this PR. Can you move the code so that it’s another metastore implementation for the Hive connector, like Glue, rather than a separate connector?

@apc999 Could you tell me how split addresses are generated when cache is on? Particularly,

What are the split addresses when object is not cached?
What are the split addresses when object is cached?
Is object cache local to Presto worker or is it distributed with respect to worker nodes?

@sopel39
To answer q1 and q2, when a database is accessed through alluxio catalog service, split addresses will always be alluxio URIs regardless of whether the file is cached in alluxio or not. If it is cached, accessing this alluxio URI will result in a direct read from alluxio memory, otherwise alluxio will fetch the data from the under file system (hdfs or object store) and cache it when presto first reads the data.
As for q3, right now there is no affinity between the cache and presto worker. The first time a presto worker reads certain block of data,  that data is cached on that machine. if subsequent table scans happen on the same machine, then it is a short-circuit read directly from the local machine, otherwise it will be a remote read.

@electrum I don't think the test failure is related to this PR, PTAL. Thanks

Merged, thanks!


Social share buttons

Can we please have social share buttons on each post?

Probably not in the existing template but if someone else wants to create a new one that has it, by all means do so.

can i work on this issue


Determine Stepsize of Transient Analysis

Hi,
is there a possibility to dertermine the stepsize of the transient analysis?
I thought the "step" variable in the constructor would do that but i tried Stepsize "double.MaxValue", "1e-127" and even "double.MinValue" and it didnt change the resolution of the results at all.
In your Tutorial "Getting started/Transient analysis/" you say that the given step size is used only to calculate the initial time step of the simulation. Does this mean i don't have the opportunity to set the real timestep?
Kind regards and once again thank you for your work and the really fast replys you are giving!

Hi!
Spice# (like all Spice simulators) uses a variable timestep which is calculated depending on the "error" it thinks it is making. To enforce a strict maximum timestep you can set the MaxStep property instead of just the initial Step property.

Thank you for your help!


Add build-gpep517 pipeline for Python

This PR adds a new Python pipeline called: python/build-gpep517.
In case if upstream project does not provide neither pyproject.toml or setup.py, so python/build and python/build-wheel pipelines does not work as expected since both pipelines uses build module and depends on aforementioned files to run correctly. Invoking the python-build with -m gpep517 rather than -m build, resolves this issue.

Why not just use gpep517 directly in python/build?  As far as I can tell gpep517 does handle legacy setup.py scripts?

As far as I can tell gpep517 does handle legacy setup.py scripts?

Hmmm, how can we be sure that it won't break the existing as-is flow? I was thinking to invoke gpep517 module if there isn't any setup.py.

Any thoughts? @kaniini
Does it make sense to check pyproject.toml and setup.py files whether exist, if so use as-is, otherwise fallback to gpep517 way?

How should we proceed here?
cc @imjasonh


Extras not installed by pip-sync

Extras are not installed by pip-sync when specified in a requirements.txt file. For example, adding the line example-package[extra] to the file, and running pip-sync, installs example-package but not the dependencies specified by the [extra] option.
I have noticed this when installing local editable packages. But it also happens when installing packages from PyPI, for example requests[security].
Environment Versions

OS Type: macOS
Python version: 3.9.15
pip version: 22.3.1
pip-tools version: 6.11.0

Steps to replicate

Create and activate a Python environment
Install requirements: python -m pip install pip-tools setuptools
Create a minimal package by creating a directory with a pyproject.toml:

[project]
name = "example"
version = "0.0.0"
dependencies = ["toml"]

[project.optional-dependencies]
extra = ["exceptiongroup"]

Create a requirements.txt with a single line:

-e ".[extra]"

Run pip-sync: pip-sync requirements.txt
Run pip list

Expected result
All dependencies specified by the [extra] option are installed (exceptiongroup in this case).
Actual result
Dependencies specified by the [extra] option are not installed (exceptiongroup in this case).

This is currently expected, as pip-sync is only designed to run against lock files. You might check out some discussion about it in not-accepted PR #907, which changes that behavior so that in your case all dependencies would be installed.

To be clearer:
The expected usage of pip-tools here would be to name that file requirements.in, use pip-compile on that to generate requirements.txt, then use pip-sync on that.

Thanks for the quick reply AndydeCleyre. Got you. I'll close this then.
I found this when I tried to modify a build process on an inherited project where this was not done. I'll have to modify it to match expected usage of pip-tools.


Question about formatting fat32

Hi
How specifically are you formatting these SD cards so that they meet the requirements of the fat32 library?
I can read the mbr (first sector and verify 55 AA ) but then I always fail after that. Doing a hexdump of the card shows the partition type as 0b (11).  Not 12.
If I go ahead and set the partition type to 11 and run again it fails on stage 2.  Last two bytes in the block are 55 and ff. I believe this to be the sector containing the BPB.

Hi David
Type 11 seems to be using CHS (cylinder/head/sector) addressing rather than
LBA. It seems very odd for an SD card. I'm surprised if any system actually
formats SD cards that way as it's quite an obsolete way of doing things, it
only really made sense on early physical hard disks.
I'm not sure what changes between the two types in practice, but I would
guess possibly the way the partition start is specified in the partition
table, and possibly some fields within the FAT32 filesystem itself.
Hopefully not too much though!
On Thu, 28 Jul 2022, 14:37 David Latham, @.***> wrote:

Hi
How specifically are you formatting these as cards so that they meet the
requirements of the fat32 library?
I can read the mbr (first sector and verify 55 AA ) but then I always fail
after that. Doing a hexdump of the card shows the partition type as 0b
(11). Not 12.
If I go ahead and set the partition type to 11 and run again it fails on
stage 2. Last to bytes in the block are 55 and ff. I believe this to be the
sector containing the BPB.
—
Reply to this email directly, view it on GitHub
https://github.com/gfoot/sdcard6502/issues/3, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHE43MZLZDVVULLDE6EDO3VWKEIDANCNFSM545JYN5A
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks for the reply.
I'm in the process now of zeroing the whole card to get rid of noise.  So I can actually match the bytes on the card with the spec.
I think fdisk allows for setting the partition table to lbs format.  I'll try that.

Yes you can just use fdisk or gparted to change the partition type, and
then run mkfs.vfat or whatever it's called to format it. I don't remember
whether I formatted mine under Linux or Windows. The built in format tool
in Windows actual can't format large SD cards as FAT32 anyway, you need
third party tools for that.
On Thu, 28 Jul 2022, 22:35 David Latham, @.***> wrote:

Thanks for the reply.
I'm in the process now of zeroing the whole card to get rid of noise. So I
can actually match the bytes on the card with the spec.
I think fdisk allows for setting the partition table to lbs format. I'll
try that.
—
Reply to this email directly, view it on GitHub
https://github.com/gfoot/sdcard6502/issues/3#issuecomment-1198657065,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHE43KBRRODCRYUDJWXN23VWL4KZANCNFSM545JYN5A
.
You are receiving this because you commented.Message ID:
@.***>

Okay. I got it working. I used my little netbook running debian to format the card with fat32 lba mbr and one partition. It's an 8gb sdhc.
I had to port your code to ca65 to get it to work on my setup. (I'm running OS/1 from dbuchwald).
I think some of the issues I was having were to do with trying to port the code. After a break and a redo, it's all working. I've learned loads from this project. Thanks very much.


Doco update - add URL for Discovery Consulting and Yelcho

@grahamrobbo needs to get HTTPS sorted for his website!
Sorry Robbo, I may have just removed another chance for you to be a contributor to the ABAPgit project... perhaps get HTTPS sorted and then you can do another pull request 😃

So embarrassed by my web site #justsayin 😊


Netty Webpage Fault [Not Code]

Netty Adopters page has an EpollServer example in the bottom which I should not be there as per my knowledge.
Screenshot attached.

@normanmaurer Please have a look

All netty.io/wiki pages apparently:
https://netty.io/wiki/writing-a-commit-message.html

Yeah, I know that. But I found EpollServer example at

the bottom of https://netty.io/wiki/writing-a-commit-message.html also. Interesting,

It looks like EpollServer example is pretty much everywhere on the website at the bottom.

Whatever is generating the HTML content for <div id="wiki-footer" class="mt-5 mb-0 wiki-footer gollum-markdown-content"> is including it somehow.

I checked Revisions of Wiki pages and looks like someone added it for no reason. Maybe we should put restrictions on who can edit Wiki pages.

Seems like this was fixed by the last deploy of the website

How about adding some restrictions on Wiki so only selected people can change it?


Why Use Pump? documentation

Looks good to me.  I'll adjust the commit messages during the merge to keep the commits separate; However, my laptop battery is dead now.

Rebase-merged as 45adfc35aeeeaac5678ee63d20a6cdf985f6ceaf and 24914f31a8fe2125d9c66469d19f8e4b2587ecc6
❤  ❤  @terinjokes


Can't change mRowHeaderWidth (private field in class)

I need a way to change the width of all row headers, but right now I can't find an easy way of doing this without needing to extend the TableView class and overriding nearly all layout methods (due to the way mRowHeaderWidth is used and the fact there is no way to change the value)

Hi @PerMagnusH,
TableView has got a method which is public for this purpose.
tableView.setRowHeaderWidth(int rowHeaderWidth)


Validate HCA Data Repository Projects for source datasets contain primary data (FASTQ files)

Need
The HCA Data Repository is meant to hold the primary data (FASTQ) files associated with the source datasets. We will validate that this is true and report on it as one of our validations.
Approach
Update the refresh service to indicate if the HCA Data Repository project has primary data.  This can be done by examining the Azul projects response and looking at the fileTypeSummaries of a project response hit and looking for format to be fastq or fastq.gz.
For example

Complete, Thanks!


GraphQL not building schema

Description
Even with the default starter project the GraphQL schema has nothing more than path and context. I can't grab id, path, title, slug, content, date or anything else. It seems I have done something wrong, but I followed the install step by step, several times with no luck.   I don't know that there's any other information I can give.
Steps to reproduce
Install Gridsome as defined on the Gridsome site.
Expected result
GraphQL schema should display all the nodes in this query:

query {
  allPage {
    context
    id
    type
    title
    slug
    path
    component
    pageQuery {
      type
      content
      options
    }
    content
    date
  }
}

Actual result
The schema returns:
path
context
Even this query returns errors after a "stock" install

query {
  allPage {
    id
    title
  }
}

Environment

Libs:
@gridsome/cli v0.2.1
gridsome v0.6.6

Browser:
- [x] Chrome (desktop) version 75.0.3770.142
- [ ] Chrome (Android) version XX
- [ ] Chrome (iOS) version XX
- [ ] Firefox version XX
- [ ] Safari (desktop) version XX
- [ ] Safari (iOS) version XX
- [ ] IE version XX
- [ ] Edge version XX

For Tooling issues:
- Node version: 12.4.0
- Platform:  Mac

The Page type in the schema is very limited and not very useful atm. It just lists the paths for the registered pages on your site :) You should instead add your own data with custom fields etc. It can either be done with source plugins or manually with the Data Store API.

OK. That helps.
If I want to avoid .md files, will Gridsome still dynamically generate routes for .vue files in the /pages directory? Because I'm not able to access anything in GraphQL when trying to run v-for to build out the menu. I guess I'm dense, not getting how to get the nodes into GraphQL.
I realize that I need to create the data structures, but is this not allowed?

use: '@gridsome/source-filesystem',
      options: {
        path: 'docs/**/*.vue',
        typeName: 'CodeSample'
      }

Yes, Vue components in src/pages will have their own routes automatically. But it's not possible to automatically generate a menu for them atm. So I would recommend settings the menu up manually instead for now. We might allow some front matter data (like title etc.) for the pages that will be available in the schema, but it's not implemented yet.
Using *.vue files with the @gridsome/source-filesystem plugin won't work because there is no transformer that can parse the files.

Yes. I figured that out last night. Thanks for confirming.
However, I did discover the mdvue plugin that's being built and I might be able to use that. It looks nearly ready, do you have a potential release date? In the meantime, I will just build out the menu manually.


Dataflow error in a default of a type-ascribed argument results in a type error instead of being propagated

There seems to be some issue when a default argument value returns a dataflow error and that argument has a type ascription resulting in a typecheck. Weirdly, it is not happening always. Below I present a repro where it consistently (wrongly) fails with a type error, but later I also show attempted repros where it actually works as expected.
Run the following repro (it assumes code from #7072 - which is waiting to be merged as of this moment):
from Standard.Base import all
import Standard.Base.Runtime.Context

from Standard.Table import Table, Value_Type

from Standard.Database import Database, SQLite, In_Memory, Column_Description, Update_Action
from Standard.Database.Internal.Upload_Table import default_key_columns

main =
    c = Database.connect (SQLite In_Memory)
    Context.Output.with_disabled <|
        t = c.create_table "table123" [Column_Description.Value "Y" Value_Type.Char] primary_key=Nothing

        r0 = default_key_columns t
        IO.println r0

        src = Table.new [["Y", ["d", "e", "f"]]]
        r1 = src.update_database_table t Update_Action.Insert
        IO.println r1

Expected behaviour
It should print the same dataflow error twice (in the r1 call, the dataflow error from the default value should get propagated to the return value).
(Error: (Illegal_Argument.Error 'Could not determine the primary key for table enso-dry-run-table123. Please provide it explicitly.' Nothing))
(Error: (Illegal_Argument.Error 'Could not determine the primary key for table enso-dry-run-table123. Please provide it explicitly.' Nothing))

Actual behaviour
Instead, in the update_database_table case, it results in a type error:
(Error: (Illegal_Argument.Error 'Could not determine the primary key for table enso-dry-run-table123. Please provide it explicitly.' Nothing))
Execution finished with an error: Type error: expected `key_columns` to be Vector | Nothing, but got Error.
        at <enso> Table.update_database_table(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Database\0.0.0-dev\src\Extensions\Upload_In_Memory_Table.enso:123-124)
        at <enso> weird_type_error.main<arg-1>(weird_type_error.enso:18:14-61)
        at <enso> Runtime.with_disabled_context(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Base\0.0.0-dev\src\Runtime.enso:228:5-81)
        at <enso> Context.with_disabled(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Base\0.0.0-dev\src\Runtime.enso:181:9-79)
        at <enso> weird_type_error.main(weird_type_error.enso:11-19)

It looks like the type check introduced by the ascription key_columns : Vector | Nothing somehow ignores the fact that the argument is a dataflow error to be propagated and sees that an 'Error' does not (obviously) fit the expected type.
I have spent some time trying to figure out a simpler example, but failed to do so. This bug seems to trigger under some specific conditions. But the repro above works reliably for me (100% replayability).
Below is a repro that didn't work (i.e. it actually works as expected):

Details
from Standard.Base import all
import Standard.Base.Errors.Illegal_State.Illegal_State

default_value t =
    if t == "should error" then Error.throw (Illegal_State.Error "error in defaults") else ["X", "Y"]

foo (t : Text) (x : Vector | Nothing = default_value t) =
    t + " - " + x.to_text

type Bar
    Value y

Bar.baz : Text -> Vector | Nothing -> Text
Bar.baz self (t : Text) (x : Vector | Nothing = default_value t) =
    t + " - " + self.y.to_text + " - " + x.to_text

main =
    #IO.println (foo "X")
    #IO.println (foo "should error")

    b = Bar.Value 42
    #IO.println (b.baz "X")
    IO.println (b.baz "should error")

and another one
from Standard.Base import all
import Standard.Base.Errors.Illegal_State.Illegal_State

foo_unchecked x =
    10 + x + 100

foo_checked (x : Integer) =
    10 + x + 100

compute_default x =
    if x == 10 then Error.throw (Illegal_State.Error "ISE: from defaults") else x*10

bar_unchecked x (y = compute_default x) =
    x + y

bar_checked (x : Integer) (y : Integer = compute_default x) =
    x + y

main =
    err = Error.throw (Illegal_State.Error "ISE: error message")

    r1 = foo_unchecked err
    IO.println r1

    r2 = foo_checked err
    IO.println r2

    r3 = bar_unchecked 1
    IO.println r3

    r4 = bar_checked 1
    IO.println r4

    r5 = bar_unchecked 10
    IO.println r5

    r6 = bar_checked 10
    IO.println r6

So the bug only seems to occur under some specific conditions, that I could not track down yet.

I've just executed:
enso$ git log | head -n 2
commit b243a5f5292bf01c810cd37ecd2f04ee751c1601
Author: somebody1234<EMAIL_ADDRESS>enso$ cat >test.enso
from Standard.Base import all
import Standard.Base.Runtime.Context

from Standard.Table import Table, Value_Type

from Standard.Database import Database, SQLite, In_Memory, Column_Description, Update_Action
from Standard.Database.Internal.Upload_Table import default_key_columns

main =
    c = Database.connect (SQLite In_Memory)
    Context.Output.with_disabled <|
        t = c.create_table "table123" [Column_Description.Value "Y" Value_Type.Char] primary_key=Nothing

        r0 = default_key_columns t
        IO.println r0

        src = Table.new [["Y", ["d", "e", "f"]]]
        r1 = src.update_database_table t Update_Action.Insert
        IO.println r1
^D
enso$ sbt
sbt:enso> runEngineDistribution --run test.enso
Nothing
(Database Table enso-dry-run-table123)

e.g. I don't see any error.


Create .htaccess

@aidan-casey @innocenzi do any of you have up-to-date knowledge about apache? I'm fine merging this, but don't understand what it does 😅

@brendt it checks if the path is an existing file or directory. If not, it sends the request to the index file. I'd be fine merging this for alpha, but in the long run I think we should be writing a deployment guide for Apache and Nginx.

@brendt ..., but in the long run I think we should be writing a deployment guide for Apache and Nginx.

A first small step:  https://github.com/tempestphp/tempest-docs/pull/40  @defautset @aidan-casey


Using order by might lead to wrong results if the limit of 10000 is reached

Due to the fact that order by is executed in SquashQL and not in the DB, if the result contains more than the allowed limit (10K), the returning result might be incorrect.

https://stackoverflow.com/questions/7802944/explicitly-specify-sort-order-for-mysql-query


Add support for AllowFileAccessFromFileURLs (Android)

There is support for allowUniversalAccessFromFileURLs, but not for allowFileAccessFromFileURLs.
Opening a local file (file://) works fine on iOS, but not in Android. On Android it is needed to set this flag. See Android API
It seems like a small fix to support this.

@all-contributors please add @femiveys for code

I can't find the Pull request related to this?

I tried to push it but I don't have permission
Op do 20 jun. 2019 om 14:33 schreef Thibault Malbranche <
<EMAIL_ADDRESS>
I can't find the Pull request related to this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/react-native-community/react-native-webview/issues/654?email_source=notifications&email_token=AAJR2N52IND6PK275FELQN3P3N2J3A5CNFSM4HZJRFQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYFI2GA#issuecomment-504007960,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJR2N4UGXQCFI6SV4Q35ZDP3N2J3ANCNFSM4HZJRFQQ
.

you need to fork and create a pull request from your fork :)

Im happy to put a pr in for this if no one else is started it?

I have done a PR for this: https://github.com/react-native-community/react-native-webview/pull/695

@femiveys WebView already has the prop allowFileAccess to do this

I don't think that's the case:
setAllowFileAccess and setAllowFileAccessFromFileURLs are 2 different things. See links to Android API.

You're right, my bad.


Export: export options in "share"/"view"/"plan chooser" are inconsistent

"Request file" terminology inconsistent with "export menu" which calls same choice "raw data"
"PDF" and "Shapefile" options should be available from plan chooser and share -- no reason to force a user to view the plan just to export in a different format.

Since we now have an Export menu on the map view, may we remove the export buttons from the plan chooser?
It seems like this would be a good compromise, because the export menu contains visual exports -- exporting the spatial data as well as the raw data. For users, I think it would make more sense to be able to export a plan after confirming that the representation of the plan on the map (visually) is accurate, or at least the expected result, as opposed to exporting from the plan chooser or the share tab.
Exporting from the chooser or the share tab gives a limited amount if information to the user, such as plan name, description, and # of districts. However, exporting from the map view shows which districts are missing/at targets, etc. It seems more useful in the scenario when you are viewing the map.

I confered with Mike, and we don't want to see it removed at this
point. Moving it off the plan chooser complicates bulk (which we'd
like to enhance at some point see 147), and also means that if you are
logged in you have to copy the plan just do download it.  The latter
wouldn't be an issue if the 108 enhancement were in place.
Maybe we should defer this to our call, and think about how
108/147/254/257 fit together and what makes sense to do now, and would
be most consistent with future desired enhancements?
On Wed, Dec 7, 2011 at 4:00 PM, David Zwarg
<EMAIL_ADDRESS>wrote:

Since we now have an Export menu on the map view, may we remove the export buttons from the plan chooser?
It seems like this would be a good compromise, because the export menu contains visual exports -- exporting the spatial data as well as the raw data. For users, I think it would make more sense to be able to export a plan after confirming that the representation of the plan on the map (visually) is accurate, or at least the expected result, as opposed to exporting from the plan chooser or the share tab.
Exporting from the chooser or the share tab gives a limited amount if information to the user, such as plan name, description, and # of districts. However, exporting from the map view shows which districts are missing/at targets, etc. It seems more useful in the scenario when you are viewing the map.

Reply to this email directly or view it on GitHub:
https://github.com/PublicMapping/DistrictBuilder/issues/254#issuecomment-3053347

--

Micah Altman, Ph.D. http://redistricting.info           Twitter: @drmaltman
Senior Research Scientist, Institute for Quantitative Social Science, Harvard U.
Director of Archiving and Acquisitions, IQSS;
"Entia non sunt multiplicanda sine necessitate" - Dr. Invincibilis
(Corollary, "Ad indicia spectate.")

Mostly updated in other tickets. Still minor inconsistency -- is it possible to add shapefile download in plan chooser and share tab?


"Neither" Option in Contest

sometimes ya get those cards that you really dont like and i dont wanna vote for either of them

Hi,
This has been removed from Contest some weeks ago (you were able to just hit F5 and vote again), now after skipping at least 5 votes you will be prompted with one of the skipped votes until you finally vote again.
The main idea was to avoid too much cheating, since some people were able to run bots that would just skip every vote until a specific idol or card showed up.
I hope you understand, I am still looking for a better solution that would enable voters to skip some votes, but that's the better solution I come up with for now.
Cheers,


Added more details

added further detail to intraframe direction prediction
Added some more resources, most notably wiki.multimedia.cx and the HEVC book
The company was not called TrueMotion, TrueMotion was the name of the first codec the Duck Corporation produced
Further clarification on the differences between h264/h265.

added jongbel h264 analyzer. Its a neat tool for seeing h264 streams, and it even comes with its own nifty decoder. It could be useful now that the intel pro analyzer link has been dead for some time. The free trial works great.


Look for GPG agent socket also in $XDG_RUNTIME_DIR/gnupg

It is implemented in similar way here https://github.com/maximbaz/yubikey-touch-detector/blob/ce2874a7a89dd7141195744a39bbd537ee243d24/detector/ssh.go#L22

@felixfontein any chance to get it merged?

@getsops/maintainers can someone else please take a look at this? I think this looks fine, but I don't know what impact //go:build !appengine has for example. Thanks.

//go:build !appengine was removed since it is not required.

@tomaszduda23 thanks for your contribution! And sorry it took so long...


Serialization Optimization potential

I have learned a lot on performance by checking this class out but also found a few places where it could be further optimized based on my tests. Important: I am testing on ABAP 1909 Developer Edition running on Docker Desktop on Windows 11 with an AMD CPU --> about as far from typical SAP ABAP systems as possible, therefore some conclusions might not apply to real world systems.

Timestamp conversion to ISO format can be enhanced. Seems that the multiple access to the string with offsets is bad for performance. Using two helper variables (char 15 for normal timestamp char 23 for long version) is much faster. I assume this has to do with the string optimization limitations of the ABAP Kernel.
Concatenate seems to perform worse if string and character are mixed. Therefore switching from `string` literals to 'character' literals for the timestamps, date and time concatenates is faster.
Added an initial check for cl_abap_typedescr=>typekind_char (outside of boolean) to avoid calling the escape logic for empty values.
Added a check to escape_json to only escape in case any escape-character is present. This is done with a CA-compare. Since the two-character cr_lf contains the one-character newline we only need to check for those four characters (assumption without knowing it is that cr_lf contains newline on any supported architecture). Will be worse if the majority of escape_json really need to be escaped but I assume that the majority of calls will not need ot be escaped.

As a rough figure when parsing 50000 sbook entries those updates reduce execution time for me by ~35%. When just parsing a table with timestamps the timestamp optimization reduces the overall runtime by ~25%.
I have a branch with those changes on my fork https://github.com/bpreissler/abap-to-json/tree/Serilaization-Optimizations.

Hi @bpreissler ,
First of all, thanks a lot for the feedback, analysis, and suggestions!
Second, sorry for the late reply - I was on vacation, but now I am back :)
Regarding your suggestions:

I think the reason for the slow performance on serialization of timestamps is not the cost of string access, but missing string concatenation optimization, which is not applied because the target string is used also as a source. I have corrected that using helper string variables.
The problem with concatenation, IMHO, is not a mixture of types, but in the usage of string literals when they are not necessary. In C++ (ABAP Kernel) strings are allocated on the heap, but text field literals, with a fixed length, are allocated on the stack. Allocation on the heap is much slower than on the stack. But the recommendation is correct. So, I have replaced this usage of string literals and others, where it is not optimal, to use text field literals.

Was already corrected by me, but was not published yet. Now the changes are also out.

Can you please validate the latest state and give feedback?
And please share your test report, so that I can use it on my side also.
BR, Alex.

Hi @alexey-arseniev,
thx a lot for the explanations.
I have uploaded the test class https://github.com/bpreissler/abap-to-json-perf (if you wonder I have updated/modified my /ui2/cl_json to be on the current PL18 update so that I can use this as comparison to my z_ui2_json changes).
Regarding point 2 & 3 that looks good for me, too.
Point 1 - the timestamps ... at least in my environment the difference between the helper variable you introduced and c(15) / c(23) fields is still massive for me with ~25% on my timestamp table test. Maybe the string optimization is just not working properly in my environment.
Regards,
Bastian

Hi Bastian,
thanks for the test report! I was able to validate the change and see that you are right. Working with text fields helper variables is much faster compared to strings.
I have updated the code and pushed a new version. Please validate.
BR, Alex.

Hi Alex,
looks great, also works for me.
The only last thing that comes to my mind would be to move the conversion logic determination up to the structure element type determination so that all the checks on type and conversion settings are done only once. That way a bunch of IF's would be skipped and there would be less options for the case statement. However as those should be quite fast I assume this would maybe lead to a lower one-digit improvement at the cost of a lot of added complexity which like doesn't make it worth going for.
I think we can therefore close this issue.
Regards,
Bastian

Hi Bastian,
Yes, you are right, the conversion detection can be taken out and recalculated for structures. This will bring benefits for big tables.
I have done it (pushed it to the main), please verify. Your report shows significant improvements.
BR, Alexey.

Wow, the difference is bigger than I expected. Great :) No more ideas to speed the serialization up.


Use a separate page rather than a modal for sign in/register

A few reasons for this:

We need to be able to redirect people to sign in where necessary.
Doesn't work well on mobile.

Done


"#if XIP" block in qemu_x86 DTS always evaluates to false

The intent here seems to be to properly set memory if CONFIG_XIP=y, but the qemu_x86 target does enable CONFIG_XIP, yet the amount of SRAM gets set to 8188K anyway.
Causes hard to debug runtime hangs if the amount of RAM used exceeds 4096, instead of failing at build time.

Not sure how this snuck in, but we don't allow use of CONFIG_ ifdef's in dts anymore.

This is not a CONFIG_* and it was there for almost 2 years, without any effect basically. We do not define XIP anywhere AFAIK
#if XIP
        #define DT_SRAM_SIZE            DT_SIZE_K(4096)
#else
        #define DT_SRAM_SIZE            DT_SIZE_K(8188)
#endif


Weird if statements in SimpleImputer.py

I was browsing some code in SimpleImputer.py and I really have no idea how this multi-line double if statement with a 0 != 0 in it works? Little cleaning job I guess

https://github.com/automl/Auto-PyTorch/blob/master/autoPyTorch/pipeline/components/preprocessing/tabular_preprocessing/imputation/SimpleImputer.py#L91

if len(dataset_properties['numerical_columns']) \
        if isinstance(dataset_properties['numerical_columns'], List) else 0 != 0:
    add_hyperparameter(cs, numerical_strategy, CategoricalHyperparameter)

solved


Log in - Error message “Too many attempts doesn't appeared and info is blinking when tap on resend link

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Launch the app
In the login screen, enter an email that doesn't have an Expensify account yet
In the "Resent link" page - Disable the internet connection in the device
Go back online and click/tap on the "Resend Link" button
Verify you see “Link has been resent” with a green dot on the left
Verify you received 2 magic link emails in your email inbox - Do not validate the account yet
Back in NewDot, click “Resend Link” a bunch of times
Verify you see an error message “Too many attempts, please wait a few minutes and try again” with a red dot on the left

Expected Result:
An error message “Too many attempts, please wait a few minutes and try again” displays with a red dot on the left
Actual Result:
Error message “Too many attempts doesn't appeared and info is blinking when tap on resend link
Workaround:
Unknown
Platform:

Where is this issue occurring?

iOS
Android

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: No
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/206574494-168c4c2f-5201-40c3-a8fe-81cdeb499aab.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

QA team is Fail steps in TR https://expensify.testrail.io/index.php?/tests/view/2990810

I'm not able to reproduce this. Steps I'm taking

Go to https://staging.new.expensify.com/ in desktop
In the login screen, enter an email that doesn't have an Expensify account yet
Click Resend link page
Get a "You've reached the maximum login attempts, which is 3. Please wait a few minutes and try again."
This message appears when online or offline - not glitching

iOS: v1.2.36-4

Go to https://staging.new.expensify.com/ in desktop
In the login screen, enter an email that doesn't have an Expensify account yet
Click Resend link page
Get a "You've reached the maximum login attempts, which is 3. Please wait a few minutes and try again."
This message appears when online or offline - not glitching

@kbecciv Can you please clarify the steps so I can try and replicate them?

@Christinadobrzyn This issue reproduced only in IOS and Android app, which environment did you try?

Steps are listed correct, also you can try using steps following the TR steps as well https://github.com/Expensify/App/issues/13447#issuecomment-1343419057

I can replicate following the TR steps. I don't see the 'too many attempts' message is blinking and the red dot seems to indicate an issue (there's a green dot when something is successful). Can you clarify the issue?


Shouldn't it be a System text instead of Staff text??

Since the tempo change will apply to the whole system anyway.

The tempo elements are, the indicative text isn't as it is not possible in the plugin framework to create system texts.
Once MuseScore allows us to create this as a system text, I will adjust the plugin accordingly

See also https://musescore.org/en/node/120256 where this was first investigated/discussed.


Move provider examples to doc book

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Checklist:

[ ] squashed commits into logical changes
[ ] includes documentation
[ ] adds unit tests
[ ] adds or updates e2e tests

@alexander-demicev this needs rebase also


sap.m.DateTimePicker: AM/PM not localized for Korean locale

OpenUI5 version: latest
Browser/version (+device/version): N/A
Any other tested browsers/devices(OK/FAIL):
URL (minimal example if possible): https://openui5.hana.ondemand.com/explored.html?sap-ui-language=ko#/sample/sap.m.sample.DateTimePicker/preview
User/password (if required and possible - do not post any confidential information here):
Steps to reproduce the problem:

Bring up sample and click on calendar icon to bring up DateTimePicker
Observe that the AM/PM text is not localized

What is the expected result? Expected that AM/PM would be localized for Korean locale
What happens instead? English AM/PM is shown instead.
Any other information? (attach screenshot if possible) Appears that other locales (e.g. https://openui5.hana.ondemand.com/explored.html?sap-ui-language=zh_CN#/sample/sap.m.sample.DateTimePicker/preview) are localized.

Hi @cmastrandrea ,
Thank you for your finding. I have created an internal incident:<PHONE_NUMBER>
Regards,
Diana

Hello,
Seems that the issue is fixed. I am closing the issue.
Best regards,
Tereza


Decide whether to base the devcontainer on mcr.microsoft.com/devcontainers/python

See https://github.com/devcontainers/images/tree/main/src/python
Pros:

Already has zsh so should be a lot quicker to start a new devcontainer (with common-utils feature) - needs testing
Contains additional tools like pipx

Cons:

More change
Another tie into Microsoft ecosystem

@callumforrester @garryod thoughts?

Don't think the tie in would be too strong, but generally prefer composition (features) instead of a monolithic base container. Then again, features can be rather slow

I agree with @garryod on tie in. The slowness of features has definitely reduces adoption in DAQ so it is something we're looking to do something about.
An alternative might be to build our own base image out of precisely the features we want and use that?

I think the microsoft one is precisely that, base python image with common-utils in it...

Benchmarking the existing container with just dev dependencies (python-copier-template-example):

From "rebuild container without cache" to venv installed: 1m59s
From "rebuild container" to venv installed: 1m19s
If we remove `common-utils":
From "rebuild container without cache" to venv installed: 1m06s
From "rebuild container" to venv installed: 36s
But no zsh
If we switch to mcr.microsoft.com/devcontainers/python:
From "rebuild container without cache" to venv installed: 1m43s
From "rebuild container" to venv installed: 38s
Need https://github.com/gilesknap/docker-outside-docker/blob/9ca46bf22c0daa11cebed1259a476e810b6ad550/.devcontainer/devcontainer.json#L36-L39
And we get "WARNING: Running pip as the 'root' user can result in broken permissions" for some reason in the postCreateCommand even though the pip install in the terminal is ok

I think common utils is giving more than just zsh. Maybe.

Do you get your ssh keys and github access inside the container without them?
Also X11 forwarding may or may not work without common utils

We used to have manual workarounds for the above but I prefer the native MS approach if possible.

I think common utils is giving more than just zsh. Maybe.

Do you get your ssh keys and github access inside the container without them?
Also X11 forwarding may or may not work without common utils
what about other git global config ?

We used to have manual workarounds for the above but I prefer the native MS approach if possible.

Above is not true as I tried it here https://github.com/gilesknap/docker-outside-docker/tree/no-common-utils.
If you do not use MS base image and remove common-utils feature you need only add an install of git and all of the above still work.

One thing that helps massively is going back to how it was originally, removing the feature, and installing it manually in the Dockerfile:
RUN git clone https://github.com/devcontainers/features --depth=1 && USERNAME=none bash features/src/common-utils/install.sh

From "rebuild container without cache" to venv installed: 1m36s
From "rebuild container" to venv installed: 35s

If we switch to uv too then then we can shave another 20s off this
The git clone is a hack, we should really be extracting from the OCI artifact at ghcr.io/devcontainers/features/common-utils:2, but I have no idea how to do that!
@garryod any ideas?

One thing that helps massively is going back to how it was originally, removing the feature, and installing it manually in the Dockerfile:

@coretl would it be even faster if we did that and prebuilt it remotely and pulled from Github?

I would rather avoid maintaining a container build for every version of python we want to support that triggers a rebuild each time there are security updates... If we only pay the 1min cost on first run of the dev container that doesn't seem too bad to me...
If we want to go down the container build root then we should use the devcontainers cli to make these in the copier template CI, but we need to work out when to trigger them...

Ok, here is the version with oras:
# Add common utils features
COPY --from=ghcr.io/oras-project/oras:v1.2.0 /bin/oras /bin/oras
RUN mkdir /tmp/common-utils \
    && cd /tmp/common-utils \
    && /bin/oras pull ghcr.io/devcontainers/features/common-utils:2 \
    && tar xvf devcontainer-feature-common-utils.tgz \
    && USERNAME=none ./install.sh

I'm now pretty certain this is not the right path to take. What we are after is a good out of the box experience that can be customized by the user (prompt, dotfiles, etc.). VSCode has 2 ways to do this, features and dotfile support.
Some possible routes:

Remove all features from the project, and provide instructions to add user features (and plugins) with at least one sensible default setup (based on zsh) that people can copy and paste with a prompt, completion and persistent history
As 1 but put some useful apt install commands in the dockerfile (like zsh) so that they are cached
As 1 but make the default setup based on bash instead

I've got a mild preference for 3, then shipping the result as a python-copier-template devcontainer feature, but then the question will be which prompt to use.
I'm also not sure how zsh completion works, and whether bash completion can be made to be as good as it with a bit of configuration...
@garryod @callumforrester @gilesknap @GDYendell thoughts?

Some useful links I found:

A persistent bash history feature
A minimal zsh feature
A one-liner installer for zsh and oh-my-zsh that could go in a dotfile
A suggestion that common-utils is too big and should be broken up

Hmm, sorry about this @gilesknap but I think we should drop ZSH. The rest of this template is very opinionated and goes the route of "there is one supported way to do this". Given that supporting ZSH and bash is already causing complexity I think we should only support one and it should be the one that more people are going to use.

Hmm, sorry about this @gilesknap but I think we should drop ZSH. The rest of this template is very opinionated and goes the route of "there is one supported way to do this". Given that supporting ZSH and bash is already causing complexity I think we should only support one and it should be the one that more people are going to use.

@callumforrester I totally agree. This discussion has made it clear that the only thing we are really getting from common-utils is zsh.

People who want zsh can always add common-utils via "dev.containers.defaultFeatures" in user settings anyway...

Will adding common-utils in user settings make the start up as slow as it is currently?
I have until very recently favoured the simplicity of bash in devcontainers, but now I am pretty set on zsh having adopted some modern terminal features that are too painful to make work in bash, so I expect I will do that. Or do it manually in my devcontainers dotfile install script, which has no caching.

Will adding common-utils in user settings make the start up as slow as it is currently?

Yes, unless you choose a container image that already has zsh and oh-my-zsh in it... You may be better off doing the one-liner installer in your dotfile as that is a little bit faster, but it will still be done on every container launch

having adopted some modern terminal features that are too painful to make work in bash

Interesting, which features are those?

I've made a PR to integrate the new bash-config lightweight feature which tries to reproduce the best aspects of having zsh without requiring the expense of common-utils.
See https://github.com/DiamondLightSource/python-copier-template/pull/216

The pip install in onCreateCommand is now leading the bottleneck charts maybe 'uv' is our next step?

I think uv deserves its own issue, also unless we can somehow alias it to pip I think it's a breaking change (so copier template 3.0)


[WIP] Fix propagation of grad_combo_fn

Summary
Fix the propagation of grad_combo_fn.
Details and comments

This would actually be much easier resolved (and be a cleaner design) by having a state dictionary (or some other name). So instead of
            return ListOp(oplist=self._oplist[offset],
                          combo_fn=self._combo_fn,
                          coeff=self._coeff,
                          abelian=self._abelian,
                          grad_combo_fn=self._grad_combo_fn)

we would have
            return ListOp(oplist=self._oplist[offset],  **self._state)

This would also mean that adding a new property means changing the code in only one place instead of everywhere where a ListOp is created.

This would actually be much easier resolved (and be a cleaner design) by having a state dictionary (or some other name)... we would have

 return ListOp(oplist=self._oplist[offset],  **self._state)

If  _state is a property of self you can make build this dictionary on the fly from the self so like you want there is just one place to update the i,e, the property state which builds this out. Or just have a method that you pass the new primitive and it builds the new ListOp with that using values for the other params from self and passes it back.


undefined method `auth_at_provider_path'

As in the tutorial and and example app I place it in my view:
<%= link_to 'Login with Facebook', auth_at_provider_path(:provider => :facebook) %>
and I actually could not find the place where it is defined neither in sorcery code, nor in example app code.

sorry, didn't configure oauth, need to delete


Custom mod blocks breaks textures when Terrain.png uses textures above 16x (Better than Adventure)

Bottom image is rose and cobweb in world.
To recreate, increase Bta's Terrain.png resolution by 200%  with nearest neighbor, use it via texture pack, and install the "Cave Uberhaul!" Mod from the BTA discord.
Cave uberhaul creator was contacted; after discussion "Do you think it's a bug with halplibe then?" "yea pretty sure. I just changed the order in which the custom blocks get registered, the textures are still messed up in 32x but now theres different blocks overlapping with the flowers etc"

The image with the area in red is the 2x scale Terrain.png. The other image with the area in the selection highlight is the area that Terrain.png at it's native resolution overlaps with 2x scale Terrain.png when anchored the top right corner. It's my speculation that the texture placing logic is simply not accounting for the change in resolution scale, and writing in areas that would normally be empty on a 1x scale Terrain.png.

Should be fixed in v2.3.0 (#12)


Authentication succeeds for users with incorrect passwords

I'm using Nginx 1.4.6 and the latest version of this module, and noticing that passwords appear to not be being validated.
Password validation is correct when using a config like:
ldap_server local {
    url ldaps://ldap.example.org/ou=Users,dc=example,dc=org?uid?sub?(objectClass=person);
    require valid_user;
}

An attempt to use a valid account with a false password fails (re-asking for authentication with a 401), and a valid account with a correct password succeeds.
If I use this, however, authentication succeeds for a user matching the given conditions, but no password validation is applied:
ldap_server local {
    url ldaps://ldap.example.org/ou=Users,dc=example,dc=org?uid?sub?(objectClass=person);
    group_attribute "uniqueMember";
    group_attribute_is_dn on;
    require user 'uid=user1,ou=Users,dc=example,dc=org';
    require user 'uid=user2,ou=Users,dc=example,dc=org';
    require group 'cn=myorg,ou=groups,dc=example,dc=org';
    satisfy any;
}

Auth succeeds if the user ID in a request is set as either user1, user2, a user that's part of the given group, but it succeeds with the password set as anything.  Auth fails, however, if say user3 is attempted, seemingly because the user fails to match the require statements.
I'm not yet running a recompiled Nginx with debug enabled, but I will shortly.  No errors or the like appear in the console output for a non-daemon Nginx compiled normally.

hi @davidjb
I will appreciate any direction towards the solving and assist me to reach the goal for centos7.
I'm following and using https://github.com/kvspb/nginx-auth-ldap . So my docker file looks like this.
RUN cd ~
&& git clone https://github.com/kvspb/nginx-auth-ldap
&& git clone https://github.com/nginx/nginx
&& cd /root/nginx
&& /root/nginx/auto/configure
--add-module=/root/nginx-auth-ldap
--with-http_ssl_module
--with-debug
--conf-path=/etc/nginx/nginx.conf
--sbin-path=/usr/sbin/nginx
--pid-path=/var/log/nginx/nginx.pid
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--with-stream
--with-stream_ssl_module
--with-debug
--with-file-aio
--with-threads
--with-http_gunzip_module
--with-http_gzip_static_module
--with-http_v2_module
--with-http_auth_request_module
&& make install
&& cd ..
&& rm -rf nginx-auth-ldap
&& rm -rf nginx
now i have difficulty in setting of nginx.conf, basically what I want to achieve is as soon as the login window pop up users will give their own credentials including user name and password which will eventually validated against LDAP container (CN=devops,OU=project,OU=services,OU=groups,o=orgname). So that means I can't have the password stored in my config file. Can you help me to build the setup ?
at this moment, I setup my config file like this, which I doubt it will work
under http tag:
ldap_server ldap {
url "ldaps://XXXX:636/ou=people,o=orgname?uid?(objectclass=person);"
require valid_user;"
group attribute name which contains member object
group_attribute member;
search for full DN in member object
group_attribute_is_dn on;
matching algorithm (any / all)
satisfy any;
require user "CN=devops,OU=project,OU=services,OU=groups,o=myorg";
}
and in location tag
location / {
#auth_basic "Restricted Content";
#auth_basic_user_file /etc/nginx/pass/.htpasswd;
auth_ldap "Closed content";
auth_ldap_servers ldap;
root html;
index index.html index.htm;
}
how can I do this ?
Regards,
Manas

@manas86 As that's a support request, best to put this on StackOverflow or similar where someone can assist you.


chore(ui): Add icons pin and x-axi-alt

Adds two icons using addicon.py utility

I have read the CLA Document and I hereby sign the CLA

recheck