Suggestion for using R in WSL - run RStudio in WSL through a Linux graphical desktop

I really enjoyed your blog about running R on Ubuntu on WSL.
I don't know if this is the best way to contact you - anyway - there is another method you could include.
You can install a graphical desktop environment in WSL and then run the Ubuntu version of RStudio in there.
I have followed online which show how to install MATE desktop and KFCE4 desktop on WSL (I can't make GNOME 3 work), e.g. here.
Below is some more possibly helpful info:

You need an X11 client for Windows e.g. Vcxsrv
Install MATE desktop in WSL with:

sudo apt update && apt upgrade -y
sudo apt install ubuntu-mate-desktop --no-install-recommends -y
sudo dpkg-reconfigure dbus && service dbus restart
sudo apt-get install mate-desktop-environment-extras
sudo apt install ubuntu-mate-icon-themes

With the X11 client running in Windows launch a graphical session within WSL with

mate-session

In WSL install R as you've done
Install RStudio by downloading the deb file from one of their install pages. I have been using the preview version from here.
Then install the .deb file with gdebi (changing the name of the .deb file as required)

sudo apt-get install gdebi-core
wget https://s3.amazonaws.com/rstudio-ide-build/desktop/bionic/amd64/rstudio-1.3.938-amd64.deb
sudo gdebi rstudio-1.3.938-amd64.deb

One quirk is that sometimes clicking the RStudio icon to launch it I just get a black window. When that happens I simply launch RStudio as sudo, e.g. sudo rstudio, and it works (having made sure rstudio is on my $PATH) (I guess not ideal running as sudo).
Here's a screenshot of my Windows 10 RStudio on the left and an X11 window running Ubuntu RStudio in WSL on the right.

cheers
Tom

Hi @remlapmot! Thanks for reaching out. I'm definitely going to try this out. Depending on how it goes, I'll either update my previous post or create a new one.
This Ask Ubuntu thread may interest you. They also discuss using an X11 client to use RStudio Desktop running in the WSL, but they used various other X11 clients. Have you tried either MobaXTerm or X410?

Great @jdblischak .
I haven't tried those other X11 clients but as far as I understand they basically have the same features. For example, VcXsrv can be run in windowed mode or (the whole Linux) desktop mode as well.
If you write a batch file to launch your WSL with a graphical desktop / windowed mode you change the relevant X11 executable/.xlaunch file. My batch file is here. It's a modified version from one of the tutorials.
Another method - Docker Desktop using WSL 2 engine
There is sort of another method of running R/RStudio which involves WSL 2. Docker Desktop on Windows can use a WSL 2 engine. In Docker Desktop settings tick Enable the experimental WSL 2 based engine.

You can launch any of the rocker, https://www.rocker-project.org/ containers.
As the rocker main page says you can launch base R with
docker run --rm -ti rocker/r-base

Or you can launch RStudio (I think server) with
docker run -e PASSWORD=yourpassword --rm -p 8787:8787 rocker/rstudio

You then access RStudio in your browser at http://localhost:8787 (username is rstudio, password yourpassword).

RAM issue with WSL 2
Another helpful point about WSL 2 is that it is useful to set the max amount of memory you want it to be able to use. Otherwise the Vmmem process ends up using almost all of your RAM. Hopefully this bug will be fixed in subsequent WSL 2 versions. The current fix is to specify memory and swap in a .wslconfig file. Mine is here.

@remlapmot Thanks again for all the advice and ideas. I recently returned to experimenting with my WSL1 setup, and I attempted to run RStudio in a graphical user interface. I successfully installed VcXsrv  and was able to launch a graphical user interface.
Unfortunately the only thing I couldn't get working was running RStudio. With versions 2022.02.1-461, 1.3.938 (the version you used in your instructions), and 1.2.5042, I get the following error:

bash: /usr/bin/rstudio: cannot execute binary file: Exec format error

From Issue https://github.com/rstudio/rstudio/issues/3615, the advice is to try RStudio 1.1. But this fails to install due to a missing dependency (I vaguely recall running into this error years ago when it was first released, but I don't have much motivation to resurrect RStudio 1.1).
Below is explicitly what I attempted. Are you currently able to run RStudio Desktop in WSL1? If yes, what version? Thanks!
# 2022.02.1-461 - Exec format error
wget https://s3.amazonaws.com/rstudio-ide-build/desktop/bionic/amd64/rstudio-2022.02.1-461-amd64.deb
sudo gdebi rstudio-2022.02.1-461-amd64.deb
rstudio
# bash: /usr/bin/rstudio: cannot execute binary file: Exec format error
sudo apt remove rstudio

# 1.3.938 - Exec format error
wget https://s3.amazonaws.com/rstudio-ide-build/desktop/bionic/amd64/rstudio-1.3.938-amd64.deb
sudo gdebi rstudio-1.3.938-amd64.deb
rstudio
# bash: /usr/bin/rstudio: cannot execute binary file: Exec format error
sudo apt remove rstudio

# 1.2.5042 - Exec format error
wget https://download1.rstudio.org/desktop/trusty/amd64/rstudio-1.2.5042-amd64.deb
sudo gdebi rstudio-1.2.5042-amd64.deb
rstudio
# bash: /usr/bin/rstudio: cannot execute binary file: Exec format error
sudo apt remove rstudio

# 1.1.463 - missing libgstreamer0.10-0
wget https://download1.rstudio.org/rstudio-1.1.463-amd64.deb
sudo gdebi rstudio-1.1.463-amd64.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading state information... Done
This package is uninstallable
Dependency is not satisfiable: libgstreamer0.10-0

For your missing libgstreamer error I found this page here. Maybe one of the answers will help?
I have upgraded all my WSL instances to WSL version 2 - as WSL2 file system operations are noticeably faster than on WSL1.
There are a few more things you can try.
RStudio server - (I find most convenient)
I now use RStudio server running on my WSL 2 but I interact with it using a browser in Windows. I can't remember if this was one your previous methods. To use RStudio server

Download from say

wget https://s3.amazonaws.com/rstudio-ide-build/server/bionic/amd64/rstudio-server-2022.02.1-461-amd64.deb

Or go to Server Version section of one of their downloads pages, e.g. https://www.rstudio.com/products/rstudio/download/preview/

Try the installation with gdebi or dpkg. I think I remember that I needed at least 1 Linux system dependency to get this to install successfully - I think I just Googled the error message after the first time I ran the command below

sudo dpkg -i rstudio-server-2022.02.1-461-amd64.deb

Once installed, you need start to an RStudio server session with sudo powers (I think this is so it's accessible from a browser in Windows), so at WSL terminal run

sudo rstudio-server start

Then in your Windows browser go to http://localhost:8787 and you should find RStudio server running in that browser window - the beauty of WSL 2 (and probably WSL 1) is that you can access both your WSL and Windows files in this.

Once you've finished your work, you stop your rstudio-server sessions back in your WSL terminal with

sudo rstudio-server stop

Electron version of RStudio
For the development versions of RStudio they have Electron builds on their dailies website
https://dailies.rstudio.com/rstudio/spotted-wakerobin/electron/bionic/
https://dailies.rstudio.com/rstudio/spotted-wakerobin/electron/macos/
https://dailies.rstudio.com/rstudio/spotted-wakerobin/electron/windows/
You could try the bionic one.
Certainly on macOS the Electron version opens noticeably faster than the usual build.
Bundled graphical WSL apps
Instead of installing the X11 stuff yourself Microsoft's WSLg and the GWSL app in the Microsoft Store appear to have bundled it up more conveniently

Microsoft's wslg https://github.com/microsoft/wslg
GWSL https://www.microsoft.com/en-us/p/gwsl/9nl6kd1h33v3?activetab=pivot:overviewtab

@remlapmot Again, thanks for the very thorough answer!
Electron dailes

For the development versions of RStudio they have Electron builds on their dailies website

I liked this idea, so I tried this first. I didn't have much luck. I finally got it to launch by specifying --no-sandbox, but it was just a blank white window.
# WSL terminal
wget https://s3.amazonaws.com/rstudio-ide-build/electron/bionic/amd64/rstudio-2022.06.0-daily-270-amd64.deb
sudo gdebi rstudio-2022.06.0-daily-270-amd64.deb

# started xlaunch.exe
export DISPLAY=localhost:0
mate-session

# In MATE terminal
$ rstudio
[370:0406/094151.472708:FATAL:setuid_sandbox_host.cc(158)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /usr/lib/rstudio/chrome-sandbox is owned by root and has mode 4755.
Trace/breakpoint trap (core dumped)
[372:0100/000000.638758:ERROR:zygote_linux.cc(646)] write: Broken pipe (32)

$ ls -l /usr/lib/rstudio/chrome-sandbox
-rwxr-xr-x 1 root root 52536 Apr  5 00:47 /usr/lib/rstudio/chrome-sandbox

$ sudo rstudio
[0406/094307.885930:FATAL:electron_main_delegate.cc(292)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
Trace/breakpoint trap (core dumped)

$ rstudio --no-sandbox
# window opens, but it is blank white

$ sudo rstudio --no-sandbox
# window opens, but it is blank white

RStudio Server

I interact with it using a browser in Windows. I can't remember if this was one your previous methods

Yep, exactly. My current approach is to use RStudio Server. I use the OpenCPU infrastructure to install and run RStudio Server, but other than that the approaches have the same effect.
Microsoft Store

Instead of installing the X11 stuff yourself Microsoft's WSLg and the GWSL app in the Microsoft Store appear to have bundled it up more conveniently

I don't have access to the Microsoft Store on my work laptop. That is one of the main reasons I have been hesitant to migrate to WSL2. It was a colossal pain to get WSL1 installed (compared to how easy it was via the Microsoft Store on my personal laptop).
Missing libgstreamer

For your missing libgstreamer error I found this page here. Maybe one of the answers will help?

Yeah, this reminded me that I had old notes on how to fix this problem. I found my code for pinning old versions of libgstreamer with APT. This was from back in 2017. If I'm going to write a blog post advising people how to install RStudio Desktop in WSL, I don't want to use such outdated versions of libgstreamer and RStudio. If I can't get a more modern version working, it doesn't seem worth the trouble.

In case helpful - you can upgrade WSL 1 instances to WSL 2 at the command line (but I don't know if this would need admin rights)

Ah, nice! I'm going to give this a try on my personal laptop first. I'm too afraid of borking my work laptop.

I have run it twice - it ran quickly without error both times! (definitely felt a bit nervous pressing Enter!)

For my future self or anyone else interested in trying this, below is a dump of the links I consulted:
https://www.rstudio.com/products/rstudio/download/preview/
https://www.rstudio.com/products/rstudio/older-versions/
https://ywjheart.wordpress.com/2019/01/09/install-mate-desktop-on-wsluse-linux-desktop-on-windows-10/
https://github.com/Microsoft/WSL/issues/637
https://sourceforge.net/p/vcxsrv/wiki/Using VcXsrv Windows X Server/
https://sourceforge.net/p/vcxsrv/wiki/VcXsrv %26 Win10/
https://gist.github.com/remlapmot/23af202e79dc31cecb66736c5c45a2a7
https://github.com/rstudio/rstudio/issues/3615
https://sourceforge.net/projects/vcxsrv/
https://askubuntu.com/questions/864260/issue-with-dependency-of-r-studio-installation-in-ubuntu-16-10
https://learn.microsoft.com/en-us/windows/wsl/install#upgrade-version-from-wsl-1-to-wsl-2

thanks John - very nice of you; and very helpful additional links.


Support rollup plugins or use @rollup/plugin-strip to remove debugger statements.

Feature request
Customize the use of the rollup plugins or use parameters (such as --strip) when executing the command to enable the @rollup/plugin-strip package.
Motivations
Debugger statements used in development are not used in production, so I want to use the plugins to remove these debugger statements.
Whether to support the use of additional configuration files to customize the rollup plugins?
Alternatives
No response
Additional context
No response
Contributions

[ ] I plan to open a pull request for this issue
[ ] I plan to make a financial contribution to this project

I'm not sure how common it is for devs to want to produce distribution code while still having debugger in the source and auto stripping it. Is there a precedent of bundlers supporting this?
Regarding customization, this tool basically configures Rollup for you so I'd like for it to export a function so users can configure Rollup too. Something like:
rollup.config.js
import { inferConfig } from 'pkgroll'

export default extendConfig({
	plugins: [
		// ...
	]
})

When I think about it, it's not very common to include a debugger in the class or function to be export, so this feature may not be needed.
However, it may be possible to set plugins or other parameters using a custom rollup config in future, which is optional and works fine without rollup.config.js in file list.
Incidentally, when will you merge this PR and release a new version?

Oh yeah, I dropped the ball on that one. It requires a breaking change release but I'll try to pick it up soon.


Branch OV - Sort by parents instead of time + PP #577

for testing you can use https://github.com/bigld/abapgit_dbtests_TABL.git and https://github.com/bigld/abapgit_dbtests.git
there a commits with changed timestamps and some to test the compress function

#577

I'm a bit concerned about performance but lets just try this, merging, thanks 👍


How do I change phone numbers when using templates to generate envelopes

Create templates that require phone validation. How do I change phone numbers when using templates to generate envelopes

Is there any interface that can modify the recipient as templateRoles does ?

See https://developers.docusign.com/docs/esign-rest-api/how-to/phone-auth/
The node code for a recipient is like this:
let signer1 = docusign.Signer.constructFromObject({
    email: args.signerEmail,
    name: args.signerName,
    recipientId: "1",
    routingOrder: "1",
    deliveryMethod: "Email",
    tabs: signer1Tabs,
    identityVerification: { workflowId: args.workflowId, steps: null, "inputOptions":[{"name":"phone_number_list","valueType":"PhoneNumberList","phoneNumberList":[{"countryCode":args.countryCode,"code":"1","number":args.phoneNumber}]}], "idCheckConfigurationName":""}

Just replace the Signer object with a TemplateRole object, but the rest should be the same.

How do I get signer1Tabs content?  I want to generate the envelope with the defined template

How do I get signer1Tabs content?  I want to generate the envelope with the defined template
------------------ 原始邮件 ------------------
发件人:                                                                                "docusign/code-examples-node"                                                                                @.>;
发送时间: 2022年6月16日(星期四) 上午10:37
@.>;
<EMAIL_ADDRESS>主题: Re: [docusign/code-examples-node] How do I change phone numbers when using templates to generate envelopes  (Issue #40)
Just replace the Signer object with a TemplateRole object, but the rest should be the same.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

The last error occurs because you're using a templateId which requires templateRole so you cannot specify recipients too.
I would recommend using the composite template model that allow the most flexibility.

Let us know if you still need help on this


More optimizations

Small optimizations which give additional 20%.
After this change perf report produces the following results:
+  29,81%  decode-193e1634  decode-193e16349b64641b  [.] _$LT$flif..numbers..rac..Rac$LT$R$GT$$u20$as$u20$flif..numbers..rac..RacRead$GT$::read::h508eb70fc761b920
+   9,77%  decode-193e1634  decode-193e16349b64641b  [.] _$LT$flif..numbers..rac..Rac$LT$R$GT$$GT$::input::h253f8d0752fff545
+   9,69%  decode-193e1634  decode-193e16349b64641b  [.] flif::numbers::near_zero::read_near_zero_inner::h256a4ce2b265169e
+   8,76%  decode-193e1634  decode-193e16349b64641b  [.] flif::decoding_image::DecodingImage::channel_pass::hb308cc595457d6c5
+   8,73%  decode-193e1634  decode-193e16349b64641b  [.] flif::maniac::ManiacTree::process::h4d320e7bd5217b70
+   6,99%  decode-193e1634  decode-193e16349b64641b  [.] flif::maniac::pvec::core_pvec::h8ca20e326f507b18
+   6,70%  decode-193e1634  decode-193e16349b64641b  [.] flif::numbers::chances::UpdateTable::next_chance::h71ce1347b9944369
+   3,18%  decode-193e1634  decode-193e16349b64641b  [.] core::ptr::drop_in_place::hb237506e6fab0973
+   2,79%  decode-193e1634  decode-193e16349b64641b  [.] flif::decoding_image::DecodingImage::get_core_vicinity::h2b420043a0b71844
+   2,72%  decode-193e1634  decode-193e16349b64641b  [.] _$LT$flif..components..transformations..ycocg..YCoGg$u20$as$u20$flif..components..transformations..Transform$GT$::crange::h2782ff09b64e64de
+   2,43%  decode-193e1634  decode-193e16349b64641b  [.] _$LT$flif..components..transformations..bounds..Bounds$u20$as$u20$flif..components..transformations..Transform$GT$::crange::h392ffb4cea3b89d6
+   2,31%  decode-193e1634  decode-193e16349b64641b  [.] flif::decoder::make_core_guess::h57797d22840f0483
+   1,23%  decode-193e1634  decode-193e16349b64641b  [.] _$LT$R$u20$as$u20$flif..numbers..near_zero..NearZeroCoder$GT$::read_near_zero::ha729b2315a24094f

Are there any more optimizations you're planning with this PR or is it good to merge?

No, that's it for now. You can merge it.


broken link

Problem description
on the page:
https://docs.docker.com/docker-for-windows/dashboard/
there is a link to the asp.net version of the frontend in the text:

The example voting application contains:
A front-end web app in Python or ASP.NET Core which lets you vote between two options
which leads to 404....

Problem location

I saw a problem on the following URL:

https://docs.docker.com/docker-for-windows/dashboard/
Project version(s) affected

Suggestions for a fix

check the docu and fix the links.

@deugister Thank you for the feedback. I've created PR to fix the broken links issue.


Fixed JEI Compat not working

Re-done how the RecipeType is registered (Switched to Deferred)

Fixes #10

<3 amazing. and of course its the one repo i forgot to reenable github actions on so im typing this comment partly to remind myself to do that


Update and format Simplified Chinese localisation

Pretty simple one
Extra: i would like to help contribute to the translation in Corsace open (cuz there's some weird translation), but i didnt find the localization strings here, do i have any way to help translate Corsace open?

join discord server and message me


Add git ignores for build artifacts

Some stray files produced by autogen/configure/make show up as uncommitted files, this ignores them.

Nice.
How about adding one .gitignore file on root level?

Well, I'm not familiar enough with the tree to know the proper ones, and you already had them split out by dir.  Up to you.

I would prefer updating https://github.com/jgarzik/univalue/blob/master/.gitignore (maybe have a look at https://github.com/bitcoin/bitcoin/blob/master/.gitignore).
Less files, more centralized git configuration.

typically IME,

if it is duplicated, put it at the top level
otherwise local

Updated per jgarzik's guidlines


.gitignore *.pyc files

You may want to add a gitignore file to your repository, so that runtime artifacts such as all the .pyc files are not submitted to the repository; and then delete those that are already in there.

@quietlyconfident Sure we'll do that.

Merged and updated the package.


AS3 Project Debug: Unable to add viarabiles to watchlist

While debuging, right-click on variables, select add to watchlist has no effect, only cursor jump to watchlist with an empty line.
nextgenas 6.0-SNAPSHOT, macOS 11.12.6, vscode 1.16

Correct, this feature has not been implemented. It may appear in a future version.


Update how the Screen Manager removes a graphic from a template

Bug Report
Currently, the screen manager API uploads a transparent image and uses that image name whenever the graphic needs to be removed from the template. This is not necessary as we can just use an "empty" Image in the graphic or secondaryGraphic field of a Show request.
Show show = new Show();
show.setMainField1("removing graphic");
show.setGraphic(new Image("", ImageType.DYNAMIC));
sdlManager.sendRPC(show);

OS & Version Information

Android Version: n/a
SDL Android Version: 4.10
Testing Against: n/a

Closing this issue because it is not possible to set an "empty" image for the secondaryGraphic field of a Show request due to a long standing bug with Core. The screen manager will still need to use the current logic for clearing the secondaryGraphic, which means there will be two sets of logic for removing the image. It is possible to do so, but this increases the likelihood of introducing bugs.


Start ProxyServer once for each test class

When running this from a JUnit test class with both @ClassRule and @Rule, the RecorderRule separates out the concerns of starting the server and starting the recorder.  If running the recorder directly, the previous behavior is maintained; that is, it will automatically start the server.
A new configuration property was introduced to create the proxy on startup - it defaults to true, but is overridden by the JUnit rule startup.  Also, a TapeProvider interface was introduced to allow dynamic loading of tapes from the ProxyServer, so that each time a test method is run with a new Tape, it's retrieved correctly in the proxy routines.
There's also code to allow users to specify a proxyPort of '0' - when this happens, the actual port is looked up and attempted to be bound; if it fails, the process retries until it finds an open port.  The actual port is also sent into the proxyOverrider to set up the JDK proxy properties correctly; it used to always follow the config value, which is incorrect when '0' is used.
Lastly, a bug fix - when the ProxyConfiguration specified a default tape mode, it would be ignored because the @Betamax annotation specified a default already that would override it; I introduced an UNKNOWN tape mode that returns an Optional.absent().

@strmer15 I can't believe I haven't commented on this yet, but I'll try to look at this over the weekend.

@cowboygneox It's all good - I'm just glad you've gotten this project back up on it's feet.  Looks like I need to re-merge with master anyways to resolve some conflicts from your recent changes.

@strmer15 This is going to take a bit to review. Perhaps we should arrange a time when we can be online together to discuss?

@cowboygneox Sorry for the wait, trying to fix these unit test failures, they always work locally but fail on travis.
I'm generally online in the afternoon US/Eastern, I can also try to schedule some time on Saturday or Sunday as well.

I can't seem to find time to work on this, plus I'm thinking my whole approach to this isn't working well - I'm going to close and try to rework it sometime.


CSV export of statistics has missmatching columns on weekly reports

Describe the Bug

When a user creates a CSV export for a weekly report, we add a new column that is not included in the CSV format (see screenshot). Normally, the upper columns should slide one further.

I would like to work on this issue. Which are the files I should work with?

@svojtkova Awesome! I guess this is a very good issue to start with. I guess you only have to take a look at https://github.com/digitalfabrik/integreat-cms/blob/develop/integreat_cms/static/src/js/analytics/statistics-charts.ts and find a good way of dealing with the special case of weekly exports and adjust the matrix accordingly :)

Hello @ulliholtgrave do you have deadline till when it is needed to be done? I had hard time to run it but now it works. I have one more question how can I get to the page where I can download the report? I searched on the web but I cannot find it anywhere.

Hey @svojtkova
We don't have any urgent need right now, but it would be good if this would be ready in the next 1-2 weeks. Unfortunately, our Matomo server for the reports is strictly internal and we would have to provide you with an API key for that. However, the reports used will be compiled into a JSON here https://github.com/digitalfabrik/integreat-cms/blob/develop/integreat_cms/cms/views/statistics/statistics_actions.py . Does that help you already?
Otherwise you can open a draft PR with your changes, then we could have a look at them.

@ulliholtgrave That didnt help me. But what could help is how does constant chart output look like in line 180 (const chart = Chart.instances[0]) It is inside function exportStatisticsData in file statistics-charts.ts.

@ulliholtgrave
Peter and me couldn't reproduce the error. We got properly formatted csv (region 29=Augsburg). Please can you post in which cases exactly the bug occurs?

@Gaston69 you have to select "weekly" in the evaluation select box.
Then, two columns are used by the date, but the first line only has one leading empty box, so the label for the first language is in the column of the last date of the week.

Ah, now I saw, that I didn't click 'Ansicht aktualisieren' before clicking export. Got it.


soc: samd51: Fix xosc32 initialization

This change sets the startup timeout and control gain to non-reserved values. Without these changes, all demos for the Wio Terminal board fail to get past osc32k_init.

Hi @mbirthelmer ,
Changes are fine but commit message is empty. You need to explain why you are doing this changes as described in the coding guidelines.

Hi @mbirthelmer,
The merge window will close this week. So, you need make changes ASAP to get this in.

Hi Gerson,
I was informed by my manager that I’m not allowed to contribute to OSS
(even personal projects) without special permission, so i can’t work on
this any further, sorry!

Marcel

On Tue, Jun 11, 2024 at 4:55 AM Gerson Fernando Budke <
@.***> wrote:

Hi @mbirthelmer https://github.com/mbirthelmer,
The merge window will close this week. So, you need make changes ASAP to
get this in.
—
Reply to this email directly, view it on GitHub
https://github.com/zephyrproject-rtos/zephyr/pull/73961#issuecomment-2160573126,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABCBE2GLUBWDTPETVKQVGYDZG3QTFAVCNFSM6AAAAABJAHCDPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRQGU3TGMJSGY
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi Gerson, I was informed by my manager that I’m not allowed to contribute to OSS (even personal projects) without special permission, so i can’t work on this any further, sorry! - Marcel
…
On Tue, Jun 11, 2024 at 4:55 AM Gerson Fernando Budke < @.> wrote: Hi @mbirthelmer https://github.com/mbirthelmer, The merge window will close this week. So, you need make changes ASAP to get this in. — Reply to this email directly, view it on GitHub <#73961 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCBE2GLUBWDTPETVKQVGYDZG3QTFAVCNFSM6AAAAABJAHCDPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRQGU3TGMJSGY . You are receiving this because you were mentioned.Message ID: @.>

Hi @mbirthelmer ,
Since you let marked the option "Maintainers are allowed to edit this pull request." I'm allowed to continue your work due to your restrictions. I hope I can ear from you again in the future when you are on a place that value you and your work.

Resuming on #74104


optimize condition expr using blendv

Enhancement
Conditional expressions run too slowly in Starrocks, and some vectorization can be used to optimize branch execution
case :
 SELECT SUM(IF(LO_SUPPLYCOST + 10000 > 100000, P_SIZE, LO_REVENUE) ) FROM ssb.lineorder_flat;

For CASE WHEN is a bit more complicated, we can prioritize the optimization of the non-nullable case
case:
 SELECT SUM(CASE WHEN LO_SUPPLYCOST + 10000 > 100000 then 1 WHEN LO_SUPPLYCOST + 10000 > 90000 then 2 WHEN LO_SUPPLYCOST + 10000 > 70000 then 3 WHEN LO_SUPPLYCOST + 10000 > 50000 then 4 else 0 END) FROM ssb.lineorder_flat;

1FE 1BE 1PAR

time cost

origin
14.43

remove column view and builder
6.64

SIMD multi selector
1.98

vectorized case when:
INPUT: select_vecs = [vec1,vec2] select_datas = [vdata1, vdata2]
1. add [1, 1, 1, 1] to select_vecs # padding len(selecter) == len(datas)
2. set selected = [0, 0, 0, 0]; # to mark if it has been selected
3. set selected_dst = [undefined]; # final select result
4. for each select_vec in select_vecs, each select_data in select_datas:
	set not_selected = ~selected
    will_select = not_selected_vec & select_vec;
    selected_dst = select_if(will_select, selected_dst, select_data)
5. store selected_dst


Style 3D axis + quickfixes

The 3D axis class now uses latex labels instead of regular text labels (inkeeping with the book style).
Positioning of negative 2D axis labels now changed to be more readable.
Forgot to use mathbf in innerproduct_projectionvectorline.

Looks good

Zullen we deze gewoon naar main pushen?


add index on history

hi,
I am using the following sql request to monitor batch performance and feed an ElasticSearch instance. Could you please add an index on end_date and applicationname ? An index already exist on applicationname (called hist_application). So the end_date column could be added to this one if you wish.
ps : i use end_date as filtering criteria because this is only way to get finished job in a monotone order.

I'm afraid you've forgotten to include the "following query" ;-)
If this is a query on the History table, there are actually no indices out of the box (save the PK), but you are free to add your own. See http://jqm.readthedocs.io/en/jqm-all-1.3.6/admin/history.html for more details.


Is it possible to add a new sheet as well?

Hello!
First, I would like to thank you for this awesome approach to send data to Google Sheets. It is really helpful.
I would like to know if by using this approach I can create another sheet on the document itself from Xamarin and then send the data to that new sheet created, This would be great due to it can bring good posibilities to categorize data better.
Thanks in advance,
Victor

I found out that we can do it yeah! by using Google Apps Script code. Thank you.


Support runtime resolution change

This will automatically detect if the rendering resolution has changed, and will re-generate the command buffer if it has. This check should be a negligible amount of overhead, but if there's some more appropriate 'onresize' type event, let me know.

I'm not sure whether it notifies you of this automatically or not, but I've done the tweaks :)


Is there any chance to support submenu?

this sidebar menu is awesome, and I want to know do you have plan to support submenu ?

hello, any update on this one? thank you so much!


HDDS-2336. Fix TestKeyValueContainer#testRocksDBCreateUsesCachedOptions

What changes were proposed in this pull request?
Fix testRocksDBCreateUsesCachedOptions, which is failing because each test method in TestKeyValueContainer introduces a new entry in MetadataStoreBuilder.CACHED_OPTS, since Configuration does not implement equals.  Thus testRocksDBCreateUsesCachedOptions passes by itself, but fails when the whole test class is run.
https://issues.apache.org/jira/browse/HDDS-2336
How was this patch tested?
Unit test.  No other code changed.

https://github.com/elek/ozone-ci-q4/blob/2dcb51dec5ccd866604c361d62f8b04c1188e17b/pr/pr-hdds-2336-74zpv/unit/output.log#L488-L489:
[INFO] Running org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.529 s - in org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer

Findbugs being fixed in #62, checkstyle in #64.
@swagle @mukul1987 please review

Thanks @mukul1987 for reviewing and merging it.


no mutation error

Should this error on mutation?
  const formattedQuery = {
    ...query,
  }

  delete formattedQuery['access-token']

Yes. delete should be entirely forbidden.
You probably want const { 'access-token': unused, ...formattedQuery } = query; instead.

It probably just doesn't check for it yet, but it should.

I believe it should be possible to write a simple rule for this, by adding a visitor for the UnaryExpression node, and checking that the node.operator === 'delete'


cataclysm-dda olny english language again.

Describe the bug
stable version and git(override to current commit) version.
Steps To Reproduce
Steps to reproduce the behavior:
install cataclysm-dda.
Expected behavior
no non-english language.
Screenshots

Notify maintainers

@mnacamura @DeeUnderscore
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-darwin"`
 - host os: `Darwin 22.0.0, macOS 13.0`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.10.3`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

The old solution https://github.com/NixOS/nixpkgs/pull/74064.
Current not work:
https://github.com/NixOS/nixpkgs/blob/ff8e8d4b4f7c934e16546beefce9e547b6b65d5c/pkgs/games/cataclysm-dda/common.nix#L45-L49

Hey, I opened a PR that might fix this at #186521. If you are able to build the PR under Darwin and see if that helps, it would be appreciated.

@DeeUnderscore Thank you!


Codespaces terminal showing too many ⚠

Steps to Reproduce:

Create a new codespace in desktop using the vscode repro
:bug: the terminal shows multiple ⚠ (single tab, insider terminal, terminal tab list)

I'd expect to not see the single tab nor the insider terminal warning
GitHub Codespaces: v0.10.4
Version: 1.57.0-insider
Commit: 541a5d4fa70ab3095f23b28b88914e1f6a8f65c4
Date: 2021-05-28T10:00:38.762Z
Electron: 12.0.9
Chrome: 89.0.4389.128
Node.js: 14.16.0
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 20.3.0

I think this is as-designed, though I thought we had gotten rid of the one in the terminal contributed by js debug?

The config terminal should probably be able to opt out of the warning somehow since this looks broken.


Incompatible TypeError message from calling __init__ method with insufficient arguments

Feature
The TypeError message from calling __init__ method with insufficient arguments is incompatible with CPython implementation. I guess this bug occurred from below lines:
https://github.com/RustPython/RustPython/blob/60edbdd427b3d552239b122b903aa89fb1501057/vm/src/builtins/function.rs#L188-L194
CPython-side
>>> class A:
...   def __init__(self, x):
...     ...
...
>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'x'

RustPython
>>>>> class A:
.....   def __init__(self, x): ...
.....
>>>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Missing 1 required positional arguments: x

I fixed it in #2941 🥳

Some parts of this issue was fixed by #2941 but there is a typo.
Current execution result is same with below:
>>>>> class A:
.....   def __init__(self, x): ...
.....
>>>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional arguments: x

So:

arguments should be argument when the length of arguments is one.
x argument should be wrapped with '.

Further more, when the arguments are multiple, the last argument should be next on and word.
RustPython
>>>>> class A:
.....   def __init__(self, a, b, c, d, e): ...
.....
>>>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 5 required positional arguments: a, b, c, d, e

CPython
>>> class A:
...   def __init__(self, a, b, c, d, e): ...
...
>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 5 required positional arguments: 'a', 'b', 'c', 'd', and 'e'

Sorry for make you cumbersome 😂. I thought formatting is far less important than missing method name in error message and there are quite a lot such formatting miss in legacy code

Sorry for make you cumbersome joy. I thought formatting is far less important than missing method name in error message and there are quite a lot such formatting miss in legacy code

Thanks to fix this issue. 🙏🏻 I didn't intend to make you feel sorry. 😂 I'm stuck at many issues while resolving A and this issue is also one of them so I left the comment to record my analyze.
Thanks to fix this issue, again. 🙇🏻♂  👍🏻


Performance for Android

Hi.
Is there any way to improve performance on Android? I've noticed that when using Calendar.List on Android, it takes a bit longer than it does on iOS. On iOS, it appears almost instantly, while on Android, there's a brief moment when the screen is blank.
import { Calendar, toDateId } from "@marceloterreiro/flash-calendar";
import React from "react";
import { View } from "react-native";

const index = () => {
  const today = toDateId(new Date());
  return (
    <View style={{ flex: 1 }}>
      <Calendar.List
        calendarMinDateId={today}
        calendarInitialMonthId={today}
        onCalendarDayPress={(date) => console.log(date)}
      />
    </View>
  );
};
export default index;

Do you see the same behavior when using the raw Shopify FlashList? Additionally, are you seeing this in production or in development? As explained by FlashList docs, you must test it in release mode.


Support Null Safety

Also please update dependencies.
[app] flutter pub get
Waiting for another flutter command to release the startup lock...
Running "flutter pub get" in app...
Because no versions of flutter_launcher_icons match >0.9.0 <0.10.0 and flutter_launcher_icons 0.9.0 depends on yaml ^3.0.0-nullsafety.0, flutter_launcher_icons ^0.9.0 requires yaml ^3.0.0-nullsafety.0.

And because image_res >=0.2.1 depends on yaml ^2.1.16, flutter_launcher_icons ^0.9.0 is incompatible with image_res >=0.2.1.

So, because beginning depends on both image_res ^0.2.3 and flutter_launcher_icons ^0.9.0, version solving failed.
pub get failed (1; So, because beginning depends on both image_res ^0.2.3 and flutter_launcher_icons ^0.9.0, version solving failed.)
exit code 1

Any progress on this? Currently still unusable because of the dependency issue.

Just made a PR, waiting approval...
In the meantime, if you need to use the package with null-safety, you can use my fork:
dev_dependencies:
  image_res:
    git:
      url: https://github.com/roipeker/image_res.git
      ref: nullsafety-update

#4 has been merged.


Accessing pins by physical location on raspberry pi doesn't seem to work

Gist with the code:
https://gist.github.com/motiooon/eb5be4823d3e3a133c50
Error stack:
/home/pi/projects/johnny-five/node_modules/raspi-io/lib/index.js:231
throw new Error('Unknown pin "' + pin + '"');
^
Error: Unknown pin "P1-16"
at Raspi.Object.defineProperty.value [as $271828_5$] (/home/pi/projects/johnny-five/node_modules/raspi-io/lib/index.js:231:13)
at Raspi.Object.defineProperty.value [as pinMode] (/home/pi/projects/johnny-five/node_modules/raspi-io/lib/index.js:240:43)
at Raspi. (/home/pi/projects/johnny-five/start.js:16:9)
at Raspi.emit (events.js:92:17)
at /home/pi/projects/johnny-five/node_modules/raspi-io/lib/index.js:204:10

As we discussed in gitter, this is actually the expected behavior, and using Johnny-Five instead of raspi-io directly will get you the pin mapping you want.


Allow specifying nearest config directory as working directory

Currently, I can specify that the eslint working directory should be

workspace folder
linted file directory
"auto" inferred (the meaning of which is not very clear)
specific sets of directories or directory globs

Consider a layout for an Angular project that uses Nightwatch for end-to-end testing. Nightwatch uses property names for some configuration options that need to be quoted like e.g. ms:edgeOptions. I want to enforce that code we write in typescript uses camelCase properties, but the nigtwatch.conf.js file (which needs to be JS not TS for reasons I don't understand - but at least that makes the pathspec easy to wildcard) needs to relax that constraint.
So, the directories look like
- nightwatch/
    - nightwatch.conf.json
    - .eslintrc.json
    - tsconfig.e2e.json
    - globals/
        - globals.ts
    - page_objects/
        - a-page.ts
    - tests/
        - page.spec.ts
- angular.json
- .eslintrc.json
- tsconfig.json
- package.json
- node_modules/...
- src/
    - app/
        - app.component.ts
    - index.html

... where nightwatch/.eslintrc.json extends the root .eslintrc.json.
And I have a workspace open to the root of this project. To get the Typescript ESLint plugin parser working, I have overrides[0].parserOptions.project set to ["tsconfig.e2e.json"]. Running ESLint manually from within the nightwatch/ directory picks up all the proper rules and lints appropriately. Running ESLint manually from the project root properly uses the nightwatch config for that directory and the root config for everything else. The ESLint VSCod(e/ium) plugin, however, fails to lint any files under nightwatch/ because
Parsing error: Cannot read file '/path/to/project/root/tsconfig.e2e.json'.

(also fwiw ng lint works exactly correctly as well).
That's because - I assume - the plugin is trying to run ESLint from the project root but specifying the configuration file as the one in the directory it's trying to lint. As a manual ESLint command, it would look like npx eslint -c ./nightwatch/.eslintrc.json ./nightwatch . That command fails with the same messages that the plugin gives me.
Instead, ESLint should allow me to configure it to use the directory containing the .eslintrc.json file it's using as the working directory.
Interestingly, in both "location" and "auto" modes, if I change the nightwatch/.eslintrc.json's overrides[0].parserOptions.project to ["nightwatch/tsconfig.e2e.json"], first of all it starts to successfully lint files under nightwatch/ - but then if I change that property back to just ["tsconfig.e2e.json"] it'll keep successfully linting those files. I have no idea why it can "fix" itself like this and only after I manually intervene.

Instead, ESLint should allow me to configure it to use the directory containing the .eslintrc.json file it's using as the working directory.

I am not sure if this will be possible since the working directory is the starting directory from which eslint (the npm module) will search for the .eslintrc.json file. Without that starting directory it will very likely not work at all.


Update README for error handling of CBOR extraneous data

Changes:

Update "Quick Start" to mention CBOR settings allow trade-offs between speed, security, encoding size, etc.

Update "Quick Start" to mention new funcs in v2.5.0 can return remaining bytes after decoding first CBOR data item:

    func UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error)
    func DiagnoseFirst(data []byte) (string, []byte, error)

Update "Status" to more explicitly mention changes should be reviewed before upgrading from v2.4 (or older) to v2.5.0 due to error handling bug fixes for extraneous data in Unmarshal, etc.

I try to avoid changing default behavior of CBOR codec, but PR #380 for Unmarshal was a bug fix rather than new opt-in setting for reasons mentioned in that PR.
How to handle extraneous data as non-error in v2.5.0
Use UnmarshalFirst or Decoder.Decode to handle extraneous data in CBOR as non-error.

@x448 Please take a look (not urgent).  Thanks!


Expense - Missing background image in the combined report

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.8-1
Reproducible in staging?: Y
Reproducible in production?: N
Logs: https://stackoverflow.com/c/expensify/questions/4856
Issue reported by: Applause-Internal team
Action Performed:

Go to staging.new.expensify.com
Go to DM with user that has no unsettled expense.
Submit one manual expense (only one).
Click on the expense preview to go to the combined report.

Expected Result:
There will be background image in the combined report.
Actual Result:
There is no background image in the combined report.
This issue only happens in the combined report (one transaction view).
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/9141d4e8-77a6-4cd6-bb35-1e84357dd2b4

View all open jobs on GitHub

We think this issue might be related to the #collect project.

Production
https://github.com/user-attachments/assets/dac0e61f-202f-4bd1-9ecc-fd90e05b301b

I'm not seeing this behavior on staging v9.0.8-3
https://github.com/user-attachments/assets/f76689dc-899b-403f-8d48-ddbbb8d21700

Oh, yes thank you @kavimuru - i can reproduce the issue on staging now. I'm fairly confident this is due to https://github.com/Expensify/App/pull/45042/
It's a pretty minor bug, so I don't think it needs to be a blocker but we should spin up a follow up issue to resolve the bug.
cc: @nkdengineer @Ollyws  @thienlnam

@NikkiWines I will create a PR tomorrow morning

@Ollyws The PR is here.

Summarizing payment on this issue:

Contributor:  @nkdengineer $250 via Upwork -- I extended the offer to you, please accept when you get a chance! https://www.upwork.com/nx/wm/offer/103368089
Contributor+: @nkdengineer $250 via NewDot -- Please request

Upwork job is here: https://www.upwork.com/jobs/~018eb452959d186422

@stephanieelliott This issue is the regression from my PR.

Ah ok thanks @nkdengineer! Closing this then.


fixed typo

@johnbuhay small typo fix

Coverage remained the same at 76.329% when pulling ac297db20aee00f4cdb1bf79ab1f53ae851b175f on rhinof:master into 60007d8cbc0915ead1e718dce3aeb26aa4c0ffc3 on wework:master.


Virtual Keyboard not added

Hello, I installed the latest version of the driver 3.1.0 and activated it like mentioned in the README. The driver seem to be loaded and activated but I don't see the virtual keyboard. How should I trouble shoot this?
Thank you,
~/Downloads❯ systemextensionsctl list
1 extension(s)
--- com.apple.system_extension.driver_extension
enabled	active	teamID	bundleID (version)	name	[state]
*	*	G43BCU2T37	org.pqrs.Karabiner-DriverKit-VirtualHIDDevice (1.7.0/1.7.0)	org.pqrs.Karabiner-DriverKit-VirtualHIDDevice	[activated enabled]
~/Downloads❯ defaults read /Applications/.Karabiner-VirtualHIDDevice-Manager.app/Contents/Info.plist CFBundleVersion
3.1.0

https://imgur.com/wtiIfrp
Edit: Tried version 2.1.0 (with clean uninstall) and still have the same problem.

@linktohack I'm having the same issue. Were you able to fix it? thx

You need to run client to add virtual devices after the driver is activated.
Example client:
https://github.com/pqrs-org/Karabiner-DriverKit-VirtualHIDDevice/tree/main/examples/virtual-hid-device-service-client

Thanks for the reply @tekezo
This is the output of make run
..al-hid-device-service-client❯ make run                                                                                Karabiner-DriverKit-VirtualHIDDevice/git/main
sudo ./build/Release/virtual-hid-device-service-client

Press control+c to quit.

pqrs::karabiner::driverkit::driver_version::embedded_driver_version: 10700
pqrs::karabiner::driverkit::client_protocol_version::embedded_client_protocol_version: 4
connected
driver_activated 0
driver_connected 0
driver_version_mismatched 0
virtual_hid_keyboard_ready 0
virtual_hid_pointing_ready 0
virtual_hid_keyboard_ready 0
virtual_hid_pointing_ready 0
virtual_hid_keyboard_ready 0
virtual_hid_pointing_ready 0
virtual_hid_keyboard_ready 0
virtual_hid_pointing_ready 0

Again
..al-hid-device-service-client❯ systemextensionsctl list                                                                                Karabiner-DriverKit-VirtualHIDDevice/git/main
1 extension(s)
--- com.apple.system_extension.driver_extension
enabled	active	teamID	bundleID (version)	name	[state]
*	*	G43BCU2T37	org.pqrs.Karabiner-DriverKit-VirtualHIDDevice (1.7.0/1.7.0)	org.pqrs.Karabiner-DriverKit-VirtualHIDDevice	[activated enabled]
..al-hid-device-service-client❯ defaults read /Applications/.Karabiner-VirtualHIDDevice-Manager.app/Contents/Info.plist CFBundleVersion               Karabiner-DriverKit-VirtualHIDDevice/git/main
3.1.0


Add support for skipping non-dist blob push

Add image handler to skip non-distributable blobs.
Add flag to ctr explicitly enable non-dist blob push -- breaking change, can flip this flag if needed, just figured it's best to have the expected handling of this blobs as a default.

/test pull-containerd-node-e2e


Update the foxy rqt_topic branch to foxy-devel.

We are about to merge some changes that are not backwards
compatible with dashing.
Signed-off-by: Chris Lalancette<EMAIL_ADDRESS>

I'm going to go ahead and merge this one so we can move forward with https://github.com/ros-visualization/rqt_topic/pull/31.


Math range error in decision function for large v-values

If v-values gets too big, the decision function will fail with math range error.
How to reproduce:
n_subjects        : 100
mechanism         : GA
behaviors         : cr,ignore
stimulus_elements : us,cs,none
start_v           : cs->ignore: 0, default: -1
alpha_v           : cs->ignore: 0.0, default: 0.2
beta              : 1
behavior_cost     : cr:1, default: 0
u                 : us:4, default: 0

@variables p_none:0, stop_condition:300

@phase Pavlovian_cond100  stop:cs=stop_condition
START            | NONE(p_none) | CS
CS    cs         | US
US    us            | START
NONE  none       | START

@phase Pavlovian_cond75(Pavlovian_cond100)  stop:cs=stop_condition
CS    cs         | US(0.75)  | NONE

@phase Pavlovian_cond50(Pavlovian_cond100)  stop:cs=stop_condition
CS    cs         | US(0.5)   | NONE

@phase Pavlovian_cond25(Pavlovian_cond100)  stop:cs=stop_condition
CS    cs         | US(0.25)  | NONE

@phase Pavlovian_cond0(Pavlovian_cond100)  stop:cs=stop_condition
CS    cs         |  NONE

@run Pavlovian_cond100 runlabel: p100
@run Pavlovian_cond75 runlabel: p75
@run Pavlovian_cond50 runlabel: p50
@run Pavlovian_cond25 runlabel: p25
@run Pavlovian_cond0 runlabel: p0

xscale: cs
@figure Test of contingency theory for Pavlovian conditioning
#@subplot 221 - {'xlabel':'Trial','ylabel':'Probability',
'title':'Probability of CR  (thick line is average)'  }
subject: average
runlabel:  p100
@pplot cs->cr {'linewidth':3,'label':'p(US|CS)=1'}
runlabel:  p75
@pplot cs->cr {'linewidth':3,'label':'p(US|CS)=0.75'}
runlabel:  p50
@pplot cs->cr {'linewidth':3,'label':'p(US|CS)=0.50'}
runlabel:  p25
@pplot cs->cr {'linewidth':3,'label':'p(US|CS)=0.25'}
runlabel:  p0
@pplot cs->cr {'linewidth':3,'label':'p(US|CS)=0.0'}
@legend

@markusrobertjonsson this most likely comes from exp, which overflows easily with a large argument. You can work around this by first calculating relative probabilities. Let v(s->bk) be the largest v of all the v(s->bi) we have. You can then calculate:
p(s -> bi) / p(s -> bk) = exp( beta*( v(s->bi) - v(s->bk ) ) )

these numbers will range between 1 (for i==1) and 0. You can then add them all to obtain the normalization factor to transform them into actual probabilities. In fact, the sum of all relative probabilities is S = sum(i) p(s->bi) / p(s->bk) and you can see that a relative probability divided by S is the same as the probability we want.

Fixed on issue branch 92. Merged to master.


Typo in IdleChannelHandler is assigning a scheduled task to the wrong member field

By browsing the code I think I found a typo in IdleChannelHandler.makeAllTimeoutTask (see here).
When makeAllTimeoutTask is rescheduling, the resulting Scheduled<T> is assigned to scheduledReaderTask instead of scheduledAllTask.

Yup, looks like a typo to me. I think it ought to be possible to reproduce a bug there in a unit test, which would make a nice contribution along with a fix. Do you think you could do that?

Working on this one


ceph is about to revamp health details

This means all the places where ceph-ansible pokes at the output of ceph -s might need to be adapted.
When this lands, any test(s) that go against ceph master will probably start breaking
https://github.com/ceph/ceph/pull/15643

we are now using -f json as much as possible, I guess we can close this.


Laravel Linter

This pull request includes changes and recommendations for crafting your application "The Laravel Way". Feel free to commit any additional changes to the shift-77893 branch.
Before merging, you need to:

Checkout the shift-77893 branch
Review all pull request comments for additional changes
Thoroughly test your application (no tests?, no CI?)

Much of the lint detected may be automatically fixed by running the Laravel Fixer or tasks within the Shift Workbench.

:warning: Shift found uses of the old array() syntax. Laravel adopted the short array syntax [] since it became available in PHP 5.4.

:warning: Shift found instances of string based class references. Laravel adopted the ::class static property since it became available in PHP 5.5. You should update your code to use references like App\SomeModel::class instead of strings like 'App\SomeModel'.

:x: The following configuration files differ from the Laravel defaults. If you are modifying configuration values directly, consider using an environment variable instead to keep these files maintainable. You should compare your configuration files against the latest version and merge any updates.

[ ] config/app.php
[ ] config/auth.php
[ ] config/broadcasting.php
[ ] config/cache.php
[ ] config/database.php
[ ] config/filesystems.php
[ ] config/hashing.php
[ ] config/logging.php
[ ] config/mail.php
[ ] config/queue.php
[ ] config/services.php
[ ] config/session.php
[ ] config/view.php

:warning: The dates property was deprecated in Laravel 8.0 and will be removed in a future version of Laravel. Instead, you should use the casts property with a datetime type. Shift found the dates property in the following files:

[ ] app/Models/Journal.php
[ ] app/Models/JournalTransaction.php

:warning: The following files reference Laravel facades through the global namespace. For example, you're referencing \DB or importing use DB. Instead import Illuminate\Support\Facades\DB and reference DB.
While global references are allowed through aliases, you should import the facade explicitly. This can improve clarity not only for developers, but static analysis used by your IDE.

[ ] app/Console/Commands/DevInstall.php
[ ] app/Console/Commands/EmailTest.php
[ ] app/Console/Commands/ProcessQueue.php
[ ] app/Http/Controllers/Admin/RankController.php
[ ] app/Http/Controllers/Admin/TypeRatingController.php
[ ] app/Http/Controllers/Frontend/AirportController.php
[ ] app/Http/Requests/ImportRequest.php
[ ] app/Repositories/SettingRepository.php
[ ] app/Support/ClassLoader.php

:warning: The following classes do not extend the standard Laravel Model and Controller class. This may add complexity which makes your application harder to upgrade. Often a trait may be used instead of inheritance. You should review the following classes:

[ ] app/Models/Acars.php
[ ] app/Models/Aircraft.php
[ ] app/Models/Airline.php
[ ] app/Models/Airport.php
[ ] app/Models/Award.php
[ ] app/Models/Bid.php
[ ] app/Models/Enums/AcarsType.php
[ ] app/Models/Enums/ActiveState.php
[ ] app/Models/Enums/AircraftState.php
[ ] app/Models/Enums/AircraftStatus.php
[ ] app/Models/Enums/Days.php
[ ] app/Models/Enums/ExpenseType.php
[ ] app/Models/Enums/FareType.php
[ ] app/Models/Enums/FlightType.php
[ ] app/Models/Enums/FuelType.php
[ ] app/Models/Enums/ImportExportType.php
[ ] app/Models/Enums/JournalType.php
[ ] app/Models/Enums/NavaidType.php
[ ] app/Models/Enums/PageType.php
[ ] app/Models/Enums/PirepFieldSource.php
[ ] app/Models/Enums/PirepSource.php
[ ] app/Models/Enums/PirepState.php
[ ] app/Models/Enums/PirepStatus.php
[ ] app/Models/Enums/UserState.php
[ ] app/Models/Expense.php
[ ] app/Models/Fare.php
[ ] app/Models/File.php
[ ] app/Models/Flight.php
[ ] app/Models/FlightField.php
[ ] app/Models/FlightFieldValue.php
[ ] app/Models/Journal.php
[ ] app/Models/JournalTransaction.php
[ ] app/Models/Kvp.php
[ ] app/Models/Ledger.php
[ ] app/Models/Module.php
[ ] app/Models/Navdata.php
[ ] app/Models/News.php
[ ] app/Models/Page.php
[ ] app/Models/Permission.php
[ ] app/Models/Pirep.php
[ ] app/Models/PirepComment.php
[ ] app/Models/PirepFare.php
[ ] app/Models/PirepField.php
[ ] app/Models/PirepFieldValue.php
[ ] app/Models/Rank.php
[ ] app/Models/Role.php
[ ] app/Models/Setting.php
[ ] app/Models/SimBrief.php
[ ] app/Models/SimBriefXML.php
[ ] app/Models/Subfleet.php
[ ] app/Models/Typerating.php
[ ] app/Models/UserAward.php
[ ] app/Models/UserField.php
[ ] app/Models/UserFieldValue.php

:warning: The following controllers contain actions outside of the 7 resource actions (index, create, store, show, edit, update, destroy). For more details, review the docs or watch Cruddy by Design to see if you may rework these into resource controllers.

[ ] app/Http/Controllers/Admin/AircraftController.php
[ ] app/Http/Controllers/Admin/AirportController.php
[ ] app/Http/Controllers/Admin/DashboardController.php
[ ] app/Http/Controllers/Admin/ExpenseController.php
[ ] app/Http/Controllers/Admin/FareController.php
[ ] app/Http/Controllers/Admin/FlightController.php
[ ] app/Http/Controllers/Admin/MaintenanceController.php
[ ] app/Http/Controllers/Admin/ModulesController.php
[ ] app/Http/Controllers/Admin/PirepController.php
[ ] app/Http/Controllers/Admin/RankController.php
[ ] app/Http/Controllers/Admin/SubfleetController.php
[ ] app/Http/Controllers/Admin/TypeRatingController.php
[ ] app/Http/Controllers/Admin/UserController.php
[ ] app/Http/Controllers/Api/AcarsController.php
[ ] app/Http/Controllers/Api/AirlineController.php
[ ] app/Http/Controllers/Api/AirportController.php
[ ] app/Http/Controllers/Api/FleetController.php
[ ] app/Http/Controllers/Api/FlightController.php
[ ] app/Http/Controllers/Api/MaintenanceController.php
[ ] app/Http/Controllers/Api/PirepController.php
[ ] app/Http/Controllers/Api/StatusController.php
[ ] app/Http/Controllers/Api/UserController.php
[ ] app/Http/Controllers/Frontend/FlightController.php
[ ] app/Http/Controllers/Frontend/LanguageController.php
[ ] app/Http/Controllers/Frontend/PirepController.php
[ ] app/Http/Controllers/Frontend/ProfileController.php
[ ] app/Http/Controllers/Frontend/SimBriefController.php
[ ] app/Http/Controllers/System/ImporterController.php
[ ] app/Http/Controllers/System/InstallerController.php
[ ] app/Http/Controllers/System/UpdateController.php

:warning: Shift detected the following HTTP components using the Request or Input facade to access the request. Within these components you should leverage the request object which is automatically injected to any controller action or Middleware.

[ ] app/Http/Controllers/Admin/Traits/Importable.php

:warning: Shift found inline validation in the following controllers. Unless you require this level of control, you should use a Form Request to encapsulate this validation logic and keep your controller clean. You may automate this conversion with the Laravel Fixer.

[ ] app/Http/Controllers/Admin/FileController.php
[ ] app/Http/Controllers/Admin/FlightController.php
[ ] app/Http/Controllers/Auth/LoginController.php
[ ] app/Http/Controllers/Frontend/ProfileController.php
[ ] app/Http/Controllers/System/InstallerController.php

:warning: Shift detected the following HTTP components accessing the authenticated user through Auth::user(). Within these components, you may access the authenticated user through the request object as well. Doing so limits your dependencies by leveraging objects already available.

[ ] app/Http/Controllers/Admin/DashboardController.php
[ ] app/Http/Controllers/Admin/PirepController.php
[ ] app/Http/Controllers/Api/FlightController.php
[ ] app/Http/Controllers/Api/PirepController.php
[ ] app/Http/Controllers/Api/UserController.php
[ ] app/Http/Controllers/Auth/LoginController.php
[ ] app/Http/Controllers/Frontend/DashboardController.php
[ ] app/Http/Controllers/Frontend/FlightController.php
[ ] app/Http/Controllers/Frontend/PirepController.php
[ ] app/Http/Controllers/Frontend/ProfileController.php
[ ] app/Http/Controllers/Frontend/SimBriefController.php

:warning: Laravel contains many built-in helpers functions which allow you to write more expressive code or leverage core functionality from anywhere in your app.
Shift found the following code which could use a helper instead:

[ ] Config::get()

:warning: Shift found opportunities to use the built-in Blade directives like @auth, @csrf in the following views:

[ ] resources/views/admin/app.blade.php
[ ] resources/views/layouts/default/airports/show.blade.php
[ ] resources/views/layouts/default/app.blade.php
[ ] resources/views/layouts/default/auth/passwords/email.blade.php
[ ] resources/views/layouts/default/nav.blade.php
[ ] resources/views/layouts/default/profile/index.blade.php
[ ] resources/views/system/importer/app.blade.php
[ ] resources/views/system/installer/app.blade.php
[ ] resources/views/system/updater/app.blade.php

:warning: To improve performance by caching your configuration, Laravel recommends you only use the env() helper within configuration files and use config() everywhere else. Shift found potential uses of env() in:

[ ] app/Http/Controllers/Auth/RegisterController.php

:x: Laravel 5.3 added a top-level routes folder for storing your app routes. You should create this folder and move your routes into one of the default api.php or web.php route files.

:x: Shift found files using the Route methods within your app folder. You should consolidate your routes into one of the default route/api.php or route/web.php route files.

[ ] app/Providers/RouteServiceProvider.php

:information_source: Shift detected controller namespaces being set in your RouteServiceProvider. Laravel 8 began registering routes using static class references instead of namespace prefixes and action strings.

:warning: Laravel uses PSR-4 autoloading for the app/ folder. While you may configure your application to load these files within composer.json, all files under the app folder should be classes with a PSR-4 compliant namespace.
The following PHP files are missing a namespace:

[ ] app/Database/migrations/2017_06_07_014930_create_settings_table.php
[ ] app/Database/migrations/2017_06_08_0000_create_users_table.php
[ ] app/Database/migrations/2017_06_08_0001_roles_permissions_tables.php
[ ] app/Database/migrations/2017_06_08_0005_create_password_resets_table.php
[ ] app/Database/migrations/2017_06_08_0006_create_sessions_table.php
[ ] app/Database/migrations/2017_06_08_191703_create_airlines_table.php
[ ] app/Database/migrations/2017_06_09_010621_create_aircrafts_table.php
[ ] app/Database/migrations/2017_06_10_040335_create_fares_table.php
[ ] app/Database/migrations/2017_06_11_135707_create_airports_table.php
[ ] app/Database/migrations/2017_06_17_214650_create_flight_tables.php
[ ] app/Database/migrations/2017_06_21_165410_create_ranks_table.php
[ ] app/Database/migrations/2017_06_23_011011_create_subfleet_tables.php
[ ] app/Database/migrations/2017_06_28_195426_create_pirep_tables.php
[ ] app/Database/migrations/2017_12_12_174519_create_bids_table.php
[ ] app/Database/migrations/2017_12_14_225241_create_jobs_table.php
[ ] app/Database/migrations/2017_12_14_225337_create_failed_jobs_table.php
[ ] app/Database/migrations/2017_12_20_004147_create_navdata_tables.php
[ ] app/Database/migrations/2017_12_20_005147_create_acars_tables.php
[ ] app/Database/migrations/2018_01_03_014930_create_stats_table.php
[ ] app/Database/migrations/2018_01_08_142204_create_news_table.php
[ ] app/Database/migrations/2018_01_28_180522_create_awards_table.php
[ ] app/Database/migrations/2018_02_26_185121_create_expenses_table.php
[ ] app/Database/migrations/2018_02_28_231807_create_journal_transactions_table.php
[ ] app/Database/migrations/2018_02_28_231813_create_journals_table.php
[ ] app/Database/migrations/2018_02_28_232438_create_ledgers_table.php
[ ] app/Database/migrations/2018_04_01_193443_create_files_table.php
[ ] app/Database/migrations/2019_06_19_220910_add_readonly_to_roles.php
[ ] app/Database/migrations/2019_07_16_141152_users_add_pilot_id.php
[ ] app/Database/migrations/2019_08_30_132224_create_notifications_table.php
[ ] app/Database/migrations/2019_09_16_141152_pireps_change_state_type.php
[ ] app/Database/migrations/2019_10_30_141152_pireps_add_flight_id.php
[ ] app/Database/migrations/2020_02_12_141152_expenses_add_flight_type.php
[ ] app/Database/migrations/2020_02_26_044305_modify_airports_coordinates.php
[ ] app/Database/migrations/2020_03_04_044305_flight_field_nullable.php
[ ] app/Database/migrations/2020_03_05_141152_flights_add_load_factor.php
[ ] app/Database/migrations/2020_03_06_141152_flights_add_pilot_pay.php
[ ] app/Database/migrations/2020_03_06_141153_fares_add_type.php
[ ] app/Database/migrations/2020_03_09_141152_increase_id_lengths.php
[ ] app/Database/migrations/2020_03_09_141153_remove_subfleet_type_index.php
[ ] app/Database/migrations/2020_03_11_141153_add_simbrief_table.php
[ ] app/Database/migrations/2020_03_27_174238_create_pages.php
[ ] app/Database/migrations/2020_03_28_174238_airline_remove_nullable.php
[ ] app/Database/migrations/2020_03_28_174238_page_icon_nullable.php
[ ] app/Database/migrations/2020_06_09_141153_pages_add_link.php
[ ] app/Database/migrations/2020_07_21_141153_create_user_fields.php
[ ] app/Database/migrations/2020_09_03_141152_aircraft_add_mtow.php
[ ] app/Database/migrations/2020_09_30_081536_create_modules_table.php
[ ] app/Database/migrations/2020_11_20_044305_modify_pages_size.php
[ ] app/Database/migrations/2020_11_26_044305_modify_download_link_size.php
[ ] app/Database/migrations/2021_01_17_044305_add_hub_to_subfleets.php
[ ] app/Database/migrations/2021_02_10_044305_change_acars_vs_type.php
[ ] app/Database/migrations/2021_02_11_044305_update_sessions_table.php
[ ] app/Database/migrations/2021_02_23_150601_aircraft_add_fuelonboard.php
[ ] app/Database/migrations/2021_02_23_205630_add_sbtype_to_subfleets.php
[ ] app/Database/migrations/2021_03_01_044305_add_aircraft_to_simbrief.php
[ ] app/Database/migrations/2021_03_05_044305_add_kvp_table.php
[ ] app/Database/migrations/2021_03_18_161419_add_disableactivitychecks_to_roles.php
[ ] app/Database/migrations/2021_03_25_213017_remove_setting_removebidonaccept.php
[ ] app/Database/migrations/2021_04_05_055245_flights_add_alphanumeric_callsign.php
[ ] app/Database/migrations/2021_04_10_055245_migrate_configs.php
[ ] app/Database/migrations/2021_05_21_141152_increase_icao_sizes.php
[ ] app/Database/migrations/2021_05_28_165608_remove_setting_simbriefexpiredays.php
[ ] app/Database/migrations/2021_06_01_141152_discord_fields.php
[ ] app/Database/migrations/2021_06_04_141152_discord_private_channel_id.php
[ ] app/Database/migrations/2021_11_23_184532_add_type_rating_tables.php
[ ] app/Database/migrations/2021_11_27_132418_add_hub_to_aircraft.php
[ ] app/Database/migrations/2022_01_10_131604_update_awards_add_active.php
[ ] app/Database/migrations/2022_02_11_124926_update_users_add_notes.php
[ ] app/Database/migrations/2022_03_09_152342_ignore_admin_activity_checks.php
[ ] app/Database/migrations/2022_07_12_142108_add_notes_to_airports.php
[ ] app/Database/migrations/2022_08_20_213507_add_callsign_to_airlines.php
[ ] app/Database/migrations/2022_12_17_211036_add_callsign_to_users.php
[ ] app/Database/seeds/DatabaseSeeder.php
[ ] app/helpers.php

:information_source: As noted, much of the lint detected above can be automatically fixed using the Laravel Fixer. Save yourself time and clean up your codebase quickly with this new Shift.


Losing Mac resource forks (and all extended attrs) over 1 KB

Output of restic version
restic 0.16.5 compiled with go1.22.4 on linux/amd64
What backend/service did you use to store the repository?
plain filesystem
Problem description / Steps to reproduce
restic fairly quietly loses the contents of any extended attributes > 1 KB. (I've submitted a patch upstream—see the link below—in case you want to avoid reading my novel.)

On a Linux host, create a file with an extended attribute of > 1KB. You may use setfattr, but my own situation was fairly involved: a TrueNAS SCALE deployment (which is Debian-based) running OpenZFS (with xattr=sa on the relevant dataset, though that may not matter), holding a resource-fork-bearing Mac file moved over via SMB. (Phew!) The resource fork in question was 161554 bytes; the data fork was 797075.

% RESTIC_PASSWORD=whatever RESTIC_REPOSITORY=/mnt/tank/stuff/someuser/restic-repo restic backup TheChallengingFile
repository 38dc2a0e opened (version 2, compression level auto)
created new cache in /mnt/tank/stuff/someuser/.cache/restic
no parent snapshot found, will read all files

can not obtain extended attribute user.DosStream.AFP_Resource:$DATA for TheChallengingFile:

Files:           1 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repository: 779.424 KiB (344.639 KiB stored)

processed 1 files, 778.394 KiB in 0:00
snapshot 67fadfa5 saved

Note the "can not obtain extended attribute" error above. (This should perhaps be a fatal error, since it indicates the loss of all data for most legacy Mac apps.)
mkdir restic-restore

% RESTIC_PASSWORD=whatever RESTIC_REPOSITORY=/mnt/tank/stuff/someuser/restic-repo restic restore 67fadfa5 --target restic-restore
restoring <Snapshot 67fadfa5 of [/mnt/tank/stuff/someuser/TheChallengingFile] at 2024-07-20 19:41:14.147516317 -0700 PDT by someuser@bags> to /mnt/tank/stuff/someuser/repro-restore
Summary: Restored 2 files/dirs (778.394 KiB) in 0:00

Expected behavior
Ideally, restoring a backed up file would reproduce its original extended attrs. Restic does make an effort at this, and the trouble is really in the particulars, for which I've submitted a patch upstream.
Actual behavior

The original file had several extended attributes:
% getfattr TheChallengingFile
# file: TheChallengingFile
user.DOSATTRIB
user.DosStream.AFP_AfpInfo:$DATA
user.DosStream.AFP_Resource:$DATA

The original size of the (base64 encoded) user.DosStream.AFP_Resource:$DATA attr was…
% getfattr -n 'user.DosStream.AFP_Resource:$DATA' TheChallengingFile | wc -c
215466

The size of that attr on the restored file was 0, so the whole resource fork was lost. Indeed, the whole attr was absent.
% getfattr -n 'user.DosStream.AFP_Resource:$DATA' repro-restore/TheChallengingFile
repro-restore/TheChallengingFile: user.DosStream.AFP_Resource:$DATA: No such attribute
0

However, the smaller attrs survived:
% getfattr repro-restore/TheChallengingFile
# file: repro-restore/TheChallengingFile
user.DOSATTRIB
user.DosStream.AFP_AfpInfo:$DATA

Do you have any idea what may have caused this?
The problem lies in the xattr package, which restic uses for extended-attribute handling. In xattr.go, it states...
// The theoretical maximum xattr value size on MacOS is 64 MB. On Linux it's
// much smaller at 64 KB. Unless the kernel is evil or buggy, we should never
// hit the limit.

That 64 KB limit is not actually so on TrueNAS, and the error codes are different as well. This is shown by the 215466-byte result above and verification on the Mac side via SMB, where the resource fork of the ZFS-stored version of the file is of equal size to the APFS-stored original.
I've submitted a patch at https://github.com/pkg/xattr/pull/73.
Recommendation
If and when my patch is accepted by xattr, it would be great if restic would update to a version that includes it. Then all 4 of us legacy Mac people would have a viable backup solution on TrueNAS. (Kopia doesn't even try for extended attrs.)
Did restic help you today? Did it make you happy in any way?
I love the simplicity of its CLI! I will be quite happy if I can just overcome this last hurdle to putting it into production. :-D Thanks for all your hard work!

Upstream patch is now merged.

Note the "can not obtain extended attribute" error above. (This should perhaps be a fatal error, since it indicates the loss of all data for most legacy Mac apps.)

The proper result would be for backup to return exit code 3 (some files could not be backed up) in these cases. The main problem is that we still have to find a way to properly distinguish between warnings and important errors when reading the xattrs without creating a mess in the code.

I've submitted a patch at pkg/xattr#73.

Thanks :-) !

That 64 KB limit is not actually so on TrueNAS, and the error codes are different as well. This is shown by the 215466-byte result above and verification on the Mac side via SMB, where the resource fork of the ZFS-stored version of the file is of equal size to the APFS-stored original.

I'm somewhat surprised that the resource forks are stored in extended attributes even though the linux kernel imposes a limit of 64kb. Normally Samba just stores them as separate files.

Upstream patch is now merged.

I'll open a PR to update the library. Restic's repository format is currently not optimized for larger xattrs. See https://github.com/restic/restic/issues/3643 for more context.

The proper result would be for backup to return exit code 3

I'm all in favor! :-)

I'm somewhat surprised that the resource forks are stored in extended attributes even though the linux kernel imposes a limit of 64kb.

Yes, I'm not sure what's going on either. It sure doesn't seem to be imposing it. I wouldn't put it past TrueNAS to make some kernel patches.

Normally Samba just stores them as separate files.

Went digging a bit. According to the Samba docs, if using vfs_fruit's fruit:resource = xattr, it will "use a xattr, requires a filesystem with large xattr support and a file IO API compatible with xattrs, this boils down to Solaris and derived platforms and ZFS". Perhaps TrueNAS made some kernel adjustments to allow the Full Power Of ZFS to shine through? For the record, I didn't touch the Samba config, so I have whatever TrueNAS's defaults are. There's nothing explicit in the config file about vfs_fruit one way or the other. Though I certainly have no ._ AppleDouble files on my drive.

I'll open a PR to update the library version used by restic.

That was quick (as was xattr's release)! Impressive!

Restic's repository format is currently not optimized for larger xattrs. See https://github.com/restic/restic/issues/3643 for more context.

Thanks for the warning! Sure beats dropping them on the floor, though.
Thanks again for the quick patch! :-D

Yes, I'm not sure what's going on either. It sure doesn't seem to be imposing it. I wouldn't put it past TrueNAS to make some kernel patches.

The Linux kernel includes code to hard limit the xattr size to 64kb. Storing or reading larger xattrs is impossible without patching the kernel.

The exit code handling still has to be fixed.

use a xattr, requires a filesystem with large xattr support and a file IO API compatible with xattrs, this boils down to Solaris and derived platforms and ZFS. Perhaps TrueNAS made some kernel adjustments to allow the Full Power Of ZFS to shine through?

Just a wild guess here; but TrueNAS originally stems from FreeNAS and the FreeBSD world, where the 64-KiB-limit does not necessarily apply. For me, this sounds like a bug where this edge-case on Linux host systems was not taken into account (with 2022, full/official Linux support is comparatively young; and AFAIK it's still mostly used on FreeBSD hosts).

I am running TrueNAS Scale EEL and the TrueCloud throws this error still:
can not obtain extended attribute user.DosStream.AFP_Resource:$DATA for .....<many files>
Hopefully fixed in next TN release?


Add sandboxing by account

Currently we support only one simulation stack. Make it so that each account have their own simulation stack

After our conversation today, I think we should have one sandbox per account, and each tab is connected to one sandbox.  A sandbox can have multiple associated tabs.  Each sandbox would have one simulation stack.  When you switch accounts, switch sandboxes (which means your simulation stack switches to the stack of that sandbox).

depends on https://github.com/DarkFlorist/TheInterceptor/issues/46


[WIP] Adding Basic Auth Support

Sorry for the amount of Commits. I am still getting used to using Git! I added support for Basic Auth using the HTTPListener methods. I would suggest using HTTPS when using Basic Auth, because the password is clear text in the request.
This should work on Linux. I do not expect this to be merged yet. I would like to work on tests firsts. I will also work on trying this in Linux. I did some testing on PowerShell core on Windows.

I gave it a shot to switch to using [System.Net.AuthenticationSchemes] as type for the parameter. I believe we should stick to using ValidateSet instead of passing in the correct type. This allows us to limit what auth schema's we allow. I also ran into an issue where the assembly wasn't loaded in the ISE to create the object before calling Start-Polaris. I think it may also help with future proofing, for when we switch to Kestrel. @tylerl0706 Thoughts?

Does this do any sort of validation on allowed users?

@awickham10 There is a method IsInRole method on the User object in the request. But, I don't believe it works with basic auth. Let me try adding negotiate auth and see if IsInRole works. The worst case, is you could add middleware that checks membership with Get-ADGroupMember.

@awickham10 I added NTLM and Negotiate auth. Here is an example below on how to check if the user is in a group. Can you test this when you have some time?
`Import-Module C:\scripts\Polaris\Polaris.psd1
New-PolarisGetRoute -Path "/helloworld" -Scriptblock {
$IsInGroup = $request.User.IsInRole('DOMAIN\AD group')
$Response.Send($IsInGroup)
}
Start-Polaris -Port 443 -Https -Auth Negotiate`

Just out of curiosity, have you been testing on Linux at all?
This could just mean throwing an "Unsupported platform" exception.

I have not. I am starting to do research to figure out which Auth Schemes are supported on Linux. It seems searching corefx issues are the only way to find this information. I have tested on core on Windows, and that does work as expected.

I did some work on this tonight. It will now throw errors if you you try to use anything other than Basic or Anonymous auth on Linux. I did test Basic Auth on linux and it does work. I tested the other auth providers on PowerShell for Windows and Core. I can work on tests to make sure the expected errors are thrown.
If we are able to create users in the test scripts, we can probably test most auth providers.

@Tiberriver256 I added tests validating that errors were thrown, when applicable. Do you think more tests should be added? There was a random one that failed on Linux. But, I think it was a false positive. Is there a way to fire off AppVeyor again?

@TylerLeonhardt - Did you want to see any other tests added on this?


MOM6 in GEOS - Part 1

This PR enables the checkout of MOM6 source code and stages it with existing MOM5.
It also adds a plug.  The plug and the source code are not used, as a result, no change in output (compile time or runtime) is expected.  GuestOcean_GridComp.F90 is left untouched.
A second PR will make use of this new code. Changes will be limited to the GuestOcean_Grid Comp.F90 and accompanying CMakeLists.txt
This is ongoing work related to issue GEOS-ESM/GEOSgcm_GridComp#70
Change-Id: I75bf29d16e626f68d4bafadb3ba3ecac909cd9ff

@aerorahul I assume branch = geos is just until MOM6 is finalized and working?

@aerorahul I assume branch = geos is just until MOM6 is finalized and working?

@mathomp4 This is the branch of MOM6 on our fork.  Are you thinking of creating something along the lines of geos/master , geos/develop. In which case this will eventually become geos/develop or a tag when we have active MOM6 development.
This is not to be confused with the FMS branch geos which contains the latest changes from xanadu tags.

@aerorahul I assume branch = geos is just until MOM6 is finalized and working?

@mathomp4 This is the branch of MOM6 on our fork. Are you thinking of creating something along the lines of geos/master , geos/develop. In which case this will eventually become geos/develop or a tag when we have active MOM6 development.
This is not to be confused with the FMS branch geos which contains the latest changes from xanadu tags.

@aerorahul I just wanted to make sure that the branch is what you wanted rather than a fixed tag. I'm guessing you have development ongoing and will provide a tag the next time develop merges into master?

@aerorahul I assume branch = geos is just until MOM6 is finalized and working?

@mathomp4 This is the branch of MOM6 on our fork. Are you thinking of creating something along the lines of geos/master , geos/develop. In which case this will eventually become geos/develop or a tag when we have active MOM6 development.
This is not to be confused with the FMS branch geos which contains the latest changes from xanadu tags.

@aerorahul I just wanted to make sure that the branch is what you wanted rather than a fixed tag. I'm guessing you have development ongoing and will provide a tag the next time develop merges into master?

yes. That will be fine for now.  Once we can actually exercise this option, we can start tagging.

Okay. Give me some more time. I might have to do some fun to make this generally supported. I think I might have to rebuild bits of Baselibs because of the use of https URLs in the submodules inside of MOM6. Hmm...

Hmm. I might be able to escape this with a few lines of change in @env changing order of modules. Unexpected, but possible.

@mathomp4 Can't we just put an Externals.cfg in the MOM6 fork and remove the submodules? Seems easy that way.
One of the repo is kdtree and that is not changing or going away.

@aerorahul This is something that will happen at some point anyway, may as well figure it out now. The problem is that if you load g5_modules and then try to run checkout_externals you get a lot of:
Checking out externals: geoschem_gridcomp, mom6, ERROR:root:Command '[u'git', u'submodule', u'update', u'--init', u'--recursive']' returned non-zero exit status 1
ERROR:root:Failed with output:
    Submodule 'pkg/CVMix-src' (https://github.com/CVMix/CVMix-src.git) registered for path 'pkg/CVMix-src'
    Submodule 'pkg/MOM6_DA_hooks' (https://github.com/MJHarrison-GFDL/MOM6_DA_hooks.git) registered for path 'pkg/MOM6_DA_hooks'
    Submodule 'pkg/geoKdTree' (https://github.com/travissluka/geoKdTree.git) registered for path 'pkg/geoKdTree'
    Cloning into '/gpfsm/dswdev/mathomp4/Models/CURLTEST-GEOSgcm-develop-with-MOM6part1/GEOSgcm/src/Components/<EMAIL_ADDRESS>    fatal: unable to access 'https://github.com/CVMix/CVMix-src.git/': Protocol "https" not supported or disabled in libcurl
    fatal: clone of 'https://github.com/CVMix/CVMix-src.git' into submodule path '/gpfsm/dswdev/mathomp4/Models/CURLTEST-GEOSgcm-develop-with-MOM6part1/GEOSgcm/src/Components/@GEOSgcm_GridComp/GEOSogcm_GridComp/GEOSocean_GridComp/GuestOcean_GridComp/MOM6_GEOSPlug/@mom6/pkg/CVMix-src' failed
    Failed to clone 'pkg/CVMix-src'. Retry scheduled
    Cloning into '/gpfsm/dswdev/mathomp4/Models/CURLTEST-GEOSgcm-develop-with-MOM6part1/GEOSgcm/src/Components/<EMAIL_ADDRESS>    fatal: unable to access 'https://github.com/MJHarrison-GFDL/MOM6_DA_hooks.git/': Protocol "https" not supported or disabled in libcurl

It looks like this is due to the libcurl in SIVO-PyD and the stupid old version of OpenSSL on discover SLES 11. Without g5_modules loaded:
(1008)(master) $ ldd /gpfsm/dulocal/sles11/other/SLES11.3/git/2.21.0/libexec/git-core/git-http-push | grep curl
	libcurl.so.4 => /gpfsm/dulocal/sles11/other/SLES11.3/git/2.21.0/lib/libcurl.so.4 (0x00002aaaaaaad000)

with g5_modules loaded:
❯ ldd /gpfsm/dulocal/sles11/other/SLES11.3/git/2.21.0/libexec/git-core/git-http-push | grep curl
	libcurl.so.4 => /usr/local/other/SLES11.3/SIVO-PyD/1.25.1_gcc6.3_mkl<IP_ADDRESS>/lib/libcurl.so.4 (0x00002aaaaaaae000)

But, I think we can avoid this simply by loading GEOSenv after SIVO-PyD so that the LD_LIBRARY_PATH is uncontaminated.

@aerorahul You should have a pull request into your feature/mom6-struct branch on GEOSgcm where I've advanced ESMA_envto 1.3.3 in Develop.cfg. That, I think, solves the issue simply. If you agree with the change, merge and let me know and I'll do another test to be sure.

@mathomp4 Once FMS dependency is satisfied, this branch feature/mom6-struct will need to be brought up to date with the latest develop.  I don't expect any conflicts, since this work is completely new w.r.t. existing code base.


vonage.errors.AuthenticationError: Authentication failed. Check you're using a valid authentication method.

Hola tengo este error : vonage.errors.AuthenticationError: Authentication failed. Check you're using a valid authentication method.
Linux xubuntu 22.04
Python 3.10
pip3 install vonage
Cuando trato de ejecutar el siguiente codigo :
#################################### Codigo #############################################
import os
from os.path import join, dirname
VONAGE_APPLICATION_ID = "f6e2b6af-2610-408d-99f9-c41e2248e03d"
VONAGE_APPLICATION_PRIVATE_KEY_PATH = "/home/android/Proyectos/Corato_V2/private.key"
TO_NUMBER = **********
FROM_NUMBER = **********
import vonage
client = vonage.Client(
application_id=VONAGE_APPLICATION_ID,
private_key=VONAGE_APPLICATION_PRIVATE_KEY_PATH,
)
client.messages.send_message({
"channel": "whatsapp",
"message_type": "text",
"to": "**********",
"from": "***********",
"text": "This is a WhatsApp text message sent using the Vonage Messages API",
})
-----https://github.com/Vonage/vonage-python-code-snippets/blob/main/messages/whatsapp/send_text.py
$ python3  send_text.py
#######################################################################################
Ya intente con key y secret pero no funciona, que estoy haciendo mal ???
Gracias de antemano.
Stack trace :
Traceback (most recent call last):
File "/home/android/Downloads/ejemplos_vonage/messages/whatsapp/send_text.py", line 23, in
client.messages.send_message(
File "/home/android/Downloads/ejemplos_vonage/env/lib/python3.10/site-packages/vonage/messages.py", line 25, in send_message
return self._client.post(
File "/home/android/Downloads/ejemplos_vonage/env/lib/python3.10/site-packages/vonage/client.py", line 231, in post
return self.parse(
File "/home/android/Downloads/ejemplos_vonage/env/lib/python3.10/site-packages/vonage/client.py", line 277, in parse
raise AuthenticationError("Authentication failed. Check you're using a valid authentication method.")

Hi, to use WhatsApp as a message channel for the Messages API, you must first register a WhatsApp Business Account to be associated with the WhatsApp messages you send. WhatsApp also have to verify that your business is legitimate, which gives you a green tick. You can read more about this here: https://developer.vonage.com/en/messages/concepts/whatsapp
You can register for a WhatsApp account with Vonage in the developer dashboard.
Hope this helps!

I've gotten this issue as well but when using Messages API to send an SMS. I think I've narrowed it done to an issue with the application_id but not sure. Created another issue here in case it's different from this one.

Hi! Im getting this auth error as well with the Voice api. I made sure the applicationi_d and all other secrets are correct. It initially works but after a certain amount of time I receive the following when trying:
ERROR: Error when triggering call: Authentication failed. Traceback (most recent call last):   File "/app/services/vonage_service.py", line 69, in trigger_vonage_call     response = vonage_client.voice.create_call(                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/lib/python3.11/site-packages/vonage/voice.py", line 29, in create_call     return self._client.post(            ^^^^^^^^^^^^^^^^^^   File "/usr/local/lib/python3.11/site-packages/vonage/client.py", line 295, in post     return self.parse(            ^^^^^^^^^^^   File "/usr/local/lib/python3.11/site-packages/vonage/client.py", line 395, in parse     raise AuthenticationError("Authentication failed.") vonage.errors.AuthenticationError: Authentication failed.
A restart of the backend makes it temporarily work again

Hi, this is quite a general error that's returned in a lot of cases. My suspicion is that the JWT used by your application is expiring as it has a default timeout of 15 minutes. You can change this using the auth method in the client class by setting a value for exp as an argument.
Hope this helps!

Note on the above: a new JWT is generated every request, but if an individual request takes time or isn't actioned due to async code etc., this can cause this problem. I'll close this issue now as the original reason it was kept open was to signpost the need to register a WhatsApp business channel, I think it's better now to have people create their own issues when they run into them.


The "Top-friends" list will order itself from best friend to least friend.

functionality added with commit: 5415a93


#0: Resolve Versim failure after Virtual Coordinates commit

Ticket
No ticket.
Problem description
Virtual Coordinates commit broke Versim runs. The Versim driver does not support querying Virtual/Translated coordinates.
What's changed
Make Virtual Coordinates == Physical Coordinates for Versim.
Checklist

[ ] Post commit CI passes
[ ] Blackhole Post commit (if applicable)
[ ] Model regression CI testing passes (if applicable)
[ ] Device performance regression CI testing passes (if applicable)
[ ] (For models and ops writers) Full new models tests passes
[ ] New/Existing tests provide coverage for changes

fyi @vtangTT


openssl/ssl.h file not found when attemping to build

Hi there,
I just tried to compile my project after adding hyper as a dependency and received this error:
$ cargo build --release
   Compiling openssl-sys-extras v0.7.14
   Compiling openssl v0.7.14
Build failed, waiting for other jobs to finish...
error: failed to run custom build command for `openssl v0.7.14`
process didn't exit successfully: `/Users/fabioberger/Documents/projects/rust/lighthouse/target/release/build/openssl-5464f8f6e728c35a/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("x86_64-apple-darwin")
OPT_LEVEL = Some("3")
PROFILE = Some("release")
TARGET = Some("x86_64-apple-darwin")
debug=false opt-level=3
HOST = Some("x86_64-apple-darwin")
TARGET = Some("x86_64-apple-darwin")
TARGET = Some("x86_64-apple-darwin")
HOST = Some("x86_64-apple-darwin")
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = None
HOST = Some("x86_64-apple-darwin")
TARGET = Some("x86_64-apple-darwin")
HOST = Some("x86_64-apple-darwin")
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-m64" "-fPIC" "-o" "/Users/fabioberger/Documents/projects/rust/lighthouse/target/release/build/openssl-5464f8f6e728c35a/out/src/c_helpers.o" "-c" "src/c_helpers.c"
cargo:warning=src/c_helpers.c:1:10: fatal error: 'openssl/ssl.h' file not found
cargo:warning=#include <openssl/ssl.h>
cargo:warning=         ^
cargo:warning=1 error generated.
ExitStatus(ExitStatus(256))

command did not execute successfully, got: exit code: 1

--- stderr
thread 'main' panicked at 'explicit panic', /Users/fabioberger/.cargo/registry/src/github.com-1ecc6299db9ec823/gcc-0.3.37/src/lib.rs:952
stack backtrace:
   1:        0x102e36ab9 - std::sys::backtrace::tracing::imp::write::h00e948915d1e4c72
   2:        0x102e3c9e0 - std::panicking::default_hook::_{{closure}}::h7b8a142818383fb8
   3:        0x102e3bd30 - std::panicking::default_hook::h41cf296f654245d7
   4:        0x102e3c2f6 - std::panicking::rust_panic_with_hook::h4cbd7ca63ce1aee9
   5:        0x102e1da94 - std::panicking::begin_panic::heb5541c22bb99ccf
   6:        0x102e29fee - gcc::fail::h035267235bacb8f3
   7:        0x102e29a86 - gcc::run::h10dd5abb3da241f3
   8:        0x102e1fa7e - gcc::Config::compile::h0f13da9d578bb106
   9:        0x102e1c805 - build_script_build::main::hb7741c2c34a9f03d
  10:        0x102e3cfca - __rust_maybe_catch_panic
  11:        0x102e3b7fe - std::rt::lang_start::h53bf99b0829cc03c

Any help with how to resolve this would be greatly appreciated!
Fabio

I ran into the same issue, installing a new version of OpenSSL worked for me, by for example running brew install openssl. And then adding the code below to your ~/.profile.
export OPENSSL_INCLUDE_DIR=`brew --prefix openssl`/include
export OPENSSL_LIB_DIR=`brew --prefix openssl`/lib

Basically the rust-openssl package needs headers > v1.0 (?) and the default ones in Mac OS are from version 0.9.8, see https://github.com/sfackler/rust-openssl#osx.

One thing to watch out for: the openssl environment variables need an absolute path since the openssl build runs in a different  directory than where cargo is invoked. The same error is reported for relative paths as for openssl just being missing.

If you're running this on a Mac, you may want to use the security-framework feature instead. In the long term, this is my plan: https://github.com/hyperium/hyper/issues/907#issuecomment-255509020

I'm facing the same, although i download OPENSSL for windows (openssl-win64) nothing changed , still  the same message "openssl/bn.h" no such file or directory .. any one can help , note: i am trying to run a cryptographic proof of data integrity written in C on windows8 machine ..