Basic wireguard IPv6 (NAT) support

Following my comment (https://github.com/pivpn/pivpn/discussions/1394#discussioncomment-2159928) I tried to integrate IPv6 for pivpn.

This PR adds basic IPv6 support for Wireguard using IPv6 NAT. IPv6 NAT is more or less ugly but works fine, since requests are desired  be coming from the public IPv6 of the VPN server and not the client IPv6 (in most cases).

Since the IPv6 interface can differ from the IPv4 interface it's configurable. Usually it's the same.
I am not sure what has to be done to have all tests run successfully since IPv6 has : and :: which might cause some trouble.
I have no experience with CLI GUI stuff, hence I hope someone can adjust the code if it's not working

WG IPv6 basically works on multiple installations I am maintaining. Now some people need to test and help finalize it to make this work with pivpn.
Edit: I fixed lint errors but don't know what is wrong with the setup tests since the setup seems to finish without errors but wireguard doesn't seem to be able to start. Can someone please check on that?
Thanks!

Tried it out, log said:
ip6tables v1.8.2 (legacy): Empty interface is likely to be undesired
Try `ip6tables -h' or 'ip6tables --help' for more information.

I can't get it to connect. Also, shouldn't there be an IPv6 subnet in the Address field in the configs like this?

Thanks for that info!
Reading your post-error comments was fun! IPv6 can be annoying sometimes but it's actually not much different than IPv4.

@DerDanilo not sure if you are local testing before hand, but just want to make sure you  please do try some local testing before pushing to the MR, Travis CI has usage limits and even tho they are quite kind in raising them if needed i'd rather avoid it.
Once all tests are passing and you feel code is ready to go let me know.
I will try to find some time during the week to assist, review and test.

@4s3ti I just launched a clean test machine today to do more testing even though I thought I tested everything already. Found the bug with the missing IPv6dev reference, which is fixed now.
The annoying thing with local tests is that the install script will always install the master upstream repo (into pivpnFilesDir="/usr/local/src/pivpn"), which can be changed by hand of course, but the branch then is wrong, since I am not pushing against the fork master obviously. Maybe this can be improved for future testing, allowing providing such option on the CLI via options or maybe just env vars. 'custom_git_repo_url' and 'custom_git_branch' (to checkout after clone).
These options then should also be referenced in the tests to have those test properly. I don't have much experience with Travis and hence don't know how that behaves.
Looking forward to have this fixed. Any help is appreciated.

Other question: Can we add an option to disable any script interaction with the firewall at all? I understand that this should be handled by the script for folks that don't work on it otherwise. But it is a valid option to have the script not fiddle with any firewall if the user wishes so. Could be an option in the unattended file and should possible be an additional question in the whiptail dialog.

I am really struggling to find time :(
@DerDanilo there are ways to test the pipelines locally .. but to be honest .. the easy way is really to just push and look at the build :p
What i usually do is to spin up a cloud instance and try to replicate the build environment and see what happens.
I usually use hetzner as my cloud provider for this kind of things, and i've noticed it gets quite close to the  build environment, however, but i believe any clean debian system might work.

There is also the issue with iptables being replace by nftables. We could also add support for that.

I'm planning to do this.

Why? Most people don't fiddle with IPv6 on their system, hence link local addresses are enabled by default.

I mean actual IPv6 internet connectivity, a global unicast address and possibly a reachability test.

I think that it should be clear that any firewall in front of the server needs editing to make this work.

That comment was a suggestion for the user above, clearly you can't change the router firewall from the Pi, unless UPnP is enabled (I think automatic port forwarding was requested in the past).

It is also not required if one wishes to connect to a DNS name which is the best option here. Those can contain dynamic IPs without any config change requirements. With dual stack a DNS name is anyways the best option, if one wishes to use that. So people either put their IP there or put a DNS name.

Though currently the whiptail prompt has options for the IPv4 address or a domain name. In case IPv6 is supported, there are some options:

Show both IPv4 and IPv6 address, and create two configurations if IP is selected instead of the DNS name.
Only provide the option for DNS same and say why it is the best option.

For the sake of dynamic IPv6 addresses (yes some providers even rotate IPv6 subnets) and that most pis have the 'privacy/private' IPv6 option enabled I skipped that.

Ok, speaking of this, I think the script should ask to set a static IPv6 address too.

There is also the issue with iptables being replace by nftables. We could also add support for that.

I'm planning to do this.

Would you mind including an option to not touch any firewall rules at all in the same PR?
Should at least be in the unattended file but maybe also in the whiptail commands.
Something like "it looks like you're using iptables/ufw/nftables, should pivpn configure your firewall for wireguard?"

Why? Most people don't fiddle with IPv6 on their system, hence link local addresses are enabled by default.

I mean actual IPv6 internet connectivity, a global unicast address and possibly a reachability test.

Maybe we can just do a ping6 google.com or something like this. If this works out IPv6 should be fine. What do you think?
Any suggestion how to implement that?
We should maybe also provide an option that disables any IP checks in case of problems there.

I think that it should be clear that any firewall in front of the server needs editing to make this work.

That comment was a suggestion for the user above, clearly you can't change the router firewall from the Pi, unless UPnP is enabled (I think automatic port forwarding was requested in the past).

UPnP is not reliable enough I think. But yes, it could also be an option to add support for that. Please don't make this the default.
I suggest you also include this feature in your firewall PR (nftables).

It is also not required if one wishes to connect to a DNS name which is the best option here. Those can contain dynamic IPs without any config change requirements. With dual stack a DNS name is anyways the best option, if one wishes to use that. So people either put their IP there or put a DNS name.

Though currently the whiptail prompt has options for the IPv4 address or a domain name. In case IPv6 is supported, there are some options:
1. Show both IPv4 and IPv6 address, and create two configurations if IP is selected instead of the DNS name.

2. Only provide the option for DNS same and say why it is the best option.

Why put a static IP into the config? All configs have to be updated if any IP was changed.
Are you suggesting to create two different config files per user if the user chooses to use both IP stacks instead of a DNS name?
I suggest to simply say that a user may use a single IP as Endpoint or a DNS based endpoint to support dual stack.
From some reddit post:

Wireguard for Android/iOS will prefer IPv4 if the endpoint hostname you specified successfully resolves AAAA and A, to help  with roaming between potentially IPv4-only networks. You can enforce IPv6 as transport by using a AAAA-only endpoint. wg-quick will prefer IPv6 if possible.

AFAIK pivpn doesn't yet support to properly update existing profiles. I am also missing an option to override the "Allowed IPs" var per user. Yes users may simply add the nets their routing through the VPN but we have some cases where changing the list on a per user basis is very useful and avoids editing the config afterwards.

For the sake of dynamic IPv6 addresses (yes some providers even rotate IPv6 subnets) and that most pis have the 'privacy/private' IPv6 option enabled I skipped that.

Ok, speaking of this, I think the script should ask to set a static IPv6 address too.

Are you referring to the part where the script will change the servers IP address? In general I think that this should be removed because the user should at least understand how to configure an IP address when going for a VPN.
As said; IPv6 on common internet providers are also dynamic for privacy reasons. Hence that should not be the default at least.

Would you mind including an option to not touch any firewall rules at all in the same PR?

Ok.

Maybe we can just do a ping6 google.com or something like this. If this works out IPv6 should be fine. What do you think?

Should be enough according to my limited knowledge of IPv6.

We should maybe also provide an option that disables any IP checks in case of problems there.

Which problems could arise by checking for connectivity?

UPnP is not reliable enough I think. But yes, it could also be an option to add support for that. Please don't make this the default.

Definitely not default. UPnP is often seen as a security risk so I'm not sure if we want to support such feature, although it would simplify the work for the user.

Are you suggesting to create two different config files per user if the user chooses to use both IP stacks instead of a DNS name?

Yes, but see below too.

AFAIK pivpn doesn't yet support to properly update existing profiles.

Yes, this is quite implicitly requested.

I suggest to simply say that a user may use a single IP as Endpoint or a DNS based endpoint to support dual stack.

Which one? IPv4, IPv6, or ask the user?

Are you referring to the part where the script will change the servers IP address? In general I think that this should be removed because the user should at least understand how to configure an IP address when going for a VPN.

I'm referring to the part of the script that sets the static LAN IP. In the same way, I think it should set a static IPv6 but warn that some ISPs change the prefix from time to time (even if is not recommended by RIPE: https://www.ripe.net/publications/docs/ripe-690). If we are removing tasks that the script has a chance to perform, then the script itself is less useful to the user.

External firewall rules, simply not required because the admin doesn't want any checks. DNS names being blocked. etc.

@4s3ti can you add an IPv6 address to install.pivpn.io?

Let the user choose, so ask. Suggest IPv4 if in doubt because of the mentioned mobile restrictions.

ACK

If you want to spend time for that okay fine with me. Just please don't let this PR stale forever because nobody finds time to do all of the suggested. We should sort the suggested and split parts into different PRs.

Separate pull requests. I will test this as soon as i have time IRL, and find out why the checks fail.

@4s3ti can you add an IPv6 address to install.pivpn.io?

I would advise against using such subdomain for that. Rather create a specific domain for that purpose.
I can provide simple PHP scripts to provide the public IP (v4 or v6) of the device connecting to the subdomain.
I suggest: ipv4.pivpn.io and ipv6.pivpn.io for that.
This way we'd also have the public IP and can asure that this will work for the installation script. Doesn't really require anything. Just PHP on the server.

If you want to spend time for that okay fine with me. Just please don't let this PR stale forever because nobody finds time to do all of the suggested. We should sort the suggested and split parts into different PRs.

Separate pull requests. I will test this as soon as i have time IRL, and find out why the checks fail.

I am happy to further help improve pivpn but I rather go for smaller packages adding improvements than one or two huge PRs.
@4s3ti Can we maybe discuss out of public Github regarding the dev env and testing? Maybe a discord group or something that also allows voice. Maybe you can create one and provide the info in the repo README ("for devs only, no support!" with manual server approval from your side).

@4s3ti can you add an IPv6 address to install.pivpn.io?

Not sure ... need to investigate that.

@4s3ti Can we maybe discuss out of public Github regarding the dev env and testing? Maybe a discord group or something that also allows voice. Maybe you can create one and provide the info in the repo README ("for devs only, no support!" with manual server approval from your side).

We have matrix where we can chat .. also can make a dev only room on matrix .. would that suffice?

We have matrix where we can chat .. also can make a dev only room on matrix .. would that suffice?

Where can I sign up?

@4s3ti can you add an IPv6 address to install.pivpn.io?

Not sure ... need to investigate that.

Hetzner cloud for a few bugs is available with both.

Hetzner cloud for a few bugs is available with both.

I know .. but I have everything running on Google Kubernetes service, so configuring IPv6 might require some tricks .. need to investigate that
As for matrix .. the easiest place is here: https://element.io/  the only reason why i am not opening up a discord is because it becomes way to much stuff to manage. the room is on the README ... and during the weekend can spin up a devs room or something. will think the best approach.

I would advise against using such subdomain for that. Rather create a specific domain for that purpose.

The rationale behind install.pivpn.io is that the user has likely visited the website so it is implicitly trusted and available.

lets discuss further in matrix. To much discussion here.

I would advise against using such subdomain for that. Rather create a specific domain for that purpose.

The rationale behind install.pivpn.io is that the user has likely visited the website so it is implicitly trusted and available when the script is being run.

We might switch to that URL in the future. For the time being I implemented just testing at google.com as this is most probably also allowed and google.com supports IPv6 of course.

After discussing with @4s3ti in matrix we decided to disable IPv6 tests in travis since travis doesn't really support IPv6 properly. It's only available in the LXC container which are only running on ARM which brings other problems to the table.
I implemented the following:

A switch to disable IPv6 config
IPv6 is enabled by default IF
--> IPv6 simple ping test, that will automatically disable ipv6 if not working (using google.com for now, may be switched to install.pivpn.io in future, no IPv6 address at the moment)
allow disabling IPv6 via unattended file switch (mainly for testing exclusion, since ipv6 should be used if available)

We need someone to test this now without and with unattended file. Disabling IPv6 via unattended file works already otherwise travis tests would fail.


carafity

mobile view

hi, it will take four days. it will be completed on 15-12-24.

https://vercel.com/ali-razas-projects-bdba9888/crafity
hi check it


[semantic-sil] Update 45 SILGen tests for ownership.

Very roughly this increases the total coverage of SILGen tests with ownership
enabled to ~20%. This handles most of the tests with characters in between a-d.
rdar://33358110

@swift-ci smoke test and merge

For those looking in, this means at a high level that the ownership verifier is now enabled while that test is being emitted and we print things slightly differently.


Fix exploding icon badge placement on mobile.

@keybase/react-hackers, this is ready for review.

Certainly not! I'll investigate.

@buoyad, what device were you seeing this on? I'm not seeing it in the Simulator or on a physical iPhone 6.

Weird. It does it in the simulator but not on device.

That took entirely too long to figure out.


Adds the ability to make custom timers repeat

I'd like the ability to have custom timers loop themselves over and over again automatically so this PR provides one way to accomplish that.
It checks for the repeat keyword when creating a custom timer.
i.e.
/tt custom [label] [duration] repeat [tooltip]

Alternatively, this could be added to a new command. e.g. /tt repeating?
This also adds the /tt stop [label] command to delete custom timers.

Apologies for the not squashing the commits, this was all just written up in the Github web editor

Squashing isn't an issue, but the fact you wrote it up in web editor means it wasn't tested.  The feature is fine to merge, but it needs a version number increase and should have been tested for at least a few days not written right on git and PRed.

It's been tested in game, I just don't have git set up on this laptop so it's copy pasted from local onto the editor.
Testing has been just setting up multiple repeating and custom timers, watching them loop and canceling them one by one.
Test run:
/tt custom test 5s repeat
/tt custom test2 5s repeat
/tt custom test3 15s repeat
/tt custom test4 1m
/tt custom test5 2m
-- watch timers loop
/tt stop test2
/tt stop test4
/tt stop test3
/tt stop test5
/tt stop test

Added a version bump.


[bug:shell] volume.balance -force failed and data seemed lost

I'm using systemd to run volume services.
After executed volume.balance -force, I found that some volumes services is down.
part of journalctl -xe logs:
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 file_util.go:23] Folder /mnt/oss-d10/v10 Permission: -rwxr-xr-x
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 volume_loading.go:89] readSuperBlock volume 135 version 3
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 volume_loading.go:126] volumeDataIntegrityChecking failed verifyIndexFileIntegrity /mn
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 volume_loading.go:131] loading sorted db /mnt/oss-d10/v10/135.sdx error: unexpected fi
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 volume_loading.go:89] readSuperBlock volume 143 version 3
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 volume_loading.go:89] readSuperBlock volume 190 version 3
Mar 26 04:29:33 ts97 weed[2055]: I0326 04:29:33  2055 volume_loading.go:141] loading leveldb /mnt/oss-d10/v10/143.ldb
Mar 26 04:29:33 ts97 weed[2055]: panic: runtime error: invalid memory address or nil pointer dereference
Mar 26 04:29:33 ts97 weed[2055]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xe0db94]
Mar 26 04:29:33 ts97 weed[2055]: goroutine 108 [running]:
Mar 26 04:29:33 ts97 weed[2055]: github.com/chrislusf/seaweedfs/weed/storage.(*SortedFileNeedleMap).Close(0xc000edf440)
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/needle_map_sorted_file.go:97 +0x14
Mar 26 04:29:33 ts97 weed[2055]: github.com/chrislusf/seaweedfs/weed/storage.(*Volume).load.func1()
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/volume_loading.go:32 +0x42
Mar 26 04:29:33 ts97 weed[2055]: github.com/chrislusf/seaweedfs/weed/storage.(*Volume).load(0xc0000e5b00, 0x1, 0x1, 0x1, 0xa0501d)
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/volume_loading.go:185 +0x1699
Mar 26 04:29:33 ts97 weed[2055]: github.com/chrislusf/seaweedfs/weed/storage.NewVolume({0x7ffc904e0f6b, 0x10}, {0x7ffc904e0f6b, 0x10}, {0x0,
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/volume.go:59 +0x1d8
Mar 26 04:29:33 ts97 weed[2055]: github.com/chrislusf/seaweedfs/weed/storage.(*DiskLocation).loadExistingVolume(0xc000d1f760, {0x28e83f0, 0x
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/disk_location.go:131 +0x369
Mar 26 04:29:33 ts97 weed[2055]: github.com/chrislusf/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes.func2()
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/disk_location.go:171 +0x9a
Mar 26 04:29:33 ts97 weed[2055]: created by github.com/chrislusf/seaweedfs/weed/storage.(*DiskLocation).concurrentLoadingVolumes
Mar 26 04:29:33 ts97 weed[2055]: /github/workspace/weed/storage/disk_location.go:168 +0xc5

Now only 2 of 13 volume services are able to start up, is there any way to fix this, or recover the data?

@rykinia  version ?

@rykinia rykinia

Now only 2 of 13 volume services are able to start up, is there any way to fix this, or recover the data?

It seems that you can delete or better rename the *.sdx files to another extension

@rykinia rykinia

Now only 2 of 13 volume services are able to start up, is there any way to fix this, or recover the data?

It seems that you can delete or better rename the *.sdx files to another extension

The version is 2.9.0
I delete the .sdx files and the volume servers can be started, thanks a lot @kmlebedev
I'm using volume.move this time, and this command works fine.

Before I executing volume.balance --force, the disks of volume services 1~8 was full, and I saw there was a grpc exception at the end of the command progress. Maybe the issue is related to that.

not clear what to do for this particular issue. Closing for now.


[FR]: Clean up kurtosis files download helptext

Background & current behavior
In a user feedback session today we saw a user struggle mightily getting kurtosis files download to work. He kept entering:
kurtosis files download enclave ....bunch of different attempts here...

and was very confused what a resource identifier is. We eventually determined:

Having the way to enter enclave be --enclave in kurtosis run but be a positional arg here was confusing to him and
The enclave string in the helptext was confusing to him
He wasn't sure what an identifier was (though it is in the helptext, to be fair - think the pressure of a quickstart experience session might have contributed)

Desired behavior
Long:
Download the given files artifact from the given enclave to your machine. The files artifact and enclave are specified by identifier (name, UUID, or shortened UUID). Read more about identifiers here: https://docs.kurtosis.com/reference/resource-identifier

also the enclave argkey is replaced with enclave-identifier
(addressing the consistency of enclave arguments is something else percolating in my brain, for later)
How important is this feature or improvement to you?
Painful, the lack of this feature makes using Kurtosis frictionful.

I'm not on this ticket but continuing our discussion from yesterday; is this a "breaking change" for you?

@h4ck3rk3y nope - don't think the user has to take any action to fix, so not a breaking change


Allow @embroider/addon-shim: ^1.0.0 as dependency

fix https://github.com/adopted-ember-addons/ember-moment/issues/377

I sent this via DM, but for others, here is the plan:
I would de this series of PRs:

drop support for node < 18 (This is a breaking change, we won't release right away)
add @ember/string to the test app
then we land your embroider update PR
then we do a v2 addon

Why use old range || new range?

not sure, if there is something braking between v0.50 and 1.x... but we can also remove v0.50 if you want

I don't think anything would break, unless folks are pinning the older versions, which it's their own fault at that point


The change in CMakeLists.txt need to be ported to spdm-emu.

The change in CMakeLists.txt need to be ported to spdm-emu.
Originally posted by @qizhangz in https://github.com/DMTF/libspdm/issues/617#issuecomment-1034650018

Ideally the CMakeLists.txt from libspdm should just be used by spdm-emu instead of having a separate copy.

@ekohandel Are you able to take this issue?

Sure @steven-bellock, I am doing some prototyping with the spdm-emu and had to make these changes to make it work. I will just clean those up and submit a PR to spdm-emu. You can assign it to me.
I believe we are blocked on updating the spdm-emu -> libspdm submodule hash until this is done so I will make it a higher priority.

https://github.com/DMTF/spdm-emu/pull/54 created to address this.

Merged. Close it.

For further spdm-emu issue, please report to spdm-emu repo.


fix 97484

This PR fixes #97484

Check remotes before adding

Thanks for approving.
To be clear, actually !was used expecting that getDomainsOfRemotes returns only those domains which are asked for. But it is not and returning domains which are not asked for in weird format. It was due to the bad contract of getDomainsOfRemotes


Server returns html document with [object Promise]

So i was really curious with this go ssr approach and went to try it but there was several issues.

I had problems running docker container, running on M1 MBP and this dockerfile worked for me

FROM node:18-alpine as frontend

RUN npm install -g<EMAIL_ADDRESS>
WORKDIR /app
COPY client ./client/

WORKDIR /app/client
RUN pnpm install --frozen-lockfile && pnpm build

FROM --platform=linux/amd64 ubuntu:latest as backend

# RUN apt update

ARG GO_VERSION
ENV GO_VERSION=${GO_VERSION}

RUN dpkg --add-architecture amd64
RUN apt-get update && apt-get upgrade
RUN apt-get install -y wget git gcc libc-dev make build-essential g++-x86-64-linux-gnu libc6-dev-amd64-cross gcc-aarch64-linux-gnu binutils:amd64 libc6-amd64-cross libstdc++6-amd64-cross

RUN ln -s /usr/x86_64-linux-gnu/lib64/ /lib64

ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/lib64:/usr/x86_64-linux-gnu/lib"
# RUN apt install gcc libc-dev make build-base

RUN wget -P /tmp "https://dl.google.com/go/go1.22.2.linux-amd64.tar.gz"

RUN tar -C /usr/local -xzf "/tmp/go1.22.2.linux-amd64.tar.gz"
RUN rm "/tmp/go1.22.2.linux-amd64.tar.gz"

ENV GOPATH /go
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH
RUN mkdir -p "$GOPATH/src" "$GOPATH/bin" && chmod -R 777 "$GOPATH"

WORKDIR /app

COPY --from=frontend /app/dist ./dist

COPY pkg/ ./pkg/
COPY go.mod go.sum Makefile main.go ./
RUN go mod download

RUN make build

FROM --platform=linux/amd64 ubuntu:latest as final

WORKDIR /app

COPY --from=backend /app/build/ /app/

EXPOSE 8080

CMD ["/app/server"]

And Makefile, not sure if it did anything but it is what it is
build:
	# build the backend
	env GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o $(BUILD_DIR)/$(APP_NAME) main.go

As title says server returns a Promise in a string so for that i had to find a way to wait for it, on v8 repo in issues i found this posted

func resolvePromise(ctx *v8go.Context, val *v8go.Value, err error) (*v8go.Value, error) {
	if err != nil || !val.IsPromise() {
		return val, err
	}
	for {
		switch p, _ := val.AsPromise(); p.State() {
		case v8go.Fulfilled:
			return p.Result(), nil
		case v8go.Rejected:
			return nil, errors.New(p.Result().DetailString())
		case v8go.Pending:
			ctx.PerformMicrotaskCheckpoint() // run VM to make progress on the promise
			// go round the loop again...
		default:
			return nil, fmt.Errorf("illegal v8go.Promise state %d", p) // unreachable
		}
	}
}

And in a nutshell what we need to do is this
func (r *Renderer) Render(urlPath string) (string, error) {
	iso := r.pool.Get()
	defer r.pool.Put(iso)

	ctx := v8go.NewContext(iso.Isolate)
	defer ctx.Close()

	iso.RenderScript.Run(ctx)

	renderCmd := fmt.Sprintf(`ssrRender("%s")`, urlPath)
	val, err := ctx.RunScript(renderCmd, r.ssrScriptName)
	result, _ := resolvePromise(ctx, val, err)

	if err != nil {
		if jsErr, ok := err.(*v8go.JSError); ok {
			err = fmt.Errorf("%v", jsErr.StackTrace)
		}
		return "", nil
	}

	return result.String(), nil
}

Maybe for the future there should be a check if a value of a script is a Promise then we wait for it otherwise just return a string.
I'm not sure if you want to add this edits but for people who will come upon this might help

Ty for posting this. :)

Thanks! I updated packages and added check for promise


renaming zarr_metadata to v04.zarr_metadata breaks downstream libraries

See bioio Issue#14 for an example on how this breaks downstream libraries.
Is the plan to have the two (v0.5 and 0.4) zarr_metadata modules going forward? Should developers adjust to the new module name? Could an alias be used?

Hi @TimMonko , thanks for the report. 🙏
Yes, we will have more, e.g. v0.6, zarr_metadata modules in the future.
We could have an alias module, ngff_zarr.zarr_metadata, that imports everything from the module that is the default in to_ngff_zarr (currently v0.4). Would that address the bioio use case?

To me, that sounds reasonable. Hopefully @toloudis can give feedback, since they are the developer of the writer that uses ngff-zarr. I'm just a user of bioio myself lol

@TimMonko @toloudis thanks -- implemented in #123 and released in ngff-zarr 0.10.1.

Thank you for the speedy update!

Thanks for this!  We are a little behind in working with ome-zarr 0.5 (ramping up some work on supporting that now-ish, starting with reading/viewing) but at the same time I'd like to acknowledge that ngff-zarr is a really nice library.  Ultimately what bioio needs is a nice metadata DataClass-like model, or as close to that as we can get. (Not to discount the tricky work of converting huge dask delayed arrays to zarr)  ome-types is another library that does something similar with metadata model.


[FLOC-2285] Postgres tests improvements

PR into master will be merged as soon as tests are slightly happier. Actual passing tests will be in a stacked PR.

Oops wrong branch.


UnicodeDecodeError in Masonry with MSYS2

[X] I am on the latest Poetry version.

[X] I have searched the issues of this repo and believe that this is not a duplicate.

[X] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: Windows 10 with MSYS2 20180531

Poetry version: 0.12.11

Link of a Gist with the contents of your pyproject.toml file: https://github.com/gaphor/gaphor/blob/master/pyproject.toml

Issue
When trying to pip install my app I am getting a UnicodeDecodeError.
$ pip3 install --user .
Processing c:/tools/msys64/home/dyeaw/gaphor
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... error
    Complete output from command C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp237x9yfr:
    Traceback (most recent call last):
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 64, in prepare_metadata_for_build_wheel
        hook = backend.prepare_metadata_for_build_wheel
    AttributeError: module 'poetry.masonry.api' has no attribute 'prepare_metadata_for_build_wheel'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 207, in <module>
        main()
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 197, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 67, in prepare_metadata_for_build_wheel
        config_settings)
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 95, in _get_wheel_metadata_from_wheel
        whl_basename = backend.build_wheel(metadata_directory, config_settings)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\api.py", line 40, in build_wheel
        poetry, SystemEnv(Path(sys.prefix)), NullIO(), Path(wheel_directory)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\wheel.py", line 48, in make_in
        wb = WheelBuilder(poetry, env, io, target_dir=directory, original=original)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\wheel.py", line 38, in __init__
        super(WheelBuilder, self).__init__(poetry, env, io)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\builder.py", line 42, in __init__
        self._meta = Metadata.from_package(self._package)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\metadata.py", line 50, in from_package
        meta.description = f.read()
      File "C:/tools/msys64/mingw64/lib/python3.7\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 7221: character maps to <undefined>

    ----------------------------------------
Command "C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp237x9yfr" failed with error code 1 in C:/Users/dyeaw/AppData/Local/Temp/pip-req-build-clnw4_9m

It looks like Poetry is not using an encoding when it calls open() to load the description.
I am opening the issue here after exploring the cause in MSYS2 first: https://github.com/msys2/MINGW-packages/issues/5155

Save your pyproject.toml in utf-8 encoding again should be alright?
To handle different encodings, it needs to depend on chardet without a guarentee resolving this problem.
Reference: https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text

@drunkwcodes My pyproject.toml, and the rest of my project is in UTF-8. The issue is that Poetry is calling open() with no encoding specified, and the default encoding for Windows is cp1252.

Sorry. Here it is. Please try to change msys2's encoding setting to utf-8.
Or use chcp 65001 command.
And I can say it will not change as long as the git windows defaults remains.
The change will lead to raising encoding error to the majority of git-windows users.
https://stackoverflow.com/questions/28481410/how-to-set-console-encoding-in-msys

@drunkwcodes Thanks for trying to research possible solutions. The correct default/preferred encoding of the contents of files for Windows users is cp1252. This is what is returned when I call locale.getpreferredencoding(). It is not a solution for me to change my default encoding. This issue here is that Poetry is opening pyproject.toml, which should be UTF-8 for all users, without specifying the encoding to use.

@danyeaw Sorry for not having a solution for you to change your default encoding, sir. What a lovely night.

I believe https://github.com/sdispater/poetry/issues/1030 is related to this issue. I don't think it's safe to assume that the locale will always be set to UTF-8 on systems where poetry (or packages using poetry, such as pendulum) might be installed.

We  (me and @rhorenov) had problem to install pendulum==2.0.4 on MS Windows with encoding windows-1250.
If the package was to be built (e.g. pendulum), it failed on special characters in metadata generated by  poetry as it was opening files for build using default console encoding.
Installing packages from wheels worked well.
I am going to create PR for master to fix this issue (adding explicit encoding where files are open for writing in text mode).

Can @danyeaw or anyone else check, that the latest poetry version 0.12.17 is not not suffering this problem?

Closed by commit e9a13f8 👍

Greate @danyeaw
Can you close the issue? I do not seem to have such option.


Shell Sort

Submission Title
Shell Sort
Submission Type
Algorithm
Submission Description
The method starts by sorting pairs of elements far apart from each other, then progressively reducing the gap between elements to be compared.
Folder Related To
Algorith => Sorting_Algorithm => Shell_Sort =>Shell_Sort.extension
Contributors

[ ]  C++ -
[ ]  C -
[x]  Python -
[ ] Java -
[ ] JavaScript -
[ ] TypeScript -
[ ] Golang -
[ ] Ruby -
[ ] php -
[ ] Swift -
[ ] C# -

I want to work on this

Please assign this to me for python

I would like to work on this in Javascript

I would very much like to work on this algorithm in C++ !

Hey! can I work on this in C.

Hi, I'd like to work on it using Java. Could you please assign me to this?

I would like to work on it in swift.

I would like to work on this in C++. Can you please assign it to me?

Can you please assign this to me in C?

thanks everyone :v:


7.2.Set Matrix zero approach-2

My code is clear and well understanding
Please assign it to me

Need to add a description and your approach. Thanks, Closing as of now.


Danbooru: when loged in, tags with many pages will return only 1 page of results.

What steps will reproduce the problem?
Select Danbooru as a source, login in account with hashed pass (i'm using gold account), search tag that should return many pages (i used 1girl, solo and other general tags)
What is the expected behavior? What do you get instead?
Expected behavior - multipaged search resault. Instead it will only find 1 page and 20 pics
How often does this problem occur? On which sources?
every time on Danbooru
What version of the program are you using? On what operating system?
nightly 2928cf38 windows 10x64
Please provide any additional information below
Can be due to a gold acc status. tags with a few pages will work as intended (testet on shiika_sadamasa tag, returns 6 pages ~100 images)

what i ment originally was that:

And when i press "get all" it will get 20 pics.

While on the danbooru

This count will be used as long as you enable the "Get detailed tags" in the settings, which will load the wiki info, tag info, etc. You can also give the Regex API top priority but I don't really recommend this.

But note that even with this setting you'll get this:

For which I'm working on a fix.

In that case that's very strange.
Do you properly see two "loading" lines in your log when you load results?
[19:42:10.801][Info] [danbooru.donmai.us][Html] Loading page https://danbooru.donmai.us/posts?login=Bionus&password_hash=*****&limit=20&page=1&tags=solo
[19:42:10.808][Info] [danbooru.donmai.us][Xml] Loading page https://danbooru.donmai.us/posts.xml?login=Bionus&password_hash=*****&limit=20&page=1&tags=solo
[19:42:11.864][Info] [danbooru.donmai.us][Html] Receiving page https://danbooru.donmai.us/posts?login=Bionus&password_hash=*****&limit=20&page=1&tags=solo
[19:42:15.272][Info] [danbooru.donmai.us][Html] Parsed page https://danbooru.donmai.us/posts?login=Bionus&password_hash=*****&limit=20&page=1&tags=solo: 20 images, 25 tags (25), -1 total (1823000), -1 pages (91150)
[19:42:15.293][Info] [danbooru.donmai.us][Xml] Receiving page https://danbooru.donmai.us/posts.xml?login=Bionus&password_hash=*****&limit=20&page=1&tags=solo
[19:42:15.553][Info] [danbooru.donmai.us][Xml] Parsed page https://danbooru.donmai.us/posts.xml?login=Bionus&password_hash=*****&limit=20&page=1&tags=solo: 20 images, 0 tags (371), -1 total (-1), -1 pages (-1)

Could it be that the parsing of tags failed?
In my case for example I properly get [Html] Parsed page [...] 25 tags (25).
Could you share your log so I can check that? (make sure to remove your password hash before posting it here)

[22:47:34.103][Info] [danbooru.donmai.us][Html] Loading page https://danbooru.donmai.us/posts?login=darknestone&password_hash=******&limit=20&page=1&tags=solo [22:47:34.104][Info] [danbooru.donmai.us][Xml] Loading page https://danbooru.donmai.us/posts.xml?login=darknestone&password_hash=******&limit=20&page=1&tags=solo [22:47:47.181][Info] [danbooru.donmai.us][Xml] Receiving page https://danbooru.donmai.us/posts.xml?login=darknestone&password_hash=******&limit=20&page=1&tags=solo [22:47:47.196][Info] [danbooru.donmai.us][Xml] Parsed page https://danbooru.donmai.us/posts.xml?login=darknestone&password_hash=******&limit=20&page=1&tags=solo: 20 images, 0 tags (391), -1 total (19), -1 pages (1) [22:47:47.240][Info] [danbooru.donmai.us][Html] Receiving page https://danbooru.donmai.us/posts?login=darknestone&password_hash=******&limit=20&page=1&tags=solo [22:47:47.251][Info] [danbooru.donmai.us][Html] Parsed page https://danbooru.donmai.us/posts?login=darknestone&password_hash=******&limit=20&page=1&tags=solo: 20 images, 0 tags (389), -1 total (19), -1 pages (1)

can't understand how to post code properly, sorry

Indeed, it would seem that when you are logged in with a premium account, Grabber can't parse tags properly anymore.
Not having a gold account it's going to be pretty hard to fix by myself, but if you can give a hand I'd be able to fix the regex and tag parsing for gold users. :+1:
Could you:

Open "https://danbooru.donmai.us/posts?login=darknestone&password_hash=******&limit=20&page=1&tags=solo" (make sure to re-put your password hash in the url)
Press Ctrl+U to show the source code of the page
Copy and paste everything here (or save it as a file and join it to a comment)

Then with said source code I'll see how I can fix the parsing regex.

can't understand how to post code properly, sorry

For multi-line code, you can wrap it with ``` at the beginning and end to make a code block.

source code.txt

was this issue fixed in the nightly Bionus? I'm only getting this issue now in the last couple of days. Looks like Danbooru has changed something on their site with the way searching for tag is done.

@rushuna86 that's not linked to this issue, it's because they changed the text for the "next page" button from > to a font-awesome icon.


Fix pgsql migration test

This PR try to fix missed database close.

Codecov Report

Merging #12844 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #12844      +/-   ##
==========================================
+ Coverage   43.08%   43.09%   +0.01%
==========================================
  Files         655      655
  Lines       72329    72329
==========================================
+ Hits        31160    31173      +13
+ Misses      36114    36107       -7
+ Partials     5055     5049       -6

Impacted Files
Coverage Δ

modules/log/event.go
57.54% <0.00%> (-1.89%)
:arrow_down:

modules/git/repo.go
46.70% <0.00%> (+0.50%)
:arrow_up:

services/pull/check.go
48.46% <0.00%> (+0.76%)
:arrow_up:

modules/git/utils.go
77.04% <0.00%> (+3.27%)
:arrow_up:

modules/charset/charset.go
73.03% <0.00%> (+4.49%)
:arrow_up:

modules/indexer/stats/queue.go
76.47% <0.00%> (+11.76%)
:arrow_up:

modules/indexer/stats/db.go
60.86% <0.00%> (+17.39%)
:arrow_up:

modules/util/timer.go
85.71% <0.00%> (+42.85%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ace8a28...2604b46. Read the comment docs.


[WIP][SPARK-28666] Support saveAsTable for V2 tables through Session Catalog

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review https://spark.apache.org/contributing.html before opening a pull request.

Test build #108893 has started for PR 25402 at commit 9781ae8.

Test build #108893 has finished for PR 25402 at commit 9781ae8.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108893/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13967/
Test PASSed.

Test build #108897 has started for PR 25402 at commit 99ba64d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13971/
Test PASSed.

Test build #108897 has finished for PR 25402 at commit 99ba64d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108897/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14053/
Test PASSed.

Test build #108985 has started for PR 25402 at commit f73feb8.

Test build #108985 has finished for PR 25402 at commit f73feb8.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class UnresolvedDataSourceV2Relation(table: Table) extends LeafNode with NamedRelation

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108985/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14060/
Test PASSed.

Test build #108992 has started for PR 25402 at commit aac9503.

Test build #108996 has started for PR 25402 at commit 428e82a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14063/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14064/
Test PASSed.

Test build #108997 has started for PR 25402 at commit d9f478d.

cc @cloud-fan @rdblue @jzhuge

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14065/
Test PASSed.

Test build #108998 has started for PR 25402 at commit e489a16.

Test build #108992 has finished for PR 25402 at commit aac9503.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class CatalogTableAsV2(v1Table: CatalogTable) extends UnresolvedTable

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108992/
Test FAILed.

Test build #108996 has finished for PR 25402 at commit 428e82a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108996/
Test FAILed.

Test build #108997 has finished for PR 25402 at commit d9f478d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108997/
Test FAILed.

Test build #108998 has finished for PR 25402 at commit e489a16.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108998/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14119/
Test PASSed.

Test build #109056 has started for PR 25402 at commit 762f873.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14120/
Test PASSed.

Test build #109056 has finished for PR 25402 at commit 762f873.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109056/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14129/
Test PASSed.

Test build #109065 has started for PR 25402 at commit 06cf1f9.

Test build #109065 has finished for PR 25402 at commit 06cf1f9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109065/
Test FAILed.

Test build #109080 has started for PR 25402 at commit 0bd93ae.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14144/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14146/
Test PASSed.

Test build #109081 has started for PR 25402 at commit 99ae156.

Test build #109081 has finished for PR 25402 at commit 99ae156.

This patch fails to generate documentation.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109081/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14148/
Test PASSed.

Test build #109084 has started for PR 25402 at commit 673d95a.

Test build #109080 has finished for PR 25402 at commit 0bd93ae.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #109084 has finished for PR 25402 at commit 673d95a.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109084/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109080/
Test FAILed.

Test build #109102 has started for PR 25402 at commit 673d95a.

Test build #109102 has finished for PR 25402 at commit 673d95a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109102/
Test PASSed.

Test build #109114 has started for PR 25402 at commit a70e726.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14180/
Test PASSed.

Test build #109114 has finished for PR 25402 at commit a70e726.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class CatalystDataToAvro(
case class Milliseconds(child: Expression, timeZoneId: Option[String] = None)
case class Microseconds(child: Expression, timeZoneId: Option[String] = None)
case class IsoYear(child: Expression) extends UnaryExpression with ImplicitCastInputTypes
case class Epoch(child: Expression, timeZoneId: Option[String] = None)
case class DeleteFromTable(
case class DeleteFromStatement(
case class DeleteFromTableExec(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109114/
Test PASSed.

thanks, merging to master! Please address https://github.com/apache/spark/pull/25402#discussion_r313962683 in a followup.

Thanks @cloud-fan!

@brkyvz sql/catalyst/src/main/java/org/apache/spark/sql/sources/v2/internal/UnresolvedTable.scala is in src/main/java directory. Is this intended?


[SPARK-48124][CORE] Disable structured logging for Connect-Repl by default

What changes were proposed in this pull request?
The pr is followup https://github.com/apache/spark/pull/46383, to disable structured logging for Connect-Repl by default.
Why are the changes needed?
Before:

After:

Does this PR introduce any user-facing change?
No.
How was this patch tested?
Manually test.
Was this patch authored or co-authored using generative AI tooling?
No.

cc @gengliangwang

@panbingkun could you double check if the test failure is related?

@panbingkun the test failure is probably not related. Could you verify it on your local env?

@gengliangwang
Sure, it is successful to execute it on local env, as following:

Thanks, merging to master


nixos/nixos-installer: use temporary directory on target filesystem

Motivation for this change
This should fix 'No space left on device' on systems with low amount of RAM when there is a need to build something like Linux kernel.
Also fix https://github.com/NixOS/nixpkgs/issues/77481
How to test this:

boot installer on machine with <=2G of RAM
mount filesystems, generate config
add this to configuration:

{
  boot.kernelPatches = [ {
    name = "crashdump-config";
    patch = null;
    extraConfig = ''
      CRASH_DUMP y
      DEBUG_INFO y
      PROC_VMCORE y
      LOCKUP_DETECTOR y
      HARDLOCKUP_DETECTOR y
    '';
  } ];
}

start nixos-install

Without this patch nix build will fail to unpack kernel sources into /tmp (which is on tmpfs on installation).
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[X] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

cc @Mic92

Maybe this should be behind a flag or something?

Maybe this should be behind a flag or something?

This should be default behavior. Installer should work under any conditions without obscure flags or environment variables.
Temporary directory can still be changed using TMPDIR environment variable by expert users but i don't see much reason to do so.

@GrahamcOfBorg test installer


Add backwards compatibility tests for automation

Add backwards compatibility tests for the plugin to automate end-to-end upgrade paths for faster releases. Developer documentation for implementing bwc tests and hooking them to CI: https://github.com/opensearch-project/opensearch-plugins/blob/main/TESTING.md#backwards-compatibility-testing

Backward compatibility tests are added on the performance-analyzer plugin side.
https://github.com/opensearch-project/performance-analyzer/pull/69
PA Reader process or performance-analyzer-rca  runs as a separate process independent of opensearch. Hence BWC tests cannot be added for them.


HADOOP-18128. Fix typo issues of outputstream.md

Description of PR
How was this patch tested?
For code changes:

[ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

There are some other typos(such as implmentation, satisifed), please modify them together. Thank you.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 47s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 33s

trunk passed

+1 :green_heart:
mvnsite
1m 28s

trunk passed

+1 :green_heart:
shadedclient
55m 18s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  1s

the patch passed

+1 :green_heart:
blanks
0m  1s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
1m 21s

the patch passed

+1 :green_heart:
shadedclient
20m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

79m 56s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4025/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4025

Optional Tests
dupname asflicense mvnsite codespell markdownlint

uname
Linux f822bc0bb4fb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / eb42b3f5370fe597633dd3b5ce06d5169f212325

Max. process+thread count
544 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4025/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

There are some other typos(such as implmentation, satisifed), please modify them together. Thank you.

ok, got it.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 56s

trunk passed

+1 :green_heart:
mvnsite
1m 25s

trunk passed

+1 :green_heart:
shadedclient
53m 49s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m  1s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
1m 15s

the patch passed

+1 :green_heart:
shadedclient
20m 12s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

77m 59s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4025/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/4025

Optional Tests
dupname asflicense mvnsite codespell markdownlint

uname
Linux fcadc50b1bc2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 8647fd3a621007c84b93931fe6a5293c6ad08474

Max. process+thread count
550 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4025/2/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@ted12138 Thanks for your contribution. @tomscut Thanks for your review! Merged.


Issue-Allow numbers.

Automated Review Comments
Issues in README.md:

(1, 'Line exceeds 80 characters')
(5, 'Line exceeds 80 characters')
(6, 'Line exceeds 80 characters')
(13, 'Line exceeds 80 characters')
(27, 'Line exceeds 80 characters')
(30, 'Line exceeds 80 characters')
(33, 'Hardcoded string found')
(36, 'Hardcoded string found')
(38, 'Hardcoded string found')
(40, 'Hardcoded string found')
(41, 'Hardcoded string found')
Issues in index.js:
(1, 'Line exceeds 80 characters')
(1, 'Hardcoded string found')
(6, 'Hardcoded string found')
(7, 'Hardcoded string found')
(11, 'Hardcoded string found')
(15, 'Hardcoded string found')
(18, 'Hardcoded string found')
(21, 'Hardcoded string found')
(24, 'Hardcoded string found')
(25, 'Hardcoded string found')
(34, 'Hardcoded string found')
Issues in index.ts:
(5, 'Line exceeds 80 characters')
(6, 'Hardcoded string found')
(7, 'Hardcoded string found')
(9, 'Hardcoded string found')
(12, 'Hardcoded string found')
(15, 'Hardcoded string found')
(18, 'Hardcoded string found')
(21, 'Hardcoded string found')
(30, 'Hardcoded string found')
(32, 'Hardcoded string found')
Issues in test/example.js:
(13, 'Line exceeds 80 characters')
(18, 'Hardcoded string found')
(20, 'Hardcoded string found')
(21, 'Hardcoded string found')

I got it.


Fix docs refering to old installer

For example here: https://docs.epinio.io/installation/install_epinio_on_k3d.html#install-epinio-on-the-cluster
It's in many places

other: https://docs.epinio.io/explanations/detailed-push-process.html#additional-things (git grep -i "system domain")

@jimmykarily

For example here: https://docs.epinio.io/installation/install_epinio_on_k3d.html#install-epinio-on-the-cluster
It's in many places

I do not see a reference to the old installer here nor in the pages reachable from that section.
I suspect that one of the rewrites in the last month mooted this.


refacto: [NPM] Handle <IP_ADDRESS>/0 ipblock and de-duplicated elements in except field

Reason for Change:

To handle special IPBlock (<IP_ADDRESS>/0) and de-duplicated elements in except in translation logic.
Issue Fixed:

Requirements:

[X] uses conventional commit messages

[ ] includes documentation
[X] adds unit tests

Notes:

/azp run


fix(apple): Use keychain from the tunnel process only

This fixes another long-standing bug with the Apple client: Keychain groups.
Apple's Keychain docs are woefully unclear and lacking on the Keychain.
These are the main takeaways:

Apple wants you to use the "Data protection keychain" on macOS which allows it to behave like an iOS keychain. That opens up the door for possible to sync to iCloud (which we don't use).
Data protection keychain items, it appears, cannot be created by Network Extensions.
However, we can save to the regular keychain (by default the system keychain for root procs like us), which is file-based.
Keychain items can be shared (both read/write) between apps, but not between users. The tunnel process and gui process run as different users. The only way for this to happen is to use the old file-based Keychain and use very deprecated APIs to allow both "users" access, which is what we were doing before.
To fix this, we limit all keychain operations to the tunnel proc only. The GUI passes the auth token in during the startTunnel call, which the system then passes to our PacketTunnelProvider class.

This uses the file-based Keychain, but since we need to use that keychain as the root tunnel proc, we don't have much choice. The "Allow access" dialog bug on macOS 12 is fixed by the fact that we are only accessing it from the same user (tunnel proc) that created it now.

@AndrewDryga This was tested on macOS 12.7.4 and the keychain issue is fixed.

Fixes #2324 (finally!)

On Windows and Linux I'm planning to go the other way and have the GUI process store the token in the keyring

Yeah, I considered that as well, but we need headless on the Apple clients because on iOS our GUI process can be reaped by the system at any time.

So if the GUI and tunnel process are both reaped, the tunnel must be able to bring itself back without the GUI helping? Makes sense


Tool extraction failure

This issue is reporting a bug in the code of Perseus. Details of the scope will be available in issue labels.
The author described their issue as follows:

error with perseus serve immediately after running perseus new

The steps to reproduce this issue are as follows:

$ perseus new appname
Your new app has been created! Run perseus serve -w to get to work! You can find more details, including about improving compilation speeds in the Perseus docs (https://arctic-hen7.github.io/perseus/en-US/docs/).
$ cd appname/src
$ perseus serve -w
🔏 Generating Cargo lockfile...✅
📥 Installing external tools...❌
Error: couldn't extract 'wasm-opt' (do you have the necessary permissions?)

Caused by:
failed to iterate over archive
Caused by:
failed to fill whole buffer
A minimum reproducible example is available at <>.

Hydration-related: false
The author is willing to attempt a fix: true

Tribble internal data
dHJpYmJsZS1yZXBvcnRlZCxDLWJ1ZyxBLWNsaSxhdXRob3Itd2lsbGluZy10by1pbXBs

Hmm, that's extremely weird. What's your OS?

@arctic-hen7 Ubuntu 22.04, also worth mentioning (thought I posted with issue) i'm on perseus 0.4 beta
This wasn't happening before, just started happening, which is what led me to try generating a new app
It sounds like a file perm error, where is wasm-opt being axtracted to?

Hmm, same. It's extracted to your system cache directory, which should be something like ~/.cache/perseus-cli/tools for you. Could you try clearing that directory? Perseus should use the local dist/ directory if it finds the systemwide cache unusable, which implies that this might have been caused by a corruption of some sort. I've experienced some weird Cargo failures causing tool reinstallation in the past, so it could be something like that.

UPDATE: This is now occurring on CI, so I think this might be a critical bug that's worked its way into the installation process somehow.

Okay, this has been caused by the wasm-opt team releasing a new version without including any release artifacts. The CLI defaults to the latest version, and it can't find the artifacts, hence the problem! Add --wasm-opt-version version_110 to the first Perseus command you run, and everything will work perfectly. (You won't need to add this flag again.)

See WebAssembly/binaryen#5282.

thank you! that fixed it. Fixed by running:
perseus serve -w --wasm-opt-version version_110

subsequent runs work without the flag:
perseus serve -w

The artifacts for wasm-opt's parent package have now been released for v111, so this problem should no longer affect anyone! I'll leave this issue open until the upstream one is definitely closed, and then I'll remove the warning in the README.


scan differences in phase-1 track ntuples: scanTC checks track candidates

this is not yet meant for merging to cmssw
In the context of comparing to tracks found in CKF but not in mkFit I was using this script to select reasonably good cases to follow.
The default formatting includes details of the seed and the track candidate with reco and simhit info.
scanTC(ta, tb, minSimPt=0., nEv=10, iteration=9)
would printout differences for the first 10 events looking at tracks in iter 9 (pixelLess).
The hardcoded logic for the detailed printouts requires that the CKF track passed the high purity selections and has all hits matched to sim
The column headings are missing.
A somewhat random example with my comments added after #
# event 5 tc idx 3324 from sim track idx 20548
5 tc  3324  sim  20548  only in a
# seed idx 16620 label 7931 pt eta phi 0.6477  2.0136  0.8917  nhits 2 last hit xyz 12.7604  37.0352  151.1987 px py pz ...
see 16620 7931  0.6477  2.0136  0.8917  2  12.7604  37.0352  151.1987 -0.0112  0.6476  2.3825
# if present, other seeds matching to the same sim will be printed as well
# glued hit from the seed (P and S would be for single pixel and strip hits) at idx 57475 subdet layer 6 1  (x y z) cm err xx yy zz "m"ono idx 71533 (x y z) "s"tereo idx 71527 (x y z)
g 57475  6   1 ( 12.62  32.98  137.20) cm  2.48e-04  7.43e-04  2.30e-03 m 71533 ( 13.12  34.30  137.04) s 71527 ( 12.96  34.24  137.36)
g 60043  6   2 ( 12.67  36.72  151.20) cm  2.52e-04  8.36e-04  2.86e-03 m 74812 ( 11.99  34.76  151.03) s 74802 ( 12.13  34.49  151.36)
# track candidate idx 3324 seed idx 16620 x y 10.3173  21.1181 pca Pt Eta Phi 0.4922  2.0317  0.7104 nhits 15 npixel 0 sim Idx 20548 pdgId 11 vtxId 7823 match fraction 1.000 simpt simeta 0.4781  2.0458
tcand 3324 16620  10.3173  21.1181 0.4922  2.0317  0.7104 15  0 20548	11 7823 1.000 0.4781  2.0458
#partial printout of hits follows
# "S"trip idx 25492 subdet layer side 4   2  0 (x y z) cm simIdx 10685 (x y z) hitpdgId 11 (simpt simpz) [SH for seed hit]
S 25492  4   2  0 ( 12.60  25.83  90.62) cm 10685  ( 10.32  21.17  90.62)	11 (  0.47   1.81)
S 25487  4   2  1 ( 11.96  25.90  90.86) cm 10684  ( 10.34  21.23  90.86)	11 (  0.48   1.81)
S 27208  4   3  0 ( 12.12  26.11  103.57) cm 10683  ( 11.34  24.42  103.57)	11 (  0.48   1.81)
S 27204  4   3  1 ( 11.85  25.95  103.81) cm 10680  ( 10.08  23.71  103.80)	22 (  0.00  -0.00)
S 71533  6   1  0 L27 ( 13.12  34.30  137.04) cm 10686  ( 12.70  33.18  137.04)	11 (  0.48   1.79) SH
...
# additional hits matching to the same simtrack are printed at the end
S 27206  4   3  1 ( 10.89  26.39  103.81) cm 10681  ( 11.35  24.49  103.81)	11 (  0.48   1.81) extra

The last updates have a few more parameters added.
I was using the following to compare a few events in iteration=4:
nt.scanTC(t132Xorig, t132X127moduw, minSimPt=0.2, maxSimPt=0.9, minSimVT = -2, maxSimVT = 2.5, nEv=1, nSkip=8, iteration=4, minSimFrac=0.75, pSeeds=True, pCHits=True, matchToSignal=False, matchTK=False, debug=False, pSimsMTV=False, pNCandLost=3)
the last flag pNCandLost is to control printouts for checking candidates present in the reference but not in the target: 0 for minimal info; 1 prints track candidate, 2 also its seed, 3 also all hits related to it.


Vulnerability roundup 99: kotlin-1.4.0: 1 advisory [5.3]

search, files

[ ] CVE-2020-29582 CVSSv3=5.3 (nixos-20.09)

Scanned versions: nixos-20.09: 95ce0f52ec1.
Cc @nequissimus

@ckauhaus This can be closed now since Kotlin has been updated to 1.5.10


raft: add our very first design.md

@yichengq @bdarnell
I am also going to draw the state machine states in ascii when I am patient...
Add our very first design about the Progress. I plan to talk about how we model raft as a pure state machine, the abstraction of tick and the node/multipNode in the future.


Use up or down gesture close image, image will show in background

if i remove setInitialPageIndex() method, it's ok, no this issue

same issue https://github.com/thiagoperes/IDMPhotoBrowser/issues/163


base model, instruct model, and fine-tuned base model

Three models

The pre-trained model (model A) is used as the base model.
We finetuned these base models by using our own dataset (Get model B).  // this is used as enhanced models
Instruct tuned models (model C)  ---// this is used as baseline right now for Table 1 and 2

And in the overleaf, I compared the performance of these instruct-tuned models (model C) with the model fine-tuned with our dataset (model B) in the previous version.

We do need another two tables similar to table 1 and 2  but use instruct models (model C) as baseline, then fine-tune baseline (instruct models) to get model D.
New two tables will show model D's improvements over model C.

I think the argument that "fine-tuning instruct models will get worse translation performance" is settled.
Your existing experiments show that "deepseek-coder-33b-instruct shows the greatest improvement".
Please confirm this so we do not need to debate this anymore. @bin123apple


getCameraPosRotForStep does not produce correct results

Create a step,  add a text to it, move it too far. (out of the step helper plane)

Fixed :)


Can't add latest patternfly/react-core in syndesis/ui

This is a...

[ ] Feature request
[x] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report
[ ] Documentation issue or request

Description
Unable to add @patternfly/react-core as a dependency to the @syndesis/ui package. When I do, and then attempt to perform a build, I receive the following error;

FWIW this worked before (recently) when we were POCing the AlertGroup component, maybe it's related to the move toward monorepo?
I've checked with others from the PF team and they've confirmed the ability to add latest react-core to a sub package in a monorepo/lerna structured app, so thinking the issue may be on our end.

Did you check out #140? I open that to give you a working starting point for this. Problem was yarn upgrading typescript to v3.4 which isn’t playing nice some bits of our codebase

Thx @riccardo-forina - hadn't seen that yet. I'll give it a run here in just a bit!


Some issues regarding the running mode of the Steady database

Steady build method: [QuickStart](https://eclipse.github.io/steady/user/tutorials/)
Question 1：View backend http://localhost:8033/bugs/ Found only 501 cve information, while there are 729 cve information in kb-importer/data/.kaybee/repositories/github.com_ sap.project-kb_vulnerability-data/statementsand 731 cve information in kb-importer/data/statements. I suspect that Steady's database import was not completed or was interrupted for some reason, so I want to manually import the information. Firstly, the three information items in kb-importer/data/steady.sh were modified: USER_ TOKEN、BACKEND_ URL、KB_ IMPORTER_ PATH, and then run steady.sh. May I ask if this can manually import vulnerability information.
Question 2：I attempted to rebuild steady by [QuickStart](https://eclipse.github.io/steady/user/tutorials/). Prior to this, I deleted the relevant docker images and found that the backend information of steady had not changed after rebuilding. http://localhost:8033/bugs/ ,The CVE information displayed on the webpage is consistent with the information before reconstruction. I checked the content in the kb-importer/data and found that the CVE information is empty and gradually increasing. Therefore, I would like to ask why there is still previous data on the backend and deleting the docker images cannot delete all the previous steady data. What should I do if I need to completely delete steady data.
I would greatly appreciate receiving a response!

Hi @11111821
Q1: More information about how data are imported from project-KB to eclipse steady (and how to manually do it) is available at ¶. If you relied on the default configuration of .env, you run the tool with KB_IMPORTER_SKIP_CLONE=True. This means that only the subset of CVEs was processed (those whose changed code is already available in the project-kb repository). We opted for such default as the runtime increases once repositories have to be cloned. To process the entire list of vulnerabilities you can modify the configuration to KB_IMPORTER_SKIP_CLONE=False
Q2: the data already imported is stored in a docker volume (visible in the docker-compose.yaml). To delete the steady data you can list the existing volumes with docker volume ls and delete the corresponding volume.
I hope this helps!


[bug][v2][android] Failed to create a symbolic link

Describe the bug
I try to create an Android app with Nuxt 3.13.1.
I'm on Windows 11, I followed all the Windows and Nuxt 3 prerequisites but when I launch the android dev command and I choose the "Medium_Phone_API_35" emulator I got this error :
Error Failed to create a symbolic link from
"E:\\Projects\\brain\\src-tauri\\target\\aarch64-linux-android\\debug\\libapp_lib.so"
to file
"E:\\Projects\\brain\\src-tauri\\gen/android\\app/src/main/jniLibs/arm64-v8a\\libapp_lib.so"
(file clobbering enabled): IO error: Incorrect function. (os error 1)

Reproduction

Follow the v2 Windows prerequisites
Create a fresh new Nuxt 3 app using npx nuxi@latest init app
Instal Tauri CLI with npm install -D @tauri-apps/cli@next
Init Tauri npx tauri init
Follow the example configuration from https://v2.tauri.app/start/frontend/nuxt/
Try to start the android dev server with npx tauri android dev
Choose the Medium_Phone_API_35 emulator

Expected behavior
The Nuxt start page launched on the emulator, I guess
Full tauri info output
[✔] Environment
    - OS: Windows 10.0.22631 x86_64 (X64)
    ✔ WebView2: 128.0.2739.67
    ✔ MSVC: Visual Studio Build Tools 2022
    ✔ rustc: 1.81.0 (eeb90cda1 2024-09-04)
    ✔ cargo: 1.81.0 (2dbb1af80 2024-08-20)
    ✔ rustup: 1.27.1 (54dd3d00f 2024-04-24)
    ✔ Rust toolchain: stable-x86_64-pc-windows-msvc (default)
    - node: 20.12.2
    - pnpm: 8.8.0
    - yarn: 1.22.19
    - npm: 10.7.0

[-] Packages
    - tauri 🦀: 2.0.0-rc.10
    - tauri-build 🦀: 2.0.0-rc.9
    - wry 🦀: 0.43.1
    - tao 🦀: 0.30.0
    - @tauri-apps/api : not installed!
    - @tauri-apps/cli : 2.0.0-rc.12

[-] Plugins

[-] App
    - build-type: bundle
    - CSP: unset
    - frontendDist: ../dist
    - devUrl: http://localhost:3000/
    - framework: Vue.js (Nuxt)
    - bundler: Webpack

Stack trace
Compiled plugins/server.mjs in 8622.28ms
Compiled plugins/client.mjs in 8675.9ms
Compiled types/plugins.d.ts in 9793.97ms
Vite client built in 575ms
Vite server built in 2030ms
warning: hard linking files in the incremental compilation cache failed. copying files instead. consider moving the cache directory to a file system which supports hard linking in session dir
?\E:\Projects\brain\src-tauri\target\x86_64-linux-android\debug\incremental\app_lib-2bc6ctgt3lhcg\s-gzqpludq51-062b9uj-working

warning: `app` (Lib) generated 1 warning
Finished `dev' profile [unoptimized + debuginfo] target(s) in 58.55s
Info symlinking lib "E:\\Projects\\brain\\src-tauri\\target\\x86_64-linux-android\\debug\\libapp_lib.so" in jniLibs dir "E:\\Projects\\brain\\src-tauri\\gen/android\\app/src/main/jniLibs/x86_64"
Nuxt Nitro server built in 27645 ms
Vite client warmed up in 1ms
Error Failed to create a symbolic link from "E:\\Projects\\\brain\\src-tauri\\target\\x86_64-linux-android\\debug\\libapp_lib.so" to file
"E:\\Projects\\brain\\src-tauri\\gen/android\\app/src/main/jniLibs/x86_64\\libapp_lib.so" (file clobbering enabled): I0 error: Incorrect function. (os error 1)

Additional context
No response

I think this is a known issue as flutter had the same message reference.
We probably also want to handle this error and display Try moving your Tauri project to the same drive as your Tauri CLI or something like this.

I think this is a known issue as flutter had the same message reference. We probably also want to handle this error and display Try moving your Tauri project to the same drive as your Tauri CLI or something like this.
@Theo-Rige can you confirm the project is on a different drive than the cli running?

Yes, my project is on an external drive.

Wondering if the different path segment separators could cause this?
gen/android\\app/src/main/jniLibs/x8 mixed with windows \\ in the same path. Idk how windows handles this for symlink creation. @FabianLars do you have a setup to reproduce?

do you have a setup to reproduce?

I don't have an external drive with enough space to build a mobile app 😂
But with E:\ being an external drive (assuming they mean something like an usb drive) we already know the issue. fat32 and exfat, the most common formats for those drives, simply do not support symlinks.

I'm on Windows 10,  when i run "npm run tauri android dev" command  that return this error :
symlinking lib "D:\project\Android\android-rust\src-tauri\target\x86_64-linux-android\debug\libandroid_rust_lib.so" in jniLibs dir "D:\project\Android\android-rust\src-tauri\gen/android\app/src/main/jniLibs/x86_64"
Failed to create a symbolic link from "D:\project\Android\android-rust\src-tauri\target\x86_64-linux-android\debug\libandroid_rust_lib.so" to file "D:\project\Android\android-rust\src-tauri\gen/android\app/src/main/jniLibs/x86_64\libandroid_rust_lib.so" (file clobbering enabled): IO
error: 拒绝访问。 (os error 5)
tauri info :
[✔] Environment
- OS: Windows 10.0.19045 x86_64 (X64)
✔ WebView2: 129.0.2792.79
✔ MSVC: Visual Studio 生成工具 2022
✔ rustc: 1.80.0 (051478957 2024-07-21)
✔ cargo: 1.80.0 (376290515 2024-07-16)
✔ rustup: 1.27.1 (54dd3d00f 2024-04-24)
✔ Rust toolchain: stable-x86_64-pc-windows-msvc (default)
- node: 20.10.0
- yarn: 1.22.21
- npm: 10.2.3
[-] Packages
- tauri 🦀: 2.0.2
- tauri-build 🦀: 2.0.1
- wry 🦀: 0.44.1
- tao 🦀: 0.30.3
- cargo-tauri 🦀: 1.0.0
- @tauri-apps/api : 2.0.2
- @tauri-apps/cli : 2.0.2

It worked for me once I enabled windows developer mode. I only used a single partition/drive though.

I'm experiencing a similar issue when running npm run tauri android dev on Windows 11. The error message is:
VITE v5.4.10  ready in 5034 ms

  ➜  Local:   http://localhost:1420/
   Compiling tauri v2.0.6
   Compiling tauri-plugin-shell v2.0.2
   Compiling deep v0.1.0 (C:\tauri_projects\deep\src-tauri)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 46.20s
    Info symlinking lib "C:\\tauri_projects\\deep\\src-tauri\\target\\x86_64-linux-android\\debug\\libdeep_lib.so" in jniLibs dir "C:\\tauri_projects\\deep\\src-tauri\\gen/android\\app/src/main/jniLibs/x86_64"
Failed to create a symbolic link from "C:\\tauri_projects\\deep\\src-tauri\\target\\x86_64-linux-android\\debug\\libdeep_lib.so" to file "C:\\tauri_projects\\deep\\src-tauri\\gen/android\\app/src/main/jniLibs/x86_64\\libdeep_lib.so" (file clobbering enabled): IO error: Cannot create a file when that file already exists. (os error 183)
    Error Failed to create a symbolic link from "C:\\tauri_projects\\deep\\src-tauri\\target\\x86_64-linux-android\\debug\\libdeep_lib.so" to file "C:\\tauri_projects\\deep\\src-tauri\\gen/android\\app/src/main/jniLibs/x86_64\\libdeep_lib.so" (file clobbering enabled): IO error: Cannot create a file when that file already exists. (os error 183)

Project Setup:
I created the project using npm create tauri-app@latest.
During setup, I selected Svelte and TypeScript as the framework and language.
tauri info:
[✔] Environment
    - OS: Windows 10.0.26100 x86_64 (X64)
    ✔ WebView2: 130.0.2849.52
    ✔ MSVC:
        - Visual Studio Build Tools 2022
        - Visual Studio Community 2022
    ✔ rustc: 1.82.0 (f6e511eec 2024-10-15)
    ✔ cargo: 1.82.0 (8f40fc59f 2024-08-21)
    ✔ rustup: 1.27.1 (54dd3d00f 2024-04-24)
    ✔ Rust toolchain: stable-x86_64-pc-windows-msvc (default)
    - node: 23.1.0
    - npm: 10.9.0
    - bun: 1.1.32
    - deno: deno 1.44.1

[-] Packages
    - tauri 🦀: 2.0.6
    - tauri-build 🦀: 2.0.2
    - wry 🦀: 0.46.3
    - tao 🦀: 0.30.3
    - @tauri-apps/api : 2.0.3
    - @tauri-apps/cli : 2.0.5

[-] Plugins
    - tauri-plugin-shell 🦀: 2.0.2
    - @tauri-apps/plugin-shell : 2.0.1

[-] App
    - build-type: bundle
    - CSP: unset
    - frontendDist: ../build
    - devUrl: http://localhost:1420/
    - framework: Svelte
    - bundler: Vite

Attempts to Resolve:

Deleted the existing libdeep_lib.so file
Ran the terminal as administrator
Enabled Windows Developer Mode
Turned off Windows Defender to rule out any interference

None of these steps resolved the issue.
Additional Information:

Drive Configuration:

I have two drives on my system:

C: SSD (NTFS format)
D: HDD (NTFS format)

I attempted to create and run the project on both the C: and D: drives, but the issue occurs on both.

Other PC without Partitions:

I tested the same project on another PC that doesn't have multiple partitions (only a single drive), and it worked without any issues.

Please let me know if there's any additional information I can provide to help resolve this issue.


windows: libwebrtc-sys conflicts with other crates that use C/C++

It looks like the prebuilt WebRTC library downloaded by the libwebrtc-sys build links the windows C Runtime Library statically, as opposed to dynamically, which causes linker errors when using other Rust crates that link to the C Runtime Library.
When compiling an application that uses both LiveKit and other crates that include C code via build.rs and the cc crate, the Zed project is getting a large number of linker errors like this:
libcpmt.lib(xstol.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MT_StaticRelease' doesn't match value 'MD_DynamicRelease' in libtree_sitter_yaml-df6b0adf8f009e8f.rlib(2e40c9e35e9506f4-scanner.o)

Here is the CI build where this error (and many similar ones) occurred.
The error mentions an object file libtree_sitter_yaml-..-scanner.o, which is just one of the crates that Zed depends on that uses C++. Here is the build.rs for that crate. You can see it's pretty standard - just including some C code via the cc crate.

Could you try using crt-static, as described in #277?


[1.10.2] "Joined [team name]" won't go away

Progression-1.10.2-0.6.7.jar
http://i.imgur.com/CtsrR9e.png
Simply won't go away even after changing team name.

I've noticed that too, "Joined Single Player" doesn't go away and sometimes when i finished a quest, the "New Criteria" popup doesn't go away either it seems (manual claiming was set to false).
Also version Progression-1.10.2-0.6.7

I haven't forgotten about this. I'll try to get back to fixing these things next week.

This bug seems to be arising still when Better Achievements is present.

They override the vanilla system and use their own local variables instead of the underlying vanilla ones, so makes sense why it wouldn't work.


Can segcontrast be implemented on S3DIS without changing a lot of code?

Can segcontrast be implemented on S3DIS without changing a lot of code?

Hi, it should be possible. The only thing that is necessary is to check how the clustered instances look like (output from this function). You can then adjust the hyperparameters from the plane fitting and from DBSCAN to extract good clusters from the S3DIS data.

Hi, it should be possible. The only thing that is necessary is to check how the clustered instances look like (output from this function). You can then adjust the hyperparameters from the plane fitting and from DBSCAN to extract good clusters from the S3DIS data.
Hopefully you can update how to change the loading of the custom dataset to fit the Segcontrast pipeline, such as S3DIS

On our new paper we have a new method for pre-training and in there we have some tips on how to implement a new dataloader for the model: https://github.com/PRBonn/TARL


If there was no cask installed, when invoking the start --upgrade command, new casks will never be upgraded

Would it harm if we would always run brew upgrade --cask -v even if no cask is installed?
Also, aren't there formulae which cloud also require SUDO_ASKPASS, to be auto upgraded?
Also the hint message should always be displayed, and not only when casks are already installed.
The following lines would need to be slightly updated.
start.rb
    if args.upgrade?
      auto_args << " && #{Autoupdate::Core.brew} upgrade --formula -v"

      if (HOMEBREW_PREFIX/"Caskroom").exist?
        if ENV["SUDO_ASKPASS"].nil? && !args.sudo?
          opoo <<~EOS
            Please note if you use Casks that require `sudo` to upgrade,
            you need to use `--sudo` or define a custom `SUDO_ASKPASS`
            environment variable.

          EOS
        end

        greedy = args.greedy? ? " --greedy" : ""
        auto_args << " && #{Autoupdate::Core.brew} upgrade --cask -v#{greedy}"
      end

Not stale

not stale

not stale