How to replace gstreamer from the examples gstreamer-receive and gstreamer-send

How could I replace gstreamer with something more lightweight c lib
or pure go lib if I only want to work with audio (non-GUI app)?
Any ideas?
Thanks in advance :)

you can describe what you want to do in more detail.

I would like to make a simple and lightweight cli audio chat app using pions webrtc.
Gstreamer seems to be a heavyweight dependency for me.

I think you can do this. But still there will be two dependencies this libssl developers want to remove it and I think this is very correct.
and you receive audio device pcm audio you can mux it to g711 alaw / ulaw
but webrtc does not support as I understand it pcm, alaw, ulaw, aac need encode to opus

you need find go project use opus https://github.com/hraban/opus
you need receive raw pcm audio you microphone
pcm -> encode to opus
next look sample and use (opus, err := peerConnection.NewRTCTrack)  opus.Samples <-  send packet opus
you need write communicate over http or tcp or any other proto for key exchange
write receive look sample it read Chanel and decode packet use https://github.com/hraban/opus to pcm
send pcm packet to sound output device

if you write, it no need gui and depending libssl, and libopus

libssl <- I would really like that the developers deleted this dependence
and then one would say proudly that this is on pure golang ;)

I can do this a little later, since I was on vacation.

What about https://github.com/nareix/joy4
Looks like a full bodied replacement of gstreamer to me.
What do you think ?
On Tue, 4 Sep 2018, 13:31 Andrey Semochkin<EMAIL_ADDRESS>wrote:

I can do this a little later, since I was on vacation.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pions/webrtc/issues/102#issuecomment-418333248, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ATuCwjIV90D26dYCr1xQ4CBIIMgIP_nqks5uXmR8gaJpZM4WYlTw
.

What about https://github.com/nareix/joy4

I don't think it has an opus decoder/encoder in it.

No it doesn't.
But I am thinking it's smarter to add it to that lib because it's setup for
muxing and transcoding that's all.
On Tue, 4 Sep 2018, 15:33 Joeky Zhan<EMAIL_ADDRESS>wrote:

What about https://github.com/nareix/joy4
I don't think it has an opus decoder/encoder in it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/pions/webrtc/issues/102#issuecomment-418368506, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ATuCwjnt3hFGb0me_ENtACbSC3ZtlOubks5uXoEfgaJpZM4WYlTw
.

Cross-platform is better,
but for now I am working on linux so I am using alsa lib to play the audio for testing.
And the following code does not work, it only produces noises.
diff --git a/examples/gstreamer-receive/main.go b/examples/gstreamer-receive/main.go
index 05174db..2466112 100644
@@ -8,9 +8,10 @@ import (
 	"bufio"
 	"encoding/base64"

+	"github.com/cocoonlife/goalsa"
 	"github.com/pions/webrtc"
-	"github.com/pions/webrtc/examples/gstreamer-receive/gst"
 	"github.com/pions/webrtc/pkg/ice"
+	"gopkg.in/hraban/opus.v2"
 )

 func main() {
@@ -46,14 +47,54 @@ func main() {

 	// Set a handler for when a new remote track starts, this handler creates a gstreamer pipeline
 	// for the given codec
+
+	const sampleRate = 48000
+	const channels = 2
+
+	dec, err := opus.NewDecoder(sampleRate, channels)
+	if err != nil {
+		fmt.Println("Decoder error")
+		panic(err)
+	}
+
+	out, err := alsa.NewPlaybackDevice("default", 1, alsa.FormatS16LE, sampleRate, alsa.BufferParams{})
+	if err != nil {
+		panic(err)
+	}
+
+	// require ALSA device
+	if out == nil {
+		fmt.Println("Device is nil!!")
+	} else {
+		fmt.Println(out)
+	}
+	// close device when finished
+	defer out.Close()
+
 	peerConnection.Ontrack = func(track *webrtc.RTCTrack) {
 		codec := track.Codec
 		fmt.Printf("Track has started, of type %d: %s \n", track.PayloadType, codec.Name)
-		pipeline := gst.CreatePipeline(codec.Name)
-		pipeline.Start()
+
+		pcm := make([]int16, 100000)
 		for {
 			p := <-track.Packets
-			pipeline.Push(p.Raw)
+
+			_, err := dec.Decode(p.Raw, pcm)
+
+			if err != nil {
+				fmt.Println("Decode error")
+				panic(err)
+			}
+
+			if pcm != nil {
+				// play!
+				out.Write(pcm)
+
+				fmt.Println("before:")
+				fmt.Println(p.Raw)
+				fmt.Println("after:")
+				fmt.Println(pcm)
+			}
 		}
 	}

Lots of info here, will try my best to respond to everything!
Is GStreamer required.
Nope! We have some examples that use GStreamer, but we also have others that save VP8 straight to disk with no extra dependencies save-to-disk
Dropping libssl
me and @trivigy talk about this a lot. It is going to be quite the undertaking, but we are going to get it done! I have a list of small issues I want closed, after that we can cut a new release and start another big feature push.
Adding more examples like libopus, ALSA, joy4
I would love this! The bar for getting code into /examples is really low for me. As long as they build, no lint errors I have no problem. You can add any new dependencies you want etc... just to show off what you can do with pion-WebRTC
Sample code
You should pass p.Payload that is just the body of the RTP packets. p.Raw includes the RTP headers etc...

You should pass p.Payload that is just the body of the RTP packets. p.Raw includes the RTP headers etc...

Thanks! I finally heard some voices, but the voice is so deep, like I am using a voice changer or something..

Nice! If you can hear voices I don't think there is anything left to tune with pion-WebRTC, since it just transports the audio buffers. If it was a bug at that level it would completely corrupt it (not change the pitch)
I would try playing with things like rate+format+channels and see if any of that helps.

I would try playing with things like rate+format+channels and see if any of that helps.

Change const channels = 2 to const channels = 1 the deep voice was fixed.
Thanks again! You guys are amazing!

I look at you it turned out to be done. It is perfectly.

Awesome glad to hear it worked @joeky888 :) Is there anything else I can do to help out, or mind if I close the issue?
If you are interested I would love to have your /examples in the repo, could really help another developer in the future!

mind if I close the issue?

Wait, I didn't add the recording code there. I'll make it as soon as possible.

I would love to have your /examples in the repo.

Sure, just do it. But it only works on Linux. I would like to rewrite it using ether oto or openal.

Edit:
I will close it anyway. I will update my code and post it here, thanks!

Hello guys, me again.
I am trying to finish the recording part in examples/gstreamer-send.
Once again the browser only produces noises.
Here is how I did it:
diff --git a/examples/gstreamer-send/jsfiddle/demo.js b/examples/gstreamer-send/jsfiddle/demo.js
index 08de6db..fcbba10 100644
@@ -27,7 +27,7 @@ pc.onicecandidate = event => {
   }
 }

-pc.createOffer({offerToReceiveVideo: true, offerToReceiveAudio: true}).then(d => pc.setLocalDescription(d)).catch(log)
+pc.createOffer({offerToReceiveVideo: false, offerToReceiveAudio: true}).then(d => pc.setLocalDescription(d)).catch(log)

 window.startSession = () => {
   let sd = document.getElementById('remoteSessionDescription').value
diff --git a/examples/gstreamer-send/main.go b/examples/gstreamer-send/main.go
index 008679f..47a1c6a 100644
@@ -8,9 +8,13 @@ import (
 	"bufio"
 	"encoding/base64"

+	"github.com/cocoonlife/goalsa"
 	"github.com/pions/webrtc"
-	"github.com/pions/webrtc/examples/gstreamer-send/gst"
+
+	// "github.com/pions/webrtc/examples/gstreamer-send/gst"
 	"github.com/pions/webrtc/pkg/ice"
+	"github.com/pions/webrtc/pkg/media"
+	"gopkg.in/hraban/opus.v2"
 )

 func main() {
@@ -89,7 +93,56 @@ func main() {
 	fmt.Println(base64.StdEncoding.EncodeToString([]byte(answer.Sdp)))

 	// Start pushing buffers on these tracks
-	gst.CreatePipeline(webrtc.Opus, opusTrack.Samples).Start()
-	gst.CreatePipeline(webrtc.VP8, vp8Track.Samples).Start()
+	const channels = 2
+	const sampleRate = 48000
+	const format = alsa.FormatS16LE
+	const pcmsize = 480
+
+	recorder, err := alsa.NewCaptureDevice("default", channels, format, sampleRate,
+		alsa.BufferParams{BufferFrames: 0, PeriodFrames: pcmsize, Periods: pcmsize})
+
+	if err != nil {
+		panic(err)
+	}
+	defer recorder.Close()
+
+	enc, err := opus.NewEncoder(sampleRate, channels, opus.AppVoIP)
+	if err != nil {
+		panic(err)
+	}
+
+	pcm := make(chan []int16)
+
+	/* Recording audio */
+	go func() {
+		p := make([]int16, pcmsize)
+
+		for {
+			recorder.Read(p)
+			pcm <- p
+		}
+	}()
+
+	/* Send audio */
+	go func() {
+		p := make([]int16, pcmsize)
+		opusdata := make([]byte, 10000)
+
+		for {
+			p = <-pcm
+
+			n, err := enc.Encode(p, opusdata) // pcm to opus
+			if err != nil {
+				panic(err)
+			}
+			opusdata = opusdata[:n] // Remove unused space after encoding to opus
+
+			opusTrack.Samples <- media.RTCSample{Data: opusdata, Samples: uint32(pcmsize)}
+		}
+	}()
+
+	// gst.CreatePipeline(webrtc.Opus, opusTrack.Samples).Start()
+	// gst.CreatePipeline(webrtc.VP8, vp8Track.Samples).Start()
+
 	select {}
 }

I would be appreciated with your help! :)

I created a repo to replace gstreamer
Links: https://github.com/joeky888/pions-examples
Feel free to use it!

I added a packetizer to add rtp headers (I think)
still no luck :(
diff --git a/examples/gstreamer-send/main.go b/examples/gstreamer-send/main.go
index 008679f..5f65d67 100644
@@ -1,6 +1,8 @@
 package main

 import (
+	"crypto/rand"
+	"encoding/binary"
 	"fmt"
 	"io"
 	"os"
@@ -8,9 +10,14 @@ import (
 	"bufio"
 	"encoding/base64"

+	"github.com/cocoonlife/goalsa"
 	"github.com/pions/webrtc"
-	"github.com/pions/webrtc/examples/gstreamer-send/gst"
+
+	// "github.com/pions/webrtc/examples/gstreamer-send/gst"
 	"github.com/pions/webrtc/pkg/ice"
+	"github.com/pions/webrtc/pkg/media"
+	"github.com/pions/webrtc/pkg/rtp"
+	"gopkg.in/hraban/opus.v2"
 )

 func main() {
@@ -26,11 +33,19 @@ func main() {
 		panic(err)
 	}

+	const channels = 2
+	const sampleRate = 48000
+	const format = alsa.FormatS16LE
+	const pcmsize = 480
+	const mtu = 1400
+	PayloadTypeOpus := uint8(webrtc.DefaultPayloadTypeOpus)
+	opuscodec := webrtc.NewRTCRtpOpusCodec(PayloadTypeOpus, sampleRate, channels)
+
 	/* Everything below is the pion-WebRTC API, thanks for using it! */

 	// Setup the codecs you want to use.
 	// We'll use the default ones but you can also define your own
-	webrtc.RegisterDefaultCodecs()
+	webrtc.RegisterCodec(opuscodec)

 	// Create a new RTCPeerConnection
 	peerConnection, err := webrtc.New(webrtc.RTCConfiguration{
@@ -61,14 +76,14 @@ func main() {
 	}

 	// Create a video track
-	vp8Track, err := peerConnection.NewRTCTrack(webrtc.DefaultPayloadTypeVP8, "video", "pion2")
-	if err != nil {
-		panic(err)
-	}
-	_, err = peerConnection.AddTrack(vp8Track)
-	if err != nil {
-		panic(err)
-	}
+	// vp8Track, err := peerConnection.NewRTCTrack(webrtc.DefaultPayloadTypeVP8, "video", "pion2")
+	// if err != nil {
+	// 	panic(err)
+	// }
+	// _, err = peerConnection.AddTrack(vp8Track)
+	// if err != nil {
+	// 	panic(err)
+	// }

 	// Set the remote SessionDescription
 	offer := webrtc.RTCSessionDescription{
@@ -89,7 +104,74 @@ func main() {
 	fmt.Println(base64.StdEncoding.EncodeToString([]byte(answer.Sdp)))

 	// Start pushing buffers on these tracks
-	gst.CreatePipeline(webrtc.Opus, opusTrack.Samples).Start()
-	gst.CreatePipeline(webrtc.VP8, vp8Track.Samples).Start()
+
+	recorder, err := alsa.NewCaptureDevice("default", channels, format, sampleRate,
+		alsa.BufferParams{BufferFrames: 0, PeriodFrames: pcmsize, Periods: pcmsize})
+
+	if err != nil {
+		panic(err)
+	}
+	defer recorder.Close()
+
+	enc, err := opus.NewEncoder(sampleRate, channels, opus.AppVoIP)
+	if err != nil {
+		panic(err)
+	}
+
+	pcm := make(chan []int16)
+
+	/* Recording audio */
+	go func() {
+		p := make([]int16, pcmsize)
+		for {
+			recorder.Read(p)
+			pcm <- p
+		}
+	}()
+
+	/* Send audio */
+	go func() {
+		p := make([]int16, pcmsize)
+		opusdata := make([]byte, 1000)
+
+		ran := make([]byte, 4)
+		_, err = rand.Read(ran)
+		if err != nil {
+			panic(err)
+		}
+
+		ssrc := binary.LittleEndian.Uint32(ran)
+
+		packetizer := rtp.NewPacketizer(
+			mtu,
+			opuscodec.PayloadType,
+			ssrc,
+			opuscodec.Payloader,
+			rtp.NewRandomSequencer(),
+			sampleRate,
+		)
+
+		for {
+			p = <-pcm
+
+			n, err := enc.Encode(p, opusdata) // pcm to opus
+			if err != nil {
+				panic(err)
+			}
+			opusdata = opusdata[:n] // Remove unused space after encoding to opus
+
+			packets := packetizer.Packetize(opusdata, pcmsize)
+
+			raw, err := packets[0].Marshal()
+			if err != nil {
+				panic(err)
+			}
+
+			opusTrack.Samples <- media.RTCSample{Data: raw, Samples: uint32(pcmsize)}
+		}
+	}()
+
+	// gst.CreatePipeline(webrtc.Opus, opusTrack.Samples).Start()
+	// gst.CreatePipeline(webrtc.VP8, vp8Track.Samples).Start()
 	select {}
 }

Hey @joeky888 really nice work with those examples!
You mind posting what isn't working as a finished code sample (gist or in that linked repo) I should have time this weekend to check it out!
Would you be ok with me importing your examples? I will mark you as a coauthor on the commit, but would love if other people could try them out easy!
thanks

You mind posting what isn't working as a finished code sample

Here it is: https://github.com/joeky888/pions-examples/tree/master/alsa-send

Would you be ok with me importing your examples?

I don't mind, just do it!


BigInt should be handled by native codec instead of extension codec

It is not a good choice to handle 64bit integers with extension codec. While 64 bit integers are built-in types of msgpack and BigInt is a built-in type of javascript, solving 64 bit problem with extension codec is not an compatible way when making cross-language systems. If javascript developers use an extension codec to support 64bit integers, then developers in other languages will have to write an corresponding extension codec, while msgpack implementation in other languages (c++, c, python, java, etc.) natively support mapping 64 bit integers into their language-specific type.
To support BigInt, I think this library should map 64 bit integers to BigInt natively or this library should add support to register custom codec for built-in types.
original issue #114

BigInt is not compatible with numbers (i.e. 64-bit floating-point numbers) in JavaScript, and BigInt is not compatible with 64-bit ints in general, so I suppose implicit mapping of BigInt and 64-bit ints should cause problems.
However, if one knows what they are doing, its mapping could be helpful for cross-platform cooperation, so I'd like to consider an optional feature to map BigInt and 64-bit ints.

Glad to hear that you have take mapping BigInt and 64-bit integers into consideration. But I don't think it is optional. Yes, BigInt is not compatile with javascript Number as Number in javascript is something like double, but no, BigInt should be compatible with 64-bit integers of msgpack as integers and floats are different things in general and as the existence of BigInt.asIntN and BigInt64Array.

+1

I also believe adding native BigInt support to handle MessagePack's native 64 bit integers is the right way to go. I also see @gfx's concern, because MessagePack tends to pick the smallest representation when encoding numbers, which could lead to an annoying schism on the JS side.
For example, if I have an u64 in Rust that should be encoded with MessagePack, MessagePack will happily encode it as a 7-bit integer if the value fits, which would come out as a number on the JS side. But if the value were larger than 2^32 - 1 it would use the 64-bit encoding, which would suddenly come out as a BigInt on the JS side. This can be annoying if you have to deal with both (incompatible) types on the JS side.
The current "solution" at least supports number up to 2^53 - 1 consistently on the JS side, but it's not a solution at all for anyone who needs numbers bigger than that. So I agree having an optional feature to opt-in to the use of BigInt would be the right way to go that I believe would satisfy everyone.

I've just made #211, which is a PR to add BigInt support in an opt-in and flexible way. Feedback/support would be appreciated.

Thank you for your effort, @jasonpaulos, but I've made another PR for bigint support: https://github.com/msgpack/msgpack-javascript/pull/223 , which introduces an option useBigInt64. It's not so much flexible. It introduces a new mapping for JavaScript's bigint and MessagePack's int64/uint64, as JavaScript's DataView does (it uses DataView's native features, indeed).
I'll release it at the end of March 2023 if nobody has objections.

I responded in that PR with an issue I noticed: https://github.com/msgpack/msgpack-javascript/pull/223#issuecomment-1484111129
We are currently using an old fork of this library with bigint support. I would very much like to stop using that fork and use the original library again, but the issue I pointed out in my comment would prevent us from migrating.


Dependency cleanup

Ensures jcl-over-slf4j in all cases
Aligns and bumps mockito version
Aligns and bumps HikariCP
Bumps spring to avoid CVE vulnerability warning
Bumps redis and spring-data-redis

:+1:

👍


enable prometheus for scale-out

Changes
add changes from the following commits to enable prometheus in scale-out cluster
https://github.com/futurewei-cloud/arktos-perftest/commit/e5f6915b1603e29fc76f6ec6af3739573165ddd8
https://github.com/futurewei-cloud/arktos-perftest/commit/c808cf317b2a0d6b16a672f1cfe81709c3c54a2b

@sonyafenge other than the changes here, is there anything else we need to do to enable prometheus when starting the cluster?

@sonyafenge other than the changes here, is there anything else we need to do to enable prometheus when starting the cluster? for example, env variables like PROMETHEUS_SCRAPE_ETCD=true ENABLE_PROMETHEUS_SERVER=true

no, we don't need any env variables because this is a temporally solution without switch variables.  ENABLE_PROMETHEUS_SERVER=true is the one supported by community and arktos code, I have a quick check and cannot get this one access outside of cluster.

Remove the "do not merge" label as Qian confirmed the verification is done.

/approve

/lgtm


Java Port Licensing

I've ported this implementation to java and added support for teams. I've linked this repo in the code for attribution, is there anything else I'd be required to do license wise?
https://github.com/DeinFreund/ZKForumParser/blob/master/src/whr/WholeHistoryRating.java
For now it's being used to compare against Elo accuracy in an online game, but it might replace Elo there in the future.

I've created a rewrite in .NET (with a couple additions) and would like to know, too, how the license should be handled. I want to give credit, but I also don't want to make it seem like the original author is responsible for bugs that I introduced :)


Add instance map for clues

Is nothing more frustrating when you have to click on world map for every single step and inconter that laggy map.Add instance map that open automatically

I don't know how you managed to open this as a PR, but this should be closed and re-opened as an issue/feature request.
Click here


Parsing: Numeric Separators

What version of this package are you using?
<EMAIL_ADDRESS>What problem do you want to solve?
Allow numeric separators (https://v8.dev/features/numeric-separators).

What do you think is the correct solution to this problem?
Numbers using numeric separator should not throw parsing error.
Are you willing to submit a pull request to implement this change?

Please update to standard 16.


wrong gas density and no pressure dependence

The calculation of density
https://github.com/DUNE/garsoft/blob/137e7a62a29ff2b409759db102c4865420d2883b/DetectorInfo/DetectorPropertiesStandard.cxx#L189
gives too high a density by a factor of nearly 10 and it does not depend on pressure.
This has been copied for the TOAD service as well:
https://github.com/DUNE/garsoft/blob/137e7a62a29ff2b409759db102c4865420d2883b/DetectorInfo/DetectorPropertiesStandardToad.cxx#L189
To do -- add dependence on pressure and get the density right at 10 bar and 294K (21C, 70F).  This is used in the dE/dx calculation.

All closed with PR 15


fork: Add documentation and licensing

I forked your branch to incorporate the licensing and contribution guides from my initial PR.  I'll close that out and we can go with yours

nice 👍
merged 👍


Support k8s for job submission and management

Why are these changes needed?
Support using k8s to manage job lifecycles, including job submission, initialization, termination and clean-up.

TODO:

[x] add README for k8s job management tutorial

change in docker/driver.py is added to use k8s client apis for job management, now the driver will support "default", "docker" and "k8s" modes.
add a yaml generator for automating generation of k8s container configs.
add new example k8s configs in benchmark

Related issue number

Checks

[x] I've included any doc changes needed for https://fedscale.readthedocs.io/en/latest/
[x] I've made sure the following tests are passing.
Testing Configurations

k8s

[x] Dry Run (20 training rounds & 1 evaluation round)
[x] Cifar 10 (20 training rounds & 1 evaluation round)
[x] Femnist (20 training rounds & 1 evaluation round)

Regression 1: docker

[x] Cifar 10 (20 training rounds & 1 evaluation round)
[x] Femnist (20 training rounds & 1 evaluation round)

Regression 2: default

[x] Cifar 10 (20 training rounds & 1 evaluation round)
[x] Femnist (20 training rounds & 1 evaluation round)

I think we have too many example *.yml for different setups. I think we should:

create two folders under /configs: docker_deploy and k8s_deploy, and remove others;
Highlight the changes in *.yml (e.g., use_container: docker) and add pointers to the corresponding README.

Please feel free to post your thoughts here. Thank you!

I think we have too many example *.yml for different setups. I think we should:

Create two folders (examples) under /configs: docker_deploy and k8s_deploy. Each folder has only an example.yml (e.g., femnist.yml), and remove others (e.g., in femnist folder);
Highlight the changes in *.yml (e.g., use_container: docker) and add pointers to the corresponding README.

Please feel free to post your thoughts here. Thank you!

Gotcha. Should I leave the old yamls in the original folders?

I think we have too many example *.yml for different setups. I think we should:

Create two folders (examples) under /configs: docker_deploy and k8s_deploy. Each folder has only an example.yml (e.g., femnist.yml), and remove others (e.g., in femnist folder);
Highlight the changes in *.yml (e.g., use_container: docker) and add pointers to the corresponding README.

Please feel free to post your thoughts here. Thank you!

Gotcha. Should I leave the old yamls in the original folders?

For now, I think we can group them into these two folders. e.g., move all k8s.ymls to the k8s

I think we have too many example *.yml for different setups. I think we should:

Create two folders (examples) under /configs: docker_deploy and k8s_deploy. Each folder has only an example.yml (e.g., femnist.yml), and remove others (e.g., in femnist folder);
Highlight the changes in *.yml (e.g., use_container: docker) and add pointers to the corresponding README.

Please feel free to post your thoughts here. Thank you!

Gotcha. Should I leave the old yamls in the original folders?

For now, I think we can group them into these two folders. e.g., move all k8s.ymls to the k8s

Sounds good. I grouped configs by deployment methods, and highlighted changes in yml by comments, also updated docs/tutorial.md.

Move default configs in default_deploy back to the original folders. (commit: 8297af1e2ea7c5a89a69de3e4dbe3a86b75b6489)

Looks good to me know. Thank you!


⚠ Bofrost.lu has degraded performance

In e59d9af, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1347 ms

Resolved: Bofrost.lu performance has improved in 3720c59 after 313 days, 3 hours, 39 minutes.


Shutdown implementation which hooks into /world/Del

Add shutdown handling for rust-g
Move log closing to said proc.
Implement job detaching for said proc.

Wonder if this would fix our need to have a patch on the HTTP client that has a designated create and shutdown method.

where do they go when they get detatched.

where do they go when they get detatched.

No where, we just drop our reference to them which means that they become free standing threads. There's no easy way to hook cancelation logic in and leaving them orphaned won't cause any problems

"Detaching" the threads would mean they may still exist after the module is unloaded which will result in a crash. It doesn't do anything.


Import search strategy

[ ] path to search and which order
[ ] support dylib, .a and .dll

Actually just searching for the right file extension should be enough. Zig's std.DynLib already abstract away OS differences.


mountpoint remains in use after restore snapshot

System information
Distribution Name    |  Ubuntu
Distribution Version  | 16.04
Linux Kernel             | 4.4.0-72-generic
Architecture              | x86_64
ZFS Version             | <IP_ADDRESS>-0ubuntu15
SPL Version             | <IP_ADDRESS>-0ubuntu4
zfsutils-linux Version| <IP_ADDRESS>-0ubuntu16
Problem:
I use lxd
after restore container from shapshot cannot add new snapshot or restore again
until restart container
Step to reproduce the problem:
launch container:
lxc launch debian/jessie
Creating popular-kitten
Starting popular-kitten

and create snapshot:
lxc snapshot popular-kitten
let`s see configs and stats:
lxc info popular-kitten
Name: popular-kitten
Remote: unix:/var/lib/lxd/unix.socket
....
Profiles: default

lxc profile show default
...
pool: main-pool

zfs get mounted main-pool/containers/popular-kitten
NAME PROPERTY VALUE SOURCE
main-pool/containers/popular-kitten mounted yes -

zfs get mounted main-pool/snapshots/popular-kitten
NAME PROPERTY VALUE SOURCE
main-pool/snapshots/popular-kitten mounted yes -

now restore container from snapshot:
lxc restore popular-kitten snap0
and see, what changes
zfs get mounted main-pool/snapshots/popular-kitten
NAME PROPERTY VALUE SOURCE
main-pool/snapshots/popular-kitten mounted yes -

zfs get mounted main-pool/containers/popular-kitten
NAME PROPERTY VALUE SOURCE
main-pool/containers/popular-kitten mounted no -

if try to make new snapshot or restore again:
lxc snapshot popular-kitten
error: Failed to mount ZFS filesystem: filesystem 'main-pool/containers/popular-kitten' is already mounted
cannot mount 'main-pool/containers/popular-kitten': mountpoint or dataset is busy

lxc restore popular-kitten snap0
error: Failed to mount ZFS filesystem: filesystem 'main-pool/containers/popular-kitten' is already mounted
cannot mount 'main-pool/containers/popular-kitten': mountpoint or dataset is busy

but container still work:
lxc info popular-kitten
Name: popular-kitten
Remote: unix:/var/lib/lxd/unix.socket
Architecture: x86_64
Created: 2017/04/25 07:17 UTC
Status: Running

lxc exec popular-kitten bash
root@popular-kitten:~# uptime
 07:34:06 up 8 min, 0 users, load average: 0.00, 0.02, 0.03

after restart container:
lxc restart popular-kitten
zfs get mounted main-pool/containers/popular-kitten
NAME PROPERTY VALUE SOURCE
main-pool/containers/popular-kitten mounted yes -

on another server this problem missmatch:
Distribution Name    |  Ubuntu
Distribution Version  | 16.10
Linux Kernel             | 4.8.0-46-generic
Architecture              | x86_64
ZFS Version             | <IP_ADDRESS>-0ubuntu4.1
SPL Version             | <IP_ADDRESS>-2
zfsutils-linux Version| <IP_ADDRESS>-0ubuntu4.1
this bug on ubuntu bug tracker:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1686036

I can confirm this bug, it affects a lot of our containers too on very different host machines. It started somewhat recently, I think LXD 2.10 and above, but I'm not 100% sure.

We do more {m,u}ounting in post-storage-api LXD instances so it's likely that this bug would happen more often. So what I'm thinking is that the zfs filesystem for the container get's lazily unmounted at some point and then we hit the zfs bug where zfs when it detects that someone is still holding a reference to the dataset (e.g. when the container is still running) it doesn't allow you to remount it and fails with EBUSY. I think I have a temporary workaround for this now in #3229.


Build fails with JDK11

It seems that the java-allocation-instrumenter dependency fails on JDK11, by the looks of it because of a transitive dependency on an outdated version of ASM:
WARNING: Failed to instrument class.
java.lang.IllegalArgumentException
	at com.google.monitoring.runtime.instrumentation.asm.ClassReader.<init>(ClassReader.java:160)
	at com.google.monitoring.runtime.instrumentation.asm.ClassReader.<init>(ClassReader.java:143)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:177)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:210)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.transform(AllocationInstrumenter.java:157)
	at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
	at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
	at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:167)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.bootstrap(AllocationInstrumenter.java:141)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.premain(AllocationInstrumenter.java:117)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.google.monitoring.runtime.instrumentation.AllocationInstrumenterBootstrap.premain(AllocationInstrumenterBootstrap.java:48)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
Oct 21, 2018 6:49:27 PM com.google.monitoring.runtime.instrumentation.AllocationInstrumenter instrument

See this failed build. This failed with OpenJDK and Oracle JDK alike.

I don't have OpenJDK 11 installed yet.
Do you happen to have a fix proposal?

I don't know if the library in question is still maintained; the latest version is being used. Memory profiling can be done with JMH's GC profiler, which works with JDK11, so one solution would be to use JMH instead.

I have raised the issue upstream. Let us see if they respond.
Presumably, this failure is specific to one test which we can workaround on JDK11, right?

The library sure looks like it is maintained... there are commits this month... but they may be due for a new release.

We have updated the instrumenter library to its latest version.

If you want to discuss dropping support for JDK8, I am game, but I think we need some kind of justification considering that it effectively comes for free at this time.

Ok. Let us close this issue given that I effectively did as you recommended and dropped the memory subpackage.

What I mean is 6 builds at a time seem to get throttled. OpenJDK8 will be a going concern for a while so I would not suggest dropping it, but oraclejdk8 is nearly EOL. *jdk9 and *jdk10 are already obsolete, so I think you could get away with:

openjdk8
oraclejdk11
openjdk11

Good.


Inputpicker is not responsive?

Hello, i try to implement inputpicker to bootstrap 4 column container, but it lost responsiveness behavior by bootstrap.
I also tried to use width : '100%', which doc said will be "equal with input", still no responsive.
How can i resolve this? Thanks

Hi @duckzringz ,
Can you please give me more details about your issue such as a screenshot or source code? So I can do the further investigation. Thanks.

Hi @ukalpa ,
Here's sample of my code:

            Type

                <input id="type" class="form-control" name="type" value="" />

            Name

                <input id="name" class="form-control" name="name" value="" type="text">

This is normal browser display. It is correct, displayed as "col-8" class

But for responsive design (also for mobile devices), it's not automatically responsive

And for "landscape" switch, not automatically responsive

I think the problem is page "refresh". It will correctly responsive if the page refreshed.
But for mobile devices, it should be automatically "switched" for portrait/landscape mode, without refreshing the page.
(Take screenshot 3 above, regular textbox is adjusted automatically according to device width, without page refresh).
So if there any workaround, it would be great, Thanks

Hi @duckzringz ,
Thank you very much for your suggestion, I have added the responsive to the inputpicker, please update to the latest version and check again.

Hello @ukalpa ,
This is just awesome!!
It works, even though sometimes it doesn't on mobile upon switching portrait to landscape (vice versa), but i think it only occurs 2 out of 10 tries.
It's fine i think, thank you so much


State the documentation link near the top of the readme

I kept having to scroll around to find the docs link, usually it's at the very top. It may save some people a little time.

The button named doc|Doxygen links to the documentation. Do you think that if we were to rename the label on this one it would have caught your attention, or is it better to simply have the text you added? (we could do both too)

@kozlovic Ah I didn't even see that. I personally don't look at those much because some repos just add a ton of meaningless badges. Now that I know about the badge it's 100% up to you.

Ok, I will merge, but could you do me a favor and amend your commit to add [ci skip] in the summary (or anywhere in the commit). This is to prevent triggering Travis build that we don't need for such change. From your branch
git commit --amend
<edit the summary to add [ci skip] in there> then save
git push --force

Thanks!

Thanks!


Core: Check the efficiency of VlogReasoner:writeInferences

Some Experiments with a huge number of facts can be done to check how efficient the method is and whether we would need to obtain the facts directly from VLog instead of using VLogToModelConverter to create objects for them.

Related to #135


Bouncy .45 magnum

About the PR

draft for review and i'm not done
requires engine pr
added bouncy .45 magnum bullets that ricochet off of walls
works pretty good but is REALLY janky (bullets retroreflecting, passing through walls, passing between walls, bouncing inside walls, etc.)
Why / Balance

mostly for the funny
the damage is reduced
gonna make them manufacturable
Technical details

Media

[ ] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl:

add: You can now create .45 magnum rounds that bounce!

I'm not sure if these should be a bullet or an item. There's a traitor item called the mechanical eyepatch that does this for all bullet types: https://github.com/Citadel-Station-13/Citadel-Station-13/pull/13011
I'll think about it, but I feel like instead of having it just be a bullet it should be something that's granted.

okay so turns out this is literally impossible to make reasonable due to how collisions work

basically, unless i crank the size way the hell up it breaks completely, making it impossible to make this work AND fun to play against at the same time


User profile error message

The error message of editing user profile doesn't seem to be correct. @ckcollab Is this module usable?

This was a bug with a front-end library we are using. Wrote a custom validator so it should the working now.


Sentry: display message to indicate how to opt out from Sentry

Description
It is likely that some users who have initially opted in for sentry (crash report) will be annoyed. They need to be able to opt out at anytime, hence this should be documented. One suggestion is to display a message like:
If you don't want to send crash report anymore, edit the file ${SCT_DIR}/bin/sct_launcher and delete the line that starts with "export SENTRY_DSN"

Ideally, this message should be displayed everytime an error is sent. I don't know if there is a method in raven that enables to add custom message upon connection.

Fixed by #1753


error when patching content on client side only

Version
2.5.16
Reproduction link
https://github.com/neyb/nuxt-bug-clientRendering
Steps to reproduce

npm i
npm run dev
access http://localhost:3000/
refresh (f5)

What is expected?
browser should show
hello
gentlemen
What is actually happening?
browser show
hello

This is a pretty sever bug to me, as it make you cannot use libraries.
I already logged this bug to nuxtjs. Actually I think it is linked to vue so I post it there (I would have prefer to wait nuxt team to tell me that it's a vuejs bug, but they seems to all be dead).
nuxt bug links : https://cmty.app/nuxt/nuxt.js/issues/c7361 & https://github.com/nuxt/nuxt.js/issues/3515
error in console
[nuxt] Error while initializing app TypeError: Cannot read property 'toLowerCase' of undefined
    at emptyNodeAt (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:5493)
    at VueComponent.patch [as __patch__] (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:6118)
    at VueComponent.Vue._update (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:2656)
    at VueComponent.updateComponent (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:2784)
    at Watcher.get (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:3138)
    at new Watcher (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:3127)
    at mountComponent (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:2791)
    at VueComponent.webpackJsonp../node_modules/vue/dist/vue.runtime.esm.js.Vue.$mount (C:\Users\GS8930\dev\project\bugnuxt\node_modules\vue\dist\vue.runtime.esm.js:79

after trying to understand what happens, it seems that vuejs is trying to mount a component in a comment that is kind of a placeholder for the component that s only rendered on client
condition to reproduce the bug

node is only rendered on client (with a vif)
vif is using a data (also work with computed)
vif is on a custom component
vif with same condition has not been used before in template

note

uncommenting  make the x-div patch successfull
:ok:

hello

ladies and

<x-div v-if="onClient">gentlemen</x-div>

display
hello
ladies and
gentlemen

:beetle:

hello

<x-div v-if="onClient">gentlemen</x-div>

ladies and

display
hello

workaround
if bug does not occure in a library component, it is possible to move the vif on a div surrounding the component : <x-div v-if="onClient">gentlemen</x-div> =>
<x-div >gentlemen</x-div>

Please, do not post repros using other libraries as the error could come from there. Make sure it doesn't come from the original library first, then post a bug report with a boiled down repro (without the external lib, here, Nuxt). Thanks


Need to add an ability to prevent dropping a piece that is being dragged.

Use case:
The user clicks and drags a Knight but moves it in a straight line and/or to an illegal square. Ideally the drag-n-drop would query a new event to determine if dropping the piece is allowed in a given location, and if not, display user visual feedback through {cursor: not-allowed;} and snap-back.
    /**
     * Signature: function(event: { piece: string, sourceSquare: string, targetSquare: string, allowDrop: boolean }) => void
     */
    canDrop: PropTypes.func,

Note: The allowDrop property should default to true, and if set to false by the function, indicate to the user that the piece can not be dropped.

Ok, this should fit nicely with ReactDnD's canDrop method.

I think this is outside the scope of chessboardjsx.  I want to keep the logic of the game separate from the board.
Signature: function(event: { piece: string, sourceSquare: string, targetSquare: string, allowDrop: boolean }) => void
There would be no way for chessboardjsx to provide an allowDrop param in any meaningful ways since it doesn't know the rules of chess.  Best to handle that with chess.js before passing the position to chessboardjsx:
HumanVsHuman
I'm not sure if I want to give control of the cursor to the consumer at this point.  I think a snapback as in the demo coupled with highlighting the legal squares is enough to show the player what moves are allowed.  Making some sort of cursor prop seems a little too niche

perhaps draggable could take a boolean for the whole board or an object for a particular square.  draggable={ {'e5': false } }.


Parsing error

This is the first time I'm trying to use a userscript. I'm getting the following error when trying to compile the script:

What can I do?

This is not a userscript, but a user style. It's designed to be used with extensions like Stylus, which you can find in the nearest extension store.

Fantastic, thank you for responding so quickly and solving the issue.


Issues with generating points1 files for numerous plots

Describe the Problem
A user has noticed issues with the content in the .points1 file for the following plots:

line
box
bar
ROC <- cannot reproduce for ROC (works for Python 3.6 and Python3.8)

Refer to Github discussions Issue  #1594
This was reproduced using Python 3.8 and the third-party packages defined in the requirements.txt of the feature_208_py38.
This problem does not exist when using Python 3.6 and the related third-party packages.
Expected Behavior
Provide a clear and concise description of what you expected to happen here.
Environment
Describe your runtime environment:
1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop)
2. OS: (e.g. RedHat Linux, MacOS)
3. Software version number(s)
To Reproduce
Describe the steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
Post relevant sample data following these instructions:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-help-desk#ftp
Relevant Deadlines
List relevant project deadlines here or state NONE.
Funding Source
2702691
Define the Metadata
Assignee

[ ] Select engineer(s) or no engineer required
[ ] Select scientist(s) or no scientist required

Labels

[ ] Select component(s)
[ ] Select priority
[ ] Select requestor(s)

Projects and Milestone

[ ] Select Organization level Project for support of the current coordinated release
[ ] Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
[ ] Select Milestone as the next bugfix version

Define Related Issue(s)
Consider the impact to the other METplus components.

[ ] METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy

Bugfix Checklist
See the METplus Workflow for details.

[ ] Complete the issue definition above, including the Time Estimate and Funding Source.
[ ] Fork this repository or create a branch of main_<Version>.
Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
[ ] Fix the bug and test your changes.
[ ] Add/update log messages for easier debugging.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into main_<Version>.
Pull request: bugfix <Issue Number> main_<Version> <Description>
[ ] Define the pull request metadata, as permissions allow.
Select: Reviewer(s) and Linked issues
Select: Organization level software support Project for the current coordinated release
Select: Milestone as the next bugfix version
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Complete the steps above to fix the bug on the develop branch.
Branch name:  bugfix_<Issue Number>_develop_<Description>
Pull request: bugfix <Issue Number> develop <Description>
Select: Reviewer(s) and Linked issues
Select: Repository level development cycle Project for the next official release
Select: Milestone as the next official version
[ ] Close this issue.

Focus on the line plot, the  code for generating the plots is identical in the box and bar plot.  There is a difference in behavior found in this code:
mask = np.array(all_filters).all(axis=0)
When run with Python 3.6, this code behaves as expected, but when using Python 3.8.12, the mask is different, and when used in self.series_data = self.input_data.loc[mask], the series_data series is full of NaNs.

this issue is related to a difference in behavior in the Python 3.6 vs Python 3.8 version of determining whether a pandas dataframe Series value is found in a list of values.  This is illustrated in the following simple code snippet to replicate the code in the line_series.py _create_series_points() method:
import sys
import pandas as pd
strlst = ['60000','120000', '180000']
dct = {'model':'HRRR', 'fcst_lead':[120000, 180000]}
df = pd.DataFrame(dct)
print(str(sys.version_info.major) + "." + str(sys.version_info.minor))
strlst = ['60000','120000', '180000']
dct = {'model':'HRRR', 'fcst_lead':[120000, 180000]}
df = pd.DataFrame(dct)
for match_val in df['fcst_lead'].isin(strlist):
print(match_val)
Output looks like this:
3.8
False
False
3.6
True
True
Which is why we are seeing NaN's in the subset dataframe after applying the mask mask = np.array(all_filters).all(axis=0), because the resulting Series of found fcst_leads is all False.
We will need to find a work-around that gives consistent results between Python 3.6 and Python 3.8.

In the _create_series_points method in the _series.py (plot=line, bar, box)
replaced original code:
all_filters.append((self.input_data[field].isin(filter_list)))
        # filter by provided indy
        all_filters.append((self.input_data[self.config.indy_var].isin(self.config.indy_vals)))

with:
all_filters.append((self.input_data[field].isin(filter_list)))
        # filter by provided indy

        # Duck typing is different in Python 3.6 and Python 3.8, for
        # Python 3.8 and above, explicitly type cast the self.input_data[self.config.indy_var]
        # Panda Series object to 'str' if the list of indy_vals are of str type.
        # This will ensure we are doing str to str comparisons.
        if isinstance(self.config.indy_vals[0], str):
            indy_var_series = self.input_data[self.config.indy_var].astype(str)
        else:
            # The Panda series is as it was originally coded.
            indy_var_series = self.input_data[self.config.indy_var]


Coding Challenge : Sudoku

How about creating a Sudoku game ? There is a lot of fun things to do :

checking if a grid is valid
highlight errors (same number on row/line/square)
show possible numbers in each cell
generate a random grid
create a solver

I have implemented most of those in my version (live version) and it was harder than I expected

Hello, Please vote in the Rainbow  Topics repository.
Relevant issues:
Generator: https://github.com/CodingTrain/Rainbow-Topics/issues/531
Solver: https://github.com/CodingTrain/Rainbow-Topics/issues/172


Failed to run post install script for guardrails/toxic_language

Nltk is one of many optional dependencies of guardrails.
@xinzaifeixiang1992, could you provide more details about your problem?


Greaseweazle PCB - BOM

Hi,
Please add to Greaseweazle PCB BOM list
with Regards
Pawel
PS. please describe diffrents gw-26 vs greaseweazle

I have also added a note on USBPU, as this must not be populated unless the Pill is modified. You would normally only do this is the BP pull-up resistor is incorrect value, which these days I think would be very rare.


Email adresses put in bold are broken in the rendering

Detected on https://symfony.com/doc/current/contributing/code_of_conduct/care_team.html which has the following content in the source:
Here are all the members of the CARE team (in alphabetic order). You can contact
any of them directly using the contact details below or you can also contact all
of them at once by emailing<EMAIL_ADDRESS>
The docs builder currently render this like that:

Here are all the members of the CARE team (in alphabetic order). You can contact
any of them directly using the contact details below or you can also contact all
of them at once by emailing <a<EMAIL_ADDRESS>class="reference internal">**care@symfony.com**:

the ** before the email is taken as part of the email reference, which breaks the mailto link (the link goes to the wrong email) and is probably also what breaks the bold rendering.

I think it's a duplicate of #128, which adds a failing test for this (but sadly not a solution).

indeed. I looked at issues but not at PRs.


Assist users to hide .env file from webroot

There should be warnings/suggestion on Readme page that .env file in webroot can be world-readable by default in many server configurations.
According ie. webserver logs or Cloudflare security logs, bots scanning .env files from websites are very used hacking attempt.
Screenshot from Cloudflare Firewall:

dotenv files should ideally not be deployed at all, and if they are, they should not be placed in the webroot directory,

Thanks Graham for clarifying this.
I strongly suggest to make that clear in README also. For me following arguments encourages to use  dotenv in production:

NO editing virtual hosts in Apache or Nginx
NO adding php_value flags to .htaccess files
EASY portability and sharing of required ENV values


⚠ Bofrost.at has degraded performance

In 9b42022, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1440 ms

Resolved: Bofrost.at performance has improved in 62200ec after 6 minutes.


Increase parity between dev and prod instances when triggering sync/link

Type of change

[ ] 🐛 Bug fix
[ ] 🌟 New feature
[ ] 🔨 Breaking change
[x] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[x] @clerk/clerk-js
[ ] @clerk/clerk-react
[ ] @clerk/nextjs
[ ] @clerk/remix
[ ] @clerk/types
[ ] @clerk/themes
[ ] @clerk/localizations
[ ] @clerk/clerk-expo
[x] @clerk/backend
[ ] @clerk/clerk-sdk-node
[ ] @clerk/shared
[ ] @clerk/fastify
[ ] @clerk/chrome-extension
[ ] gatsby-plugin-clerk
[ ] build/tooling/chore

Description
This PR

Deprecates and prepares the ground for the removal of __clerk_referrer_primary which ended up to be redundant
Triggers sync/link for development instances only if client uat is missing
Refactors the steps we take in order to load clerk-js properly for each flow

[x] npm test runs as expected.
[x] npm run build runs as expected.

This PR has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.


Frozen data models; Compiled templates

I suggest to allow frozen trees into a data model. Example:
function makeSubModel(val) {
	return Object.freeze({
		key1: Object.freeze({
			key2: val
		})
	});
}
var data = {
	frozenSubModel: makeSubModel('init')
}
new Vue({
	// …,
	data: data
});
data.frozenSubModel = makeSubModel('changed');

The current version rejects the frozen model: "Cannot define property:__ob__, object is not extensible." But frozen objects don't need to be observed.

I have several instances of Vue that share the same template. Is there a means to compile the template once?
My current code:
var myTpl = '
…
';
var vm1 = new Vue({
	// …,
	template: myTpl
});
var vm2 = new Vue({
	// …,
	template: myTpl
});

I would like to have:
var compiledTpl = Vue.compile('
…
');
var vm1 = new Vue({
	// …,
	template: compiledTpl
});
var vm2 = new Vue({
	// …,
	template: compiledTpl
});

+1 for both those suggestions. I'm going to hope that Vue.component already pre-compiles templates but I don't know for sure.

+1 for both those suggestions. I'm going to hope that Vue.component already pre-compiles templates but I don't know for sure.

Template compilations are cached as long as they are the same string.
The frozen object usage seems interesting, I can take a look after my vacation :)

Thanks.
(Just an informal question) Do you plan to integrate Format.js? I am considering contributing for this feature.

@tarh I personally have no plan in doing it, feel free to contribute!


node 8.1.0 upgrade issue

I've upgrade node version, from 8.0.0 to 8.1.0, when i want run vue cli (many projects), command line freeze on "Project description" step ,  but when downgraded node to 8.0.0 it's works!
windows 10
node 8.1.0
vue-cli 2.8.2

Yeah true happening after upgrade

Good idea to upgrade again, as something changed and it's working now @RezaHaidari

Experiencing the same issue with Node 8.1.0 on Linux Mint 18.2 (kernel 4.8)

Hmm @dzift , on osx brew version 8.1.0_1 is pushed that somehow fixed  the problem, don't know what's changed ~!

Still experiencing the problem on Arch. Any ideas for a workaround?

It's still going on.

For now we can just clone the template repo, edit package.json and npm install.

Node regression, see https://github.com/nodejs/node/pull/13560
Ought to be fixed with node 8.1.1, scheduled for June 13th 2017 https://github.com/nodejs/node/issues/13557#issuecomment-307565799

Thanks a lot for letting us know @elgrafico
Closing since it's a node bug. Hold on to node 8 until tomorrow 😄


Web3ValidatorError error when use sendSignedTransaction on HB blockchain

Description
As an blockchain developper, i'm actualy working on smart contract ERC-721 POC. I have a Hyperledger Besu blockchain working on my VM with 4 nodes, built with Qorum Quickstart (https://besu.hyperledger.org/stable/private-networks/tutorials/quickstart). This blockchain is configured for 0 fees.
I can develop, build, deploy, use my contract on Remix connected on my Hyperldeger Besu blockchain, but when i want to deploy the same smart contract from Visual Studio Code i have some mistakes with the signature of my transaction for deploy the contract.
Acceptance Criteria
Can build and sign my transaction for deploy the contract on my Hyperledger Besu blockchain.
Steps to Reproduce (Bug)

Launch Hyperledger Besu blockchain with ./resume.sh from the blockchain folder
Compile my smart contract and retrieve BIN, ABI and JSON files from bash script. Truffle dependencies on my VIsual Code had never worked, it's why i'm using this script.
Contract code on Test.sol :

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.17;

contract Test {
    uint public count;

    function increment() external {
        count += 1;
    }
}

command to launch the script :
./compile.sh contract.sol
bash script :
fileNameWithoutpath=$(basename $1)
#retrive file name without extension
fileNameWithoutPathAndExtension=${fileNameWithoutpath%.*}

echo "${bold}*************************************"
echo "Compile smart contract to BIN and ABI files from SOL files"
echo "*************************************${normal}"
solcjs --bin --abi $1 -o ../../build/contracts/
echo "fileNameWithoutPathAndExtension => $fileNameWithoutPathAndExtension"
solc $1 --combined-json abi,bin > ../../build/contracts/$fileNameWithoutPathAndExtension.json

for file in ../../build/contracts/*; do
    # Rename each file from this folder
    mv "$file" "../../build/contracts/${file##*_}"
done

Once my contract was built, i want to create transaction to deploy it on my Heperldger Besu Blockchain.

This is the genesis file of it :
{
    "config" : {
      "chainId" : 1337,
      "homesteadBlock": 0,
      "eip150Block": 0,
      "eip155Block": 0,
      "eip158Block": 0,
      "byzantiumBlock": 0,
      "constantinoplefixblock" : 0,
      "qbft": {
        "epochlength": 30000,
        "blockperiodseconds": 5,
        "requesttimeoutseconds": 10
      }
    },
    "nonce" : "0x0",
    "timestamp" : "0x58ee40ba",
    "gasLimit" : "0x1fffffffffffff",
    "difficulty" : "0x1",
    "number": "0x0",
    "gasUsed": "0x0",
    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "mixHash" : "0x63746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365",
    "extraData": "0xf87aa00000000000000000000000000000000000000000000000000000000000000000f8549493917cadbace5dfce132b991732c6cda9bcc5b8a9427a97c9aaf04f18f3014c32e036dd0ac76da5f1894ce412f988377e31f4d0ff12d74df73b51c42d0ca9498c1334496614aed49d2e81526d089f7264fed9cc080c0",
    "coinbase" : "0x0000000000000000000000000000000000000000",
    "alloc" : { DELETED FOR THE GITHUB ISSUE }
  }

I create my transaction, sign it and send it from the file "public_tx.js" called from this command : node public_tx.js
This is the content of the file :

const { FeeMarketEIP1559Transaction } = require("@ethereumjs/tx");
// const Common = require("@ethereumjs/common").default;
const { Common, Chain, Hardfork } = require("@ethereumjs/common");
const ethUtil = require("ethereumjs-util");
const {Web3} = require("web3");
const path = require("path");
const fs = require("fs-extra");
const web3 = new Web3("http://<IP_ADDRESS>:8545");
const to = "0x0000000000000000000000000000000000009999"

// Define the network parameters
const chainId = 1337; // The chain ID of your Besu network
const hardfork = "london"; // The hardfork name of your Besu network

async function publicTx() {
  const customCommon = Common.custom({
    chainId: chainId,
    hardfork: hardfork
  });
  const account = web3.eth.accounts.privateKeyToAccount(
    "0x8f2a55949038a9610f50fb24b5883af3b4ecb3c3bb792cbcefbd1542c692be64",
    true
  );

  const contractJsonPath = path.resolve(
    __dirname,
    "../../build/contracts/Test.json"
  );
  const contractJson = JSON.parse(fs.readFileSync(contractJsonPath));
  const contractAbi = contractJson.abi;
  const contractBinPath = path.resolve(
    __dirname,
    "../../build/contracts/Test.bin"
  );
  const contractBin = fs.readFileSync(contractBinPath);
  const contractConstructorInit =
    "000000000000000000000000000000000000000000000000000000000000002F";

  const txnCount = await web3.eth.getTransactionCount(account.address);

  const dataBuild = "0x" + contractBin + contractConstructorInit
  console.log('dataBuild ==> ',dataBuild)

  const rawTxOptions = {
    nonce: web3.utils.numberToHex(txnCount),
    maxPriorityFeePerGas: "0x0", // The max priority fee per gas in wei
    maxFeePerGas: "0x0", // The max fee per gas in wei
    gasLimit: "0x186A0", // The gas limit in wei
    to: "0x0000000000000000000000000000000000009999", // The address of the contract you want to deploy
    value: "0x00", // The value in wei
    data: dataBuild
  };

  console.log('account ==> ', account)
  const privateKeyBuff = Buffer.from(
    account.privateKey,
    'hex'
  );

  const tx = FeeMarketEIP1559Transaction.fromTxData(rawTxOptions, { customCommon });

  const signedTx = tx.sign(privateKeyBuff);
  var serializedTx = signedTx.serialize();
  console.log("serializedTx ==> " ,serializedTx);

  const pTx = await web3.eth.sendSignedTransaction(
    "0x" + serializedTx.toString("hex").toString("hex")
  , function(error, receipt) {
    if (!error) {
      console.log('receipt => ' ,receipt)
    } else {
      console.log('error => ' ,error)

    }
  });
  console.log("tx transactionHash: " + pTx.transactionHash);
  console.log("tx contractAddress: " + pTx.contractAddress);
}

publicTx();

result of console.log for dataBuild :
0x608060405234801561000f575f80fd5b506101468061001d5f395ff3fe608060405234801561000f575f80fd5b5060043610610034575f3560e01c806306661abd14610038578063d09de08a14610056575b5f80fd5b610040610060565b60405161004d9190610097565b60405180910390f35b61005e610065565b005b5f5481565b60015f8082825461007691906100dd565b92505081905550565b5f819050919050565b6100918161007f565b82525050565b5f6020820190506100aa5f830184610088565b92915050565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52601160045260245ffd5b5f6100e78261007f565b91506100f28361007f565b925082820190508082111561010a576101096100b0565b5b9291505056fea26469706673582212201c3c48dec99e9cf77132e4073590b6c55580f5c057b9a16678a26b3bd45b716c64736f6c63430008150033000000000000000000000000000000000000000000000000000000000000002F
result of console.log for account:
  address: '0xFE3B557E8Fb62b89F4916B721be55cEb828dBd73',
  privateKey: '0x8f2a55949038a9610f50fb23b5884af3b4ecb3c3bb792cbcefbd1542c692be64',
  signTransaction: [Function: signTransaction],
  sign: [Function: sign],
  encrypt: [Function: encrypt]
}

result of console.log for serializedTx:
Uint8Array(492) [
    2, 249,   1, 232,   1, 128, 128, 128, 131,   1, 134, 160,
  148,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
    0,   0,   0,   0,   0,   0,   0, 153, 153, 128, 185,   1,
  131,  96, 128,  96,  64,  82,  52, 128,  21,  97,   0,  15,
   87,  95, 128, 253,  91,  80,  97,   1,  70, 128,  97,   0,
   29,  95,  57,  95, 243, 254,  96, 128,  96,  64,  82,  52,
  128,  21,  97,   0,  15,  87,  95, 128, 253,  91,  80,  96,
    4,  54,  16,  97,   0,  52,  87,  95,  53,  96, 224,  28,
  128,  99,   6, 102,
  ... 392 more items

Just after these console.log i have a error :
/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-validator/lib/commonjs/validator.js:73
                throw new errors_js_1.Web3ValidatorError(errors);
                      ^
Web3ValidatorError: Web3 validator found 1 error[s]:
value "0x2,249,1,232,1,128,128,128,131,1,134,160,148,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,153,153,128,185,1,131,96,128,96,64,82,52,128,21,97,0,15,87,95,128,253,91,80,97,1,70,128,97,0,29,95,57,95,243,254,96,128,96,64,82,52,128,21,97,0,15,87,95,128,253,91,80,96,4,54,16,97,0,52,87,95,53,96,224,28,128,99,6,102,26,189,20,97,0,56,87,128,99,208,157,224,138,20,97,0,86,87,91,95,128,253,91,97,0,64,97,0,96,86,91,96,64,81,97,0,77,145,144,97,0,151,86,91,96,64,81,128,145,3,144,243,91,97,0,94,97,0,101,86,91,0,91,95,84,129,86,91,96,1,95,128,130,130,84,97,0,118,145,144,97,0,221,86,91,146,80,80,129,144,85,80,86,91,95,129,144,80,145,144,80,86,91,97,0,145,129,97,0,127,86,91,130,82,80,80,86,91,95,96,32,130,1,144,80,97,0,170,95,131,1,132,97,0,136,86,91,146,145,80,80,86,91,127,78,72,123,113,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,95,82,96,17,96,4,82,96,36,95,253,91,95,97,0,231,130,97,0,127,86,91,145,80,97,0,242,131,97,0,127,86,91,146,80,130,130,1,144,80,128,130,17,21,97,1,10,87,97,1,9,97,0,176,86,91,91,146,145,80,80,86,254,162,100,105,112,102,115,88,34,18,32,28,60,72,222,201,158,156,247,113,50,228,7,53,144,182,197,85,128,245,192,87,185,161,102,120,162,107,59,212,91,113,108,100,115,111,108,99,67,0,8,21,0,51,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,192,128,160,52,50,225,140,187,125,5,209,45,142,251,176,206,178,8,64,131,49,4,73,77,57,12,182,167,170,7,151,166,172,88,15,160,8,244,95,158,148,147,217,202,246,101,194,121,38,108,83,78,161,148,56,68,231,33,13,173,150,83,21,75,44,115,17,222" at "/0" must pass "bytes" validation
    at Validator.validate (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-validator/lib/commonjs/validator.js:73:23)
    at Web3Validator.validate (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-validator/lib/commonjs/web3_validator.js:35:32)
    at bytesToUint8Array (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/node_modules/web3-utils/lib/commonjs/converters.js:69:32)
    at hexToBytes (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/node_modules/web3-utils/lib/commonjs/converters.js:114:42)
    at Object.<anonymous> (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/rpc_method_wrappers.js:429:171)
    at Generator.next (<anonymous>)
    at /home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/rpc_method_wrappers.js:24:71
    at new Promise (<anonymous>)
    at __awaiter (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/rpc_method_wrappers.js:20:12)
    at /home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/rpc_method_wrappers.js:426:20 {
  innerError: undefined,
  code: 1100,
  errors: [
    {
      keyword: '0',
      instancePath: '/0',
      schemaPath: '#0',
      params: {
        value: '0x2,249,1,232,1,128,128,128,131,1,134,160,148,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,153,153,128,185,1,131,96,128,96,64,82,52,128,21,97,0,15,87,95,128,253,91,80,97,1,70,128,97,0,29,95,57,95,243,254,96,128,96,64,82,52,128,21,97,0,15,87,95,128,253,91,80,96,4,54,16,97,0,52,87,95,53,96,224,28,128,99,6,102,26,189,20,97,0,56,87,128,99,208,157,224,138,20,97,0,86,87,91,95,128,253,91,97,0,64,97,0,96,86,91,96,64,81,97,0,77,145,144,97,0,151,86,91,96,64,81,128,145,3,144,243,91,97,0,94,97,0,101,86,91,0,91,95,84,129,86,91,96,1,95,128,130,130,84,97,0,118,145,144,97,0,221,86,91,146,80,80,129,144,85,80,86,91,95,129,144,80,145,144,80,86,91,97,0,145,129,97,0,127,86,91,130,82,80,80,86,91,95,96,32,130,1,144,80,97,0,170,95,131,1,132,97,0,136,86,91,146,145,80,80,86,91,127,78,72,123,113,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,95,82,96,17,96,4,82,96,36,95,253,91,95,97,0,231,130,97,0,127,86,91,145,80,97,0,242,131,97,0,127,86,91,146,80,130,130,1,144,80,128,130,17,21,97,1,10,87,97,1,9,97,0,176,86,91,91,146,145,80,80,86,254,162,100,105,112,102,115,88,34,18,32,28,60,72,222,201,158,156,247,113,50,228,7,53,144,182,197,85,128,245,192,87,185,161,102,120,162,107,59,212,91,113,108,100,115,111,108,99,67,0,8,21,0,51,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,192,128,160,52,50,225,140,187,125,5,209,45,142,251,176,206,178,8,64,131,49,4,73,77,57,12,182,167,170,7,151,166,172,88,15,160,8,244,95,158,148,147,217,202,246,101,194,121,38,108,83,78,161,148,56,68,231,33,13,173,150,83,21,75,44,115,17,222'
      },
      message: 'value "0x2,249,1,232,1,128,128,128,131,1,134,160,148,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,153,153,128,185,1,131,96,128,96,64,82,52,128,21,97,0,15,87,95,128,253,91,80,97,1,70,128,97,0,29,95,57,95,243,254,96,128,96,64,82,52,128,21,97,0,15,87,95,128,253,91,80,96,4,54,16,97,0,52,87,95,53,96,224,28,128,99,6,102,26,189,20,97,0,56,87,128,99,208,157,224,138,20,97,0,86,87,91,95,128,253,91,97,0,64,97,0,96,86,91,96,64,81,97,0,77,145,144,97,0,151,86,91,96,64,81,128,145,3,144,243,91,97,0,94,97,0,101,86,91,0,91,95,84,129,86,91,96,1,95,128,130,130,84,97,0,118,145,144,97,0,221,86,91,146,80,80,129,144,85,80,86,91,95,129,144,80,145,144,80,86,91,97,0,145,129,97,0,127,86,91,130,82,80,80,86,91,95,96,32,130,1,144,80,97,0,170,95,131,1,132,97,0,136,86,91,146,145,80,80,86,91,127,78,72,123,113,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,95,82,96,17,96,4,82,96,36,95,253,91,95,97,0,231,130,97,0,127,86,91,145,80,97,0,242,131,97,0,127,86,91,146,80,130,130,1,144,80,128,130,17,21,97,1,10,87,97,1,9,97,0,176,86,91,91,146,145,80,80,86,254,162,100,105,112,102,115,88,34,18,32,28,60,72,222,201,158,156,247,113,50,228,7,53,144,182,197,85,128,245,192,87,185,161,102,120,162,107,59,212,91,113,108,100,115,111,108,99,67,0,8,21,0,51,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,192,128,160,52,50,225,140,187,125,5,209,45,142,251,176,206,178,8,64,131,49,4,73,77,57,12,182,167,170,7,151,166,172,88,15,160,8,244,95,158,148,147,217,202,246,101,194,121,38,108,83,78,161,148,56,68,231,33,13,173,150,83,21,75,44,115,17,222" at "/0" must pass "bytes" validation'
    }
  ]
}

I maked some research on google but i don't find any topic with this error.
Expected behavior: can deploy my contract
Actual behavior: error displayed on console when sendSignedTransaction is used
Frequency: Every time i use this function
Logs (if a bug)
Please post relevant logs from Besu (and the consensus client, if running proof of stake) from before and after the issue.
Versions (Add all that apply)

Software version: [besu --version] : using docker  image: hyperledger/besu:${BESU_VERSION:-latest}
Java version: [java -version]
OS Name & Version: [cat /etc/*release] : Ubuntu 22.04.1 LTS
Kernel Version: [uname -a] : Linux blockchain 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Virtual Machine software & version: [vmware -v]
Docker Version: [docker version] : 20.10.12
Cloud VM, type, size: [Amazon Web Services I3-large]
Consensus Client & Version if using Proof of Stake: [e.g. Teku, Lighthouse, Prysm, Nimbus, Lodestar] : using QBFT

Smart contract information (If you're reporting an issue arising from deploying or calling a smart contract, please supply related information)

Solidity version [solc --version] : Version: 0.8.21+commit.d9974bed.Linux.g++
Repo with minimal set of deployable/reproducible contract code - please provide a link : can't provide link with my companie safety rules
Please include specifics on how you are deploying/calling the contract : step 3
Have you reproduced the issue on other eth clients : smart contract can be build / compil / deploy and use on Remix connected to my blockchain with remixd

Additional Information (Add any of the following or anything else that may be relevant)

Besu setup info - genesis file, config options : Genesis provided on top of the issue
System info - memory, CPU : VM with 4 CPU / Go RAM / 150Go SSD

@louloub The latest fork in your genesis file is "constantinoplefixblock" : 0,
however the script uses FeeMarketEIP1559Transaction which requires the more recent `londonBlock" hard fork.
I would recommend upgrading your genesis to london or later.
Note, if you're using free gas with london, you need extra config:
https://besu.hyperledger.org/stable/private-networks/how-to/configure/free-gas#4-enable-zero-base-fee-if-using-london-fork-or-later

Also, are you using the latest version of quickstart? Because the latest version should already have this config built in https://github.com/Consensys/quorum-dev-quickstart/blob/master/files/besu/config/besu/QBFTgenesis.json#L14-L17

Hi @siladu thanks for your answers !
I modified my genesis.json file like this and remove / run the blockchain, and i have the same error.
genesis.json :
"config" : {
      "chainId": 1337,
	    "homesteadBlock": 0,
	    "eip150Block": 0,
	    "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
	    "eip155Block": 0,
	    "eip158Block": 0,
	    "byzantiumBlock": 0,
	    "constantinopleBlock": 0,
	    "petersburgBlock": 0,
	    "istanbulBlock": 0,
	    "muirglacierblock": 0,
	    "berlinBlock": 0,
	    "londonBlock": 0,
	    "zeroBaseFee": true,
    	    "contractSizeLimit":<PHONE_NUMBER>,
	    "qbft": {
	      "blockperiodseconds": 5,
	      "epochlength": 30000,
	      "requesttimeoutseconds": 10
	    }
    },
    "nonce" : "0x0",
    "timestamp" : "0x58ee40ba",
    "gasLimit" : "0x1fffffffffffff",
    "difficulty" : "0x1",
    "number": "0x0",
    "gasUsed": "0x0",
(....)

Hello !
I thought I had solved the problem with this line of code, but I have a new problem after this modification.
code line :
const result = Buffer.from(serializedTx).toString("hex");
const pTx = await web3.eth.sendSignedTransaction(
    result,
    function (error, receipt) {
      if (!error) {
        console.log("receipt ==> ", receipt);
      } else {
        console.log("error ==> ", error);
      }
    }
  );

new issue :
/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/utils/get_transaction_error.js:48
            error = new web3_errors_1.TransactionRevertInstructionError(_reason, undefined, transactionReceiptFormatted);
                    ^

TransactionRevertInstructionError: Transaction has been reverted by the EVM
    at /home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/utils/get_transaction_error.js:48:21
    at Generator.next (<anonymous>)
    at /home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/utils/get_transaction_error.js:24:71
    at new Promise (<anonymous>)
    at __awaiter (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/utils/get_transaction_error.js:20:12)
    at getTransactionError (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/utils/get_transaction_error.js:33:12)
    at Object.<anonymous> (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/rpc_method_wrappers.js:441:100)
    at Generator.next (<anonymous>)
    at fulfilled (/home/adminlogin/BESU_NETWORK/v5/contracts/EOBLOCK v2/node_modules/web3-eth/lib/commonjs/rpc_method_wrappers.js:21:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  innerError: undefined,
  reason: 'Internal error',
  signature: undefined,
  receipt: undefined,
  data: undefined,
  code: 402
}

Does i create new issue on github ?

Hello @louloub
Keep the thread here for now. I haven't quite figured out what's going on with web3, but have been ok deploying contracts with ethers.js
Something like this below works fine on Besu on the quickstart, hopefully will unblock you
const path = require('path');
const fs = require('fs-extra');
var ethers = require('ethers');

// RPCNODE details
const { tessera, besu } = require("../keys.js");
const host = besu.rpcnode.url;
const accountPrivateKey = besu.rpcnode.accountPrivateKey;

// abi and bytecode generated from simplestorage.sol:
// > solcjs --bin --abi simplestorage.sol
const contractJsonPath = path.resolve(__dirname, '../../','contracts','Test.json');
const contractJson = JSON.parse(fs.readFileSync(contractJsonPath));
const contractAbi = contractJson.abi;
const contractBytecode = contractJson.evm.bytecode.object

async function main(){
  const provider = new ethers.providers.JsonRpcProvider(host);
  const wallet = new ethers.Wallet(accountPrivateKey, provider);
  const nonce = await provider.getTransactionCount(wallet.address);
  const feeData = await provider.getFeeData();
  const gasLimit = await provider.estimateGas({from: wallet.address, value: ethers.utils.parseUnits("0.01", "ether")});
  var txn = {
    chainId: 1337,
    type: 2,
    nonce: nonce,
    to: "0x0000000000000000000000000000000000009999",
    value: ethers.utils.parseEther("0.01"),
    data: '0x'+contractBytecode,
    gasLimit: gasLimit,
    maxPriorityFeePerGas: feeData["maxPriorityFeePerGas"],
    maxFeePerGas: feeData["maxFeePerGas"],
  };
  const signedTx = await wallet.signTransaction(txn);
  provider.sendTransaction(signedTx).then(console.log);
}

if (require.main === module) {
  main();
}

module.exports = exports = main

Cheers
Josh

Hi @joshuafernandes !
Thanks for your time :)
I followed your code and modify some code line for finally deploy correctly my contract with ethers.js and can use one function ! (now i have another issue for using call function, i will create new issue i think)
This is my code :
const { ethers, JsonRpcProvider } = require("ethers");
const fs = require("fs");
const solFile = fs.readFileSync("../Test.sol", "utf8");
const solc = require("solc");

const provider = new JsonRpcProvider("http://localhost:8545");
const privateKey =
  "0x0ba970ff5e9b9c4ce2435fdf94a998af0b5892c4f6f7cb9c3fcb0772ae98ab54";
const wallet = new ethers.Wallet(privateKey, provider);
const receiver = "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73";

buildContract();

async function buildContract() {
  const input = {
    language: "Solidity",
    sources: {
      "Test.sol": {
        content: solFile,
      },
    },
    settings: {
      outputSelection: {
        "*": {
          "*": ["*"],
        },
      },
    },
  };
  const output = JSON.parse(solc.compile(JSON.stringify(input)));
  const bytecode = output.contracts["Test.sol"]["Test"].evm.bytecode.object;
  const abi = output.contracts["Test.sol"]["Test"].abi;

  const factory3 = new ethers.ContractFactory(abi, bytecode, wallet);
  const nonce = await provider.getTransactionCount(wallet)
  const contractDeployed = await factory3.deploy({
    gasLimit: "0x4e85c",
    gasPrice: "0x0",
    nonce: nonce
  });

  console.log("contractDeployed ==> ", contractDeployed);

  const contractDeployedAddress = await contractDeployed.getAddress();
  console.log("contractDeployedAddress ==> ", contractDeployedAddress);

  const sampleContract = new ethers.Contract(
    contractDeployedAddress,
    abi,
    wallet
  );
  const tx = await sampleContract.increment({
    gasLimit: "0x4e85c",
    gasPrice: "0x0",
  });

  console.log("tx ==> " ,tx);

Hi @louloub
Really good to hear :) I've added in some examples based on this specifically (EIP-1559) in the quickstart yesterday and got it to deploy & read + write https://github.com/Consensys/quorum-dev-quickstart/blob/master/files/besu/smart_contracts/scripts/public/hre_1559_public_tx.js
Have also stuck to the ethers type of calls and contractfactory (I find this abstractions nicer to work with). Let me know if this works for your token as well and feel free to submit a PR if its lacking functionality
Cheers
Josh

Hi @joshuafernandes
Thanks for your answer ! I just updated my previous comment because i forgot to preciss that i'm back to NO EIP1559 blockchain version.  Actualy i can use Ethers.js to deploy contract from .sol files and use "increment" function . But i have issue for simple "call" function. I'm working on it and i will create new issue for this :)

Hi @louloub
Sounds good! This one should work in the future if you need one that is EIP-1559 https://github.com/Consensys/quorum-dev-quickstart/blob/master/files/besu/smart_contracts/scripts/public/hre_1559_public_tx.js#L69 .
For a normal one this should get you going https://github.com/Consensys/quorum-dev-quickstart/blob/master/files/besu/smart_contracts/scripts/public/hre_public_tx.js#L53 I have a get/set operation after deployment. Let me know if that works, else create a new issue
Cheers

Thanks @joshuafernandes and @louloub for your comments - I think this issue can be resolved now. If not, feel free to reopen.
@louloub if the new issue is related to the quickstart scripts rather than Besu itself, then https://github.com/Consensys/quorum-dev-quickstart might be a better place for it

Hi @joshuafernandes !
Back with new informations :)
On my previous message i say that i can deploy and use increment function but can't use call function. But after lot of test i can say that my contract was not deployed correctly. I don't understand how it's possible that i can use increment function...
When i add on this code the line  after the contract factory line
const test = await contractDeployed.waitForDeployment();
console.log(test)

This error was triggered :
Error: transaction execution reverted (action="sendTransaction", data=null, reason=null, invocation=null, revert=null, transaction={ "data": "", "from": "0xd48ab4450681d37CbfbDcf21e0b1eE9405045A9F", "to": null }, receipt={ "_type": "TransactionReceipt", "blockHash": "0xf2f71da4fc4c820604b82eb23ab6d9dc5ab43a4ef45685b4db38e934eab40e53", "blockNumber": 128208, "contractAddress": "0xc6aB7ED563D09a3CE72d4e812b8DDc004e9661fA", "cumulativeGasUsed": "321628", "from": "0xd48ab4450681d37CbfbDcf21e0b1eE9405045A9F", "gasPrice": "0", "gasUsed": "321628", "hash": "0x429a6967d601a2b6c57b3366a86c62777b5d40088327e2efb3f2a2f80ebc42c2", "index": 0, "logs": [  ], "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "root": null, "status": 0, "to": null }, code=CALL_EXCEPTION, version=6.7.1)

We can see on the TransactionReceipt the status 0 for reverted transaction
So now the question is why my contract can be deployed.
Do you have any ideas ?
Thanks for your time


feat: added an Anime Template Portfolio

Hello @coder-ralph has this been updated? Please mark the resolved comments as done if they are already addressed. Thank you <3

I'm sorry, Sir, I forgot to click the "resolve conversation" button, but I want to inform you that all the comments have been resolved. The pull request is now ready for review/merge. Thank you Sir for your feedback, and please let me know if there's anything else I can improve.

I'm sorry, Sir, I forgot to click the "resolve conversation" button, but I want to inform you that all the comments have been resolved. The pull request is now ready for review/merge. Thank you Sir for your feedback, and please let me know if there's anything else I can improve.

Thank you, no problem @coder-ralph will review it as soon as I can thanks

Closing and reopening the PR to re-run the Github workflow

Reopening


One page query

Paging at SQL databases is almost like this:
var count = await source.CountAsync();
var items = await source.Skip((pageIndex - 1) * pageSize).Take(pageSize).ToListAsync();

Both statements result in database queries. Depending on the query (including JOINs, WHEREs, ...) both database queries may last very long.
What if EntityFramework Core introduces a PaginatedList<T> construct which will be populated by a single database query?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 15e2038b-fa1b-897c-32ee-7173c7d1d909
Version Independent ID: 000c5a97-c1b2-9a48-289d-83ec4f0c0c98
Content: Tutorial: Add sorting, filtering, and paging - ASP.NET MVC with EF Core
Content Source: aspnetcore/data/ef-mvc/sort-filter-page.md
Product: aspnet-core
Technology: aspnetcore-data
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

On SQL Server I reduced the execution time from 3+8 seconds (COUNT and SELECT ... statements) to 7 seconds using common table expression:
;WITH Query AS (
SELECT *
FROM (
	SELECT
	    p.Id,
		p.IsActive,
		v.Title,
		v.Body,
		...

  )
)

SELECT *
FROM Query
CROSS JOIN (SELECT COUNT(*) AS Total FROM Query) x
ORDER BY [Id]
OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY

@tdykstra would should consider cutting sorting, filtering, and paging -
That's pure EF and not a core area for ASP.NET Core to document.
@aureole82  we're a doc team for ASP.NET Core. You'd probably have better luck at StackOverflow.


ServerError struct does not have Temporary method

When attempting to assert that the errors.Cause(err) of a returned error from a service is Temporary for the new ratelimiting requirements:
type temporary interface {
        Temporary() bool
}

func IsTemporary(err error) bool {
        te, ok := errors.Cause(err).(temporary)
        fmt.Printf("IS TEMPORARY %#v %t\n", errors.Cause(err), ok)
        return ok && te.Temporary()
}

This prints:
IS TEMPORARY igdb.ServerError{Status:429, Msg:"too many requests", Temp:true} false

errors.Cause() returns a plain igdb.ServerError, not *igdb.ServerError, and a plain igdb.ServerError does not implement temporary. Only *igdb.ServerError has the required Temporary method:
https://github.com/Henry-Sarabia/igdb/blob/dcac61d5a6ea36eb1265af5e5f0cfdf479adf14f/error.go#L72
Is there any reason we can't define this method on igdb.ServerError as that will cover both plain struct and pointer?

Looking at it now, there is no reason the method should be defined on the pointer.
Good looking out!
Temporary will be igdb.ServerError's method in the next patch.

Fantastic! Thanks for the update. 👍


chore: add move assign operator for MultiLockGuard

Besides, after move a vector will be in a unspecific state, according to the standard. We need to use clear() to make the state specific, again.

@Sandra-Amina-Boss Please stop such meaningless reviews and sending spam notifications to me.


Create UBQ bonding curve to burn UBQ and mint NFTs

Create the UBQ bonding curve smart contract (fork)
Update UI component
Implement the logic "Buy with any token"

Get the tokens that the user has
Exchange to UBQ
Send UBQ to bonding curve contract (effectively burning it)
Mint UbiquiStick NFT
Transfer the UbiquiStick NFT to the buyer

@codeawayaks We offer solving bounties one at a time to minimize blocking other developers to move things forward. Which one issue you want to work on first?

@sergfeldman I am very interested about this task.
But I need your explain a bit more.

@sergfeldman I am very interested about this task. But I need your explain a bit more.

@pufferdev0807 Please, post your clarifying questions here.
We will make a decision on assigning you after you ask questions.

hello @sergfeldman I am very familiar with bonding curve (bancor formula) as you can see here https://github.com/meTokens/meTokens-core

I get what you want to do here :smile:
let me know if I can work on it

@zgorizzo69 Already did most of the mathematical modeling for this just need to refine a few parameters https://www.desmos.com/calculator/o311zdypy1, we can hop on a call for more details if the decision is to have you working on this

Sure ! Let's wait for @sergfeldman @pavlovcik decision then

hello @sergfeldman I am very familiar with bonding curve (bancor formula) as you can see here https://github.com/meTokens/meTokens-core and I get what you want to do here 😄 let me know if I can work on it

That's dope but I feel like this particular bounty is quite low in our priority list. Given your experience with this project and Solidity expertise, I'd rather have you focus on the more important issues you have also expressed interest in.

Note for the bounty hunters.
This issue should be divided to smaller tasks with the specific deliverables.
For example,

Update the UI component.
Implement the logic "Buy with any token".

Please, suggest smaller tasks when apply for this bounty.

This issue should be divided to smaller tasks with the specific deliverables. For example,

Update the UI component.
Implement the logic "Buy with any token".

Please, suggest smaller tasks when apply for this bounty.

I encourage you to break up the issue into separate issues. It should be on us to do that.


Implementing Segment Tree

I would like to work on implementing segment tree in javascript 😇

You are welcome.

Please write the tests for that

Ok ✌


Multiple Single Frame acquisitions require StopAcquisition

Describe the bug
The Genicam SFNC document mentions that executing the StopAcquisition() command is optional in Single Frame acquisition Mode. See following image from genicam SFNC page 165:

However when I try to run multiple singe frame acquisitions without a StopAcquisition(), I get an error.
To Reproduce
Execute the following snippet:
import os
from pathlib import Path
from importlib.util import find_spec

genicam_path = Path(find_spec("genicam").origin).parent
os.environ["PATH"] += os.pathsep + str(genicam_path)

from harvesters.core import Harvester

h = Harvester()

h.add_file(str(genicam_path / 'TLSimu.cti'))

h.update()

ia = h.create_image_acquirer(0)

ia.remote_device.node_map.get_node('AcquisitionMode').value = 'SingleFrame'

ia.start_acquisition() # Success

with ia.fetch_buffer() as buffer:
    pass # just return the buffer to queue

ia.start_acquisition() # fail: `GenTL exception: Operation cannot be performed. Responsible module is busy. (Message from the source: The acquisition process is running now.) (ID: -1022)`

Expected behavior
The second ia.start_acquisition() should not throw an error
Desktop (please complete the following information):

OS: Windows 10
Python: 3.7
Harvester: harvesters==1.2.0
genicam==1.0.0
GenTL Producer: TLSimu
Camera: TLSimu

Additional context
Could also very well be a misunderstanding by me of the genicam spec.

@StefanD986 Thank you for the comment. Could you give me some time to think about that? It could be a bit delicate issue for Harvester. /Kazunari

@StefanD986 Hi, sorry having kept you waiting for such long. It's a very trivial question but you mean you would like to use the start_acquisition method to acquire a single image? And does it mean you want an `ImageAcquire

I think the short answer is:

you would like to use the start_acquisition method to acquire a single image?

yes

you want an ImageAcquire object to keep holding the acquired image until the next star_acquisition call?

No, I only expect it to hold it until leaving the with ia.fetch_buffer() as buffer: construct. After that I expect the buffer would be returned to genicam so that it can be re-used.
Long answer would be: This is how I understand the Single Frame acquisition of genicam:

ImageAcquirer allocates multiple image buffers, which are then put in a queue for buffers that are free to use.
If I say start_acquisition, the camera will acquire a frame, one buffer is removed from the above mentioned queue, the image is saved in there. Then the (filled) buffer is put into another queue for buffers filled with image data.
With ia.fetch_buffer one buffer is pop'ed from the buffers filled with image data-queue
I do whatever needs to be done with the picture
If I am done with the picture, the buffer is returned to the buffers that are free to use-Queue (this is done automatically through the ia.fetch_buffer context manager)
The next start_acquire would repeat the process from 2.
Calling stop_acquisition would de-allocate all buffers

Essentially when in SingleFrame acquisition mode, I want to avoid to do start_acquisition do-something-with-the-image stop_acquisition repeatedly, because each start_acquisition needs to allocate and each stop_acquisition de-allocate the buffers each time. And the genicam standard seems to allow that.
Hope that clarifies it a bit?

Not all manufacturers support or require the same settings for using an external trigger, you may want to look for a manual specific to your camera instead of generic Genicam specs. As I recently commented on the closed issue 77 (repeating here since a closed issue may not get so much attention): I have a camera where repeated acquisitions by external trigger need to be done by using AcquisitionMode='SingleFrame', calling my patched
ia.start_acquisition(force_unlimited_count=True)
only once and ia.stop_acquisition() only once when I have as many frames as I want. This works also at high frame rates (tested 1 kHz). This camera has no 'MultiFrame' mode, which seems from Genicam docs to have been more expected/typical.
So if your camera behaves in the same way, the patch I attach may be a way to achieve what you want without needing to call start_acquisition() again, just
fetch_buffer(timeout=more_than_expected_trigger_separation_in_seconds)
again as many times as you want. I also configured
ia.timeout_for_image_acquisition = more_than_expected_trigger_separation_in_milliseconds
(its default is 1 ms) but I didn't test if that was really needed.
I suspect that your approach of wanting to call start_acquisition() before every new frame could be a bit slower if made possible, since I guess it would "re-arm" (validate all settings again, maybe reallocate buffers too).

@StefanD986 Hi. Thank you for your clarification and it helped me to clearly understand your assertion in detail. One thing I can say now is it's just an issue about the design of Harvester and I would like to cover the usage that you expect in a natural way as much as possible. I know this is just an awkward excuse but Harvester is one of my purely private projects and it's done in my spare time that I can have when my family allows. It's been a while since you opened this ticket but I hope you can keep waiting a bit more. Could you? I hope you do not get disappointed with Harvester. /Kazunari

@kazunarikudo yes no problem. Its not a blocking problem for me at the moment. As a workaround I am now doing a software-triggered multiframe acquisition, and I am simply using the software trigger to acquire a single frame.

@erik-mansson thanks a lot for the tip and the patch. Luckily the camera I need to support at the moment have multiframe acquisition, so I can use my workaround. But I also have another camera, which I have not tested yet, so you patch may still come in handy.

I suspect that your approach of wanting to call start_acquisition() before every new frame could be a bit slower if made possible, since I guess it would "re-arm" (validate all settings again, maybe reallocate buffers too).

If I understand the genicam standard correctly, re-allocating buffers should not be necessary I deduct that from GenTL standard section 5.2 Acquisition chain, where the allocation and announcing of the buffers is described before Start Acquisition. (Its a bit lengthy so I won't copy it here).
Re-validating settings should also not be necessary, if I understand the section about the AcquisitionStart command (section 5.5.3 in the GenICam SFNC) correctly:

AcquisitionStart command must validate all the current features for consistency before starting the Acquisition. This validation will not be repeated for the subsequent acquisitions unless a feature is changed in the device.

But yeah could also very well be that I am misunderstanding it or practice differs from theory.

@StefanD986, good that things are OK for you.
I saw the GenICam documentation about AcquisitionStart yesterday, and I remember it saying that somthing like Arm or ReArm exists too. If ReArm is done, I think it kind of forces a full initialization. I was just fearing that Harvesters might be causing a ReArm or maybe setting some parameter that causes the "feature change in device", or guessing that there are buffers allocated also on the Harvesters level when ImageAcquirer.start_acquisition() is called. So the allocation/performance part of my comment is just guesswork and apparently you are well aware of such risks.

@StefanD986 Hi, Stefan. I have just committed a change for your use case to _issue_150 branch. Could you try that out when you can and let me know if there's something block you, please? I would appreciate it if you could try the development version with the possible AcquisitionMode values on camera models that you have. /Kazunari

Hi, thank you for the fix. Unfortunately I do not have the camera available all the time (its a shared asset and its currently in use). So it might take a couple of weeks before I can try out the fix with a physical camera.
How shall we proceed? Do you want to wait with the merge until I can test it with the physical camera? (This would be fine for me, but I would understand if you want to merge it ASAP to prevent the branches diverging too much)

@StefanD986 Hi. Thank you for your reply. It's not urget so I'll wait for your feedback. I see a value in the suggested usage so lookeing forward to hearing from you again. Thank you. /Kazunari

@StefanD986 Hi. This is just a friendly reminder. I'm awaiting your feedback. ;-)

@StefanD986 How's it going? I'd appreciate if you could give me your feedback. Thank you!

Hi, I'm sorry I still need some more time :( . The camera feature is on our roadmap for this year, so I will definitely come back to it, however there are some other things planned in the meantime.

@kazunarikudo Hi, I can finally confirm that this PR fixes the issue! 🎉
Apologies that it took so long. The camera that I needed for testing is a scarce resource in our company. Thank you for your patience!

@StefanD986 Thank you for your feedback!


Update to API version v1

@liggitt @jwforres

looks pretty good to me. before merging this, create a branch from master for v1beta3. when cutting a release for this, increment a major version to 1.0.0

@liggitt I was looking at pkg/api/graph/test/bare-rc.yaml, which has a service account, but misread it... :(

I've tested this in Cockpit roughly and it works. We updated our core code to the v1 API a few days ago and the object describer was one of the things that is remaining. CC @sub-mod

LGTM


Regarding fine-tuning on new action spaces

Hi,
Thanks for sharing this exciting work. I want to clarify the steps to fine-tune pre-trained open-vla on a different action space like discrete keyboard inputs (e.g., tfds.features.ClassLabel type in action config for the dataset). To achieve that, could you explain the steps (e.g., config updates or the proper way to add different action encoders)?
To note,  I have already followed the steps to export my custom dataset to the x-embodiment format using the rlds dataset builder README, so I am mainly talking here about the follow-up steps to fine-tune on the exported dataset assuming a different action space.
I added a new ActionEncoding enum value named KB (1 dim) and adjusted the materialize.py to stop action normalization as follows:
elif dataset_kwargs["action_encoding"] is ActionEncoding.KB:
          dataset_kwargs["absolute_action_mask"] = [True]
          dataset_kwargs["action_normalization_mask"] = [False]

This gives an error when running the finetune.py on the custom data (exported to RLDS format) as the dataset loader loads the 1-d action as None (I loaded the exported dataset using tfds.builder_from_directory and inspected an episode steps, the class label integer value is present as per the tfds.features.ClassLabel feature config) :
Inside openvla/prismatic/vla/datasets/rlds/dataset.py:
if absolute_action_mask is not None:
            if len(absolute_action_mask) != traj["action"].shape[-1]:
                raise ValueError(
                    f"Length of absolute_action_mask ({len(absolute_action_mask)}) "
                    f"does not match action dimension ({traj['action'].shape[-1]})."
                )

Exception:
File "openvla/prismatic/vla/datasets/rlds/dataset.py", line 194, in restructure  *
        raise ValueError(
    ValueError: Length of absolute_action_mask (1) does not match action dimension (None).

I solved the issue by changing the action feature type to be tf.features.Tensor with shape (1,) and type float, changing the encoder masking to assume 1 dim masking instead of the default 8 space one and adjusting the ActionTokenizer class to count for the fact that my action is already discrete.