Are sounds implemented yet?

HI,
anything I am missing to get sounds to work?
here's my init:
Push.init({

            bagde: true,
            sound: true,
            alert: true
        });

BTW, they seem to work fine  on iOS if I am using one with push simulators.

Hi, still having an issue with sounds... Do I need to put my own sound file somewhere in the project?

@adamgins try the 2.3.3 - I've updated the path to the sound file. It should be placed in the meteor public folder.
Meteor puts the app in sub folder application so its updated now - but I'm not 100% sure it will work, meteor actually runs a small server on the device so we are running at http://meteor.local
If you cant get sound to work we might have to try and use the server prefix instead.

Hello @adamgins Did you get the sound to work?
I tried quite a few combinations including "sound" : "somefile.aiff".
I merely want the default native alert tone to play when the push is delivered to the device. Just "sound": true does not work.

yep, I have same issue. It works fine on Android but cannot get sound to work on iOS. Been flatout on other stuff to come back and look at it. @raix wondering if it works for you?

Havent tested this part fully yet

@insaneinc01, @adamgins, This may be a long shot, but does this work:
sound: 'default'

if in call, or:
"sound": "default"

if in config.

It's been a while since I have looked at this issue. Anyone get sounds working on iOS yet?

@hellogerard @adamgins
No luck yet. Tried
sound: 'default'
both in the call and config, does not work.
I'm trying only on iOS though.

@hellogerard @adamgins
Push.send({
         from: 'Test',
         title: 'Test',
         text:msgText,
         sound: '/audio/mySound.mp3',
         badge: 1,
         query: { }
      });

works fine for me after adding org.apache.cordova.media plugin

@maabed sounds cool - If you guys can confirm that adding the cordova.media plugin solves this we should prop. consider making the media plugin a dependency?

@maabed tried your suggestion, but it still doesn't work :( I did add org.apache.cordova.media.
I have placed the mySound.mp3 file in the Public folder. Sorry I might be missing something very obvious here.

ok! I finally got the sounds working. Just the default native iOS push sound, not customized sounds.
config.push.json never worked for me. It always gets stuck on the Meteor splash screen. So I used @spencercarli suggestion here: https://github.com/raix/push/issues/21#issuecomment-69647337
if (Meteor.isServer) {
  Meteor.startup(function () {
    Push.debug = true;

    Push.Configure({
      apn: {
        passphrase: 'test1011',
        certData: Assets.getText('PushChatCert.pem'),
        keyData: Assets.getText('PushChatKey.pem'),
        gateway: 'gateway.sandbox.push.apple.com'
      },
      production: false
    });
  });
}

if (Meteor.isClient) {
  Meteor.startup(function() {
    Push.Configure({
      "sound": true
    });
    Push.debug = true;
  })
}

And in push.send call:
Push.send({from: 'push',title: 'Congratulations',text: 'You can now Push to this device!', sound: 'blahblah', query: {}})

Note the value - sound: 'someString'.
Thanks @maabed

regarding:

config.push.json never worked for me. It always gets stuck on the Meteor splash screen

Have you tried using $ meteor --production?
I've filed an issue on this but they want a replication - I'm not sure I can

Hi @raix ,
--production works! Thanks for that!
Btw, since my requirement is only to trigger the default Push sound on iOS, the org.apache.cordova.media plugin is really not needed.

@insaneinc01 super :)
I actually thought the sound was HTML5 why the media plugin is a bit of a surprise.

Hi folks, great stuff... just catching up here. So, am I reading correctly:

we don't actually need the plugin?
We just need to add the:

 Push.Configure({
      "sound": true
    });

to the client?
3) sorry a bit confused what's in the sound:<string>? is it just 'default'?
thanks

Ah by the way, if I add the Push.Configure above, I get: Uncaught Error: Push.Configure should not be called more than once!
I am only calling it once in the client code. But I am using the config.push.json

The config.push.json creates the specific Push.Configure in each environment - so you should only use one of the config methods - The sound should be set true in the config and in the push send you reference the sound file as string

thanks… I was just trying that … but seemed to be getting an issue after installing Cordova media plugin.. do I actually need it?
Adam Ginsburg
<EMAIL_ADDRESS>tel: +61 411 151 732
http://buzzy.buzz
On 9 Feb 2015, at 5:59 pm, Morten N.O. Nørgaard Henriksen<EMAIL_ADDRESS>wrote:

The config.push.json creates the specific Push.Configure in each environment - so you should only use one of the config methods - The sound should be set true in the config and in the push send you reference the sound file as string
—
Reply to this email directly or view it on GitHub.

I dont think the media is required - but I'm not sure - playing sound on ios is triggy

@adamgins The Cordova media plugin is not needed if you only need to trigger the default native push sound on iOS.
In the Push.Send call, the "sound": "default" or "sound": "yes" is needed. I guess any string value here will trigger the default push sound.
Push.send({from: 'push',title: 'Congratulations',text: 'You can now Push to this device!', sound: 'blahblah', query: {}})

Thanks @insaneinc01 ok sound playing now while app is open only. Is there something else I need to do so it plays if the app is closed?

Hi, this started working for whilst the app was closed... not sure if it was an udpate...anyway, closing this. Thanks for all the help folks.

Followed all the ideas and examples shown here to do a custom sound. However, I am still not getting the custom sounds to play. I either get the default notification sound, or no sound at all. My config.push.json looks like this:

I was only able to get vibrations and sound working in iOS after adding 'sound':'default' in the Push.send method. Unfortunately, I've had no success in using custom sounds.
I've tried all three of: 'www/application/{public,app,nothing}/sound.mp3' with no success.
I've even installed cordova-plugin-media, which I had to use the older commit for because of a cordova-android version > 5.0.0-dev requirement of the latest branch. Unfortunately, this package did nothing. I am going to try listing it as a dependency to this package.
At this point, I can get a default sound and vibrate on iOS, only a vibrate on Android. Still trying to figure out how to get sound working on android. I changed settings on my phone to allow sounds, but still no luck.
Will keep posted.

I have tried for weeks trying to get a custom sound to play on android
using raix push with no success. I have used the methods you described as
well as many others with no success. I'll post if I figure it out. Great
package! If custom sounds would work it would be perfect.
On Nov 16, 2015 12:16 AM, "Streemo"<EMAIL_ADDRESS>wrote:

I was only able to get vibrations and sound working in iOS after adding
'sound':'default' in the Push.send method. Unfortunately, I've had no
success in using custom sounds.
I've tried all three of: 'www/application/{public,app,nothing}/sound.mp3'
with no success.
I've even installed cordova-plugin-media, which I had to use the older
commit for because of a cordova-android version > 5.0.0-dev requirement of
the latest branch. Unfortunately, this package did nothing. I am going to
try listing it as a dependency to this package.
At this point, I can get a default sound and vibrate on iOS, only a
vibrate on Android. Still trying to figure out how to get sound working on
android. I changed settings on my phone to allow sounds, but still no luck.
Will keep posted.
—
Reply to this email directly or view it on GitHub
https://github.com/raix/push/issues/9#issuecomment-156944113.

I haven't tested sound yet since shifted to the new cordova plugin - please let me know if you find a bug
(I do take pr's :) )

Solved! This may be a work around, but it does work and it works consistently. None of the above methods worked for me, however the following does work.
You will want to add a new folder under the following path in your project > .meteor/local/cordova-build/platforms/android/res/  , add a folder called raw . In the raw folder add your sounds.
The resulting folder is the exact location that the sounds are being referenced from in android. To see for yourself, you can run your app on a device and log the errors. Do the following:

Build your application to your phone
in the terminal type: adb logcat >> emulator.log
Send yourself a push notification
Check the log. Notice the following error in the log:
Failed to open file 'android.resource://com.yourappname/res/raw/filename

--- Indeed the app is trying to reference  a file that does not exist. Thats why creating a new directory called raw and placing the file inside it does the job.
*** PLEASE NOTE! IN THE PUSH NOTIFICATION - I ALSO REMOVED THE FILE EXTENSION ***.mp3
Push.send({
from: 'push',
title: title,
text: usernameFrom + ' ' + message,
badge: badge,
query: {
userId: userIdTo
},
sound: 'soundNameWithoutExtension'
});
I hope this helps.
Matt

I'll try this out and let you know if I can replicate the success. It's a
bummer that this kind of stuff is not normalized.
On Nov 24, 2015 2:11 PM, "mvgalle"<EMAIL_ADDRESS>wrote:

Solved! This may be a work around, but it does work and it works
consistently. None of the above methods worked for me, however the
following does work.
You will want to add a new folder under the following path in your project

.meteor/local/cordova-build/platforms/android/res/ , add a folder called
raw . In the raw folder add your sounds.

The resulting folder is the exact location that the sounds are being
referenced from in android. To see for yourself, you can run your app on a
device and log the errors. Do the following:

Build your application to your phone
in the terminal type: adb logcat >> emulator.log
Send yourself a push notification
Check the log. Notice the following error in the log:
Failed to open file 'android.resource://com.yourappname/res/raw/filename

--- Indeed the app is trying to reference a file that does not exist.
Thats why creating a new directory called raw and placing the file inside
it does the job.
*** PLEASE NOTE! IN THE PUSH NOTIFICATION - I ALSO REMOVED THE FILE
EXTENSION ***.mp3
Push.send({
from: 'push',
title: title,
text: usernameFrom + ' ' + message,
badge: badge,
query: {
userId: userIdTo
},
sound: 'soundNameWithoutExtension'
});
I hope this helps.
Matt
—
Reply to this email directly or view it on GitHub
https://github.com/raix/push/issues/9#issuecomment-159422348.

@mvgalle brilliant solution!
Here's where it's documented in the cordova plugin

@mvgalle it's works! thanks

i can't get it working to play the default sound on IOS.
in client/main.js i have:
if (Meteor.isClient) {
Meteor.startup(function() {
Push.Configure({
"sound": true
});
Push.debug = true;
})
}
in config.push.json i have:
{
"apn-dev": {
"passphrase": "xxxxxxxxxx",
"key": "key.pem",
"cert": "cert.pem",
"gateway": "gateway.sandbox.push.apple.com"
},
"production": false,
"badge": true,
"sound": true,
"alert": true,
"vibrate": true
}
the alert is working and badge too but the sound is not working

@mvgalle Great! Thanks for sharing, this works. If anyone is still having issues with this problem, here's a summary of what worked for me, from start to finish, using a custom modification of raix:push [1]:
Create myalert.mp3 and myalert.wav.
Load myalert.wav into this package via package.js:
...
api.addAssets('lib/public/myalert.wav','client');
api.export('Push')
...

Next, we need to add myalert.mp3 into a special folder to work with android. see @mvgalle's post
#in your project's root
mkdir .meteor/local/cordova-build/platforms/android/res/raw
cp path/to/myalertmp3 .meteor/local/cordova-build/platforms/android/res/raw/myalert.mp3

Your config.push.json should look like this:
{
  ...
  "sound":true
}

Your calls to Push.send should look like this:
Push.send({
  ...
  apn:{
    //this works because we added 'myalert.wav' as an asset to the custom push package.
    sound: "www/application/packages/yourname_push/lib/public/myalert.wav",
    ...
  },
  gcm: {
    //this will refer to the file we placed in '.meteor/local/.../android/res/raw'
    sound: "myalert",
    ...
  }
})

[1] To create a custom modification of raix:push, use git clone to pipe the source code of raix:push into yourproject/packages/push. Then modify  the name in package.js to yourname:push. I did this because I needed to modify some source code of raix:push. If you don't need to modify the source, then you can just put myalert.wav into yourproject/public, and meteor should load it into the www/application/... directory during the build.

Clever @Streemo !  Thanks for posting this!

nice guys - should we add the solution to the docs or somehow improve the api?

Is this eventually handled by this package? Perfect solution would be to keep the audio file somewhere and add a filepath to mobile.config so that it's automatically added into res/raw folder when building.

Is this eventually handled by this package? Perfect solution would be to keep the audio file somewhere and add a filepath to mobile.config so that it's automatically added into res/raw folder when building.

The cordova-build-override folder might help http://guide.meteor.com/mobile.html#advanced-build

anyone tested
Possible solution for iOS (NOT TESTED):
I assume for iOS you can use similar workflow but place the sound file in public folder and refer the file with full path of cordova build:
for iOS?

@java99 Exactly.
For iOS simply place the file in the public folder and refer to it like so:
sound: "www/application/app/sound.wav"

@philipthomasme where do you write this piece of code ?

@sahanDissanayake I think you specify it in Push.send configuration, see my post above.

yep sweet :) working through it right now.
wasn't 100% sure because you had ... after the sound is defined.. But I will give this a shot
Also I'm thinking from your post that For Android it is mp3 and for IOS it is WAV format ?
Thanks

I can only confirm that for Android it's MP3. For iOS I didn't implement it yet. But I've seen people using .wav for iOS in the posts above.

great I will confirm soon :)

So the following code does not generate a sound :(
Push.send({
    from: 'push',
    title: "title",
    apn:{
        sound: "www/application/app/sound.wav"
      },
    gcm: {
        sound: "sound"
    },
    text: "text",
    badge: 12,
    payload: {
        title: "title",
    },
    query: {
    }
});

@maabed Perfect solution on the top.. Everything is working
So Android sound file is MP3
and IOS uses WAV
( I had the ipad on silent mode.. DUHH!!! ) Thanks

hi @maciejsaw since you are one of the few poeple that got the plugin to work on Android maybe you might have an idea
There are no badge on the icon and also if you send multiple push notifications from the server then one notification on the phone is replaced by the next notification on ANDROID, Which means there is always one ONE notification on tray 😢

I have the same problem of only one notification but I didn't try to solve
it yet.
With the badges I wouldn't worry too much if it doesn't work, android os
doesn't have a consistent way to present badges, only some custom home
screens / launchers have it.
On 00:01, Wed, May 25, 2016 sahanDissanayake<EMAIL_ADDRESS>wrote:

hi @maciejsaw https://github.com/maciejsaw since you are one of the few
poeple that got the plugin to work on Android maybe you might have an idea
There are no badge on the icon and also if you send multiple push
notifications from the server then one notification on the phone is
replaced by the next notification on ANDROID, Which means there is always
one ONE notification on tray 😢
<EMAIL_ADDRESS>My cordova-plugins
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/raix/push/issues/9#issuecomment-221414050

For custom sounds you are unable to use MP3, you need to use WAV or CAF. If you have an mp3 you can simply run the following to convert your mp3 via the terminal.
afconvert -f caff -d LEI16@44100 -c 1 <<file>>.mp3 <file>>.caf

The solution that @maciejsaw proposed worked for me. To make it work in iOS I had to add nested iOS configuration found here. If it's set in the root of configuration object as described in the docs, iOS doesn't pick it up.
Push.Configure({
...
      "ios": {
        "sound": true,
        "badge": true,
        "alert": true
      }
});

We're on version 3.0.3-rc.2 of the push plugin, I'm not sure if this issue was addressed in later versions.

Does anyone know why the sounds stopped working according to this scheme?


[SPARK-16216][SQL] Write Timestamp and Date in ISO 8601 formatted string by default for CSV and JSON

What changes were proposed in this pull request?
Currently, CSV datasource is writing Timestamp and Date as numeric form and JSON datasource is writing both as below:

CSV
// TimestampType
1414459800000000
// DateType
16673

Json
// TimestampType
1970-01-01 11:46:40.0
// DateType
1970-01-01

So, for CSV we can't read back what we write and for JSON it becomes ambiguous because the timezone is being missed.
So, this PR make both write Timestamp and Date in ISO 8601 formatted string (please refer the ISO 8601 specification).

For Timestamp it becomes as below: (yyyy-MM-dd'T'HH:mm:ss.SSSZZ)
1970-01-01T02:00:01.000-01:00

For Date it becomes as below (yyyy-MM-dd)
1970-01-01

FYI, CSV and JSON uses DateTimeUtils.stringToTime(...) to read data and infer schema. This is already okay with ISO 8601 dates. So, this PR only deals with writing.
This would not affect reading at all but only for writing.
How was this patch tested?
Unit tests were added in CSVSuite. For JSON, existing tests cover them.

Test build #62584 has started for PR 14279 at commit 27de16d.

cc @rxin, @srowen and @deanchen

Test build #62584 has finished for PR 14279 at commit 27de16d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62584/
Test FAILed.

It seems due to https://issues.apache.org/jira/browse/LANG-982 I will look into this deeper

Test build #62597 has started for PR 14279 at commit e2e0886.

Test build #62597 has finished for PR 14279 at commit e2e0886.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62597/
Test PASSed.

Test build #62642 has started for PR 14279 at commit f80dd53.

Test build #62642 has finished for PR 14279 at commit f80dd53.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62642/
Test PASSed.

Test build #62817 has started for PR 14279 at commit 10cc553.

Test build #62817 has finished for PR 14279 at commit 10cc553.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62817/
Test PASSed.

Would be great to get this in too. Currently using a hack where we iterate through all the date columns at the end of our run and manually converting the values to the string values formatted as yyyy-MM-dd before writing to csv/json.

@HyukjinKwon is this good to go as far as you're concerned? @deanchen I assume you've reviewed and are OK with it.

@srowen ltgm, thanks for chiming in!

I think it is okay. Just to make sure,

This writes date and timestamps fine:
val path = "/tmp"
val date = java.sql.Date.valueOf("1970-01-01")
val timestamp = java.sql.Timestamp.valueOf("1970-01-01 11:46:40.0")
val data = Seq((date, timestamp)).toDF("date", "timestamp")

// Write CSV data
data.write.format("csv").save(s"$path/csv")
spark.read.format("text").load(s"$path/csv").collect().foreach(println(_))

// Write JSON data
data.write.format("json").save(s"$path/json")
spark.read.format("text").load(s"$path/json").collect().foreach(println(_))

prints below:
[1970-01-01,1970-01-01T11:46:40.000-08:00]
[{"date":"1970-01-01","timestamp":"1970-01-01T11:46:40.000-08:00"}]

Able to be read back fine.
val path = "/tmp"
val date = java.sql.Date.valueOf("1970-01-01")
val timestamp = java.sql.Timestamp.valueOf("1970-01-01 11:46:40.0")
val data = Seq((date, timestamp)).toDF("date", "timestamp")

// Roundtrip in reading and writing CSV data
data.write.format("csv").save(s"$path/csv")
val roundtripCsvDF = spark.read.schema(data.schema).format("csv").load(s"$path/csv")
roundtripCsvDF.collect().foreach(println(_))
roundtripCsvDF.printSchema()

// Roundtrip in reading and writing JSON data
data.write.format("json").save(s"$path/json")
val roundtripJsonDF = spark.read.schema(data.schema).format("json").load(s"$path/json")
roundtripJsonDF.collect().foreach(println(_))
roundtripJsonDF.printSchema()

prints below:
[1970-01-01,1970-01-01 11:46:40.0]
root
   |-- date: date (nullable = true)
   |-- timestamp: timestamp (nullable = true)

[1970-01-01,1970-01-01 11:46:40.0]
root
   |-- date: date (nullable = true)
   |-- timestamp: timestamp (nullable = true)

FastDateFormat is thread-safe and compatible with SimpleDateFormat (See https://github.com/apache/commons-lang/blob/LANG_3_2/src/main/java/org/apache/commons/lang3/time/FastDateFormat.java#L30-L37).

I might have to make a follow-up for..

making DatatypeConverter.parseDateTime to be safe with early dates

enabling TimestampType tests in JsonHadoopFsRelationSuite

merging CSVFileFormat.verifySchema() for verifying schemas before writing (see CSVFileFormat.scala#L87) with CSVRelation. makeConverter here just like the other data sources. For example, JSON (see JacksonGenerator.scala#L115-L119)

I like this change. It does introduce a reasonably significant behavior change in the output, even if the output is better. CC @rxin who brought that up. The problem with changing CSV to match JSON output is that the JSON output is 'wrong'.
I suppose a narrower change here would be to have JSON output a timestamp (number of seconds since the epoch) to match CSV, and then expose an optional date format to control the output. That's less change, but changes JSON. WDYT?
Here, it's always using ISO-8601 by default right?
What's the issue with dates before 1970, just that they don't parse correctly?

+1 to default formats that are unambiguous regarding timezone

Yes, it uses ISO 8601 by default and yes, it seems there is an issue for parsing dates before 1970.

Are we able to read the output back once written as JSON/CSV?

Yes, as shown in https://github.com/apache/spark/pull/14279#issuecomment-236469454
(but we should still manually give the schema as inferring DateType and TimestampType is not supported in JSON and DateType is not inferred in CSV )

@HyukjinKwon are you OK with the current state of this PR? you mentioned a few follow-ups and I'm wondering if you think they're important to include here.
@rxin how do you feel about it? it's a good change I think, but would need a note in release notes
I know Cody and Dean are supportive FWIW

@srowen I think it is okay as a separate PR.
I believe adding the things I said for a follow-up will mess up the change and make this hard to be reviewed. Nevertheless, I don't mind adding it here together if any of you think they should be included here.
Just FYI, the listed ones (for a follow-up above) are,

an existing issue.
enabling the related tests (which was disabled by 1).
cleaning up codes (which I avoided here to make the review easier).

OK, I'd like to merge this for master if there are no other comments. I'm waiting more than usual to make sure

Out right now. Will take a look at this tonight or tomorrow.

@HyukjinKwon can we also allow users to set the output date/timestamp format? This is a breaking change, and I really worry about that. With this change and without making it configurable, there will be no way for Spark 1.x and Spark 2.0.1 to generate the same output.

@rxin, Sure! I can add dateFormat option for both. Do you want me to do this here (or follow-up)?

That'd be great. I will review it asap when you do it.

Sure, thanks!

Test build #63915 has started for PR 14279 at commit 8bcdd59.

Let me please rebase as it is now different a lot.

Test build #63915 has finished for PR 14279 at commit 8bcdd59.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63915/
Test FAILed.

Test build #63917 has started for PR 14279 at commit c4900de.

Test build #63917 has finished for PR 14279 at commit c4900de.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63917/
Test FAILed.

Test build #63918 has started for PR 14279 at commit 4171a3c.

I will take another look tomorrow. Please feel free to take a look further because it's almost complete I think.

Test build #63918 has finished for PR 14279 at commit 4171a3c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63918/
Test PASSed.

Do we need a separate setting for time format?

Test build #63954 has started for PR 14279 at commit 2bd9cbf.

@rxin Ah, I see. Yes, it seems we need for each to allow writing dates and timestamps as the the original format.

Let me please add timestampFormat. As CSV already has dateFormat option for both DateType and TimestampType, Please let me use dateFormat for both if timestampFormat is not set if it makes sense.

Default to dateFormat for timestampFormat seems  wrong? Because in that case all timestamps will be written as yyyy-mm-dd only ignoring the rest?

Oh, default for dateFormat will be null and they will use yyyy-MM-dd'T'HH:mm:ss.SSSZZ for timestamps and yyyy-MM-dd for dates by default. Test cases for the default for both are added in CSVSuite and it is already being tested for JSON  in JsonSuite

But if dateFormat is set to yyyy-MM-dd, then suddenly timestamp output won't show anything past the date part?

If dateFormat is for DateType and timestampFormat for TimestampType, I think both can be independently handled although I am worried of backwards compatibility because dateFormat for CSV were applied to both TimestampType and DateType in 2.0.

Test build #63954 has finished for PR 14279 at commit 2bd9cbf.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63954/
Test PASSed.

If we are introducing breaking changes to fix the bugs here, let's fix it for real. (definitely a problem if we can't specify dateFormat and timestampFormat separately).

BTW I think this is pretty important for 2.0.1 release.

Yup, thanks for informing me.

Sigh.. I will proceed but please let me note that this will also bring a breaking change for reading as well not only writing..
It seems there is a backwards compatibility problem with using default date format for dateFormat and timestampFormat instead of null when reading.
When it was null, it uses DateTimeUtils.stringToTime for TimestampType and DateTimeUtils.millisToDays  for DateType. It seems okay with DateTimeUtils.millisToDays for DateType but the problem is with DateTimeUtils.stringToTime().
It seems DateTimeUtils.stringToTime() for TimestampType supports to read several other date formats such as

2000-01-01T00:00GMT+01:00

2000-01-01T00:00+01:00

2000-01-01 00:00:00.0

As previous Spark version writes 2000-01-01 00:00:00.0, setting the default value for timestampFormat to ISO8601 format will also break the backwards compatibility.
we should explicitly set this for reading as well not only writing for the same timestamps handling between Spark 1.x and Spark 2.x.

Oh, wait, I think I can handle this. Please ignore the comments I just left. Sorry for noise.

Test build #64061 has started for PR 14279 at commit 7e61674.

Test build #64061 has finished for PR 14279 at commit 7e61674.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64061/
Test FAILed.

Test build #64065 has started for PR 14279 at commit 405e723.

Test build #64065 has finished for PR 14279 at commit 405e723.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64065/
Test PASSed.

Test build #64326 has started for PR 14279 at commit af8250e.

Test build #64326 has finished for PR 14279 at commit af8250e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64326/
Test PASSed.

Merged build finished. Test PASSed.

LGTM - merging to master/2.0. Thanks!

@HyukjinKwon could you backport this 2.0? It has merge conflicts so I cannot cherry pick it.

@hvanhovell Sure!


Akka Test Kit Equivalent

Trying to work out how you would go about testing a simple actor that sends a message back on the parent context. It would be good if you had example of how testing is done with middle-ware or some kind of TestKit like Akka.
Trying to test this simple actor and not sure how to go about it.
    public class DeviceLookupActor : IActor
    {
        private readonly IDeviceLookup _deviceLookup;

        public DeviceLookupActor(IDeviceLookup deviceLookup)
        {
            _deviceLookup = deviceLookup;
        }

        public async Task ReceiveAsync(IContext context)
        {
            switch (context.Message)
            {
                case Started _:
                    break;
                case LookupDevice _:
                    await _deviceLookup.LookupAsync().MatchAsync(
                        device => context.Parent.Tell(new DeviceFound(device)),
                        () => context.Parent.Tell(new DeviceNotFound()));
                    break;
            }
        }

        public class LookupDevice
        {
        }

        public class DeviceFound
        {
            public DeviceFound(Device device)
            {
                Device = device;
            }

            public Device Device { get; }
        }

        public class DeviceNotFound
        {
        }
    }
```

Just to clarify writing a interaction based unit test is not a problem but if I wanted to test it through the actor system runtime then it's not clear if there is a simple way to do this.

Our goal is to have enough injection points so that this can be easily done w/o having a special testkit library for it.
We do however have the test mailbox we use in our internal tests, which might be useful to extract.
This lets you write synchronous code and then inspect what has passed through it.
But the general idea is that it should be done with middleware.
I will create an example for it

Just a quick note:
                        device => context.Parent.Tell(new DeviceFound(device)),
                        () => context.Parent.Tell(new DeviceNotFound()));

In order to leverage outgoing middleware, you need to use context.Tell(target, message)
This way, the message is passed through the current actors outgoing pipeline.

Thanks for the clarification regarding the outgoing pipeline and a testing example would be useful.

A test kit was introduced by #469 - should we close this?

Yeah, I think this issue is only still open because we didn't go through old issues :)


disable unicorn/prevent-abbreviations for spec/test files

It's annoying and not particularly necessary in non-production code


Syntax error in latest version

This is a (multiple allowed):

[x] bug

Swiper Version: 5.0.0.

all the esm files have a syntax issue for the translate function

@nolimits4web thanks for the quick fix.
would you consider merging a pr to add support for shadow dom in swiper ? we really need it and we can upgrade the code

@ghaiat if you send it, then sure will merge it ;)

@nolimits4web here you go , let me know what you think
https://github.com/nolimits4web/swiper/pull/3215
Cheers


fix: LDCL-54: Modify getDrupalBaseUrl to always return general base url

ehm, no? why would we? when the caller wants to use
useRuntimeConfig().public.drupalCe.drupalBaseUrl
he should simply do so.
BUT - we should point that out in the documentation. this method will automatically use serverDrupalBaseUrl on the server. If the general base URL is always needed simply use   return useRuntimeConfig().public.drupalCe.drupalBaseUrl
Generally, we should have documentation for all of our provided API helpers. If not, let'S add it.


Option to disable javascript clipboard manipulation

It would be nice to have configuration option that disables javascript clipboard API( https://www.w3.org/TR/clipboard-apis/ ). A feature like this is configurable in firefox using parameter dom.event.clipboardevents.enabled. Here are some rationale for the feature: http://superuser.com/questions/703744/how-do-i-prevent-read-more-www-site-com-from-being-injected-with-non-tynt-jav

Sounds like a good idea if it's not implementable via an extension.

In that superuser thread the Kill Evil extension https://chrome.google.com/webstore/detail/kill-evil/epieehnpcepgfiildhdklacomihpoldk was mentioned.
In Scriptsafe there is also "Options/Fingerprint Protection/Prevent Clipboard Interference" to check.

Sounds like it's already been done, so I'm closing this.


Make the verify-genome command output LaTeX optionally

As of now, there is no interesting information in the verify genome command that could go into the report.


Projects feed new

resolves https://github.com/NEARBuilders/gateway/issues/311
https://github.com/NEARBuilders/gateway/assets/46527178/d20294be-aa56-453e-bb98-0d18a631f546
Current Folder Structure:

Other changes we should make:

[ ] grid vs list view toggle should use state, then debounce update to Storage API
[ ] projects/list and projects/card are both cards -- let's combine into one with two different variants, or choose better naming (list is misleading)
[ ] define heading and subheading CSS in the root -- something like "font-family" should be global

I have trouble understanding the naming convention of files
in home you have used Build, in projects and components you have used card
I think we decided to use with PascalCase right for all the components? like a react component generally have, and the folders in lowercase with hyphens
Can you also please upload the video of change, so we know how it looks?

I have fixed the file and folder naming and a view has been added.

@itexpert120 can we how 2 items/cards per row? would that look better? I think it looks a bit congested right now, or should we leave it right now until the sidebar navbar is finalised?
Also why do we have project inside components? we have Cards component inside home in pages, since it's specific to home page I believe the list and cards are specific to projects itself, isn't it?

We can further discuss it later in tomorrow's meeting. This PR is just for implementing the designs.


how to navigate to another UIViewController, choose something and return it to the first VC?

On my first UIViewController, I have a button which leads to another VC with a list of entries. When I select an entry on this, I would like to return it back to the first UIViewController.
But currently I really have problems implementing this with this architecture...

Hi,
You can use the delegate pattern.
For instance AddBookPresenter notifies its delegate (BooksPresenter) that a Book was added.
You can do something similar, notifying that an entry was selected, by passing back either the entry itself or the selected index path.
Regards,
Cosmin

sorry, didn't see that you already answered long time ago... thanks a lot!
I'm using this architecture in my current project, would love to open issues if I have questions/discussion material, if that's ok for you!


DOC: Add GitHub Desktop Contribution Instructions

What's wrong with the existing documentation
There is No instructions given for contributing to the project using GitHub Desktop.
There should also be the Step about how to contribute in the project using Github desktop as many of contributor's that.
Add ScreenShots
Some thing like this:

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@Suchitra-Sahoo can you please assign this issue to me?


Date Object disappears after case-converter use

Hi,
I'm facing a problem where dates keys converted through case-converter seems to be emptied.
Here, user is an object returned from my ORM. That's why attributes level is removed after toJSON() call.
> user.attributes.created_at
2016-08-17T09:56:04.236Z
  __proto__:Date
> user.toJSON().created_at
2016-08-17T09:56:04.236Z
  __proto__:Date
> caseConverter.toSnakeCase(user.toJSON()).created_at
Object
  __proto__:Object

toCamelCase have the same behavior (except the key is name createdAt, of course...).
Is this a bug or am I missing something ?
Thanks !

Hi @DeadEye0112 I'll try to take a look at it this evening

Hey ! Thanks ! :D

I run this small test
const date = new Date()
const converted = toSnakeCase(date)
date instanceof Date // true
converted instanceof Date // false

What's probably happening is that toSnakeCase is messing up with Date's prototype and as a result, we get back a corrupted Date object.
I think case-converter should leave alone native objects methods, but I would like to hear @Moezalez's input on this one.

@Gpx Yes, I suppose so.
Ideally case-converter should just convert two type of objects:

Arrays
"Key-Value objects"

Not sure how to check for that. Any idea?

I confirm your statement @Gpx. From what I see, case-converter copy only keys and values. The problem with Date Object is that it contains no keys but the Object still have a value.

What about adding a simple check if (obj instanceof Date) we don't convert it?

That would fix the problem for Date's prototypes but are we sure that there are no other prototypes with a similar behavior ? (Error prototype has probably the same behavior)

This is my worry, too.
For now I propose that we just check for enumerable properties, as in my PR:
https://github.com/travelperk/case-converter/pull/10

@DeadEye0112 update to 1.0.1 and you should be fine. If you run into any issues, let us know.


Foundry.toml Config behavior for tests dir

Component
Forge
Describe the feature you would like
@mattsse suggested opening an issue for this
Dir structure
├── src
│   └── Greeter.sol
└── tests
    ├── Greeter.t.sol
    ├── blacksmith
    │   ├── Blacksmith.sol
    │   ├── Errors.bs.sol
    │   └── Greeter.bs.sol
    └── fixtures
        ├── Fixture1.sol
        └── Fixture2.sol

I am trying to do a two-step compile to run some code-gen (blacksmiths) that gets used in my test files.  Right now, I'm separating my src files from my tests in the directory structure src/ and tests/, run forge build to compile src, run the code-gen, and then forge test --contracts tests.  The behavior I was expecting that if I set test='tests' option in the foundry.toml that it would do forge test --contracts tests behind the scenes when I do forge test
Additional context
repo for more context

it looks like setting the test='tests' in the foundry.toml works now or it's being detected by default, but forge build --contracts src compiles everything in the src and test directories instead of just src


IconLayer API Audit

renamed getScale to getSize
renamed size to sizeScale
moved color fallback to the default accessor

@shaojingli @gnavvy @ibgreen @heshan0131 @apercu @abmai

@ibgreen getSize returns a number in pixels - how is it not an absolute size?

getSize returns a number in pixels - how is it not an absolute size?
Yes now that reread the code it looks like that is indeed the case. I probably read the old comment. We're all good then!


EthStorage Devs Meeting #107 Agenda

Meeting Info

Dec 10, 2024, 11:00 UTC+8
Duration: 60 minutes
Meeting link
Anyone can comment on the agenda where they would like this to be discussed.

Agenda

Protocol/App/Research progress
BD/Marketing progress

Goal:

EthStorage Mainnet Ready

Updates:

CircleCI

https://github.com/ethstorage/optimism/pull/106
Enable PR
Filed several issues

Code review

Long term DA:

https://github.com/ethstorage/es-node/pull/339
https://github.com/ethstorage/es-node/pull/338
https://github.com/ethstorage/es-node/pull/337
https://github.com/ethstorage/es-node/pull/325
https://github.com/ethstorage/es-node/pull/340

es-node

https://github.com/ethstorage/es-node/pull/345

TODO:

EigenDA investigation

Goal:

Prepare WSC L2 testnet launch

Updates(since Saturday):

recovered the new devnet from expired seq-window by using singular batch
PR:

https://github.com/ethstorage/optimism/pull/116
https://github.com/ethstorage/optimism/pull/117

review:

https://github.com/ethstorage/optimism/pull/119
https://github.com/ethstorage/es-node/pull/340

TODO:

Figure out why custom gas token is not activated in the new devnet though deploy config seems correct.
Fix https://github.com/ethstorage/optimism/issues/118
Prepare a tool to handle scalar: https://github.com/ethstorage/optimism/issues/93
Investigate https://github.com/ethstorage/optimism/issues/82

Research how to use custom RPC and ask users to pay gas fees.
https://docs.google.com/document/d/19EX5bYmQQadJoM_2u9BPZ6OKWOpXGLLQtpwyLMblwp0/edit?usp=sharing

Archive service x OP Stack demo
https://github.com/ethstorage/es-node/pull/340
https://github.com/ethstorage/es-node/pull/345
https://github.com/ethstorage/optimism/issues/111

Walrus: https://docs.google.com/document/d/18h3-s_j9c7Zg1ybU92EdsvX3hwxPkxnE/edit

Read the white paper, run walrus site builder, join co-learn group, summarize.
prepare PPT: https://docs.google.com/presentation/d/1D9RL1qmHAKPo46-th0BUi6cssVrF8TazcbobnEdH8vg/edit#slide=id.p14

EthStorage x Taiko:

[x] investigation: read docs and go through the taiko codes (Merge the latest code)
[x] integration test
[ ] Run on Holesky chain (Under test)

Goal:

SWC Public Testnet

Updates:

Go through https://github.com/ethstorage/es-node/blob/main/ethstorage/archiver/guide-devnet.md and fix some issues
Study Hyperliquid
Polish talk topics for Soul Gas Token & SWC
Test SGT on the latest devnet and find an issue that the custom gas token is not applied in op-deployer
Discuss the scalar variable setup for QKC https://github.com/ethstorage/optimism/issues/93

TODO:

Promote long-term DA
More tests on devnet

Goal:

E2E test of 4-ary dispute game
E2E test of SGT

Updates:

Cannon E2E test  https://github.com/ethstorage/optimism/pull/107 -- Done
Study SGT doc and how to do SGT E2E test
Rerun https://perfect-amphibian-929.notion.site/SWC-Devnet-1282e493e88380648486d4961eb91a8d

TODO:

SGT E2E test

New layer2 update for binance and upbit
Prepare for super world computer launch
EigenDA meeting, utilize their DA for L2 blob

This week

Attending taipei blockchain week


Webservice - external kubernetes-client-javascript library

Description
This PR removes the local copy of the kubernetes-client-javascript library (i.e. the webservice/k8s_library folder, hence the huge number of files deleted) in favor of the version available at LiqoTech/kubernetes-client-javascript, to avoid code duplication and unify the maintenance efforts.
Although the library has been configured as dependency, it is apparently not installed correctly by npm install and requires some manual intervention (hence the first step in the Dockerfile). @GabriFila  Could you please have a look whether this could be easily fixed? I believe it should be necessary to change a little the package.json file of the library but I am a totally newbie in this field. For the moment, I will keep the PR as draft.
@palexster The kube-score check started failing due to a version update changing the behavior of a test (BTW, should we specify a version tag to prevent this kind of issues in the future?). Now, according to the kube-score suggestion, I deleted the livenessProbe of the frontend deployment, but any alternative solution would be fine to me.
How Has This Been Tested?
This PR does not introduce any functional changes visible to the users. It has been tested building the frontend docker image both locally (checking that the resulting image worked correctly) and with the CI (the action succeeded).

The npm install works locally, but not in the container build. As @giorio94 found at this link, the unsafe-perm must be explicitly set with the config command.
Also 2 more files need to be modified

.eslintignore (update)
,prettierignore (delete)

@GabriFila Thanks! Should be fixed now. When you have time, the PR should be ready for the review (I separated the deletion commit from the other which changes the build logic for clarity).


Update releases JSON format

Currently the JSON format for releases in /releases is :
{
  "<version_1>: "<date_1>",
  "<version_2>: "<date_2>",
...
  "<version_n>: "<date_n>"
}

This format is not extensible : only a date can be provided. Using a format like the following one could make it extensible (e.g. open to new properties that could eventually used for more automation):
{
  "versions": [
    {
      "number": "<version_1>"
      "date": "<date_1>"
    },
    {
      "number": "<version_2>"
      "date": "<date_2>"
    },
...
    {
      "number": "<version_n>"
      "date": "<date_n>"
    }
  ]
}

This will make it possible to add more properties at product level and at version level.
I do not particularly have "extensions" ideas for the time being, but I have the feeling we should not wait to do that. The more we will add automation scripts, the more work it will be if one day we want to change the format.

This issue is linked to https://github.com/endoflife-date/releases.json, but that may be a bit difficult to implement such formats.

Having versions grouped under their cycle would be best if possible. I would make possible for instance to automate the retrieval of EOL date (for example for https://learn.microsoft.com/en-us/lifecycle/products/?terms=Azure DevOps).

Been wanting to do this (See Step 1 on this discussion for eg), so big +1 for creating this.
I like the proposed schema, can be improved with additional fields (such as link for eg) over time.
One important extension is to support EOL and development dates. For example, for several manual scrapes, like:

COS
GKE
Palo Alto Pages
EKS
HAProxy
ROS

we scrape pages that already include the EOL data. This is data that can automatically be patched (if upstream changes) using latest.py.
-1 for release cycle grouping, since it can get quite confusing. We have releaseCycle labels that latest.py uses, and it still has to make some guesses.

I do not particularly have "extensions" ideas for the time being, but I have the feeling we should not wait to do that. The more we will add automation scripts, the more work it will be if one day we want to change the format.

+1
(Puts long-term hat on). I think of this as an ingredient dataset to our main repo. Ideally, this repo by itself will be helpful to other projects by itself as well. So ideally, this would closely match the releases.json format by itself, with any additional bits getting patched on top by the main repo. But a core-idea is for this to be a repo containing all versions, which we aren't accounting for in our releases.json schema (yet). So maybe something to patch there as well? We already have a supportedReleases key, having another releases key (same as versions in your proposal above) won't make much difference - it's gonna be optional anyway.
Since this would be a breaking change for latest.py, maybe we can patch latest.py first to support both old and new schema. That lets us incrementally change the automation here over a few separate PRs.

We could also automatically create/update purls using repology (https://github.com/endoflife-date/endoflife.date/pull/1778#issuecomment-1359140137).

Yes, that is definitely needed. Need to write the script for that. Unsure about keeping the generated PURLs separate, or overwriting the md files (like how we do with latest.py). Probably going with latter.

Yes, overwriting is better IMHO.

Another improvement: we could also indicate the origin of the data (npmjs, maven, git...).

We can provide a list of PURLs for the data-source. pkg:rubygems/bundler points to https://rubygems.org/gems/bundler, but is more standardized.
Another use-case for our frontend would be to directly link to relevant releases from the latest column.

use-case for DotNet to track EOL: https://github.com/endoflife-date/endoflife.date/issues/2234

This should have been closed with #274. Release-level data will be supported after #297 is merger.
Other ideas, such as supporting product-level data (for PURLs) or data origin will be handled in separate issues if there is some need.


fix: DEV-2510: add guard with error message for non incremental, non sequential datasets. (#1752)

Timeseries data is required to be ordered by an incrementing sequence to work with the TimeSeries tag. This adds a check when mapping the timestamps to find an out of order or decrementing value in the series.
https://labelstud.io/tags/timeseries.html

/help


feat: btc switch type segwit/native/legacy

Allows user to switch between segwit, segwit native & legacy bitcoin by passing the correct scriptType & bip32Params to pertinent chainAdapters calls.

Can you provide screenshots of the UI changes?

Can you provide screenshots of the UI changes?

done


Can this active tab indicator be rendered?

hi there, thanks for publishing this component !
We need to create a tab bar (top) that has an active indicator per the screenshot. I will try your component shortly, however in advance of that do you know if this custom tab indicator can be implemented with your component ?

@ajonno Yeah, you could use an image for this indicator.

nice one. via renderIndicator ?

yes

dude that was so fast! ok thanks will give it a whirl shortly

need some guidance, im doing some testing but can't seem to modify the behavior of the renderIndicator prop.
onRenderIndicator = (props) => {
     return (
         <View style={styles.indicator}>
            <Text>displays?</Text>
         </View>
     );
};

here's the TabBarTop setup:
<TabBarTop
       {...props}
       pressColor='rgba(251, 172, 48, 1)'
       onTabPress={this.onHandleTabItemPress}
       renderLabel={this.onRenderLabel(props)}
       renderIndicator={this.onRenderIndicator(props)}   <=== added
       indicatorStyle={styles.indicator}
       tabStyle={styles.tab}
       tabWidth={90}
       style={styles.tabbar}
/>

however there doesnt appear to be any change to the indicator. obviously im just popping some text in the onRenderIndicator method to test but is it instead expecting an  or .. ?
thanks for your help.

You actually have to use TabBar instead of TabBarTop for custom indicator - https://github.com/react-native-community/react-native-tab-view#tabbar-
And it must be a function, so
__renderIndicator = params => props => (...)

You can copy the code from here. Change Animated.View to Animated.Image for your image - https://github.com/react-native-community/react-native-tab-view/blob/master/src/TabBarTop.js#L70

oh i see. we need the custom indicator for a TopTabBar. any chance you can add this capability?

sorry but i dont quite follow ?  firstly we need a TapBarTop component ie. display tab bar top of view on both ios/droid.
is this currently not possible ? the onRenderIndicator method that I previously displayed (above) is invoked. however if i return the following from this method (below), there is no change to the TopTabBar indicator.
onRenderIndicator = props => {
   const { width, position } = props;
   const translateX = Animated.multiply(position, width);

  return (
  <Animated.Image
    source={require('../images/myIndicator.png')}
    style={[ styles.indicator, { width, transform: [ { translateX } ] } ]}
  />
  );
};

@ajonno TabBarTop is just a TabBar with indicator and some styling.

ahh sorry i should have checked the return .... i was assuming it was a separate component. all good now thanks!
render() {
    return (
      <TabBar
        {...this.props}
        renderLabel={this.props.renderLabel || this._renderLabel}
        renderIndicator={this._renderIndicator}
      />
    );
  }


Use URL instead of relative path for documents in the repository

This is a cosmetic documentation change.
This PR uses URL instead of relative path for documents in the repository
404 page will be displayed when trying to refer to "Changelog" link and "changelog entry format" link in the following URL.
https://github.com/rubocop-hq/rubocop/blob/master/.github/PULL_REQUEST_TEMPLATE.md
Therefore, This PR changed from relative path to URL.

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[ ] Added tests.
[ ] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Thanks!


Request Elastic APM to records spans for the current request

Is your feature request related to a problem?
Transaction sampling with the details of the spans is quite a useful feature. I would like to be request a specific transaction to be sampled.
Describe the solution you'd like

Add a new header e.g. 'trace-profile:true' or we can add a flag in `tracestate:profile=true'
This should indicate to the APM to save all the span information for the request

Describe alternatives you've considered

Additional context

That's great @felixbarny. I will try it out.
There are times we know one of our users/clients is having a slow experience on the application or one of our job runs are taking longer than expected. It will be great if we can set a header and get the APM to trace the full request giving us all the spans and telling what are the parts that are most expensive for that particular request.

Hi @chintohere did you managed to get it working using the sampled flag Felix mentionned ?
I'm doing some cleanup, and this issue does not seem to require any code change within the agent, thus I'll close this issue for now.
Feel free to further discuss this in our forum if needed.


KPI Alert Range bar value issues fixed

KPI Alert Range bar Not Update Correct value #848

@ChartistDev @Santhoshkumar1023 Any specific reason for the severity to be done as 1?

@manassolanki according to @Santhoshkumar1023 severity was defaulted to 1 previously. The bug was that it was mistakenly set to 90.


Gradle 4.9 + lombok 1.18 = build failed

Now lombok not comatible with gradle 4.9. Please, fix it.
With Gradle 4.8.1 all works perferct.

Can you test this using the edge release? Although not mentioned in the changelog, we did improve gradle support.

Can you test this using the edge release? Although not mentioned in the changelog, we did improve gradle support.

I have also tested and could confirm that 1.18.0 and 1.16.22 is not working with Gradle 4.8
e.g when I add to my class XXX annotation @Builder then occurred compilation error "Cannot find symbol XXXBuilder" but if I downgrade to Gradle 4.6, the code is being compiled correctly.
I am convinced that Lombok should be aligned to recent version of Gradle.

for whoever is also running in this issue:
In the the gradle tutorial in the oficial website it uses older version of the net.ltgt.apt plugin, which was causing my gradle4.9+java10 build to fail.
Upgrading net.ltgt.apt plugin to version 0.18 fixed the problem.
Ref:

Thanks, it does indeed.


[BUG] createTableIfNotExists logs message with info level error when table exists

Describe the bug
createTableIfNotExists logs an message with level ERROR when a table exist. The code works fine, but makes tracking the log from this library problematic.
Exception or Stack Trace
To Reproduce
Use method createTableIfNotExists on an existing table store entry and see that it logs.
Code Snippet
tableServiceClient.createTableIfNotExists("foo")
Expected behavior
It should create the table and ignore any duplicate table warnings - that is, not log anything, at least not in ERROR level. debug would be preferable.
Screenshots
No
Setup (please complete the following information):

OS: Linux (Not relevant)
IDE: Intellij
Library/Libraries: com.azure:azure-data-tables:12.3.8+ has this problem. com.azure:azure-data-tables:12.3.7 is fine.
Java version: 17
App Server/Environment: N/A
Frameworks: N/A

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

verbose dependency tree (mvn dependency:tree -Dverbose)
exception message, full stack trace, and any available logs

Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[ x ] Bug Description Added
[ x ] Repro Steps Added
[ x ] Setup information Added

Thanks for the feedback @knubonrk.  @jairmyree could you please follow up?

@knubonrk Thank you for reaching out, I will follow up on this shortly.

@knubonrk Thank you for your report. Our team will look into adjusting the logging in the future. You can track issue here. Because this is not a bug, I am going to close out this report. If you have any additional concerns, please see our SUPPORT.md for more information. Thank you very much.


[0.9.1] data with same timestamp and tags gets overwritten

Is it a limitation of influx that only one point can exist with the same tags and timestamp? If a collision does occur, does the old one get overwritten?
I can reproduce with this script:
from influxdb import InfluxDBClient
import time
from uuid import uuid4
token = str(uuid4())
t = int(time.time())
cli = InfluxDBClient(host="localhost",port="8086",database="test",timeout=30)
cli.create_database("test")
events=[{
        'database': "test",
        "retentionPolicy": "event_data",
        'points': [
            {
                'measurement': 'events',
                'tags': {"location":"Mars", "sector":"12a","token":token},
                'time': t,
                'fields': {"lifeform-count":600}
            }
        ]
    },{
        'database': "test",
        "retentionPolicy": "event_data",
        'points': [
            {
                'measurement': 'events',
                'tags': {"location":"Mars", "sector":"12a","token":token},
                'time': t,
                'fields': {"lifeform-count":700}
            }
        ]
    }]
for e in events:
    cli.write(e)

print cli.query("""SELECT * FROM events WHERE "token"='{0}'""".format(token))

Output: It only lists one point even through I wrote two.
eric@mothership:~$ python influx_test.py
ResultSet({'(u'events', {u'sector': u'12a', u'token': u'4ed89066-cef1-44d2-a201-f058bdfc8e43', u'location': u'Mars'})': [{u'lifeform-count': 700, u'time': u'2015-09-17T18:17:07Z'}]})

Is it a limitation of influx that only one point can exist with the same tags and timestamp?

Yes, this is by design.

If a collision does occur, does the old one get overwritten?

Yes, it is silently overwritten.
This is documented but not well. I opened https://github.com/influxdb/influxdb.com/issues/324 to address that.

@beckettsean  So, are points always overwritten when repeatedly sending something like { data: 123 }?
Or are they overwritten only when the data being sent has a time field set to some constant value which remains the same?

@halt-hammerzeit: https://docs.influxdata.com/influxdb/v0.13/troubleshooting/frequently_encountered_issues/#writing-duplicate-points

I'm new to time series database, I got confused by this and wasted one afternoon in finding out why, until I find this post. I'm currently adding an uuid tag to prevent points overwriting each other, is this a good practice?

@beckettsean I am a little confused about the overwriting the duplicate data. The overwrite will first check if there is any existing duplicate data (e.g., with the same measure, tag, timestamp), and if it has been found, what will do next ? It will directly modify the existing data in TSM file on disk ? If so, the performance will be affected.  Or, it will do like the deletion operation.
As I know, the deletion operation will add a delete tag to the corresponding data, and when compaction occurs, it will make the garbage collection.
Can you give me some explanations?
Thanks.


Did you forget the node14 versions?

The Hub docker mentions the existence of a node14-alpine version for the latest versions of strapi 3.5.2 but no image has been published with the node 14 version for 2 months (since 3.4.1 version)

As far as I'm aware Strapi relies on sharp as a dependency for image compression and sharp doesn't have a node 14 version just yet.

Any news about why there is no build targets for node14? According to https://github.com/lovell/sharp/issues/2235 there is no problem with node14 and sharp?


PlayTorchJSIModule not found - ios & android m2 macbook pro

Version
0.2.4
Problem Area
react-native-pytorch-core (core package)
Steps to Reproduce
follow these instructions on a 2023 m2 MacBook pro (not positive this is the cause) until you get to the section "Use PlayTorch SDK"
up until this point the expo app will work correctly for each of the following commands npx expo start npm run ios npm run android
once you add the import
import { torch } from 'react-native-pytorch-core';
The exception from line 27 of install-jsi.ts will be thrown when you run npx expo start
If you run npm run ios I get an extremely long xcodebuild error, I have posted what I think is the relevant portion below
› Compiling react-native Pods/libevent » signal.c
    Run script build phase '[CP-User] Generate app.config for prebuilt Constants.manifest' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'EXConstants' from project 'Pods')
    Run script build phase 'Bundle React Native code and images' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'testapp' from project 'testapp')
    Run script build phase 'Start Packager' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'testapp' from project 'testapp')

› 0 error(s), and 3 warning(s)

CommandError: Failed to build iOS project. "xcodebuild" exited with error code 65.
To view more error logs, try building the app with Xcode directly, by opening /Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/ios/testapp.xcworkspace.

Command line invocation:
    /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -workspace /Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/ios/testapp.xcworkspace -configuration Debug -scheme testapp -destination id=CAC7A930-B0AD-4927-9DA6-DDCD8DA30303

User defaults from command line:
    IDEPackageSupportUseBuiltinSCM = YES

Prepare packages

Computing target dependency graph and provisioning inputs

if you run npm run android I get the following error
BUILD FAILED in 3s
5 actionable tasks: 4 executed, 1 from cache
Error: /Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/android/gradlew exited with non-zero code: 1
Error: /Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/android/gradlew exited with non-zero code: 1
    at ChildProcess.completionListener (/Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/node_modules/@expo/spawn-async/build/spawnAsync.js:52:23)
    at Object.onceWrapper (node:events:628:26)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1091:16)
    at ChildProcess._handle.onexit (node:internal/child_process:302:5)
    ...
    at Object.spawnAsync [as default] (/Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/node_modules/@expo/spawn-async/build/spawnAsync.js:17:21)
    at spawnGradleAsync (/Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/node_modules/@expo/cli/build/src/start/platforms/android/gradle.js:72:46)
    at Object.assembleAsync (/Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/node_modules/@expo/cli/build/src/start/platforms/android/gradle.js:52:18)
    at runAndroidAsync (/Users/rbmdotdev/workspace/rbmdotdev/portfolio/test-app/node_modules/@expo/cli/build/src/run/android/runAndroidAsync.js:31:24)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

removing the import allows npx expo start to work again however npm run ios and npm run android still throw the same error
Expected Results
I expext to be able to follow the instructions on this tutorial https://playtorch.dev/docs/tutorials/get-started-expo-app/ and get a working app
Code example, screenshot, or link to repository
No response

I have the same issue, just to help clarify:

PlayTorchJSIModule is not found -> this happens when scanning the Expo QR on Expo Go, correct?
I also have problems following the steps in the https://playtorch.dev/docs/tutorials/get-started-expo-app/ tutorial.

2a. torchlib-lite seems to not have.. something built for arm yet. I needed to disable arm64 as a build target for simulators
2b. After disabling arm64 build targets, I encounter this:
❌  (/Users/rs/Projects/uni/capstone/app/node_modules/expo-modules-core/ios/Swift/Functions/ConcurrentFunctionDefinition.swift:49:5)

  47 |
  48 |     // Switch from the synchronous context to asynchronous
> 49 |     Task { [arguments] in
     |     ^ 'Task' is only available in iOS 13.0 or newer
  50 |       let result: Result<Any, Exception>
  51 |
  52 |       do {

❌  (/Users/rs/Projects/uni/capstone/app/node_modules/expo-modules-core/ios/Swift/Functions/ConcurrentFunctionDefinition.swift:49:5)

  47 |
  48 |     // Switch from the synchronous context to asynchronous
> 49 |     Task { [arguments] in
     |     ^ 'init' is only available in iOS 13.0 or newer
  50 |       let result: Result<Any, Exception>
  51 |
  52 |       do {

I already tried reinstalling the expo-modules-core but still can't continue.
The biggest problem here is that the tutorial/docs is outdated and we can't even start a new project following the steps.

After more digging, seems like https://github.com/facebookresearch/playtorch/issues/195 is related.
I managed to run the tutorial steps, modified:

change the first step (creating expo app) with SDK 47:
I think this is related to the newer RN version incompatibility, as i didn't encounter issue 2b i described above.

npx create-expo-app my_app --template blank@47

once you have ran npx expo prebuild, run xed ios and disable arm64 bulid target. Follow these steps:

from: https://developer.apple.com/forums/thread/672541?answerId=722978022#722978022
now you should be able to run npm run ios and continue the tutorial

I occasionally experience the same issue, but only in debug builds. In release builds it never happens.


feat: download file in document list

Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:

[x] Target branch: Please verify that the pull request targets the dev branch.
[x] Description: Provide a concise description of the changes made in this pull request.
[x] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[ ] Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
[ ] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
[x] Testing: Have you written and run sufficient tests for validating the changes?
[x] Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
[x] Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following:

BREAKING CHANGE: Significant changes that may affect compatibility
build: Changes that affect the build system or external dependencies
ci: Changes to our continuous integration processes or workflows
chore: Refactor, cleanup, or other non-functional code changes
docs: Documentation update or addition
feat: Introduces a new feature or enhancement to the codebase
fix: Bug fix or error correction
i18n: Internationalization or localization changes
perf: Performance improvement
refactor: Code restructuring for better maintainability, readability, or scalability
style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
test: Adding missing tests or correcting existing tests
WIP: Work in progress, a temporary label for incomplete or ongoing work

Changelog Entry
Description

Re-add the download button as link
Use FileAPI
Store file-id in meta data of documents
Only show download button, if file-id is available

Added

Download button for documents

Additional Information

It might make sense to migrate the documents to the files API sooner or later. The files table also contains all the inline uploaded files, which are not available in the documents table
If backend and frontend running on different hosts/ports than the browser might show a preview instead of downloading the file.

Cool feature, I was looking for something similar.
Do you think you could add a new column in the table with the date the document was added?

@jonathan-rohde WDYT ^^

Sounds good to me to add that info in general to the view.
But I do not see it necessarily part of this PR, could be another one.
Furthermore I am currently not updating the PR as the document view is changing and I first want to await those changes to update my code.

Cool ! Thank you for your answer, make sense.


Add VL53L1X

Untested, ending for the day possibly, and wanted a couple of UF2's for testing next

Ignoring the build size, this is now ready for review @brentru

Holding until https://github.com/adafruit/arduino-esp32/issues/18 is merged and tested. This PR will increase the available application flash and resolve the funhouse build error.

@tyeth  Why is funhouse failing? Is this PR simply using the old workflow and can't be updated until we merge?


fix(SNP-1234): Correct Logo Position

Changes:

correct logo position
update version

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (bb27cdf) to head (29040db).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #32   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            2         2
  Lines           24        24
=========================================
  Hits            24        24

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


UI for adding children when booking event

EPIC: #1062
Dependency: #1042
Allow a user to add children OTG in case you're bringing in somebody or simply go through the simplified registration/book a ticket process.
NB - Keep flow for ‘New applicant’ as is for Ticketing Admin.

May not be needed when we have add child as part of the registration flow.
Thoughts @rosalanghammer?

May still be needed for users who have already registered but haven't added kids. Needs re-examination when we look at registration and booking flows so leave open for now.

Adding children is now supported through the new booking flow


Check installed methods

hmm, i quite like the error handling here.

No worries, I moved the code to a different branch ^^


Update index.html

Remove the brackets from the anchor tag while you replace the placeholders with actual url.

Remove the brackets from the anchor tag while you replace the placeholders with actual url.

Done

fix conflits, try adding your profile in between

still blocked because of conflicting files

I will create a new pull request


Fix typo in workspace extension activation failure message

Fix this typo seen when selfhosting

/assign @sandy081

@sandy081 it's unclear to me what's stopping this from merging.

Not clear to me too. Let me close and reopen to try re-triggering the checks


Countour plotting needs transposition

Hey, I noticed a bug in your contour plotting as in this function: https://github.com/igul222/improved_wgan_training/blob/master/gan_toy.py#L163. You have to transpose the height values for the contour using .transpose()
i.e. plt.contour(x,y,disc_map.reshape((len(x), len(y))).transpose())
The reason the results look ok for the gaussian toy examples is because they are symmetrical. Try removing a few gaussians from the 8gaussian example like this and see what happens:
    elif DATASET == '8gaussians':
        scale = 2.
        centers = [
            (1,0),
            #(-1,0),
            #(0,1),
            (0,-1),
            (1./np.sqrt(2), 1./np.sqrt(2)),
            (1./np.sqrt(2), -1./np.sqrt(2)),
            #(-1./np.sqrt(2), 1./np.sqrt(2)),
            #(-1./np.sqrt(2), -1./np.sqrt(2))
        ]

This might improve your results on the swiss roll as well since it's not perfectly symmetrical.
Had me scratching my head for a bit on my own unsymmetrical dataset wondering why the controus looked so wrong haha.
Cheers

Good catch!


not able to signup with email

I've followed project database
I am not getting any response & not able to signup with email.
Console :

google Play services out of date.  Requires 11910000 but found 11055480

@RajuSE that means you need to install a newer version of Google Play services on the device you're using. If you're using a real device this will happen automatically, if you're using an emulator just update the image and create a new emulator or use one of the images with the Play Store on it.


Update documentation

Pull Request Test Coverage Report for Build 265

1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.4%) to 90.402%

Totals

Change from base Build 264:
0.4%

Covered Lines:
2317

Relevant Lines:
2563

💛  - Coveralls


Custom tag per RessourceModel not available anymore

The feature added in PR https://github.com/Batix/rundeck-ansible-plugin/pull/60 is not available anymore in the latest release. For what reason it has been removed? Is it possible to get it back?

Sorry about that, seems like @yassineazzouz based the major rewrite on code that didn't include it or removed it accidentally. I'm sure it will be added back (PRs are always welcome). :)

Hi @UnsignedLong , Apologies for that, as @frozenice mentioned I have missed that in the code rewrite I will restore it ASAP.
Thanks,

Hi @UnsignedLong ,
I restored the PR in release 2.0.2, can you check and let me know if that's ok ?
Thanks,

Closing due to inactivity.


Support for Ubuntu 24.04 LTS

Ubuntu 24.04 LTS (Noble Numbat) is planned for April 25, 2024. We would like to get the support for a new core image based on this new version as some customers will require it.
A similar feature request was created to support Ubuntu 22.04. Based on this comment
Test the following releases

[ ] #2241
[ ] arm64 generic
[ ] arm64 rpi4

[ ] https://github.com/kairos-io/kairos/issues/2249

Artifacts and Cleanup

[ ] add in flavors.json
[ ] remove 23.10
[ ] what about 20.04

@mudler @cloudmaniac What does that mean for our other LTS releases? Do we support them all the way till their EOL dates? Or do we plan to roll the versions and only keep a certain number of LTSs? e.g. stop building 20.04 artifacts as soon as 24.04 is released

@mudler @cloudmaniac What does that mean for our other LTS releases? Do we support them all the way till their EOL dates? In the case of 20.04 this is 2030 for example. Or do we plan to roll the versions and only keep a certain number of LTSs? e.g. stop building 20.04 artifacts as soon as 24.04 is released

It would make sense to me to just keep LTSes until their end of standard support for now (not EOL) - all the rest can be cut off (intermediary releases, latest). But I'd like to hear what @cloudmaniac 's thoughts are on it.
https://wiki.ubuntu.com/Releases for reference

Blocks RPi https://github.com/kairos-io/kairos/issues/2249

It would make sense to me to just keep LTSes until their end of standard support for now (not EOL) - all the rest can be cut off (intermediary releases, latest). But I'd like to hear what @cloudmaniac 's thoughts are on it.

I didn't forgot about that question: I agree with that statement @mudler but I'm still checking internally as I may not have the final word on this. 😅

btw @cloudmaniac with current master of Kairos, you should be able to build 24.04 for amd64, feel free to test it out and let me know if you find any issues. We are not yet producing artifacts ourselves, but this way you can already find out if there's any requirements that aren't met by the base image.

Regarding the artifact cleanup, the EOL for 23.10 is July 24, I think we should move that into an independent ticket because we might even have to delay that a bit to avoid braking the flow of anyone depending on this version. But if we do so, we must be very clear that the version is not supported so security wise is a bad idea


Two questions regarding the 'abide.npy' dataset.

Hello, I'd like to ask two questions regarding the 'abide.npy' dataset.

Is 'abide.npy' obtained after processing data following the command git clone https://github.com/preprocessed-connectomes-project/abide and using python download_abide_preproc.py -d rois_cc200 -p cpac -s filt_global -o data/?

How are the dimensions of 'final_timeseires' in ABIDE (1009, 200, 100) and 'final_pearson' (1009, 200, 200) determined, and what do they represent?

Thank you.

Here is the repo https://github.com/Wayfear/FBNETGEN/tree/main/util/abide I used to generate the abide dataset.
'final_timeseires' in ABIDE (1009, 200, 100)  means (sample_size, roi_size, time-series length); 'final_pearson' (1009, 200, 200) means  (sample_size, roi_size, roi_size)

I'd like to ask another question.
In the HGSUM model, is the "timeseries" data used solely for dimensionality checking, or does it serve other purposes as well?
Thank you.

I do not have a "HGSUM" model in my repo. Can you point out the code in this repo?

Oh, sorry, I made a mistake. I was asking about the BrainNetworkTransformer model.

Yes, I do not use timeseries for BNT model

Thanks you!


fisherman→fisher

Hi @jhipster! Fisher is the continuation of and successor to Fisherman. Could you please rename your repository, tags, and mentions of "fisherman" and replace them with "fisher"?
About changing the name of the repository; it is safe to do it. When you rename a repository, username or organization account, old links redirect to the new location. Changing the name of your repository will not result in broken-links.
See: https://blog.github.com/2013-05-16-repository-redirects-are-here

Hi @jorgebucaran  i already saw the change, but didn't had time to update everyhing. Will do soon thx for the heads up!

Awesome, thanks!


:sparkles:Add LGTM to the SAST check

According to https://github.com/apps/lgtm-com
"LGTM is a code analysis platform for identifying vulnerabilities early and preventing
them from reaching production". It's used by systemd, lxc and a lot of other large
open source projects. The check is
still kind of broken in the sense that it fails to detect
projects where every PR is analyzed by LGTM before getting merged
but it's better than nothing I guess.

/ok-to-test sha=03c92279e12aeb4b0235d5cd878008fdf89e3013

I'm not sure what TestSARIFOutput is but judging by https://github.com/ossf/scorecard/runs/4146238436?check_suite_focus=true it seems to have been failing since the latest commit was merged so I don't think it has anything to do with this PR.

I'm not sure what TestSARIFOutput is but judging by https://github.com/ossf/scorecard/runs/4146238436?check_suite_focus=true it seems to have been failing since the latest commit was merged so I don't think it has anything to do with this PR.

Thanks. I've send a PR to fix it https://github.com/ossf/scorecard/pull/1233

According to https://github.com/apps/lgtm-com "LGTM is a code analysis platform for identifying vulnerabilities early and preventing them from reaching production". It's used by systemd, lxc and a lot of other large open source projects. The check is still kind of broken in the sense that it fails to detect projects where every PR is analyzed by LGTM before getting merged but it's better than nothing I guess.

Can you elaborate? Function sastToolInCheckRuns does list merged PRs and look for Slug, see https://github.com/ossf/scorecard/blob/main/checks/sast.go#L138 Is this insufficient for LGTM? Is there some improvement we should make?

Is this insufficient for LGTM?

I'm not sure. I need to figure out why scorecard says that "23 commits out of 30 are checked with a SAST tool". It could be it has something to do with "neutral" checks where the source code wasn't changed and LGTM was skipped in a way.

Is this insufficient for LGTM?

I'm not sure. I need to figure out why scorecard says that "23 commits out of 30 are checked with a SAST tool". It could be it has something to do with "neutral" checks where the source code wasn't changed and LGTM was skipped in a way.

Thanks. Let's not block this PR on this problem. Can you create a tracking issue for it so we don't forget? If you're interested in looking into, help is appreciated!

Looks like in PRs like https://github.com/systemd/systemd/pull/21266 the status of LGTM is "neutral" because "LGTM.com did not detect changes to any C/C++ files". It can be fixed by not skipping PRs where cr.Conclusion is neutral:
+                       if cr.Conclusion != "success" && cr.Conclusion != "neutral" {
                                continue
                        }

With that patch applied scorecard says that "29 commits out of 30 are checked with a SAST tool", which is correct because apparently in https://github.com/systemd/systemd/pull/21266 LGTM got lost somehow. Though I'm not sure flukes like that should affect the aggregate score.

Looks like in PRs like systemd/systemd#21266 the status of LGTM is "neutral" because "LGTM.com did not detect changes to any C/C++ files". It can be fixed by not skipping PRs where cr.Conclusion is neutral:
+                       if cr.Conclusion != "success" && cr.Conclusion != "neutral" {
                                continue
                        }

With that patch applied scorecard says that "29 commits out of 30 are checked with a SAST tool", which is correct because apparently in systemd/systemd#21268 LGTM got lost somehow. Though I'm not sure flukes like that should affect the aggregate score.

That sounds good. Please send us a PR and cc me on it, I'll LGTM it.

This PR is ready to merge code-wise. @olivekl we only need your ack for the documentation.

LGTM! Thanks, @evverx!

Thanks.


Reduce global state mutation in quilc

Replace most global state mutation in quilc with let bindings.
This is "option 5" from the discussion in #192. Turns out the state mutation was almost entirely confined to quilc::process-options, so switching to let bindings everywhere wasn't too bad.
The only remaining global state mutation occurs in quilc::setup-debugger which is called from quilc::entry-point, but not quilc::%entry-point. Removing this last bit of state mutation is also possible, but slightly uglier since it requires either peeking under the covers of sb-ext:disable-debugger to figure out what additional vars to let-bind, or else running the body of quilc::entry-point in an unwind-protect that calls quilc::enable-debugger on exit. Since the whole point of quilc::%entry-point is presumably to provide a non-debugger-mangling entry point for interactive use, it didn't seem worth it to take this final step.
Fixes #192

What about a macro called with-debugger-disabled or so?

What about a macro called with-debugger-disabled or so? CC @rigettizach

I did consider such a macro, but decided against it since I though it might seem like a Gratuitous Use of Macros (TM). Especially because I assumed that entry-point (as opposed to %entry-point) is only likely to be called when running the quilc executable and hence didn't seem like much point restoring the debugger state when the next thing that happens is process exit. I'm happy to add it though if maintainers feel it's a good idea.

@appleby You're right, it is a bit of a gratuitous use of a macro, but then again, it's not a very confusing macro. I would look at @notmgsk's recent work on temporarily disabling the debugger with his with-mocked-function-definitions macro: https://github.com/rigetti/qvm/blob/master/app/tests/utils.lisp#L3
See its use here: https://github.com/rigetti/qvm/blob/master/app/tests/suite.lisp#L179
We ought to get this macro included in Fiasco itself, so everybody can use it.

Personally, I'm all in favor of small macros that make your life better. I just erred on the side of caution since opinions vary and I wasn't 100% sure it was necessary.
There are some small complications. For example, setup-debugger only disables the debugger if the forest-sdk feature is not present; otherwise, it sets *debugger-hook* to 'command-line-debugger. So it's not quite as simple as
(with-debugger-disabled
   ;; body of entry-point here)

One option is to rename the macro with-debugger-setup, but that has a slightly bad mouthfeel. As you say, with-debugger-disabled is general enough to possibly be included in fiasco, but with-debugger-setup feels like a single-use macro.
Alternatively, the +/- forest-sdk conditional feature logic of setup-debugger could be moved into entry-point, like so:
#+forest-sdk
(let ((*debugger-hook* 'command-line-debugger))
  (entry-point-thunk) ; or something with a better name
#-forest-sdk
(with-debugger-disabled
  (entry-point-thunk)))

One final complication I just noticed: build-app.lisp also calls quilc::setup-debugger when preparing the executable image. I'm not sure why. Possibly to ensure the debugger is setup even when build-app is passed the --unsafe option, which causes it to use %entry-point rather than entry-point, which would otherwise skip debugger setup?
Or maybe I've misunderstood your intention, and you mean to leave the debugger twiddling in entry-point.lisp alone, and that with-debugger-disabled is meant to be macro for use only in testing, similar to @notmgsk's with-mocked-function-definitions? And in that case, setup-debugger could be mocked to a no-op?
Clearly, I need to think about this a bit more. Posting rambles for comments in case anyone has better ideas.

I would rather just setup the debugger in build-app.lisp, and avoid it in the entry-point. I don't think the debugger is re-enabled upon "boot", but maybe that's why both instances are there. I forgot.

I would rather just setup the debugger in build-app.lisp, and avoid it in the entry-point.

Ok, I will give that a try later and see how it goes.

I just pushed a new commit that removes the call to quilc::setup-debugger from quilc::entry-point.
Here is the overly-long-winded description of this small change.
I manually tested the effect of this change with the following quilc variants / make targets:

quilc
quilc-unsafe
quilc-sdk-barebones

I did not test quilc-sdk, because it apparently expects libs in /usr/local/lib/rigetti/<whatever>. I did try the docker-sdk target thinking that the docker build might put the libs in the right place for me, but that also fails at startup. See the end of this comment for the exact error message from running the docker-sdk build, in case that is not an expected error.
I tested this change by executing each of the following forms at the top of quilc::%entry-point, to make sure the behavior is the same before and after removing the call to setup-debugger in entry-point:

(break)
(error ...)
(invoke-debugger ...)

The good news: behavior appears to be the same before and after. That is, whatever debugger state is modified by the call to setup-debugger in build-app.lisp does persist across the call to sb-ext:save-lisp-and-die. In no case was I dropped into a debugger. "Safe" variants that use entry-point print a one-line summary of errors, and print a backtrace for anything that would normally invoke the debugger. The "unsafe" variant that uses %entry-point prints a backtrace in all cases.
Two small surprises.

The call to setup-debugger can also be removed from build-app.lisp, apparently without consequence. I think the reason is that the make target invokes sbcl with the --non-interactive flag, which implies --disable-debugger. Apparently, the disabled debugger persists in the executable image.

as a consequence of (1), conditionally setting *debugger-hook* to 'command-line-debugger when the forest-sdk feature is present appears to have no effect. Or at least I could not determine how to tickle it. To clarify, the definition of setup-debugger looks like this:

(defun setup-debugger ()
  #+forest-sdk
  (setf *debugger-hook* 'command-line-debugger)
  #-forest-sdk
  (disable-debugger))

When the forest-sdk feature is present (i.e. in the {quilc,docker}-sdk{,-barebones} variants), *debugger-hook* is indeed set to 'command-line-debugger, but sbcl essentially ignores it because sb-ext:*invoke-debugger-hook* is already set to sb-debug::debugger-disabled-hook, due to (1).
Note that I didn't test any of this in ccl because I didn't see an easy way to produce a ccl executable since build-app.lisp is sbcl-specific. But I did notice that ccl has it's own implementations of {enable,disable}-debugger in app/src/impl/clozure.lisp, so it's possible the setting of *debugger-hook* is there for ccl.
Error message when trying to run docker image produced by make docker-sdk
It seems libzmq.so is present in the docker image, but it's located at /usr/lib/x86_64-linux-gnu/libzmq.so.
$ docker run --rm -it quilc-sdk
Unhandled SIMPLE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                    {10006B85B3}>:
  Error opening shared object "/usr/local/lib/rigetti/libzmq.so":
  /usr/local/lib/rigetti/libzmq.so: cannot open shared object file: No such file or directory.

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {10006B85B3}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SIMPLE-ERROR "Error opening ~:[runtime~;shared object ~:*~S~]:
  ~A." {1000EC56D3}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SIMPLE-ERROR "Error opening ~:[runtime~;shared object ~:*~S~]:
  ~A." {1000EC56D3}>)
2: (INVOKE-DEBUGGER #<SIMPLE-ERROR "Error opening ~:[runtime~;shared object ~:*~S~]:
  ~A." {1000EC56D3}>)
3: (ERROR "Error opening ~:[runtime~;shared object ~:*~S~]:
  ~A." "/usr/local/lib/rigetti/libzmq.so" "/usr/local/lib/rigetti/libzmq.so: cannot open shared object file: No such file or directory")
4: (SB-SYS:DLOPEN-OR-LOSE #S(SB-ALIEN::SHARED-OBJECT :PATHNAME #P"/usr/local/lib/rigetti/libzmq.so" :NAMESTRING "/usr/local/lib/rigetti/libzmq.so" :HANDLE NIL :DONT-SAVE NIL))
5: (SB-ALIEN::TRY-REOPEN-SHARED-OBJECT #S(SB-ALIEN::SHARED-OBJECT :PATHNAME #P"/usr/local/lib/rigetti/libzmq.so" :NAMESTRING "/usr/local/lib/rigetti/libzmq.so" :HANDLE NIL :DONT-SAVE NIL))
6: (SB-SYS:REOPEN-SHARED-OBJECTS)
7: (SB-IMPL::FOREIGN-REINIT)
8: (SB-IMPL::REINIT)
9: ((FLET SB-UNIX::BODY :IN SAVE-LISP-AND-DIE))
10: ((FLET "WITHOUT-INTERRUPTS-BODY-14" :IN SAVE-LISP-AND-DIE))
11: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))

unhandled condition in --disable-debugger mode, quitting

make docker-sdk is for producing a binary to extract for our downloadable SDK. It's not meant to be run on its own.

make docker-sdk is for producing a binary to extract for our downloadable SDK. It's not meant to be run on its own.

Got it. Thanks.

@ecpeterson Will do.

Pushed additional commits removing both *isa-descriptor* and *isa-descriptors* hash-table. Not sure if the latter is wanted, but it was only referenced in lookup-isa-descriptor-for-name, so I just moved the moral equivalent into that function.
It also occurred to me that I could add some tests for option parsing, piggybacking on the aforementioned with-mocked-function-definitions macro. I actually started writing some, but figured I would check here first to see if such tests are even desired.  Personally, I think the tests could be useful for validating this PR, but would likely be low-value tests going forward. Adding the tests would presumably entail copy/pasting the definition of with-mocked-function-definitions from the qvm test package.


Few 'bugs' in latest Alpha 6

If you install Hyperion Control addon and has two instances (one is adalight and the other is in my example forwarder to esp2866 device) and you disable the option for menu colors in addon, the addon disables both System | Platform Capture: (X11) when you stop video, but when you select a video only System | Platform Capture: (X11) for primary is enabled and the other one is not.

Also it would be nice to have an option to zero the stream to adalight device (send black to shut it down). For example even if menu lights are disabled and you press stop in the middle of the video. Ta last colour is then send to device the whole time. If you disable the led device and reenable it then the colors are blanked.

please use the template and DONT IGNORE IT
thanks!

@iLLiac4 On 2, see #855, when disabling the device it should go black and no updates occur unless you configured a "Refresh time".


Support streaming the download and extraction of zstd bundles

We can save time installing the bundle by streaming the download and extraction phases, so that CPU time previously spent waiting on the network can now be spent extracting what we've received so far.
Some implementation notes:

Streaming is enabled behind a feature flag.

We implement streaming only for Zstandard so that in the event of a bug with streaming, we can fallback to a download first then extract method.

The Actions toolkit HTTP client downloads the full response body, so we can't use it to stream the extraction.  Therefore, we use the lower-level Node APIs.  But these don't handle redirects, so we add a dependency on follow-redirects to avoid implementing error-prone redirect handling ourselves.

We add some performance measurements so we can see how much faster streaming is compared to downloading first then extracting.

This PR also adds some logging improvements to the setup CodeQL step, removing the verbose tar extraction output, and only echoing commands for which we're streaming the output.

Merge / deployment checklist

[ ] Confirm this change is backwards compatible with existing workflows.
[ ] Confirm the readme has been updated if necessary.
[ ] Confirm the changelog has been updated if necessary.

I think I'm running a newer version of npm on my machine which is reformatting the package.json files.


rpIdHash validation incorrectly requires rpId to exactly match origin domain

I believe the webauthn spec allows rpId to be a registrable suffix of the origin domain, but https://github.com/apowers313/fido2-lib/blob/master/lib/validator.js#L365 only allows it to be exactly equal to the client origin domain.
Maybe Fido2Lib options.rpId should be used here, and rsl might need to be checked to verify that options.rpId is a registrable suffix of clientDataJSON.origin.

Same feature is worked on here: https://github.com/webauthn-open-source/fido2-lib/pull/52 Will close here in the meantime


VSCode no longer in repo for RHEL/Fedora/CentOS

Visual Studio Code is notifying me of an update, but for over a week after running the dnf update multiple times, I haven't seen a new package come down.
I checked and sure enough the repository that is still listed here:
https://code.visualstudio.com/docs/setup/linux#_rhel-fedora-and-centos-based-distributions
No longer has vscode listed. I believe it is an oversight.

Worth mentioning, the Download link points me at the instructions above, here is the repo in question.


Added chucker dev extension to help debug network requests

Related Issue
Description
How Can It Be Tested
Screenshots (If Applicable)
Additional Comments
Checklist

[ ] New tests were added/Existing Modified

@jumaallan  Made the requested changes