[SPARK-39489][CORE] Improve event logging JsonProtocol performance by using Jackson instead of Json4s

What changes were proposed in this pull request?
This PR improves the performance of org.apache.spark.util.JsonProtocol by replacing all uses of Json4s with uses of Jackson JsonGenerator. In my benchmarking so far, I have seen ~2x speedups for reading events and ~3x speedups for writing them.
JsonProtocol is used by EventLoggingListener to emit JSON event logs capturing the firehose of SparkListener events. The history server uses ReplayListenerBus to parse the event logs and replay the event stream in order to rebuild and display a terminated Spark application's web UI.
Today, JsonProtocol uses the Json4s library's ASTs when writing and reading JSON. This existing approach was chosen because we require fine-grained control over how the JSON is generated (so the listener events themselves can't be directly passed to an ObjectMapper). JsonProtocol needs to be backwards- and forwards-compatible so that a single history server application can display UIs from multiple versions of Spark: as a result, the code has special logic for handling fields which might be missing in event logs produced from older Spark versions.
Performance profiling revealed that JsonProtocol spends a significant proportion of its time in Json4s AST calls. Much of this overhead comes from temporary Scala collections which are allocated when constructing the ASTs or when extracting fields from them.
This PR aims to improve performance by replacing all direct uses of Json4s with calls to Jackson's JsonGenerator, a low-level imperative API for generating JSON.
As a simple example, let's consider the SparkListenerApplicationStart event: this produces a JSON record like:
{"Event":"SparkListenerApplicationStart","App Name":"Spark shell","App ID":"app-20220615090229-0000","Timestamp":1655308948551,"User":"joshrosen"}

With Json4s, this is expressed as:
def applicationStartToJson(applicationStart: SparkListenerApplicationStart): JValue = {
    ("Event" -> SPARK_LISTENER_EVENT_FORMATTED_CLASS_NAMES.applicationStart) ~
    ("App Name" -> applicationStart.appName) ~
    ("App ID" -> applicationStart.appId.map(JString(_)).getOrElse(JNothing)) ~
    ("Timestamp" -> applicationStart.time) ~
    ("User" -> applicationStart.sparkUser) ~
    ("App Attempt ID" -> applicationStart.appAttemptId.map(JString(_)).getOrElse(JNothing)) ~
    ("Driver Logs" -> applicationStart.driverLogs.map(mapToJson).getOrElse(JNothing)) ~
    ("Driver Attributes" -> applicationStart.driverAttributes.map(mapToJson).getOrElse(JNothing))
  }

With Jackson JsonGenerator, this is expressed as:
def applicationStartToJson(
      applicationStart: SparkListenerApplicationStart,
      g: JsonGenerator): Unit = {
    g.writeStartObject()
    g.writeStringField("Event", SPARK_LISTENER_EVENT_FORMATTED_CLASS_NAMES.applicationStart)
    g.writeStringField("App Name", applicationStart.appName)
    applicationStart.appId.foreach(g.writeStringField("App ID", _))
    g.writeNumberField("Timestamp", applicationStart.time)
    g.writeStringField("User", applicationStart.sparkUser)
    applicationStart.appAttemptId.foreach(g.writeStringField("App Attempt ID", _))
    applicationStart.driverLogs.foreach(writeMapField("Driver Logs", _, g))
    applicationStart.driverAttributes.foreach(writeMapField("Driver Attributes", _, g))
    g.writeEndObject()
  }

The JsonGenerator code is more verbose but is significantly faster. This example is a relatively straightforward conversion, but there's a bit of trickiness in more complex cases (such as serialization of task metrics or resource profiles). I will leave comments in the GitHub PR review to highlight some of the less straightforward cases.
This is a big PR, but most of the changes were made in a semi-automated fashion with regex find-replace macros.
Why are the changes needed?
Improving JsonProtocol performance has several benefits:

Decreased history server load times, especially for long-running Spark applications.
Improved EventLoggingListener throughput: this reduces the likelihood that the will fall behind and drop events (which can lead to confusing and inconsistent Spark UIs).
Reduced resource consumption on the driver, which may improve the throughput of task scheduling.

In addition, this is a stepping-stone towards eventually being able to remove our Json4s dependency:

Today Spark uses Json4s 3.x and this causes library conflicts for end users who want to upgrade to 4.x; see https://github.com/apache/spark/pull/33630 for one example.
To completely remove Json4s we'll need to update several other parts of Spark (including code used for ML model serialization); this PR is just a first step towards that goal if we decide to pursue it.
In this PR, I continue to use Json4s in test code; I think it's fine to keep Json4s as a test-only dependency.

Does this PR introduce any user-facing change?
No.
How was this patch tested?
Correctness
There are two aspects to consider:

Ability for old HistoryServer versions to read new logs: I performed tests using randomly-generated events and confirmed that the old and new implementations produce JSON output that is character-for-character identical: see a review comment for full details.
Ability to read event logs produced by old Spark versions: I believe this is covered by backwards-compatibility tests in JsonProtocolSuite, SQLJsonProtocolSuite, and HistoryServerSuite. I looked through the git history of JsonProtocol and found a few cases where PRs that modified it did not add compatibility tests: for each of these cases, I added new test cases in JsonProtocolSuite.

Performance
So far, I have only measured performance via laptop microbenchmarks.
To measure event parsing performance, I generated a large event log by running (1 to 5).foreach(_ => spark.range(1, 100 * 1000, 1, 100 * 1000).foreach(_ => ())) then used the history server's logs to measure the time taken to load the UI. This PR's improvements cut the loading time from ~34 seconds to ~18 seconds.
To measure event logging overheads, I used YourKit asynchronous CPU profiling to to measure spark.range(1, 100 * 1000, 1, 100 * 1000).foreach(_ => ()). For this benchmark, this PR's changes cut total time in logEvent (including callees) from ~9.5 seconds to ~2.9 seconds.

Can I ask a newbie questions about version compatibility / optional field handling? The existing code is capable of being lenient in some cases and ignoring certain extra/missing fields. Does the translation from Json4s -> Jackson preserve that behavior?

Yes, it should.
There are two different types of compatibility that we need to consider:

Compatibility for reading JSON produced by old versions of Spark:

When we add a new field to an existing event type, we need the read path to treat that field as optional. Both the old and new code achieve this via the jsonOption function, which allows us to control behavior when a field is missing at read time.
There are some existing tests for this in JsonProtocolSuite. These tests remove fields from the JSON (simulating JSON generated by old versions that don't emit those fields) and tests that events still parse.
There are also some older eventlogs which are used by history server tests: https://github.com/apache/spark/tree/master/core/src/test/resources/spark-events
As long as we've been consistent about marking new fields with jsonOption, then I think that the existing tests + code review to ensure that we've preserved jsonOption should give us pretty high confidence in terms of our ability to parse old logs.

Writing event JSON which is readable by old HistoryServer versions:

JsonProtocolSuite has tests which serialize events and check that Json4s parses the generated JSON into the same tree as an expected JSON response (stored in a string literal); see assertJsonStringEquals. Right now this PR is replacing that use of Json4s with Jackson, but I will update this to check that the trees are considered equal by both Jackson and Json4s (and will leave a comment in the code explaining why we have this compatibility test).
A stronger guarantee would be to check that the Json4s and Jackson versions of JsonProtocol produce character-for-character identical output. I am going to experiment with running a test where I copy the existing Json4s version of JsonProtocol to a class named OldJsonProtocol then generate random events (likely via ScalaCheck) and check that the Json output is exactly identical. If this test passes in all cases then we have strong assurance that our new output will be compatible with old history servers. I'm not planning to check this test in because I don't want to have two versions of JsonProtocol that need to be kept in sync when making updates (and I think the risk of future changes introducing breakage which wouldn't be caught by another test is small).

I think (but will confirm) that our Json4s code path is using Jackson under the hood (behind the Json4s interfaces), which gives me further confidence that things should be compatible; if the underlying json parsers were different then we might have to worry about corner-cases in how numbers are parsed, etc., but hopefully they're using the same low-level reader.
I'll post an update here once I've had a chance to run those more extensive compatibility experiments.

+CC @shardulm94, @thejdeep

I submitted https://github.com/apache/spark/pull/37027 to fix the pre-existing JsonProtocol bugs that I found during my testing. Once that PR merges I will fix conflicts in this PR.

I've fixed merge conflicts and pushed another change to add typechecking during JSON node extraction: this doesn't matter when parsing well-formed / expected JSON, but could make a difference if invalid JSON is presented; see https://github.com/apache/spark/pull/36885#discussion_r911493733 for details.
I removed the "WIP" tag; I think this PR should now be ready for a final round of review.

Thanks everyone. I'm merging this to master (Spark 3.4.0).
(The latest test failure is unrelated / a pre-existing flaky test; my most recent commit only added a code comment)

SPARK-42403 is filed as a release blocker for Apache Spark 3.4.0.

cc @gengliangwang, @HyukjinKwon , @cloud-fan , @mridulm , too.

@dongjoon-hyun, thanks for the ping. I think that I spot the problem: it looks like the File Name field could contain a null value and the old code was likely handling JNothing -> null.asInstanceOf[String] implicitly.
I think we can fix this by either modifying extractString to handle this case or by special-casing this for FileName. I'd prefer the former approach because it's probably more robust.

Ya, actaully, I tried to relax the requirement in extractString, but it could miss other cases.
So, I'm currently proposing this one.

https://github.com/apache/spark/pull/39972

I'll convert it to a normal PR after adding a test case properly.

I like your approach as a surgical fix.
I'm trying to think through if there are any other cases where we can get null values for strings:

It can't happen for Properties because Java Properties forbids null values.
Many of the listener event string fields are things like executorId, block IDs, hostnames, etc., none of which can possibly be null at the time that the event logs are written.
We might get nulls in the ExceptionFailure TaskEndReason because an exception can have a null message.

Given this, I'm thinking that I'd prefer to fix this by changing extractString to check whether the value is textual or null.

I've submitted my proposed fix at https://github.com/apache/spark/pull/39973


Level up system

In order for the player to grow stronger, an experience system will need to be implemented which allows the character to level up, gain higher stats and progress further in the game.

To do this, we'd go about it as shown below:

Add an expGiven property to goomba.js. This will represent the value of experience awarded to the player for killing a goomba.

Within goomba.js, we create a method called kill( ), which will be called when this entity is killed (health <= 0).

The kill( ) method will have code similar to this:
var player = this.getEntitiesByType( EntityPlayer )[0];
// Award player the experience gained by killing the Goomba
player.totalExp += this.expGained;
// Decrease experience to next level
player.nextExp -= this.expGained;
// Call a level up function from main.js
ig.game.levelUp(player);

Within main.js, we need to write a function which will take the player as a parameter, and check if nextExp <=0, which signifies a level up. If this condition is true, we then want to add some values to the player's stats (def, str, mag), etc.

On future iterations, we'd like to add stats with some random modifiers to make things more interesting. We'd also like to implement some sort of experience gained function which will take in parameters such as the level of the enemy, type of enemy, etc, instead of having the player gain 50 experience at a time.

Implemented by #60.


Failures with tzdata-2022b

With update of tzdata in Fedora from 2022a to 2022b, I started to observe several test failures:
1)
Time.local timezone changes correctly adjusts the timezone change to 'CEST' on 'Europe/Amsterdam' FAILED
Expected [0, 0, 0, 16, 5, 1940, 4, 137, true, "WEST"] == [0, 40, 1, 16, 5, 1940, 4, 137, true, "CEST"]
to be truthy but was false
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/shared/local.rb:13:in `block (5 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/shared/local.rb:12:in `block (4 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/local_spec.rb:5:in `<top (required)>'
2)
Time.mktime timezone changes correctly adjusts the timezone change to 'CEST' on 'Europe/Amsterdam' FAILED
Expected [0, 0, 0, 16, 5, 1940, 4, 137, true, "WEST"] == [0, 40, 1, 16, 5, 1940, 4, 137, true, "CEST"]
to be truthy but was false
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/shared/local.rb:13:in `block (5 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/shared/local.rb:12:in `block (4 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/mktime_spec.rb:5:in `<top (required)>'
3)
Time.new timezone changes correctly adjusts the timezone change to 'CEST' on 'Europe/Amsterdam' FAILED
Expected [0, 0, 0, 16, 5, 1940, 4, 137, true, "WEST"] == [0, 40, 1, 16, 5, 1940, 4, 137, true, "CEST"]
to be truthy but was false
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/shared/local.rb:13:in `block (5 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/shared/local.rb:12:in `block (4 levels) in <top (required)>'
/builddir/build/BUILD/ruby-3.1.2/spec/ruby/core/time/new_spec.rb:11:in `<top (required)>'
Finished in 73.148502 seconds
3817 files, 30898 examples, 133657 expectations, 3 failures, 0 errors, 0 tagged

Reporting this against tzdata, I was told that this is expected change and the test needs to be fixed. Not sure if the correct change is bumping the year from 1940 => 1970 though, because it seems that 1940 was originally chosen on purpose.

Interesting, so Europe/Amsterdam in 1940 is not CEST but "WEST"?
Does the test pass by changing to 1970? PR welcome BTW.

because it seems that 1940 was originally chosen on purpose.

Because of https://github.com/ruby/spec/commit/fb2bf67c3575ec3357dc883cbe6d927930c1a53c ?
Changing the expectation to WEST wouldn't work against older tzdata, so using a different year seems more reliable.

I investigated this issue.

Because of https://github.com/ruby/spec/commit/fb2bf67c3575ec3357dc883cbe6d927930c1a53c ?

It seems that the commit above added the test cases. But according to the comment for tzdata RPM package issue ticket on Fedora, and checking the tzdata project, here is the caused commit. This is intentional. The change is new in tzdata version "2022b".
https://github.com/eggert/tz/commit/35fa37fbbb152f5dbed4fd5edfdc968e3584fe12
On ruby/ruby, I found the commit to comment out the tests. My guess is that it is a temporary workaround to pass the tests. And the "2022c" is not accurate. It should be "2022b".
The tzdata 2022c removed Amsterdam Mean Time
https://github.com/ruby/ruby/commit/b7577b4d9e0fd92522fc30e10fe712e245adee8c

Changing the expectation to WEST wouldn't work against older tzdata, so using a different year seems more reliable.

I would suggest this patch for the ruby/ruby and (ruby/spec).
diff --git a/spec/ruby/core/time/shared/local.rb b/spec/ruby/core/time/shared/local.rb
index 997b7186f1..bd33fa01ac 100644
--- a/spec/ruby/core/time/shared/local.rb
+++ b/spec/ruby/core/time/shared/local.rb
@@ -6,18 +6,18 @@
     end
   end

-=begin
   platform_is_not :windows do
     describe "timezone changes" do
       it "correctly adjusts the timezone change to 'CEST' on 'Europe/Amsterdam'" do
         with_timezone("Europe/Amsterdam") do
-          Time.send(@method, 1940, 5, 16).to_a.should ==
-            [0, 40, 1, 16, 5, 1940, 4, 137, true, "CEST"]
+          [
+            [0, 40, 1, 16, 5, 1940, 4, 137, true, "CEST"],
+            [0, 0, 0, 16, 5, 1940, 4, 137, true, "WEST"]
+          ].should include(Time.send(@method, 1940, 5, 16).to_a)
         end
       end
     end
   end
-=end
 end

 describe :time_local_10_arg, shared: true do

Could you make a PR with that?
Also would it work to change 1940 by 1970? I think that would be simpler.

Sure. I will send the PR with the 1970.

I sent the PR: https://github.com/ruby/spec/pull/939.


Idea: Support a local to-do list

Since we have a deliver local feature, even supporting running local tests, wouldn't it be cool to also have support for some sort of local/offline task list?
Let's brainstorm on this some day 🍺 or code something at the next code café.

Having your issues as code has been a long discussion @JKrag and I know @lakruzz have spoken about this for some time.. We could look into this and groom this more at the Code Cafe

I believe that GitHub already provided us a relatively easy format to persist this using the JSON of (for instance)
 curl -s -i "https://api.github.com/repos/Praqma/ghreport/issues"
We could store and maintain it locally in a .git/issues folder and then basically mimic (roughly) the features of the GHI cli, but making them operate on the local issues.
to sync with the remote we would  then add a push|pull issues git-extension.
@JKrag @groenborg

@lakruzz Files located in the .git folder is not synced and can not be added as far as I understand, right?

@groenborg
Right we would have to make that happen - hence the comment:

to sync with the remote we would then add a push|pull issues git-extension.


Withdraws marked as liquidations in Trade History

Subject of the issue
WithdrawAllMargin txs are considered liquidations by the current implementation of #833.
It is probably desired to not treat margin withdrawals als "trades" or "liquidations" ? Would need to double check also with Trade History if those are added there or not.

Txs in screenshot:
1, 2

@platschi spoke with @koredefashokun - the fix here is non impactful and only adds a new conditional check that a withdrawalAll does not trigger this liquidation creation (the conditional also applies to deposits but shouldnt matter).
this should fix the issue with only marking trade entities that are actually liquidations correctly. Just noticed this issue is still open, might be able to close it with this current issue
https://github.com/Kwenta/kwenta/issues/813


Fix colorscheme to theme matching

Fixes #1067 for real.

thanks!

FWIW this made my colors all wrong. When I revert this CL locally they return to normal. I'm using base16-ocean.
Perhaps there's something particular about my config that I need to set? (I don't set any airline options at all, just install both plugins and set my colorscheme to base16-ocean). Any debugging info I can provide?


User guide for migrating jobs from collab to dev

resolution_fixed   |    by<EMAIL_ADDRESS>
Provide a guide to existing users of accesscollab on how they can start running jobs from accessdev
Mostly how to sync home directories, at this stage work under the assumption that collab will remain available while people are still using

Issue migrated from trac:110 at 2024-01-31 17:34:49 +1100

<EMAIL_ADDRESS>changed status from new to closed

<EMAIL_ADDRESS>changed resolution from `` to fixed

<EMAIL_ADDRESS>commented

https://trac.nci.org.au/trac/access/wiki/Moving from Accesscollab


Remove empty animations

@lilleyse does gltf-pipeline already do this?
e.g.
        "animation_1": {
            "channels": [],
            "count": 16,
            "parameters": {
                "TIME": "animAccessor_0"
            },
            "samplers": {}
        },

Replaces https://github.com/KhronosGroup/glTF/issues/333

No it doesn't.
That actually goes against the spec, where minItems of channels and samplers is 1.
https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/schema/animation.schema.json

Though this used to be allowed in the 1.0 spec and just wasn't handled here.

Ah, I would say OK to close this then, thanks!


Thank you for creating this!

Just found it today on the marketplace, and really like it so far.

Thanks! I'm glad you liked it) Wonderful variants will be available soon.

Just found it today on the marketplace, and really like it so far.

Thanks! I'm glad you liked it) Wonderful variants will be available soon.


TensorFlow 1 & 2 Support

Currently importing adanet with tensorflow 2.0.0alpha gives the error 'ModuleNotFoundError: No module named 'tensorflow.contrib''
Are there plans to extend compatibility to the new TF API?

@ushmalramesh: We'll be adding TF 2.0 support slowly but surely over the next few months, but don't expect full TF 2.0 until end-of-summer '19 and once TF 2.0 is out of pre-release.

Fixed in adanet==0.8.0 release.

@cweill does it mean, that now we can test adanet with tf2?

@cweill does it mean, that now we can test adanet with tf2?

I don't think so because it seems that adanet.Estimator still depends on contrib.Head,

@lc0 Yes, you should now be able to test TF2. @shendiaomo: AdaNet currently supports the new estimator heads, such as tf.estimator.RegressionHead() and tf.estimator.BinaryClassHead().


Win10Support

Hello,
thank you very much for the tool. I wanted to test it with two Win10 in the lab and could not start it on the "victim" OS. Namely  it says that this app can't be run on that pc. Per default .net is installed but probably higher version. If I try to install .net 4.0 i get an error saying that it is already installed.
Can you help what else needs to be done in order to start it?
Thanks

.NET Framework 4.5 is pre-installed on Windows 10 so something else is blocking it.

Can it be that it is because I am using trial win version so something is not activated. Antivirus is disabled (real time) - Do you know what else can be?

What message is shown when u start it?

it is shown generic  "This app can’t run on your PC. To find the version for your pc check with your publisher" message. I disabled UAC with slider only not through GPO, I enabled real admin account and I tried to use run as admin. Also it was written that some SmartScreen is possible as a cause so I disabled that one as well. I restarted the vm to be sure that change was done and now I am getting something else - Operation did not complete successfully because the file contains a virus or potentially unwanted software. It seems that somethign from above did the trick. But still long way from the result. I will check further. Nice for you as well to know that there are some built it def features

It's most likely this SmartScreen feature, it prevents all unsigned executables from running. But there is a small clickable label called "More info". Once you click this there appears a button to ignore the warning.


Do not assume multiple births unless explicitly stated.

Describe the bug
Multiple birth was implemented such that if the multiple births integer is present and N is set as indicator, parser assumes that deceased is true, and a .multipleBirthInteger is created.
However, the desired behavior is:
BOOL || NUMBER = Answer
-----------------------------
Y + number = integer number
N + number = N
Y + blank = Y
N + blank = N
blank + number = number   << in this case assume the number is meaningful data
blank + blank = nothing

To Reproduce
Use the value for boolean | number:  `|N|16|'
Expected behavior
This should produce  "multipleBirthBoolean": false

In Release 1.0.9


i don't know what to do about this

it says me this error in node.js after I type "npm install" - "npm WARN<EMAIL_ADDRESS>No repository field."
anyone know what is this? xd, i feel a bit dumb about what i ask, but i'm really new about this
EDIT:
When i do "node index..js"
it hosts the map, but no one show up in that map, it is just like picture.

this map should working with https://github.com/jussihi/PUBG-map-hack,
just run both of them.

i already did, but still nothing.

did you load the driver and read memory successfully?

well with that i think i need help, because i'm new at this as i said and probably i don't even know what i'm doing what what should look right, if you can contact me on discord - freshz#8664

if you have problem with load&read memory, you should ask jussihi, he is the author, just follow this: https://github.com/jussihi/PUBG-map-hack/issues

got sent back here :D he said that it is nodejs side problem

lol.
when it says :
D:\somepath>node index.js
server running at localhost:7890

means the map it's running up with no error, check this.

yeah, the server is like running, but none of the players showing up on the map.

ok, and now try to debug your usermode app , found out did it send the data to the map. set some breakpoint in CURLWrapper.hpp, and run it up.

it says that windows sdk version (...} was not found, but i already got it installed

so it's going into kernel driver part... shame, I can't compile the driver too.  that's why I told you to ask jussihi.

but I have to debug right that whole usermode app folder or what, like I'm in confusion right now haha

you need the driver, usermode app and the map.  both of them.  if any one of them got some error,  the whole system could't working. good luck


Replace static deployment files with template yamls

Use yaml templates to handle app configs
Added checksum annotation for handle pod restarts on config change

Tested by deploying to local k8s, didn't test backend integrity as I didn't host a db
Will need to update deployment values before merging
refs #265

@rickmak updated, tested frontend by deploying to local k8s

Merged and pushed, need to update the likedao-deploy repos.


[#251] fix PscStream read_stream company_number regex

Otherwise, company numbers such as SC808869 are not detected, since they do not contain only digits.
References https://github.com/openownership/register/issues/251 , during which it was discovered as part of Transformer PSC streaming work.

@StephenAbbott, I think this could cause some streamed statements to be missed. I didn't notice the bug when I checked the existing regex. We'll have to fill in the gaps via a bulk import, which we were planning to do anyway.

References https://github.com/openownership/register/issues/244 .


Browserview settings window

Please provide the following information:
Summary
This PR is mainly for ensuring the settings page works as expected, as a result some other parts have improved. But some effects on the main page doesn't still happen (like changing the tabs in the main window when adding/removing a server, although server is there and attached)
for now use the settings page to move through tabs
Additional Notes
It is based on previous PR work, please review that other before this one. once that one is merged, changes will go down by a lot :)

PR updated (don't really get how it ended with merge conflicts if it was based from the previous branch)


Use zero-copy DuckDB vector conversion for primitive types.

This patch uses BufferView class to zero-copy convert DuckDB vector into FlatVector. The patch works only for primitive types where a simple copy was previously used.

@ienkovich Thank you, Ilya. There is a typo in the PR description: VectorView -> BufferView. And, also in PR title: trivial -> primitive. Do we have any tests running in CI that go through this code path?

Thanks! Fixed. This code path is tested by Duck wrapper tests and Parquet reader tests.

LGTM.
Should be merged latest by tomorrow.


Mat/Pat - Employee Information Validations

Tab -  Employee Information
AC

[x] Name Ministry and Email field values is present (testing only the critical fields)
[x] "Incorrect or missing information can cause delays in the processing of your leave application" can be left empty and next tab "Alternate contact information" should show up

Note
"Incorrect or missing information can cause delays in the processing of your leave application" is mandatory only on form submission so will test that as part of another ticket

script - https://bcgov.sharepoint.com/:u:/r/teams/03991/Shared Documents/General/Selenium/Selenium Scripts/Maternity Parental L%26A/MaternityEmployeeInfo_1886.side?csf=1&web=1&e=gt56fc

Working great - thanks Fazil!

Perfect - Good to go PO A = closed


Simplify actions in Edit Playlist

Users don't understand the ... symbol

Closed by pull request #169


Add some httpbin-like functionality to web component tester

Enables iron-ajax tests to be standalone and not hit the third party httpbin.org service.

LGTM with changes to use the wct logging system.


[$1000] Suggestion text below company address is not visible when we don't select any suggested option but it is visible in every other cases

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open the app
Open Workspaces
Open any workspace
Open connect bank account
Click on connect manually
Fill in step 1 information (eg data: routing number = 210021002, acc no = 1234)
In company address, type in address but don't select any option and after typing, select city below
Observe that suggestion text below company address is gone
Fill in other details, click on continue and click on back and obserce that suggestion text below company address is back
Click on back few times and click on startover, again fill in step 6 details and observe that company address has suggestion text below it
Type in address and select any one suggestion and observe that company address has suggestion text below it

Expected Result:
App should display suggestion text below company address if user decides to fill it on their own and not use the suggested addresses
Actual Result:
App does not display suggestion text below company address if user decides to fill it on their own and if suggestions were still displayed before user tries to move to fill in other details
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

Version Number: 1.2.88-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:
https://user-images.githubusercontent.com/43996225/227250842-bc253546-6f19-4bd5-97b1-2778e8171123.mp4
https://user-images.githubusercontent.com/43996225/227250929-6267a11f-4548-4ddf-be78-3398e62e9764.mp4
Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1679566976110789
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~018ed613b675853bc5
Upwork Job ID:<PHONE_NUMBER>190702592
Last Price Increase: 2023-03-30

Triggered auto assignment to @zanyrenney (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

I'm not sure if this constitutes a bug, so need to check on this and bring a conversation to bug0.

I guess looking at this again, it is inconsistent so makes sense to progress forward here.

Job added to Upwork: https://www.upwork.com/jobs/~018ed613b675853bc5

Current assignee @zanyrenney is eligible for the External assigner, not assigning anyone new.

Triggered auto assignment to Contributor-plus team member for initial proposal review - @eVoloshchak (External)

Triggered auto assignment to @iwiznia (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

Thanks for your proposal @Prince-Mendiratta

Proposal
Please re-state the problem that we are trying to solve in this issue.
Suggestion text not toggles and shows once we move focus to the other elements without selecting/pressing any suggested option
What is the root cause of that problem?
Currently, onPress of the option from the GooglePlacesAutocomplete we are setting the displayListViewBorder to false which shows the hint text. But on blur of the textInput of GooglePlacesAutocomplete textInputProps we are not setting displayListViewBorder to false which causes this issue.
What changes do you think we should make in order to solve the problem?
We need to update the onBlur inside textInputProps of the GooglePlacesAutocomplete with
onBlur: () => setDisplayListViewBorder(false),
https://github.com/Expensify/App/blob/b091e3c747dc39e9642c65199ee31d46b09d81ef/src/components/AddressSearch.js#L213-L236
We are not passing onBlur from the parent for any occurrences so we can take out props.onBlur.

Proposal
Please re-state the problem that we are trying to solve in this issue.
App does not display hint text under the company address is the user click outside of the dropdown without selecting an option.
What is the root cause of that problem?
In this line https://github.com/Expensify/App/blob/4ee21e95a38858df28e46caf54dc97e1bca9bc7a/src/components/AddressSearch.js#L236, the hint is only displayed if the displayListViewBorder is false.
The displayListViewBorder is set to false if the user selects an option, but is not if the user blurs out of the GooglePlacesAutocomplete.
What changes do you think we should make in order to solve the problem?

We need to set the displayListViewBorder to false inside InteractionManager.runAfterInteractions when the user blurs the address search input and the option list.
This can be done by updating this line https://github.com/Expensify/App/blob/4ee21e95a38858df28e46caf54dc97e1bca9bc7a/src/components/AddressSearch.js#L241 to
onBlur: (event) => {
    if (containerRef.current && event.relatedTarget && !containerRef.current.contains(event.relatedTarget)) {
        setDisplayListViewBorder(false);
    }
    props.onBlur(event);
}

The containerRef is the ref of the address search container (could be this one) that we can set via useRef.
The related target check is required here because without it, when we select an option, the onBlur will still trigger, setting displayListViewBorder to false will make the auto complete component re-render before onPress is called, making selecting an option not working.
Currently there's a bug in our react-native-google-places-autocomplete fork here https://github.com/Expensify/react-native-google-places-autocomplete/blob/e12768f1542e7982d90f6449798f0d6b7f18f192/GooglePlacesAutocomplete.js#L860 that is not passing the event to onBlur, we need to fix that as well. The main react-native-google-places-autocomplete lib already fixed is so we can just update our fork to the latest
What alternative solutions did you explore? (Optional)

We can wrap the setDisplayListViewBorder(false); inside InteractionManager.runAfterInteractions as well instead of using related target check
The relatedTarget check can be added upstream in the library instead of in our code

Result
Working well after the fix:
https://user-images.githubusercontent.com/113963320/229275934-9e6971a6-6c5e-4688-9716-afb4be087812.mov

@iwiznia, @eVoloshchak, @zanyrenney Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

@eVoloshchak can you review the proposals above please?

@Prince-Mendiratta's and @Pujan92's proposals use the same approach: call setDisplayListViewBorder(false) when GooglePlacesAutocomplete's input is blurred. They both resolve our issue, but as @tienifr's pointed out in their proposal, there is a bug: when we select an option, the onBlur will still trigger, setting displayListViewBorder to false will make the auto-complete component re-render before onPress is called, making selecting an option not working. So essentially you can't select an address
@tienifr's proposal addresses this issue.

We need to set the displayListViewBorder to false inside InteractionManager.runAfterInteractions when the user blurs the address search input and the option list.

Not sure I fully understand, do you propose we use both InteractionManager.runAfterInteractions and relatedTarget check or just one of these?

@eVoloshchak sorry for the confusion, either one of these will fix the issue, but I prefer to use the relatedTarget check

Proposal looks good

Hi @eVoloshchak @iwiznia We need to update react-native-google-places-autocomplete in our fork before. The PR on react-native-google-places-autocomplete fork is ready for review

@iwiznia Friendly bump, could you take a look at 2 PRs on this issue? Thanks

Reviewed the fork and posted in slack to see how we handle the unsigned commits

Sounds good. Putting on hold now.

@iwiznia, @eVoloshchak, @zanyrenney, @tienifr Whoops! This issue is 2 days overdue. Let's get this updated quick!

Not overdue
We should remove the signed commits requirement in all of the forks soon, which would allow us to merge the PR for this issue

This shouldn't be on HOLD anymore, the first PR was merged, only https://github.com/Expensify/App/pull/16875 is left, currently in review

Reviewing label has been removed, please complete the "BugZero Checklist".

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.3.6-0 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/16875

If no regressions arise, payment will be issued on 2023-05-04. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[ ] [@eVoloshchak] The PR that introduced the bug has been identified. Link to the PR:
[ ] [@eVoloshchak] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[ ] [@eVoloshchak] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[ ] [@eVoloshchak] Determine if we should create a regression test for this bug.
[ ] [@eVoloshchak] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
[ ] [@zanyrenney] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

Hi @zanyrenney , @iwiznia , @eVoloshchak ping for payment.

I was OOO. Catching up now

Currently having an issue on Upwork with the Job posting:

The PR that introduced the bug has been identified. Link to the PR: https://github.com/Expensify/App/pull/9207

The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: https://github.com/Expensify/App/pull/9207#issuecomment-1540489201

A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: I don't think additional discussion is needed, this is a visual bug, there is nothing we can do besides automated UI tests

Determine if we should create a regression test for this bug.
Is it easy to test for this bug? Yes
Is the bug related to an important user flow? (For example, adding a bank account) Yes
Is it an impactful bug? No
The bug has a very low impact, but it's easy to test and is related to connect bank account flow. I think we might want to add a simple regression test

Regression Test Proposal

Open Workspaces > select any workspace > connect bank account
Click on connect manually
Fill in step 1 information (eg data: routing number = 210021002, acc no = 1234)
In the company address, type in address but don't select any option, wait for the list with addresses to load
Click outside the list (ex: click on the city field)
Verify that the suggestion text below the company address is displayed

Do we agree 👍 or 👎

At Accountex so won't get to this regression test + payout for a few days, reassiging.

Cool I'll get to this today!

@sophiepintoraetz Sorry to bring you in on this too Sophie! I am OoO until Monday and in the interest of these payments being overdue, I wanted to make sure they didn't have to wait until then.
Once you and @iwiznia have agreed if the bonuses are due, then the contracts here can be paid. I've already paid the issue reporter!

Yep, agree on bonuses

Ionatan is speedy, so I'm doing the payments. Wrap-up steps are yours, Sophie!

Payments done

QA steps requested here.


Hendrycks Math extraction rule seems too strict

Hi thanks for the library! It seems that the way how math answer is extracted, i.e. https://github.com/EleutherAI/lm-evaluation-harness/blob/bcb4cbf464668450f1969a4f1bef28f8638d1648/lm_eval/tasks/hendrycks_math/utils.py#L20-L24, may be too strict.
For example, the following answer: ... some reasoning logic ... Thus the answer is \[ \boxed{42} \] is not extracted, because it is not a $.

Hi! This is based on the original code, but we can add another metric to show alongside by adding a flexible-extract filter, as gsm8k does it. PR welcome!


Adds support for type slicing in Entity deserialisation.

Example of type slicing: a particle writes Person {age, height} to a handle, and another particle reads Person {age}. Previously this would throw an exception when deserialising, because height is not defined. Now unknown fields in the RawEntity will now be silently ignored when deserialising. This is a slight hack, but the behaviour is consistent with what we do for wasm.

cl/308174642


Style/MethodCallWithoutArgsParentheses should have an IgnoredMethods setting

The Style/MethodCallWithArgsParentheses cop allows some methods to be ignored using the IgnoredMethods setting. Oddly, the Style/MethodCallWithoutArgsParentheses cop does not.

Expected behavior
Given the following code
s()

If I add the following to .rubocop.yml:
Style/MethodCallWithoutArgsParentheses:
  IgnoredMethods: ['s']

I expect RuboCop not to report an offense for Style/MethodCallWithoutArgsParentheses.
Actual behavior
RuboCop complains about the parameter and reports an offense for Style/MethodCallWithoutArgsParentheses:
Warning: unrecognized parameter Style/MethodCallWithoutArgsParentheses:IgnoredMethods found in /home/matijs/Projects/ripper_ruby_parser/.rubocop.yml
Inspecting 1 file
C

Offenses:

foo.rb:1:2: C: Do not use parentheses for method calls with no arguments.
s()
 ^

Steps to reproduce the problem
Create a file foo.rb with:
s()

And a file .rubocop.yml with:
Style/MethodCallWithoutArgsParentheses:
  IgnoredMethods: ['s']

Run RuboCop:
rubocop foo.rb

RuboCop version
$ rubocop -V
0.50.0 (using Parser <IP_ADDRESS>, running on ruby 2.4.1 x86_64-linux)

Good point. 🙂 It's also a bit unfortunate the author went with IgnoredMethods, because I've been trying to find a good abstraction for Whitelist/Blacklist. 😅

Good point. 🙂 It's also a bit unfortunate the author went with IgnoredMethods, because I've been trying to find a good abstraction for Whitelist/Blacklist. 😅

It's not like we can't change this. ;-)

There are unfortunately a ton of inconsistencies from the early days of the project.

Indeed. Another one I've noticed is the varied use of Unnecessary, Useless, Unneeded. (Probably influenced on how frustrated the author was at the offense at the time of writing. 😛)

Works like a charm, thanks!


Changing of underlying type broke pulumi-aws elasticache.Cluster snapshotArn

As part of the pulumi-aws v4.0 upgrade, we took onboard a breaking change from the upstream TF provider. This changed the aws.elasticache.Cluster to ensure that snapshotArns changes from multiple items to 1 item
This means that we went snapshotArns being a []string to snapshotArns being a string
We now get the following error for users:
error: unmarshaling urn:pulumi:production::at-platform::aws:elasticache/cluster:Cluster::platform's instance state: could not read field snapshot_arns: '' expected type 'string', got unconvertible type '[]interface {}', value: '[]'

This feels we either need to have a migration in place that handles this (so support for migrations) or that we have a bug in our bridge that doesn't handle the change in state as part of the diff

This changed the aws.elasticache.Cluster to ensure that snapshotArns changes from multiple items to 1 item

In general, I believe any change like this should be flagged as a breaking change, and then in general we should decide to override MaxItemsOne: boolRef(false), in our layer to force this back to being an array for Pulumi users, until we are willing to take a breaking change release to upgrade.

need to have a migration in place that handles this (so support for migrations)

To enable the breaking change when we do want to take it - we do need to have the ability to allow this to work.  We should likely open a separate tracking issue on adding that general feature.

@lukehoban this was added as part of a major release of the provider and noted in the changelog

We just ran into this issue today. Is there any guidance here other than having to manually fix the stack using export > import as mentioned like what is mentioned here: https://github.com/pulumi/pulumi-aws/issues/1458#issuecomment-825333911
This is a potential blocker for us upgrading to 3.x as well.


Allow to switch the Open Prices endpoint in Dev Mode

Problem

Allow to switch the Open Prices endpoint in Dev Mode
Add a system similar to the main Product Opener url switching

why

we're rewriting the backend

Part of

#525

@monsieurtanuki @g123k We're thinking of migrating in the next few days.

@teolemon I should be able to code the issue today.


Is there anyway to make selected aria fixed ?

Hi fengyuanchen , I wanna ask you how I can make the select aria fixed ? like facebook when you upload an profile picture , same issue ? thanks

Are you mean this: https://github.com/fengyuanchen/cropperjs/blob/master/examples/fixed-crop-box.html


Outdated version on openblas.net

The latest 0.3.28 release does not appear on http://www.openblas.net/.

yes, I know, I'm visiting family this weekend


Stabilize Real User Monitoring Spans and Attributes

https://github.com/signalfx/gdi-specification/blob/main/specification/semantic_conventions.md#real-user-monitoring-spans-and-attributes needs to be stable for https://github.com/signalfx/gdi-specification/issues/240

@t2t2 Can you please take this issue?


allow posting failed and successful encounters

Is your feature request related to a problem? Please describe.
Currently the scripts do not publish failed encounters.
Describe the solution you'd like
Add configuration which allows posting failed encounters in addition to the successful ones.
Describe alternatives you've considered
A possible alternative is the manual interactive script for cherry-picking which encounters you want to publish. This may be suitable for some advanced users, but may not meet the needs of someone who just wants to publish everything.

Preferably, this should be done by changing the current global configurations for uploading into configurations per guild which control both uploading and formatting.

This is supported in the v3.0.0 release. You can optionally configure this for each guild


List items in bucket

A handy feature would be the ability to list objects. I was thinking maybe if on a get request a file isn't found, try list objects with the the path as the prefix.
I can do a PR if you think this is a good idea.

Your ideas sound great!
I think it's important to have every directory be listed by default, as the vibe I'm getting from this project is "a super easy way to publicly expose r2", and have options to enable checking for .list or not to list directories by default.
Parsing the accept header sounds good. I would much rather add an extra header to my fetch() rather than have to parse any sort of HTML. If we include the option to return JSON, would you still want the HTML to look like nginx or maybe something nicer?

I think it's important to have every directory be listed by default, as the vibe I'm getting from this project is "a super easy way to publicly expose r2", and have options to enable checking for .list or not to list directories by default.

That sounds good, I think if listing is enabled (so DIRECTORY_LISTING = true under [vars] in wrangler.toml or similar), it should check for /prefix/.nolist, otherwise it disables listing completely (or checks for .list, up to you).
I think the HTML should be relatively simple, preferably no external resources should be loaded and no JS. But it can look nice, as long as there isn't like 1000 lines of HTML or something. I'm not sure what to expose, though. Ideally there isn't a HEAD request for every single file in the directory, but I'll have to see what metadata is exposed in list.
It would be awesome if you want to send a PR for this, even if it only implemented a fraction of this. If not, let me know and I'll gladly do it myself. Thanks for the feedback!

Sure! I'll get it done when I have time.
I'm just thinking that .nolist and .list would be features that won't be used often, but also have the downside of slowing down requests because of the extra head request. Maybe it could be optional?

I'm just thinking that .nolist and .list would be features that won't be used often, but also have the downside of slowing down requests because of the extra head request. Maybe it could be optional?

You're right, maybe we don't need it. I'll try to think of an alternative in the meantime. Maybe an object with the same name as the directory? Since we have to check for it anyway.

An idea I had was using the VS code icon set, which has icons for each file extension or folder name. What do you think?
We could host the icons in a Cloudflare Pages project and link to them when the list page is generated.

Hmm, I'm not sure if that would be used often, though. Do people typically host code, etc under S3 buckets? It seems like a lot of the icons in the bundle wouldn't be used.
I would prefer simpler icons, like Chalice, and not have any external dependency on another Pages project, etc.
Basically, for accessibility I want to be able to view it through lynx easily.
For example, making it look good with sakura would be better than something like a full-blown vue application.

What do you think of this design?

The icon pack is Feather Icons.

Hmm, I'm not sure if that would be used often, though. Do people typically host code, etc under S3 buckets? It seems like a lot of the icons in the bundle wouldn't be used.

Very true. The VS code icon set does have a wide range of icons for all sorts of files, but there's also way too many icons for code.

I would prefer simpler icons, like Chalice, and not have any external dependency on another Pages project, etc.

Yeah, you're right, everything should be inside this repo. Something like Chalice would be nice, but common formats for S3 buckets like video, archives, and audio would all be just "other" and have the same icon... Feather Icons as suggested by @zadeviggers may be promising?

Basically, for accessibility I want to be able to view it through lynx easily.
For example, making it look good with sakura would be better than something like a full-blown vue application.

Cool, I'll keep that in mind. I'm just using a giant JS template string right now. The one zadeviggers made looks good though.

@zadeviggers What do you think of this design? It's just html and css. I've also written a little function that takes a list of files and breadcrumbs and generates the html, for use in the worker.  The icon pack is Feather Icons.

This looks good! One personal preference, but not a requirement, is if the layout were from top-to-down then left-to-right, because that's usually how things are displayed, I think.

@ben476 Something like Chalice would be nice, but common formats for S3 buckets like video, archives, and audio would all be just "other" and have the same icon... Feather Icons as suggested by @zadeviggers may be promising?

Yup, I didn't think of that, I think Feather would be good if we have all those icons. We should categorize with content-type rather than filenames if possible, but if they aren't returned in a list operation, we might have to use filenames.

@ben476 Cool, I'll keep that in mind. I'm just using a giant JS template string right now. The one zadeviggers made looks good though.

:+1: Sounds good!

The way Caddy does it is a browse.html that gets templated. Maybe this would be a better solution?

Added support for some basic listing for now in #20, looks good to me so closing!
Will version bump and release shortly.

Released 1.0.0 in 52a7521! 🎉


Make build folder of adapter-node self-sufficient

Currently you can't just take the content of the build folder to run the server in production.
The server won't start without an existing package.json file which is configured with type: "module".
This PR creates a package.json file in the build folder.
To avoid errors after starting the server because if dependencies which should be in dependencies instead of devDependencies of package,json, this PR also extends the build process to create a production ready package.json file.
Can't you just use the project's package.json and lockfile? Why do you need to generate a new one?
I believe the compiler is smarter than I am, so I let it decide which packages should be dependencies in a production package.json.
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[ ] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[x] This message body should clearly illustrate what problems it solves.
[ ] Ideally, include a test that fails without this PR but passes with it.

Tests

[ ] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[ ] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. All changesets should be patch until SvelteKit 1.0

Thank you. This is very adapter-node specific, so I don't think we need to add a new generatePackageJson API, the logic should be contained within that adapter.
Having said that, this makes me nervous. Putting a generated package.json in the output folder without a lockfile means that the production build will end up using different versions of packages, which is a non-starter. But you can't just copy the lockfile, because it won't match the generated package.json.
I think it's better if people are responsible for including/excluding files themselves.

One option could be to fix the currently installed versions in the generated package.json. Then there are no surprises. It will be the same versions installed in production that you had when you developed it.
Otherwise I would argue, that just copying the package.json from the project is the same as letting people include/exclude files themselves, since this is what most of them (from my experiences on the discord) do. They copy the package.json and then get surprised if everything explodes on the next update. At least it is consistent.

Can't you just use the project's package.json and lockfile? Why do you need to generate a new one? (The answers to questions like these would be good to include in the PR description and code comments)

I believe the compiler is smarter than I am, so I let it decide which packages should be dependencies in a production package.json.

One option could be to fix the currently installed versions in the generated package.json. Then there are no surprises. It will be the same versions installed in production that you had when you developed it.

This works fine for your immediate dependencies. It doesn't work for their dependencies — that's the problem lockfiles solve.

Can't you just use the project's package.json and lockfile?

This seems like a good approach to me. I'm not sure if there are any drawbacks — obviously the file will include stuff that doesn't make any sense (like a "dev" script, etc) but if it's only being used to determine which dependencies to install then that's probably fine.

I believe the compiler is smarter than I am, so I let it decide which packages should be dependencies in a production package.json.

You'd be surprised! Giving users the ability to control which dependencies are included in a production build (by putting them in pkg.dependencies instead of pkg.devDependencies) is easier to understand, more consistent with the logic that determines what stuff gets bundled, and offers an escape hatch for when someone inevitably hits a bizarre edge case.

Using the same package.json and lockfile sounds like the right path to me. Sure it'll have stuff in it you don't need, but those will be just lines in a text file. You should be installing dependencies with npm ci --prod when you're preparing your prod build. If you're using Docker, for example, you should be using a multi stage build so that you install all dependencies to build the app and then just install prod dependencies in the final stage and copy over the contents of build.

One option could be to fix the currently installed versions in the generated package.json. Then there are no surprises. It will be the same versions installed in production that you had when you developed it.

This works fine for your immediate dependencies. It doesn't work for their dependencies — that's the problem lockfiles solve.

true

Can't you just use the project's package.json and lockfile?

This seems like a good approach to me. I'm not sure if there are any drawbacks — obviously the file will include stuff that doesn't make any sense (like a "dev" script, etc) but if it's only being used to determine which dependencies to install then that's probably fine.

I believe the compiler is smarter than I am, so I let it decide which packages should be dependencies in a production package.json.

You'd be surprised! Giving users the ability to control which dependencies are included in a production build (by putting them in pkg.dependencies instead of pkg.devDependencies) is easier to understand, more consistent with the logic that determines what stuff gets bundled, and offers an escape hatch for when someone inevitably hits a bizarre edge case.

If you don't know how to decide which goes where you build your app and try to run it. You hit the first missing dependency, you update package.json, then try again. Rinse and repeat. In my project I had 7 such dependencies. Then you copy everything on the server and try to run it. Then you hit the missing package.json issue. This is very frustrating.
How about making it optional via adapter configuaration parameter.
Another approach could be svelte-kit sync or svelte-check to hint that some packages should be moved.

The reason why I generated the package.json file comes from the question which packages go where? Is a specific package a devDependency or is it a prod dependency? How do you decide? This question was asked a lot of times on discord. If you don't know you may put everything in devDependencies and sort it out when you want to deploy.
If you don't know what to look for after building you may end up in the rinse and repeat cycle described earlier. Since you put everything into devDependencies (or you missed something in dependencies), some dependencies may not have been bundled (I also did an issue earlier #5849) .
If you know what to look for you can build once and then search for imports and find those which don't import from a relative path. Relative path import means the dependency has been bundled. Some dependencies can't be bundled so they are imported from node_modules.
Again, how do you decide which package is a prod dependency? One Answer was: "When you need it in production it is a prod dependency.". May work, but may not be completely true. You may not miss any packages, but you may add more packages into prod dependencies then necessary. Which is fine I guess.
From my observations everything that is not treeshakable is not bundled. So if it's esm, it is usually bundled.
 The compiler does the bundling, so it knows what can be bundled and what can't. I let it decide and generate the package.json accordingly.

I'm sorry, I still don't follow. The logic is very simple: if something is a prod dependency, it will be excluded from the generated bundle, and must be installed alongside your deployment: https://github.com/sveltejs/kit/blob/9dbcce994161ec7191a8a8ea5a510437232dd1ce/packages/adapter-node/index.js#L52-L62
This is described in the README: https://github.com/sveltejs/kit/tree/master/packages/adapter-node#deploying

some dependencies may not have been bundled

In what circumstance? If that's happening, it's a very strange bug and it needs a reproduction.

The problem I'm trying to solve starts earlier.
Let me give you an example. These are your dependencies. Which one is a prod dependency and which one is devDependency?
"@felte/reporter-svelte": "1.1.2",
"@felte/validator-vest": "1.0.9",
"@playwright/test": "1.25.0",
"@prisma/client": "4.2.1",
"@sveltejs/adapter-node": "1.0.0-next.89",
"@sveltejs/kit": "1.0.0-next.480",
"@tailwindcss/forms": "0.5.2",
"@tailwindcss/typography": "0.5.4",
"@types/nodemailer": "6.4.5",
"@typescript-eslint/eslint-plugin": "5.33.0",
"@typescript-eslint/parser": "5.33.0",
"autoprefixer": "10.4.8",
"eslint": "8.21.0",
"eslint-config-prettier": "8.5.0",
"eslint-plugin-svelte3": "4.0.0",
"felte": "1.2.2",
"form-data": "4.0.0",
"got": "12.3.1",
"mdsvex": "0.10.6",
"nodemailer": "6.7.7",
"paseto": "3.1.0",
"postcss": "8.4.16",
"postcss-load-config": "4.0.1",
"prettier": "2.7.1",
"prettier-plugin-svelte": "2.7.0",
"promise-retry": "2.0.1",
"soap": "0.45.0",
"svelte": "3.49.0",
"svelte-check": "2.8.0",
"svelte-preprocess": "4.10.7",
"tailwindcss": "3.1.8",
"tslib": "2.4.0",
"typescript": "4.7.4",
"vest": "4.6.2",
"vite": "3.1.0"

@tylkomat just make everything a dev dependency and be done with it

@benmccann This is exactly what I'm trying to explain. If you do that you'll end up in the rinse and repeat cycle I described earlier. At least you did before #6301 was merged. Nonetheless you'll end up with 7 dependencies that are not bundled.
I created a stripped down example of the problem here, which just includes 3 of the dependencies.
It works fine in dev mode. When you build it the server starts fine.
Before #6301 the server wouldn't start because dependencies where missing, I also can provide an example for that.
Now the server starts fine, but when you access the page it explodes, which is even worse since you now expect everything to be fine. Also the error doesn't help you much to identify what the issue could be.
The server is build as a sibling to the source folder to simulate copying the content of the build folder to a server.
In a second step you have to copy the package.json to the build folder (../no-bundling-prod), but don't run npm install, since we're expecting the dependencies to be bundled.

The issue here isn't that dependencies aren't being bundled, it's that dependencies that shouldn't be bundled (because they're not compatible with esbuild in esm mode) are being bundled. Ultimately this is a bug in esbuild (https://github.com/evanw/esbuild/issues/1921) — there's no reason why require('util') should be turned into __require('util'), given that it's a built-in module — but it's something we have to deal with for now.
This PR (if we ignore the complexity around lockfiles etc) would change that by making everything external by default. But that's undesirable since it increases the size of deployments.
I think the best solution might be to work around https://github.com/evanw/esbuild/issues/1921, perhaps by generating CommonJS output instead of ESM. I've opened a PR that implements that idea: #6855

Thanks @Rich-Harris. Looks like #6896 solved it.


Hermes WeakRef Support in React Native

Description
Hermes has support for WeakRefs, but we're missing the integration code on React Native to properly enable.
From Original Issue: https://github.com/react-native-community/discussions-and-proposals/issues/741

From https://github.com/facebook/hermes/issues/658 and https://github.com/facebook/hermes/issues/1195, it seems that this requires action on the React Native side (enabling the microtask-queue flag for Hermes and then routinely calling drainMicrotasks).

@cortinico I think<EMAIL_ADDRESS>which was released yesterday is affected by this: https://github.com/apollographql/apollo-client/issues/11543
They introduced the toBeGarbageCollected.ts which uses WeakRef: https://github.com/apollographql/apollo-client/commit/7d939f80fbc2c419c58a6c55b6a35ee7474d0379#diff-83faa655149377a65d05a36458355e0a84925b36d1ebe017f68196c27a369670

Generally, @apollo/client in all versions is impacted by this - we use WeakRefs for our internal LRU WeakCache implementation and in React Native we have to fall back to default behaviour which is a lot more memory-consuming - things stay in memory even if they aren't used in the rest of the app until the cache will reach it's maximum size and things will be cache-collected because of that.
It still works, and it's not growing indefinitely, but it could be a lot better (like on every other platform) if WeakRef were available.

Is there any updates on this?

Is there any updates on this?

Nope we'll be sharing updates here once we have them

This is fixed by #43396, which was just merged. Please take into account that this will only work in the new RN architecture.

This is fixed by #43396, which was just merged. Please take into account that this will only work in the new RN architecture.

This is fixed by #43396, which was just merged. Please take into account that this will only work in the new RN architecture.

@rubennorte Awesome news! So I'm using constructs like WeakRef and FinalizationRegistry inside my new-arch RN project, basically doing nothing. Can you please tell me how to use this new functionality you say it was implemented?

You'd need to use RN from the main branch because this was just merged. I can reopen the issue and close it when we release 0.75 (which is going to be the version this is shipped in).
If you want to test it now, you can clone the repository and build the RNTester app :)

@rubennorte If I need this functionality before 0.75, do you think it's feasible to have them exposed through JSI (i.e. WeakRef, FinalizationRegistry) or I'll bump into some blockers?

@rubennorte Thank you. If I need this functionality before 0.75, do you think it's feasible to have them exposed through JSI (i.e. WeakRef, FinalizationRegistry) or I'll bump into some blockers?
Incidentally, do you know the estimated release date for 0.75 or how can I access such a roadmap?

Unfortunately, this depends on having support for microtasks in Hermes, which is only supported with the new event loop that's only on main. You'll have to wait for 0.75. The branch cut for 0.74 was done recently, so this might take a while (possibly end of Q2 or beginning of Q3).

@rubennorte, will this new work on a new event loop give us also a solution for long-running background C++ callbacks that require access to the JSI runtime object? I'm solving this by continuously polling for a new JSI runtime, the same as N-API (which hits hard on the battery).

@rubennorte, will this new work on a new event loop give us also a solution for long-running background C++ callbacks that require access to the JSI runtime object? I'm solving this by continuously polling for a new JSI runtime, the same as N-API (which hits hard on the battery).

Sorry I don't have context about that. Can you share more details?

Will FinalizationRegistry be supported as well or only WeakRefs get supported?

Will FinalizationRegistry be supported as well or only WeakRefs get supported?

That's a question for Hermes, but I think FinalizationRegistry isn't supported yet.

How can we enabled WeakRef for the legacy architecture too? I get ReferenceError: Property 'WeakRef' doesn't exist, js engine: hermes (RN 0.75, old architecture).

How can we enable WeakRef for the legacy architecture too? I get ReferenceError: Property 'WeakRef' doesn't exist, js engine: hermes (RN 0.75, old architecture).

Unfortunately it's not possible, as WeakRef require support for microtasks, which is only implemented in the new architecture.

@rubennorte, is it planned to be implemented for the legacy arch too or we hope that everyone will eventually switch to the new one? I'm curious if you know any strategy to mock it in the meantime or if we should just accept the memory handling constrains of it.

@rubennorte, is it planned to be implemented for the legacy arch too or we hope that everyone will eventually switch to the new one? I'm curious if you know any strategy to polyfill it in the meantime or if we should just accept the memory issues of mocking it.

It would require a lot of work to support it in the legacy architecture because there are a lot of missing pieces, so we're not going to do that.
AFAIK there's no way to polyfill it in a way that retains its semantics (in other words, you'll have to keep things in memory).


Update version: Gyan.FFmpeg version 6.0

Pull request has been created with WinGet Automation :rocket:
Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/114829&drop=dogfoodAlpha

@wingetbot run


[99] feat: Header

Closes #99
Ignored Desktop user menu icon change for Desktop.

this is a good start! I like how you added the hamburger menu (yay responsive!).
some other things

in the meet the mangrove team mentioned they wanted the scroll to top button on the header, as well as a 'next' button for the forms page. I believe Al mocked this up
could you also add the fixed footer to this pr?

We captured that in the standup. In general we shouldnt scope creep in PRs, just make new tickets for new features. :) (Good catch for sure on the form nav component though, I didnt even notice it was a thing)


should be able to search file namespace by id too

https://{{apic}}:{{port}}/api/v1/file/namespace/config?id=3085587b-d226-4abd-8509-3b7756285165
{
"response": [
{
"nameSpace": "config",
"name": "2960-client.txt",
"downloadPath": "/file/3085587b-d226-4abd-8509-3b7756285165",
"fileSize": "512",
"fileFormat": "text/plain",
"md5Checksum": "c905b02d675babf2f954ba485cc244f1",
"sha1Checksum": "f72b936d02c0eece173179da516c78dfd3045f2f",
"id": "3085587b-d226-4abd-8509-3b7756285165"
}
],
"version": "1.0"
}
Same as earlier issue sort by name

Same as #17

@aradford123 as @LiFengPro mentioned this would be a documentation bug from APIC-EM's perspective if it does not list all the available API endpoints.
Marking this issue as invalid till then.


Create CODE_OF_CONDUCT.md

@ChakibChemso No plans to add this kind of document at this moment. Actually, it was added and later removed some years ago.

I have no opinion and defer to raysan5 completely.
I also note that the document has a direct link to (https://www.raylib.com/index.html). There is not anything there that supports handling of questions or complaints around the proposed code of conduct.  There is also no identification of "community leaders."

ok!


Save plans linked to order_items

Hi!
Just an idea and I would like to know what you guys think.
For other purposes we need to know which plan is linked to the order line. We use it to synchronize with our administration tool.
Currently there is no way of knowing which plan is linked, yeah it’s possible to do it based on the plan description but if they even change that’s gonna be a problem.
Would you guys accept a pull request with this feature? And if yes, we should discuss which is the best way to do this.
With regards!

Hi,
The order item is linked to the subscription through orderable. The plan is stored on the subscription. So you should be able to run a join query including both.
In v2 the order items table will contain other items than subscriptions too, so storing the plan there does not make sense.
If necessary for your use case you could add it in your app and use the cashier events to populate it.
Hope that helps!

Hi Sander,
I understand how cashier works. Except in this case al my orders items are linked to the orderable_type Laravel\Cashier\Subscription

I will try to explain. I know i can get the current subscription for this invoice line. Except if the plan has changed in the past. the join displaying incorrect data (the current actual plan, for an old invoice). Because the customer had a different plan on the time the invoice was generated.
this way you will never know for past invoices with plan changes what the actual plan was at the time of generating the invoice.

I understand your use case. I think solving it with events as described above is the easiest way to go. Another direction to explore is using a preprocessor.
Either way you'll have to add the plan column and take care of populating it, Cashier will not be doing this.


SSH Agent forwarding in devpace

Is your feature request related to a problem?
Yes, we would like to be able to update dependencies when running devspace commands like dev/pipeline. I tried using
proxyCommands:
      - gitCredentials: true
        skipContainerEnv: true

to no avail
Which solution do you suggest?
Docker has following solutions
docker run --mount type=bind,source=$SSH_AUTH_SOCK,target=/ssh-agent \
             --env SSH_AUTH_SOCK=/ssh-agent \
             foo-image

and docker-compose this
something:
    container_name: something
    volumes:
        - $SSH_AUTH_SOCK:/ssh-agent # Forward local machine SSH key to docker
    environment:
        SSH_AUTH_SOCK: /ssh-agent

so forwarding the ssh agent would achieve that. However as pointed out here it might be challenging
https://github.com/coderanger/kitchen-kubernetes/issues/5
Which alternative solutions exist?
updating my ssh/.config file to include ForwardAgent yes allows me to ssh in to my pod and run git pulls / dependency updates. However when running devspace dev that block automatically gets overwritten, and even if it's enabled running a pipeline command to update my dependencies it does not pick up the agent.
this is not ideal, but if it's the only way it would be nice to have devspace have a flag to turn on ForwardAgent when it updates the ssh config of the host.
Additional context
We do not want to introduce our ssh keys into our image.
I am @mawkbagawk in the GoFundMe slack and our discussions on this topic happened here in the #loft-support-for-gofundme channel
https://gofundme.slack.com/archives/C0356D10LDC/p1681168730628459

i would also love this feature

Agreed this would be a great add. Being able to forward the agent could also provide a workaround for the gitCredential helper not being available during initial sync (#2349)


Fix issue with strong reference making library contain itself

EventSource should not reconnect if another app has set its reference to nil.
var eventSource = EventSource()
// do connect
eventSource.close()
eventSource = nil
// EventSource will still reconnect after this, if it is timed after it has started doing the async after
This pull request fixes that issue

Codecov Report

Merging #87 into master will decrease coverage by 3%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #87      +/-   ##
==========================================
- Coverage   84.52%   81.51%   -3.01%
==========================================
  Files           8        8
  Lines         588      752     +164
  Branches        0       45      +45
==========================================
+ Hits          497      613     +116
- Misses         91      110      +19
- Partials        0       29      +29

Impacted Files
Coverage Δ

EventSource/EventSource.swift
94.46% <100%> (+1.16%)
:arrow_up:

EventSourceTests/SplittedEvents.swift
80% <0%> (-8.24%)
:arrow_down:

EventSourceTests/ConfigurationTests.swift
77.27% <0%> (-7.64%)
:arrow_down:

EventSourceTests/EventSourceTests.swift
76.88% <0%> (-5.59%)
:arrow_down:

EventSourceExample/ViewController.swift
55.73% <0%> (-5.05%)
:arrow_down:

EventSourceTests/MockNSURLSessionDataTask.swift
100% <0%> (ø)
:arrow_up:

EventSourceTests/TestableEventSource.swift
100% <0%> (ø)
:arrow_up:

EventSourceExample/AppDelegate.swift
14.28% <0%> (+0.95%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update daf42d5...599036f. Read the comment docs.

self does not retain the DispatchQueue.main singleton


Cannot see Vue.js devtools on vue.js v1.0.22

Vue.js version
1.0.22
Steps to reproduce
I Just upgraded to 1.0.22 and I cannot see [Vue.js devtools] in chrome (https://chrome.google.com/webstore/detail/vuejs-devtools/nhdogjmejiglipccpnnnanhbledajbpd)
OS: ubuntu 14.04
Chromium Version: 49.0.2623.108 Ubuntu 14.04 (64-bit)

Bummer.
I just had to restart chrome and it works.
Sorry for that.
False alarm.


fix for Portuguese

In the older langpacks, the language id for Portuguese was lowercase, while the folder was capitalized. This PR restores that and the resulting vsix package is properly capitalized, matching the extension gallery entry.
This PR fixes #15639

Pull Request Test Coverage Report for Build 923297226

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 43.369%

Totals

Change from base Build 922835645:
0.0%

Covered Lines:
26602

Relevant Lines:
55738

💛  - Coveralls


Wrapped func is recalled when result is None.

if not result in L50 recall wrapped function when result is an empty queryset like queryset.count() == 0.

I just ran into this and found it rather surprising. Any falsy value ([], "", False, {}...) will be regarded as a cache miss.

I just ran into this and found it rather surprising. Any falsy value ([], "", False, {}...) will be regarded as a cache miss.

Any plan to merge this PR anytime soon?


Document determinism

This PR does two documentation-y things:

Adds a section in the README on how to achieve deterministic output w.r.t. file contents, with a link to a method that restores file modification times.
Adjusts the -help text to point out more clearly -m does and that it helps with deterministic output.

This addresses #99

As requested, @jcchavezs - I hope this is what you had in mind. Let me know if you want to tweak the language in any way (:

I really love it. Let's wait a couple of days for some feedback.

(I pushed one more update to fix a typo in the readme bit)


Parse one statement at a time.

Codecov Report

Merging #5 (fea96c2) into main (3df7e7b) will decrease coverage by 0.13%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
- Coverage   93.32%   93.19%   -0.14%
==========================================
  Files          28       28
  Lines        1694     1690       -4
==========================================
- Hits         1581     1575       -6
- Misses         71       73       +2
  Partials       42       42

Impacted Files
Coverage Δ

internal/sql/parsing/parser/parser.go
99.50% <50.00%> (-0.50%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3df7e7b...fea96c2. Read the comment docs.


Ability to identify tier type / namespace type

Currently, SBE doesn't have an idea what tier or namespace type is used.
Related issue #97

Fixes by #504

Fixes by #504


Changes in custom library are not visible in built application

[x] bug report
[ ] feature request

Area
- [ ] devkit
- [ ] schematics

Angular CLI: 6.0.8
Node: 8.10.0
OS: linux x64
Angular: 6.0.6
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.6.8
@angular-devkit/build-angular     0.6.8
@angular-devkit/build-optimizer   0.6.8
@angular-devkit/core              0.6.8
@angular-devkit/schematics        0.6.8
@angular/cdk                      6.0.0
@angular/cli                      6.0.8
@angular/material                 6.0.0
@ngtools/webpack                  6.0.8
@schematics/angular               0.6.8
@schematics/update                0.6.8
rxjs                              6.2.1
typescript                        2.7.2
webpack                           4.8.3

Repro steps

generate library with ng g library
Install custom library in application
make changes to library
install library again

Mention any other details that might be useful
Hi  I'm having problems using my custom library i have built. The library itself builds fine without any issues. I can pack it up and install it as a local library in my angular app.
The strange thing is, that as soon as i make changes to my library and try to update it in my angular app (i.e. installing the newly generated .tgz file again), the changes are not reflected in the app.
I have narrowed the problem down to the buildOptimizer option in my angular.json, as soon as set it to false, the app does reflect changes i have made previously.
I looked into the bundles installed under node_modules and everything is alright there, but somehow the older version of my code is weaved into the main.js anyways.
Can somebody point me in the right direction, because i'm really stuck here.

Solved it, somehow the buildoptimizer used old code from its cache. I deleted it under<EMAIL_ADDRESS>and everything is working as expected now


Original renditions not really original

Issue Summary
When uploading an animated WebP image to Wagtail and rendering it with the original option, the image is not animated anymore.
It appears that some conversion of the image is applied although the original is requested. I am not sure if this is a bug or if I am looking for a feature to by pass the image manipulation during rendition generation. E.g. a "copy" option, which only copies the image file from original_images to images without any manipulation applied.
Steps to Reproduce

Start a new project with wagtail start myproject
Edit the following files

# home/models.py
from django.db import models

from wagtail.core.models import Page
from wagtail.images import get_image_model_string, edit_handlers

class HomePage(Page):
    image = models.ForeignKey(
        get_image_model_string(),
        on_delete=models.SET_NULL,
        null=True,
        blank=True,
    )

    content_panels = Page.content_panels + [
        edit_handlers.ImageChooserPanel('image')
    ]

{% extends "base.html" %}
{% load static wagtailimages_tags %}

{% block content %}

Home

    {% image page.image original %}
{% endblock content %}

# myproject/settings/base.py
...

WAGTAILIMAGES_FORMAT_CONVERSIONS = {
    'webp': 'webp',
}

Make migration, apply migrations.
Create a super user, start the server, visit the admin
Upload an animated WebP image. E.g. https://ezgif.com/images/format-demo/butterfly-small.webp
Select the uploaded image as the image on the home page.
View the home page and notice that the image is not animated anymore.

I have confirmed that this issue can be reproduced as described on a fresh Wagtail project: (yes / no)

Technical details

Python version: 3.9.9
Django version: 4.0.4
Wagtail version: 2.16.2
Browser version: Firefox Nightly 101.0a1, Safari 15.4

Hi @tbrlpld - this reminds me of #3210. On that issue there's a comment https://github.com/wagtail/wagtail/issues/3210#issuecomment-285027754 (albeit from 2017, so may warrant reconsideration) with a recommended workaround to use image.file.url as a way to serve the raw, original image, bypassing any Wagtail rendition code. Would this work for your purpose?

Wow. Thanks @chosak, that issue describes what I was thinking but way more eloquent and succinctly! Thank for the pointer! I somehow missed that one in my search.
My issue is also fairly similar. I only started thinking of animated WEBP after GIF resizing with Wand was crashing my containers and servers.
I guess the image.file.url work around should work. It will just require a custom image model and some form of detection logic (or a flag).
Since you are also already suggesting the "raw" filter in #3210 already, this issue is not really adding anything new to the discussion and I will close in favor of that one.


Rename packages

Simple package renaming. No logic has changed.

How was the naming adjusted? Perhaps in a separate PR we can add a markdown file with instructions on how you generate the API and how you get the correct package name?


hpx::lcos::barrier() results in deadlock

Expected Behavior
According to the documentation, using hpx::lcos::barrier("barrier_name") without specifying the number of localities or the rank should assume all the localities, and the current locality as the rank.
Actual Behavior
Deadlock.
A similar problem is true for gather_here() and broadcast_to() - it requires at least the locality count to compile/work correctly.
Steps to Reproduce the Problem
The Phylanx project is using HPX, and in a distributed run, APEX is using HPX collectives to do event unification at the end of execution.

Configure HPX with MPI support and APEX enabled
Configure Phylanx
Run a Phylanx distributed example with the environment variable APEX_OTF2=1 set.  It won't deadlock with the current code, but if these lines are changed to use default argument values, it will deadlock:
https://github.com/khuck/xpress-apex/blob/f666086dc469f48c220997d1b6b121e01d3c3afc/src/apex/otf2_listener.cpp#L1453
https://github.com/khuck/xpress-apex/blob/f666086dc469f48c220997d1b6b121e01d3c3afc/src/apex/otf2_listener.cpp#L1588

Specifications
... Please describe your environment

HPX Version: current master
Platform (compiler, OS):  linux GCC 7

h/t @stevenrbrandt

This has been resolved by switching to the C++20 barrier implementation.

@hkaiser  Thanks!


Create a launch deck for Collaborative Experiment Tracking.

Context
This is part of a wider collaborative experiment tracking product launch strategy #74 .
It captures the creation of a short presentation focused on the new collaborative experiment tracking feature #1295 on Kedro-Viz.
Goal
The goal is to create awareness and adoption of this feature.

Closing in favor of #75


New FBD Routine Example

I've been using the library quite a bit recently and really appreciate the work you've put into it.
One thing that I've found difficult to figure out is creating a new FBD routine and adding a sheet to it. I am able to open existing sheets and manipulate them, but I haven't quite figured out how to start one from scratch. Any advice?

Yeah, working with FBD is not easy. Modeling them in a simple intuitive way was hard, and to be honest, I still consider the FBD code somewhat of a work in progress. I want to refine it when I get more time, but if something isn't working correctly, I would not be surprised. I have not done extensive testing on creating FBD and importing them into Logix.
But to answer your question, you should be able to create a new Sheet and add Block and Wire objects to it. Here is a simple example.
var sheet = new Sheet { Number = 1 };
var id1 = sheet.AddAt(100, 100, Block.IREF("InputTag"));
var id2 = sheet.AddAt(100, 200, Block.IREF(100));
var id3 = sheet.AddAt(200, 100, Block.ADD("Add_Block"));
var id4 = sheet.AddAt(300, 100, Block.OREF("OutputTag"));
sheet.Add(new Wire { FromID = id1, ToID = id3, ToParam = "SourceA" });
sheet.Add(new Wire { FromID = id2, ToID = id3, ToParam = "SourceB" });
sheet.Add(new Wire { FromID = id3, ToID = id4, FromParam = "Dest" });

Things to note here:

When you call Add or AddAt you get back the ID of the block that was added (these have to be unique for the import to work so the code will just find the next available id and assign it to the block you add).
All FBD Blocks can be created using the Block class which has static factory methods for creating all the known FBD elements found in Logix (some may be missing or incorrect they have not all been tested).
You can use the IDs returned by Add/AddAt to add wires to the diagram which connect blocks/pins. You will have to know the defined pin names to make this work.

Obviously, even a simple example like this gets complicated. But once your sheet is all configured, adding it to a routine should be simple. It is crucial to set the correct RoutineType, SheetSize, and SheetOrientation for FBD diagrams for them to be imported.
var routine = new Routine(RoutineType.FBD)
{
    Name = "Test",
    SheetSize = SheetSize.A0,
    SheetOrientation = SheetOrientation.Landscape
};

routine.Content<Sheet>().Add(sheet);

Hope this helps.

Aha, I see what I was doing wrong. I wasn't passing the Routine constructor the "RoutineType.FBD", I was trying to change the Type parameter after creating the Routine. When I did that, I wasn't able to add a sheet because there was no FBDContent.
Things are working as expected now, thanks again for your help!

Thanks for pointing that out. I can probably improve that by making it update the child content element when you set Type after constructing the Routine. I'll add it to my list of things to look at.


Fix role order in reset playbook

Proposed Changes

see https://github.com/techno-tim/k3s-ansible/pull/92#discussion_r979367257
Checklist

[ ] Tested locally
[ ] Ran site.yml playbook
[ ] Ran reset.yml playbook
[x] Did not add any unnecessary changes
[x] 🚀

Thank you!!


Add basic proxy app

Don't make into separate repo, don't use create-react-app

Will be turned into entirely separate project

will make into addon


The buttons are not aligned in center

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

hi @shivvamm can u assign me this issue

@shivvamm which buttons are not alligned?

@shivvamm which buttons are not alligned?

If would be good if they are aligned in centre all of them

@shivvamm but in different mobile views its differently alligned

Yea it is left aligned which is bit odd

ok work on it

Yes sir on it I will send the pr soon ☺

i would like to work on this issue . can you please assign me?

@dvlsatya u can work

@dvlsatya any update?

Sir, can I work in this issue too ? I will try to align all these buttons on center and if I see any update while I am working I will tell you here or ping you on discord to tell me your opinion

@tasneemahmed123 u can work

I would like to work on this issue.

Its not required as of now


Address a few more places that used the old spelling

xref: https://github.com/scientific-python/lazy-loader/issues/107

Thank you!


Get all events since event with id?

I'm wondering how replay is done best. My intuition was to take "THE_LAST_EVENT_SEEN"-id from the revisionGuard of my event-denormalizer and read all events from the event store that come after that one.
However, I still didn't find this feature and wonder how this should be done instead... What is the desired way to do this (or am I completely wrong with this thinking)?

In the projects I've seen so far... for the replay use case (new read model) we/they do replay each event from the beginning.... => getEvents https://github.com/adrai/node-eventstore#query-your-events
and for occasionally missing events => getEventsByRevision
and only for edge cases when denormalizers where down for a longer time... they do get the "THE_LAST_EVENT_SEEN" containing an occurredAt field and use getEventsSince

Thanks for the fast reply... the last point is how I do it currently. However, I quite often had events that occurred at the same time (yes, exactly same time down to milliseconds). Guess I just augment my function so it ignores everything that comes before the last seen event ID.


Retrieve sample for FAIL checks

Description
Probably extends what was mentioned in https://github.com/sodadata/soda-core/issues/1497 but it would be nice to be able to define a sample set to be returned.
Example
If the check is:
product_stock >= 50:
      product_stock query: |
        SELECT unitsinstock + unitsonorder
        FROM products

We will se something like:
[product_stock >= 50] FAIL (check_value: 39.0)

Knowing SODA under the hood already manages the query:
SELECT unitsinstock + unitsonorder
FROM products

Would be ideal to ask for samples to be returned using derived query:
SELECT *
FROM products
WHERE NOT unitsinstock + unitsonorder >= 50
LIMIT 10

NOTE: Added a limit to be defined somewhere 😄

SODA-975

Hi @IraitzM,
Soda supports this through "failed row checks": https://docs.soda.io/soda-cl/failed-rows-checks.html. Can you validate this answers your question?

Thanks @mathissedestrooper! Yes, I was aware of that... But I did not check if returns a query when asking from a notebook. Will check.
My question was more related to getting the query so that it could be used while using Soda-Core (as a way to check while you are developing your tests, more interactivelly). But sure, it would be similar to the failed-rows-checks in Soda-Cloud.