Use rowid when the table doesn't have a pk

Instead of having to create a new table to add a new pk when the
table doesn't have one, lets just use the sqlite built-in unique
identifier accessible by 'rowid'.
@drippsj this should probably fix that issue for you.
There's one more issue that you might encounter because of this like https://github.com/viaforensics/viaextract-main/blob/develop/viaextract/ui/preview/base.py#L582.
On that db you provided me, there are some bool cols with 70 as a value. That makes those asserts fail and not open the preview.
I'm wondering if we should change those asserts to warnings...

Looks good to me :+1:

@hackedbellini With this branch, I'm now able to open the table in recovered.db that was causing the issue, but I get a traceback when I try to preview the rows from that table:
Traceback (most recent call last):
  File "/home/santoku/.virtualenvs/viaextract/src/datagrid-gtk3/datagrid_gtk3/ui/grid.py", line 662, in on_treeview_cursor_changed
    self.selected_record_callback(record)
  File "viaextract/ui/workspace.py", line 863, in show_record
    self.delete_confirmation_msg)
  File "viaextract/ui/preview/__init__.py", line 100, in get_preview_widget
    artifact_type, artifact_row)
  File "viaextract/ui/preview/sql.py", line 108, in __init__
    self, application, field_types, record_type, record)
  File "viaextract/ui/preview/artifact.py", line 41, in __init__
    data=dict(record)
  File "viaextract/ui/preview/base.py", line 206, in __init__
    self.populate(fields, data)
  File "viaextract/ui/preview/base.py", line 527, in populate
    handler_method(field_name, field_value)
  File "viaextract/ui/preview/base.py", line 583, in _populate_boolean
    '{} is not a boolean: {}'.format(field_name, field_value)
AssertionError: external_control is not a boolean: 70

I ran into a similar traceback while testing Javier's branch here. I wonder if his fix might also resolve this?

@hackedbellini Update: Using Javier's branch on viaextract-main with this branch for datagrid fixes the issue I reported above.


[test] Reenable stdlib/StringIndex, requiring optimized_stdlib

This test was disabled in https://github.com/apple/swift/pull/62875 because the addition of the new UTF-16 breadcrumbs tests seriously regressed DebugAssert runs on CI.
It turns out that even without that test, this test case is simply not appropriate to run with unoptimized stdlib builds -- e.g. the index interchange tests also take an extraordinarily long time to finish.
Let's reenable this test, but only run it with optimized stdlibs.
rdar://103934958

@swift-ci smoke test and merge

@swift-ci smoke test macOS platform

@swift-ci smoke test macOS platform


Bubble Sort

❗Write Code in Only One Language ( CPP , C , Java , Python )
Give File Name :  BubbleSort  ✅ In respective language folder
📑 DESCRIPTION
BubbleSort

Given an Integer N and a list arr. Sort the array using bubble sort algorithm.
##Your Task:
You don't have to read input or print anything. Your task is to complete the function bubblesort() which takes the array and it's size as input and sorts the array using bubble sort algorithm.
Example 1:
Input:
N = 5
arr[] = {4, 1, 3, 9, 7}
Output:
1 3 4 7 9

Example 2:
Input:
N = 10
arr[] = {10, 9, 8, 7, 6, 5, 4, 3, 2, 1}
Output:
1 2 3 4 5 6 7 8 9 10

Constraints:
1 <= N <= 103
1 <= arr[i] <= 103

please assign this to me

@Sayan2908 Ask for others too

@Sayan2908 Ask for others too Like you can ask for kartsuba i will assign that to you


ensure files from the tar stream are created with proper permissions

also bug https://bugzilla.redhat.com/show_bug.cgi?id=1479130

@jim-minter ptal, in reference to https://issues.jboss.org/browse/CLOUD-1982?_sscc=t

@rcernich fyi.  after this merges we'll have to updated openshift w/ the change, it'll be part of 3.6.1 at this point.  In the meantime the workaround would be to perform permission fixing in your assemble script.

Awesome.  Thanks @bparees!

Oops :(  Fix lgtm, please update the unit test.

@jim-minter unit tests updated.

hopefully it'll pass on Windows too? [test]

assuming it passes the tests, lgtm

finally.  @jim-minter please give a final review, i had to tweak the permission operations in the unit test a bit to make windows happy.

lgtm, please merge

[merge]
let the backporting begin.

backport to 3.6 branch: https://github.com/openshift/source-to-image/pull/788/files


Thresholds for clusters

Perhaps this could be connected to #15. Definitely to Sam's ideas.
Threshold could be a small number, say 0.001 of the entry value or absolute, and we could walk the matrix as a graph and find connected components based on threshold distance between points. This way all points would start separated (and could be plotted using different colors). Then points would start joining into clusters and fewer colors would show up on the plots. We could possibly see the clusters forming, and later joining together.
At first clusters would not be full graphs, but eventually we could report when they turn into clusters according to our current definition.

New plotting and components address this to some extent.


Want to know how to measure distance of detected objects using camera and radar sensor in CARLA

I have used camera RGB and radar i am getting vector 3D of each pixels generated by RADAR and for that i have used the code provided in carla documentation. i want to measure distance from an object using the image captured by camera. how to do that
any help is appreciated
thanks
Sagar Varvariya

the same question here...

Hello, thank you for your answer
I want to get the distance to each car.
Can you please guide me how to do this, what the radar sensor detects and how to set it only for cars.

@PatrickFSLin I used following code to measure average distance of detected objects. the following code you will find it on carla documentation as well.

https://carla.readthedocs.io/en/latest/tuto_G_retrieve_data/#set-advanced-sensors
https://carla.readthedocs.io/en/latest/ref_sensors/#depth-camera

refer this documents as well
RADAR sensor
    rad_bp = world.get_blueprint_library().find('sensor.other.radar')
    rad_bp.set_attribute('horizontal_fov', str(35))
    rad_bp.set_attribute('vertical_fov', str(20))
    rad_bp.set_attribute('range', str(20))
    rad_location = carla.Location(x=2.0, z=1.0)
    rad_rotation = carla.Rotation(pitch=5)
    rad_transform = carla.Transform(rad_location, rad_rotation)
    rad_ego = world.spawn_actor(rad_bp, rad_transform, attach_to=ego_vehicle,
                                attachment_type=carla.AttachmentType.Rigid)
    actor_list.append(rad_ego)

    def rad_callback(radar_data):
        velocity_range = 7.5  # m/s
        current_rot = radar_data.transform.rotation
        for detect in radar_data:
            azi = math.degrees(detect.azimuth)
            alt = math.degrees(detect.altitude)
            # The 0.25 adjusts a bit the distance so the dots can
            # be properly seen
            fw_vec = carla.Vector3D(x=detect.depth - 0.25)
            carla.Transform(
                carla.Location(),
                carla.Rotation(
                    pitch=current_rot.pitch + alt,
                    yaw=current_rot.yaw + azi,
                    roll=current_rot.roll)).transform(fw_vec)

            def clamp(min_v, max_v, value):
                return max(min_v, min(value, max_v))

            norm_velocity = detect.velocity / velocity_range  # range [-1, 1]
            r = int(clamp(0.0, 1.0, 1.0 - norm_velocity) * 255.0)
            g = int(clamp(0.0, 1.0, 1.0 - abs(norm_velocity)) * 255.0)
            b = int(abs(clamp(- 1.0, 0.0, - 1.0 - norm_velocity)) * 255.0)
            world.debug.draw_point(
                radar_data.transform.location + fw_vec,
                size=0.075,
                life_time=0.1,
                persistent_lines=False,
                color=carla.Color(r, g, b))

Array tranfer of obtained data
            points = np.frombuffer(radar_data.raw_data, dtype=np.dtype('f4'))
            points = np.reshape(points, (len(radar_data), 4))

            print(points)

 # code convert array into list and measure distance
            L = []
            pointslist=points.tolist()
            for i in range(len(pointslist)):
                L.append(pointslist[i-1][-1])

            ave = sum(L)/len(L)
            print(ave)

    rad_ego.listen(lambda radar_data: rad_callback(radar_data))
    print(rad_ego.listen)

good luck

Hello, thank you for your answer I want to get the distance to each car. Can you please guide me how to do this, what the radar sensor detects and how to set it only for cars.

@PatrickFSLin I used following code to measure average distance of detected objects. the following code you will find it on carla documentation as well.

https://carla.readthedocs.io/en/latest/tuto_G_retrieve_data/#set-advanced-sensors
https://carla.readthedocs.io/en/latest/ref_sensors/#depth-camera

refer this documents as well
RADAR sensor
    rad_bp = world.get_blueprint_library().find('sensor.other.radar')
    rad_bp.set_attribute('horizontal_fov', str(35))
    rad_bp.set_attribute('vertical_fov', str(20))
    rad_bp.set_attribute('range', str(20))
    rad_location = carla.Location(x=2.0, z=1.0)
    rad_rotation = carla.Rotation(pitch=5)
    rad_transform = carla.Transform(rad_location, rad_rotation)
    rad_ego = world.spawn_actor(rad_bp, rad_transform, attach_to=ego_vehicle,
                                attachment_type=carla.AttachmentType.Rigid)
    actor_list.append(rad_ego)

    def rad_callback(radar_data):
        velocity_range = 7.5  # m/s
        current_rot = radar_data.transform.rotation
        for detect in radar_data:
            azi = math.degrees(detect.azimuth)
            alt = math.degrees(detect.altitude)
            # The 0.25 adjusts a bit the distance so the dots can
            # be properly seen
            fw_vec = carla.Vector3D(x=detect.depth - 0.25)
            carla.Transform(
                carla.Location(),
                carla.Rotation(
                    pitch=current_rot.pitch + alt,
                    yaw=current_rot.yaw + azi,
                    roll=current_rot.roll)).transform(fw_vec)

            def clamp(min_v, max_v, value):
                return max(min_v, min(value, max_v))

            norm_velocity = detect.velocity / velocity_range  # range [-1, 1]
            r = int(clamp(0.0, 1.0, 1.0 - norm_velocity) * 255.0)
            g = int(clamp(0.0, 1.0, 1.0 - abs(norm_velocity)) * 255.0)
            b = int(abs(clamp(- 1.0, 0.0, - 1.0 - norm_velocity)) * 255.0)
            world.debug.draw_point(
                radar_data.transform.location + fw_vec,
                size=0.075,
                life_time=0.1,
                persistent_lines=False,
                color=carla.Color(r, g, b))

Array tranfer of obtained data
            points = np.frombuffer(radar_data.raw_data, dtype=np.dtype('f4'))
            points = np.reshape(points, (len(radar_data), 4))

            print(points)

 # code convert array into list and measure distance
            L = []
            pointslist=points.tolist()
            for i in range(len(pointslist)):
                L.append(pointslist[i-1][-1])

            ave = sum(L)/len(L)
            print(ave)

    rad_ego.listen(lambda radar_data: rad_callback(radar_data))
    print(rad_ego.listen)

good luck

have you done this part where you can get the distance of each car?


Improve service usage

Changes
Improves service usage for UserInputService by using game:GetService() and a better variable name.
Checks
By submitting your pull request for review, you agree to the following:

[x] This contribution was created in whole or in part by me, and I have the right to submit it under the terms of this repository's open source licenses.
[x] I understand and agree that this contribution and a record of it are public, maintained indefinitely, and may be redistributed under the terms of this repository's open source licenses.
[x] To the best of my knowledge, all proposed changes are accurate.

Thanks as always @Flikter!


[release/1.5 backport] content/local: inline sys.StatATimeAsTime()

cherry-pick of https://github.com/containerd/containerd/pull/5633
relates to https://github.com/docker/cli/pull/3154
The sys.StatATimeAsTime() utility is currently only used in a single place,
but because it's living in the "sys" package, also brings in other dependencies,
such as Microsoft/hcsshim.
This patch inlines the code from sys.StatATimeAsTime(), to remove that dependency.

moved out of draft, because https://github.com/containerd/containerd/pull/5633 was merged

LGTM


remove outdated calendar links

The linked calendar appears to be updated (with the exception of the wallet meeting), so I don' think this matters?

Nice! Close?


not working on dynamically generated table

hi
i have a table with the row are generated by a script. after i generate them i call the jquery selector for jeditable but nothing happens ... if i put a static row , the script work ... it's not working only in the generated rows

any help here please !!!

Could you provide a working copy like jsfiddle or jsbin to look into?

Hello, this is due to the way jeditable is written. Something I'm willing to fix down the road. Right now the workaround is to retrigger the jeditable on the element after it appeared.


Screenshots in non tiff format do not save to history

Clipmenu was able to work with more graphics formats - I have screenshot set to jpg format to save sapce and now I have to use clipmenu back, cause those are not stored in the history when doing screenshot to clipboadr with ctrl pressed.
Thanks for soon update!
Good work !

I have the same issue. I wish there was a real equivalent to ShareX on Mac...

me too

Hi!  Wanted to know if adding other image types such as jpg will be supported eventually?


DX: benchmark.sh - ensure deps are updated to enable script working across less-similar branches

ker@dus:~/github/PHP-CS-Fixer λ bash benchmark.sh 2.19b 3.0b
2.19 mean:45.3485 total:453.4850 rounds:10
3.0  mean:31.7666 total:317.6665 rounds:10

~30% improvement, huh?

Coverage remained the same at 91.53% when pulling 7aa596312bbb7abef18ba4c46fe8a32929ec069b on keradus:2.19b into 714e1922c6c598c8d032f166fdfbe86217e43bfd on FriendsOfPHP:2.19.


[C++][Parquet] Investigate / improve performance of Thrift utilities

We're currently using Thrift serialization / deserialization functions that allocate and deallocate Thrift C++ objects and memory with each invocation. The Impala codebase (and probably others) have some code we can learn from to be a bit more efficient:
https://github.com/apache/incubator-impala/blob/master/be/src/rpc/thrift-util.h#L42
Reporter: Wes McKinney / @wesm
Assignee: Wes McKinney](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=wesm) / @wesm
PRs and other links:

GitHub Pull Request #3268

Note: This issue was originally created as PARQUET-690. Please see the migration documentation for further details.

Wes McKinney / @wesm:
Issue resolved by pull request 3268
https://github.com/apache/arrow/pull/3268


Dynamic regions for flexible content placement

A lot of sites we created in FeinCMS required content to be positioned with a lot more flexibility than FeinCMS offered.

Sometimes some arbitrary content items just needed to be wrapped together inside a container with special styling (background, border, etc.)
Occasionally a website needed a more flexible grid-like approach. For example, one page needed to have 3 rows, 2 columns each (6 regions), but another page 4+2+3 regions (9 regions!), and a lot more variations.

You could argue that this is easily solved by making a lot of different templates with a lot of region tabs. Probably, but that wouldn't work for case 1. And for case 2 that would mean a lot of searching for templates and spreading of content items across region tabs. Not so user friendly.
We should envision a more flexible approach to content generation and placement. Dynamic regions, where the user can play with different layouts and region styles (within the rules set by the website designer), could be a revolutionary addition to FeinCMS. One that could attract a lot of attention to the project.
Think of a toolbox of layout types, not unlike the current content types you put in a region. Every layout type has it's own customization options (like content types). Some layout types can be nested. Some can only contain particular content types. And of course a new user interface in the admin to accomodate this kind of flexibility.
What do you think?
Wouter

That sounds interesting!
Most of the time, we have had enough flexibility by providing a replacement for feincms_render_region though which knows how to insert some custom HTML code before and after certain content types. Still, making these variations more customizable by the website manager might be a good addition to FeinCMS.


pgvector extension is not installed

https://github.com/langchain-ai/langchain/blame/e51bccdb2890fa193ce7eb5bf7e13c28afef4dc4/libs/langchain/langchain/vectorstores/pgvector.py#L117
@hwchase17

I ran into this too, I added a script during provisioning to run a query to set enable it.
    engine = sqlalchemy.create_engine(con_str)
    conn = engine.connect()

    try:
        with Session(conn) as session:
            statement = sqlalchemy.text("CREATE EXTENSION IF NOT EXISTS vector")
            session.execute(statement)
            session.commit()
    except Exception as e:
        return return_failure(f"Error install vector extension! {str(e)}", e)


HTTPS access to opensearch 500 error [BUG]

Opensearch version 2.0, k8s deployment with 3 nodes
In the same business scenario:
When opensearch is connected using HTTP, there are no issues and the business is normal
When Opensearch uses HTTPS to connect, it occasionally reports a 500 error, and the server also found through testing that one of the nodes was unable to return data normally

Maybe you can check the logfiles at this date and time to get a stack trace or further information about the cause.

@suwei999 can you please add the request that causes the 500 and the complete response with the error?

@suwei999 Closing this issue as there isn't enough information to act upon here. Please feel free to open a new issue if you can provide more details as requested above. Thanks!


Need help with querying latest pool of a stake address

I am starting out with cardano db sync and needed help to query latest pool of a stake address?

This is what i could come up with:
select latest_pool.addr_id, latest_pool.pool_id, ph.view, stakes.stake_address, stakes.amount
from (
         select distinct on (addr_id) addr_id, pool_id
         from epoch_stake
         order by addr_id, epoch_no DESC
     ) latest_pool
         join (
    select saa.addr_id, saa.stake_address, sum(saa.amount) amount
    from stake_address_addresses saa
    group by saa.stake_address, saa.addr_id
) stakes on stakes.addr_id = latest_pool.addr_id
         join pool_hash ph on ph.id = latest_pool.pool_id;

Are you aware of the interesting queries document? This query is pretty close to what you want.

Yes, I did it though but I don't know if it's right.
select latest_pool.addr_id, latest_pool.pool_id, ph.view, stakes.stake_address, stakes.amount
from (
select distinct on (addr_id) addr_id, pool_id
from epoch_stake
order by addr_id, epoch_no DESC
) latest_pool
join (
select saa.addr_id, saa.stake_address, sum(saa.amount) amount
from stake_address_addresses saa
group by saa.stake_address, saa.addr_id
) stakes on stakes.addr_id = latest_pool.addr_id
join pool_hash ph on ph.id = latest_pool.pool_id;``


[6.0][Concurrency] Refine getResumeFunctionForLogging to avoid reading invalid future contexts.

When using a future adapter, the resume context may not be valid after the task starts running. Only peer through the adapter when we're starting to run.
rdar://126298035

@swift-ci please test

Explanation: Our os_signpost code tries to peer through some wrapper functions when posting the address of the resume function. This change avoids doing this in cases where the context data may no longer be valid.
Original PR: https://github.com/apple/swift/pull/73796
Reviewed by: @ktoso
Risk: Low. We now signpost the wrapper functions themselves instead of trying to dig into the context, which is strictly safer. This changes the info we provide to Instruments, but code inspection and testing indicate that it tolerates the change.
Resolves: rdar://126298035
Tests: Manually tested with a reproducer.


Allocation pools

please test this

thinking about this patch again, I did not test that allocation_pools can be updated correctly, so it may not work the same in prod...


(Feature) Remove automatic load addressbook owners

Closes #1391

Travis automatic deployment:
https://pr1406--safereact.review.gnosisdev.com/app

Travis automatic deployment:
https://pr1406--safereact.review.gnosisdev.com/app

Test:

Create a safe giving names to the owners. Checking that the safe and owners are added to the address book
Added a new owner to the safe. New owner is added to the address book
Edited the name of the safe. The name changes in the address book
Created a new safe. Add the address that were added to the address safe in previous test. The names were filled automatically
During the creation edited those names filled automatically. They change in the address book and settings > owners list as well.
Loaded a safe. Address that were in the addressbook had their names filled automatically
Edited the names there and they changed in the address book and settings > owners table as well
Removed an address from the address book. Created/loaded a safe and the address deleted did not get a name filled automatically anymore.

Looks good to me

Travis automatic deployment:
https://pr1406--safereact.review.gnosisdev.com/app

Travis automatic deployment:
https://pr1406--safereact.review.gnosisdev.com/app

Retested this ticket after the changes, repeating what I've detailed in the previous comment and adding:

Replacing an owner in the settings with an account that was already in the address book and editing the name. After the replacement the name was edited in the address book.

Looks good to me

Retested this ticket after the changes, repeating what I've detailed in the previous comment and adding:

Replacing an owner in the settings with an account that was already in the address book and editing the name. After the replacement the name was edited in the address book.

Looks good to me

CC @dasanra

Travis automatic deployment:
https://pr1406--safereact.review.gnosisdev.com/app


Strict lint rule to only allow for CSS attributes to be camel case in tokens

Lint rule change; see CS

Where did this change request come from? Was it noticed elsewhere? I thought that the style lint setup should catch these sort of issues?

it wasn't that strict; but I've changed this PR to make it be

@joshwooding
Added lint rule, which is fine for core - but if you look at the failing list I think we may not be able to do it like this, depending on how strict we want to go...
Tokens like --calendar-day-outOfRange-text-color, I think outOfRange is better than out-of-range, in this scenario (personal preference), anything that stands as one piece should be camel case - but things like -adornmentGap should be -adornment-gap, because it would make sense to have -adornment-color, -adornment-size... etc - which can only be determined by us if it makes sense or not, not a lint rule
Thoughts?


socket hang up when using node-inspector

Socket hang up when using node-inspector to pause the execution, to be more exact, the socket hang up when the execution is continued.
node_modules/hotel/node_modules/http-proxy/lib/http-proxy/index.js:119
    throw err;
    ^

Error: socket hang up
    at createHangUpError (_http_client.js:239:15)
    at Socket.socketCloseListener (_http_client.js:271:23)
    at emitOne (events.js:95:20)
    at Socket.emit (events.js:182:7)
    at TCP._onclose (net.js:477:12)

This issue seems to have the same root cause with this PR:
https://github.com/typicode/hotel/pull/61

Hey, thank you for the PR and link to http-proxy issue :+1:
By the way, I'm not familiar with node-inspector how could I reproduce the error?

@typicode Install node-inspector via npm i -g node-inspector, then start the nodejs app with --debug to enable node's debug mode, i.e. node --debug app.js. Node will listen to the port 5858 to wait for  node-inspector, then run node-inspector.
node-inspector will listen to the port 8080 and take a parameter to specify the node app's debug port, i.e. http://<IP_ADDRESS>:8080/?port=5858. Visit the url then set a breakpoint on the app's source code from the panel.
Finally, trigger the breakpoint from the node app's page i.e. go to http://app.dev, and it'll pause on the breakpoint then resume its execution, and you'll encounter the issue.

Awesome, thanks :)

Fix released in v0.4.9


[Menu][Dropdown] Right menu sub-menu expanded to the wrong side

Hello.
Please, look at this fiddle: http://jsfiddle.net/donnicky/u1zLx2ow/
Sub-menu is opened so it covers dropdown: https://s3.amazonaws.com/donnicky_screenshots/2017-12-11_04-29-49_9055cc1b-9a03-46a3-b101-95120f725931.png

@stale Ping.

That was fixed in the community fork Fomantic-UI
See your adjusted jsfiddle here http://jsfiddle.net/lubber/Lpmxu58b/


Remove tracks functionality (calculatePageOrder)

In #339 we discussed removing the older tracks functionality (calculatePageOrder). See discussions below. If we remove that support, we should ensure app developers are notified in DevTools when still using the configuration option in triggers. We should also make sure we support all use-cases using newer expressions - especially changing the page order in some way. It seems we already have some (undocumented) support for changing the page order using ILayoutNavigation (search for that type and its usages), where the next page can be configured per-layout file. If we enhance these properties to support expressions, we should be covered enough to be able to remove support for calculatePageOrder.
Discussion:

https://github.com/Altinn/app-frontend-react/issues/339#issuecomment-1340766828
https://github.com/Altinn/app-frontend-react/issues/339#issuecomment-1340812897
https://github.com/Altinn/app-frontend-react/issues/339#issuecomment-1340873406

Relevant issues:

#306

This issue implies hiding and reordering of pages. Hiding is supported with expressions, while we need to look into the usage of reordering - are there any real cases of reordering of pages?

Oppdatere:

Kurs
Dokumentasjon
Eksempler på uttryk
Oppådatere converterings tool

Fjernes fra frontend
TImebox for å teste ut automatikk

Re-opening. As mentioned above, we also need a strategy for migrating existing apps, update documentation, etc.


[FLINK-25079][table-common] Add some initial assertj assertions for table data and types apis

What is the purpose of the change
Added some basic utilities for testing table apis, like:

assertThat(DataType)
assertThat(LogicalType)
assertThat(RowData)

Also add FlinkAssertions in flink-test-utils-junit to add some assertions and utilities to use together with assertj. Also deprecate the old matchers for hamcrest.
Brief change log

Add TableAssertions to encapsulate all the assertion constructors for our custom assertions
Add FlinkAssertions
Refactored some tests from junit assertions and hamcrest to assertj

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

@twalthr this is ready for another pass


Tracking issue: non-deterministic java apps

There are several ways to package a Java app inside Nixpkgs, and most of those package the final app into a .jar file. However, .jar files are not reproducible/deterministic by default.
How to check for reproducibility?
Why are .jar files not deterministic?
When creating a .jar file Java will place a META-INF/MANIFEST.MF file inside the archive which will have the current date as its creation date, so it's non-deterministic.
There also might be a problem with the fact, that changing a .properties file while building will put the current timestamp inside the file as a comment. These can usually be patched out without too much of a hassle.
One could just download the pre-built .jar-s and not worry about this, but I think it's important to build stuff from source.

Here are some ways packages build java apps inside Nixpkgs:

using javac+jar by themselves (not too common)
using ant/apacheAnt (usually using bundled jars for dependencies)
using maven (there's a generic builder: maven.buildMavenPackage)
using gradle (there's no generic builder for it yet, but some packages have found a way to patch it properly)

Possible solutions to achieve determinism
There was a setup-hook called canonicalize-jars-hook which aimed to solve this problem by unwrapping and rewrapping jars with the creation dates set to a fixed time. However, this process had some flaws

doesn't preserve compression-state (whether an entry is just stored or is compressed)
weird workarounds needed for messed up jar files
messes up MANIFEST.MF (is fixed in a PR listed below)

A PR fixing this has been merged and now canonicalize-jars-hook lives on as stripJavaArchivesHook (https://github.com/NixOS/nixpkgs/pull/296549)
My previous attempts at fixing this:

https://github.com/NixOS/nixpkgs/pull/278322
https://github.com/NixOS/nixpkgs/pull/286805

If we don't want to or can't use patchJavaArchivesHook for some reason, we could use the tools provided by the build systems:
Apache Ant
You can set modificationtime for the <jar> (and possibly <war>) tasks, usually in the build.xml files
Like this: <jar modificationtime="0" ...>...</jar>
To automate this you could use a script like this inside postPatch
# Fix jar timestamps for reproducibility
substituteInPlace build.xml \
    --replace-fail '<jar ' '<jar modificationtime="0" '

(Note, that this also matches a space character after the jar word, so that it doesn't accidentally match other tasks starting with jar, though there's not likely to be one anyway)
You could also use a dedicated xml modification tool like xmlstarlet
# Fix jar timestamps for reproducibility
xmlstarlet ed -L -a //jar -t attr -n modificationtime -v 0 build.xml

I tried to create setup-hook which does this automatically: https://github.com/NixOS/nixpkgs/pull/294516
However this would only work for ant projects
Maven
You can set the project.build.outputTimestamp property
You can do this by adding -Dproject.build.outputTimestamp=1980-01-01T00:00:02Z to the args given to the mvn command
or by patching the pom.xml file to have this:
<project>
  ...
  <properties>
    ...
    <project.build.outputTimestamp>1980-01-01T00:00:02Z</project.build.outputTimestamp>
  </properties>
  ...
</project>

Gradle
If the project uses Groovy, you can add the following lines to build.gradle
tasks.withType(AbstractArchiveTask) {
    preserveFileTimestamps = false
    reproducibleFileOrder = true
}

or if the project uses Kotlin you can add the following to build.gradle.kts
tasks.withType<AbstractArchiveTask> {
    isPreserveFileTimestamps = false
    isReproducibleFileOrder = true
}

Without a build system
If the packaging script of an app uses the jar command directly, you could use the --date flag to specify a build date. Note, this flag does not exist on versions before jdk17.
Example:
jar --date="1980-01-01T00:00:02Z" --create Program.class > test.jar

Though I'd say that using stripJavaArchivesHook is your best bet, because it should work with any build tool

Progress with making everything deterministic:
was already deterministic before opening this issue

jogl (ant)
prismlauncher (cmake's UseJava (javac+jar backend))
swt (javac+jar)

Fixed

[x] abcl (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278525)
[x] arduino-core-unwrapped (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297566)
[x] astral (javac+jar) (PR: https://github.com/NixOS/nixpkgs/pull/297131)

also puts the .jar files into a zip file, needed to call strip-nondeterminism manually

[x] axis2 (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294597)
[x] brmodelo (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297196)
[x] calcoo (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278530)
[x] dataexplorer (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297189)
[x] dayon (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278532)
[x] domination (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294968)
[x] DisnixWebService (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294709)
[x] fop (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294614)
[x] freetts (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294631)
[x] gcs (ant->golang) (PR: https://github.com/NixOS/nixpkgs/pull/279271)
[x] gogui (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294739)
[x] hdfview (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297160)
[x] hydra-ant-logger (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278740)
[x] ili2c (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278561)

.properties file editing patched out

[x] jameica (ant (PR: https://github.com/NixOS/nixpkgs/pull/294619)
[x] jasmin (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278567)
[x] javacc (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278577)
[x] javaCup (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278572)
[x] java-service-wrapper (ant) (PR: https://github.com/NixOS/nixpkgs/pull/295154)
[x] jdepend (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278534)
[x] jedit (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297255)
[x] jetbrains.jcef (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297717)
[x] jna (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297193)
[x] jpsxdec (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278739)
[x] jugglinglab (ant->maven) (PR: https://github.com/NixOS/nixpkgs/pull/278788)
[x] ktfmt (maven) (PR: https://github.com/NixOS/nixpkgs/pull/280283)
[x] libbluray withJava (ant) (PR: https://github.com/NixOS/nixpkgs/pull/295055)
[x] martyr (ant) (PR: https://github.com/NixOS/nixpkgs/pull/279424)
[x] mkgmap (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294708)
[x] openrocket (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297566)
[x] pattypan (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278590)
[x] processing (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297566)
[x] projectlibre (ant) (PR: https://github.com/NixOS/nixpkgs/pull/294958)
[x] scenebuilder (maven) (PR: https://github.com/NixOS/nixpkgs/pull/287319)
[x] sleuthkit (ant) (PR: https://github.com/NixOS/nixpkgs/pull/295064)
[x] trimmomatic (ant) (PR: https://github.com/NixOS/nixpkgs/pull/278776)
[x] xtreemfs (ant) (PR: https://github.com/NixOS/nixpkgs/pull/297566)

attempted, seems difficult

[ ] dbeaver (maven)

many failed attempts to update
uses tycho to fetch dependencies

puts timestamps into files and mirror-site names into the directory names
probably impossible ATM

[ ] mindustry (gradle)

Inconsistent file name order

This is partially fixed in https://github.com/Anuken/Mindustry/pull/9443

The texture packer still has some ordering randomness inside the Arc library.

[ ] i2p (ant)

not deterministic (unstable <servlet> order inside .xml files in the .war files)

could be done

[ ] freemind (ant)

not deterministic (.properties file being edited)

[ ] gephi (maven)

needs default plugin version fix
needs fixed outputTimestamp

[ ] jffi (ant)
[ ] sweethome3d (ant)
[ ] tvbrowser (ant)

not attempted

[ ] buck (ant)
[ ] jetbrains stuff (ant)
[ ] libreoffice? (ant)
[ ] polymake (ant)
[ ] rabbitmq-java-client (ant)

uses python2

[ ] rstudio? (ant)
There might be other packages that build jars which I didn't list here

Maybe this hook should be added to the java packaging docs

yes please!

Excellent work.

I was looking around on nixpkgs and it looks like ant actually does have a way to set the modification-time for the created jars. Here's the only example that's inside nixpkgs:
https://github.com/NixOS/nixpkgs/blob/6f5673f731614db0f22d9af504ac844a1112a17c/pkgs/development/java-modules/jogl/default.nix#L50-L52
It's editing the adding a fixed modificationtime attribute to the jar task.
Docs of the jar ant task: https://ant.apache.org/manual/Tasks/jar.html
After some more looking, it turns out that even the jar program itself has the --date argument.
jar --date="1980-01-01T00:00:02Z" --create Program.class > test.jar

This will create a jar file with all files inside having the set timestamp. Though this is not in the older versions of java. The first version to have this inside nixpkgs is jdk17. Not sure exactly which version added it because it is heavily underdocumented.
After some more searching, it looks like gradle also has something like this built in:
tasks.withType(AbstractArchiveTask) {
    preserveFileTimestamps = false
    reproducibleFileOrder = true
}

So, in conclusion, it looks like every major way of packaging has a built-in solution for fixed timestamps. I am a bit sad that I did not look into this more thoroughly earlier.
Still, I believe that having a hook, which works well can still be useful, as it will allow us to avoid having to patch every java app.

I decided that using the built-in method is a better solution, so I opened https://github.com/NixOS/nixpkgs/pull/294516, which moves away from canonicalize-jars-hook

It might also be helpful to have a tool to detect common pitfalls in making Java packages deterministic. For example, we could find all jar files in the outputs and inspect them for timestamps that suggest non-determinism. We could then cause that build to fail unless some sort of "allowNondetermism = true" flag is set. This could be a setuphook but at some point we may want a buildJavaPackage builder that just applies common-sense things like this and can be easily documented.

A new thing I discovered was the existance of .jmod files. AFAICT they are present inside every jdk. strip-nondeterminism supports patching these files, however I am not sure if it's the best idea to have all jdks as a dependency of strip-nondeterminism


Create EditProfile activity

working on it
parent issue #27

Focus more on event related functionality rather than profile based
We are not using any authentication info for getting data from API

@nikit19 @iamareebjamal Shall I work on this?

Not necessary for now. Look for bugs, there are many

@nikit19 I am working on it


Upgrading to 11.1 breaks ClickOnce deployment

Describe the bug
Running locally works fine using either my IDE or dotnet run.
When packing the application using a variation of the following command:
msbuild MyApp.sln -t:restore,publish -p:Configuration=Release -p:PublishProfile="${{env.PUBLISH_PROFILE}}" -p:PublishDir="${{env.PUBLISH_PATH}}" -p:ApplicationRevision=${{github.run_number}} -p:PublishUrl="${{env.DEPLOY_DEST}}" -p:InstallUrl="${{env.DEPLOY_DEST}}"

It builds and publishes properly but, when trying to run the actual artifact produced by this, I get the following error:
Avalonia.Markup.Xaml.XamlLoadException: No precompiled XAML found for MyApp.App.App, make sure to specify x:Class and include your XAML file as AvaloniaResource
   at void Avalonia.Markup.Xaml.AvaloniaXamlLoader.Load(object obj)
   at void Tesseract.App.App.Initialize()
   at void Avalonia.AppBuilder.SetupUnsafe()
   at void Avalonia.AppBuilder.Setup()
   at AppBuilder Avalonia.AppBuilder.SetupWithLifetime(IApplicationLifetime lifetime)
   at int Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime(AppBuilder builder, string[] args, Action<IClassicDesktopStyleApplicationLifetime> lifetimeBuilder)
   at int MyApp.App.Program.Main(string[] args)

In this case, my Program.cs builds the Avalonia app this way:
public static AppBuilder BuildAvaloniaApp() {
  AppBuilder.Configure<App>().UsePlatformDetect().LogToTrace().UseReactiveUI();
}

Where App is class in the MyApp.App project.
My only possible thought here is the ClickOnce is weird and not properly packing the app, but this doesn't explain why it worked fine with Avalonia 11 and 10.
To Reproduce
Try to deploy an app using ClickOnce and msbuild?
Expected behavior
The output app should run as it did for v11.0
Avalonia version
11.1.3
OS
Windows
Additional context
Unfortunately this is a Company app so I'm limited in the source code I can share. Hopefully the conditions causing the bug rings a bell to anyone more intimate with the changes from 11.0 => 11.1!
I should also mention that my csproj for this project look a LOT like the default csproj from the 11.1 template, and so should run fine.

I have the same problem. 👍

@heuristicAL in case you have a support agreement the team may be able to help further. Otherwise try to reproduce in a simple sample and upload it here

Exactly the same problem here. This when deploying with ClickOnce.

Fixed by #16835

I checked the current beta version containing the PR, but I got the same issue. Is it only for me?

@cauefelipe1 I also still have the same issue.

Can't reproduce it with 11.2.0-beta2. I tried both dotnet publish --no-build and msbuild MyApp.sln -t:restore,publish (as stated in this bug). Both published working applications with XAML compilation.

How are you publishing the project? Can you share the source code?

@TomEdwardsEnscape, it works fine if I publish it in a folder, but when I create a ClickOnce profile and publish it that way, I get the error above.

The repro steps in the bug are missing the important step of actually publishing via the ClickOnce system. This is not the same as executing the given MSBuild command, and there are still issues due to the way ClickOnce generates its file layout.
The "entry point assembly" is captured at evaluation time. This means that even though our target executes at the correct time, there is a copy of the original file path that it doesn't know about and so can't update. As a result, the assembly being published doesn't have XAML compile output (all referenced assemblies are OK though).
I can think of three ways to fix this:

We replace the _DeploymentManifestEntryPoint item in InjectAvaloniaXamlOutput. I've tried this and it works. But the item has an underscored name, which indicates that it's considered private and may be changed or removed at any time. (It also has a metadata item that we have to replicate. Instead of hardcoding it we could write a task to copy all metadata to the new item.)
Microsoft updates their targets file so that the value isn't captured at evaluation time. There is no reason for them to do this, as the item isn't used until targets start executing. But that only fixes one instance of this problem.
We start overwriting the original build outputs from CSC instead of redirecting paths to our own outputs. Capturing the original paths at evaluation time wasn't a scenario that I considered during #13840. Somebody else could capture the values now or in the future. However, overwriting the originals is unexpected behaviour and runs the risk of reintroducing the original issue.

Is step 1 something that I can do by my self?

If you want a quick fix right now, you can add this to the project you are publishing:
<Target Name="ClickOnceAvaloniaXaml" AfterTargets="InjectAvaloniaXamlOutput"
        Condition="'@(AvaloniaResource)@(AvaloniaXaml)' != '' AND $(EnableAvaloniaXamlCompilation) != false">
  <ItemGroup>
    <!-- ClickOnce support -->
    <_DeploymentManifestEntryPoint Remove="@(_DeploymentManifestEntryPoint)" />
    <_DeploymentManifestEntryPoint Include="@(_AvaloniaXamlCompiledAssembly)">
      <TargetPath>$(TargetFileName)</TargetPath>
    </_DeploymentManifestEntryPoint>
  </ItemGroup>
</Target>

@TomEdwardsEnscape Thanks! That worked like a charm!

@maxkatz6 please re-open this bug.

I've listed three possibilities above, I'd like to hear what Avalonia team members think.

Good findings @TomEdwardsEnscape. Why isn't that capture in a target...

I can think of three ways to fix this:
1. We replace the `_DeploymentManifestEntryPoint` item in `InjectAvaloniaXamlOutput`. [...]
2. Microsoft updates their targets file [...]
3. We start overwriting the original build outputs from CSC [...]

In my opinion, 2. would be the best. I'll try to open a PR for MSBuild, but I wouldn't be surprised if the change is rejected. After all it's been working this way for what, 20 years now?
Even if it's accepted, we would still need a workaround for a while. I say we go with 1. to avoid reintroducing the previous issues with overwriting the original file. Yes, the item is private, but it hasn't changed for years and is unlikely to change at all (jinxing it). Feel free to open a PR.


Use brace_style for struct instantiation

New struct instances don't adhere to brace_style. For example, when brace_style = "AlwaysNextLine", a new struct instance will look like this:
let value = MyStruct {
    something: 1
}

When it should look like this:
let value = MyStruct
{
    something: 1
}

Originally posted by @Sibz in https://github.com/rust-lang/rustfmt/issues/3376#issuecomment-70128656

This also applies when using macro variables:
$foo {
   bar: 1
}

Thanks for opening all these, but I'm going to close all of them, at least for the time being, pending conclusion of the strategy discussion in #3376

Alright. Perhaps given the introduction of a new option fitting this scope they can be reopened 🤛🏻👍


[SP-2637][PDI-14699] Encrypted passwords do not work in Salesforce / …

…IBM WebSphere MQ steps

It is backport for https://github.com/pentaho/pentaho-kettle/pull/2380

Build Commands:
Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f plugins/salesforce/build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f plugins/salesforce/build.xml jacoco-integration checkstyle publish-local

Cleanup:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
plugins/salesforce/src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnection.java
plugins/salesforce/test-src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnectionIT.java
plugins/salesforce/test-src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnectionTest.java
Unit test coverage change:
org.pentaho.di.trans.steps.salesforceinput.SalesforceConnection:

Branch Change :  + 4.05%
Complexity Change :  + 7.24%
Instruction Change :  + 7.67%
Line Change :  + 11.53%
Method Change :  + 16.23%

Integration test coverage change:
org.pentaho.di.trans.steps.salesforceinput.SalesforceConnection:

Branch New Coverage : 4.86%
Complexity New Coverage : 8.38%
Instruction New Coverage : 11.46%
Line New Coverage : 18.18%
Method New Coverage : 27.27%

Checkstyle violations:
See: https://github.com/pentaho/pentaho-coding-standards
File /home/buildguy/project/plugins/salesforce/src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnection.java

Message: ',' is preceded with whitespace. at Line: 864 Column: 92
Message: ',' is preceded with whitespace. at Line: 872 Column: 92
Message: ',' is preceded with whitespace. at Line: 880 Column: 92
Message: ',' is preceded with whitespace. at Line: 888 Column: 92

The PR was updated with fix checkstyle violations

@bmorrise, Could you review this backport for #2380?

Build Commands:
Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f plugins/salesforce/build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f plugins/salesforce/build.xml jacoco-integration checkstyle publish-local

Cleanup:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
plugins/salesforce/src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnection.java
plugins/salesforce/test-src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnectionIT.java
plugins/salesforce/test-src/org/pentaho/di/trans/steps/salesforceinput/SalesforceConnectionTest.java
Unit test coverage change:
org.pentaho.di.trans.steps.salesforceinput.SalesforceConnection:

Branch Change :  + 4.05%
Complexity Change :  + 7.24%
Instruction Change :  + 7.67%
Line Change :  + 11.53%
Method Change :  + 16.23%

Integration test coverage change:
org.pentaho.di.trans.steps.salesforceinput.SalesforceConnection:

Branch New Coverage : 4.86%
Complexity New Coverage : 8.38%
Instruction New Coverage : 11.46%
Line New Coverage : 18.18%
Method New Coverage : 27.27%


fix: Fix various memory leaks problems

Which issue does this PR close?

Closes #884.
Rationale for this change

Please refer to the comments in https://github.com/apache/datafusion-comet/issues/884 for details.
What changes are included in this PR?

Fixes ArrowSchema and ArrowArray base structure leaks in JVM
Fixes AQE coalesce partition leaks by closing the ArrowStreamWriter

How are these changes tested?
It is pretty hard to add tests for this fix, so we manually tested this and relying on existing tests to make sure that it does not break anything.

Codecov Report
Attention: Patch coverage is 23.25581% with 33 lines in your changes missing coverage. Please review.

Project coverage is 55.01%. Comparing base (9d8730d) to head (a0c46ee).
Report is 22 commits behind head on main.

Files with missing lines
Patch %
Lines

...ain/scala/org/apache/comet/vector/NativeUtil.scala
0.00%
25 Missing :warning:

.../scala/org/apache/comet/vector/ExportedBatch.scala
0.00%
7 Missing :warning:

.../scala/org/apache/spark/sql/comet/util/Utils.scala
0.00%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #890      +/-   ##
============================================
- Coverage     55.16%   55.01%   -0.15%
+ Complexity      857      854       -3
============================================
  Files           109      110       +1
  Lines         10542    10592      +50
  Branches       2010     2020      +10
============================================
+ Hits           5815     5827      +12
- Misses         3714     3750      +36
- Partials       1013     1015       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@Kontinuation I revised the current approach in https://github.com/apache/datafusion-comet/pull/893. So the ArrowSchema and ArrowArray are allocated in native. By doing that, I think we don't need to release the JVM structures.

@Kontinuation I revised the current approach in #893. So the ArrowSchema and ArrowArray are allocated in native. By doing that, I think we don't need to release the JVM structures.

Yes. They are allocated by the native side in #893,  so they should be released by the native side accordingly.

Thank you @Kontinuation
I will merge this first. And in #893, I will remove ExportedBatch as it allocates array/schema structures in native.


update Documenter to v1

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (8126819) 87.40% compared to head (77ab590) 87.38%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #567      +/-   ##
==========================================
- Coverage   87.40%   87.38%   -0.03%
==========================================
  Files          10       10
  Lines        3026     3027       +1
==========================================
  Hits         2645     2645
- Misses        381      382       +1

Files
Coverage Δ

src/abstractinterpret/abstractanalyzer.jl
85.10% <ø> (ø)

src/abstractinterpret/inferenceerrorreport.jl
90.12% <90.90%> (-0.29%)
:arrow_down:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Svnapot spec confusion

My team and I are confused by the two bullet points in the Svnapot chapter.

If the encoding in pte is valid according to Table 5.1, then instead of returning the original value of pte, implicit reads of a NAPOT PTE return a copy of pte in which pte.ppn[pte.napot_bits − 1 : 0] is replaced by vpn[i][pte.napot_bits − 1 : 0]. If the encoding in pte is reserved according to Table 5.1, then a page-fault exception must be raised.

The indexing of pte.ppn here is apparently a bit index. (It doesn't make sense otherwise.) But that is different than the indexing of pte.ppn elsewhere, including in the heading of Table 5.1 just above this paragraph, which uses pte.ppn[0] to refer to bits [18:10] of the PTE.
Also, what is the i in vpn[i]? If that refers to the level, as i does elsewhere, then the bit index doesn't make sense for anything except i=0.
I suppose you could write: pte.ppn[i][min(8, pte.napot_bits - 9 * i - 1)  : 0] is replaced by vpn[i][min(8, pte.napot_bits - 9 * i - 1): 0] for all i <= (pte.napot_bits  - 1) / 9. Or something crazy like that.

Implicit reads of NAPOT page table entries may create address-translation cache entries mapping a + va.vpn[j] × PTESIZE to a copy of pte in which pte.ppn[pte.napot_bits − 1 : 0] is replaced by vpn[0][pte.napot_bits − 1 : 0], for any or all j such that j[8 : napot_bits] = i[8 : napot_bits], all for the address space identified in satp as loaded by step 0.

I believe what this second paragraph is trying to say is that implementations are permitted to cache a 64KB page as a series of 4KB pages, or any other in-between size. Yes? I base this on the commentary that follows which indicates that software should do sfence.vma on every 4KB chunk when a NAPOT PTE changes.
But I don't see how this paragraph leads to that result.
I assume that the j here is analogous to the i in Step 2 of 4.3.2, where i refers to the tablewalk level and has a range of 0-2 in Sv39. How then is j[8 : napot_bits] relevant, since napot_bits=4? Those bits will always be 0.
It seems that a + va.vpn[j] × PTESIZE will always point to a PTE. So maybe this paragraph has to do with caching intermediate (non-leaf) PTEs?
What is i in this paragraph? There is only one reference to it. It must not refer to the tablewalk level, as i does elsewhere. But I don't know what else it might be.
What is "step 0"? Section 4.3.2 has no step 0.

Unless Andrew or someone solidly answers this question, this should be directed to @daniellustig - who I believe wrote the text in question.
Insofar as the belief of what the second paragraph is saying - that is correct.  An implementation may cache individual 4KB pages, or a 64KB napot page, or anything in between.  And software can't assume anything in this regard and, since the semantics of sfence.vma were (intentionally) not changed, it must do an sfence.vma for each 4KB page within a 64KB napot page.

It's true that since only 64 KiB NAPOT pages are currently defined, i must be 0.  But Dan's formulation was generalized to handle NAPOT superpages (i > 0), too.  I think generalizing the description was a defensible choice: should NAPOT superpages ever be defined, only the table need be altered.
i is defined implicitly in Table 5.1 above, which is how we know i must be 0 for the time being.
You are right that we were missing an [i] in pte.ppn[i][pte.napot_bits − 1 : 0], and that j was misdefined (it should just be an integer 0..511, not an index into vpn).  And step 0 should be step 1.  Thanks for pointing these out; I've fixed them in 54eb7d664d6f0814a86c57971c8e5abcc3050915
I believe the remaining points are stylistic.
I'll leave this open so y'all and @daniellustig can check my work.

It's true that since only 64 KiB NAPOT pages are currently defined, i must be 0. But Dan's formulation was generalized to handle NAPOT superpages (i > 0), too. I think generalizing the description was a defensible choice: should NAPOT superpages ever be defined, only the table need be altered.

I think my confusion is that I was assuming napot_bits would continue growing, e.g. napot_bits=12 for page size of 16 MiB. If you define the 16 MiB page size with i=1 napot_bits=3 then it works.
I will have to think about the new paragraph 2. Am I right about what this is intended to say? (Storing 64KB as smaller pages is allowed.)

If you define the 16 MiB page size with i=1 napot_bits=3 then it works.

Right, that's how it would be defined.

Am I right about what this is intended to say? (Storing 64KB as smaller pages is allowed.)

Yep, for example, a legal implementation of this extension is to ignore the N bit (modulo exception behavior, anyway).

Thanks for the fixes @scottj97 @aswaterman.  Andrew's commit and explanations LGTM.
FWIW the table in the commentary part of the Svnapot spec also gives examples of other possible future encodings, including for superpages with i=1 (currently stopping just one row short of your 16MB example, but the pattern is there.)


fluxctl: 1.21.1 -> 1.21.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/weaveworks/flux/releases.
meta.description for fluxctl is: "CLI client for Flux, the GitOps Kubernetes operator"
meta.homepage for fluxctl is: "https://github.com/fluxcd/flux"
meta.changelog for fluxctl is: ""
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.21.2 with grep in /nix/store/gfsa7f3vinxnxnkhja09r5jbi4ykzdda-fluxctl-1.21.2

directory tree listing: https://gist.github.com/ec2ce6ed263ff1e2001b3185d21759bd

du listing: https://gist.github.com/d7a859d5749036fe592c466bc7db1da5

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
fluxctl

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/gfsa7f3vinxnxnkhja09r5jbi4ykzdda-fluxctl-1.21.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A fluxctl https://github.com/r-ryantm/nixpkgs/archive/6669fc24a98124a9f26fd7a0f274c743dc720e1e.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/gfsa7f3vinxnxnkhja09r5jbi4ykzdda-fluxctl-1.21.2
ls -la /nix/store/gfsa7f3vinxnxnkhja09r5jbi4ykzdda-fluxctl-1.21.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

fluxctl

Maintainer pings
cc @Gonzih @Br1ght0ne @kalbasit for testing.

Result of nixpkgs-review pr 113561 at 6669fc24 run on aarch64-linux 1

1 package built:

fluxctl

1 suggestion:

error: no-flags-array
buildFlagsArray is only intended to be used in Bash, not as a Nix attribute.
Near pkgs/applications/networking/cluster/fluxctl/default.nix:22:3:
   |
22 |   buildFlagsArray = [ "-ldflags=-s -w -X main.version=${version}" ];
   |   ^


Missed import for sys

Changelog description
Added missing import to sys
Status

[x] Code base linted
[x] Ready to go

I'm not sure what I need to do for Travis. It looks like there are issues unrelated to this PR?

please rebase to fix CI

Do you want to rebase this, or shall I just incorporate the edits into another PR?


User variable substitution (lookup) for the RelativePath value for a Bicep Module

Support variable substitution on the path string used via RelativePath in a Bicep Module?!
i.e. I don't want to hard code the '../templates-base/01-azuredeploy-OMS.bicep'
I would prefer to maintain a single lookup table at the variables for all references, then just lookup the path via the variable:
DeploymentInfoObject.OMS

WHY?
It's easier to maintain all of the paths together... and clone new deployments for new resource types.

Sounds like this may be possible via #444

Also keeping in mind this will also have to be supported via New-AzTemplateSpec after the template is decompiled?


removed id from hashCode and equals

Closes #9

Codecov Report

Merging #10 into master will decrease coverage by 0.05%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             master      #10      +/-   ##
============================================
- Coverage     77.84%   77.79%   -0.06%
+ Complexity      574      573       -1
============================================
  Files            55       55
  Lines          1652     1648       -4
  Branches        165      164       -1
============================================
- Hits           1286     1282       -4
  Misses          244      244
  Partials        122      122

Impacted Files
Coverage Δ
Complexity Δ

src/org/smof/element/AbstractElement.java
100% <100%> (ø)
12 <0> (-1)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05a35a6...dd9187a. Read the comment docs.


added missing overflow-hidden to avatar preset for circle shape

Avatar circle shape preset is missing "overflow-hidden" class, you guys are using a circular pic for example , so the error is not visible

No way! I was about to make a pull request to fix this same issue!
Mind If I add mine and let the code reviewers see which one is more in line with the project needs?

No way! I was about to make a pull request to fix this same issue!
Mind If I add mine and let the code reviewers see which one is more in line with the project needs?

sure

Mentioned this PR just in case.

Hi, thank you for your PR.🌟 I merged this one https://github.com/primefaces/primevue-tailwind/pull/159 for this issue.


Lazy loading violation on Subpages in strict mode

Description
A common practice in laravel applications is to have Model::shouldBeStrict() checks active in non-prod environments:
https://laravel.com/docs/10.x/eloquent#configuring-eloquent-strictness.
Currently, when strict mode is activated, a LazyLoadingViolationException is thrown in:
Z3d0X\FilamentFabricator\FilamentFabricatorManager::setPageUrl()
in the line if (filled($page->children)) when attempting to save a SubPage (a page with parent) from the filament admin panel.
Reproduction steps

Activate strict mode by adding Model::shouldBeStrict(); to the boot() function of the AppServiceProvider.php

    public function boot()
    {
        Model::shouldBeStrict(); // should normally only be active in non-prod environments
    }

Create a Page and a Child Page through the filament admin panel. When saving the Child Page, the Exception will be thrown

Further comments:
The error does not happen if the line if (filled($page->children)) is replaced with if ($page->children()->count() > 0 && filled($page->children)), which seems to suggest that the Violation is not - strictly speaking - a lazy loading violation, but a missing attribute call violation.

I believe is it is eagerly loaded (last addressed in https://github.com/Z3d0X/filament-fabricator/pull/56).
Nevertheless I tried your reproduction steps, I'm unable to reproduce the error. Could you please provide with a reproduction repository, so i could investigate this issue further?

fixed in #96
Released v1.2.1 & v2.0.2 with the fix


ApplicationToken

I was wondering where we would get the application token value in the app.config

This code works only with Oauth1 keys which had apptoken. It will not work for Oauth2. You need to follow these steps for Oauth2 as we do not support non user interaction model there.
For desktop apps/server apps/console apps and automated service type apps for Oauth2
There is no way to bypass the Oauth process. You will have to do it at least once to make api calls.
So, if you have a desktop app/service then you can use Postman or Oauth2 playground we have, add the app keys and then get the Oauth2 tokens.
Hardcode them in your application. Then keep on refreshing access token using refreshtoken api call until refresh token expires in 100 days.
Then again get new access tokens by playground or postman and repeat api calls until refresh token expires. This would be a cycle of hardcoding tokens every 100 days.
Always save the latest refresh token in your code as once used refresh tokens can expire in 24hrs.
OAuth2 playground-https://developer.intuit.com/v2/ui#/playground
On the developer.intuit.com->Your app tab->Keys->Add this redirect url under your Prod app keys->Redirect url-https://developer.intuit.com/v2/OAuth2Playground/RedirectUrl
Postman-
https://developer.intuit.com/docs/0100_quickbooks_online/0400_tools/0012_postman
On the developer.intuit.com->Your app tab->Keys->Add this redirect url under your Prod app keys ->Redirect url-https://www.getpostman.com/oauth2/callback
1)You get access token and refresh token manually using Postman or your web code where user authorizes
2)You hard code TOKENs in your code by saving the access token and latest refresh token in DB/permanaent store as access token expires every 1 hr and refresh token every 24hrs.
3)You can make calls without User interaction after refreshing access token using refresh token api once you get the access token from step 1 .
4)Every time refresh token api is called(check any of the web sample apps for oauth2 here-https://github.com/IntuitDeveloper), you have to save the latest value for access token and refresh token in DB.
5)Use the latest refresh token to get new access token and this process continues until 100 days.
6)Then again start with getting a new access token or whenever you have an expired token for various reasons. You will get a 401 error when access token expires or a 400 error when refresh token call fails to refresh access token. Read details in the docs - https://developer.intuit.com/app/developer/qbo/docs/develop/authentication-and-authorization.


Support mock.shop as a storeDomain

Builds on top of #985 , and is related to the new onboarding in #913
Tiny change to support mock.shop as a storeDomain. Any call to storefront.query(...) will hit the mock.shop instead of the storefront API.
Note that mock.shop doesn't work for every query. For example, it throws permission errors when trying to read customer information.
🎩 To test it, add PUBLIC_STORE_DOMAIN="mock.shop" to the .env file in demo-store template, and comment out the customer { ... } selection part of the CartFragment in root.tsx (not supported in mock.shop).

@teddyhwang @darius-gai  — the permission errors for customers, any thoughts on that? intentional? want to keep it?

@teddyhwang @darius-gai — the permission errors for customers, any thoughts on that? intentional? want to keep it?

I don't recall but we might have just forgotten that scope. The only scope we were intentional about not adding was checkout mutations. feel free to add the scope and generate a new token!


Fixed inheritance of  text-anchor and baseline-shift attributes

use GetInheritedAttribute() to correctly handle "Inherit" value
fixes inheritance of these attributes from groups
force redraw of SvgUse children to ensure correct style inheritance
fixes alignment in types-basic-01-f.svg and text-text-12-t.svg tests

Note: the US flag test (first commit) is unrelated to the actual fix. I just added it after I broke it accidentally, to avoid regressions of #243.


Support configure for barrier interval

Problem Description
Now we use the const value for barrier interval. See code in https://github.com/singularity-data/risingwave/blob/main/src/meta/src/barrier/mod.rs#L177
Frequently checkpoint will produce a lot of sst files in level 0.
barrier with 100ms (main branch)

barrier with 1 second

Problem Solution
Support configuration for barrier interval and update the default value to 1 second
Future Discussion
If the number of files left in level0 increases too large, it will cost more resource to compact or read hummock. Maybe we shall think about  adjusting the barrier interval  dynamically according to the increase or decrease of the total number of files in L0, or other factors.

Previously we set the interval to 5000ms with debug profile and relies on manual FLUSH in the tests to make it run faster. However, the recent introduction of RearrangedChain has led us to set back to 100ms...

After RearrangedChain is used, every MV creation will last for at least one epoch, which will make the test running much slower if we still use 5000ms. We may investigate it more in the next few days to find a better workaround.
Some executors only yield outputs on the barrier. Several bugs were exposed with much more frequent barriers.

For production, I'd agree on setting a reasonable interval or finding some mechanism to decide one.

Already supported.


Comment knitter agent get pod error.

This fixes a bug (I didn't recall the detail) when agent fails to get pod. This can be optimized in the future.
cc @guangxuli

Pull Request Test Coverage Report for Build 50

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 35.538%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

knitter-agent/context/get_pod_action.go
0
1
0.0%

Totals

Change from base Build 48:
0.0%

Covered Lines:
5630

Relevant Lines:
15842

💛  - Coveralls


Bundle is including development and production versions of react

Apologies for opening another issue but seeing as my previous was closed without resolution, I had no choice. Previous issue https://github.com/webpack/webpack/issues/5848
Do you want to request a feature or report a bug?
Bug

What is the current behavior?
Bundling react not in production results in both 'react.production.js' and 'react.development.js' being included.
If the current behavior is a bug, please provide the steps to reproduce.
This is running a simple ./node_modules/.bin/webpack from the root of this project.
See the output here where both production and development builds are included.
Looking at the bundle profile you can see both react.development.js and react.production.js being required despite the if (NODE_ENV === 'production') check.

What is the expected behavior?
Only have 'react.development.js' bundled when running a development build.
If this is a feature request, what is motivation or use case for changing the behavior?
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
Node: 8.6.0
Webpack: 3.8.1

This is working as expected. NODE_ENV is not set to a compile-time constant value (no DefinePlugin). webpack doesn't know which value process.env.NODE_ENV will have at runtime, and can't assume any branch of the if-else is untaken. It have to include both versions. At runtime it will only load the dev version.
cc @gaearon

ok makes sense but that's causing us issues with having create-react-class unminified and react minified as per https://github.com/facebook/react/issues/9999

Solved this by explicitly defining NODE_ENV: JSON.stringify('development') in webpack.


[FEATURE]: get back generated values from $defaultFn

Describe what you want
Hi, I'm using $defaultFn to generate ids (to have uuids), I would like to have generated values with $defaultFn in the insert result (at first I thought that I can get back from MySqlInsert result with insertId but its value is always '0' - generated values only appear in statement)
So for the moment I have $defaultFn in my id columns (to make it optional through type inferences) but I need to explicitly create ids to have them back

Would also love to see this, makes it hard if I need that id to reference in code down the line.

I would like to submit a PR for this. I just don't know what would be the best approach.
The problem is how we should return it? It has to be a way that is driver agnostic. Specially when you're inserting an array.

I think this can be closed as it is now fixed by the recently released $returningId() function.
See https://github.com/drizzle-team/drizzle-orm/issues/1681#issuecomment-2222297214:

We just released a custom .$returningId() function for MySQL that should cover your case.
Please check the documentation here: https://orm.drizzle.team/docs/insert#insert-returningid
It should work both for serials and $default with the primary key defined
release notes with version: https://github.com/drizzle-team/drizzle-orm/releases/tag/0.32.0


unexpected keyword argument 'grad_wrt' when using TRAKer

I'm following the MSCOCO demo script in the folder examples. I got an error like this:
traker = TRAKer(
  File "/home/user_name/.conda/envs/sd38/lib/python3.8/site-packages/trak/traker.py", line 177, in __init__
    self.gradient_computer = gradient_computer(
TypeError: __init__() got an unexpected keyword argument 'grad_wrt'

I didn't change the initialization:
traker = TRAKer(
    model=model,
    task=task,
    gradient_computer=DiffusionGradientComputer,
    proj_dim=2048,
    train_set_size=len(loader_train.dataset),
    save_dir='./MSCOCO_trak_results',
    device='cuda'
)

I'm using traker 0.3.2. Can you help me locate the bug?
###############################################################
I also got this warning, and I put it here though it seems to be unrelated to the error.
The configuration file of this scheduler: DDPMScheduler {
  "_class_name": "DDPMScheduler",
  "_diffusers_version": "0.15.1",
  "beta_end": 0.02,
  "beta_schedule": "linear",
  "beta_start": 0.0001,
  "clip_sample": true,
  "clip_sample_range": 1.0,
  "dynamic_thresholding_ratio": 0.995,
  "num_train_timesteps": 1000,
  "prediction_type": "epsilon",
  "sample_max_value": 1.0,
  "thresholding": false,
  "trained_betas": null,
  "variance_type": "fixed_small"
}

 has not set the configuration `clip_sample`. `clip_sample` should be set to False in the configuration file. Please make sure to update the config accordingly as not setting `clip_sample` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` file

  deprecate("clip_sample not set", "1.0.0", deprecation_message, standard_warn=False)

Thanks!

Looking back after several months, using traker==0.2.2 diffusers==0.15.1 makes it work, although according to the setup.py traker 0.3.2 should also work.


When will the new community edition Docker image be released?

We have a small frontend modification that needs to be released in the new community version, but we've been stuck on Python package dependencies for the backend environment. Now we've successfully developed the feature we need through the API preview, but we want to contribute it to the new Redash community and hope that it can be included in the next Docker image release.

你卡在Python构建的哪一步了？有没有截图发出来？

Which version of Python should I use?
I am currently using Python 3.9, but it seems that the dependency of gevent=1.4.0 cannot be installed. Can I use gevent=22.10.2 instead?

Python 3.7 is too old for us as many dependencies cannot be used with Python 3.7.x.

Python 3.9, you can install gevent==1.3.4, if you can't run it, you can add greenlet==0.4.17. I tested it, it works.

Make a PR with it?
If it gets through the CI fine, then we can merge it.
Btw - there are 4 outstanding existing PR's in the repo, created by dependabot.  Those four seem to be non-trivial to upgrade though and I don't have much spare time this week.  If someone else can take a look at them, that would be awesome. :smile:

Python 3.9, you can install , if you can't run it, you can add . I tested it, it works.gevent==1.3.4``greenlet==0.4.17

 Rolling back uninstall of gevent   Moving to /usr/local/lib/python3.9/dist-packages/gevent-22.10.2.dist-info/    from /usr/local/lib/python3.9/dist-packages/~event-22.10.2.dist-info   Moving to /usr/local/lib/python3.9/dist-packages/gevent/    from /usr/local/lib/python3.9/dist-packages/~event ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-w4dbemnq/gevent_c5b4074345d34c5f9286fac00114a361/setup.py'"'"'; __file__='"'"'/tmp/pip-install-w4dbemnq/gevent_c5b4074345d34c5f9286fac00114a361/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-sehr8liw/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/gevent Check the logs for full command output.
After modifying the two dependency versions mentioned above, I am still unable to install gevent properly.

gevent                22.8.0
greenlet              1.1.3

gevent 22.8.0 greenlet 1.1.3

@Avey777 You can test it.

gevent 22.8.0 greenlet 1.1.3

Great!
Currently, there is no documentation on the dependencies for different versions. It would be great to have documentation on dependencies for different major Python versions in the future.
Also, Debian now comes with Python 3.9 installed by default.

gevent 22.8.0 greenlet 1.1.3

Great!
Currently, there is no documentation on the dependencies for different versions. It would be great to have documentation on dependencies for different major Python versions in the future.
Also, Debian now comes with Python 3.9 installed by default.

Lately, i will add this info.

@gaecoli Has this been done? I'm not seeing anything in our Docker Hub repo, nor wiki page with info, etc.
Am I looking in the wrong place? 😄

Sorry, I edit wiki page, but not push. I reopen this issue.

No worries at all. :smile:

@justinclift How can i push my new docker image to our docker hub?

No worries at all. 😄

Looking forward to the release of our community edition's first Redash image as soon as possible.

@gaecoli With this new docker image, is it a stable release build, or is it built from our our current master branch (aka "a development build"), or is it something else?
Just so we know how to talk about it, as our potential users will definitely want to know. :smile:

@justinclift secrets should be in github, I haven't had a chance to look at changing build number info yet, so we might want that before merging?
I think for releases we'll want a manual action trigger from a specific branch, which will come in a later PR.
https://github.com/RedashCommunity/redash/pull/25

Good stuff with that PR @hazmeister. :smile:
Looking at the build number info in master at the moment, it's currently saying 11.0.0-dev which seems fine for prerelease things.  So, I've just reviewed and merged the PR.
Any idea if the PR merge itself will generate a preview docker build, or if it'll start happening with the next merge, or ?

k, well this sounds like the preview docker builds should be about under control now, so that's good progress.
Next step will be to get updated, secure "release" builds done.
@hazmeister and @gaecoli You both seem to be working on that bit, but separately yeah?  If that's the case, would it be useful to work together on it or something?

Still getting used to GitHub Actions.  Referencing the environment allowed reading the secrets in.  We now have a preview image. I'll look at manual actions for the release branches later this week.

Awesome!
Ironically, the Cypress tests didn't pass this time around, but that's probably just a false positive.  Having used Cypress before with a different project, it was flaky there too.
So, am re-running that failed job now and (in theory) it should pass a 2nd time around.  Or maybe 3rd, we'll see. :wink:
But, working preview image now, that's excellent! :smile:

Cool, the re-run of the failed job worked first time around. :smile:
@hazmeister Would it makes sense to have the upload of the preview build (to Docker Hub) only happen after the CI tests successfully pass?
Am kind of thinking that uploading a build that fails CI is probably not what we want.
Looking at the existing ci.yml file, ordering seems to be done using the needs line.  eg:

https://github.com/RedashCommunity/redash/blob/bb1d45ba3e1690a8acef192935916c2d6e5476d4/.github/workflows/ci.yml#L24
https://github.com/RedashCommunity/redash/blob/bb1d45ba3e1690a8acef192935916c2d6e5476d4/.github/workflows/ci.yml#L10-L20

No idea if they can cross-reference things in other .yml files, or if we need to move the contents of docker-preview.yml into ci.yml, etc.
Thoughts? :smile:

version: "2"
x-redash-service: &redash-service
  image: redashcommunity/redash:preview
  depends_on:
    - postgres
    - redis
  env_file: /opt/redash/env
  restart: always

services:
  server:
    <<: *redash-service
    command: server
    ports:
      - "5500:5000"
    environment:
      REDASH_WEB_WORKERS: 4

  scheduler:
    <<: *redash-service
    command: scheduler
    environment:
      QUEUES: "celery"
      WORKERS_COUNT: 1

  scheduled_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "scheduled_queries,schemas"
      WORKERS_COUNT: 1

  adhoc_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "queries"
      WORKERS_COUNT: 2

  redis:
    image: redis:5.0-alpine
    container_name: redash-redis
    restart: always

  postgres:
    image: postgres:9.6-alpine
    container_name: redash-postgres
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: always

Can I install Redash like this? Should I write the .env file in the docker-compose.yml or do I need to create a separate .env file?

version: "2"
x-redash-service: &redash-service
  image: redashcommunity/redash:preview
  depends_on:
    - postgres
    - redis
  env_file: /opt/redash/env
  restart: always

services:
  server:
    <<: *redash-service
    command: server
    ports:
      - "5500:5000"
    environment:
      REDASH_WEB_WORKERS: 4

  scheduler:
    <<: *redash-service
    command: scheduler
    environment:
      QUEUES: "celery"
      WORKERS_COUNT: 1

  scheduled_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "scheduled_queries,schemas"
      WORKERS_COUNT: 1

  adhoc_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "queries"
      WORKERS_COUNT: 2

  redis:
    image: redis:5.0-alpine
    container_name: redash-redis
    restart: always

  postgres:
    image: postgres:9.6-alpine
    container_name: redash-postgres
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: always

Can I install Redash like this? Should I write the .env file in the docker-compose.yml or do I need to create a separate .env file?

It is not necessary to create a .env file separately. But it is recommended that you separate the redis service and postgres service from docker and deploy them on the machine separately, so that there is no risk of data loss.

x-redash-environment:` &redash-environment
  REDASH_LOG_LEVEL: "INFO"
  REDASH_REDIS_URL: "redis://redis:6379/0"
  REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
  REDASH_RATELIMIT_ENABLED: "false"
  REDASH_MAIL_DEFAULT_SENDER<EMAIL_ADDRESS>  REDASH_MAIL_SERVER: "email"
  REDASH_ENFORCE_CSRF: "true"
  REDASH_GUNICORN_TIMEOUT: 60
  # Set secret keys in the .env file

x-redash-environment:` &redash-environment
  REDASH_LOG_LEVEL: "INFO"
  REDASH_REDIS_URL: "redis://redis:6379/0"
  REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
  REDASH_RATELIMIT_ENABLED: "false"
  REDASH_MAIL_DEFAULT_SENDER<EMAIL_ADDRESS>  REDASH_MAIL_SERVER: "email"
  REDASH_ENFORCE_CSRF: "true"
  REDASH_GUNICORN_TIMEOUT: 60
  # Set secret keys in the .env file

Do I not need to add the above information to the docker-compose.yml file? Following the above steps did not result in a successful deployment.

... deploy them on the machine separately, so that there is no risk of data loss.

That's not the full picture.
With the PostgreSQL service part... :
  postgres:
    image: postgres:9.6-alpine
    container_name: redash-postgres
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: always

... the volume line there is important:
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data

That tells Docker to actually store the PostgreSQL database files on the host, rather than inside the container.  So if you destroy and rebuild the container, then the database doesn't get destroyed at the same time.
That being said, you should have some kind of backup of the database happening anyway, just in case the whole host system dies for some reason.

... deploy them on the machine separately, so that there is no risk of data loss.

That's not the full picture. With the PostgreSQL service part... :
  postgres:
    image: postgres:9.6-alpine
    container_name: redash-postgres
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: always

... the volume line there is important:
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data

That tells Docker to actually store the PostgreSQL database files on the host, rather than inside the container. So if you destroy and rebuild the container, then the database doesn't get destroyed at the same time.
That being said, you should have some kind of backup of the database happening anyway, just in case the whole host system dies for some reason.

Yes.

@Avey777 The env_file line in your above stuff is important:
  env_file: /opt/redash/env

It's saying those values need to be in the /opt/redash/env file.
Is that where you're putting them? :smile:

@Avey777 The env_file line in your above stuff is important:
  env_file: /opt/redash/env

It's saying those values need to be in the /opt/redash/env file.
Is that where you're putting them? 😄

I need to know where my configuration file is located. Are you saying that if I do not have env_file: /opt/redash/env in my docker-compose.yml file, then my deployment should be successful? Will the data in the .env file be generated automatically?

@Avey777 The env_file line in your above stuff is important:
  env_file: /opt/redash/env

It's saying those values need to be in the /opt/redash/env file.
Is that where you're putting them? 😄

I need to know where my configuration file is located. Are you saying that if I do not have env_file: /opt/redash/env in my docker-compose.yml file, then my deployment should be successful? Will the data in the .env file be generated automatically?

You just put your env in /opt/redash/env. NOT need .env file. If your env_file is .env, like env_file: .env, so you need touch a .env file. But your env_file is /opt/redash/env, so you don't need .env file.

Ahhh.  Ok, lets clarify things so we're on the same page.  Are you trying to do a new production deployment using that image, or are you trying to get this running in a development environment?
The docker-compose.yml file is a bit different, depending on what you're trying to do.
The one you have above, looks like it's for a production deployment, so it used /opt/redash/env instead of .env.
Does that make sense? :smile:

@Avey777  You don't have to pay attention to the naming of env_file, because it doesn't matter, the important thing is that you put your environment variables in a file and mount this file under env_file.

Ahhh. Ok, lets clarify things so we're on the same page. Are you trying to do a new production deployment using that Docker Hub image, or are you trying to get this running in a development environment?
The docker-compose.yml file is a bit different, depending on what you're trying to do.
The one you have above looks like it's for a production deployment, so it uses instead of ./opt/redash/env``.env
Does that make sense? 😄

I am deploying in a production environment.

Hmmm, as background there as two main files:

The docker-compose.yml file

This tells docker compose which services to start up, where to grab image files from, etc.

The environment file

This holds key=value information, and is passed to all of the services when they start.  This helps the services to know things like "what is the database address?", "what is the timeout value for gunicorn?" and so on.
Things like this:
REDASH_LOG_LEVEL: "INFO"
REDASH_REDIS_URL: "redis://redis:6379/0"
REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
REDASH_RATELIMIT_ENABLED: "false"
REDASH_MAIL_DEFAULT_SENDER<EMAIL_ADDRESS>REDASH_MAIL_SERVER: "email"
REDASH_ENFORCE_CSRF: "true"
REDASH_GUNICORN_TIMEOUT: 60

The docker-compose.yml needs to know where to find the environment file, so that's what the env_file: /opt/redash/env line inside it is for.  As long as the path there points to a readable file with those value in it, you should be good.

I am deploying in a production environment.

Ok.  Give me a few minutes, I'll see what needs changing for the existing Newdash setup script.  That one was based on the Redash one a while ago, and should be mostly ok.  Probably just needs the docker image name changed.

@Avey777 As a temporary step for now, try running this setup script:
https://github.com/getnewdash/setup/blob/master/setup.sh
I've updated it to use the Redash Community preview build:
We should probably copy the useful changes from that setup repo to our own setup repo.  Then I can delete the that Newdash setup repo.

Just tried that out here, and it worked ok.
I'd be nervous about actually running this build in production though, as it's not based upon a stable release.

Do I need to manually create the .env file and write its contents for it to be recognized?

Hmmm, as background there are two main files:

The docker-compose.yml file

This tells which services to start up, where to grab image files from, etc.docker compose

The environment file

This holds = information, and is passed to all of the services when they start. This helps the services to know things like "what is the database address?", "what is the timeout value for gunicorn?" and so on.key``value
It looks like this:
REDASH_LOG_LEVEL: "INFO"
REDASH_REDIS_URL: "redis://redis:6379/0"
REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
REDASH_RATELIMIT_ENABLED: "false"
REDASH_MAIL_DEFAULT_SENDER<EMAIL_ADDRESS>REDASH_MAIL_SERVER: "email"
REDASH_ENFORCE_CSRF: "true"
REDASH_GUNICORN_TIMEOUT: 60

The needs to know where to find the environment file, so that's what the line inside it is for. As long as the path there points to a readable file with those values in it, you should be good.docker-compose.yml``env_file: /opt/redash/env

Do I need to manually create the .env file and write its contents for it to be recognized?

Hmmm, as background there are two main files:

The docker-compose.yml file

This tells which services to start up, where to grab image files from, etc.docker compose

The environment file

This holds = information, and is passed to all of the services when they start. This helps the services to know things like "what is the database address?", "what is the timeout value for gunicorn?" and so on. keyvalue ``
It looks like this:
REDASH_LOG_LEVEL: "INFO"
REDASH_REDIS_URL: "redis://redis:6379/0"
REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
REDASH_RATELIMIT_ENABLED: "false"
REDASH_MAIL_DEFAULT_SENDER<EMAIL_ADDRESS>REDASH_MAIL_SERVER: "email"
REDASH_ENFORCE_CSRF: "true"
REDASH_GUNICORN_TIMEOUT: 60

The needs to know where to find the environment file, so that's what the line inside it is for. As long as the path there points to a readable file with those values in it, you should be good. docker-compose.ymlenv_file: /opt/redash/env ``

Do I need to manually create the .env file and write its contents for it to be recognized?

你写的文件路径不是.env所以不需要创建一个.env的文件。只有你的env_file挂载的文件名字才需要创建。

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

有没有启动日志？贴出来我帮你看看。这样提问我不知道到底哪里有问题？

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

REDASH_LOG_LEVEL: "INFO"
REDASH_REDIS_URL: "redis://redis:6379/0"
REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
REDASH_RATELIMIT_ENABLED: "false"
REDASH_MAIL_DEFAULT_SENDER<EMAIL_ADDRESS>REDASH_MAIL_SERVER: "email"
REDASH_ENFORCE_CSRF: "true"
REDASH_GUNICORN_TIMEOUT: 60
这种写法似乎是无法解析的，你在文件中改为
REDASH_LOG_LEVEL=INFO
这样的写法试试。我目前线上的写法是这样的。

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

类似这样

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

 类似这样

I have successfully deployed the application using the .env method. However, it did not work when I tried to write the parameters into docker-compose.yml as well as the method you suggested. Therefore, I followed the method of configuring with .env file to accomplish the deployment.

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

 类似这样

I have successfully deployed the application using the .env method. However, it did not work when I tried to write the parameters into docker-compose.yml as well as the method you suggested. Therefore, I followed the method of configuring with .env file to accomplish the deployment.

OK.

@Avey777 如果线上用户用的话建议将 server 和 worker 节点的并发度调高一些，不然容易卡死。

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

 类似这样

I have successfully deployed the application using the .env method. However, it did not work when I tried to write the parameters into docker-compose.yml as well as the method you suggested. Therefore, I followed the method of configuring with .env file to accomplish the deployment.

在 worker 节点和 server 节点下添加这个配置就好了。

However, why is it that even after I commented out the env_file and executed the docker-compose.yml, the container still could not start properly?

 类似这样

I have successfully deployed the application using the .env method. However, it did not work when I tried to write the parameters into docker-compose.yml as well as the method you suggested. Therefore, I followed the method of configuring with .env file to accomplish the deployment.

在 worker 节点和 server 节点下添加这个配置就好了。

@Avey777

但是，为什么即使我注释掉了env_file并执行了docker-compose.yml，容器仍然无法正常启动？

类似这样

我已经使用 .env 方法成功部署了应用程序。但是，当我尝试将参数写入 docker-compose.yml 以及您建议的方法时，它不起作用。因此，我遵循了使用 .env 文件进行配置的方法来完成部署。

在 worker 节点和 server 节点下添加这个配置就好了。

@Avey777

Cool

Cool, sounds like it's working now. 😄

Yes, it's great to hear that Redis is working properly for you. Thank you for sharing the deployment steps in a way that you can understand. You can find them here: https://github.com/RedashCommunity/redash/discussions/26.


Bump types-setuptools from <IP_ADDRESS> to <IP_ADDRESS> (#5328)

Pull request opened by the merge tool on behalf of #5328

Merged in ab4283c1947515164a09d6a7ce7a4225b205a8c2


Add floating point range and truncation analysis.

There is significant generalization that can be done for this analysis - just the bare bones for the moment.
Ideally these attributes start to build on each other and share common elements, but since we don't have many yet, it is hard to see how to modularize. Open to suggestions.

PTAL


MQTT format

Sorry. Format is correct? Can not parse data

Sorry. I convert json

I guess you figured it out.


Update rune checks.

Changed values for 6.86 rune locations.

@loopuleasa I thought 0c91866f4a65bace5a15ceb17989c39ccaf6e006 fixed these rune checks.  No?

I did merge a rune check update, but maybe this one has more accurate values. So I am overwriting them.
Both should be pointing in the right spot, so there's no reason for me to prefer one over the other, I just go with the latest.

@loopuleasa The numbers are substantially different, I'll have to check it out and see which is wrong.

@don4of4 the map was changed tho
tell me what you find out


ACM ImportedAt certificates not JSON serializable

Imported certificates fail with:
datetime.datetime(2017, 1, 9, 13, 30, 31, tzinfo=tzlocal()) is not JSON serializable
because the ImportedAt property is still a datetime object rather than an ISO format date string.

Coverage decreased (-0.01%) to 56.556% when pulling e1722b5bad5a1fb4774c7751a6d8ef8034da53cb on sysboy:fix_acm_imported_at into 90c02638a315c78535869ab71a8859d17e011a6a on Netflix:develop.

Just merged this exact same fix under another PR a minute ago.
Thanks!


Create PotLock Styled Components Kit

Problem
Need a styled component library so people can easily fork and we can rapidly development components
User Story:
As a developer, I want to create PotLock styled components, so that I can rapidy prototype
Description
A set of buttons, styled componened kits to rapidly prototype bos components without repeating code
Acceptance Criteria
Outline what needs to be done
Limitations

Dependent on a designer

Resources

outline links and relevant resources, references implementations
Examples of Styled Component Kits
[ ] https://www.boshacks.com/#/nomination.ndctools.near/widget/NDC.StyledComponents
[ ] https://www.boshacks.com/#/nearui.near/widget/index

@codingshot we pretty much have this - all inputs, buttons, donations info etc are componentized
The code needs some cleaning up but that's a separate ticket.