[Improvement][Common template API | Standalone import tool] About migrating to DolphinScheduler

About migrating to DolphinScheduler
It is believed that many users are attracted by the functions of DS, but not everyone can easily migrate the original scheduling configuration to DS, especially the scheduling platform with thousands of jobs such as data warehouse.
Most users really need a tool for easy data migration, and I am one of them, and I need to migrate more than 6000 jobs.
So after reading most of the code, I wrote a simple script, which can directly generate the job information of batch configuration into the database. Of course, I will continue to migrate my flows in Azkaban to DS:

https://github.com/reele/process_definition_import_tool

Therefore, is it necessary to add a common template import interface in DS, so that the data of other scheduling platforms can be formatted and exported, and then directly imported into DS? Or implement a common stand-alone tool for data conversion?
Which version of DolphinScheduler:
-[1.3.6-release]
关于迁移到 DolphinScheduler
相信有很多用户都被DS的功能所吸引，但并不是所有人都能便捷的将原有的调度配置迁移至DS，尤其是数据仓库这类拥有数千个作业的调度平台。
大多数用户确实需要一个便捷数据迁移的工具，我就是其中之一，而且我需要迁移的作业数达到6000多个。
所以我在看过大部分代码之后，写了一个简单的脚本，可以将批量配置的作业信息直接生成到数据库中，当然后续我也准备继续把我在Azkaban中的配置也迁移至DS：

https://github.com/reele/process_definition_import_tool

所以，是否有必要在DS中加一个通用模版导入的接口，可以实现其他调度平台的数据格式化导出，然后直接导入至DS？再或者实现一个通用的独立工具用来做数据转换？

General-purpose tools may be better. We can create a new warehouse for this purpose. Does anyone have any suggestions for this?

About migrating to DolphinScheduler
It is believed that many users are attracted by the functions of DS, but not everyone can easily migrate the original scheduling configuration to DS, especially the scheduling platform with thousands of jobs such as data warehouse.
Most users really need a tool for easy data migration, and I am one of them, and I need to migrate more than 6000 jobs.
So after reading most of the code, I wrote a simple script, which can directly generate the job information of batch configuration into the database. Of course, I will continue to migrate my flows in Azkaban to DS:

https://github.com/reele/process_definition_import_tool

Therefore, is it necessary to add a common template import interface in DS, so that the data of other scheduling platforms can be formatted and exported, and then directly imported into DS? Or implement a common stand-alone tool for data conversion?
Which version of DolphinScheduler:
-[1.3.6-release]

In my opinion, this function is generally implemented by users themselves. There are too many scheduling platforms (including self-development), complex structures and more uncertainties, which makes it difficult to parse and initialize into DS general format (especially on front-end, how to initialize location to define the coordinates of each task?), we can collect the user's own scripts, summarize them to the tool set on the DS official website, and download and modify them by ourselves.

In my opinion, this function is generally implemented by users themselves. There are too many scheduling platforms (including self-development), complex structures and more uncertainties, which makes it difficult to parse and initialize into DS general format (especially on front-end, how to initialize location to define the coordinates of each task?), we can collect the user's own scripts, summarize them to the tool set on the DS official website, and download and modify them by ourselves.

In my understanding, the main reasons for migration difficulties are as follows:

Job data structure is complex and difficult to be solved by simple mapping
Difficult for non-developer user

Typically, the migration of large numbers of jobs (thousands) is done with little regard for the DAG graphics, and the dependencies are likely to be too complex for the data warehouse user to develop with the DAG graphical interface, like mine(One of the simpler):

In general, the scheduler requires the following basic data:
1.Task definition
2.Task Dependencies
3.Scheduling of tasks
4.DAG drawing definition
5.Task group definition
I think, 1-3 are required and 4,5 is optional.
If the common interface of 1-3 can be implemented, the data migration will be very friendly to other platforms.


Add prefix to scripts

Adds --prefix argument to scan_bucket.py and display_infected.py to limit the objects considered for each.
For buckets with many (>100,000) the scan_bucket.py and display_infected.py scripts can incur long running times and non-negligible API costs to iterate an entire bucket. The --prefix option passes a string to the ListObjectsV2 Prefix parameter which limits the objects to iterate and consider.
Note: the long runtime/incurred costs issue remains present in the existing --limit argument as it currently iterates the entire bucket before trimming the returned objects with a Python slice. I'm making these patches on a contract job so I don't have time to fix the pagination logic for this issue. Please advise if you'd like a separate issue to track.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Improve description for promotion rules

I think you used the “Subscription promotion rule” instead of “Subscription order promotion rule”. I know this is super-confusing, but the first one is applied to the order that generates a subscription (i.e. a customer’s first order when they sign up to the PP), while the second is applied to all orders in a subscription.

X-posting from Slack, but that's basically the gist of it. We might want to add some descriptive text/comments to the rule file, the readme, and the rule partial (so non-coder admin can understand the difference too)

Is it a possibility to change the rules name right now? If so, what about "Subscription creation order promotion rule" and "Subscription installment order promotion rule" ?

@luca-landa yep, we should be able to change them.

@aldesantis if somebody is using the extension now, their subscriptions processing will break after renaming the promotion rules classes. How should we handle this? Should we provide a script to fix the names in the database or is it just too much worry because we aren't at 1.0.0 yet?

@luca-landa I think we should make a PR for adding a migration for the promotion rule names too 🙂 Nonetheless, this is still a breaking change because someone might be relying on those class names, so I'm adding a "breaking" label.

@aldesantis created it #216, let me know if everything is okay with this


Prepare this repo to be public

Please assign the LGPL license to this repo, and fill out the README so that this is prepared to be open sourced.

Switched to Apache 2.0 ... published ...  Done!


[Tracking] DB debugging tool

It's currently difficult to debug the content of the database. Everything has to be done manually. We should have a tool to help us

Extract the difference between two databases
Convert the keys and values to something human-readable

Closed as we now have the ledger-db tool to help us with that.


Bump k8s to 1.19.12 and go.uuid to latest master in KEB

Description
Changes proposed in this pull request:

addressing security vulnerability

Related issue(s)

https://github.tools.sap/kyma/security-scans/issues/8490

Please bump the image here:  https://github.com/kyma-project/control-plane/blob/5909d9db36d62d8aa0f39ee485b3e93744bc897d/resources/kcp/values.yaml#L21

/retest


[cherry-pick][branch-3.2][Feature] support lzo block decompression (#30923)

Fixes #issue
What type of PR is this:

[ ] BugFix
[x] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.2
[ ] 3.1
[ ] 3.0
[ ] 2.5

[FE Incremental Coverage Report]
:heart_eyes: pass : 0 / 0 (0%)

[BE Incremental Coverage Report]
:heart_eyes: pass : 29 / 29 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/util/compression/block_compression.cpp
29
29
100.00%
[]


fix: Backup old wallet salt key to gaia

Try out this version of the Hiro Wallet - download extension builds.

This PR setup a backup of the user wallet salt in Gaia encrypted. This is to fix #2238
This will be used to pass to the dApp later
Test1
To reproduce you will have to simulate a wallet upgrade (see https://github.com/hirosystems/stacks-wallet-web/pull/2172)

Install the live wallet (https://github.com/hirosystems/stacks-wallet-web/releases/tag/v3.2.0), place it in a folder hiro-wallet
Setup a wallet (note down the seed)
Look in the localStorage (open dev tool, tab Application , select Local Storage chrome:extension) and note down the stacks-wallet-salt
Install the new wallet (using the same instruction as the simulation steps)
it will be on the unlock screen, unlock it
Check the localStorage: You should have an entry with key = 'salt-used-in-app-key-in-gaia' and the value is the salt you noted down

Test 2

Now completely uninstall the wallet or use another browser, and install this build again.
Enter the same seed as Test1
Look in the localStorage: You should have an entry with key = 'salt-used-in-app-key-in-gaia' and the value is the salt you noted down

another test is to login with a dapp in Test1
close the browser and update with the new build.
Go to the dapp logout/login, the choose account popup should be triggered.
You should have an entry with key = 'salt-used-in-app-key-in-gaia' and the value is the salt you noted down

@Eshwari007 @timstackblock If you can have a look at this that would be great !

@beguene test 1 was successful for me but test 2 was NOT. There was no backup salt saved in localStorage when I reinstalled the wallet.

@beguene test 1 was successful for me but test 2 was NOT. There was no backup salt saved in localStorage when I reinstalled the wallet.

Similar test findings as @timstackblock and @fbwoolf

@aulneau @yknl Please give your feedback on this PR we would like it to be put line ASAP

@timstackblock this PR is not current. @beguene should this one be closed to avoid confusion?


Investigate startup issues with Kestrel

@anurse @pakrym @halter73

Performance

I too can use random software engineering terms. Multithreading. Concurrency. Thunk.
What's the context on this? Kestrel being slow to start-up?

Yep, the issue you've seen when trying to start the server for every test.

Who is looking into this?

I don't understand what this issue is, so closing this.


Add support for Topiom Rowing Machine

The Bluetooth module is supported by KinoMap and FitShow apps and is displayed as FS-4F3000 within them but does not show up as a supported device within TrackMyIndoorWorkout.
I can help implement / provide any debug information if you point me in the right direction.

I mainly support FTMS compatible devices. The main question is if the machine exposes the 0x1826 Bluetooth Low Energy (BLE) service.

When trying my app, make sure that you exit any app on any other device you used the fitness machine with. Even if an application is in the background it may hold the Bluetooth connection to the machine and the console won't advertise itself so it won't be visible to any other app.
In case my app doesn't show the machine then you can turn off the compatibility filtering in the Expert preferences section. This will show every Bluetooth device which are advertising, not only the ones my app deems compatible. Does the machine show up? If yes, then click on the row (but not the play icon) and that will open an accordion section with details about the machine. What services does the machine advertise (Service UUIDs)?
In case of an unsupported protocol, or the machine not showing up you can also try the NRF Connect app. That's a generic Bluetooth app which is capable of connection to any device and even logging traffic.
If the machine is FTMS compatible (or other BLE standard, like CSC cycling sensor, power meter sensor, RSC sensor) and I don't show it then I'd need to debug the situation. I developed an internal logging mechanism into my app which is also under the Expert preferences: you can set the logging level to INFO and then later export the logs.
In case of proprietary protocol the situation is hard. The best would be if I can use a similar machine. What make and model is it? It's not impossible to develop a support without having the machine, such as happened with Schwinn 170 / 270 / 570. However recently I learned that the support has problems (#433) and I was still not able to solve because I don't have those machines. Are you a developer by any chance?

Ok, so it's a rower. In this case my app expect the device to advertise 0x1826 FTMS Fitness Machine service support and within that a 0x2AD1 Rower Data characteristic (https://github.com/oesmith/gatt-xml/blob/master/org.bluetooth.characteristic.rower_data.xml).
Maybe it's just that the console doesn't explicitly advertise that. However it's infeasible for my app to actually connect to everything it sees in order to determine if the support is fine or not. I'd say that you can try to disable the compatibility filtering and try to connect to the device (preferably also logging enabled).
With Kinomap stuff is more manual: you need to pick the type of your machine. One application I'm also curious about - and this is easier to use the NRF Connect because it's a normal user app - is GymTrakr. Please give that a try and report back if it can simply support the rower's console.

The machine is advertising the following service UUIDs 180A, 180F, FFF0 and looking them up it may be that FFF0 is a propriety protocol?
GymTrakr doesn't connect.
The logs when trying to connect are:
{FIND_DEVICES}  {startScan}  {Scan initiated}  {16 June 2023 12:12:46.136 PM}  {INFO} {FIND_DEVICES}  {ScanResult}  {ScanResult{device: BluetoothDevice{id: CB:F0:01:00:30:4F, name: FS-4F3000, type: BluetoothDeviceType.le, isDiscoveringServices: false, _services: [], advertisementData: AdvertisementData{localName: FS-4F3000, txPowerLevel: null, connectable: true, manufacturerData: {1002: [115, 48, 79, 48, 0, 0, 0, 0, 0, 213]}, serviceData: {}, serviceUuids: [0000180a-0000-1000-8000-00805f9b34fb, 0000180f-0000-1000-8000-00805f9b34fb, 0000fff0-0000-1000-8000-00805f9b34fb]}, rssi: -72, timeStamp: 2023-06-16 12:12:46.247316}}  {16 June 2023 12:12:46.256 PM}  {INFO} {logVersion}  {version}  {1.1.140 build 140}  {16 June 2023 12:12:49.819 PM}  {ERROR} {FIND_DEVICES}  {goToRecording}  {Could not infer sport of the device}  {16 June 2023 12:12:49.824 PM}  {ERROR} {FIND_DEVICES}  {ScanResult}  {ScanResult{device: BluetoothDevice{id: CB:F0:01:00:30:4F, name: FS-4F3000, type: BluetoothDeviceType.le, isDiscoveringServices: false, _services: [], advertisementData: AdvertisementData{localName: FS-4F3000, txPowerLevel: null, connectable: true, manufacturerData: {1002: [115, 48, 79, 48, 0, 0, 0, 0, 0, 213]}, serviceData: {},  serviceUuids: [0000180a-0000-1000-8000-00805f9b34fb, 0000180f-0000-1000-8000-00805f9b34fb, 0000fff0-0000-1000-8000-00805f9b34fb]}, rssi: -72, timeStamp: 2023-06-16 12:12:46.247316}}  {16 June 2023 12:12:49.835 PM}  {INFO} {logVersion}  {version}  {1.1.140 build 140}  {16 June 2023 12:13:01.518 PM}  {ERROR}
I'll log some traffic with NRF Connect next and we may be able to determine if it's using a standard protocol but not advertising it as such.

I tried to determine Bluetooth support by web searches for the model, but the results are kind of inconclusive. Some reviews only mention Bluetooth support in general (which with some other machine only means capability to pear a BT heart rate monitor to the console). Some articles mention that apps support the console, but I never saw FTMS as a keyword mentioned. The fact that GymTrakr is not recognizing it also points towards a proprietary protocol. Sometimes it's hard to crack those (for example #275).
We have a few venues. One is to get help from the manufacturers, since if Kinomap supports it then there must be some proprietary documentation. The other is reverse engineering. For that the key is to remember what strokes per minute and what resistance level you were on and pair specific data packets to those. For example row with light resistance but high spm, and also high resistance lower spm. Also remember pace / speed. Then the specific bytes might be identifiable. That's how I reverse engineered the Precor Spinner Chrono Power as the first supported device when I started this app. But I was not able to crack the Bowflex #275 yet.

One more thing: does this machine have different versions with different consoles? When articles are inconclusive then sometimes this is the case. Sometimes the Bluetooth support require some extra pay for the "smarter" console. Sometimes console firmware updates could bring in new capabilities as well.

Shot the company an email, it could never hurt:
"I'm a mobile application developer who'd like to support your rowers. My application is https://trackmyindoorworkout.github.io/
Recently a customer of yours was asking for support: https://github.com/TrackMyIndoorWorkout/TrackMyIndoorWorkout/issues/436
My searches were inconclusive, so I curious if your consoles are capable of Bluetooth FTMS Rower Data support?
Does your bluetooth support depend on console version, or firmware version?
Since Kinomap and a few applications seems to support your in case you are using a proprietary protocol would you be able to provide me a copy. I doesn't have to be fancy, but only enough so I can develop a support.
My application is free, free of ads, and open source."
In case of Bowflex I was never able to connect even after LinkedIn forensics, however this is a smaller company so they might be less bureaucratic and willing to talk to a tiny fish like me.

There are at least two consoles one newer "TM3" and mine being the "TM2".
Cheers for reaching out to the company, lets see if they reply.
In the mean time I've logged some traffic when connected with KinoMap.
As its a water rower, I believe the only transmitted value with be RPM, then all others will be derived? For example watts and distance calculations are wildly different in KinoMap compared to what the LCD on the rower shows.
Here's the log:
Log 2023-06-19 10_48_50.txt

I got a response back from the company:

Dear Csaba,
All of our current monitors are Bluetooth-compatible, and we no longer offer the older version of displays. Rest assured that our current version is functioning properly. Thank you for your attention and continued support of our products.
If you have any further questions or concerns, please feel free to reach out.
Best regards,
Doris

I'm following up about FTMS or documentation. It's promising that they are communicating.

I'm able to calculate calories from heart rate (#154), however it can be extremely off especially between various sports (see for example #369). Calorie tune feature helps to apply a linear modification factor to that value.
Then I'm able to calculate watts from calorie delta. In case of cycling I have formulas to calculate speed based on watts. Different sports need different formulas though, because running, rowing or swimming won't be affected as much by wind resistance for example, but other physical factors could come in strong. See #398.
This wouldn't be able to tell spm (strokes per minute), and the transitive multi stage approximations would result in widely varying numbers. Although right now I have several machines which leave out some key measurements from the FTMS data for whatever reason (for example Schwinn IC4/IC8 is missing the calories. So far Schwinn 170 / 270 / 570 is the bike which has a proprietary protocol and I truly use multi stage transitive approximations (calories -> watts -> speed): #240
I'd still want to identify some data in the stream other than the heart rate. Another venue could be to look for console upgrade if Topium has any (for example water rower has S3 -> S4 upgrade https://www.waterrower.com/us/products/s3-to-s4-monitor-upgrade-2) or Bluetooth Com module (FTMS compatible, can plug in on the console https://www.waterrower.com/us/products/waterrower-commodule).
I've also seen some hardware projects:

https://github.com/zpukr/ArduRower
https://github.com/laberning/openrowingmonitor

I can co-operate to tune-in multi-stage approximation like with the the Schwinn 170, it sounded working fine (although it seems to have unrelated problems right now #433).

One more thing: in case the shaft of the resistance wheel is exposed, you might be able to attach a cycling cadence sensor (CSC Bluetooth standard) on it. This is how I support the Old Danube kayak ergometer: #384
This is Old Danube specific right now, but I can extend / modify it if it's applicable. That would be able to provide some kind of a data, although the spm is not correlated to the flywheel speed in a linear way, because there's also resistance setting.


PHP7 compatibility issue

Hello,
I'm upgrading my project to php7 and I'm using a report generated by phpcs with PHPCompatibility. I got two errors related to the bundle and I would like to know if this is something you decided not to change because it can't cause any damage or we should provide a PR.
I'm using doctrine 2.4.8 but I just checked in the repository and the master branch contains the same code. Here are the errors:
FILE: .../orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php
----------------------------------------------------------------------
FOUND 2 ERRORS AFFECTING 2 LINES
----------------------------------------------------------------------

917 | ERROR | 'clone' is a reserved keyword introduced in PHP version
|       | 5.0 and cannot be invoked as a function (T_CLONE)
|       | (PHPCompatibility.PHP.ForbiddenNamesAsInvokedFunctions.Found)
942 | ERROR | 'clone' is a reserved keyword introduced in PHP version
|       | 5.0 and cannot be invoked as a function (T_CLONE)
|       | (PHPCompatibility.PHP.ForbiddenNamesAsInvokedFunctions.Found)
I think that deleting the parenthesis and maybe assigning the value first to a variable will correct it.
Is anybody already using the bundle with php7? Can you give me a feedback?

@anacicconi the parenthesis are indeed not needed but there's no problem at all. You can check here too: https://3v4l.org/chnBj

Closing as invalid.

Ok, thanks for your answer @lcobucci.


About SharpCompress.Common.EntryStream

why EntryStream override Stream's function ,but not supported

EntryStream is a forward-only subsection of a file.  Since it decompresses you cannot seek it.  I imagine that's what you're running into.


Ensure config file contains required params

Coverage increased (+0.5%) to 66.418% when pulling 887568c8bda1bda74c072c9217891ae45907e49b on validate-json into 313ca8f8ecf21261c0f33301cc833fc454eece5f on master.


Gnome 3.34.1

Motivation for this change
Things done
Staging updates will come in a separate pr.

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Notify maintainers
cc @

We'll just say this point release was very needed and proactive :smile:

Yeah, the zero releases tend have a few issues :/
Built everything (and a large part of the reverse dependencies). Think this is good to go tbh.

moved evince back to tarball as they now contain meson
removed gio-unix ld flag from gnome-terminal as that's fixed upstream
ran up a VM to check that gdm/gnome-session works correctly

@hedning Seems good to me also. Will run some checks now and probably merge shortly.

gnome-shell and mutter is late updating, so we'll just do them in a separate pr.


License link on the landing page is broken

Go to http://testng.org/doc/index.html
Click Apache 2.0 link in the bottom of the page
http://testng.org/license page appears with 404 response code
Please, update the link or create corresponding page

Fixed.


Great work

I can see there's not much going on anymore in this repo, but I'd like to say I really appreciate your work. It's good stuff, thankyou.

Great to hear, thanks!!


Is it possible to get a list of available placeholders in the document?

ex:
template = Sablon.template(File.expand_path("~/Desktop/template.docx"))
template.get_placeholders # ['title','author', ...]

@Xorg1984 currently there is no public API to access the fields. Also I'm not sure how easily such a function can be generalized. The fields can be nested within loops and conditionals, they can also use local variables. This makes the expression not very useful outside of it's context.
What are you trying to achieve?

my task - get all document placeholders (this will be a simple strings), show interface to set values to placeholders and then generate a new document with values
maybe parse document independently?  what way you recommend?

Sablon was not built with that use-case in mind. Therefore I won't introduce public API to get the output you'd like. The behavior would be too brittle and not clear when loops and conditions are in play.
Having said that, you can use the internal building blocks of Sablon to extract the data you need. Following is a simple script executed against a fixture template:
path = "/Users/senny/Projects/sablon/test/fixtures/recipe_template.docx"

parser = Sablon::Parser::MailMerge.new
fields = []

Zip::File.open(path).each do |entry|
  entry_name = entry.name
  content = entry.get_input_stream.read
  if entry_name == 'word/document.xml' || entry_name =~ /word\/header\d*\.xml/ || entry_name =~ /word\/footer\d*\.xml/
    document = Nokogiri::XML(content)
    fields += parser.parse_fields(document)
  end
end

fields.map(&:expression)
# => ["=title", "=intro", "ingredients:each(ingredient)", "=ingredient.amount", "=ingredient.name", "ingredients:endEach", "=preparation", "=metadata.generator", "=details.author.name"]

Depending on your templates you might just need to parse word/document.xml.

Thanks!

@avdovin how did you get on this, did you manage to get what you want working?
I am also looking to read all merge_fields from the document so the user can fill out the fields in the interface.
Was the code in the comment sufficient? https://github.com/senny/sablon/issues/16#issuecomment-103446793
Thanks


Mistakes in the .pdf docs

Hi,
here a collection with mistakes I found in the .pdf docs.
-7.5, Gabor Noise: the "do_filter" point is listed twice.
-4.1 Shader Types, mention the type "light", but light brings up a compile error. (Unknown shader type: light).
Not a big deal, but nice to fix once.
Cheers,
Thomas

I can’t find the issue with “do_filter” in the current docs—looks like that has been fixed.
On page 2 it says “Surface and light shaders are the same thing”. Given this, perhaps it makes sense to omit mention of “light” shader types on page 13.

Fixed now, thanks.


Replace gMaps with OSM

Ahoj,
how about replacing the proprietary Google Maps with OpenStreetMaps? Open Knowledge, free software and so on...

duplicate of #70


Min zooms for railways don't make sense

The description to railway=rail in the streets layer says

ways with service=* on zoom level 8+, other ways on zoom level 10+

For railways service is yard, spur, siding, or crossover. These are less important than ways without a service tag, and styles like osm-carto render them as less significant. To include them on z8 before the main lines seems like an obvious error that the 8 and 10 got flipped.
cc @Nakaner

The documentation is wrong. The Tilemaker implementation writes all railway tracks on zoom level 10+ but railway=rail/narrow_gauge without service=* on zoom level 8+.
Writing unimportant features on lower zoom levels than important features would not make sense.


🛠 [TASK] : Login / Connect Wallet / Confirmation

Summary
Wallet connect confirmation
Description
As a user, I need a wallet connect confirmation screen, so that I can check a few key details to have confidence my wallet was properly connected.
GIVEN user confirms wallet connect in browser extension
WHEN wallet connect is completed
THEN show confirmation modal with connected wallet details
AND THEN add new wallet to "linked wallets" in My Account
AND THEN update logged in profile panel on header
Relevant details to include on modal:

Checksum
ADA balance
Current stake pool delegated (nice to have)
Existing registrations (eg voter, proposer)

completed with mve2


[BUG] Azure.Data.Tables throws an exception and failes TableTransactionAction when object to delete does not exist

Library name and version
Azure.Data.Tables 12.6.1
Describe the bug
Lets start with TableClient.DeleteEntity. If you call this with a partitionKey and rowKey that does not exists (also for Etag.All or without Etag), it returns a Response with a 404, while the documentation says:
If the entity exists, the Response indicating the result of the operation. If the entity does not exist, null
However, it does not throw an Exception.
If you use a TableTransactionAction with TableTransactionActionType.Delete and one of the entries does not exist, a RequestFailedException exception will be thrown and the batch will be rollbacked, probably because an individual entry was not existent and thus resulted in a 404 .
Expected behavior

For TableClient.DeleteEntity to match te documentation to return null when the item did not exist
For TableClient.SubmitTransaction with a TableTransactionAction that has some TableTransactionActionType.Delete to succeed even if the to be delete items did not exist

Actual behavior

TableClient.DeleteEntity returns a Response with a 404 instead of null when the item did not exist.
TableClient.SubmitTransaction with a TableTransactionAction that has some TableTransactionActionType.Delete throws a RequestFailedException when some of the items to be deleted do not exist.

Reproduction Steps
Response exists = _TableClient.DeleteEntity("partitionKey", "rowkey", ETag.All);
Assert.IsTrue(null, exist);

Response exists = _TableClient.DeleteEntity("partitionKey", "rowkey");
Assert.IsTrue(null, exist);

List<TableTransactionAction> deletes = new List<TableTransactionAction>();
deletes.Add(new TableTransactionAction(TableTransactionActionType.Delete,
                        new TableEntity { PartitionKey ="partitionkey", RowKey = "rowkey") }));

_TableClient.SubmitTransaction(deletes)

Environment
Azure WebApp

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Hi @edwinvdb Sorry you encountered this problem.
The documentation bug is a known issue being tracked by #25561 - This also relates to a larger issue tracked by #25626 which we should be able to address in the near term which allows configuration of acceptable error responses that should not throw.
Regarding the actual behavior, throwing the 404 is currently intended for the Delete operation when the item does not exist, and this behavior is consistent with the guideline that all failure cases will throw by default.
In the case of batch operations, this behavior is controlled by the service, and cannot be changed by the client. This is the intended behavior of transactional batch operations where all operations succeed "all or nothing". The documentation can be found here

Hi @christothes , thanks for your quick response.
You say:
"Regarding the actual behavior, throwing the 404 is currently intended for the Delete operation when the item does not exist, and this behavior is consistent with the guideline that all failure cases will throw by default."
But this isn't true, the DeleteEntity won't throw, but it will just return a response with a 404 when there was nothing to delete. This seems quite reasonable to me: when you ask to delete something, you don't want it to exist and as that was already the case, from a functional perspective there is no error. This also aligns with normal SQL database behavior for a DELETE statement.
For insert/updates we have an Upsert operation, maybe there could also be a "DeleteIfExists" operation?
You write "This is the intended behavior of transactional batch operations where all operations succeed "all or nothing""
I agree with the "all or nothing" part, but the DeleteEntity operation succeeds code-flow wise, even if the object to delete did not exist, as no exception is thrown and you just get a statusresponse back with a 404. This is different from CreateEntity, which does throw when the entity already exists.
So it seems to me there is a discrepancy here between a DeleteEntity that kind of succeeds without an exception and a batch of Deletes that don't succeed and throw an exception. Deleting an entity only throws within the context of a batch operation and that seems inconsistent.
What is your view on this?

Hi @christothes , thanks for your quick response.
You say: "Regarding the actual behavior, throwing the 404 is currently intended for the Delete operation when the item does not exist, and this behavior is consistent with the guideline that all failure cases will throw by default."
But this isn't true, the DeleteEntity won't throw, but it will just return a response with a 404 when there was nothing to delete. This seems quite reasonable to me: when you ask to delete something, you don't want it to exist and as that was already the case, from a functional perspective there is no error. This also aligns with normal SQL database behavior for a DELETE statement.

Ah, you are correct. Thanks for the correction here - I had confused the fact that we actually do internally throw, but just swallow the exception. We did want Delete to be treated as non-failing operations, since to try and reason about the state of the entity before the Delete is inherently a race condition.

For insert/updates we have an Upsert operation, maybe there could also be a "DeleteIfExists" operation?
You write "This is the intended behavior of transactional batch operations where all operations succeed "all or nothing"" I agree with the "all or nothing" part, but the DeleteEntity operation succeeds code-flow wise, even if the object to delete did not exist, as no exception is thrown and you just get a statusresponse back with a 404. This is different from CreateEntity, which does throw when the entity already exists.
So it seems to me there is a discrepancy here between a DeleteEntity that kind of succeeds without an exception and a batch of Deletes that don't succeed and throw an exception. Deleting an entity only throws within the context of a batch operation and that seems inconsistent.
What is your view on this?

Yes, the resulting behaviors are inconsistent, but, as mentioned above, we swallow the failure issued by the service for the Delete operation as an enhancement that we can control on the SDK side. For batch operations, we are limited by the service side behavior. I think the best we can do in this case is to improve the documentation to avoid unexpected behavior.

Yes, the resulting behaviors are inconsistent, but, as mentioned above, we swallow the failure issued by the service for the Delete operation as an enhancement that we can control on the SDK side. For batch operations, we are limited by the service side behavior. I think the best we can do in this case is to improve the documentation to avoid unexpected behavior.
Yes, I understand. Hopefully the service will improve to support "DeleteIfExists" batch operations ;-)

I'll leave closing the issue up to you, as I don't know if this issue should remain for the documentation improvement.

I'll leave closing the issue up to you, as I don't know if this issue should remain for the documentation improvement.

I'll track this with #30494


It's not user-friendly to show the code error info when input the password

It's not user-friendly to show the code error info when input the password which has no 6 characters long :）

We will refine error messages in next release by #807


Use observer pattern with threads observers instead of queues for starting workers

Something like this:
https://en.wikipedia.org/wiki/Observer_pattern#Python

Probably not necessary. Current solutions works.


[BUG] -

Internal/External
External   otherwise.
Area
Other
Cardano-node issues connecting to tesnet relay after syncing 100%
Summary
After the node is synced 100%. I constantly get DNS Subscription error. This node seems to be running perfectly then after an hour I get an error.
Steps to reproduce
Steps to reproduce the behavior:

on a centos7 docker container install cardano-node from https://hydra.iohk.io/build/16669273/download/1/cardano-node-1.35.0-linux.tar.gz
Start cardano-node

Expected behavior
node is started and node is properly syncing.
System info (please complete the following information):

OS Name: Centos
OS Version 7.7.1908
Node version
cardano-node 1.35.0 - linux-x86_64 - ghc-8.10
git rev 0cd6878769a10097e7c780e15efb9919fb94f669
CLI version
cardano-cli 1.35.0 - linux-x86_64 - ghc-8.10
git rev 0cd6878769a10097e7c780e15efb9919fb94f669

If the git revision is reported to be<PHONE_NUMBER>000000000000000000000000000000, please use cabal build instead of cabal install to build the binaries because cabal install does not record the git revision in the binaries.
Screenshots and attachments
[d5117cb8:cardano.node.DnsSubscription:Error:2230] [2022-07-10 19:10:13.59 UTC] Domain: "relays-new.cardano-testnet.iohkdev.io" Application Exception: <IP_ADDRESS>:3001 ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Mary StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (AlonzoEra StandardCrypto)) (': * (ShelleyBlock (Praos StandardCrypto) (BabbageEra StandardCrypto)) ('[] *))))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Mary StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (AlonzoEra StandardCrypto)) (': * (ShelleyBlock (Praos StandardCrypto) (BabbageEra StandardCrypto)) ('[] *))))))))) (ServerAgency TokNext TokMustReply)
[d5117cb8:cardano.node.ErrorPolicy:Notice:2023] [2022-07-10 19:10:13.60 UTC] IP <IP_ADDRESS>:3001 ErrorPolicySuspendConsumer (Just (ApplicationExceptionTrace ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Mary StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (AlonzoEra StandardCrypto)) (': * (ShelleyBlock (Praos StandardCrypto) (BabbageEra StandardCrypto)) ('[] *))))))))) (Tip HardForkBlock (': * ByronBlock (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyEra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Allegra StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (ShelleyMAEra 'Mary StandardCrypto)) (': * (ShelleyBlock (TPraos StandardCrypto) (AlonzoEra StandardCrypto)) (': * (ShelleyBlock (Praos StandardCrypto) (BabbageEra StandardCrypto)) ('[] *))))))))) (ServerAgency TokNext TokMustReply))) 20s
[d5117cb8:cardano.node.DnsSubscription:Notice:2321] [2022-07-10 19:10:14.67 UTC] Domain: "relays-new.cardano-testnet.iohkdev.io" Connection Attempt Start, destination [2a05:d01c:321:2101:0:1:0:1]:3001
[d5117cb8:cardano.node.DnsSubscription:Notice:2322] [2022-07-10 19:10:14.70 UTC] Domain: "relays-new.cardano-testnet.iohkdev.io" Connection Attempt Start, destination <IP_ADDRESS>:3001
[d5117cb8:cardano.node.DnsSubscription:Notice:2323] [2022-07-10 19:10:14.73 UTC] Domain: "relays-new.cardano-testnet.iohkdev.io" Connection Attempt Start, destination [2a05:d01c:321:2100:0:1:0:1]:3001
[d5117cb8:cardano.node.DnsSubscription:Notice:2324] [2022-07-10 19:10:14.75 UTC] Domain: "relays-new.cardano-testnet.iohkdev.io" Connection Attempt Start, destination <IP_ADDRESS>:3001
[d5117cb8:cardano.node.DnsSubscription:Notice:2322] [2022-07-10 19:10:14.77 UTC] Domain: "relays-new.cardano-testnet.iohkdev.io" Connection Attempt End, destination <IP_ADDRESS>:3001 outcome: ConnectSuccessLast
[d5117cb8:cardano.node.ErrorPolicy:Notice:2023] [2022-07-10 19:10:14.77 UTC] IP [2a05:d01c:321:2101:0:1:0:1]:3001 ErrorPolicySuspendConsumer (Just (ConnectionExceptionTrace (SubscriberError {seType = SubscriberParallelConnectionCancelled, seMessage = "Parallel connection cancelled", seStack = []}))) 1s
[d5117cb8:cardano.node.ErrorPolicy:Notice:2023] [2022-07-10 19:10:14.77 UTC] IP [2a05:d01c:321:2100:0:1:0:1]:3001 ErrorPolicySuspendConsumer (Just (ConnectionExceptionTrace (SubscriberError {seType = SubscriberParallelConnectionCancelled, seMessage = "Parallel connection cancelled", seStack = []}))) 1s
[d5117cb8:cardano.node.ErrorPolicy:Notice:2023] [2022-07-10 19:10:14.77 UTC] IP <IP_ADDRESS>:3001 ErrorPolicySuspendConsumer (Just (ConnectionExceptionTrace (SubscriberError {seType = SubscriberParallelConnectionCancelled, seMessage = "Parallel connection cancelled", seStack = []}))) 1s

Additional context
I believe this could be due to the fact that I am running the docker container on my macbook. Maybe the connection could've been interrupted.

Hi @doncoriolan; Is this issue still relevant when using the 1.35.3 release?

Closing this. Please reopen if this is still relevant.


Fix gif in README

Rename gif is currently build_tsconfig.gif, changed to rename.gif.

Hi @jsilvermist, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

Thanks! Sorry for the delay!


Failed to resolve:'com.afollestad:material-dialogs:<IP_ADDRESS>

When I am run this ticker app,then during building gradle,an error is occur Shows as - Failed to resolve:'com.afollestad:material-dialogs:<IP_ADDRESS>

Please help me,to solve this.thanks in advance!

@viveksonihmr are you get solution of above problem...?
if you get it please help me to solve it
i'm getting same problem

The latest version of the app shouldn't have this problem.  Try pulling the latest from git and cleaning your build.
This should resolve the issue, but if it doesn't, feel free to re-open this!

Thanks jens! This problem have been solved,Now its working fine.
:)
On Tue, Nov 17, 2015 at 3:39 AM, Jens Knutson<EMAIL_ADDRESS>wrote:

The latest version of the app shouldn't have this problem. Try pulling the
latest from git and cleaning your build.
This should resolve the issue, but if it doesn't, feel free to re-open
this!
—
Reply to this email directly or view it on GitHub
https://github.com/spark/photon-tinker-android/issues/1#issuecomment-157130898
.

--
Er.Vivek Soni
Integrated Circuit Interface Lab
College of Electronics and Radio Engineering
Kyunghee University(KHU),Suwon,South Korea


ReferenceError: Urls is not defined when clicking on Online logo

This has happened two times in Firefox 66.0 on Windows 10, see Sentry for more information
https://sentry.io/organizations/dotkom/issues/968331210/?project=204971&referrer=github_plugin
ReferenceError: Urls is not defined
  at modules (external "Urls":1:1)
  at __webpack_require__ (webpack/bootstrap 519b222291c20d2217fa:54:1)
  at modules (frontpage-519b222291c20d2217fa.js:304:63)
  at __webpack_require__ (webpack/bootstrap 519b222291c20d2217fa:54:1)
  at modules (frontpage-519b222291c20d2217fa.js:245:75)
...
(9 additional frame(s) were not displayed)

So this probably happens because of the Urls being loaded as a separate script to the main page and other scripts. It is imported into the other JS files as if it was a JS module, but it is actually just added to the global scope (I think?). If it is not added in time for the other scripts to run, all JS that uses it will break on load. Which means basically nothing important on the front-page will display.
You can test this out for yourselves by going to the network tab in dev tools and block the /jsreverse/ endpoint.
I happens maybe a few times a year, and I don't know if we really care?


Prepare 0.18.3 release

Summary
Bump version numbers for 0.18.3 release.  No fixes to current release notes are necessary this time (there's only one note, anyway).
Details and comments
This version release is currently blocked on the following PRs being merged to main, and subsequently being backported to stable/0.18.

[x] #7071 (backport PR #7074)
[x] #7037 (backport PR #7077)
[x] #7019 (backport PR #7076)

All blockers are now merged, and we're ready for the release.

Before we merge this we need to check the release notes. For example there is a typo here: https://github.com/Qiskit/qiskit-terra/blob/stable/0.18/releasenotes/notes/sx-error-map-408f28a6a0c05fd2.yaml#L6 Let's check the output in the docs ci job and make sure it all looks good before we merge this and tag the release

Ah yeah, of course the new notes that I blocked this release weren't in my branch when I did the build test earlier.  I'd checked all the incoming ones in text form when I wrote them in their respective PRs, but must have missed the ` ' transposition.


link to published documentation

The README.md file needs a link to the URL where the documentation for this repo is published.

The old documentation was available in https://nsls-ii.github.io/hklpy/. I don't think it's up-to-date or is currently updating via CI, as the repo was moved to a new org.

In #21, the documentation was brought up to date.  Can we get it published
now with the other bluesky documentation?
On Sat, Sep 26, 2020, 11:42 AM Maksim Rakitin<EMAIL_ADDRESS>wrote:

The old documentation was available in https://nsls-ii.github.io/hklpy/.
I don't think it's up-to-date or is currently updating via CI, as the repo
was moved to a new org.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bluesky/hklpy/issues/31#issuecomment-699518815, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AARMUMB2XGN55QDBKPMFKB3SHYKYBANCNFSM4R27X3EA
.


Error creating repository: Forbidden (HTTP 403)

Hello 👋
I used access tokens instead of a password and received this problem.

Hi, your Personal Access Token does not seem to have ability to create repositories in the current context. I'm not sure why is that, but I'm sure this is an API problem and not a hub client problem.

Okay, Thank you.


Fix algorithm for counting runs

make the algorithm simpler and add some cases where origin algorithm fails

Codecov Report

Merging #104 into master will decrease coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #104      +/-   ##
==========================================
- Coverage   86.98%   86.93%   -0.05%
==========================================
  Files          44       44
  Lines        3250     3246       -4
==========================================
- Hits         2827     2822       -5
- Misses        423      424       +1


Conformance Crud tests: Patch should be implemented

Implement HTTP Patch on entities

Sample PUT:
http://localhost:8080/v1.0/Things(1)
{"description":"This is a Updated Test Thing From TestNG"}

Sample PATCH:
http://localhost:8080/v1.0/Things(1)
{"description":"This is a PATCHED Test Thing From TestNG"}

fixed


mark unsafe the non-spec obj parameter

There are two magic variables in ansible-operator, one for the _spec and containing the whole object. If we have some jinja syntax in the annotations or in the status, markUnsafe currently misses it and it fails when injecting the variable in the playbook/role.

Hello,
Can I get a review on this change ? What can I do to help getting this merged ?
Thanks


Fixed #35661 -- Fixed test_too_many_digits_to_rander() test crash on PyPy.

I think it's worth moving this extreme case to a separate test method.

https://docs.python.org/3/library/stdtypes.html#integer-string-conversion-length-limitation
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10735
https://github.com/python/cpython/issues/95778

Thanks Michał Górny for the report.
ticket-35661

I should back-port this into 5.1, 5.0 and 4.2 right?

We worked on PyPy3.10 support in Django 5.1 (ticket-34986), so I'd only backport to stable/5.1.x.


Mark @typedef as a type declaration

This is only important right now for marking uses of deprecated tags due to the way that deprecated type references are computed. But I'm surprised it hasn't caused problems elsewhere.
Fixes #39466

@Kingwl you might also be interested in this, although the bug was in code that I wrote. =)

should include: Yes. does include: No. I'll fix that and add tests.


Initial content

Pull Requests

https://github.com/Senzing/docker-yum/pull/2
https://github.com/Senzing/docker-yum/pull/3
https://github.com/Senzing/docker-yum/pull/4
https://github.com/Senzing/docker-yum/pull/5
https://github.com/Senzing/docker-yum/pull/6
https://github.com/Senzing/docker-yum/pull/7
https://github.com/Senzing/docker-yum/pull/9
https://github.com/Senzing/docker-yum/pull/9
https://github.com/Senzing/docker-yum/pull/10
https://github.com/Senzing/docker-yum/pull/11
https://github.com/Senzing/docker-yum/pull/12
https://github.com/Senzing/docker-yum/pull/13
https://github.com/Senzing/docker-yum/pull/14


Add statistics

Codecov Report
Base: 73.91% // Head: 70.73% // Decreases project coverage by -3.17% :warning:

Coverage data is based on head (89252ff) compared to base (b47eb03).
Patch coverage: 1.20% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master      #95      +/-   ##
============================================
- Coverage     73.91%   70.73%   -3.18%
  Complexity      608      608
============================================
  Files            95       97       +2
  Lines          1825     1907      +82
  Branches        195      204       +9
============================================
  Hits           1349     1349
- Misses          407      489      +82
  Partials         69       69

Impacted Files
Coverage Δ

...ain/java/seedu/phu/model/internship/Statistic.java
0.00% <0.00%> (ø)

src/main/java/seedu/phu/ui/MainWindow.java
0.00% <0.00%> (ø)

src/main/java/seedu/phu/ui/StackedBarPanel.java
0.00% <0.00%> (ø)

...seedu/phu/model/internship/ApplicationProcess.java
47.61% <16.66%> (-14.89%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Here looks like have lots of memory waste.

I load a simplified model which has almost 50k vertices. And there is an OOM.
Here is the key code. It limits kaolin to rendering large models.
https://github.com/NVIDIAGameWorks/kaolin/blob/bcce7a95506847b3ced48fcfc3207f61488dcb42/kaolin/ops/mesh/mesh.py#L116
Is there any plan for improvement?

Hi @FavorMylikes , Thank you for your interest in Kaolin and good catch!
We might be able to optimize this by using torch.diagonal. I will try to test it on my side but do give it a try for your own use case! :)


Transfer - Attempt to transfer ".jfrog" directory in Docker repositories

[x] All tests have passed. If this feature is not already covered by the tests, new tests have been added.
[x] The pull request is targeting the dev branch.
[x] The code has been validated to compile successfully by running go vet ./....
[x] The code has been formatted properly using go fmt ./....

Test https://github.com/jfrog/jfrog-cli-core/pull/1088

Notice that the tests are failing on this PR.

Fix for the transfer tests - https://github.com/jfrog/jfrog-client-go/pull/877


Thread.new() returns null, start() not working

Godot version
4.0-dev
System information
Arch Linux
Issue description
I got this error when I try to start a thread after a Thread.new()
E 0:00:00:0840   is_type_compatible: Parser bug (please report): Trying to check compatibility of unset value type
<Erreur C++>   Condition "!p_source.is_set()" is true. Returning: true
   modules/gdscript/gdscript_analyzer.cpp:3459 @ is_type_compatible()
Steps to reproduce
Declare a new Thread with Thread.new(), launch a start()
Minimal reproduction project
No response

Duplicate of https://github.com/godotengine/godot/issues/44375.


Float weights in Kaplan-Meier

There is some unexpected behavior when fitting a weighted KM with weights that are floats. Here is a quick toy example
import pandas as pd
import lifelines
df = pd.DataFrame()
df['t'] = [1,2,3,4,5]
df['d'] = [1,1,1,1,1]
df['w1'] = [2,1,1,1,1]
df['w2'] = [2.1,0.9,0.9,0.9,0.9]
When fitting integer weights (w1), everything behaves as expected. However, when floats are included instead (w2) some unexpected behavior occurs. Running the following lines:
y = lifelines.KaplanMeierFitter().fit(df['t'],weights=df['w2'])
print(y.survival_function_)
Produces
          KM_estimate
timeline
0.0             1.000
1.0             0.375
2.0             0.375
3.0             0.375
4.0             0.375
5.0             1.000
But what is expected is:
          KM_estimate
timeline
0.0             1.000
1.0             0.632
2.0             0.474
3.0             0.316
4.0             0.158
5.0             0.000
I haven't figured out exactly where everything goes wrong, but it looks like it is due to y.event_table converting all of the counts to int32 rather than float32. Most likely something in _preprocess_inputs is rounding everything to integers for the event_table before the KM calculation. This would explain why the survival time goes to 1 at the last time point that is less than 1.

Sorry I missed this - looks pretty important.

No worries. I meant to look deeper into it and find where it goes wrong, but have been pretty bogged down lately

Alright, I think I found where it goes wrong.  In lifelines/utils/init the function survival_table_from_events() returns a pandas dataframe with all items converted to integers. It converts all the items to integers when the object is returned. Removing .astype(int) generates event_tables that have the correct integers when I tested the function locally. I don't know if removing this would have any downstream issues. I would guess integers was a stylistic choice?
Options are; (1) remove .astype(int) from a single line. Disadvantage is that event_tables won't look as clean and crisp (everything will have trailing zeroes). (2) add logic so that if weights are all integers then it is returned as integers. Otherwise, it leaves everything as a float

I think I prefer 2). I'd also like to add tests to check this as well

See #432


Verifying all ATXs before starting node sync takes a lot of time and CPU

Description
Before Node starts syncing layers it downloads and verifies all activations (ATXs).
It causes me to spend about 80 minutes syncing ATXs from scratch on devnet-300 and consumes about 100% of the CPU within this process. So this is a bug and we can significantly speed up this process.
My most significant concern is that this process will take much longer on the main net if someone wants to join the network after some time from the beginning.
I did some calculations, but it might be wrong. Fix me then :)
Network stats at the moment:

10 000 layers x 120 seconds per layer
it lived for ~13.8 days or about 20 000 minutes
60 layers per epoch = epoch №166
about 75 miners,
up to ~12500 atxs in total for that period
but most likely less, since not all miners joined right from the first epoch

What it takes

It took about 80 minutes to sync up only ATXs from scratch on my machine (M1),
~3.84 seconds per ATXs on average
some community members wrote in Discord that it takes 20 minutes and 40 minutes, so I believe it also depends on the machine and also the last synced epoch

Some extrapolation

if we'll have 400 layers per epoch in the main net,
then for the same period of the running network (~13.8 days)
there will be 25 epochs
let's say that we'll have 1000 miners instead of 75 in the main net,
then there will be about 25000 ATXs within the same period of time

I assume that the required time for syncing has a pretty linear relation
to the amount of ATXs, since we need to take each one after another.

In this case, it will take about 3 hours to sync up ATXs for the network
that runs for just a couple of weeks.

The time of syncing layers is not included in the calculations at all.
So expect that it will take some more time.
Steps to reproduce

Just start syncing the node from scratch against devnet-300

Actual Behavior
It consumes a lot of CPU (~100%) and takes a lot of time to sync up ATXs.
Expected Behavior
It syncs up much faster and does not consume so much CPU.
Environment
Please complete the following information:

OS: macOS 11.6 Big Sur, MacBook Pro (13-inch, M1, 2020). 16GB RAM
Node Version: v0.2.18-beta.0

Out of scope
Since it takes some time to sync up ATXs probably we also need to add the status (like the last synced epoch) in some GRPC API endpoint, like NodeStatus and NodeStatusStream.

i think the plan for mainnet is to have 2000 per epoch, 2 weeks/epoch basically. but number of miners is hopefully will be around 1000 - 10000.
regarding cpu, it is expected that consumption will be high during sync. but i haven't looked at sync profiles. by 100% you mean that it utilizes fully 1 core, and not all of them, right?

by 100% you mean that it utilizes fully 1 core, and not all of them, right?
Yes, right, only one core.

For 2k layers per epoch and having 1k miners it seems much better (about 5.8 days of continuous work) to sync up ATXs for the one-year age network. But, in my humble opinion, it still doesn't sound good.
Mb we can optimize it somehow after the genesis.
But for the testnet (if we won't use the same config) it still sounds unacceptable.

For 2k layers per epoch and having 1k miners it seems much better (about 5.8 days of continuous work) to sync up ATXs for the one-year age network. But, in my humble opinion, it still doesn't sound good, but it is acceptable to have it for genesis. And then hopefully we can optimize it somehow.

definitely needs to be profiled. in a year there will be 26 epochs, 1 000 atxs in each. 26 000 atxs in total. my guess would be that it should be doable to download them in a couple of hours at most.

So we'll have something about 10 minutes per layer? Then we can live with it :)
Meanwhile, we'll have to mimic these properties in the testnet.
I mean that with the current testnet parameters it will be very painful to join testnet late

What @dshulyak said 👍🏻
Regardless of layer length, epochs will be 2 weeks so the number of epochs per year is indeed 26. We can't predict the number of smeshers / ATXs, but 2k-5k in the first year sounds reasonable to me.

It doesn't make sense to me that it takes so much time, that code path definitely has to be profiled

Btw, sometimes CPU usage jumps up to 3.5-4 / 8 cores.
Process sample on my M1 if it might help:
sample-0.2.18-atxs.txt
And the log is full of such messages:
2022-11-21T13:56:07.038+0200	INFO	6a1f7.atxHandler   	validating nipost	{"node_id": "6a1f7a84dfa6d53428aa48a0a2a3ae62574743911b15fa479f285afd8e5cdd85", "module": "atxHandler", "sessionId": "1bdd20a1-a160-4a18-9bed-90aac448cfa9", "expected_challenge_hash": "0x346f9a0ab496680ba073a9a1cefb8bc68ff112383ada903e47d0ec412620b9f9", "atx_id": "084a71cab9c05cb33ea59eaec1dbe2361a8302d934e66e5800f0d7f2cc07789f", "name": "atxHandler"}
2022-11-21T13:56:07.322+0200	INFO	6a1f7.atxHandler   	processing atx	{"node_id": "6a1f7a84dfa6d53428aa48a0a2a3ae62574743911b15fa479f285afd8e5cdd85", "module": "atxHandler", "sessionId": "1bdd20a1-a160-4a18-9bed-90aac448cfa9", "atx_id": "084a71cab9c05cb33ea59eaec1dbe2361a8302d934e66e5800f0d7f2cc07789f", "epoch_id": 232, "atx_node_id": "a8a016fbc6c968ca48bac97336882efeac85cee12a7563cfc8e04405b1410993", "layer_id": 13920, "name": "atxHandler"}
2022-11-21T13:56:07.323+0200	INFO	6a1f7.atxHandler   	atx is valid	{"node_id": "6a1f7a84dfa6d53428aa48a0a2a3ae62574743911b15fa479f285afd8e5cdd85", "module": "atxHandler", "sessionId": "1bdd20a1-a160-4a18-9bed-90aac448cfa9", "atx_id": "084a71cab9c05cb33ea59eaec1dbe2361a8302d934e66e5800f0d7f2cc07789f", "name": "atxHandler"}
2022-11-21T13:56:07.323+0200	INFO	6a1f7.atxHandler   	finished storing atx in epoch	{"node_id": "6a1f7a84dfa6d53428aa48a0a2a3ae62574743911b15fa479f285afd8e5cdd85", "module": "atxHandler", "sessionId": "1bdd20a1-a160-4a18-9bed-90aac448cfa9", "atx_id": "084a71cab9c05cb33ea59eaec1dbe2361a8302d934e66e5800f0d7f2cc07789f", "epoch_id": 232, "name": "atxHandler"}
2022-11-21T13:56:07.323+0200	INFO	6a1f7.atxHandler   	got new atx	{"node_id": "6a1f7a84dfa6d53428aa48a0a2a3ae62574743911b15fa479f285afd8e5cdd85", "module": "atxHandler", "sessionId": "1bdd20a1-a160-4a18-9bed-90aac448cfa9", "atx_id": "084a71cab9c05cb33ea59eaec1dbe2361a8302d934e66e5800f0d7f2cc07789f", "challenge": "0x346f9a0ab496680ba073a9a1cefb8bc68ff112383ada903e47d0ec412620b9f9", "id": "084a71cab9", "sender_id": "a8a016fbc6c968ca48bac97336882efeac85cee12a7563cfc8e04405b1410993", "prev_atx_id": "3127e190da", "pos_atx_id": "3127e190da", "coinbase": "stest1qqqqqqztuw7tq633g22a7cnfwclsxlvg54slhxgyq6a6k", "pub_layer_id": 13920, "epoch": 232, "num_units": 4, "sequence_number": 7, "size": 3164, "name": "atxHandler"}

likely caused by https://github.com/spacemeshos/go-spacemesh/issues/3770

It should be verified on the version with that bug fixed.

I can't reproduce it anymore. So seems the issue is solved 👍


Smart Outlet not responding.

I'm having an issue with an wall outlet that I installed. I connected it to the iphone tuya app and was successful at getting the ID and Key. From that point I added all the required information into my config.json file.
The brand is CE Smart Home - Wifi Smart Outlet 2 Pack.
https://www.costco.ca/CE-Smart-Home-Wi-Fi-Receptacle,-2-pack.product.100475520.html
Here are my logs.
Apr 02 16:19:59 wwhub homebridge[1974]: [2019-4-2 16:19:59] [TuyaLan] Discovered Coffee<PHONE_NUMBER>0194c1ba37)
Apr 02 16:19:59 wwhub homebridge[1974]: [2019-4-2 16:19:59] [TuyaLan] Connected to Coffee
Apr 02 16:19:59 wwhub homebridge[1974]: [2019-4-2 16:19:59] [TuyaLan] Ready to handle Coffee with signature {"101":false,"102":0}
now i'm a little confused as to why it is using 101 and 102. only the bottom receptacle is controlled and the top one is a dumb always on receptacle.
here are the logs when I toggle it on and off from Home app.
Apr 02 11:58:21 wwhub homebridge[1974]: [TuyaAccessory] Sending Coffee {"1":true}
Apr 02 11:59:49 wwhub homebridge[1974]: [TuyaAccessory] Sending Coffee {"1":true}
Apr 02 11:59:51 wwhub homebridge[1974]: [TuyaAccessory] Sending Coffee {"1":false}
Apr 02 11:59:53 wwhub homebridge[1974]: [TuyaAccessory] Sending Coffee {"1":true}
Let me know if I can get this going.
Thank you.

101 is certainly not what I have seen around. Let's try something.

update the plugin to the latest unreleased code with npm i -g AMoo-Miki/homebridge-tuya-lan
change the device type to CustomMultiOutlet and match this:

{
  "name": "Coffee",
  "type": "CustomMultiOutlet",
  "manufacturer": "CE Smart",
  "model": "LQ-2-W3",
  "id": "xxxxxxx",
  "key": "xxxxxxxx",
  "outlets": [
    {
      "name": "Bottom",
      "dp": 101
    }
  ]
}

restart Homebridge.

Lemme know how this works.

Before posting a new issue, thought I'd comment my discoveries here. I'm receiving a very similar message. What I've found is that if trigger the device on/off in one of the apps (CE Smart or Smart Life), then it will start to work with Homebridge. The status get's updated immediately and you can use the switch.
Here's my output:
[2019-7-13 17:09:19] [TuyaLan] Ready to handle Bedside Fan with signature {"1":false,"11":0}
[2019-7-13 17:09:21] [TuyaLan] Ready to handle Milo with signature {"1":false,"11":0}
[TuyaAccessory] Sending Milo {"1":false}
[TuyaAccessory] Sending Milo {"1":true}

@bugs181, the original poster had a device that didn't use the standard(?) power datapoint identifier. However, reading your message, it sounds like your device doesn't work right off the bat. If that is the case please create a new issue so we can work together and find a solution.


Change the default theme to starry light

The current default was made rather quickly and has not aged super gracefully. Aestrix, in addition to several others, would like the default layout to change, presumably to starry light.

See #591 for progress on splitting starries up as necessary and removing default from being so tightly coupled with the site's CSS.

I think #1059 is the latest on this?


breakId fix & custom

VrpXMLWriter is a fix, currently it write 'serviceId' instead of 'breakId'.
Others are in order to define its own id to each break, with the idea to handle with multiple breaks for each vehicle.

Thanks a lot, @braktar 👍


[FEATURE] A script to vacuum expired data

📝 Feature Description
We need to develop and implement a vacuuming script to automatically delete expired card data from the database, enhancing our data security measures. This script will run as a cron job at regular intervals (e.g., every 24 hours) to maintain database hygiene and reclaim disk space.
The script should perform the following tasks:

Identify expired cards in the database using the ttl field.
Delete expired entries directly from the database.
Initiate a database vacuuming job after deletion to reclaim disk space.

🔨 Possible Implementation

Create a new script file for the vacuuming process.
Implement database connection and query functionality to identify expired cards.
Develop a secure method to delete expired entries from the database.
Integrate a mechanism to initiate database vacuuming after deletion.
Set up error handling and logging throughout the script.
Create a cron job configuration to run the script at specified intervals.
Implement performance optimizations to handle large datasets efficiently.
Add monitoring and alerting for script execution and any potential issues.

🔖 Note:
Technical Considerations:

The script must operate securely without exposing any sensitive data during the process.
Optimize the script for performance to handle large datasets efficiently, without blocking or hindering the performance of the locker.
Implement proper error handling and logging mechanisms to track the script's operation and any issues that arise.
Consider implementing a dry-run mode for testing purposes.
Ensure the script is compatible with the existing database schema and any future planned changes.

Submission Process:

Ask the maintainers for assignment of the issue, you can request for assignment by commenting on the issue itself.
Once assigned, submit a pull request (PR).
Maintainers will review and provide feedback, if any.
Maintainers can unassign issues due to inactivity, read more here.

Refer here for Terms and conditions for the contest.

Hi @Chethan-rao I want to work on it, can you please assign it to me.

Sure @Vikas350 Assigning this to you, Happy coding!

Hey @Vikas350 Un-assigning the issue, as we haven't heard back from you, and these is another issue assigned to you as well.


Add badges to README

This adds 2 badges to the README.md file:

npm version
npm install size

Thanks!


Battery format-alt no longer works

Hi there, I had been using the format-alt config directive on the battery model to toggle between short and more detailed battery info states. It seems in the latest waybar version that functionality stopped working. Thanks for your help!

I'm not sure what happened, but format-alt is working again! Closing this, sorry for the noise


Update BUG_REPORT.yml

Thank you for this suggestion, John. I'll let @davkutalek also take a look.

Thank you.I felt that this would necesitate us to categorize the bugs that should be fixed immediately and the ones that can be worked on within the weekly sprints


Failing to connect with shs errors when receiving new clients

I opened Manyverse which tried to connect only to hermies.club, and Manyverse got these logs
2021-04-26T11:38:48.925Z ssb:conn-hub failed to connect to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ= because: shs.client: error when expecting
server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

2021-04-26T11:38:56.031Z ssb:conn-hub connecting to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ=

2021-04-26T11:39:01.136Z ssb:conn-hub failed to connect to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ= because: shs.client: error when expecting
server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

2021-04-26T11:39:08.248Z ssb:conn-hub connecting to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ=

2021-04-26T11:39:13.349Z ssb:conn-hub failed to connect to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ= because: shs.client: error when expecting
server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

2021-04-26T11:39:20.164Z ssb:conn-hub connecting to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ=

2021-04-26T11:39:25.471Z ssb:conn-hub failed to connect to net:hermies.club:8008~shs:
uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ= because: shs.client: error when expecting
server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

I then restarted the server with systemctl and then the connection in Manyverse succeeded as expected.
Ideally, we wouldn't need a server restart and we clients wouldn't see the shs error upon connecting.

It also happens with other room attendants:
[ 04-26 17:33:23.505  2802: 2890 E/NODEJS-MOBILE ]
2021-04-26T14:33:23.505Z ssb:room-client we wish to connect to { name: 'tunnel', portal:
<EMAIL_ADDRESS>target:
<EMAIL_ADDRESS>}
2021-04-26T14:33:23.505Z ssb:room-client will call tunnel.connect at
@VIOn+8a/vaQvv/Ew3+KriCngyUXHxHbjXkj4GafBAY0=.ed25519 via room
@uMYDVPuEKftL4SzpRGVyQxLdyPkOiX7njit7+qT/7IQ=.ed25519

[ 04-26 17:33:28.509  2802: 2890 E/NODEJS-MOBILE ]
2021-04-26T14:33:28.509Z ssb:room-client tunnel duplex broken with { name: 'tunnel',
portal<EMAIL_ADDRESS>target:
<EMAIL_ADDRESS>} because
shs.client: error when expecting server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

The JS error masked by shs.client: error ... is:
Error: pull-reader: read exceeded timeout
    at Timeout._onTimeout (/data/data/se.manyver/files/nodejs-project/index.js:37667:10)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7)

This may turn out to not be about go-ssb-room! In the past 30min I've been connected to only JS rooms, and I detected a few (although it happened more often with go-ssb-room):
Error: pull-reader: read exceeded timeout 5000
    at Timeout._onTimeout (/data/data/se.manyver/files/nodejs-project/index.js:37667:10)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7)

2021-04-26T15:49:43.953Z ssb:room-client tunnel duplex broken with { name: 'tunnel', portal<EMAIL_ADDRESS>target<EMAIL_ADDRESS>} because shs.client: error when expecting server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

2021-04-26T15:49:43.955Z ssb:conn-hub failed to connect to tunnel:@W18NyiHagiiVE4/jm2sxzqQqIwmUw2p9pKZ0lHH9OXA=.ed25519:@THUzexG1y6kWofwiN8Lix/jNH/P6roYdlCDgpAn2HSc=.ed25519~shs:THUzexG1y6kWofwiN8Lix/jNH/P6roYdlCDgpAn2HSc= because: shs.client: error when expecting server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

shs.client: error when expecting server to accept challenge (phase 1).
possibly the server is busy, does not speak shs or uses a different application cap

no stream for incoming msg {
  req: -11,
  stream: true,
  end: true,
  value: {
    message: 'shs.client: error when expecting server to accept challenge (phase 1).\n' +
      'possibly the server is busy, does not speak shs or uses a different application cap',
    name: 'Error',
    stack: 'Error: shs.client: error when expecting server to accept challenge (phase 1).\n' +
      'possibly the server is busy, does not speak shs or uses a different application cap\n' +
      '    at abort (/data/data/se.manyver/files/nodejs-project/index.js:19583:45)\n' +
      '    at Object.cb (/data/data/se.manyver/files/nodejs-project/index.js:19592:24)\n' +
      '    at drain (/data/data/se.manyver/files/nodejs-project/index.js:37697:23)\n' +
      '    at /data/data/se.manyver/files/nodejs-project/index.js:37714:18\n' +
      '    at source (/data/data/se.manyver/files/nodejs-project/index.js:38359:9)\n' +
      '    at Timeout._onTimeout (/data/data/se.manyver/files/nodejs-project/index.js:37667:7)\n' +
      '  Error: pull-reader: read exceeded timeout 5000\n' +
      '    at Timeout._onTimeout (/data/data/se.manyver/files/nodejs-project/index.js:37667:10)\n' +
      '    at listOnTimeout (internal/timers.js:554:17)\n' +
      '    at processTimers (internal/timers.js:497:7)'
  },
  length: 1066,
  type: 2

One hypothesis, although I wouldn't assign too much probability to it: this could be one of those deweird-territory bugs, because deweird bugs occurred when an implementation was fast (in the case of ssb-db2 it was the raw log stream that was the fastest one). That would explain why this issue occurs more frequently with go-ssb-room than with JS room, because Go is faster. But I don't see exactly where this would happen, because it would have to be a source muxrpc somewhere between the tunnels. (Or maybe duplex muxrpcs also suffer from deweird bugs)
But this hypothesis explains only tunnel connection bugs between attendants, doesn't explain the bug where the client cannot connect to the go-ssb-room.

Another hypothesis, a pretty good one I think: I updated ssb-room-client to 0.13.0 on my own phone, but the other attendants in hermies.club haven't yet updated theirs, so it's not like the bug #190 is entirely gone, it may look different from my side.
This doesn't explain though, the first case where ssb-room-client couldn't connect to go-ssb-room, and after a server restart, it could.

Downgrading this from this milestone since it's unclear whether (1) is it critical or not, given that most of the time we don't experience this issue, (2) is it specific to go-ssb-room or is it general among ssb-js peers.

I can't reproduce this anymore in 21 days, let's call it done. :tada: :taco:


Add the option to have persistent switches

Add new GPIO switch class - PersistentRPiGPIOSwitch which will restore the switch state after Home Assistant restart.
As part of the change move the core switch to use async functions
This is based on my work in the original branch: https://github.com/thecode/ha-rpi_gpio/pull/266

@jdeneef kind reminder. Thanks.

will take a look, hardly any time during weekdays, sry

Hi @jdeneef , thanks for taking a look.
What did you do with this PR? It seems deleted.

sry, messed up using github, more used to local git. Anyways, your request is merged in 1.4.1, see prereleases. I updated your code to avoid flickers at startup, and had to remove some reading of gpio switches before setting from persistence, cause this is more consistent. It's working for me in prod now, please test the prerelease.

I tested it in my setup and it looks great as well. I would remove all the commented lines to clean it up :). I think the changes you made make it looks better. I mainly wanted to minimize changes but you took it further.

nice! I will cleanup in a next drop and make it a github release if no errors pop up. Will test for a few days and than move to release state.