Multiple keyboards

I've been using this for a while now and it works fantastically.  There's just one shortcoming I've noticed that is probably only an issue because of some of the ways I use it.
I use language learning apps on my phone.  I'm studying several languages including Russian.  To this end, I have two keyboards connected to my desktop, one configured for US English with some keymap changes that allow me to also use it for Norsk and Islenska, and the other configured with a Russian keymap.
Now, this works fine in typical desktop applications, web pages, etc.  But when I type in a language app on the phone, via scrcpy, on the Russian keyboard, nothing happens.  It seems scrcpy is not aware of the second keyboard.
Does this seem like something that could be readily fixed?

It seems scrcpy is not aware of the second keyboard.

If you type Russian (non-ASCII) characters, it is "expected" not to work: see https://github.com/Genymobile/scrcpy/issues/37 (even without multiple keyboards)

Aaaaah.  Oh, well....

Closing as duplicate of #37.

Please test #1426 (works on Android >= 7).


[Bug]: Use timestamp format hh not h for SRT file

The SRT export should use hh:mm:ss,mmm, but the provided format uses h:mm:ss,mmm

@ullenboom
Thanks for report! fixed in latest version and re-uploaded files to releases.


Hp serialization adjustement

The main goal of these changes is to let high performance serializers like protoStuff or google protocol buffers, serialize the SimpleRobotRules structure in order to cache it.

Thanks Josep Miquel~

cool, thanks!


fix multi-arch image tags

@cosmo0920 Looks like my last PR didn't set the image tag properly on the multi-arch manifests, this one should fix it.

It is very nice change.
I'd confirmed that this change publishes multiarch images on our DockerHub:
https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset/tags?page=1&ordering=last_updated
Thanks for your work! :+1:


Using vscode node debugger is not working anymore for Protractor

- VSCode Version: 1.14.2
- OS Version: Windows 10 Pro, 64 bit
- Node Version: v.7.9.0
- Protractor Version: 5.1.2 (Javascript test framework - http://www.protractortest.org/#/)
Launch.json
{
// Use IntelliSense to learn about possible Node.js debug attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"cwd": "${workspaceRoot}",
"program": "C:\Users\\AppData\Roaming\npm\protractor",
"args": ["${workspaceRoot}\conf.js"],
"outFiles": ["${workspaceRoot}/**/*.js"],
"sourceMaps": true,
"protocol": "inspector",
"trace": true
}
]
}
Alternatively, I've tried the following too:
{
// Use IntelliSense to learn about possible Node.js debug attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"cwd": "${workspaceRoot}",
"program": "${workspaceRoot}\node_modules\.bin\protractor",
"args": ["${workspaceRoot}\conf.js"],
"outFiles": ["${workspaceRoot}/**/*.js"],
"sourceMaps": true,
"protocol": "inspector",
"trace": true
}
]
}
- Steps to reproduce the bug
Setup the launch.json file and run the debugger.
VS code shows the error message: "Cannot launch program 'c:\Repos\leo.tests\node_modules.bin\protractor'; setting the 'outDir or outFiles' attribute might help."

In the trace logs, I keep getting the message. I would love to use the vscode debugger as it's handy as was working! It looks like it's looking for a sourcemapURL which isn't available.
vscode-node-debug2.txt
Below is the message to the client from the log file:
To client: {"seq":0,"type":"response","request_seq":2,"command":"launch","success":false,"message":"Cannot launch program '{path}'; setting the 'outDir or outFiles' attribute might help.","body":{"error":{"id":2003,"format":"Cannot launch program '{path}'; setting the 'outDir or outFiles' attribute might help.","variables":{"path":"c:\\Repos\\leo.tests\\node_modules\\.bin\\protractor"}}}}
From client: disconnect({"restart":false})
Terminated: Got disconnect request
To client: {"seq":0,"type":"event","event":"output","body":{"category":"telemetry","output":"debugStopped","data":{"reason":"Got disconnect request"}}}
To client: {"seq":0,"type":"response","request_seq":3,"command":"disconnect","success":true}

The file in .bin at least is a shell script:

You can reference protractor.cmd instead as your runtimeExecutable, or you can reference the actual .js file under node_modules/protractor as your program.


MINIFI-170 Change FlowStatusReport to be JSON

The FlowStatus report were in a format that is close to but not JSON.
FlowStatusReport{controllerServiceStatusList=null, processorStatusList=[{name='GenerateFlowFile', processorHealth={runStatus='Running', hasBulletins=false, validationErrorList=[]}, processorStats={activeThreads=0, flowfilesReceived=0, bytesRead=0, bytesWritten=1024000, flowfilesSent=0, invocations=100, processingNanos=106080516}, bulletinList=[]}], connectionStatusList=null, remoteProcessGroupStatusList=null, instanceStatus=null, systemDiagnosticsStatus=null, reportingTaskStatusList=null, errorsGeneratingReport=[]}
This makes it hard to process. The changes in this P/R result in a FlowStatus report that is valid JSON, thus enabling standard tools to be used to process the reports.

reviewing

Updates look good.  Thanks for patching those up.  Will merge this in.


[NEMO-61] Fix lost execution metric collection problem

JIRA: NEMO-61: Fix lost execution metric collection problem
Major changes:

Add metric flush request and response types in control message type.
RuntimeMaster#terminate now wait until all metrics from executors arrive.

Minor changes to note:

Expose ExecutorReqistry#viewExecutors to public.

Tests for the changes:

N/A

Other comments:

Related to NEMO-20.

resolves NEMO-61

Let's think about how to add any tests to prevent regression. :thinking:

Thanks for the work @skystar-p! I've done a quick pass for the changes.
In my humble opinion, MetricManagerMaster should be responsible for sending flush requests to MetricManagerWorkers. Then, for writing tests we only have to inject MetricManagerMaster and MetricManagerWorkers, rather than RuntimeMaster. (which is quite huge!)

@seojangho Thanks for the good idea! I will fix like you suggested.

and also please update your branch 😄


Unstable device location and slow updates

I'm using a Samsung S20+ (Android) phone with Home Assistant providing a beacon so my phone can be tracked.
In the BLE Transmitter settings I've put my advertise mode at 10Hz and my power at Ultra Low (though I've also tried at high with the same results).
I've flashed 4 ESP32s with the ESPresense firmware and placed them in my office, one at either end of a u shaped hallway, and one in the room adjacent to the office.
I've made no changes to the default configs beyond entering my co-ordinates for the map and telling it where my nodes are located.
I've measure my  RSSI@1M using the ESP32 and holding my phone 1M away from it and set it the settings area.
Despite that my device keeps getting lost (I think the beacon just stops reporting since the LastHit is 2 minutes ago and growing), and when it is found it jumps all over the place.  When I walk around it tends to take about 60 seconds before it updates where I am (which even at 1Hz should still update within a second of me walking around)
I feel like I'm missing something obvious here, but I'm not sure what it is 🤦♂

I've found the ha screen to edit those sensor settings is buggy, sometimes I need to go in and flip the settings and eventually it "takes".  Use another phone and confirm you're actually at 10hz.  Also if your missing pings it might be Ultra low is too low.  I use lowLatency and medium power.  Also make sure you've disabled maxDistance.  If you're rssi@1m is off you might be hitting the 16m max.

I'll give the nRF Connect a test, thanks.
On the maxDistance, I haven't put anything in, should I put in 0 or should I put in a hard limit on the filter to about my room size to prevent excessive overlapping when I'm not testing it?

  I'm not sure it this is an android thing or a Samsung thing or what, but it's that way on my phone and my samsung tablet as well, I'm trying to see why my bluetooth radio strength is so weak.

Ultra Low and -81 tracks.  Try medium.

Okay I tried to update to medium, and low Latency (I also tried to use Beacon Scope to see if that would work better than home assistant).  I made sure that they have no restrictions on power usages and can run in the background.  Looking at the devices information on Home Assistant I will see it's x,y,z coordinates and can see it should be in a room, but it will say n/a for the room/floor.  Then it will drop to unavailable for a minute and come back, but I haven't moved my phone and can confirm in the nRF Connect that it is sending out a signal.
I tried to bump it up to High power but I didn't see any change.  And I also went through and redid all my measurements making sure I was as close as I could get for room positions.  I'm fairly confident that my ESP32 are within a half foot of reality.
I have 6 of them spread across my upstairs, one at each corner of the upstairs and one on either end of a hallway.
It's hard to tell, but I noticed when I was listening to the espresense/rooms/# topic I was getting a lot of telemetry messages with an uptime around "3800".  Is this indicating that my devices are resetting frequently?

Uptime of 1 hour isn't good.  I am working through crashes in the betas (especially on b0).  Some super cheap esp32s do have some issues w/ uptime (honestly we're pushing the rf circuit so hard even an occasional m5 esp32 has issues).
Put this on your home assistant dashboard
type: custom:auto-entities
card:
  type: history-graph
  hours_to_show: 8
filter:
  include:
    - entity_id: sensor.espresense_*_uptime
sort:
  method: friendly_name

Share your graph with me if you can.  I just looked and v3.3.2.b1 is actually looking quite good:

I actually updated to v3.3.2b1 to try and get it stabilized and my uptime is around 140,000 which tracks for when I ran  the update on Wednesday evening.
But I'm still getting flaky/slow location.  If I drop the timeout or interval settings in the config will that improve how quickly it detects me moving about?
For the flaky location, could it be because I'm having a lot of error in my calibration settings?

I've used a laser tape measure to map out my house and made sure to include things like gaps for wall thickness, so I'm pretty confident I have things pretty spot on.  I mean I've got a lot of not straight angles that I had to do some math to figure out distances so I might have a foot of error by the end of it, but 6m of error seems a bit much.
Do I need to bring down all the ESPs and bring things up 1 at a time, dial in the calibration and then bring up the next one?

FWIW:
I know I am using older ESPresense firmware, but I thought I would share these two charts... I think they are confirming that different esp hardware behaves differently, despite all be esp32 D1 Mini, all from the same supplier AZ-Delivery, but purchased at different times.
All are esp32 D1 Mini, all running espresense v3.3.1, all with the same settings (except name).
approx half of them crash/reboot after about 1 hour, whilst the others have been running now for more than 8 days.

Yeah, the brandless stuff on amazon stinks.  But it's so much easier to prime to your house.  I feel like we should have some kind of warning on https://github.com/ESPresense/ESPresense.github.io/blob/master/base-stations.md


Instructions to install

Following the installation instructions does not actually install the plugin into Kibana. There must be something that needs to be done after cloning the repo.

Just needed to restart Kibana.


Cluster driver logo is not showing properly for non-default cloud providers

What kind of request is this (question/bug/enhancement/feature request):
UI bug
Steps to reproduce (least amount of steps as possible):

go the Global -> Tools -> Cluster Drivers and enable some cloud providers like Alibaba ACK
Add cluster and the driver's UI is not showing properly

@lvuch Not sure if it is the right solution. Cloud you have a look, please? Thanks!

Version: master-head (v2.3) (8/16/19)
@loganhz the icons look better now in current master-head. But the Baidu icon is touching the edge of the box to left. If you enlarge the image and notice the leftmost blue part of the Baidu logo.

Version: master-head (v2.3) (08/21/2019)
All the logos are now properly centered and displayed in the browser.


Boot strap issue..

where to put these ==> @navber-height @navbar-padding-vertical .Please can show that to me in example?
Those parameters are mentioned here.
http://stackoverflow.com/questions/19890700/change-navbar-height-in-bootstrap3
I am using bootstrap 3.0.0

Closing as super out of date and not something we can help with here (odd we somehow overlooked this). Searching Google for how to use Less would be your best bet :).

See https://github.com/twbs/bootstrap/blob/master/README.md#community for other community forums you could ask for help with this at.


A way to arrange cam windows

I love your pluggin. Its very useful when Im away from printers. Its piece of mind. Ive cancelled prints remotely that failed or lost adhesion. But also to know a print so doing fine. Great to show co-workers.
My request is for a way to choose and view multiple printers. Because not all printers are always printing however.
So currently if printer A and C are printing, B and D is shown idle. Tomorrow D and B maybe printing but A and C are showing @ https://www.getanywhere.io/app/devices, with idle screens. There is no way to size the web screens smaller. So currently I can only view 2 screens without scrolling. In the four printer scenario, A's printing, B idle, C's printing, D idle. But the standard 800 x 600 or even 1920 x 1080 web browser page will show 2 printers max without scrolling, and 2 could be idle:
Web page:
A printing B idle(waste)
must scroll
C printing D idle
wish i could choose active printers:
A printing C printing
B idle D idle
or 3 printers printing
Choose active printers - scale smaller to view A, C, D printing same page.

Suggestion 1st page -
Combine all printer cams single scaled window, choose which to zoom/expand. (similar to security office, where all cams can be viewed at once on 1 page)
then
Need a way to group active printers, let user decide which printers are grouped. (so idle printer doesn't waste screen)
At home on local network, pi3 are connected by ethernet on a switch, not using the on board wifi of pi3 is more reliable for me, and I can stream locally @ 1024 x 720 25fps.
camera_raspi_options="-fps 25 -x 1280 -y 720 -ex auto -awb fluorescent -b 2000000 -ISO 200 -vf -hf"
Prusa Mk3
Prusa MK2s
Taz 6
CR10s4
3ku Delta

Thank you @mylife4aiurr for a very constructive suggestion! Even though it sounds pretty difficult to implement, it definitely sheds lights on how I can further improve the user interface. I will think about it and come back with some options.
I don't fully understand your last part:

At home on local network, pi3 are connected by ethernet on a switch, not using the on board wifi of pi3 is more reliable for me, and I can stream locally @ 1024 x 720 25fps.
camera_raspi_options="-fps 25 -x 1280 -y 720 -ex auto -awb fluorescent -b 2000000 -ISO 200 -vf -hf"

Is it something that is causing a different problem, or just the same problem but described differently?

i was just showing you the screen size i use with Octoprint. I think its why I only see two printers then have to scroll.

Great thanks!

I see you are now charging users for more then 1 instance. OK. I would consider paying but I still need this issue addressed. Have 5 printers.
I would need at a minimum to have actively printing printers presented 1st to user. So I dont have to scroll to find active printers. With printers doing nothing in between the ones printing.
Wish list:
Main screen with thumbnail sized windows of all printers. Click one that I'd like to view.
I way to order the printers being viewed. Maybe I have 3 printers going. But I'd like to see the mk3 1st, taz6 2nd, delta 3rd. etc

Yes sorting the card is on my roadmap. I can definitely pull it in. The default sorting will be by status (active -> idle -> offline). You can also choose to sort by name. Does it sound like what you want?
By "clicking one to view", do you mean to view webcam feed in full screen? Or you mean screen with only 1 printer?

The sorting option is great as a minimum option.
This suggestion/option is for going beyond the minimum.... My wishlist.
If User has 3 or more printers.
Main screen is clickable icon/thumbnail view of all printers.
If I want to full screen a printer, I can click on it's smaller pic/icon to get a full screen view (the screen u already have implemented) then I could stop print, raise head, etc - your current functionality.

@mylife4aiurr I just released a new version that includes sorting.
Is the reason why you want the thumbnail view because you want to see webcam feed from all printers at a glance? If so, I'm wondering if the thumbnail images will be too tiny for you to get the info you'd like to get anyway (such as if a printer is failing). Let me know if I misunderstood you.

Yes correct, my use case for multiple printer view is while away using a cell phone. If Octoprint doesn't support a good way to scale each small window for a particular phone or tablet screen then don't bother.
Thanks for implementing a sort function...

I'm not a big fan of "click to see detail / back button to thumbnail view" user interaction. I have been in many mobile-oriented projects and it's clear to me that card-based UI that requires mostly swipes  opposed to clicks is just much better user experience. I doubt that you would like that user experience even if I implemented the thumbnail/detail UI.
At some point I may implement the feature to let users toggle between thumbnail and card view, if I'm convinced that the webcam feeds on thumbnail view won't be too cramped to be useful.
Again I appreciate your constructive suggestions. I'll leave this issue open for now in case you want to provide additional info.


Unrated Users have Rating 0 for Performance Calculations

I do not know about live prediction calculations but at least for performance calculations after the contest is done, unrated users count as having rating 0 instead of the default rating of 1400.
I believe this is because the ratingChange api response returns rating 0 but the default rating is only used if the rating is null.
One possible fix would be to also use default rating if the rating is zero. There may be some false positives who actually have rating zero but they should be negligible, as there currently is no active user who has that rating. I tested this and it seems more accurate than before.

First, sorry for the incredible latency here.

...unrated users count as having rating 0 instead of the default rating of 1400.

This is not too surprising. It is a part of the whole fake rating problem (#18 for details).

One possible fix would be to also use default rating if the rating is zero. There may be some false positives who actually have rating zero but they should be negligible, as there currently is no active user who has that rating. I tested this and it seems more accurate than before.

Note that this change does not help with users with fake ratings other than 0. Neither can I think of some other easy way to detect them. So prediction will remain inaccurate with such cases in the data.
So this is a not a complete fix, but I agree that this sounds like an improvement overall.


Series is a plural noun

I just realised while reviewing https://github.com/Inria-Asclepios/medInria-public/pull/214 (actually, I realised after it was merged unfortunately) that the word "serie" is used multiple times in the GUI. The word does not exist in english, as "series" is always plural.

@fcollot what's the correct word? a "series"?

Yep.


Autorenaming to bad names

For example, I have a variable with undefined4 type.  Dots is the part of the name.

If the string is showing "??..." for its value it indicates that there is a problem getting the bytes from memory or a character set problem.
Is 007bb17f near the end of a memory section, or have you manually assigned a character set to that string or set the default character set to something other than the default?

It is the string allocated in data section memory (not initialized). Sure, it will be without predefined bytes.


New Ruby HTTParty generator

Ruby Net/HTTP is quite verbose, and I usually use HTTParty. These commits add support for generating HTTParty code, although I've removed the support for forms/proxies for the moment.
I've followed the contributing guide, but let me know if there is anything else you need in order to merge this.

Looks great!

Thanks


Problem with editablegrid 2 and AdminLTE

Hello, I have the following problem. I have a grid that does not make me see all the columns that should. A similar error I had already come across and almasaeed I had helped solve. But with the solution you gave me I do not work in this case. How so? In the attached file you can find an image of the grid should be seeing many more columns, for accuracy should be seeing these columns here:
ID
NOME
ABILITATO
COGNOME_UTENTE
LINGUA
LIVELLO
PATH_DB
IMG
action
I do not see the action column. And theoretically there were other columns not me see. Annex are also the source of the page. Explain to me where am I wrong? thanks a lot
Attachment link: www.lazaruspascal.it/download_personali/Allegato.zip

Hi,
This issue should be reported to the plugin's maintainer since it is not an AdminLTE issue. Thank you.


Upgrade static site Ruby version

Background
The Ruby version on the static site is fairly out of date and should be updated. Our current version (2.7) will be EOL by March 2023.
Action Requested
Upgrade our Ruby version.
Acceptance criteria
Ruby is upgraded on the static site and we're no longer facing the March EOL deadline.

@emyl3 can you add any details to the description above? Want to make sure I captured what we talked about in standup yesterday

@emmastephenson @rin-skylight
My inclination is to upgrade to ruby 3.1.3 since 3.1 has been out for a year now and support will be until March 2025.
I am a little hesitant to upgrade to the latest (ruby 3.2.0) since it was released around 2 weeks ago. (see https://endoflife.date/ruby)
Do either of you see any issues with this? 🤔

@emyl3 no issues on my end! Upgrading to 3.1 sounds like the safest course of action 👍


wehahah

we have a problem, idk what it is xD

line 1
line 2
#bold
emphasize

*list 1
*list 2
*list 3
sourcecode

use
use
use

bold

emphasize


Vue Compat: deprecation ATTR_FALSE_VALUE

Overview of the problem
In my repository I have `vue-compat installed to see all the things that I must change and therefore some warnings appear.
Buefy-next version: 0.1.0
Vuejs version: 3.3.4
OS/Browser: Linux/Firefox
Description
The following warning message appears:

This is likely being caused by this:
https://github.com/ntohq/buefy-next/blob/f0348bfbc83536b76717b023821f941ac5c83ae9/src/components/dropdown/Dropdown.vue#L10
Steps to reproduce
It should happen with any usage of the <b-dropdown> component as long as vue-compat is installed (I believe other components might fall into this too, but I'm focusing on this to keep things simple).
Expected behavior
No warning at all :)

<b-datepicker> should have a similar issue:
https://github.com/ntohq/buefy-next/blob/f0348bfbc83536b76717b023821f941ac5c83ae9/src/components/datepicker/DatepickerTableRow.vue#L19


Can not find command.publishStub function

I'm trying your adonis package with adonis 6, I got an error while I try to run
node ace configure adonis6-amqp
this's the error message:
[ error ] command.publishStub is not a function
Am I need to install something?

Hi,
Sorry for late answer.
This package was developped during Adonis 6 alpha, this might be why it's not working anymore.
I'm currently working on a new version of this package that will be more expandable and suitable.


Add screenshot in the JUnit output for system tests

Summary
RSpec system tests produce a failure screenshot. The path of the screenshot is printed out when the test fails.
Proposal
Parse the test output and grab the path to the screenshot and output it the JUnit XML output.

You can configure it the hacky way if you like. I wanted to have my screenshots made with the rails screenshot helper easily accessable in Gitlab-CI, so I did the following:
module ScreenshotForCiHelper
  def take_screenshot
    super
    @screenshot_image_path = absolute_image_path.delete_prefix(Rails.root.to_s)
  end
end

RSpec.configure do |config|
  config.include ScreenshotForCiHelper, type: :system
  config.around(:each, type: :system) do |example|
    @screenshot_image_path = nil
    example.run
    example.metadata[:stdout] = "[[ATTACHMENT|#{@screenshot_image_path}]]" unless @screenshot_image_path.nil?
  end
end

Now everytime a screenshot is taken, its path is saved in the JUinit XML output like gitlab needs it.
See: https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html#view-junit-screenshots-on-gitlab

This gem is for outputting junit compatible xml. There is nothing in that standard for outputting screenshots like this. I'd suggest moving to a dedicated format that can have attachments, or writing an extension to integration your screenshot tool with rspec output :-)


Remove hash97

Because the prime modulus is less than 2**381, we can just use the top bit of the x coordinate for the parity bit. This is what we do for the current alt_bn128 implementation already. So we don't need 97 bytes for anything, 96 works just fine.

Already handled in #116


"Learn about Silver and Gold competency membership levels" link server error

Hello,
I've found a server error page and there was no feedback section at the bottom of the page.
the page where the problematic link is located and the feedback is missing:
https://docs.microsoft.com/en-us/partner-center/mpn-get-action-pack
The problematic link on that page:
Learn about Silver and Gold competency membership levels
The server error:

Thanks for the feedback, @Gerasz87 ! We've updated the link to point to the correct location. #please-close

Thanks for the quick fix!


⚠ ClamAV has degraded performance

In 0898640, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 66 ms

Resolved: ClamAV performance has improved in 2612267.


Metrics not saved when datasource changed

Make sure these boxes are checked before submitting your issue - thank you!

[ x] I have checked the superset logs for python stacktraces and included it here as text if any
[x ] I have reproduced the issue with at least the latest released version of superset
[ x] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version
master
Expected results
In explore view, when the datasource is switched, any metrics that occur in both datasources are preserved.
Actual results
If metrics are selected, and the datasource is changed, all metrics are deleted. This is a problem if you have several datasources with the same fields and you are wanting to create the same chart on all of them.
Steps to reproduce
Create a chart
Add some metrics
Change the datasource
Metrics are deleted

I will say if you change datasource, metrics options won't change if datasource have different options.
that's an issue, too.


Error when resetting quota leaderboard automatically

C:\Users\user\desktop\projects\onerealm\out\utilities\TimeUtilities.js:140
            throw new Error("negative time");
                  ^

Error: negative time
    at Object.formatDuration (C:\Users\user\desktop\projects\onerealm\out\utilities\TimeUtilities.js:140:19)
    at Object.getQuotaLeaderboardEmbed (C:\Users\user\desktop\projects\onerealm\out\managers\QuotaManager.js:524:56)
    at Timeout.run [as _onTimeout] (C:\Users\ewang\user\projects\onerealm\out\managers\QuotaManager.js:647:45)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

This error should have been fixed


SAA-326

Codecov Report
Base: 69.92% // Head: 70.19% // Increases project coverage by +0.27% :tada:

Coverage data is based on head (d740d9f) compared to base (6b90e5f).
Patch coverage: 98.18% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #177      +/-   ##
============================================
+ Coverage     69.92%   70.19%   +0.27%
- Complexity      712      728      +16
============================================
  Files           143      145       +2
  Lines          3378     3426      +48
  Branches        129      130       +1
============================================
+ Hits           2362     2405      +43
- Misses          938      939       +1
- Partials         78       82       +4

Impacted Files
Coverage Δ

...mppsactivitiesmanagementapi/entity/PrisonRegime.kt
91.66% <91.66%> (ø)

...hmppsactivitiesmanagementapi/model/PrisonRegime.kt
100.00% <100.00%> (ø)

...tivitiesmanagementapi/resource/PrisonController.kt
88.88% <100.00%> (+0.65%)
:arrow_up:

...mentapi/service/ActivityScheduleCreationService.kt
82.60% <100.00%> (+1.21%)
:arrow_up:

...vitiesmanagementapi/service/PrisonRegimeService.kt
100.00% <100.00%> (ø)

...activitiesmanagementapi/util/TransformFunctions.kt
73.02% <100.00%> (+0.75%)
:arrow_up:

...vitiesmanagementapi/entity/ActivityScheduleSlot.kt
87.71% <0.00%> (-7.02%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Update Monitor Renames and Enums

This PR addresses the following:

[ ] Add endpoint as a public property
[ ] Change filter parameter name in CreateQuery to query
[x] Start ServiceVersion numbering at 1 in LogsQueryClientOptions
[x] Start ServiceVersion numbering at 1 in MetricsQueryClientOptions
[x] Change Top to Size in MetricsQueryOptions
[ ] Add partial failure plan to LogsQueryResult
[ ] GetResult in class LogsBatchQueryResult should return a LogsBatchQueryResult
[ ] Decide if GetResult in class LogsBatchQueryResult is necessary
[x] Hide Deserialize in class LogsTable
[x] Make GetObject methods internal in class LogsTableRow
[ ] Don't return nullables everywhere
[ ] Rename NamespaceCollection with Metric prefix
[x] Spell out Qos in NamespaceClassification
[ ] Rename QueryModelFactory to MonitorQueryModelFactory - change title in autorest.md
[ ] Rename TimeSeriesElement with Metric prefix?
[x] Change Data to Values in TimeSeriesElement class
[ ] Add Collection type

Linked: https://github.com/Azure/azure-sdk-for-net/issues/23580


Fix bug: rust mark warnings as errors

Fix: #1919

Thanks for the PR - have not looked into it yet, but also created one: https://github.com/neomake/neomake/pull/2393
What's better? :)

Thanks for the PR again in the first place!
Let's continue it in https://github.com/neomake/neomake/pull/2393.


update version of azure-functions-java-library

work item: #224

Codecov Report

Merging #225 into develop will increase coverage by 0.02%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop     #225      +/-   ##
===========================================
+ Coverage    58.61%   58.64%   +0.02%
===========================================
  Files           50       50
  Lines         1590     1591       +1
  Branches       176      176
===========================================
+ Hits           932      933       +1
  Misses         596      596
  Partials        62       62

Impacted Files
Coverage Δ

...oft/azure/maven/function/bindings/HttpBinding.java
78.94% <100%> (+1.16%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 78b81c0...4b52b50. Read the comment docs.


🔵 Problem to get Typescript working in a fresh new vscode #macOS-catalina #typescript

VSCode Version: 1.43.1
Node.js : 12.8.1
OS Version: Darwin x64 19.3.0
Apple Model :	MacBook Pro 14,3
MacOS : Catalina 10.15.3

Just download vscode on a Mac Book Pro
Steps to Reproduce

npm install -g typescript seems to install or update<EMAIL_ADDRESS>correctly
but then tsc -v says tsc: command not found

same steps in video

Does this issue occur when all extensions are disabled?: Console says (when trying to disable extensions) bash: code: command not found

but then tsc -v says tsc: command not found

This means that global node modules are not properly configured on your PATH. It's an environment issue. You could try tweak the terminal.integrated.osx.shellArgs setting to [] or ["l"] to see if that fixes it.

bash: code: command not found

There's a command (F1) to install this to your PATH.


Archive state optimization

Problem description
Right now we archive state using BeaconState.serialize() which is not optimized, see https://github.com/ChainSafe/lodestar/blob/unstable/packages/state-transition/test/perf/util/serializeState.test.ts#L20
Solution description

always serialize state using state.serializeToBytes() api
use preallocated buffer pool , similar to what PersistentCheckpointStateCache does https://github.com/ChainSafe/lodestar/blob/rc/v1.19.0/packages/beacon-node/src/util/bufferPool.ts

Additional context
No response

should be done after we turn on n-historical state by default
then we have a bufferPool on chain to be used for archiving state too

we done need this considering #7005


Pin wpilib-controller version

I just updated the wpilib-controller package with changes from the
upstream PR.  We should update our code to use this at some point.
Any subsequent builds will fail if either this isn't merged in,
or someone actually updates the code to use the new calculate method.

Codecov Report

Merging #75 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #75   +/-   ##
=======================================
  Coverage   83.68%   83.68%
=======================================
  Files          21       21
  Lines        1606     1606
  Branches      160      160
=======================================
  Hits         1344     1344
  Misses        220      220
  Partials       42       42

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dda4b60...5542a50. Read the comment docs.


Error on converting mobilenet ssd model

System information

OS Platform and Distribution: Linux Ubuntu 16.04
NDK version: 19c
GCC version: 5.4.0
MACE version: v0.10.0
Python version: 2.7
Bazel version: 0.13.0

Model deploy file (*.yml)
library_name: mobilenet-ssd
target_abis: [arm64-v8a]
model_graph_format: file
model_data_format: file
models:
  mobilenet_v1: # model tag, which will be used in model loading and must be specific.
    platform: tensorflow
    # path to your tensorflow model's pb file. Support local path, http:// and https://
    model_file_path: /workspace/model/frozen_inference_graph.pb
    # sha256_checksum of your model's pb file.
    # use this command to get the sha256_checksum: sha256sum path/to/your/pb/file
    model_sha256_checksum: 74c6e4d38e2fb30f09d487160a41141c2625a435179b00e26df2ee543fac04bc
    # define your model's interface
    subgraphs:
      - input_tensors:
          - image_tensor
        input_shapes:
          - 1,300,300,3
        output_tensors:
          - detection_boxes
          - detection_classes
          - detection_scores
        output_shapes:
          - 1, 100, 4
          - 1, 100
          - 1, 100
    # cpu, gpu or cpu+gpu
    runtime: cpu+gpu
    winograd: 0

Describe the problem
NodeDef mentions attr 'Truncate' not in Op.
To Reproduce
1. cd /path/to/mace
2. python tools/converter.py convert --config=/workspace/model/mobile-net-ssd.yml

Error information / logs
Traceback (most recent call last):
  File "/root/mace/bazel-bin/mace/python/tools/converter.runfiles/mace/mace/python/tools/converter.py", line 414, in <module>
    main(unused_args=[sys.argv[0]] + unparsed)
  File "/root/mace/bazel-bin/mace/python/tools/converter.runfiles/mace/mace/python/tools/converter.py", line 213, in main
    option, FLAGS.model_file)
  File "/root/mace/bazel-bin/mace/python/tools/converter.runfiles/mace/mace/python/tools/converter_tool/tensorflow_converter.py", line 309, in __init__
    tf.import_graph_def(transformed_graph_def, name='')
  File "/root/.pyenv/versions/2.7.12/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "/root/.pyenv/versions/2.7.12/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 493, in import_graph_def
    raise ValueError(str(e))
ValueError: NodeDef mentions attr 'Truncate' not in Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: ToFloat = Cast[DstT=DT_FLOAT, SrcT=DT_UINT8, Truncate=false](image_tensor). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
Traceback (most recent call last):
  File "tools/converter.py", line 1346, in <module>
    flags.func(flags)
  File "tools/converter.py", line 853, in convert_func
    convert_model(configs, flags.cl_mem_type)
  File "tools/converter.py", line 782, in convert_model
    ",".join(model_config.get(YAMLKeyword.graph_optimize_options, [])))
  File "/root/mace/tools/sh_commands.py", line 549, in gen_model_code
    _fg=True)
  File "/root/.pyenv/versions/2.7.12/lib/python2.7/site-packages/sh.py", line 1413, in __call__
    raise exc
sh.ErrorReturnCode_1:

This error occured when call the tensorflow's import_graph_def function: tf.import_graph_def(transformed_graph_def, name='').
It seems that the model is invalid? can you load the model in tensorflow successfuly?

@alexst07 by the way, the output_shapes in yml should not have whitespaces.

@lu229 Yes, I can. In fact all models that I am trying can be found here: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
These are models from oficial tensorflow repositroy.

The Postprocess of this model include some logical operations which mace does not support. You can use ClassPredictor and BoxEncodingPredictor as output nodes.


🩹 Fix: Re-align Rate Limiter code block

On https://gofiber.io, the rate limiter section has a misaligned code block for limiter.Config{}:

This is fixed by updating components/blocks/RateLimiter.tsx to add 2 extra spaces after Max:.
Fixes: #110

Summary by CodeRabbit

Chores

Minor formatting adjustment made to the Max field in the RateLimiterBlock component for improved code consistency.

@grivera64 Now that I look at it, all the code in the main() function also has to be indented. It has 2 instead of 4 spaces.


ninôhtêpayin is not recognized as nôhtêpayiw, which doesn't get a paradigm

For whatever reason, ninôhtêpayin is not analyzed to have the lemma nôhtêpayiw:

... even though the FST recognizes the form:
hfst-lookup -q src/analyser-gt-desc.hfst
ninôhtêpayin
ninôhtêpayin	nôhtêpayiw+V+AI+Ind+1Sg	0.000000

nôhtêpayiw
nôhtêpayiw	nôhtêpayiw+V+AI+Ind+3Sg	0.000000
nôhtêpayiw	nôhtêpayiw+V+II+Ind+3Sg	0.000000

Might this be an artifact of the ambiguity concerning the lemma? (and thus related to #395)?

I downloaded the newst releases of descriptive-analyzer.hfstol and verified our app is indeed using the latest file.
It turns out the latest descriptive-analyzer.hfstol file does not recognize nôhtêpayin
$ md5sum CreeDictionary/res/fst/crk-descriptive-analyzer.hfstol
400d4a338fcd9fe928959594a8119c7b  CreeDictionary/res/fst/crk-descriptive-analyzer.hfstol
$ md5sum ~/Downloads/crk-descriptive-analyzer.hfstol
400d4a338fcd9fe928959594a8119c7b  /home/matt/Downloads/crk-descriptive-analyzer.hfstol

$ hfst-optimized-lookup CreeDictionary/res/fst/crk-descriptive-analyzer.hfstol
nôhtêpayin
nôhtêpayin      nôhtêpayin      +?

@eddieantonio @aarppe

oooh, I think you meant  ninôhtêpayin instead of nôhtêpayin?? is that a typo in the fist sentence?

Yes, there was a typo, but the issue name has things right. The inflected form should be ninôhtêpayin.

This seems to have been fixed with #419. The remainder is a duplicate in the source XML, which should become a separate issue #420.


RuntimeError: all tensors must be on devices[0]

I got this error when running 'Model_Test_OutputSaver_TrainDataset.py'.
Traceback (most recent call last):
File "/usr/Data/Emotion/Attention/Group-Level-Emotion-Recognition/Ensemble_Models/Model_Test_OutputSaver_TrainDataset.py", line 1418, in
model = train_model(model_ft, None, None, None, num_epochs=epochs)
File "/usr/Data/Emotion/Attention/Group-Level-Emotion-Recognition/Ensemble_Models/Model_Test_OutputSaver_TrainDataset.py", line 1395, in train_model
outputs, output1, output2, output3, output4, output5, output6, output7, output8, output9, output10, output11, output12, output13, output14 = model(inputs, labels, face_features_mtcnn, face_features_aligned, numberFaces, phase)
File "/usr/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 123, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/usr/local/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 133, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/usr/local/lib/python2.7/site-packages/torch/nn/parallel/parallel_apply.py", line 77, in parallel_apply
raise output
RuntimeError: all tensors must be on devices[0]
Does anyone have any idea how to solve this?
Thank you

Fixed by changing the following lines:
Line 327: model2 = nn.DataParallel(model2, device_ids=[0])
Line 1336: model_ft = nn.DataParallel(model_ft, device_ids=[0])
to


Unable to re-dump artwork

https://forum.kodi.tv/showthread.php?tid=258886&pid=2996615#pid2996615
If you delete dumped artwork from your drive you can no longer dump it again without first either changing the artwork or refreshing the movie/tv show.

What? Don't delete it if you want to keep it. That's not how this was designed. It downloads artwork then changes the path to point to the newly downloaded artwork.
The whole point of downloading artwork once is so that Kodi doesn't have to download things over and over again. If you are using this tool to redownload the same thing then what's the point?

Ok, thanks I understand how it works now, I didn't realise it changed the path in the database.


Edits to next_dfs() docstring example.

Edited the code example in the next_dfs() docstring, following the same loop style as shown in dwarfdump example found here: https://github.com/gimli-rs/gimli/blob/master/examples/dwarfdump.rs#L799

No problem! :D

Failing CI is due to https://github.com/rust-lang/rust/issues/51699 -- going to just go ahead and merge this.

This is an improvement over the previous example, which was a non-typical use. But I think this is still a non-typical example because if you're traversing starting at the root then it's good enough to stop when next_dfs returns None. Keeping track of the depth is more useful when you start at a specific entry.


Refactor kafka-producer module to use build-in spring config

use config in application.yml
spring:
kafka:
producer:
bootstrap-servers


How to interact with the server from the command line?

Hi, thanks again for this great tool!
I'm trying to get the server to work with my editor, but am having problems verifying the server is working.
To start the server, I open up one shell and run:

python -m galaxyls --tcp --host=<IP_ADDRESS> --port=2087

And to interact with it, I open up another shell and run:

socat - tcp:<IP_ADDRESS>:2087, where I can then start sending strings to the server on every new line
e.g.
TEST
<param type="
hello

And I can verify that the messages are being sent by checking the galaxy-language-server.log file to see:
DEBUG:pygls.protocol:Received b'TEST\n'
DEBUG:pygls.protocol:Received b'<param type="\n'
DEBUG:pygls.protocol:Received b'hello\n'

However, I cannot get any information back from the server. Is a response expect through this port? Or are there replies somewhere else I should be looking for?
Thanks

Hi @mtekman!
The language server is an implementation of the Language Server Protocol Specification. This protocol uses messages with a simple HTTP like header and content in JSON-RPC 2.0.
These messages should be handled directly by your editor (if it supports the LS protocol) so you don't have to manually launch or send messages to the server.
In which editor are you trying to integrate the server?

Ah okay. I knew the strings I was sending were not correctly formatted, I guess I was just surprised by the lack of any response at all -- but if that's expected, then I will need to keep debugging things on my side.
I'm trying to get it working with Emacs https://github.com/emacs-lsp/lsp-mode/pull/2333 in conjunction with the LSP package, but have run into the issues and suspected it might be the server not responding properly. Though I see now it's likely working fine it just needs proper headers from the editor.
Thanks -- I will close the issue then

Cool!
I haven't used EMACS (and I am fairly new to the LSP world), but let me know if I can help you with anything :)

@mtekman you can test the server with VSC and see if the server works. You could even try to capture the communication if that is needed.


Feature Request: Sweep config: Probabilities for categorical distribution

wandb --version && python --version && uname
wandb, version 0.8.24
Python 3.7.5
Linux
Description
I want to run a sweep with the following config:
parameters:
  param1:
    values: ["True", "False"]

But let the value True happen in 10% of the cases, and False in 90% of the cases.
Would be great if you could add this, thanks a lot!

Good idea - I will file a ticket.  One hacky workaround now would be to copy "False" 9 times.

@lukas that's how I ended up doing it, but it felt very hacky to me indeed :D Also I wasn't sure how Bayesian Optimization would deal with it – would it treat the strings as different values or as the same?

Hey @dreamflasher
In the past year we've majorly reworked the CLI and UI for Weights & Biases. We're closing issues older than 6 months. Please comment to reopen.

Please reopen

We are closing this issue due to inactivity. If you feel that it merits additional review, please provide a reproduction and add a comment explaining why. Thank you.


Monitoring - failed to upgrade project monitoring which is enabled before upgrading Rancher

What kind of request is this (question/bug/enhancement/feature request):
BUG
Steps to reproduce (least amount of steps as possible):

run Rancher v2.2.1
add a single-node cluster
enable cluster monitoring and project monitoring
upgrade Rancher to v2.2.4-rc8
go to the project monitoring page
check the checkbox Upgrade monitoring to the latest version and click Save

Result:

The app project-monitoring failed to upgrade

Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): v2.2.1 -> v2.2.4-rc8
Installation option (single install/HA): single install

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): Infrastructure Provider
Machine type (cloud/VM/metal) and specifications (CPU/memory): 4 vCPU, 8GB Ram
Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.7", GitCommit:"6f482974b76db3f1e0f5d24605a9d1d38fad9a2b", GitTreeState:"clean", BuildDate:"2019-03-25T02:52:13Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.5", GitCommit:"2166946f41b36dea2c4626f90a77706f426cdea2", GitTreeState:"clean", BuildDate:"2019-03-25T15:19:22Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Docker version (use docker version):

18.9.6

We don’t support this upgrade path. Users need to disable project monitoring first if they upgrade from 2.2.1. This will be in release note

It should be related to https://github.com/helm/helm/issues/1996


[LSP] completion use wrong column offset

Input
support following code
# typed: true
# frozen_string_literal: true

Array.ne # cursor is between n|e, offset is 7

with LSP request:
{
	"id": 6,
	"jsonrpc": "2.0",
	"method": "textDocument/completion",
	"params": {
		"position": {
			"character": 7,
			"line": 3
		},
		"textDocument": {
			"uri": "xxxx.rb"
		}
	}
}

Observed output
lsp server return:
{
	"jsonrpc": "2.0",
	"id": 6,
	"requestMethod": "textDocument/completion",
	"result": {
		"isIncomplete": false,
		"items": [
			{
				"label": "new",
				"kind": 2,
				"detail": "Class#new",
				"documentation": {
					"kind": "markdown",
					"value": "```ruby\\nsig {params(args: T.untyped).returns(T.untyped)}\\ndef new(*args); end\\n```\\n\\n---\\n\\nCalls `allocate` to create a new object of *class*'s class, then invokes\\nthat object's `initialize` method, passing it *args*. This is the method\\nthat ends up getting called whenever an object is constructed using .new."
				},
				"sortText": "000001",
				"insertTextFormat": 1,
				"textEdit": {
					"range": {
						"start": {
							"line": 3,
							"character": 5
						},
						"end": {
							"line": 3,
							"character": 7
						}
					},
					"newText": "new"
				}
			}
		]
	}
}

Expected behavior
textEdit range is wrong, should be 6-7 or 6-8.
Other issues:
Array. have no completions, must provide query to get completion. expect to return all completions without filter

Array. have no completions, must provide query to get completion. expect to return all completions without filter

This is a known issue. It's unlikely to get fixed unless someone more talented with parser generators than me looks at how to make our parser recover from parse errors better. That would be a welcome change.

This was fixed in #5081
Thanks again for the report.


Upgrade to Typescript 2.0

TypeScript 2.0 is way better and nixes the need for typings.

Frankly speaking, this is my first project using Typescript and I migrated to typescript in the process (initially it was ES6 - that's why I have babel dependencies and that's why codebase is poorly typed)
I believe moving to TS 2.0 can be done together with https://github.com/Rebilly/ReDoc/issues/79.
Can't it?

Yup :)


Music fixes

Pull request

[x] I have read and agreed to the code of conduct.
[x] I have read and complied with the contributing guidelines.
[x] What I'm implementing was assigned to me or was not being worked on and is a somewhat requested feature. For reference see our GitHub projects.
[ ] I have tested all of my changes.
(Discord started dying in the middle of my testing so I wasn't able to finish testing)

Added/Changed
Better music error handling
New audio track user data so I could do the better error handling
Added QueueMoveSubCommand
Added QueueRemoveSubCommand
Added MovableList for lists with items that the user can move.
Added the appropriate methods to cascade player for those command.
Did proper audio track user data checks as we didn't really have any checks on it before.

You're sending a few messages in this commands, do they need to be languified?

You're sending a few messages in this commands, do they need to be languified?

Yegh i forgot to do language stuff. Give me a min. Thanks for reminding me.

jenkins build


[TC-ALOGIN-12.1] - The automated script needs to be updated by removing the comment  mentioned before step 1

Description :
[TC-ALOGIN-12.1] - The automated script needs to be updated by removing the comment mentioned before step 1 because the mentioned issue has been resolved in the SDK.

YMAL Script Reference : https://github.com/project-chip/connectedhomeip/blob/master/src/app/tests/suites/certification/Test_TC_ALOGIN_12_1.yaml
Resolved Issue in SDK : https://github.com/project-chip/connectedhomeip/issues/28992

PR
https://github.com/project-chip/connectedhomeip/pull/32625

Issue resolved


fix: properties missing from server config view

Pull Request Title
Added the missing properties:

FRPS Port
FRPS Protocol

Description
Followed the pattern and wrote the code to add the missing properties properly.
Related Issue(s)
This PR addresses issue #780
/closes #780
/claim #780

@RS-labhub this was not the required solution. As you can see in the issue, daytona server config is the view that is missing properties.
Closing this PR now, feel free to open a new one with the required changes.

The bot was wrong 😄
That was just an experiment ran by algora, it was not a suggestion from us. Sorry for the confusion.


Tutorial in jupyterhub

Some ideas to improve the tutorial after I've set it up into a jupyterhub environment.
As starting jupyter hub was useless, I did not want to run anaconda-project run jupyter notebook. However packages were missing, so I ran sudo -E anaconda-project run but it seems it does start a notebook nonetheless. What does anaconda-project run do? (yes I know: it was maybe not smart to start as super user an unknown script).
Besides what's already under the "setup" page, I had to install specific packages:
conda activate project
# below two lines is for jupyter to suggest the right environment
sudo -E conda install ipykernel
sudo -E /opt/tljh/user/envs/project/bin/python -m ipykernel install --name 'holoviz-tuto'
# certainly the last line could be made more flexible as counting on this specific version of tljh

# herebelow missing packages
sudo -E conda install -c pyviz datashader=0.14.0 bokeh=2.4.3 holoviews=1.15.0
sudo -E conda install -c conda-forge numpy=1.21 fastparquet=0.8

I also made a bash script to copy the notebooks but have soft links to the data from a read-only shared folder to the jupyter user home.
I haven't tried the whole tutorial, so I hope there is no mandatory writing back into the other folders besides the tutorial folder.
if [ ! -d ~/holoviz_tutorial/ ]
then
  mkdir ~/holoviz_tutorial
else
  exit 1
fi

echo "copying tutorial notebooks"
cp -R ~/shared/holoviz_tutorial/tutorial ~/holoviz_tutorial
for folder in `ls ~/shared/holoviz_tutorial/`
do
  if [ "$folder" != "tutorial" ]
  then
    echo "linking file or folder $folder"
    ln -s ~/shared/holoviz_tutorial/"$folder" ~/holoviz_tutorial/"$folder"
  fi
done

exit 0

Good question; I'm not sure how best to set this up in jupyterhub. What anaconda-project run <command> does is to:

"Prepare" the environment specified in the anaconda-project.yml file. It's basically the equivalent of conda env create -f anaconda-project.yml (as our anaconda-project.yml files are also valid conda environment files), except that (a) the environment is stored in the current directory so that a project is entirely independent of anything else on your system, and (b) if the envt already exists, any packages specified in the .yml but not already installed are installed.
Activate the conda environment
Run the specified <command> in that environment.

Here if <command> is something defined in the .yml file (e.g. dashboard), the text of that command is used instead, including (if needed) injecting command-line arguments in some cases (e.g. when launching notebooks or dashboards as web services). If <command> isn't defined in that file, it is executed as a command-line command, e.g. echo hello.
In this case you could launch JupyterHub as the command, e.g. anaconda-project run jupyter hub, which would run the entire JH instance in the envt it creates. If the JH is solely for use with this tutorial, that seems like the easy solution. If you want to use the tutorial on an existing JH instance, then I'm not sure how best to define the environment and make it available from within JH, but hopefully the above will make that clear for people who use JH more than I do.

Thanks for the explanation. I believe anaconda-project is unnecessary in a jupyter hub context.
I could simply suggest a addition to the documentation. Although I am not sure how to deal with the information contained in the yml file, except manually installing the packages.


Dark Mode Styling Solutions

fixes https://github.com/android-graphics/sokatoa/issues/369

still not thrilled with the selected track color but at least we can read the text now. Thanks!

still not thrilled with the selected track color but at least we can read the text now. Thanks!

I'll see if there are any good themed colors in Theia to replace the yellow with


LD plot not loading

The LD plot on the variant page (e.g. https://www.ebi.ac.uk/gwas/variants/rs2249296) is not loading. Unclear whether this is associated with the last data release.

LD plot not loading cause EBI proxy times out the request after some time. the request takes too long to finish cause it makes calls to OLS which is currently unusually slow


[db-up] Pin pymssql==2.2.7

Pin pymssql==2.2.7 to avoid 2.2.8 issue https://github.com/pymssql/pymssql/issues/832

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

db-up


Add documentation for AMP support

Some things that come to mind:

How to opt in to having an alternative version
How to opt in to having a full amp page
next/amp const isAmp = useAmp() hook
How to include AMP components (timeago etc)

Merged in https://github.com/zeit/next.js/pull/7031 and we added to Learn site.


move ENV var export from buildkite script, to buildkite pipeline settings

Issue Number

#100
Overview

[ ] I have moved ENV var export from buildkite script, to buildkite pipeline settings

Comments

See here. It seems that we can't modify the environment directly from the buildkite execution script, but it needs to be set within the platform interface through the "Pipeline Settings"; which I did. I've scheduled a new build for 10:30 to see whether this worked or not.

Well.. it seems it still doesn't work. There's some nix magic going on here I suppose :| ... I am not sure to understand what's happening. I'll investigate a bit more.

Finally nailed it and buildkite just finished successfully.


No event.relatedTarget for Collapse

I have a form in a moral to do the edit job. And then the form become too big that it does not fit in the modal. Then I realise that collapse could be a good alternative for the modal.
And then I noticed that collapse event does not have the .relatedTarget property as modal events. I have to get some data from the relatedTarget to render the form.

Approximate duplicate of #15393.


Setup pre-commit

We decided on using precommit: https://pre-commit.com/
To include:

ES lint
detect secrets

Maybe we should check out the sso-keycloak pre-commit.yml and .pre-commit-config.yaml?

@BCerki proposed splitting GitHub action enforcing pre-commit from this issue. @BryanGK would that change your plans for #13 at all?

@BCerki proposed splitting GitHub action enforcing pre-commit from this issue. @BryanGK would that change your plans for #13 at all?

@wenzowski I don't think so, as my changes are for pre-commit on your local machine

Perfect, let's adopt that split then. Issue breakdown for the win! 🥇


Go bindings

So I think I finally figured out how to provide Go bindings. I've made a proof of concept here:

https://github.com/jtraglia/cgo-kzg-4844

You can import this package and use the exported C-KZG-4844 functions.
Is this something we want in this repository?

Also, there's a bug in gcc that's preventing the bindings from working with both gcc and clang. It's been fixed in a newer version of gcc, but the version with the bug is still prevalent.

https://github.com/golang/go/issues/7270

For this to "just work" with both compilers, I think we should make a code change.

We can remove the const qualifier from blobs in the function definitions.

This is simpler, but less safe.

Change the type of blobs to not use the type def in the function definitions.

A little ugly, but same level of safeness.

The first option is straight-forward. Here's the proposed changes for the second option:
diff --git a/main.go b/main.go
index c775853..8d89781 100644
--- a/main.go
+++ b/main.go
@@ -157,7 +157,7 @@ func ComputeAggregateKzgProof(blobs []Blob) ([CompressedG1Size]byte, int) {
        proof := [KzgProofSize]byte{}
        ret := C.compute_aggregate_kzg_proof(
                (*C.KZGProof)(unsafe.Pointer(&proof)),
-               *(**C.Blob)(unsafe.Pointer(&blobs)),
+               *(**[BlobSize]C.uchar)(unsafe.Pointer(&blobs)),
                (C.size_t)(len(blobs)),
                &settings)
        return BytesFromG1(proof), int(ret)
@@ -181,7 +181,7 @@ func VerifyAggregateKzgProof(blobs []Blob, commitments []Commitment, proof Proof
        var result C.bool
        ret := C.verify_aggregate_kzg_proof(
                &result,
-               *(**C.Blob)(unsafe.Pointer(&blobs)),
+               *(**[BlobSize]C.uchar)(unsafe.Pointer(&blobs)),
                *(**C.KZGCommitment)(unsafe.Pointer(&commitments)),
                (C.size_t)(len(blobs)),
                (*C.KZGProof)(unsafe.Pointer(&proof)),
diff --git a/src/c_kzg_4844.h b/src/c_kzg_4844.h
index 959c2a1..667cd49 100644
--- a/src/c_kzg_4844.h
+++ b/src/c_kzg_4844.h
@@ -104,12 +104,12 @@ void free_trusted_setup(
     KZGSettings *s);

 C_KZG_RET compute_aggregate_kzg_proof(KZGProof *out,
-                                      const Blob blobs[],
+                                      const uint8_t blobs[][BYTES_PER_BLOB],
                                       size_t n,
                                       const KZGSettings *s);

 C_KZG_RET verify_aggregate_kzg_proof(bool *out,
-                                     const Blob blobs[],
+                                     const uint8_t blobs[][BYTES_PER_BLOB],
                                      const KZGCommitment expected_kzg_commitments[],
                                      size_t n,
                                      const KZGProof *kzg_aggregated_proof,

Would we avoid this if we used structs as suggested in the comments on #37 ?

Yes, if you convert Blob to a struct it will also fix the problem. For example, this works for both compilers:
diff --git a/main.go b/main.go
index c775853..8ec5171 100644
--- a/main.go
+++ b/main.go
@@ -201,7 +201,7 @@ func BlobToKzgCommitment(blob Blob) ([CompressedG1Size]byte, int) {
        commitment := [KzgCommitmentSize]byte{}
        ret := C.blob_to_kzg_commitment(
                (*C.KZGCommitment)(unsafe.Pointer(&commitment)),
-               (*C.uint8_t)(unsafe.Pointer(&blob)),
+               *(*C.Blob)(unsafe.Pointer(&blob)),
                &settings)
        return BytesFromG1(commitment), int(ret)
 }
diff --git a/src/c_kzg_4844.c b/src/c_kzg_4844.c
index 1fd7c14..64ea998 100644
--- a/src/c_kzg_4844.c
+++ b/src/c_kzg_4844.c
@@ -1000,7 +1000,7 @@ static C_KZG_RET poly_to_kzg_commitment(KZGCommitment *out, const Polynomial p,
 static C_KZG_RET poly_from_blob(Polynomial p, const Blob blob) {
     C_KZG_RET ret;
     for (size_t i = 0; i < FIELD_ELEMENTS_PER_BLOB; i++) {
-        ret = bytes_to_bls_field(&p[i], &blob[i * BYTES_PER_FIELD_ELEMENT]);
+        ret = bytes_to_bls_field(&p[i], &blob.data[i * BYTES_PER_FIELD_ELEMENT]);
         if (ret != C_KZG_OK) return ret;
     }
     return C_KZG_OK;
diff --git a/src/c_kzg_4844.h b/src/c_kzg_4844.h
index 959c2a1..25d6aa5 100644
--- a/src/c_kzg_4844.h
+++ b/src/c_kzg_4844.h
@@ -47,7 +47,7 @@ typedef blst_fr fr_t;         /**< Internal Fr field element type */
 typedef g1_t KZGCommitment;
 typedef g1_t KZGProof;
 typedef fr_t BLSFieldElement;
-typedef uint8_t Blob[BYTES_PER_BLOB];
+typedef struct { uint8_t data[BYTES_PER_BLOB]; } Blob;

 /**
  * The common return type for all routines in which something can go wrong.

@xrchz so what do you think?

I think having go bindings in this repo would be good, and we should follow the struct approach. I haven't had time to implement that yet - feel free to add to #37 if you are up for it!

The final roadblock is the lack of exposed sha256 functions in BLST. The current way we do things (patch & build) doesn't work very well with cgo. We need a GitHub repository to expose these so we can import the package in Go. If imported, the exported functions in BLST are visible during the compilation process. The ideal solution is to expose these functions in supranational/blst but if that doesn't pan out, we'll need to fork it under the ethereum organization.
I've made a PR for this which I'm hoping fairs well:

https://github.com/supranational/blst/pull/146

Related to #23


Support for AAD Registered devices

Does this feature work with AAD Registered (vs AAD Joined) devices?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 78c66f8a-1c40-e31c-5c7e-7870b7d65e0c
Version Independent ID: 3e8301bc-8e4d-ee23-8770-2f75e170e01a
Content: Passwordless security key sign (preview) - Azure Active Directory
Content Source: articles/active-directory/authentication/howto-authentication-passwordless-security-key.md
Service: active-directory
Sub-service: authentication
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

@Pikhulya
Thanks for your feedback! We will investigate and update as appropriate.

I've reached out to the product team to confirm about this. From the document it seems that they need to be joined though.

Hybrid will be available with the next version of Win 10.
Also, for Azure AD Registered you don't sign into the machine using Azure credentials.  But you can provision a security key for the work account and then use it with supported apps.
Hope this helps!


How to install versions of tensorflow from 0,12 to 0,9 nowadays, i can't find anywhere

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce
Mobile device

It has been 18 days with no activity and the awaiting response label was assigned. Is this still an issue?

@Paulekna  Pip allows you to install specific version of TensorFlow. Something like this will do your job

pip install tensorflow==(version number)
Ex. : pip install tensorflow==1.1


add unit test to improve coverage

Codecov Report

Merging #46 (d9c5e17) into main (3ac9b08) will increase coverage by 11.89%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##             main      #46       +/-   ##
===========================================
+ Coverage   33.07%   44.96%   +11.89%
===========================================
  Files          13       14        +1
  Lines         127      129        +2
  Branches       30       30
===========================================
+ Hits           42       58       +16
+ Misses         80       66       -14
  Partials        5        5

Impacted Files
Coverage Δ

src/pages/blog/[slug].tsx
0.00% <ø> (ø)

src/components/Article.tsx
100.00% <100.00%> (ø)

src/pages/blog/index.tsx
77.77% <100.00%> (ø)

src/components/Seo.tsx
100.00% <0.00%> (+16.66%)
:arrow_up:

src/components/components.tsx
19.56% <0.00%> (+19.56%)
:arrow_up:

src/components/Pre.tsx
100.00% <0.00%> (+100.00%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ac9b08...d9c5e17. Read the comment docs.


ODK forms for web based surveys

Software and hardware versions
Aggregate v1.x.x, Survey vx.x.x, Android vx.x.x, device used...
Problem description
Steps to reproduce the problem
Expected behavior
Other information
Things you tried, stack traces, related issues...

Closing due to lack of sufficient detail.


Add HasRawWindowHandle for WindowHandle in druid (rebased against master)

Expose the HasRawWindowHandle trait impl of druid-shell for
WindowHandle into druid itself, so druid users can access it. This
is also under a raw-win-handle feature, like the druid-shell one.
Bug: #1505
This is all the work of @djeedai in #1828, just rebased against master to help nudge it along the finish line :)

merged #1828 so will close this. Thanks!


Health check to validate dialogs can be dismissed

We added the dialog dismissal check as part of the recipe but it turns out that it already too late in the process making the task fail rather than preventing the test bed from picking up tasks.
We need to move this validation as part of the health checks running on on before task in the bot to quarantine them when the dialog dismissal logic fails.
Context:
https://github.com/flutter/flutter/issues/76796

Not feasible


Update radio-image.md

I have been using this docs for my field types, please correct the 'type' => 'radio' for 'type' => 'radio-image' since it does not work using only radio.
I came accross this because I was checking an issue regarding radio image not working, and it was not working for me until i found out this small change

thanks!
I started rewriting the docs a few days ago,so all the docs you see there were basically written this week.
If you find any other typos or inconsistencies please feel free to submit as many PRs as you can!
I really appreciate the help here...
Merged. :+1:


SDL behavior in case unable to get system time from HMI

Question is related to proposal https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0120-GetSystemTime.md.
According to proposal security service will be started only in case HMI provided system time.
And what about case when HMI can not provide system time, e.g navigation module is unavailable, satellites are not available( vehicle is some where in tunnel, on underground parking ).
Is this okay that application will not be able to start secure service?
Must SDL start unsecure service or fail handshake with session start in case application can not start secure service because of absence system time?

@dboltovskyi, @AByzhynar , @KhrystynaDubovyk , please be informed about question.

@GetmanetsIrina
The handshake process will need to be postponed until receiving a corresponding notification from hmi.

@mrapitis, thanks, but what if hmi does not send corresponding notification at all by some unexpected reason?

@GetmanetsIrina in that case SDL must fail the handshake process.

@mrapitis , Thank you!


Add support for JSON claim value type

What issue does this PR address?
Add support for parsing object claim as JSON Claim Value Type

Thanks! Which exact problem are you trying to solve? Is there an issue that discusses the problem?

I have created issue on Support repository https://github.com/DuendeSoftware/Support/issues/1487

I thinks this pr will be helpfull for those who use introspection endpoint or userinfo endpoint for authenticate and load claims from there, because microsoft add object as “JSON” ClaimValueType by default not ClaimValueTypes.String


ML Challenge page - design for responsive behavior

Tasks:

[ ] P0 - Update existing mobile screen designs to reflect final designs w/ final copy

Goal: users on small/mobile screens can clearly read all ML challenge page copy
ML challenge page mobile screens - needs update to reflect final designs/copy for H1

[ ] P1 - Update/Recommend designs for low-lift mobile nav

Goal: nav doesn't look broken for users
Goal: get lift estimate from eng to see if mobile nav is feasible
Existing mobile top nav explorations

[ ] P1 - Update H1 implemented designs with final copy

can just update screen 2.4, add note about when copy was updated.
do not need to update other pages, but add large note that content doesn't reflect final
Make full frame layout use auto-layout so that dannielle can implement future copy updates herself.
H1 ML challenge page design
Final copy for ML Challenge page

OOS:

mobile "jump to nav" (is in the design file, but will not touch for implementation, remove from mobile screens)

We handed off the spec before the copy was finalized.

H1 launch
Mobile responsiveness is a launch
post launch

This one is finished - P1 - Update H1 implemented designs with final copy

@kev-zunshiwang heads up that Jun and I upped the mobile nav to a P0 -- i dont think this affects you though!


How to test velodyne dataset?

Hi author,
I run velodyne_mapping.launch file and found it was nothing displayed on rviz. Did you have any suggestions to modify launch file parameters? Thanks for your reply very much O(∩_∩)O~

Hey,
you need a .bag file containing point clouds and tfs in order to use this launch file.
You need to specify the path to that bag file in the "bagfile" parameter in the third row of the .launch file.
Make sure the .bag file contains a transform from base_link to the velodyne frame - or change the "scan_assembler_frame_id" and "sensor_frame_id" parameters in the launch file according to your setup.


Added transferinfo client APIs to ReqMgrAux

Status
ready
Description
Implemented the client APIs within ReqMgrAux module such that we can read/write/update/delete workflow transfer documents (TRANSFER).
Is it backward compatible (if not, which system it affects?)
no
Related PRs
Yes, it implements the client side of this PR touching the server side:
https://github.com/dmwm/WMCore/pull/9302
External dependencies / deployment changes
no

Tested, fixed a bug in the _updateRecords private method and now it's good to go.


remove the wifi configure hack that was needed before UI support wifi…

… cfg
Signed-off-by: Naiming Shen<EMAIL_ADDRESS>
have tested in patch in various configuration cases.

Hey @naiming-zededa -- great to this this, but what's the e2e story here? Suppose I have a new device with only WiFi (like HiKey), what do I need to configure where to make it work?

@rvs,, first need to configure in the cloud with UI, the 'network' configuration part for the wifi params; then we needed to use USB to put the 'override.json' file into the /config/DevicePortConfig/ directory.

@rvs,, first need to configure in the cloud with UI, the 'network' configuration part for the wifi params; then we needed to use USB to put the 'override.json' file into the /config/DevicePortConfig/ directory.

But with this change it should be sufficient to do it in the UI, correct?

@rvs,, first need to configure in the cloud with UI, the 'network' configuration part for the wifi params; then we needed to use USB to put the 'override.json' file into the /config/DevicePortConfig/ directory.

But with this change it should be sufficient to do it in the UI, correct?

Correct. Except for USB is used for bootstrap.

LGTM.
I think we should improve the error handling for the WiFi and APN code. Some errors are logged but we might want to fail things up front if we can't decrypt the credentials or if we can't write the files. That means returning an error from checkAndUpdateWireless and have that result in setting LastFailed/LastError in the caller. But that can be a separate PR.
I also don't know if we can extract errors when WiFi fails to authenticate and associate.
That would be useful to be able to feed back to the controller. Perhaps we are getting it as part of the wlan container logs?

today the logs have those:
linuxkit-ac1f6b70fdaa kernel: [   67.157106] wlan0: authenticate with e4:6f:13:3f:73:08
linuxkit-ac1f6b70fdaa kernel: [   67.179534] wlan0: send auth to e4:6f:13:3f:73:08 (try 1/3)
linuxkit-ac1f6b70fdaa kernel: [   67.192639] wlan0: authenticated
linuxkit-ac1f6b70fdaa kernel: [  884.618783] wlan0: deauthenticating from e4:6f:13:3f:73:08 by local choice (Reason: 3=DEAUTH_LEAVING)


Update HTTP/2 to RFCs

Motivation:
HTTP/2 RFC 7540 has been released.
Modifications:

Make changes RFC 7540 introduced since draft 17.

Result:
HTTP/2 RFC 7540 compliant code

@nmittler - FYI.  This is to address https://github.com/netty/netty/issues/3787.  You mentioned you previously reviewed the spec and also didn't see any changes that would impact us....so this PR is just updating the version strings we use.

@Scottmitch ship it

@Scottmitch Make it so! :)

Go!

Cherry-picked 4.1 (0f28bdf) master (4cac795)


DEPS.xwalk: Roll v8-crosswalk (ba5299d -> b394589)

b394589 [SIMD.js] Fix performance regression of mandelbrot benchmark

It fixes https://crosswalk-project.org/jira/browse/XWALK-3525

@rakuco , please take a look. Thanks!

@darktears , please take a look at this one. It is for Crosswalk-12.


Question on stringbuilder in pizza.cs

Hi @willportnoy. Can you explain what tostring() in pizza.cs does? I see you dont have that in other sample projects.

It's a mechanism to make the output more readable.  The default ToString will serialize the class name.  This ToString override helps when the dialog writes the completed form back to the user.  The other samples either use the json serializer to make a string out of the completed form or don't output the completed form.


[Bug] - Plugin update does not always work on cluster members

During plugin updates not all members get updated from master

Unable to recreate


[SPARK-29506][SQL] Use dynamicPartitionOverwrite in FileCommitProtocol when insert into hive table

What changes were proposed in this pull request?

When insert overwrite into hive table, enabling dynamicPartitionOverwrite when initializing FileCommitProtocol.
Why are the changes needed?

HadoopMapReduceCommitProtocol uses FileOutputCommitter to commit job output files.
FileOutputCommitter continues do FileSystem.listStatus for directories in partitions, recursively, and commits job output leaf files:
https://github.com/apache/hadoop/blob/c0a0c353e86fc3f3d4eedd91aeb9d5b9c8eb748a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L478-L490
It is inefficient when dynamically overwritting many partitions and files.
HadoopMapReduceCommitProtocol, when dynamicPartitionOverwrite is enabled, writes to staging dir dynamically, and commits written partition directories, instead of leaf files.
Does this PR introduce any user-facing change?

No
How was this patch tested?

Existing tests.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17242/
Test PASSed.

Test build #112257 has started for PR 26159 at commit 669c7fa.

Test build #112257 has finished for PR 26159 at commit 669c7fa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112257/
Test PASSed.

cc @felixcheung @cloud-fan

For hive table insertion, we insert to a fresh staging dir first. So dynamicPartitionOverwrite and normal write are logically the same IIUC. Do you mean dynamicPartitionOverwrite is better for performance?

For hive table insertion, we insert to a fresh staging dir first. So dynamicPartitionOverwrite and normal write are logically the same IIUC. Do you mean dynamicPartitionOverwrite is better for performance?

Hi, @cloud-fan I think dynamicPartitionOverwrite would keep a filesToMove set and  the times to rename file is partitions-num.
https://github.com/apache/spark/blob/f4d5aa42139ff8412c573c96a1631ef3ccf81844/core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala#L181-L183
And normal write(let file output committer algorithm version to 1), per task would commit output to a temp path.
For example:
_temporary/0/task_attempt_1/p1=v1/p2=v2/task1.parquet
_temporary/0/task_attempt_2/p1=v1/p2=v2/task2.parquet
_temporary/0/task_attempt_3/p1=v1/p2=v2/task3.parquet
After all tasks completed,  it would invoke mergePaths to merge these output.
The cost is larger for a partitioned table than dynamicPartitionOverwrite.
But there is a known issue for dynamicPartitionOverwrite,  a task may conflict with its speculation task.
I have created a PR https://github.com/apache/spark/pull/26086, can you help take a look?

@cloud-fan FileOutputCommitter commits job in an inefficient way.

List all committed task paths under job attempt path:

https://github.com/apache/hadoop/blob/c0a0c353e86fc3f3d4eedd91aeb9d5b9c8eb748a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L402-L406
https://github.com/apache/hadoop/blob/c0a0c353e86fc3f3d4eedd91aeb9d5b9c8eb748a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L329-L333

Merge paths under committed task paths to job output path. When there are conflicting path names, it merges paths/files recursively:

https://github.com/apache/hadoop/blob/c0a0c353e86fc3f3d4eedd91aeb9d5b9c8eb748a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L486-L489
But dynamicPartitionOverwrite simply renames partition paths under staging dir to job output path.

Just to confirm, it's all for performance right? Does it fix a correctness issue?

@cloud-fan Yeah, this is not about correctness.

It looks to me that the underlying issue is: MR committer is slow and we need a home-made custom committer.
I think this is a big story and needs more discussion. This PR is just a trick to bypass the MR committer by using dynamicPartitionOverwrite when the behavior/result is not affected.

HadoopMapReduceCommitProtocol is somehow a custom committer in Spark. Actually we use it with dynamicPartitionOverwrite in this way, at least for data source.

IIUC HadoopMapReduceCommitProtocol is a wrapper of the hadoop MR committer. The code you refer to that proves the inefficient is all from hadoop MR committer, right?

yes, that's right.

Test build #112496 has started for PR 26159 at commit 6593e79.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17461/
Test PASSed.

Test build #112496 has finished for PR 26159 at commit 6593e79.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112496/
Test PASSed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17726/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112781/
Test FAILed.

retest this please

Test build #112795 has started for PR 26159 at commit 6593e79.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17739/
Test PASSed.

This issue causes real problem in our production jobs. Whether to come out a home-made custom committer for this issue, is another story. IMHO, it seems an overkill, because only dynamic partition overwrite is a problem.
I added a config to control it and it is false by default. So it should be conservative and safe.
also cc @HyukjinKwon @dongjoon-hyun for some review and options.

Test build #112795 has finished for PR 26159 at commit 6593e79.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112795/
Test PASSed.

Enabling dynamic partition overwrite can cause issues like SPARK-27194. Probably better to fix that bug first.

@rezasafi thanks for referring that. Is it specified to enabling dynamicPartitionOverwrite? As the task id was fixed in setupJob, I think this is also happened without this patch?

@viirya  Hi,  for the issue mentioned by @rezasafi , I have created a ticket to fix it.
Could you help take a look? Thanks in advance!

@rezasafi thanks for referring that. Is it specified to enabling dynamicPartitionOverwrite? As the task id was fixed in setupJob, I think it is also happened without this patch?

Hi @viirya, yeah that issue happens when dynamicPartitionOverwrite is enabled. It seems that after your fix is merged if people there would be more cases that hit SPARK-27194. So was just wondering whether it is better to fix that first before suggesting to use it in more scenarios.
I haven't looked at @turboFei fix.

@rezasafi Do you have idea why it only happens when dynamicPartitionOverwrite is enabled?

I think the jira and also the discussion on the first pr that was created for it is describing the issue in more detail. Basically the current dynamic partition overwrite implementation didn't consider the case that a task may fail during the commit. If a container fail, the task that was running there will fail but its written data won't be cleaned up and the rerun of the task will hit the FileAlreadyExistsException. The discussions under PR#24142 also debated  a proposed solution because of the concern of duplicate data.

@rezasafi yea I know that PR and read some discussion there. My question is this seems not specified to dynamic partition overwrite case. I did not see an analysis about it is specified for dynamic partition overwrite case, or I miss it? The problem happens because the filename to write was produced by same params between tasks retries, so the same path was written and FileAlreadyExistsException was thrown. Even dynamic partition overwrite is disabled, I did not see there is difference in determining the path to write. So wondering if it is specified to dynamic partition overwrite or not.

We experienced that issue only when dynamic partition overwrite was set.  I don't know whether it is possible to hit this issue without dynamic partition overwrite.

@rezasafi  @viirya  For the issue you mentioned.
The reason is that the filename is only escaped with taskId and without attemptId.
We also met it when spark speculation is enabled, for that a task's filename is conflicted with its speculation task.
https://github.com/apache/spark/blob/077fb99a26a9e92104503fade25c0a095fec5e5d/core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala#L108-L118
As shown in the code, I think the risk is same when the output committer is non-FileOutputCommitter, but I have not met this case and I think it is rare.
I have read the comments under PR #24142, and it seems that there is a risk for non-FileOutputCommitter case, if a task aborted and failed to clean up its output, result would be duplicated.
I created a PR #26086, which fix the issue for dynamic partition overwrite only.

@cloud-fan Would like to hear if you have other comments? If you still have concern on this, I will close this.


GenerateReleaseNotes will have no effect in RepositoriesService.EditRelease

According to the GitHub v3 API reference, generate_release_notes is not a recognised body parameter of the endpoint to update a release. In this library, the value of
RepositoryRelease.GenerateReleaseNotes is passed to the PATCH request to the GitHub API regardless, where it will have no effect.
From what I can tell, RepositoryRelease.GenerateReleaseNotes will only have an effect when calling RepositoriesService.CreateRelease, whereas the comments imply that it has an effect everywhere.
I recognise this field can't have an effect until there is a change in the GitHub API; I think what I would prefer in the meantime would be an update to the code/documentation in this library to make it clear that RepositoriesService.UpdateRelease will not cause release note generation, as this was rather confusing to me when trying to consume this function in a downstream library.

Thank you, @coro . Would you like to prepare the PR?

Will do, Glenn, thank you!


Updating Readme

It looks like you have not pulled down the changes to the main branch. See how you have no "dot" files and you have 5000+ files changes (because you dont have the updated .gitignore etc.

Please pull changes from main branch
Please update your branch to lv-README-addName


add cloudwatch logs arn to outputs

Add cloudwatch logs arn to module outputs

@AngadiS do you know when a new release will be made available?

@AngadiS do you know when a new release will be made available?

@szesch release v4.0.1 is made available.