Question: shall we update mpif90/mpirun to mpifort/mpiexec?

Avg response time

![Issue Stats][issue stat img]

Please replace [ ] with [X] to indicate you have taken the requested action

[X] I have surrounded any code, or log output in codeblock fences
(```) if it occupies more than one line, or in single
backticks if it is short short code

Question:

The MPI 3.1 standard mentions that many implementations provide mpirun but recommends providing mpiexec.
MPI 3.1 mentions mpif90 in a similar manner but is silent on mpifort.  However, comments online indicate a preference for the latter to liberate the name of the command from a specific revision of the Fortran standard.

I don't know of any implementations that don't provide mpirun or mpif90, but it seems reasonable to switch to mpifort and mpiexec for standard-conformance and clarity where the former two are employed.  On the master branch, I find the following locations:

[x] windows-install.sh,
[x] at several places in find_or_install.sh, and
[x] at two places in report_results.sh.

Motivation: I'm introducing caf and cafrun early the new book so this is a good time to update these commands to versions that are more standard or modern.

doing this with #268

@zbeekman given the simplicity of this issue, shall we simply implement a quick fix rather than tie it to #268, of which this is presumably a tiny component?

sure, but it's not quite as trivial as it seems, because it pops up a few
places... what I should do is just cherry pick the commit off my work and
merge that.. I'll do that now
On Tue, Jul 18, 2017 at 1:07 AM Damian Rouson<EMAIL_ADDRESS>wrote:

@zbeekman https://github.com/zbeekman given the simplicity of this
issue, shall we simply implement a quick fix rather than tie it to #268
https://github.com/sourceryinstitute/OpenCoarrays/issues/268, of which
this is presumably a tiny component?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourceryinstitute/OpenCoarrays/issues/378#issuecomment-315959325,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAREPFZnd_3wmc0fcQzMN22n_9nRV1elks5sPD18gaJpZM4NUldx
.


Update zio, zio-test, zio-test-sbt to 2.0.10

Updates

dev.zio:zio
dev.zio:zio-test
dev.zio:zio-test-sbt

from 2.0.0-RC1 to 2.0.10.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "dev.zio" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "dev.zio" }
}]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by #216.


Villager placement in bed - client crash - rare world gen oddity

Crash type: Multiplayer. Fatal error to Client resulting in immediate disconnect. Server is fine.
Game version: 1.17.2 and 1.17.3
VS Village version: vsvillage_v0.2.4_vsquest_v0.0.+.zip
Loaded Mods<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Repeatable: Not easy; highly specific circumstance involving ruins added to world generation by another mod. Consistent once its happened.
What happened?:
A large ruin from Better Ruins spawned overlapping the center of a village. This led to blocks taking up space above a bed. A villager either spawned in that bed, or attempted to use it, cannot be properly placed, and caused an unavoidable client crash.
Coming within loading distance of this village with VSvillage enabled results in client crash, always. Only resolution was to temporarily disable VSvillage to allow players to leave the area. Crash continues to happen after all blocks of both ruin and village have been removed, including the bed involved. Eventually forced the area to regenerate without the mod, before adding VSvillage back to the server again.
Error log:
Game Version: v1.17.2 (Stable)
9/11/2022 6:05:35 AM: Critical error occurred
Loaded Mods<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>System.NullReferenceException: Object reference not set to an instance of an object.
at Vintagestory.GameContent.BlockEntityBed.get_MountPosition() in VSSurvivalMod\BlockEntity\BEBed.cs:line 35
at Vintagestory.GameContent.EntityBehaviorControlledPhysics.TickEntityPhysics(EntityPos pos, EntityControls controls, Single dt) in VSEssentials\Entity\Behavior\BehaviorControlledPhysics.cs:line 227
at Vintagestory.GameContent.EntityBehaviorControlledPhysics.TickEntityPhysicsPre(Entity entity, Single dt) in VSEssentials\Entity\Behavior\BehaviorControlledPhysics.cs:line 156
at Vintagestory.GameContent.EntityBehaviorControlledPhysics.onPhysicsTick(Single deltaTime) in VSEssentials\Entity\Behavior\BehaviorControlledPhysics.cs:line 122
at Vintagestory.Client.NoObf.ClientEventManager.TriggerRenderStage(EnumRenderStage stage, Single dt) in VintagestoryLib\Client\Util\ClientEventManager.cs:line 201
at Vintagestory.Client.NoObf.ClientMain.TriggerRenderStage(EnumRenderStage stage, Single dt) in VintagestoryLib\Client\ClientMain.cs:line 1174
at Vintagestory.Client.NoObf.ClientMain.MainRenderLoop(Single dt) in VintagestoryLib\Client\ClientMain.cs:line 1216
at Vintagestory.Client.NoObf.ClientMain.MainGameLoop(Single deltaTime) in VintagestoryLib\Client\ClientMain.cs:line 1103
at Vintagestory.Client.GuiScreenRunningGame.RenderToPrimary(Single dt) in VintagestoryLib\Client\MainMenu\Screens\GuiScreenRunningGame.cs:line 123
at Vintagestory.Client.ScreenManager.Render(Single dt) in VintagestoryLib\Client\ScreenManager.cs:line 669
at Vintagestory.Client.ScreenManager.OnNewFrame(Single dt) in VintagestoryLib\Client\ScreenManager.cs:line 613
at Vintagestory.Client.NoObf.ClientPlatformWindows.window_RenderFrame(Object sender, FrameEventArgs e) in VintagestoryLib\Client\ClientPlatform\GameWindow.cs:line 125
at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
at OpenTK.GameWindow.RaiseRenderFrame(Double elapsed, Double& timestamp) in C:\Users\Nexrem\Desktop\transfer\opentk\src\OpenTK\GameWindow.cs:line 476
at OpenTK.GameWindow.DispatchRenderFrame() in C:\Users\Nexrem\Desktop\transfer\opentk\src\OpenTK\GameWindow.cs:line 452
at OpenTK.GameWindow.Run(Double updates_per_second, Double frames_per_second) in C:\Users\Nexrem\Desktop\transfer\opentk\src\OpenTK\GameWindow.cs:line 375
at Vintagestory.Client.ClientProgram.Start(ClientProgramArgs args, String[] rawArgs)
at Vintagestory.ClientNative.CrashReporter.Start(ThreadStart start) in VintagestoryLib\Client\ClientPlatform\ClientNative\CrashReporter.cs:line 93

https://github.com/AreYSerious/betterruins/issues/9#issuecomment-1243453102


[AAP-23373] Topology mesh tech debt test

Issue: AAP-23373

@akus062381 I forgot to unskip it again :facepalm:, but that's actually good idea to move it to component tests

@akus062381 https://github.com/ansible/ansible-ui/blob/main/frontend/awx/administration/topology/Visualizer.cy.tsx already has very similar tests, so maybe we should keep these e2e tests in topology-view.cy.ts and just leave them unskipped? :D

@akus062381 https://github.com/ansible/ansible-ui/blob/main/frontend/awx/administration/topology/Visualizer.cy.tsx already has very similar tests, so maybe we should keep these e2e tests in topology-view.cy.ts and just leave them unskipped? :D

@jerabekjiri if that's the case, maybe we can delete the tests from this file altogether?

Test failures unrelated to this PR


Add autoprefixer

Not fully tested yet

Not sure if you want it tested yet, but looks good to me!


Please define your licence

Please choose an opensource license:
http://choosealicense.com/

and add the license name in package.json file please.


Increment version to 75, v2.2.0

Google Play release notes:
User Profiles and bug fixes
Dev release notes:

App now shows in recent apps

Config version:
edx/edx-mobile-config@e2a198c990987cb86382dfdc35f8600f12fe0378
Schedule:

@aleffert

If the rc is 2.2.0 then master should be 2.3.0. That way it's easy to distinguish the lines.

Current coverage is 16.11%

Merging #633 into master will increase coverage by +0.07% as of 78b0019

@@            master    #633   diff @@
======================================
  Files          458     458
  Stmts        19611   19611
  Branches      2850    2850
  Methods          0       0
======================================
+ Hit           3146    3161    +15
- Partial        375     378     +3
+ Missed       16090   16072    -18

Review entire Coverage Diff as of 78b0019

Uncovered Suggestions

+0.16% via ...gScaleImageView.java#436...465
+0.09% via ...gScaleImageView.java#2665...2681
+0.07% via ...gScaleImageView.java#2571...2584
See 7 more...

Powered by Codecov. Updated on successful CI builds.


Unable to add games from subdirectories

I don't see the option for this so it may be a new feature request. It would be great to add games in subdirectories.

Check out the latest release! You can now use nested folders in platforms.

No toggle, it's on by default. It should look something like this if your folder structure is:
GC

Game 1 Folder

Game 1

Game 2 Folder

Game 2

For the screenshot mine is
SNES

Bunch of games
Mario games

Super Mario World

More mario games

Hmm, the only one that I can get working is psx. Are there any logs or something that I could send you?

No there aren't logs for it. Are the nested games detected if they're not nested?

Yes, they work if they are at the root.

Ok I will look into this and get back to you. Thanks for the report!

Oh! I think I found a solution. Looks like there needs to be one valid game in the root and then the rest will populate.

For GC I put a dummy.rvz file in the root and now the games are syncing.

Can this get looked at again so a valid file/dummy isn't needed in the root for the subdirectories to get looked at?


proxy lib

hi,
in readme.md it says: 'for proxy you have to use annother lib.' Is the lib 'http-proxy' a good choice for it ? What can you recommend ? :-)
greets,
david

Never tested any to be honest. If http-proxy works, then use it.


additionally display sap_flux data

For TESS data, the default is to display pdcsap_flux.  Is there an option to also display the sap_flux alongside, as a B dataset?
See https://masttest.stsci.edu/viz/ui/#/lcviz?uri=mast%3ATESS%2Fproduct%2Ftess2023315124025-s0072-0000000842468675-0267-a_fast-lc.fits.
Is there a UI option to display additional lightcurve data from within the same file?  Or is there a programmatic way to do it?  There was some feedback on LCViz in MAST to have this functionality.

There will be in the next release - see #77 for details and let me know if you have any comments/feedback!

This looks great.  This should satisfy our criteria for the most part.  Looks like you already have a potential follow-up for over-plotting in the same viewer.  I think that would be useful to have.

Do you know about when the next release will be?

Support for choosing a single column is now released in lcviz 0.2.0.  I'll leave this issue open for now until we either support multiple columns or decide to abandon that idea.


The Hitchhikers Guide To The Galaxy

When user presses keys '4' and '2', a "Dont't Panic" image fades in and, after some time, fades out. This is a reference to the brilliant novel "The Hitchhiker's Guide To The Galaxy" by Douglas Adams.

haha great 📖


Print Map Fuctionality Implemented using Leaflet Open-Source-library

Here is the working Model of Print Functionality.

Thank You from @GisOpenSourceOrg

Thank You @HameedMulani for your Contributions @GisOpenSourceOrg

Welcome @GisOpenSourceOrg Team 👍🏻
Excited for New Projects and Issues


Dispatcher to Sidecar environment vars

The dispatcher will need to inject the following values as ENV vars to the sidecar.

METADATA_ACCESS_TOKEN
BLOB_ACCESS_TOKEN
JOB_CORRELATION_ID
LOCAL_SECRET
TOPIC_NAMES
SERVICE_BUS_URI

Also going to need:

Parent_Metadata_ID
Current_Event_ID

Done


GraphExecutor: Fix wild pointer assign when input and output are reshape

This PR is an updated version of #15121

cc @masahi @corehalt


Fix edge and edgedriver version mismatch on CI

Noticed in #2394

@jessfraz this is 100% related https://github.com/actions/runner-images/issues/9538, we're not alone!

Ok the workaround provided at https://github.com/actions/runner-images/issues/9538 works to get Webview2 to update
Invoke-WebRequest -Uri 'https://go.microsoft.com/fwlink/p/?LinkId=2124703' -OutFile 'setup.exe'
Start-Process -FilePath setup.exe -Verb RunAs -Wait


Traefik not closing tcp connections, server getting unresponsive after a bit of time

Do you want to request a feature or report a bug?
Bug
Did you try using a 1.7.x configuration for the version 2.0?

[ ] Yes
[x] No

What did you do?

I deployed my (rather simple) go application using docker and docker-compose with some monitoring (using prometheus, not related to traefik) and traefik (also in docker) by pretty much following this guide. All in all not much, 7 containers in total (with traefik). I used the cheapest Hetzner cloud vps which however should have more than enough ressources for my use case.
I can provide the compose files if needed.
What did you expect to see?
Application working normally, no issues.
What did you see instead?
After a while, the server gets totally unresponsive. Not even ssh did work anymore. When I restarted it via Hetzners web console, it started working again, but after a few days I had the same issue again. After some digging, I found this to be an error with "too many orphaned tcp connections" and thought my application was to blame, found an issue with that, fixed it, redeployed it and thought it was all good. But then, the same problem started appearing again. Being a bit short on time, I just raised all limits which got it somehow working again.
Then I finally did some more looking for the error and found this:
~# ss -sa
Total: 99822 (kernel 99836)
TCP:   99753 (estab 1, closed 99748, orphaned 1, synrecv 0, timewait 225/0), ports 0

Transport Total     IP        IPv6
*	  99836     -         -
RAW	  0         0         0
UDP	  1         1         0
TCP	  5         2         3
INET	  6         3         3
FRAG	  0         0         0

Notice the 99748 closed tcp connections which don't show up in the further stats. These didn't show up in netstat either, which I thought is a bit weird. Then I started shutting down parts of my application, one after another, then the monitoring, checking each time if the number of closed connections had gone down (spoiler: it didn't much) until there was only traefik left. When I shut it down, I noticed this:
/opt/compose/traefik# docker-compose down
Stopping traefik ... done
Removing traefik ... done
Network web is external, skipping
/opt/compose/traefik# ss -sa
Total: 127 (kernel 156)
TCP:   3904 (estab 1, closed 3901, orphaned 3667, synrecv 0, timewait 234/0), ports 0

Transport Total     IP        IPv6
*	  156       -         -
RAW	  0         0         0
UDP	  1         1         0
TCP	  3         2         1
INET	  4         3         1
FRAG	  0         0         0

The number of closed tcp connections got down pretty quickly, even dropping down to ~30 after a few minutes. Even after I started bringing up my application again. So it looks like traefik is the issue here.
I'm not quite sure if this relates to #3165 since the connections didn't show up in netstat at all, also not sure about #4864 since my application isn't using any websockets.
Output of traefik version: (What version of Traefik are you using?)

In Docker:
Version:      v1.5.4
Codename:     cancoillotte
Go version:   go1.9.4
Built:        2018-03-15_01:35:21PM
OS/Arch:      linux/amd64

(Oh wow... I don't know why I'm using such an old version of traefik... Maybe my bug already got fixed...)
I'll update my version of traefik and see if that fixes it and update here.
What is your environment & configuration (arguments, toml, provider, platform, ...)?
debug = false

logLevel = "ERROR"
defaultEntryPoints = ["https","http"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]

[retry]

[docker]
endpoint = "unix:///var/run/docker.sock"
#domain = "my-awesome-app.org"
watch = true
exposedByDefault = false

[acme]
email =<EMAIL_ADDRESS>storage = "acme.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"

Running with this docker-compose.yml:
version: '2'

services:
  traefik:
    image: traefik:1.5.4
    restart: always
    ports:
      - 80:80
      - 443:443
    networks:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.toml:/traefik.toml
      - ./acme.json:/acme.json
    container_name: traefik

networks:
  web:
    external: true

Hi @kolaente ,
Could you try with the latest stable version of Traefik  (aka. 1.7.12 at the date of this comment),
instead of 1.5.4 ?
Reason is that a lot of changes and fixes happened between these 2 versions :)

@dduportal yes, I've updated yesterday right after submitting this issue, and today the number of closed connections is hovering around 1000. Still a lot, but already a lot better since before, so it looks like this already fixed something. I'll keep a look on it over the next week and update here.

I've opened a pr to prevent others from using an outdated traefik version like me: #4927

So, after running traefik with the updated version for about 4 days without any issues, the update seems to have worked, I'll close this issue.


Give the Nethrite shulker more room

Currently netherite shulker boxes don't have much of a reason to exist since my shulker boxes dont often go in lava and if I have a netherite shulker box i will definetly have netherite armor so dying and your stuff falling into lava never happens i dont see explosion proof to be very useful either (Although both of them are good side additions) I think it would make sense to give netherite shulker boxes 1 or 2 (maybe even 3) new slots to really get me to want the netherite shulker boxes otherwise i would just be sticking with normal shulkers but more room really gives me a good reason to want them.

Id say to add 1 more row of slots would be good enough and not too much

Id say to add 1 more row of slots would be good enough and not too much

Yea that sounds good

Reinforced Shulker Boxes gives you larger shulker boxes, and the diamond and netherite boxes are 4x the size of a regular box.


Docs section: Network & Connectivity - Target information

This page has a lot of information on it that isn't specific to AVD as a "construction set" upon an ESLZ framework.
The CAF networking decision tree & common networking scenarios etc. Shouldn't live in this doc and are covered in the ESLZ networking sections for what is relevant for "corp" connected workloads which is effectively what WVD is.
This page seems more like a general CAF doc for networking which we already have covered in ESLZ and elsewhere.
This should be prescriptive around the networking considerations for WVD deployments on and ESLZ framework.
Make this doc specific to the AVD construction set networking design covering the below:

VNET design
Subnets
Micro segmentation
Peering back to hub or VWAN as per ESLZ
DNS configuration
For AD DS Join
Private Link/Endpoint
NSGs
Including suggested rules
Private Endpoints
Creation and placement
DNS Zones back to central ones in ESLZ
WVD required URLs
How to handle with and without Azure Firewall
Azure Files
Private Endpoint
SMB Multi-Channel
Considerations before enabling
(breaks NTFS GUI & have to use icacls)
How to handle internet breakout
Connectivity back to on-prem
Linking back to ESLZ network docs and just providing context as to how ESLZ sits on top of ESLZ and therefore uses the hubs

I would also add AzFW and its suggested configuration for AVD.

Changes to existing documentation: we need to overhaul the current documentation as it mostly talks of CAF and ALZ but not a lot of specifics on AVD, we will use some of the existing info in the document but in general we need to align with what other teams (like AVS Enterprise-scale network topology and connectivity for Azure VMware Solution - Cloud Adoption Framework | Microsoft Docs)  have delivered in their accelerator docs.
The newly proposed structure for the doc will include customer scenarios section (customer profile, architectural components with diagram, considerations):

Quick intro: brief info on ALZ connectivity, AVD’s networking and mentions hybrid, performance, security, and scale.
Networking components and concepts: quick description of networking concepts and components that interact with the AVD deployment: NSG, ASG, Azure Firewall, Azure virtual network, NVA, vWAN, hub&spoke, UDR, AVD traffic flow, private link, azure files networking, ANF networking.

Networking Scenarios:

1: Hub&Spoke
2: vWAN
3: Traffic inspection: NVA, secure hub, azure firewall (forced tunneling).

General design considerations and recommendations:

Hub-Spoke vs vWAN
AVD Ports and protocols requirements
AVD Bandwidth and latency
AVD Internet connectivity (Outbound)
Zero trust on AVD networking Security
Plan for IP addressing
DNS
Native vs Short path

Network and topology doc created and submitted including diagrams for scenario 1.
PR submitted: https://github.com/MicrosoftDocs/cloud-adoption-framework-pr/pull/2988

Pending:

Scenarios
-- Zero-trust: NSG/ASG for micro-segmenting east/west and forced NVA/AzFW in the hub for north/south
-- RDP Shortpath
-- NVA/AzFW in the hub to control north/south specially internet outbound
-- Cross-region example

Subjects:
-- Network policy driven governance
-- Monitoring
-- Plan for IP addressing
-- Internet inbound/outbound filtering options
-- Micro segmentation
-- Network security

Moving this issue to complete.
New issue created for networking docs and scenarios updates (#71)


InputEvent as_text now returns readable string. Added to_string for debug strings

InputEvent changed as_text() to return a display-friendly and human readable string, to be used for UI display. to_string() can be used to get a non-display friendly version which has more information for debugging. Before, some as_text() methods returned a nice readable string, and some displayed more debug-style information.
Based on Calinou comment https://github.com/godotengine/godot/issues/38326#issuecomment-621842765
This is the first commit of a larger series based on #42600.

This will likely conflict with #43591 (or the other way around, depending on which is merged first).

@madmiraal ah yep in the original pr of #42600  I completely remove the input map editor, as I make it generic (action_map_editor.cpp), so in that pr this issue does not exist. Here it seems I will have to edit the input map editor to use as_text rather than constructing its own strings everywhere.
Seeing input map editor construct the representation of the events on its own is what made me want to do this change anyway, since the text representation of an event should be consistent across the engine.

@madmiraal @akien-mga Ok, finished the above reviews up. Also added some text to the descriptions as per https://github.com/godotengine/godot/pull/43591/files#r52860139, and also rebased with newest master, which includes #43591

All sorted now, thanks!

Thanks!

Hi, I just got the issue of listening to gamepad axis showing the long readable string, but not the one needed to add an input mapping, in v4.0.beta.custom_build [767f8fb48]

I'm looking at the code in editor/input_event_configuration_dialog.cpp and can clearly see the quickfix under

// Joypad motion events will display slightly differently than what the event->as_text() provides. See #43660.

but it doesn't seem to be applied.
I couldn't make debugging work in my VS Code so I don't know why the quickfix doesn't work exactly (I'll try to rebuild the whole project with llvm but it will take time, and I have a few things to finish on my project first).
Should I open a new bug report, or let you handle this in this issue?

This PR has been merged 2 years ago. Open a new issue.


Make the rebase fail when a lifecyclehook fails

Context: https://redhat-internal.slack.com/archives/GE2HQ9QP4/p1729009589047099?thread_ts=1728898665.791679&cid=GE2HQ9QP4
For Example this job passed, but the actual lifecycle hook failed.
make: *** [Makefile:25: ocp-manifests] Error 1
Script failed with error:
Command '/working/generate-manifests.sh' returned non-zero exit status 2.
INFO - Running LifecycleHook.POST_REBASE lifecycle hook _BUILTIN_/update_go_modules.sh
Performing go modules update

cc. @RadekManak

Fixed by https://github.com/openshift-eng/rebasebot/pull/57


Help understand inconsistency in altitudes

I've been analyzing some logs and I always see this "problem".

The various altitude measures have some inconsistency IMO.
For example the local_position.z (PINK) is not at zero when arming (on ground), but it has an offset of around 8 meters. You can see the real altitude from ground in black from the distance_sensor.
Also the vehicle_global_position.alt (BLUE) seems to have the same offset w.r.t. baro data and vehicle_gps_position.alt. The first is at around 57 meters, while the latters at around 49 meters.
Another weird thing is that the "relative alt" I see on QGC matches almost perfectly the distance_sensor, so my questions are:

Isn't the "relative alt" in QGC taken from local_position.z? If not which data is sent for it?
Is it normal that the local_position.z doesn't reset to 0 when arming?
Why those offsets? Is it just the baro drifting?

The overall problem of all this, is that during AUTO modes, the altitude references are always wrong and the copter never reaches the desired waypoint because the reference and the altitude used to check the acceptance radius differ of that offset.

Isn't the "relative alt" in QGC taken from local_position.z? If not which data is sent for it?

It uses the relative altitude to the home position.

Is it normal that the local_position.z doesn't reset to 0 when arming?

local_position is initialized when the EKF initializes but not reset on arming.

Why those offsets? Is it just the baro drifting?

Not sure.

... the copter never reaches the desired waypoint because the reference and the altitude used to check the acceptance radius differ of that offset.

The waypoints should still be reached. Are you sure about this? And can you reproduce it in SITL? Or do you have a logfile to show it?

I'm closing this because there was not follow up. Re-open if you have answers.


Not utilizing gpu during image generation

hi, thank you for providing this code.
i am currently running the model schnell q2 in kaggle notebook but when it start generating the image it always shows 'using cpu backend' and it does not utilize the gpu at all. pls help
input cli:
!/kaggle/working/stable-diffusion.cpp/build/bin/sd
--diffusion-model /kaggle/working/flux1-schnell-q2_k.gguf
--clip_l /kaggle/working/clip_l.safetensors
--t5xxl /kaggle/working/t5xxl_fp8_e4m3fn.safetensors
--vae /kaggle/working/ae.safetensors
-p "A male model standing confidently against a clean white background, wearing a fitted blue t-shirt and stylish black jeans. The model has a friendly smile, with short dark hair, and is posing casually with one hand in his pocket. The lighting is bright and even, highlighting the clothing details and creating a professional e-commerce look."
--cfg-scale 1.0
--sampling-method euler
--rng cuda
--steps 2
-v
verbose:
[DEBUG] stable-diffusion.cpp:180  - Using CPU backend
[INFO ] stable-diffusion.cpp:202  - loading clip_l from '/kaggle/working/clip_l.safetensors'
[INFO ] model.cpp:793  - load /kaggle/working/clip_l.safetensors using safetensors format
[DEBUG] model.cpp:861  - init from '/kaggle/working/clip_l.safetensors'
[INFO ] stable-diffusion.cpp:209  - loading t5xxl from '/kaggle/working/t5xxl_fp8_e4m3fn.safetensors'
[INFO ] model.cpp:793  - load /kaggle/working/t5xxl_fp8_e4m3fn.safetensors using safetensors format
[DEBUG] model.cpp:861  - init from '/kaggle/working/t5xxl_fp8_e4m3fn.safetensors'
[INFO ] stable-diffusion.cpp:216  - loading diffusion model from '/kaggle/working/flux1-schnell-q2_k.gguf'
[INFO ] model.cpp:790  - load /kaggle/working/flux1-schnell-q2_k.gguf using gguf format
[DEBUG] model.cpp:807  - init from '/kaggle/working/flux1-schnell-q2_k.gguf'
WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc!
[INFO ] stable-diffusion.cpp:223  - loading vae from '/kaggle/working/ae.safetensors'
[INFO ] model.cpp:793  - load /kaggle/working/ae.safetensors using safetensors format
[DEBUG] model.cpp:861  - init from '/kaggle/working/ae.safetensors'
[INFO ] stable-diffusion.cpp:235  - Version: Flux Schnell
[INFO ] stable-diffusion.cpp:266  - Weight type:                 f16
[INFO ] stable-diffusion.cpp:267  - Conditioner weight type:     f16
[INFO ] stable-diffusion.cpp:268  - Diffusion model weight type: q2_K
[INFO ] stable-diffusion.cpp:269  - VAE weight type:             f32
[DEBUG] stable-diffusion.cpp:271  - ggml tensor size = 400 bytes
[DEBUG] clip.hpp:171  - vocab size: 49408
[DEBUG] clip.hpp:182  -  trigger word img already in vocab
[DEBUG] ggml_extend.hpp:1046 - clip params backend buffer size =  235.06 MB(RAM) (196 tensors)
[DEBUG] ggml_extend.hpp:1046 - t5 params backend buffer size =  9083.77 MB(RAM) (219 tensors)
[DEBUG] ggml_extend.hpp:1046 - flux params backend buffer size =  3732.51 MB(RAM) (776 tensors)
[DEBUG] ggml_extend.hpp:1046 - vae params backend buffer size =  94.57 MB(RAM) (138 tensors)
[DEBUG] stable-diffusion.cpp:398  - loading weights
[DEBUG] model.cpp:1530 - loading tensors from /kaggle/working/clip_l.safetensors
[DEBUG] model.cpp:1530 - loading tensors from /kaggle/working/t5xxl_fp8_e4m3fn.safetensors
[INFO ] model.cpp:1685 - unknown tensor 'text_encoders.t5xxl.encoder.embed_tokens.weight | f8_e4m3 | 2 [4096, 32128, 1, 1, 1]' in model file
[DEBUG] model.cpp:1530 - loading tensors from /kaggle/working/flux1-schnell-q2_k.gguf
[DEBUG] model.cpp:1530 - loading tensors from /kaggle/working/ae.safetensors
[INFO ] stable-diffusion.cpp:482  - total params memory size = 13145.92MB (VRAM 0.00MB, RAM 13145.92MB): clip 9318.83MB(RAM), unet 3732.51MB(RAM), vae 94.57MB(RAM), controlnet 0.00MB(VRAM), pmid 0.00MB(RAM)
[INFO ] stable-diffusion.cpp:501  - loading model from '' completed, taking 53.47s
[INFO ] stable-diffusion.cpp:518  - running in Flux FLOW mode
[DEBUG] stable-diffusion.cpp:572  - finished loaded file
[DEBUG] stable-diffusion.cpp:1378 - txt2img 512x512
[DEBUG] stable-diffusion.cpp:1127 - prompt after extract and remove lora: "a lovely cat holding a sign says 'flux.cpp'"
[INFO ] stable-diffusion.cpp:655  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s
[DEBUG] conditioner.hpp:1036 - parse 'a lovely cat holding a sign says 'flux.cpp'' to [['a lovely cat holding a sign says 'flux.cpp'', 1], ]
[DEBUG] clip.hpp:311  - token length: 77
[DEBUG] t5.hpp:397  - token length: 256
[DEBUG] ggml_extend.hpp:998  - t5 compute buffer size: 68.25 MB(RAM)
[DEBUG] conditioner.hpp:1155 - computing condition graph completed, taking 39122 ms
[INFO ] stable-diffusion.cpp:1256 - get_learned_condition completed, taking 39127 ms
[INFO ] stable-diffusion.cpp:1279 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed 42
[DEBUG] ggml_extend.hpp:998  - flux compute buffer size: 397.27 MB(RAM)
|==================================================| 2/2 - 244.56s/it
[INFO ] stable-diffusion.cpp:1315 - sampling completed, taking 490.85s
[INFO ] stable-diffusion.cpp:1323 - generating 1 latent images completed, taking 491.69s
[INFO ] stable-diffusion.cpp:1326 - decoding 1 latents
[DEBUG] ggml_extend.hpp:998  - vae compute buffer size: 1664.00 MB(RAM)
[DEBUG] stable-diffusion.cpp:987  - computing vae [mode: DECODE] graph completed, taking 49.77s
[INFO ] stable-diffusion.cpp:1336 - latent 1 decoded, taking 49.77s
[INFO ] stable-diffusion.cpp:1340 - decode_first_stage completed, taking 49.77s
[INFO ] stable-diffusion.cpp:1449 - txt2img completed in 580.60s
save result image to 'output.png'

Did you compile with the appropriate settings for your GPU?

!cmake .. -DSD_CUBLAS=ON
!cmake --build . --config Release
is this necessary to enable gpu utilization @grauho?

Yes, if you're trying to use it with a CUDA enabled graphics card you do want to build it with:
cmake .. -DSD_CUBLAS=ON
cmake --build . --config Release
as well as making sure you have the rest of the CUDA tool chain set up like it says in the README.

@grauho thanks for the help. By CUDA tool chain set you mean CUDA toolkit?
as im using kaggle notebook CUDA toolkit is properly installed in it.

@grauho thanks for the help. By CUDA tool chain set you mean CUDA toolkit? as im using kaggle notebook CUDA toolkit is properly installed in it.

No problem. Yep the CUDA toolkit, and provided it builds without error give it a shot and see if it recognizes the GPU properly.

@grauho i have tried cmake .. -DSD_CUBLAS=ON cmake --build . --config Release and it gives error /home/wiredhikari/flux-api/stable-diffusion.cpp/model.cpp:705:0: required from here /usr/include/c++/13/bits/stl_tree.h:2131:14: internal compiler error: Segmentation fault 2131 | return _Res(__j._M_node, 0); | ^~~~~~~~~~~~~~~~~~~~ 0x75fe5424531f ??? ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 0x75fe5422a1c9 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 0x75fe5422a28a __libc_start_main_impl ../csu/libc-start.c:360 Please submit a full bug report, with preprocessed source (by using -freport-bug). Please include the complete backtrace with any bug report. See <file:///usr/share/doc/gcc-13/README.Bugs> for instructions. gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:76: CMakeFiles/stable-diffusion.dir/model.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:167: CMakeFiles/stable-diffusion.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 the model is not able to use gpu how can i fix this?

That looks like a different issue, does it build normally without SD_CUBLAS enabled?

@grauho i have tried cmake .. -DSD_CUBLAS=ON cmake --build . --config Release and it gives error /home/wiredhikari/flux-api/stable-diffusion.cpp/model.cpp:705:0: required from here /usr/include/c++/13/bits/stl_tree.h:2131:14: internal compiler error: Segmentation fault 2131 | return _Res(__j._M_node, 0); | ^~~~~~~~~~~~~~~~~~~~ 0x75fe5424531f ??? ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 0x75fe5422a1c9 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 0x75fe5422a28a __libc_start_main_impl ../csu/libc-start.c:360 Please submit a full bug report, with preprocessed source (by using -freport-bug). Please include the complete backtrace with any bug report. See <file:///usr/share/doc/gcc-13/README.Bugs> for instructions. gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:76: CMakeFiles/stable-diffusion.dir/model.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:167: CMakeFiles/stable-diffusion.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 the model is not able to use gpu how can i fix this?

That looks like a different issue, does it build normally without SD_CUBLAS enabled?

Yeah it was build perfectly without SD_CUBLAS enabled but after that it is not able to build. And I am not able to run the model on my gpu.


(fix) Address issue with registry_key

Property needed to be passed along and is not attribute of class

Thanks!

Released as v2.34.9.


D455 2.42.0 Driver does not work on Win7

Required Info

Camera Model
D455

Firmware Version
Can't tell

Operating System & Version
Win 7 x64 Enterprise

Platform
PC

SDK Version
2.42.0

Issue Description
Just got a D455, and plugged it into a Win7 system that works fine with a D435.  It showed up under "Imaging Devices" in device manager, and I then updated the SDK to 2.42.0, making sure that the Win7 driver install option was checked.  I was prompted to OK the drier install, did so, and the device manager window updated a couple of times, but when done the D455 RGB and Depth were still listed under Imaging Devices, and the D455 is not recognized by anything.  The D435 still works fine.
I plugged the D45 into a Win10 system, updated to 2.42.0, and it works fine there.

Looked into this some more, and seems the driver installed with 2.42 is still 2.30 from 2019.  The .inf file for it does not have a listing for the D455, so it is not getting recognized.

Hi @mediadog   The IMU-equipped D435i is listed in the SDK release notes as not supported by Windows 7 yet.  So whilst I have not seen previous reports of issues with D455 in Windows 7, this may be related to the D455 also being an IMU-equipped model.

Hi @mediadog  Do you require further assistance with this case, please?   Thanks!

Well, is it fixed?  If the answer is "no", then my answer is "yes".
In fact, the D435i does appear in the Win7 driver .inf file.  But the D455 does not.  I hacked the .inf file to add the D455, and after doing a "have disk" update of the driver, the D455 did in fact get recognized and show up in device manager - but reaslsense-viewer does not recognize it.  So I suspect it is a Win7 librealsense issue as well as driver issue.
I was going to try hacking the .inf file so it would be reported as a D435, to see if the viewer would then recognize it, but have not had time.
Just curious, so when a bug is reported, is there any effort made to replicate it and schedule a fix?

There is no information available regarding when D435i support for Windows 7 might be supported.  If the Release notes state that support of a particular RealSense camera model for a particular platform is scheduled for a future release then usually that support is implemented but no time schedule estimate can be provided.
Confirmed bugs are taken seriously, yes.  A rapid solution is not possible in every case though.

Well this issue is concerning the D455, not the D435i so your comment is perhaps in the wrong thread.  You mentioned the D435i support as perhaps the D455 being related because of the IMU.  I mentioned the D435i is in the driver .inf file just as an aside.  And when I modified the .inf file for the D455, the driver did install for it without errors, but the viewer still did not recognize it, so it may be more than a driver issue, perhaps the Win7 library version not recognizing the model number (or just not looking for it), but I am only guessing (it works fine on Win10, but there it appears to be using a generic Windows driver).
And in any case, if it is an IMU support issue, in the cases where the D455 could be used to enhance performance in D435 applications (like mine), transitional support for the image streams would be a huge win as the D455 is a great improvement over the D435.  I am now not going to buy any more D435s unless I absolutely have to, delaying any purchases pending D455 support (and I have some very large installations on the horizon).
Where is the roadmap for targeted release features and device support?  Thanks.

Sorry, I meant D455.  This was in relation to my earlier comment in this discussion about how the reason for lack of D455 support on Windows 7 is likely for the same technical reasons that it is not supported on D435i when the D435 is.
https://github.com/IntelRealSense/librealsense/issues/8621#issuecomment-803265777
Intel do not publish roadmaps for camera features.  They are typically announced in the release notes at their time of introduction instead of schedule estimates being provided in advance.

I just looked at the v2.43.0 release page, and it still lists the D455 as supported, with the Win7 exception being just the D435i.  At least the release notes should show the D455 as not supported also.

I have passed your suggestion about the release notes to Intel.  Thanks very much!

Hi @mediadog   An official feature request has been filed with Intel for the addition of Windows 7 support of the D455.  This case should be kept open for whilst the request is being considered by the Intel RealSense team.  Thanks again!

Most excellent!  Athankew.

Hi @mediadog  In regard to the feature request for D455 support on Windows 7, could you provide a reason why you are working with Windows 7 instead of Windows 10 please?  Intel's Windows 7 installation documentation makes the statement that "Not all product features are enabled on Windows 7".
https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_win7.md

Hi @mediadog   Could you provide information about the question in the comment above, please?  Thanks!

Hi @mediadog  This case and the feature request will be closed if we do not receive a response to the earlier question in https://github.com/IntelRealSense/librealsense/issues/8621#issuecomment-860866222 after 7 days from the time of writing this.  Thanks very much.

So far trying to use Win10 for 24/7/.365 applications is problematic.  We have Win7 based installations using Kinect V2s with libfreenect2/OpenNI2 that have literally run constantly for over a year, only rebooting because of a power failure.  The amount of work required to try to get Win10 remotely stable enough for long term unattended use is considerable, and requires constant re-qualification.  We're talking public million-dollar-plus installations that can't have downtime because we missed plugging some Win10 "feature" or update.

Thanks very much @mediadog for the explanation. I will pass it back to Intel.

Closed due to ending of Windows 7 support from SDK 2.51.0 onwards.


Support Qt6/Linux via cmake

This PR corrects some cmake building issues that are present in Qt6 under Linux. This also updates the developer readme with some details.
Note that there's still a bit of guess work here with regards to the exact libraries we need. Ideally, we probably only need xcb and xcb-keysyms in addition to the normal Qt libraries. Likewise, I'm still working out a library set for fedora.
I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

Do we still need to the readme updates from this pr?

Bits of this would be helpful, yes. But it's probably a little out of date with Chris' work.


Ignore libraryID instead of throwing an error when keysope is global

@retorquere Is there any harm in removing this check/error? I've added better support for groups libraries in https://github.com/mgmeyers/obsidian-zotero-integration , but I did so by always passing along the library id to the rpc endpoints. This works for users where keyScope === 'library' but not keyScope === 'global'. It seems to make sense to me to just ignore the library ID when keyScope is global, but I might be missing something.
Alternatively, is there a way to check a user's keyScope setting via cayw/json-rpc? Maybe we could return it along with the output of user.groups? That could be another solution to this issue.

A new build that includes this PR will drop on #2392 in about 10 minutes

Can confirm build above works for my use case


How to use with unRAID br0 / macvlan?

Can the author elaborate on the design of the network connection? I'm trying to figure out the connection between the <IP_ADDRESS> gateway of the container itself and docker0 (172.17.0.x) that docker establishes for bridge mode. Because currently only bridge mode can connect containers. I'm thinking about how to connect containers in host mode or custom:br0 mode.
Thank you so much.

The method of creating a macvlan network that matches the local subnet I tried before and did not succeed. On the one hand, the default subnet of custom network br0 under the docker of the unraid platform is <IP_ADDRESS>/24, and the newly created subnet conflicts with its address. Change the range of the default br0 CIDR block to a smaller one, and other containers cannot be connected to the network. Can you provide a complete example of the RAID platform for creating a MACVLAN network for VDSM?

The host and container baota can be connected to each other.

Similarly, using the br0 network type, the baota container of <IP_ADDRESS> can be connected, and the virtual-DSM container of <IP_ADDRESS> cannot be connected.

No offense, but given that I have about 20 containers running on the Unraid Macvlan network, it's probably not an Unraid bug. I don't want to sound ungrateful because I think this project is great, but basic troubleshooting leads me to believe it's not an unraid bug. I also have my network on br0.
I'll try to disable Host access when I have a chance, and maybe try ipvlan as well.

The new version keeps getting stuck at this step.

log.txt

I don't much about coding or the virtual dsm, but is it possible to pass the MacVlan IP to the virtual dsm? Instead of trying to find a way to route the macvlan ip to the virtual ip, can we have the vdsm set as DHCP and maybe pull the ip from the macvlan?
I dont' know how much sense that makes.

I'm getting this on the logs.  The problem is I do not know how to put in the --device-cgroup-rule='c 243:* rwm'  in unraid.
I put it variable but im sure it is wrong as it is not an environment varialble.  If I put it as device it show as error.

Can anyone advise how to correct this?  Thanks

Thanks Kroese for the advise.  That work!!! Got an IP via DHCP .  Now the vDSM is on another IP on my network not bridged in unraid IP.  Can access the vdsm storage via smb on that ip.

@WarMachine228 Have you implemented the BR0 network connection to VDSM? Please describe the method in detail. Thank you.

@kroese ah yes. Enable host access did the trick.  Now can mount shares from unraid. Thank you

@Alan-dowayStudio  yes using bro network. Juat follow @kroese  instructions above. I screenshot the whole unraid docker confog page later when I get home.

This is my docker config for vDSM.  Can set a static ip on the Synology Control Panel after to set to the the IP you want.

My Extra parameter is: --stop-timeout 60 --cap-add NET_ADMIN --device-cgroup-rule='c 243:* rwm'
But as Kroese said your device-cgroup-rule number may be different.  Check your logs for the correct number,  Check my screenshot from a previous post where it showed the cgroup rule number to input.

OK, the problem of direct connection to VDSM in BR0 mode has been solved. But it still needs to occupy two IP addresses, I believe we will find the perfect solution. My specific steps are:

Set the network mode to br0, the IP address is empty (automatically obtained), the DHCP variable is Y for installation, and the console prompts: ERROR: Cannot create TAP interface. Please add the following docker settings to your container: --device-cgroup-rule='c 243:* rwm' --device=/dev/vhost-net；
Edit configuration: display the advanced view, modify the extra parameters to --cap-add NET_ADMIN --stop-timeout 60 --device-cgroup-rule='c 243:* rwm' --device='/dev/vhost-net';
Start the container, after completing the installation, set the fixed IP address in the network configuration of the DSM control panel.

Thanks @Alan-dowayStudio I did as you described. However, I am stucked a little bit before it:
"ERROR: Capability NET_ADMIN has not been set. Please add the following docker setting to your container: --cap-add NET_ADMIN"
which is interesting, as I already have set "Extra Paramters" to "--cap-add NET_ADMIN --stop-timeout 60 --device-cgroup-rule='c 243:* rwm' --device='/dev/vhost-net'"
Did you started and installed DSM before on the bridge network?

yes
发自我的iPhone

在 2023年4月30日，00:41，zilchulant @.***> 写道：

Thanks @Alan-dowayStudio I did as you described. However, I am stucked a little bit before it:
"ERROR: Capability NET_ADMIN has not been set. Please add the following docker setting to your container: --cap-add NET_ADMIN"
which is interesting, as I already have set "Extra Paramters" to "--cap-add NET_ADMIN --stop-timeout 60 --device-cgroup-rule='c 243:* rwm' --device='/dev/vhost-net'"
Did you started and installed DSM before on the bridge network?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.

Unfortunately not.
Still the same issue:

Starting Virtual DSM for Docker v3.81...
INFO: Container IP is <IP_ADDRESS> with gateway <IP_ADDRESS>
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet <IP_ADDRESS>  netmask <IP_ADDRESS>  broadcast <IP_ADDRESS>
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
inet <IP_ADDRESS>  netmask <IP_ADDRESS>
loop  txqueuelen 1000  (Local Loopback)
default via <IP_ADDRESS> dev eth0
<IP_ADDRESS>/24 dev eth0 proto kernel scope link src <IP_ADDRESS>
++ ip link add link eth0 dsm_vlan type macvlan mode bridge
++ rc=2
++ ((  rc != 0  ))
++ echo -n 'ERROR: Capability NET_ADMIN has not been set (2/1). Please add the '
ERROR: Capability NET_ADMIN has not been set (2/1). Please add the following docker setting to your container: --cap-add NET_ADMIN
++ echo 'following docker setting to your container: --cap-add NET_ADMIN'
++ exit 15

Now I am using the docker-compose addon. So my Compose File looks like this:
version: "3"
services:
    dsm:
        container_name: dsm
        image: kroese/virtual-dsm:latest
        environment:
            DISK_SIZE: "16G"
            ALLOCATE: "N"
            CPU_CORES: "2"
            RAM_SIZE: "512M"
            DHCP: "Y"
            DEBUG: "Y"
        devices:
            - /dev/kvm
            - /dev/vhost-net
        device_cgroup_rules:
            - 'c 510:* rwm'
        cap_add:
            - NET_ADMIN
        ports:
            - 5000:5000
        volumes:
            - /mnt/cache/docker/vdsm:/storage
        restart: on-failure
        stop_grace_period: 1m
        networks:
            br0:
                ipv4_address: <IP_ADDRESS>

networks:
    br0:
        external: true

The br0 Network is created by unraid and is a ipvlan network (at least for me? Didn't change it).
docker network inspect br0
[    {
        "Name": "br0",
        "Id": "<<>>",
        "Created": "2023-05-05T15:42:20.275804473+02:00", <<last boot time
        "Scope": "local",
        "Driver": "ipvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "<IP_ADDRESS>/24",
                    "IPRange": "<IP_ADDRESS>/26",
                    "Gateway": "<IP_ADDRESS>",
                    "AuxiliaryAddresses": {
                        "server": "<IP_ADDRESS>"
                    }                }            ]        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""        },
        "ConfigOnly": false,
        "Containers": {}        },
        "Options": {
            "parent": "br0"        },
        "Labels": {}    }]

Thanks, I got it working. Changed br0 to macvlan in Unraid settings. According to the dsm logs I had some network issues, however with
        networks:
            - br0

It does work. I got the message with changing device_group_rules and now it's running! Thank you very much!


Yaml parser support for flow style

The current yaml parser (and printer) makes the assumption that parsed yaml documents contain only block style elements. Add support for Yaml flow style sequences and mappings, and any other flow style yaml document elements.
example flow style sequence:
a: [foo bar, baz]

example flow style mapping:
b: {c: 3, d: 4}

flow-styled sequences are now handled as of #573.
flow-styled mappings are, as of yet, unsupported.


Added my pic

My pic

AAAAAAAAAA

let's make your PR more popular

by

adding

some

amounts

of

comments

Now

you

have

more

comments

than

others

Please, add your comments, instead of modifying my ones

Lust auf ein kaffee?

@dmitrykrivaltsevich  ping...

London, Capital of Great Britain
London is the capital of Great Britain, its political, economic, and commercial centre. It is one of the largest cities in the world and the largest city in Europe. Its population is about 8 million.
London is divided into several parts: the City, Westminster, the West End, and the East End.
The heart of London is the City, its financial and business centre. Numerous banks, offices, and firms are situated there, including the Bank of England, the Stock Exchange, and the Old Bailey. Few people live here, but over a million people come to the City to work. There are some famous ancient buildings within the City. Perhaps the most striking of them is the St. Paul's Cathedral, the greatest of English churches. It was built in the 17th century by Sir Christopher Wren. The Tower of London was founded by Julius Caesar and in 1066 rebuilt by William the Conqueror. It was used as a fortress, a royal palace, and a prison. Now it is a museum.
Westminster is the governmental part of London.
Nearly all English kings and queens have been crowned in Westminster Abbey. Many outstanding statesmen, scientists, writers, poets, and painters are buried here: Newton, Darwin, Chaucer, Dickens, Tennyson, Kipling, etc.
Across the road from Westminster Abbey is Westminster Palace, the seat of the British Parliament. The Clock Tower of the Houses of Parliament is famous for its big bell, known as "Big Ben". Buckingham Palace is the official residence of the Queen.
The West End is the richest and most beautiful part of London. It is the symbol of wealth and luxury. The best hotels, shops, restaurants, clubs, and theatres are situated there.
The Trafalgar Square is the geographical centre of London. It was named in memory of Admiral Nelson's victory in the battle of Trafalgar in 1805. The tall Nelson's Column stands in the middle of the square.
On the north side of the Trafalgar Square is the National Portrait Gallery. Not far away is the British Museum — the biggest museum in London. It contains a priceless collection of ancient manuscripts, coins, sculptures, etc, and is also famous for its library.
There are a lot of factories, workshops, and docks in the East End.

@AntonZarutsky , could you please merge it?

@AntonZarutsky ...feel free to merge this. lol


Is it possible to save the state changes?

This is very nice! - thanks for your work!
As the title says, I want to save the state of a changed graph eg added, deleted, moved nodes etc - ie NOT exporting but I guess saving changed code?
Is that / could that be possible?

Hi @philiprhoades,
I apologize for the very late reply.
Do you mean that you do not want to export the modified network graph, but you want to export each change made to the network graph as source code?
Unfortunately, there is no direct way to do this. You need to listen for the necessary change events and generate code accordingly.


Add support for importing group members from a CSV file

be9f9e39d3143b8e41f2d5d0cb9f5064ae54cf51 added support for creating invites from a list of email addresses. still gotta do accepting, doing that next


Consumer eventdata does not worke if system time isn't UTC

Link : https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/README.md#consume-events-using-an-event-processor
Code snippet:

Error Details : Will throw an exception while perform this code.

Consumes Event data failed while system time is not UTC time, it works if set the system time as UTC.

@jsquire This is a .NET issue.

@jsquire This is a .NET issue.

Yes.  We're tracking it and are reaching out for additional context and repro steps.


Update student identification field to student id

Update student identification from name to studentId
Closes #73

Closes #73


use c/storage instead of docker/docker

Use c/storage packages instead of docker/docker ones where possible.

/lgtm

Nice.


Desktop Explorer

Description of the new feature / enhancement
I would like the option that the desktop will act like a file explorer. You could save things to desktop, right click the desktop background, but it will look and manage as if it was a folder (so you could scroll, if you have too many files there, or sort by details, see preview etc'.
Scenario when this would be used?
in cluttered desktops...
Supporting information
No response

is the ask here to allow the actual desktop to be like a detail view? /needinfo

Yes, that was my thought.
Thanks.
בתאריך יום ה׳, 17 באוק׳ 2024, 02:48, מאת Clint Rutkas <
@.***>:

is the ask here to allow the actual desktop to be like a detail view?
/needinfo
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/35440#issuecomment-2418155478,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA6QVEVKAL6FU6S3DBQ5R63Z333L3AVCNFSM6AAAAABP6XECZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYGE2TKNBXHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

neat idea, but i don't think powertoys would create that.


feat: shutdown SSL connection upon closing

This PR lets the DtlsClient send a close notify alert to its peer upon closing, causing connections to be shutdown more gracefully.

Codecov Report
Base: 23.87% // Head: 23.20% // Decreases project coverage by -0.66% :warning:

Coverage data is based on head (fd9a565) compared to base (0f57232).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #21      +/-   ##
==========================================
- Coverage   23.87%   23.20%   -0.67%
==========================================
  Files           7        7
  Lines         289      293       +4
==========================================
- Hits           69       68       -1
- Misses        220      225       +5

Impacted Files
Coverage Δ

lib/src/dtls_client.dart
16.04% <0.00%> (-0.18%)
:arrow_down:

lib/src/generated/ffi.dart
29.48% <0.00%> (-0.78%)
:arrow_down:

lib/src/lib.dart
78.94% <0.00%> (-5.27%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Great job!

Hey there,
I just had to come here to thank you! I just clean installed my windows machine, and i can't thank you enough for this script! everything worked as it should, and i'm really enjoying this clean and smooth experience. Thanks for your work man, really aprecciate you take the time to care about us :)
cheers and have a nice day

I'm happy you enjoy it! And also credit to other collaborators as well :D


Disable HTML mode when having NONE as popup

close https://github.com/CartoDB/cartodb/issues/8647

waiting for disabled style for toggle to be reviewed in cartoassets (I'll bump version then), working already
CR @piensaenpixel /cc @saleiva

ping!

I'd add a test checking the behaviour BTW.

roger!

test added, review @xavijam?

:+1:


Update to nan 1.5.1 - iojs support

Updated nan to 1.5.1 for iojs support.

Was that simple then! Thanks!


Tutorial "Bayesian Hidden Markov Models" was not reproducible

I walked through the Bayesian Hidden Markov Models tutorial and noticed that the result was not reproducible.
The result are also not good.
Is this due to a difference of other packages version?
(I've copied and pasted the the tutorial, so I won't attach the code.)

Julia v1.3.1
Turing v0.10.1

That's a quite old version of Turing. Have you tried it with Turing 0.13 as well?

I updated to 0.13 but the result didn't improve so much...
I also noticed that the sample number in tutorial's gif animation is different from the code.
There should be only 100 samples but the gif is sampling 500 samples.
g = Gibbs(HMC(0.001, 7, :m, :T), PG(20, :s))
c = sample(BayesHmm(y, 3), g, 100);
# changing the sample number to 500 didn't improve the result as well.

# Extract our m and s parameters from the chain.
m_set = c[:m].value.data
s_set = c[:s].value.data

# Iterate through the MCMC samples.
Ns = 1:length(c) # there're only 100 samples

# Make an animation.
animation = @animate for i in Ns
    m = m_set[i, :];
    s = Int.(s_set[i,:]);
    emissions = collect(skipmissing(m[s]))

    p = plot(y, c = :red,
        size = (500, 250),
        xlabel = "Time",
        ylabel = "State",
        legend = :topright, label = "True data",
        xlim = (0,15),
        ylim = (-1,5));
    plot!(emissions, color = :blue, label = "Sample $$N")
end every 10;

I looks like an initialisation issue related to that seed. Below is the gif produced by Random.seed!(12345678) and a longer chain sample(BayesHmm(y, 3), g, 500).

Wow, I didn't expect that seed value would make such a big difference...
Thank you so much for letting me know!
I learnt something new.

I mean it's fixed seed, so if it's too bad, it could be hard to give sensible results for a short chain (500 here). What I did is increasing the chain length to 500, run the sample call a few times and I found some of the runs are good. Then I realized this is probably an initialization problem. I ended up trying a few seed that works and reproducible.


Security center :UI change.

Contacts you have the Recovery Key for will appear here.

Verified this issue on dev app v2.0.70(441)


Prevent test libraries from accidentally including dependency tests

I've noticed an annoying issue recently while running tests in the cadl-azure repo: running rushx test for any of the individual libraries also causes the tests to run for any libraries that the tested library depends on.  This means that running tests for cadl-azure-core also causes the tests for @cadl-lang/rest and @cadl-lang/versioning to also run.
You can see this for yourself by looking at CI logs for cadl-azure library test runs (scroll down a bit after each library's tests finish):

cadl-azure-core
cadl-azure-resource-manager
cadl-providerhub-controller

I've tracked down the problem to how we load up the JavaScript files for referenced test libraries:
https://github.com/daviwil/cadl/blob/ea85521f9ecd30cfa47817b6a796125fa01b1f7d/packages/compiler/testing/test-host.ts#L198-L219
We're basically loading all .js files indiscriminately from the test library's configured path.
If you look at the change I'm making in this PR, we use the top-level dist path as the test library path meaning that we're loading up all files in each library's dist/test path too!  Loading all the .js files under dist/test causes Mocha to register all those tests to be executed, in addition to the actual tests we're trying to run.
This change just defaults the library to dist/src to avoid pulling in all of the test files for each library.  This fixes the issue without causing any apparent adverse affects in test runs across Microsoft/cadl and Azure/cadl-azure.
I'm using this change in the following PR and the tests are all passing on cadl-azure side: https://github.com/Azure/cadl-azure/pull/2555

Wow, thanks!


Plan ID - globally unique or unique within the service?

There is a discrepancy in the requirements for Plan ID:

in the plan object doc:
https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#plan-object

id* | string | An identifier used to correlate this plan in future requests to the broker. This MUST be globally unique. MUST be a non-empty string. Using a GUID is RECOMMENDED.

In the provisioning request doc:
https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#body-2

plan_id* | string | The ID of the plan (from the catalog) for which the service instance has been requested. MUST be unique within the service. MUST be a non-empty string.

Nice catch. I vote for unique within the service. Since service id's are globally unique and required on each request, this would make it simpler and unambiguous.

@angarg12 agreed. Also, it is probably redundant to describe the format of the plan id in the provisioning request, since it is already described in the catalog endpoint where broker returns a list of plans.

In CF we require plan IDs to be globally unique. Changing this to unique within a service would cause issues with our API. But obviously I agree the discrepancy needs fixing, and I'd vote for globally unique.

I think that's a bug. While it may feel a bit inconsistent to have the name be scoped to the service while the ID is globally unique, I think it makes a lot more sense from a UX and coding perspective.
I'll open a PR to fix it.

I'll open a PR to fix it.

@duglin are you still planning on opening a PR? :)

yea, its on my todo list - I'll try to get to it later today

just looked at this and I think its been fixed already. The uniqueness text in the provision section have been removed, thus removing the inconsistency.
It still says it must be globally unique during the retrieval of the catalog, which is what we agreed to, and really the only spot it needs to be talked about since that's where the ID is generated. The provision just uses it and can't control its uniqueness at that point.

Going to close this.  We can reopen if needed.


Majority Element in an array

I am a participant and I want to implement the majority element algorithm in C++ with O(n) time complexity and O(1) space complexity.
A majority element in an array A of size N is an element that appears more than N/2 times in the array using searching concept

hi, I want to work on this issue.
kindly assign to me.

I will do it using c++

Hey, I am Divya, a participant of GSSOC'21. I can contribute a JAVA code for the same. I will make sure that my code is easily understandable and explanatory. Please assign me this issue.
Thanks.

Hey, I am Kanchan Jeswani, a GSSOC'21 participant.
I am a Competitive Programmer and C++ enthusiastic.
I would love to solve this problem using CPP with well-explained code and along with its time and space complexity.
Please assign me this issue.
Hope to hear from you.
Discord username  kanchan19#3152
@anubhavitis @dheerajkotwani @tarun26091999 @todi-2000 @geekquad @plazzy99 @rudrakshi99

you are talking about Moore's voting algo, it is already present at #392


Support all types of callable for functions and filters

Twig has full Closures support, but we can't use this feature.

What's the usage of it?

Already applied.


Add Alternate Noir Versions to our DB

With broader version support for Noir Circuits, we need to update our sample circuits to include each possible supported version.  (Bonus points for circuits with features not supported by previous versions!)

Is this ticket still needed?


[Question] Execution Time

Hello,
First of all, I want to thank you for the incredible work you done.
I run these lines:
import time
import yfinance as yf

Symbol = "AAPL"

start_time = time.time()
Asset_YF = yf.Ticker(Symbol)
print("- %s seconds" % (time.time() - start_time))

start_time = time.time()
Asset_Info = Asset_YF.info
print("- %s seconds" % (time.time() - start_time))

start_time = time.time()
Asset_Data = Asset_YF.history(period="max")
print("- %s seconds" % (time.time() - start_time))

The output:

0.005499362945556641 seconds
1.2929999828338623 seconds
0.41750001907348633 seconds

Around 1.7s of execution time. This is not huge, except in the context of web services.
I'm wondering if there is any possibility of improving this time. Is there any existing parameter (in your package) I should use to do this ? (ex:multiprocessing)
Thank you in advance,
Best Regards

I have the same issue. I'm extracting a couple of info fields from multiple tickers and it takes quite a long time as suggested above. I wonder if there is a way to speed up this.
Many thanks!

I am having the same problem for a while. Just checked it with another package finviz ... The reason I used 'PEG` ratio for yf is that the info dictionary gives key error for trailingPE when the company does not have a PE ratio yet but it returns PEG with None:


[SPARK-12154] Upgrade to Jersey 2

What changes were proposed in this pull request?
Replace com.sun.jersey with org.glassfish.jersey. Changes to the Spark Web UI code were required to compile. The changes were relatively standard Jersey migration things.
How was this patch tested?
I did a manual test for the standalone web APIs. Although I didn't test the functionality of the security filter itself, the code that changed non-trivially is how we actually register the filter. I attached a debugger to the Spark master and verified that the SecurityFilter code is indeed invoked upon hitting /api/v1/applications.

Test build #57035 has started for PR 12715 at commit b86ed6f.

Test build #57035 has finished for PR 12715 at commit b86ed6f.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57035/
Test FAILed.

Merged build finished. Test FAILed.

Test build #57039 has started for PR 12715 at commit b4d72d8.

Test build #57039 has finished for PR 12715 at commit b4d72d8.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57039/
Test FAILed.

Test build #57041 has started for PR 12715 at commit e2dbf03.

Test build #57041 has finished for PR 12715 at commit e2dbf03.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57041/
Test FAILed.

Test build #57067 has started for PR 12715 at commit 313401b.

Test build #57069 has started for PR 12715 at commit 5793600.

Test build #57069 has finished for PR 12715 at commit 5793600.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57069/
Test FAILed.

Test build #57070 has started for PR 12715 at commit c9b5c3b.

Test build #57067 has finished for PR 12715 at commit 313401b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57067/
Test FAILed.

retest this please

Test build #57082 has started for PR 12715 at commit c9b5c3b.

Test build #57070 has finished for PR 12715 at commit c9b5c3b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57070/
Test FAILed.

Merged build finished. Test FAILed.

Test build #57082 has finished for PR 12715 at commit c9b5c3b.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57082/
Test FAILed.

@JoshRosen I did not change the Jersey dependency for the docker-integration-tests project. Should we change that here or leave it alone?

Jenkins, retest this please

Test build #57155 has started for PR 12715 at commit c9b5c3b.

Test build #57155 has finished for PR 12715 at commit c9b5c3b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57155/
Test FAILed.

Test build #57174 has started for PR 12715 at commit bed5618.

Test build #57174 has finished for PR 12715 at commit bed5618.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57174/
Test FAILed.

Test build #57177 has started for PR 12715 at commit bfb178a.

Test build #57177 has finished for PR 12715 at commit bfb178a.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57177/
Test FAILed.

Merged build finished. Test FAILed.

Test build #57182 has started for PR 12715 at commit a199cb6.

Test build #57184 has started for PR 12715 at commit 5a95559.

Test build #57182 has finished for PR 12715 at commit a199cb6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57182/
Test FAILed.

Hm... Are SparkLauncher tests just flaky or could this change have directly affected that?

Jenkins, retest this please

That particular one shouldn't be affected by these changes. I'll take a closer look at it separately.

Test build #57196 has started for PR 12715 at commit 5a95559.

Test build #57184 has finished for PR 12715 at commit 5a95559.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57184/
Test PASSed.

Test build #57196 has finished for PR 12715 at commit 5a95559.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57196/
Test PASSed.

LGTM, I'll leave this for @srowen since he had a comment about licenses.

Yeah you can update the following section of NOTICE to read as follows and that should be it:
========================================================================
Common Development and Distribution License 1.0
========================================================================

The following components are provided under the Common Development and Distribution License 1.0. See project link for details.

     (CDDL 1.0) Glassfish Jasper (org.mortbay.jetty:jsp-2.1:6.1.14 - http://jetty.mortbay.org/project/modules/jsp-2.1)
     (CDDL 1.0) JAX-RS (https://jax-rs-spec.java.net/)
     (CDDL 1.0) Servlet Specification 2.5 API (org.mortbay.jetty:servlet-api-2.5:6.1.14 - http://jetty.mortbay.org/project/modules/servlet-api-2.5)
     (CDDL 1.0) (GPL2 w/ CPE) javax.annotation API (https://glassfish.java.net/nonav/public/CDDL+GPL.html)
     (COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL) Version 1.0) (GNU General Public Library) Streaming API for XML (javax.xml.stream:stax-api:1.0-2 - no url defined)
     (Common Development and Distribution License (CDDL) v1.0) JavaBeans Activation Framework (JAF) (javax.activation:activation:1.1 - http://java.sun.com/products/javabeans/jaf/index.jsp)

========================================================================
Common Development and Distribution License 1.1
========================================================================

The following components are provided under the Common Development and Distribution License 1.1. See project link for details.

     (CDDL 1.1) (GPL2 w/ CPE) org.glassfish.hk2 (https://hk2.java.net)
     (CDDL 1.1) (GPL2 w/ CPE) JAXB API bundle for GlassFish V3 (javax.xml.bind:jaxb-api:2.2.2 - https://jaxb.dev.java.net/)
     (CDDL 1.1) (GPL2 w/ CPE) JAXB RI (com.sun.xml.bind:jaxb-impl:2.2.3-1 - http://jaxb.java.net/)
     (CDDL 1.1) (GPL2 w/ CPE) Jersey 2 (https://jersey.java.net)

@srowen Should the NOTICE still contain Jersey 1 since I think we still pull it in explicitly for YARN unit tests?

Test build #57622 has started for PR 12715 at commit 70d4844.

This is looking good. Regarding the license, it depends on whether the artifacts we distribute contain Jersey 1.x or not. I have the impression it isn't at the moment. If you have the build handy and can look at the assembly and grep its contents, that would help verify whether 1.x classes are still there or not. If not, this is fine. Otherwise just restore the Jersey 1.x license statements.

Test build #57622 has finished for PR 12715 at commit 70d4844.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57622/
Test PASSed.

I didn't see any old-Jersey jars in the lib directory.

@srowen given that Jersey 1 isn't being distributed, this PR should be good to go?

LGTM; let me leave this open a short while longer for comments.

Question: will this be ported to the 1.6 branch?

No, it's too significant a change for a maintenance release.

Thanks Sean. So, just to recap regarding this, whoever is stuck trying to use spark in a web application based on Jersey 2 has currently the following options:

wait for spark 2.0;
rollback the web application to use Jersey 1.9 (to be consistent with spark);
rollback the spark version to be 1.3.1 (the only one that can be easily integrated by simply excluding the jersey-server dependecy);
replace Jersey 2 in the web application with something else (e.g. spring mvc).

Is my understanding correct?

I think the best option is to shade Jersey if you need a different version.

Merged to master/2.0


Implement OIDC signin() functionality

Implement a toolkit of functions that helps the client-side developer to:

Redirect the user to a 'discover' app, or help pop up a discover UI.
Redirect the user to a signin app for their preferred server (if the discovery is handled by the client UI, above)
Parse the callback redirect parameters and extract the user's ID Token. Extract the WebID from the ID Token.

Implemented in latest master, closing.


[hunspell] Update and fix

What does your PR fix?
Updates hunspell to 1.7.1.
Adds explicit nls feature for native language support.
Fixes #26470.
Fixes #22301.

Which triplets are supported/not supported? Have you updated the CI baseline?
unchanged, no
Tested with x64-mingw cross builds.

Does your PR follow the maintainer guide?
yes

If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?
yes

Feature tools tested successfully in the following triplet:

x86-windows
x64-windows
x64-windows-static

Feature nls tested successfully in the x64-linux triplet

Thank you!


Newly created terminal from profile is not focused after creation

Testing #124793

Create a new terminal using the profile contributed by the extension from the dropdown or from command pallette - Create New Terminal Integrated Terminal (With Profile)

🐛 Newly created terminal is not focused.

It is still not getting focused when created from dropdown action in the terminal.

@sandy081 I can't repro, can you show a gif and check for console errors?

Here you go

I do not see any console errors
Here is the code snippet to register terminal profile provider
context.subscriptions.push(vscode.window.registerTerminalProfileProvider('my-ext.terminal-profile', {
		provideProfileOptions(token: vscode.CancellationToken): vscode.ProviderResult<vscode.TerminalOptions | vscode.ExtensionTerminalOptions> {
			return {
				name: 'Profile from extension',
			};
		}
	}));


WIP: BPE Training ctc loss and label smooth loss

As metioned in #217，currently bpe training with ctcLoss and labelSmoothLoss in snowfall obtain higher wer than that of espnet.

decoding algorithm
training tool
encoder + k2 ctc decode+no rescore

k2 ctc decode in #217
espnet
2.97

k2 ctc decode in #217
snowfall
3.97 (updated on June 24 with wrong data preparation)

k2 ctc decode in #217
snowfall
3.32 (updated on June 28 by fixing data preparation mistake)

avg (10epochs, 26-35 epochs)
INFO:root:[test-clean] %WER 3.33% [1749 / 52576, 171 ins, 140 del, 1438 sub ]
INFO:root:[test-other] %WER 8.06% [4218 / 52343, 397 ins, 403 del, 3418 sub ]
avg (10 epochs, 29 - 38 epochs)
INFO:root:[test-clean] %WER 3.32% [1744 / 52576, 184 ins, 136 del, 1424 sub ]
INFO:root:[test-other] %WER 7.96% [4167 / 52343, 402 ins, 367 del, 3398 sub ]

The PROBLEM I am facing is:
Wer of snowfall trained models is still a little higher than the model of espnet trained, by 3.32% > 2.97%.
(fixed by correcting datapreparation mistake)During espnet training: loss_att and loss_ctc always have the same order of magnitude, i.e. they decrease at the same pace.
However during snowfall training, loss_att decrease sharply to even below 1.0 while loss_ctc keeps more than [30 to 100] times larger than loss_att.
espnet training log file: https://github.com/glynpu/bpe_training_log_files/blob/master/espnet-egs2-librispeech-asr1-exp-asr_train_asr_conformer7_n_fft512_hop_length256_raw_en_bpe5000_sp-train.log
snowfall training log file of wer 3.97%: https://github.com/glynpu/bpe_training_log_files/blob/master/snowfall-egs-librispeech-asr-simple_v1-train_log.txt
snowfall training log file of wer 3.32% experiment:
https://github.com/glynpu/bpe_training_log_files/blob/master/wer_3.32_June_26_snowfall_egs_librispeech-asr-simple_v1-train_log.txt
What I have tried to make compariable between espnet and snowfall are:

model structures: by loading espnet released models into snowfall successfully(using regular expression to chage key's name in state_dict, similar to this), I believe model structures are identical except parameter names.
loss functions: both use torch.nn.CTCLoss and torch.nn.KLDivLoss
normalization: both loss are normalized by batch_size in espnet and snowfall
learning rate schedule: espnet use WarmupLR, while snowfall use Noam. WramupLR(optimizer.lr=0.0025, warmup_steps=40000) and Noam(model_size=512, factor=10.0, warm_step=40000) are quite similar, though not 100 percent identical.
each batch contains utts in silimar duration: espnet use  NumElementBatchSampler and snowfall use BucketingSampler
token_ids: in total 5000 tokens. After spm tokenizer is trained,    are removed while other 4997 tokens are kept.  Then three tokens, "blank_id = 0; oov_id = 1; sos_eos_id=4999", are manully added(reference).

After fixing some bugs, wer on test-clean decrease from 3.97% to 3.32%, though it's still higher than espnet's 2.97%.

Great work!!

Sorry for my late response.
I did not notice @glynpu's comment.
First, I just want to know whether the difference comes from the training part (probably so) or other parts.

I want to check the learning curve. Could you share it?
Did you compare the best validation accuracy (note that espnet uses the teacher forcing when computing the accuracy and it would be ~95.7% as written in the log)? We can compare them if we use the same (similar) BPE size.

Also, could you point out the main script to me?
I want to check the overall training and inference flows and several hyper-parameters.

Thanks for your kindly help! @sw005320

I want to check the learning curve. Could you share it?

tensorboard for above screenshot

Did you compare the best validation accuracy (note that espnet uses the teacher forcing when computing the accuracy and it would be ~95.7% as written in the log)? We can compare them if we use the same (similar) BPE size.

Not yet. I will compare the differences between espnet and snowfall.

Also, could you point out the main script to me?I want to check the overall training and inference flows and several hyper-parameters.

Currently this pr is about training part. and #227 is focusing on decoding part.
For training, this shell scripts is the entrance.
  export CUDA_VISIBLE_DEVICES="0, 1, 2, 3"
  python3 bpe_ctc_att_conformer_train.py \
    --bucketing-sampler True \
    --lr-factor 10.0 \
    --num-epochs 50 \
    --full-libri True \
    --max-duration 200 \
    --concatenate-cuts False \
    --world-size 4 \
    > train_log.txt

Some hyper-parameters is hard-coded in bpe_ctc_att_conformer_train.py or constructor of class Conformer. Some of them are listed below:

name
value

warmup_step
40,000

att-rate
0.7, i.e. ctc_weight=0.3

lsm_weight
0.1

num_encoder_layerss
12

num_decoder_layers
6

nhead
8

atten_dim
512

For decoding part,  the latest decoding implementations is #217, and I plan to port them to espnet after it's approved and finally merged into sowfall. The entrance of decoding is here
'''
if [ $stage -le 3 ]; then
export CUDA_VISIBLE_DEVICES=2
python bpe_ctc_att_conformer_decode.py
--max-duration=20
--generate-release-model=False
--decode_with_released_model=True
--num-paths-for-decoder-rescore=500
fi
'''
A core function of decoding in bpe_ctc_att_conformer_decode.py is here

Thanks!
Did you use model averaging?
If so, how do you pick up models (best loss?), and how many?

I found it https://github.com/k2-fsa/snowfall/pull/217/files#diff-fd4e35e8e4b530ddf5ca285f24f2f92dfb6a0db691e75b4efe1dc59309654883R146-R151
Did you tune it?
It may not have a big difference, but we usually pick up the 10-best models based on the validation accuracy for averaging.

Latest results are:
  | before | current
-- | -- | --
Encoder + ctc | 3.32 | 2.98( wer of espnet released model is 2.97/3.00)
Encoder + TLG + 4-gram lattice rescore +  nbest rescore with transformer decoder with log_semering=False and remove repeated tokens | 2.73 | 2.54
Result difference between current pr and espnet is sovled by tune training hyper-parameters with following modifications:
  | feat-norm | learning-factor | warm-up steps | epoch
-- | -- | -- | -- | --
before | no | 10 | 40,000 | 40 epoch (avg=10, with 26-35 epoch)
current | yes | 5 | 80,000(around 10 epochs) | 50 epochs (avg=20 with 31-50 epochs)
Reason of previous modifications are:
I realized that in espnet, 1 epoch contains around 3000 batchs;
however, in my implementation, with max_duration=200, one epoch contains 6000 batchs.
As a matter of experience, smaller batch_size is compatible  with smaller learning rate, so half the learning rate.
Since 1 epoch contains 6000 batchs ranther than 3000 batchs now, I doulbe warm-up steps.
The module feat_batch_norm also helps, resulting 3.32 --> 3.17.
As 35 epochs --> 50 eochs, I just set it arbitrarily to see what will happen with more epochs.
BTW, I failed to increase max_duration=200 because larger max_duration is easily caused OOM, 200 seems the largest with my GPUs.

I feel that 80,000 warm-up steps are too large. It requires larger epochs to make training converged. I think you can find some optimal points with  fewer warm-up steps and comparable performance.
Also, how about using the 3000 batches?

I feel that 80,000 warm-up steps are too large. It requires larger epochs to make training converged. I think you can find some optimal points with fewer warm-up steps and comparable performance.

80,000 is calculated from:
around 10 epochs =  40,000/ 3000 (in espnet) = 80,000 / 6000(current pr)

Also, how about using the 3000 batches?

6000 batches --> 3000 batches  means max_duration = 200 --> max_duration = 400;
which will cause OOM in some batches.
I am still analysising the reason.

As we mentioned in person, I believe a problem with the current setup is that the transformer loss is being normalized (divided by the minibatch size) twice,  once in a library function and once in the training script, while the CTC loss is only normalized once.
If we had logged the 2 objectives separately, we likely would have noticed this.
I think that normalizing even once is not right, and that we should not normalize either of these objectives.  The reason is that Librispeech has a wide range of durations, and the Lhotse sampler that we are using actually puts minibatches in bins where they have about the same duration (and approximately constant total duration in seconds), so in effect, right now we have a weight per frame that rises linearly with the sentence length.  This will tend to cause convergence problems because longer sentences are harder to align.  Removing the normalization (division by len(texts)) should not require changes to learning rates, because we are using Adam and there is no weight decay.
[However, as a separate issue, I think we should experiment with a very small weight decay, which will cause the system to train/converge faster.]

FYI, espnet did not normalize the CTC and attention loss by the length.

Does anyone have any pointers to visualizations of the decoder attention in the application of transformers to ASR?  I want to get a feel for how it works.


Alluxio metrics

Is your feature request related to a problem? Please describe.
Alluxio has a bunch of metrics and we should figure out which ones are critical to user and document and figure out the best way for them to view them.

normally alluxio has UI, this is not availalble by default on databricks, can we ssh tunnel or get it somehow?
can look at metrics via command line: alluxio fsadmin report metrics
should we save certain ones to csv in log files and then have a way to graph them?  We want to be careful about saving all to disk if it adds a lot of load to the disks.

Are we going to save the alluxio metrics in Databricks usage? Would it be something configurable in our customized Docker image?

normally alluxio has UI, this is not availalble by default on databricks, can we ssh tunnel or get it somehow?

We can use ssh tunnel to access the UI.
Assume we can login to alluxio master via an egx host:
For example:
ssh egx_user@egx_host
ssh<EMAIL_ADDRESS>-p 2200 -i <private_key_file_path>

Create a tunnel from localhost 39999 port to an egx host 29999 port,
then create a tunnel from egx host 29999 port to alluxio master port 19999.
Port forwarding path is:
localhost:39999 -> egx-host:29999 -> alluxio-master:19999
Details:
Let assume:
egx_host=<IP_ADDRESS>
egx_user=chonggao
alluxio_master_host=ec2-34-211-208-96.us-west-2.compute.amazonaws.com
alluxio_master_user=ubuntu
step 1:
# forward the egx port 29999 to local 39999
# On your own machine
ssh -L 39999:localhost:29999 egx_user@egx_host
# check is listening to 39999 port on your own machine
netstat -anp | grep 39999 | grep LIS
(base) chongg@chongg-pc:~$ netstat -anp | grep 39999 | grep LIS
tcp        0      0 <IP_ADDRESS>:39999         <IP_ADDRESS>:*               LISTEN      24352/ssh

step 2:
# forward the alluxio server 1999 port to the egx local port 29999
ssh egx_user@egx_host
# enter password of egx_user
ssh -L 29999:localhost:19999 alluxio_master_user@alluxio_master_host -p 2200 -i <private_key_file_path>
# check is listening to 29999 port on egx host
ssh egx_user@egx_host
chonggao@spark-egx-02:~$ netstat -an | grep 29999 | grep LIS
tcp        0      0 <IP_ADDRESS>:29999         <IP_ADDRESS>:*               LISTEN

step 3:
# aceess the 39999 port on web browser
# On your own mathine
http://localhost:39999

but we also need to handle case we can't ssh tunnel into the master node

Seems we can only save the metrics and then find a way to graph them.
It's the case you required in the following?

should we save certain ones to csv in log files and then have a way to graph them?

Are we going to save the alluxio metrics in Databricks usage? Would it be something configurable in our customized Docker image?

The latest version we do not.
The latest version has a  config to sync logs to dbfs file system.
Because the metric files are in the log path, if we enable metric files, then metric files can be synced together with log files.
Refer to the docker readme.md:

right so can we figure out which metrics are useful.  saving all the metrics adds disk load and if we have slow disks it makes it worse to save all the metrics.  It goes along with figure out which ones are useful and user should pay most attention to.

Close this issue as the following PRs were merged:
https://github.com/NVIDIA/spark-rapids/pull/7172
https://github.com/NVIDIA/spark-rapids-container/pull/24
Note: Find the critical metrics part is moved to https://github.com/NVIDIA/spark-rapids/issues/7241


Update to version 3.5.8 and set dependencies to latest versions.

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@danclewley  https://github.com/conda-forge/rsgislib-feedstock/pull/13 updates meta.yaml to the latest syntax. I'm not sure we can move forward until cgal is also updated (https://github.com/conda-forge/cgal-feedstock/pull/38) as I think everything needs to be on the new syntax (?)
You might have more luck updating #13 to pin to the older boost in the meantime, but I'm not really sure.

Closing this in favour of #13. Once that is merged will create a new fork from master to update RSGISLib version.


[otbn,dv] Weaken an assertion about tracking busy/locking state

The NotBusyAndDone_A assertion wasn't quite correct. We can make it fail by injecting a fatal error when the core is in the middle of its initial wipe. If that happens, we don't bother doing a proper locking procedure (there's nothing important to wipe!) and the "busy_execute_d" signal stays high, but "locking" goes high.
These assertions were added by me (oops) last May and I don't think they really make much sense anyway. Drop the one that isn't true.

CHANGE AUTHORIZED: hw/ip/otbn/rtl/otbn.sv

CHANGE AUTHORIZED: hw/ip/otbn/rtl/otbn.sv


[Feature Request] Ability to edit outputs or add new ones

This might already be a planned feature in which this can be ignore. The ability to add items to the output of say cobblestone or adjust the weighting to the default outputs would be great. Additionally being able to add new recipes using any modded block/item as input or output for the extractinator would be amazing.

There is some primitive support for this already. There is an option in the config called “Supported Blocks,” which allows you to add and remove blocks, as well as set their drops and rarity. I want to expand on this feature, though, because it is difficult to understand.

Does the current system allow for any modded block/item to be an input/output or only supported mods? I'm able to edit the config and change the drops of but only vanilla items are being added and none of the support mods our in the pack I'm making.

It’s supposed to support other mods, but I have discovered a bug on current versions that prevents that functionality. I’ve now patched it, so it should work on the latest commit. Also, make sure that you add the mod id that you want to support in the “Supported Mods” part of the config.

The mod is now data-driven, so you can add new drops and change recipes with a data pack.


possibly use GeneratedFile interface

at the moment the build step that precomputes the entities will add src/entities.zig to the project that uses the package, rather than the htmlentities package itself, as the cwd doesn't change on the step
    var out_file = try std.fs.cwd().createFile("src/entities.zig", .{});
    const formatted = try zig_tree.render(allocator);
    try out_file.writer().writeAll(formatted);
    out_file.close();

I am not sure if the GeneratedFile interface of zig build would work for this, would need to explore that myself. for now, package users would need to add src/entities.zig to their gitignore which is a bit hacky
another approach would be having a dedicated zig build entities.zig step that is only used for development of htmlentities which would generate the file, keeping it vendored for distribution (which is what already happens)

Hah, whoops. The dedicated build step seems like a pretty easy OK choice; I might do that this evening to at least get it building more cleanly immediately.

Trying this approach now!