Tart fails with "Error: The Internet connection appears to be offline."

Hi, we recently updated one of our mac runners to macOS 15.0.1
sw_vers
ProductName:		macOS
ProductVersion:	15.0.1

and started experiencing this issue:
Running with gitlab-runner 16.8.0 (c72a09b6)
  on <some.runner.group> aHi6uRQSm, system ID: s_b649a000851a
Preparing the "custom" executor
00:00
Using Custom executor...
2024/10/14 10:16:52 Pulling the latest version of <some.registry.url>/<some.image_name>...
2024/10/14 10:16:52 tart command returned non-zero exit code: "Error: The Internet connection appears to be offline."
WARNING: Cleanup script failed: exit status 1
ERROR: Job failed: exit status 1

There is internet connection on the runner though, I tested by pinging various places, also by visiting different websites.
Also, the image it is trying to pull is already present on the runner. When I run tart pull and try pulling the same image, it says that it is present like so:
image is already cached and linked!

tart --version
2.18.5

The problem has not happened in earlier versions of macOS or Tart, and my hunch is that it is not related to Tart version either, but something else that Tart is doing, which I may not be aware of... Or maybe it is macOS related.
Any pointers on what things I could look at? Thank you!

Another thing I just found out:

Normally, our runner runs as a service, so with gitlab-runner start.

I now ran with gitlab-runner run, triggered a pipeline and the above issue did not repeat.

So, it may be related to running it as a service.

I ran into another issue after some more fiddling. (Updated gitlab-tart-executor to 1.19.0-cbb18ff, as earlier it was on 1.8.0, also updated gitlab-runner to 17.4.1)

On the runner mac node: gitlab-runner run
Trigger a pipeline on GitLab and watch the logs
Get the following error:

Running with gitlab-runner 17.4.1 (32fe5502)
  on <runner> aHi6uRQSm, system ID: s_b649a000851a
Preparing the "custom" executor
00:00
Using Custom executor...
2024/10/14 14:20:16 Pulling the latest version of <image>...
2024/10/14 14:20:16 tart command returned non-zero exit code: "Error: Could not connect to the server."

Check logs on the gitlab-runner:

WARNING: 2024/10/14 14:20:16 Failed to stop VM: tart command returned non-zero exit code: "the specified VM \"gitlab-534216\" does not exist"  cleanup_std=err job=534216 project=581 runner=aHi6uRQSm
WARNING: 2024/10/14 14:20:16 Failed to delete VM: VM errored: failed to delete VM gitlab-534216: tart command returned non-zero exit code: "the specified VM \"gitlab-534216\" does not exist"  cleanup_std=err job=534216 project=581 runner=aHi6uRQSm
WARNING: 2024/10/14 14:20:16 VM errored: failed to delete VM gitlab-534216: tart command returned non-zero exit code: "the specified VM \"gitlab-534216\" does not exist"  cleanup_std=err job=534216 project=581 runner=aHi6uRQSm
WARNING: Cleanup script failed: exit status 1       job=534216 project=581 runner=aHi6uRQSm
WARNING: Job failed: exit status 1
                 duration_s=0.157331167 job=534216 project=581 runner=aHi6uRQSm

Looks like it is trying to delete a VM that does not exist, then errors out.

So far I managed to solve the issue by downgrading to tart executor 1.8.0, and changing the startup from:
gitlab-runner install && gitlab-runner start
to
sudo gitlab-runner install --user <user to run with> && sudo gitlab-runner start

This is likely related to the newly introduced "Local Network" permission in macOS Sequoia and the fact that GitLab Runner's binary has no LC_UUID identifier, which is critical for the Apple's Transparency Consent and Control framework.
Can you check if the workaround in https://github.com/cirruslabs/gitlab-tart-executor/issues/85#issuecomment-2363353178 works for you?
Without re-building the GitLab Runner (or waiting for the upstream fix) the permission above cannot take effect, even if you've explicitly allowed it in the GUI.

Another clue is https://github.com/cirruslabs/tart/issues/919.
Have you tried simply restarting the host? 🤔

Restart did not work, but it is likely what you described with LC_UUID, because sudoing works. I'll test the workaround and let you know.

I built like you suggested, and ran it with:
gitlab-runner install && gitlab-runner start
did not help unfortunately.
dwarfdump shows it has UUID:
dwarfdump -u gitlab-runner
UUID: 12BD2F27-6F42-3953-84FF-B8DE2ADEE7AA (arm64) gitlab-runner

Are you sure that the invocation of gitlab-runner uses your compiled GitLab Runner from PATH, and not the one installed before?
You can also inspect ~/Library/LaunchAgents/gitlab-runner.plist to see if the first array element in ProgramArguments does indeed point to your compiled version of GitLab Runner.

Does pull work if you specify an image from the internet, e.g. ghcr.io/cirruslabs/macos-sequoia-base:latest?

It works, I tested with your example image.
Not to my knowledge, no VPN or endpoint security.

We also hit this issue, but didn't stop upgrading until we'd broken our last runner 🥇 . After a few hours of playing around, we whittled it down to these instructions:

[upgrade to macOS 15.x] - causes the breakage
brew upgrade - to get latest gitlab-runner
recompile gitlab-runner with go 1.23.2git clone -b v17.5.2 https://gitlab.com/gitlab-org/gitlab-runner.git
cd gitlab-runner
go build -ldflags="-linkmode=external" -o gitlab-runner main.go

copy rebuilt gitlab-runner into brew's pathchmod u+w /opt/homebrew/bin/gitlab-runner
sudo cp -p gitlab-runner /opt/homebrew/bin/gitlab-runner

restart gitlab-runnerbrew services restart gitlab-runner

trigger a CI job that targets that runner so it pops up the message asking if you want to grant access to devices on the local network - click YES/Accept
the job that triggers the pop up will fail but subsequent jobs should succeed. If they still fail, try toggling the setting off and back on in System Settings -> Privacy & Security -> Local Network

With that we managed to bring all 9 hosts back online.
@edigaryev do you know if there is an existing Gitlab ticket we can add some weight to to get them to build the gitlab-runner with the requisite flags?

@edigaryev do you know if there is an existing Gitlab ticket we can add some weight to to get them to build the gitlab-runner with the requisite flags?

A quick search for LC_UUID yields these two issues:

https://gitlab.com/gitlab-org/gitlab-runner/-/issues/38262
https://gitlab.com/gitlab-org/gitlab/-/issues/500056

Another point to keep in mind that soon a new minor version of Golang will be released, that will include the backported LC_UUID fix. This means that a new GitLab Runner release could get the fix automatically.

Same issue here. Waiting for the go fix is the only thing we can do: https://github.com/golang/go/issues/69992
Gitlab runner issue: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/38044

You could run the gitlab runner with sudo, or try the above fix where you click the popup on the mac.

@gitperr I've tried that, but it doesn't seem to work. With sudo, the runner doesn't seem to be working at all. It says: "running", but it does not communicate to Gitlab for some reason.

Hmm, make sure that it is using the correct config.toml and that it has a proper token there.
Did you try this when installing it?
sudo gitlab-runner install --user <user to run with>

Yes, I tried that. I rebuild it according to https://github.com/cirruslabs/gitlab-tart-executor/issues/88#issuecomment-2444035719 and this works!

Update: Also this doesn't seem to work. It seems sporadic for some reason ☹

I am experiencing the same problem.
The runner has not been changed since the last time this worked and now I am starting to experience the following situation:
Running with gitlab-runner 17.5.3 (xxx)
  on macos-runner xxx, system ID: xxx
Preparing the "custom" executor 03:31
Using Custom executor...
2024/11/04 15:25:04 Pulling the latest version of ghcr.io/cirruslabs/macos-sonoma-xcode:16...
2024/11/04 15:28:35 tart command returned non-zero exit code: "Error: The network connection was lost."
WARNING: Cleanup script failed: exit status 1
ERROR: Job failed: exit status 1

This seems to be a slightly different error message, where the connection was lost instead of not present at all, therefore I wanted to add it to this issue.
I tried restarting the host and renewing all brew installs, but it did not change the outcome.

Thanks to @waddles, this is now fixed in the latest Homebrew version of GitLab Runner (>=17.6.0).
I've created https://github.com/cirruslabs/gitlab-tart-executor/pull/94 to reflect this in the README.md.


Idea: Easy handling

Just an idea.
Right now: tracer.restart(tracee, Restart::Syscall).unwrap();
If the tracee would have a reference to the tracer, you coould allow:
tracee.restart(Restart::Syscall).unwrap();

This was actually one of the original APIs I considered, and I do like the convenience of having a self-receiving Tracee::restart() method. We currently wouldn't even need a reference back to the Ptracer struct, since the body of Ptracer::restart() doesn't actually use the &mut self, it just makes a ptrace() syscall.
The choice of an &mut self receiver encodes the constraint that the Ptracer "reserves the right" to read or mutate its internal state on every tracee restart. Also, if you restart a tracee on a thread other than the original tracer's thread, it will result in an OS error (see #3). We use phantom data to make it so Ptracer is !Send, and thus the current design disallows that error (we could similarly force Tracee to be !Send, of course).
I settled on the current design as a conservative choice. I wanted to require that a Ptracer be a broker between the user and "global resource" of ptrace() syscalls, and to have a chance to update metadata about tracees (though we don't currently do that in restart()). We could preserve this property via reference back to the Ptracer, as you suggested, but we'd probably want that to be a shared &, which might then require the introduction of interior mutability in the Ptracer.
So, all in all, I'm not sure that the UX benefit is worth the cost. That said, I'll revisit this over the coming weekend, but just wanted to share the original motivation behind the current design.

I don't want to invalidate all your thoughts, it has just been an idea. It's good to see that you considered this in your original api specifications - and I am looking forward to the weekend :-)

I did revisit this, and for now, I want to continue requiring the use of the Ptracer to restart (for the reasons above). I'm willing to revisit this again in the future, maybe after buttoning down some of the other attach modes. Thanks for prompting me to think about it again, @Beiri22!


Improved Arquillian test debugging + use non-conflicting port

https://issues.jboss.org/browse/AGPUSH-1434
https://issues.jboss.org/browse/AGPUSH-1433
How to test?

run the tests when you are already having a WildFly instance running (the ports should not conflict due to port offset set in tests)
watch the console and once Push Sender tests will be running, WildFly starts and then TestClass.war pattern deployments are being deployed (instead of UUID.war)

\o/ awesome to be able to build again with Wildfly running
:+1:

okay, commented on various config options and their meaning
rebased on latest master

thanks!
On Wed, Jun 3, 2015 at 11:07 AM, Lukáš Fryč<EMAIL_ADDRESS>wrote:

okay, commented on various config options and their meaning
rebased on latest master
—
Reply to this email directly or view it on GitHub
https://github.com/aerogear/aerogear-unifiedpush-server/pull/582#issuecomment-108257750
.

--
Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf

ship it !

merging....

landed in 98e8bbf16a4efe70bea511a9f4d9bc1842e07492

thanks, @lfryc :tada: :tada: :tada: :tada:


Check if there are pending changes to synchronize

We like to have a way to check if there are changes in the db that have not been synchronized.
There are some situations where the app might be offline, and some changes have occurred, this feature would allow us to detect this situation and react to it, allowing the app to display a warning message that it requires some connectivity.
A workaround is to combine a Realm, session change and progress listeners, to detect if the DB is in a dirty state, and if all changes have been uploaded.
We would provide an API to check if there are pending changes, and/or a flow with the current status.

all this data is already available in the sync client so we can easily expose it


Mac app doesn't start

Downloaded the application from the website for my MacOS operating system.
Can't start it. Crashes instantly.
My os: Mac OSX 14.0 (Intel)

Hey there, would you mind sharing the crash log if any is displayed? Thanks. I’ll work on fixing it promptly.

macOS 12(intel)
-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Incident Identifier: FE2B289C-F63B-4142-AC4F-CA93BB3D145C
CrashReporter Key:   A579D432-2FE8-A396-632B-DC7FA7EEB972
Hardware Model:      MacBookPro12,1
Process:             Beaver-notes [87155]
Path:                /Applications/Beaver-notes.app/Contents/MacOS/Beaver-notes
Identifier:          com.danielerolli.beaver-notes
Version:             2.2.2 (2.2.2)
Code Type:           X86-64 (Native)
Role:                Default
Parent Process:      launchd [1]
Coalition:           com.danielerolli.beaver-notes [47677]

Date/Time:           2023-12-21 20:00:29.5099 +0800
Launch Time:         2023-12-21 20:00:29.3113 +0800
OS Version:          macOS 12.7.2 (21G1974)
Release Type:        User
Report Version:      104

Exception Type:  EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Reason: CODESIGNING 1

Triggered by Thread:  0

Thread 0 Crashed:
0                                	       0x10d967ae0 _dyld_start + 0
1   ???                           	       0x10a791000 ???

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000000  rcx: 0x0000000000000000  rdx: 0x0000000000000000
  rdi: 0x0000000000000000  rsi: 0x0000000000000000  rbp: 0x0000000000000000  rsp: 0x00007ff7b576ebd8
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000000
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000000000000000  r15: 0x0000000000000000
  rip: 0x000000010d967ae0  rfl: 0x0000000000000200  cr2: 0x0000000000000000

Logical CPU:     0
Error Code:      0x00000000
Trap Number:     0

Binary Images:
       0x10d963000 -        0x10d9cefff  (*) <499010ac-3054-326e-a050-fefffb5ce89a>
       0x10a791000 -        0x10a7e0fff ??? (*) <ab3cb5d1-eb3f-388f-8c63-416a00da1aaa> ???

 I'll try and find a crash log too.
Found it! Also, seems another person also experienced a possible crash.
Hope you can fix it  ✨

Translated Report (Full Report Below)
Incident Identifier: AB8A9B14-4512-46B7-B683-33819ECD141B
CrashReporter Key:   10C707BE-1865-001F-4317-34874584DA3B
Hardware Model:      MacBookPro11,5
Process:             Beaver-notes [11780]
Path:                /Applications/Beaver-notes.app/Contents/MacOS/Beaver-notes
Identifier:          com.danielerolli.beaver-notes
Version:             2.2.2 (2.2.2)
Code Type:           X86-64 (Native)
Role:                Default
Parent Process:      launchd [1]
Coalition:           com.danielerolli.beaver-notes [6462]
Date/Time:           2023-12-19 05:02:38.7083 +0100
Launch Time:         2023-12-19 05:02:38.6163 +0100
OS Version:          macOS 14.0 (23A344)
Release Type:        User
Report Version:      104
Exception Type:  EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: CODESIGNING 1 Taskgated Invalid Signature
Triggered by Thread:  0
Thread 0 Crashed:
0                                 	       0x10aa6d040 _dyld_start + 0
1                                	       0x109510000 ???
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000  rbx: 0x0000000000000000  rcx: 0x0000000000000000  rdx: 0x0000000000000000
rdi: 0x0000000000000000  rsi: 0x0000000000000000  rbp: 0x0000000000000000  rsp: 0x00007ff7b69efb88
r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000000
r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000000000000000  r15: 0x0000000000000000
rip: 0x000000010aa6d040  rfl: 0x0000000000000200  cr2: 0x0000000000000000
Logical CPU:     0
Error Code:      0x00000000
Trap Number:     0
Binary Images:
0x10aa68000 -        0x10ab07fff  () <1289b60a-4980-342d-b1a4-250bbee392f1> ???
0x109510000 -        0x10955ffff  ()
0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
Error Formulating Crash Report:
dyld_process_snapshot_get_shared_cache failed
EOF

Full Report
{"app_name":"Beaver-notes","timestamp":"2023-12-19 05:02:38.00 +0100","app_version":"2.2.2","slice_uuid":"ab3cb5d1-eb3f-388f-8c63-416a00da1aaa","build_version":"2.2.2","platform":0,"bundleID":"com.danielerolli.beaver-notes","share_with_app_devs":0,"is_first_party":0,"bug_type":"309","os_version":"macOS 14.0 (23A344)","roots_installed":0,"name":"Beaver-notes","incident_id":"AB8A9B14-4512-46B7-B683-33819ECD141B"}
{
"uptime" : 36000,
"procRole" : "Default",
"version" : 2,
"userID" : 502,
"deployVersion" : 210,
"modelCode" : "MacBookPro11,5",
"coalitionID" : 6462,
"osVersion" : {
"train" : "macOS 14.0",
"build" : "23A344",
"releaseType" : "User"
},
"captureTime" : "2023-12-19 05:02:38.7083 +0100",
"codeSigningMonitor" : 0,
"incident" : "AB8A9B14-4512-46B7-B683-33819ECD141B",
"pid" : 11780,
"cpuType" : "X86-64",
"roots_installed" : 0,
"bug_type" : "309",
"procLaunch" : "2023-12-19 05:02:38.6163 +0100",
"procStartAbsTime" :<PHONE_NUMBER>5535,
"procExitAbsTime" :<PHONE_NUMBER>6114,
"procName" : "Beaver-notes",
"procPath" : "/Applications/Beaver-notes.app/Contents/MacOS/Beaver-notes",
"bundleInfo" : {"CFBundleShortVersionString":"2.2.2","CFBundleVersion":"2.2.2","CFBundleIdentifier":"com.danielerolli.beaver-notes"},
"storeInfo" : {"deviceIdentifierForVendor":"CB9ED85E-ABDD-5A5C-ABDB-B6B4372163C5","thirdParty":true},
"parentProc" : "launchd",
"parentPid" : 1,
"coalitionName" : "com.danielerolli.beaver-notes",
"crashReporterKey" : "10C707BE-1865-001F-4317-34874584DA3B",
"codeSigningID" : "",
"codeSigningTeamID" : "",
"codeSigningFlags" : 16777216,
"codeSigningValidationCategory" : 0,
"codeSigningTrustLevel" :<PHONE_NUMBER>,
"wakeTime" : 9166,
"sleepWakeUUID" : "E2AF9A43-788F-4907-B6DE-78FD4FAB7F8F",
"sip" : "enabled",
"exception" : {"codes":"0x0000000000000000, 0x0000000000000000","rawCodes":[0,0],"type":"EXC_CRASH","signal":"SIGKILL (Code Signature Invalid)"},
"termination" : {"flags":66,"code":1,"namespace":"CODESIGNING","indicator":"Taskgated Invalid Signature"},
"extMods" : {"caller":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"system":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"targeted":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"warnings":0},
"faultingThread" : 0,
"threads" : [{"triggered":true,"id":307241,"threadState":{"r13":{"value":0},"rax":{"value":0},"rflags":{"value":512},"cpu":{"value":0},"r14":{"value":0},"rsi":{"value":0},"r8":{"value":0},"cr2":{"value":0},"rdx":{"value":0},"r10":{"value":0},"r9":{"value":0},"r15":{"value":0},"rbx":{"value":0},"trap":{"value":0},"err":{"value":0},"r11":{"value":0},"rip":{"value":4473671744,"matchesCrashFrame":1},"rbp":{"value":0},"rsp":{"value":140701897522056},"r12":{"value":0},"rcx":{"value":0},"flavor":"x86_THREAD_STATE","rdi":{"value":0}},"frames":[{"imageOffset":20544,"symbol":"_dyld_start","symbolLocation":0,"imageIndex":0},{"imageOffset":0,"imageIndex":1}]}],
"usedImages" : [
{
"source" : "P",
"arch" : "x86_64",
"base" :<PHONE_NUMBER>,
"size" : 655360,
"uuid" : "1289b60a-4980-342d-b1a4-250bbee392f1",
"name" : ""
},
{
"source" : "P",
"arch" : "x86_64",
"base" :<PHONE_NUMBER>,
"size" : 327680,
"uuid" : "ab3cb5d1-eb3f-388f-8c63-416a00da1aaa",
"path" : "\u001f",
"name" : "\u001f"
},
{
"size" : 0,
"source" : "A",
"base" : 0,
"uuid" : "00000000-0000-0000-0000-000000000000"
}
],
"vmSummary" : "ReadOnly portion of Libraries: Total=1336K resident=0K(0%) swapped_out_or_unallocated=1336K(100%)\nWritable regions: Total=8192K written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=8192K(100%)\n\n                                VIRTUAL   REGION \nREGION TYPE                        SIZE    COUNT (non-coalesced) \n===========                     =======  ======= \nSTACK GUARD                       56.0M        1 \nStack                             8192K        1 \nVM_ALLOCATE                          4K        1 \n__DATA                              32K        2 \n__DATA_CONST                        32K        1 \n__DATA_DIRTY                        16K        1 \n__LINKEDIT                         384K        3 \n__TEXT                             960K        2 \nshared memory                        4K        1 \n===========                     =======  ======= \nTOTAL                             65.4M       13 \n",
"legacyInfo" : {
"threadTriggered" : {
}
},
"logWritingSignature" : "42b77dc67293b36d88e964b69d4a2e5fda98f2e6",
"trialInfo" : {
"rollouts" : [
{
"rolloutId" : "60356660bbe37970735c5624",
"factorPackIds" : {
  },
  "deploymentId" : 240000027
},
{
  "rolloutId" : "6246d6a916a70b047e454124",
  "factorPackIds" : {

  },
  "deploymentId" : 240000010
}

],
"experiments" : [
]
},
"reportNotes" : [
"dyld_process_snapshot_get_shared_cache failed"
]
}
Hardware
Modelnaam:	MacBook Pro
Model-ID:	MacBookPro11,5
Processor:	2,5 GHz Quad-Core Intel Core i7
processors:	1
Total cores:	4
L3-cache:	6 MB
Ram:	        16 GB
Mac OS:          14.0 (23A344)

Thanks for reporting this issue. I’ll be fixing it soon. It boils down to the code signing certificate expiring, as I don’t have an Apple Developer account. The easy fix is to buy Apple’s developer account, which I will do. Thanks for your understanding.

I've rebuilt the packages for both macOS on Apple Silicon and x86. It should work now. I hope you'll enjoy Beaver Notes despite this tiny issue.

No worries! I know how development is.
Great! I just tried it. It does startup, but the screen seems to be in debug mode?

When closing the debug mode, the screen stays white/empty.

Yeah, sorry about that. I forgot to reinstall the dependencies before rebuilding the package. Try downloading it now and see if it works. Sorry again for the inconvenience.

Success! haha

Great job, my friend


Can not build ESP-IDF A2DP-SINK demo on ESP32 C3 (IDFGH-5008)

Can not build ESP-IDF A2DP-SINK demo on ESP32 C3.
While building following error comes up -
.../hci_layer.c: In function 'hci_layer_init_env':
.../hci_layer.c:184:5: error: implicit declaration of function 'btsnd_hcic_ble_sync_sem_init'; did you mean 'btsnd_hcic_ble_set_phy'? [-Werror=implicit-function-declaration]
btsnd_hcic_ble_sync_sem_init();
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
btsnd_hcic_ble_set_phy
Is ESP32 C3 not supported for this Example?

Hi, @jassi00713
ESP32-C3 supports Low Energy Bluetooth and Wi-Fi, doesn't support Classic BT.
For more information please refer to ESP32-C3.
Thanks


Bug: Sort By Size Not Working

Describe the bug
Under snapshot when you click on size either once or twice it does not sort by size but does reorder the list of snapshots.
Steps to reproduce
Go to snapshot click on size either once or twice
Screenshots or log output

ArchiveBox version
v0.8.2

That is correct, size is not stored in DB as it's an expensive filesystem calculation, it's computed on the fly and cached only for the snapshots currently visible on the page. When you sort by size it actually sorts by number of output results recorded, which is the closest approximation that's still reasonably fast.
If you really need to sort by size I suggest just sorting the data/archive/ dir in your filesystem browser or using a CLI tool like Ncdu (both of which you'll notice take several minutes to compute sizes for everything).

The displayed sizes are accurate but it's doing a trick where it only calculates them for the page you're actively viewing (<100 snapshots at a time). The sorting needs to operate across all pages matching your active filters, so it cant be based on the displayed sizes in the current architecture without having to calculate the size of every single snapshot in the entire dataset.


Inverse of a matrix #40

Issue #40
Briefly explain your program logic -
made a calculator to find inverse of a matrix using react js and also use a library numeric which helps in finding inverse of matrix , math library is giving some issue but numeric resolves that
Screenshots(Attach 2 screenshots of your own input and output) -

By raising this PR I affirm that -

[x] My code follows the guidelines of this project.

[x] I have performed a self-review of my own code.

[x] I have commented my code.

[x] My code gives the correct output.

[x] I affirm that I strictly follow contributing guidelines and code of conduct.

Congratulations 🎉 @Sam-bhav-20 on your contribution.
Thanks for adding value to this repository. 😊


fix: readded imported privileges special case for database grants

readded imported privileges special case for database grants

It was removed in #1522, however it is needed for the edge case of shared databases (e.g. the snowflake database). In order to give custom roles the permission to use the shared databases, the privilege must be given "IMPORTED PRIVILEGES". However, in the information_schema tables the privilege is shown as "USAGE".
Without this code snippet, the terraform provider wants to regive this permission every time.
Test Plan

[x] unit tests
[x] manual test with snowflake

References
(https://docs.snowflake.com/en/user-guide/data-exchange-marketplace-privileges)

@sfc-gh-swinkler I would be great, if this small fix could be included in the next release as this bug is quite annoying ;)

@sfc-gh-swinkler Any chance, if this fix could be merged? The bug is a little anoying

@Relativity74205. Thank you, i didn't understand why this was in there previously.


Set cursor to end of text [input field]

Sometimes you want to focus an input field that has a value.

This looks an awful lot like angular-focus-caret which was brought up in pull request 10.


Examples for bit flipping

I have written an example for most of the uses of bit flipping mentioned. I have also labeled which use they are.

Well done @CBaetz!  Thanks for your contribution here!
However, I find it extremely unusual that you'd add 6 commits for something so small.  Also if you forget one character, such as the * that you forgot in one of your commits, it's best to catch that before you commit anything.
Perhaps you can try to squash your 6 commits into 1, before we merge your fork into the Master branch. You might even want to practice/test this in a separate repository of your own (which has no relation to this repository).
Other than the unusual number of commits here, you've done well on this task though!

Hi @CBaetz, I still haven't accepted this pull request for a merge into the master branch, since I think it would be a good idea for you to first squash your commits (just push to this branch of your fork, and the pull request will be automatically updated). This would also be a good learning experience for you, meaning that if a situation like this were to ever come up in a work/team setting (whether with HPQC Labs or with anyone else), you would know what to do.

I have squashed all the commits into only 2 and changed the commit message accordingly.

Thank you so much for squashing the commits! The pull request and its associated code additions, are now much easier to see. I'll take a final look as soon as I have time, and then hope to merge your contribution into the Master branch.


frost-scroll not showing icon in demo app

The add symbol is not showing in the frost-scroll demo app: http://ciena-frost.github.io/ember-frost-core/#/scroll

Closed via: https://github.com/ciena-frost/ember-frost-core/pull/346


Can gocron work with gin

I want to use gocron with gin router. But it seems that gin router and gocron block each other.
If i start goncron before router.Run, the genData will be done, but can not get response from /ex/getdata. If i start router.Run before gocron.Start(), I can get response from /ex/getdata but genData job is not executed.
func main() {
data := &Data{}
router := gin.Default()
router.GET("/ex/getdata", getData)
gocron.Every(1).Second().Do(genData)
<-gocron.Start()
router.Run(":8888")

}

I have same problem.

I back with github.com/robfig/cron and all work fine

i have same problem

I backed with github.com/robfig/cron and all work fine

could you mind to give sample implementation about it?

Please refer my sample at https://github.com/huyntsgs/exercise/blob/master/main.go#L215 and https://github.com/huyntsgs/exercise/blob/master/main.go#L63

Please refer my sample at https://github.com/huyntsgs/exercise/blob/master/main.go#L215 and https://github.com/huyntsgs/exercise/blob/master/main.go#L63

how about every (time) get data from db and send broadcast notif with gcm. @huyntsgs


Title of preferances shows as "Webpack App"

Describe the bug
The title of the preferences window is incorrect, at least on windows, (don't have Mac or Linux available to test)
Versions
<IP_ADDRESS>
To Reproduce
Steps to reproduce the behavior:

Open Ray
Goto Menu
Select Preferences
Observe the title Webpack App

Expected behavior
I believe it should either be named Ray, Preferences or Ray Preferences.
Screenshots

Desktop (please complete the following information):

OS: Windows
Version 11

Still an issue

@Sam-Apostel could you take a look?

@Sam-Apostel could you take a look?

<IP_ADDRESS> Image here as to update from the original images :)


Domain specific interface for (aircraft) VMS systems

(Copied from https://sourceforge.net/p/tulip-control/feature-requests/9/ with creation date 2012-05-30, by @murrayrm and assigned to Mumu Xu.)
COMMENTS copied from https://sourceforge.net/p/tulip-control/feature-requests/9/:
2012-05-30 by @murrayrm: Since one of the applications that we are pursue at Caltech is the application of TuLiP to aircraft vehicle management systems (including electric power systems), it would be nice to create a domain-specific interface tuned to this application. The idea would be to have functions that automatically generate environmental descriptions and specifications based on something like a line drawing of an electrical, hydraulic or pneumatic system.
2012-05-31 by @murrayrm: Robert Rogersten will be doing something related to this: going from netlists to specs.
Assignment was changed from nobody to mumuxu on 2012-06-07.

I think that the specification generation code under contrib/AES as of commit 02a407765f2f4ca995a6691587e8170fdbd385e6 is sufficient to close this issue. If additional features are desired, then open issues that are narrower in scope.


DBZ-5985 Disable to print invalid estimated row count when config "snapshot.select.statement.overrides"

https://issues.redhat.com/browse/DBZ-5985

@harveyyue Applied, thanks!


Added a loop on line 82 that trains 10 times

Find by going cmd-F or ctrl-F and typing in "asdfasdfasdf"
You can change the value i has to reach to train more times or less.
Streaks get all weird, not because of an error, but because you're training more that one time before the computer has time to update the screen.
Oh wait, never mind, you can auto-train.  Did you save the file with different line endings?

I think this is a very crucial addition and is highly unlikely to cause merge conflicts in the future. Cary is also very diligent on maintaining his github so I have no doubt this will be merged very soon.

Just want to slide in here with a "You cucked up and Cary doesn't use GitHub anyway". Thank you for your time,


Add Lambda function sample Jupyter notebook

Issue #, if available:
N/A.
Description of changes:

Sample Jupyter notebook added for ingesting data into Timestream for LiveAnalytics with a Lambda function.

Generates data using either ten kinds of time series data generators or using a user-defined schema.
Deploys a Lambda function that receives the generated data and ingests data into Timestream for LiveAnalytics.
Creates an Amazon Managed Grafana workspace.
Generates and uploads a Grafana dashboard to the workspace to view the ingested time series data.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Here is what the dashboard currently looks like with DevOps data. Each measure is given its own panel. The dashboard is flexible and accommodates any data schema.


Minor fixes

@onli, some minor fixes:

switched URL from the Makefile and Dockerfile
clarified inheritance in the README
updated image metadata in the Makefile

You fixed the version to 2.0.1 in there, but I think we should move it to 2.0.2.. I asked in the project to a link to the always newest version (the one used so far does not work anymore because we switched to github). I will merge it, but think that part should be changed later. Or am I missing something?

Yes, I switched to an hardcoded version so we can upgrade the version manually in the Dockerfile and in the Makefile, so the tags are also up to date
It's also easier for future developments to maintain a changelog if we want to


No TS types respected of data

When using useCsrfFetch, you cant specify the return data type:
const {
  data,
  error,
  status,
} = await useCsrfFetch<Log[]>('/api/admin/logs');

Expected:

For the record: Building your own useFetch function and implementing the csrf token in the header is the best solution

@Morgbn HI,
I apologize for the interruption.
I noticed that this issue has been marked as closed, yet it persists in the latest version I am using. Therefore, I would like to inquire whether it has not yet been addressed in the most recent release.


Provide "outdated" Repository information

Please show a little icon/symbol in the repository overview that the remote branch "newer" than the local branch.
You can implement it by an periodical "fetch" and use these information.
The interval of the fetch should be configurable.
This feature should be disabled by default.

Available from version 4.0 and up :heart:

Aww, this issue got mentioned on the changelog by my similar (and prior https://github.com/awaescher/RepoZ/issues/45) issue didn't.  Feels bad man.

Oh my gosh @ScottRFrost, you're right. I will definitely catch up on that. Man, how exactly you specified the feature the way I implemented it ...

... the one thing I'd like to see is a way to fetch from all repos, either automatically every {configurable} minutes ...

Thank's for the hint.


runpod accelerate launch -m axolotl.cli.train examples/mistral/config.yml --deepspeed deepspeed/zero2.json error

Please check that this issue hasn't been reported before.

[X] I searched previous Bug Reports didn't find any similar reports.

Expected Behavior
According to the README:
If you run into CUDA OOM, use deepspeed with config zero2.json:
accelerate launch -m axolotl.cli.train examples/mistral/config.yml --deepspeed deepspeed/zero2.json

Should use deepspeed to lower vram usage.
Current behaviour
Here is my output when I run the command, adapted with my config.yml path, from the readme in a runpod container with 96GB of VRAM:
!accelerate launch -m axolotl.cli.train config.yml --deepspeed deepspeed/zero2.json

The following values were not passed to `accelerate launch` and had defaults used instead:
	`--num_processes` was set to a value of `2`
		More than one GPU was found, enabling multi-GPU training.
		If this was unintended please pass in `--num_processes=1`.
	`--num_machines` was set to a value of `1`
	`--mixed_precision` was set to a value of `'no'`
	`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/bitsandbytes-0.41.1-py3.10.egg/bitsandbytes/cuda_setup/main.py:106: UserWarning:

================================================================================
WARNING: Manual override via BNB_CUDA_VERSION env variable detected!
BNB_CUDA_VERSION=XXX can be used to load a bitsandbytes version that is different from the PyTorch CUDA version.
If this was unintended set the BNB_CUDA_VERSION variable to an empty string: export BNB_CUDA_VERSION=
If you use the manual override make sure the right libcudart.so is in your LD_LIBRARY_PATH
For example by adding the following to your .bashrc: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_cuda_dir/lib64
Loading CUDA version: BNB_CUDA_VERSION=118
================================================================================

  warn((f'\n\n{"="*80}\n'
/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/bitsandbytes-0.41.1-py3.10.egg/bitsandbytes/cuda_setup/main.py:106: UserWarning:

================================================================================
WARNING: Manual override via BNB_CUDA_VERSION env variable detected!
BNB_CUDA_VERSION=XXX can be used to load a bitsandbytes version that is different from the PyTorch CUDA version.
If this was unintended set the BNB_CUDA_VERSION variable to an empty string: export BNB_CUDA_VERSION=
If you use the manual override make sure the right libcudart.so is in your LD_LIBRARY_PATH
For example by adding the following to your .bashrc: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_cuda_dir/lib64
Loading CUDA version: BNB_CUDA_VERSION=118
================================================================================

  warn((f'\n\n{"="*80}\n'
                              dP            dP   dP
                              88            88   88
   .d8888b. dP.  .dP .d8888b. 88 .d8888b. d8888P 88
   88'  `88  `8bd8'  88'  `88 88 88'  `88   88   88
   88.  .88  .d88b.  88.  .88 88 88.  .88   88   88
   `88888P8 dP'  `dP `88888P' dP `88888P'   dP   dP

[2023-10-05 19:41:31,119] [WARNING] [axolotl.validate_config:163] [PID:3646] [RANK:0] eval_batch_size != micro_batch_size. This can lead to VRAM instability.
[2023-10-05 19:41:31,126] [WARNING] [axolotl.validate_config:163] [PID:3647] [RANK:1] eval_batch_size != micro_batch_size. This can lead to VRAM instability.
[2023-10-05 19:41:31,241] [INFO] [axolotl.normalize_config:122] [PID:3646] [RANK:0] GPU memory usage baseline: 0.000GB (+0.455GB misc)
[2023-10-05 19:41:31,241] [WARNING] [axolotl.scripts.check_user_token:261] [PID:3646] [RANK:0] Error verifying HuggingFace token. Remember to log in using `huggingface-cli login` and get your access token from https://huggingface.co/settings/tokens if you want to use gated models or datasets.
[2023-10-05 19:41:31,249] [INFO] [axolotl.normalize_config:122] [PID:3647] [RANK:1] GPU memory usage baseline: 0.000GB (+0.455GB misc)
[2023-10-05 19:41:31,249] [WARNING] [axolotl.scripts.check_user_token:261] [PID:3647] [RANK:1] Error verifying HuggingFace token. Remember to log in using `huggingface-cli login` and get your access token from https://huggingface.co/settings/tokens if you want to use gated models or datasets.
[2023-10-05 19:41:31,418] [DEBUG] [axolotl.load_tokenizer:75] [PID:3646] [RANK:0] EOS: 2 / </s>
[2023-10-05 19:41:31,418] [DEBUG] [axolotl.load_tokenizer:76] [PID:3646] [RANK:0] BOS: 1 / <s>
[2023-10-05 19:41:31,419] [DEBUG] [axolotl.load_tokenizer:77] [PID:3646] [RANK:0] PAD: 2 / </s>
[2023-10-05 19:41:31,419] [DEBUG] [axolotl.load_tokenizer:78] [PID:3646] [RANK:0] UNK: 0 / <unk>
[2023-10-05 19:41:31,473] [DEBUG] [axolotl.load_tokenizer:75] [PID:3647] [RANK:1] EOS: 2 / </s>
[2023-10-05 19:41:31,473] [DEBUG] [axolotl.load_tokenizer:76] [PID:3647] [RANK:1] BOS: 1 / <s>
[2023-10-05 19:41:31,473] [DEBUG] [axolotl.load_tokenizer:77] [PID:3647] [RANK:1] PAD: 2 / </s>
[2023-10-05 19:41:31,473] [DEBUG] [axolotl.load_tokenizer:78] [PID:3647] [RANK:1] UNK: 0 / <unk>
[2023-10-05 19:41:31,723] [INFO] [axolotl.load_tokenized_prepared_datasets:130] [PID:3646] [RANK:0] Unable to find prepared dataset in last_run_prepared/79fe5144e8e385dc65045e15b51b2838
[2023-10-05 19:41:31,723] [INFO] [axolotl.load_tokenized_prepared_datasets:131] [PID:3646] [RANK:0] Loading raw datasets...
[2023-10-05 19:41:31,723] [INFO] [axolotl.load_tokenized_prepared_datasets:136] [PID:3646] [RANK:0] No seed provided, using default seed of 42
Map (num_proc=64): 100%|███████████| 2000/2000 [00:01<00:00, 1677.16 examples/s]
[2023-10-05 19:41:36,515] [INFO] [axolotl.load_tokenized_prepared_datasets:354] [PID:3646] [RANK:0] merging datasets
[2023-10-05 19:41:38,973] [INFO] [axolotl.load_tokenized_prepared_datasets:130] [PID:3647] [RANK:1] Unable to find prepared dataset in last_run_prepared/79fe5144e8e385dc65045e15b51b2838
[2023-10-05 19:41:38,974] [INFO] [axolotl.load_tokenized_prepared_datasets:131] [PID:3647] [RANK:1] Loading raw datasets...
[2023-10-05 19:41:38,974] [INFO] [axolotl.load_tokenized_prepared_datasets:136] [PID:3647] [RANK:1] No seed provided, using default seed of 42
Map (num_proc=64): 100%|███████████| 2000/2000 [00:01<00:00, 1425.44 examples/s]
[2023-10-05 19:41:46,305] [INFO] [axolotl.load_tokenized_prepared_datasets:354] [PID:3647] [RANK:1] merging datasets
Filter (num_proc=64): 100%|████████| 1980/1980 [00:00<00:00, 2691.62 examples/s]
Filter (num_proc=20): 100%|██████████████| 20/20 [00:00<00:00, 48.95 examples/s]
[2023-10-05 19:41:52,758] [INFO] [axolotl.calculate_total_num_steps:526] [PID:3646] [RANK:0] total_num_steps: 372
[2023-10-05 19:41:52,772] [INFO] [axolotl.train.train:48] [PID:3646] [RANK:0] loading tokenizer... mistralai/Mistral-7B-v0.1
[2023-10-05 19:41:52,963] [DEBUG] [axolotl.load_tokenizer:75] [PID:3646] [RANK:0] EOS: 2 / </s>
[2023-10-05 19:41:52,964] [DEBUG] [axolotl.load_tokenizer:76] [PID:3646] [RANK:0] BOS: 1 / <s>
[2023-10-05 19:41:52,964] [DEBUG] [axolotl.load_tokenizer:77] [PID:3646] [RANK:0] PAD: 2 / </s>
[2023-10-05 19:41:52,964] [DEBUG] [axolotl.load_tokenizer:78] [PID:3646] [RANK:0] UNK: 0 / <unk>
[2023-10-05 19:41:53,267] [INFO] [axolotl.train.train:56] [PID:3646] [RANK:0] loading model and (optionally) peft_config...
Filter (num_proc=64): 100%|████████| 1980/1980 [00:00<00:00, 2360.42 examples/s]
Filter (num_proc=20): 100%|██████████████| 20/20 [00:00<00:00, 58.73 examples/s]
[2023-10-05 19:41:59,075] [INFO] [axolotl.calculate_total_num_steps:526] [PID:3647] [RANK:1] total_num_steps: 372
[2023-10-05 19:41:59,087] [INFO] [axolotl.train.train:48] [PID:3647] [RANK:1] loading tokenizer... mistralai/Mistral-7B-v0.1
[2023-10-05 19:41:59,278] [DEBUG] [axolotl.load_tokenizer:75] [PID:3647] [RANK:1] EOS: 2 / </s>
[2023-10-05 19:41:59,278] [DEBUG] [axolotl.load_tokenizer:76] [PID:3647] [RANK:1] BOS: 1 / <s>
[2023-10-05 19:41:59,278] [DEBUG] [axolotl.load_tokenizer:77] [PID:3647] [RANK:1] PAD: 2 / </s>
[2023-10-05 19:41:59,278] [DEBUG] [axolotl.load_tokenizer:78] [PID:3647] [RANK:1] UNK: 0 / <unk>
[2023-10-05 19:41:59,575] [INFO] [axolotl.train.train:56] [PID:3647] [RANK:1] loading model and (optionally) peft_config...
Loading checkpoint shards: 100%|██████████████████| 2/2 [00:18<00:00,  9.06s/it]
[2023-10-05 19:42:14,140] [INFO] [axolotl.load_model:364] [PID:3646] [RANK:0] GPU memory usage after model load: 13.989GB (+0.251GB cache, +1.359GB misc)
Traceback (most recent call last):
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 52, in __init__
    config_decoded = base64.urlsafe_b64decode(config_file_or_dict).decode("utf-8")
  File "/root/miniconda3/envs/py3.10/lib/python3.10/base64.py", line 133, in urlsafe_b64decode
    return b64decode(s)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/miniconda3/envs/py3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/miniconda3/envs/py3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 38, in <module>
    fire.Fire(do_cli)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 34, in do_cli
    train(cfg=parsed_cfg, cli_args=parsed_cli_args, dataset_meta=dataset_meta)
  File "/workspace/axolotl/src/axolotl/train.py", line 76, in train
    trainer = setup_trainer(
  File "/workspace/axolotl/src/axolotl/utils/trainer.py", line 667, in setup_trainer
    training_args = AxolotlTrainingArguments(  # pylint: disable=unexpected-keyword-arg
  File "<string>", line 128, in __init__
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/transformers/training_args.py", line 1693, in __post_init__
    self.hf_deepspeed_config = HfTrainerDeepSpeedConfig(self.deepspeed)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/transformers/integrations/deepspeed.py", line 88, in __init__
    super().__init__(config_file_or_dict)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/transformers/integrations/deepspeed.py", line 78, in __init__
    super().__init__(config_file_or_dict)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 55, in __init__
    raise ValueError(
ValueError: Expected a string path to an existing deepspeed config, or a dictionary, or a base64 encoded string. Received: deepspeed/zero2.json
Loading checkpoint shards: 100%|██████████████████| 2/2 [00:14<00:00,  7.35s/it]
[2023-10-05 19:42:16,626] [INFO] [axolotl.load_model:364] [PID:3647] [RANK:1] GPU memory usage after model load: 13.989GB (+0.251GB cache, +1.359GB misc)
Traceback (most recent call last):
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 52, in __init__
    config_decoded = base64.urlsafe_b64decode(config_file_or_dict).decode("utf-8")
  File "/root/miniconda3/envs/py3.10/lib/python3.10/base64.py", line 133, in urlsafe_b64decode
    return b64decode(s)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/miniconda3/envs/py3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/miniconda3/envs/py3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 38, in <module>
    fire.Fire(do_cli)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/workspace/axolotl/src/axolotl/cli/train.py", line 34, in do_cli
    train(cfg=parsed_cfg, cli_args=parsed_cli_args, dataset_meta=dataset_meta)
  File "/workspace/axolotl/src/axolotl/train.py", line 76, in train
    trainer = setup_trainer(
  File "/workspace/axolotl/src/axolotl/utils/trainer.py", line 667, in setup_trainer
    training_args = AxolotlTrainingArguments(  # pylint: disable=unexpected-keyword-arg
  File "<string>", line 128, in __init__
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/transformers/training_args.py", line 1693, in __post_init__
    self.hf_deepspeed_config = HfTrainerDeepSpeedConfig(self.deepspeed)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/transformers/integrations/deepspeed.py", line 88, in __init__
    super().__init__(config_file_or_dict)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/transformers/integrations/deepspeed.py", line 78, in __init__
    super().__init__(config_file_or_dict)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 55, in __init__
    raise ValueError(
ValueError: Expected a string path to an existing deepspeed config, or a dictionary, or a base64 encoded string. Received: deepspeed/zero2.json
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 3646) of binary: /root/miniconda3/envs/py3.10/bin/python
Traceback (most recent call last):
  File "/root/miniconda3/envs/py3.10/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
    args.func(args)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/commands/launch.py", line 977, in launch_command
    multi_gpu_launcher(args)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/accelerate/commands/launch.py", line 646, in multi_gpu_launcher
    distrib_run.run(args)
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/root/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
axolotl.cli.train FAILED

Steps to reproduce

Run the axolotl runpod container.
Edit the mistral config.yml with my dataset
Changed alpaca_chat.py in axolotl, and axolotl-update dir from lowercase "question" to uppercase "QUESTION" same with answer.
Paste my edited config.yml in /workspace
Run shell command provided.

Config yaml
base_model: mistralai/Mistral-7B-v0.1
base_model_config: mistralai/Mistral-7B-v0.1
model_type: MistralForCausalLM
tokenizer_type: LlamaTokenizer
is_mistral_derived_model: true
load_in_8bit: false
load_in_4bit: false
strict: false
datasets:

path: cvinker/lit_test
type: alpaca_chat.load_qa
dataset_prepared_path:
val_set_size: 0.01
output_dir: ./out

sequence_len: 8192
sample_packing:
pad_to_sequence_len:
wandb_project: axolotl_1
wandb_entity:
wandb_watch:
wandb_run_id:
wandb_log_model:
gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 3
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: false
group_by_length: false
bf16: true
fp16: false
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 10
eval_steps: 20
eval_table_size: 5
eval_table_max_new_tokens: 128
save_steps:
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
bos_token: ""
eos_token: ""
unk_token: ""
Possible solution
Appears to be related to not knowing the location of the deepspeed config, and some padding complaints.
Which Operating Systems are you using?

[ ] Linux
[ ] macOS
[ ] Windows

Python Version
3.10
axolotl branch-commit
main
Acknowledgements

[X] My issue title is concise, descriptive, and in title casing.
[X] I have searched the existing issues to make sure this bug has not been reported yet.
[X] I am using the latest version of axolotl.
[X] I have provided enough information for the maintainers to reproduce and diagnose the issue.

You should put the path to the deepspeed config in your deepspeed: field

Hello, is this problem still occurring?

I have the same problem using zero2. Zero 1 works

Just a note to future readers, the directory has been renamed to deepspeed_configs. Will be closing as its stale unless can be reproduced.


plugin crashes the app

after adding plugin "flutter_facebook_login: ^1.1.1" or "flutter_facebook_login: ^1.1.1",
fail to build the apk, causing an immediate exit when open the app.
Built build/app/outputs/apk/debug/app-debug.apk.
after this step, nothing comes out.

Did you remember to follow these installation instructions in the README?
https://github.com/roughike/flutter_facebook_login/tree/bugfix/resolve-ios-build-issues#installation
If you didn't touch your AndroidManifest.xml, it will result in a crash.

Oh, Thank you. I will try it later.

Closing due to not getting a response, and due to fact that not following the installation instructions was the case for another user having the same issue :)
Feel free to comment if the problem persists.

It would be nice if those additional installation instructions were found here.
https://pub.dartlang.org/packages/flutter_facebook_login#-installing-tab-

That is something I can't change - pub doesn't allow customizing that section for package maintainers.

In android/gradle.properties i added:
android.useAndroidX=true
android.enableJetifier=true

and stop to crash...


sl-select doesn't support dynamically removing items from content

If the selected item is dynamically removed from the array bound to content, than the sl-select component becomes unusable and acts as if it is disabled. Dynamically adding items to the array that is bound to content works fine.

@Samsinite I'm unable to reproduce the disabled part of the issue. When removing an element from content, the selector still works, although it is now selecting an element that does not exist.

@alxyuu this was from whatever version was latest on Dec 2014. I've since stopped using sl-ember-components so I am unsure if this issue still exists.

@Samsinite If you would not mind sharing, what is the reason you stopped using sl-ember-components?  If it is something we can change we would be interested in the information so that we can make modifications in the repository moving forward.  Thank you.

Sure. I've outlined the main reasons below, some of which may be resolved by now.

Way too many components that I didn't regularly need or use.
Had too many issues with the select components using array promises and/or dynamically changing selection choices.
The bootstrap form components didn't seem to easily support form columns.
Some of the components required us to import all of the styling, thus requiring us to use your custom bootstrap theme which didn't always play nice with other bootstrap plugins outside of your library.

Overall, you guys built some impressive tools, it just seemed -- at the time at least -- more customized to your specific workflow and standards, which I can completely understand. Our company tends to do the same thing, and make libraries more generalized as they mature.

@alxyuu As you are unable to reproduce this I am going to close the issue.  Regarding the selection of a non-existent elemet, my recommendation normally would be to reset the selection to nothing or the label, but since we are going to be refactoring this entire component shortly, no such effort needs to/will be spent at this time.


Bullets Could Inherit Ship's Velocity

Problem

Bullets currently don't inherit the ship's velocity when fired.
With a thrust powerup, the ship is almost as fast as bullets.
Hence, bullets barely escape the ship's nozzle.
Velocity inheritance would also be more physically realistic.

Possible Fixes

Does the  Rigidbody API have any simple functions for this?

https://github.com/antfarmar/Unity-3D-Asteroids/blob/b9c0cbda61f6442a558a2b94945409720f054d78/Assets/Asteroids/Scripts/MicroBehaviours/ShipShooter.cs#L88-L93
Add the velocity of the rigidbody to the bullet.
And get rid of those comments I placed there while you are at it :)
Bullet().Fire(nozzle.position, nozzle.rotation, direction * velocity + rb.velocity);

Oh yeah, vector addition. Who knew rigidbodies had a velocity property? :laughing:
I also renamed the variables to distinguish scalars from vectors.
Commit a56a4ed
void FireBullet(Vector3 direction, float speedScalar = bulletSpeed)
{
        direction = (direction * speedScalar) + rbody.velocity;
        Bullet().Fire(nozzle.position, nozzle.rotation, direction);
}

Could probably also just collapse those 2 parameters to simply velocity and have the caller do the math.


Sweep AICLK frequency at fixed voltage

Try to find passing frequencies from entire AICLK range: 500MHz to 1GHz, at 13.5MHz steps.
Test setup:

Set voltage to something safe - 0.85V
Disable AICLK PPM (ARC doesn't throttle AICLK at all)
Run all three workloads with 300k loops
Start at 1GHz AICLK and walk the frequency down.
a. record failing loop iteration, if it's close to 300k then increase the loop count to 500k

Rerun the experiment with PPM enabled and see if board reliability related failures are affected.
Milos

Tracked through SYSENG ticket:
https://tenstorrent.atlassian.net/browse/SYS-752

V/F sweeps conducted and safe mode defined as AICLK = 500MHz, Voltage = 0.85V
Detailed results listed:
(N300, N150): https://docs.google.com/spreadsheets/d/10uWtBEkLLEM-h5TuuGQ6HW8AjhXwSjFV-cK3iVFYYIU/edit?pli=1&gid=341688123#gid=341688123
4U Galaxy: https://docs.google.com/spreadsheets/d/10uWtBEkLLEM-h5TuuGQ6HW8AjhXwSjFV-cK3iVFYYIU/edit?pli=1&gid=771792970#gid=771792970


Update config.yaml

👷 Deploy Preview for dat4lab processing.

Name
Link

🔨 Latest commit
506f15a5d2b6ccb03379629b3873b000d300c39d

🔍 Latest deploy log
https://app.netlify.com/sites/dat4lab/deploys/65243e4810f24800085d10a4


feat: Only save md5 in lock file if no sha256 is present

By saving only one of those two hashes, we reduce the lock file size a bit. This should be especially noticeable in git repos, since hashes compress poorly.
This PR is a contribution to improving https://github.com/prefix-dev/pixi/issues/1509

Could you give some statistics on file size or compresed file reductions for: https://github.com/rerun-io/rerun (e.g)?

@Hofer-Julian Maybe something like this to test the compression?
import zlib

# Step 1: Read the file contents
input_file_path = 'input.txt'
with open(input_file_path, 'rb') as f:
    file_data = f.read()

# Step 2: Compress the contents
compressed_data = zlib.compress(file_data)

# Step 3: Save the compressed data
output_file_path = 'output.txt.zlib'
with open(output_file_path, 'wb') as f:
    f.write(compressed_data)

print(f"Compressed file saved as {output_file_path}")

Could you give some statistics on file size or compressed file reductions for: https://github.com/rerun-io/rerun (e.g)?

Here you go:
nu ❯ ls --du | sort-by size | reverse
╭───┬────────────────────────────┬──────┬───────────┬───────────────╮
│ # │            name            │ type │   size    │   modified    │
├───┼────────────────────────────┼──────┼───────────┼───────────────┤
│ 0 │ conda-lock_with_md5        │ file │   1.7 MiB │ 8 minutes ago │
│ 1 │ conda-lock_without_md5     │ file │   1.6 MiB │ 8 minutes ago │
│ 2 │ conda-lock_with_md5.zip    │ file │ 241.0 KiB │ 2 minutes ago │
│ 3 │ conda-lock_without_md5.zip │ file │ 199.0 KiB │ 2 minutes ago │
╰───┴────────────────────────────┴──────┴───────────┴───────────────╯

The fraction between with and without md5 not zipped is 0.94.
The fraction between with and without md5 zipped is 0.82.
That indicates that the impact is indeed higher for compressed situations.

That's pretty significant for the compressed case :)

@tdejager @Hofer-Julian The reason we store all the metadata including the md5 and the sha256 is because a matchspec can depend on these values. E.g you can have torch[md5=123456789]. That is also the reason we store the license etc. I know this is an edge case but by removing this we can no longer statitically verify the lock-file correctly. We also break incremental updates.

Do we have examples of this being used?

Not from the top of my head no. But if we remove this field we can also remove a bunch of others: e.g. size, file_name, timestamp, etc.
But I would be opposed to that because it would potentially break a future incremental update of a lock-file. We have been advocating for storing the entire repodata record in the lock-file for exactly this reason.

See also the design goals and text about conparison with conda-lock: https://docs.rs/rattler_lock/latest/rattler_lock/#relation-to-conda-lock

Yes you are right, we might need to come up with alternative solutions though, we could revert this.
Because the large lock files are a problem :)


ENH populate model card metadata with info from config.json

This PR adds a convenience function to populate the metadata section of model cards from the info available in config.json.
I'll be adding tests, but I'm wondering what you think of the API @merveenoyan @BenjaminBossan ?

I think we can proceed in this direction. It makes sense to use what's already there in the config.json.

This finally makes the model work out of the box, example: https://huggingface.co/skops/hf_hub_example-bdc26c1f-7e82-42eb-9657-0318315f2df0
Note: the output is actually not reliable yet, for the model to get the right input, https://github.com/huggingface/api-inference-community/pull/79 needs to be merged first.
@osanseviero you might like this one ;)
Running plot_hf_hub.py (w/o the delete repo at the end) gives a nice example repo.

(There are some issues with the widget in linux, both Firefox and Chrome, and I've already told Mishig about them).

Indeed I like this one, very cool! :fire:


Correct Service User Permissions Checkboxes for Service Users

Bug Description
State of service user permissions is sometimes inaccurately displayed:

After saving a new service user, click edit and you may see some of the options you selected have been disconnected
New service user form may be populated with permissions data from the previous form

Steps to Reproduce

Impact

new UI


chore(tsc): simplify tsconfig.json using tsconfigs

I also diffed between the build artifacts of the master branch and this branch. No change. Specifically, I diffed both /snabbdom.js and /es/snabbdom.js


BUG: #17778 - DataFrame.to_pickle() fails for .zip format on MacOS and pandas 0.20.3

[x] closes #17778
[x] tests added / passed
[x] passes git diff upstream/master -u -- "*.py" | flake8 --diff
[x] whatsnew entry

Codecov Report

Merging #2 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master       #2      +/-   ##
==========================================
+ Coverage   91.23%   91.26%   +0.02%
==========================================
  Files         163      163
  Lines       50120    50129       +9
==========================================
+ Hits        45728    45749      +21
+ Misses       4392     4380      -12

Flag
Coverage Δ

#multiple
89.07% <100%> (+0.02%)
:arrow_up:

#single
40.31% <0%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pandas/io/common.py
69.74% <100%> (+0.25%)
:arrow_up:

pandas/io/pickle.py
83.63% <100%> (+2.38%)
:arrow_up:

pandas/plotting/_converter.py
65.2% <0%> (+1.81%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27bbea7...b8b7a66. Read the comment docs.

There are PermissionError on Windows AppVeyro build - I need to look closer into it.


Fix esql.query request types

With make validate, I get ✔ 174 out of 174 test request cases are passing.. I'll need to understand what is going on.

So I get 224/227 now (both locally and in CI thanks to #2690!), and all the remaining failures are related to LOOKUP and its new tables parameter that I'd like to tackle later.


Abstract over blk instead of SimpleBlock in the consensus tests

This is laying the ground work for testing with real Byron blocks.
Since we no longer know we're dealing with SimpleBlocks, we can't generate
transactions directly. We solve this by introducing the TxGen type class to
generate transactions. An instance for SimpleBlock is provided.
Remove the loggingChannel and all its Show/Condense constraints, we can
use Tracers for that.

@nfrisby I have added you as a reviewer just to keep you in the loop

bors r+


Which one is the official CDN link?

I want to use the umd+externals method of webpack to introduce moment to reduce the package size, but I don’t see a standard CDN link in the official document, and I can’t tell which one is official in some posts in the community

Official CDN link given in Documentation  - https://momentjs.com/docs/#/use-it/browser/

I don't think have "official" cdn, just a bunch of CDNs pick up the releases and bundle them. You can download from npm or the web (but from the web it uses github pages, so not likely to be blazing fast).


Check react / react-dom version is set as 15.4.2

Issue Description
Most of our current consumers are using react 15.4.2. To ensure things as passive for them, we've set that as our lower bound for supported react version in peerDeps.
In our generator, this was updated to 15.5.4. We need to review the packages and make sure the react and react-dom versions are set at 15.4.2.
Issue Type

[ ] New Feature
[ ] Enhancement
[ ] Bug
[x] Other

Expected Behavior
make sure the react and react-dom versions are set at 15.4.2 in all packages.

Looking at the code in master, it looks like only the root package has react / react-dom set to 15.5.4.
Need to check the open PRs and see what versions there at and update the generator back to 15.4.2.

Looked at all the PRs and left comments where the versions needed to be updated back to ^15.4.2. Looks like only the https://github.com/cerner/terra-core/pull/475 needs this. I also [updated the generator](https://github.com/cerner/generator-terra-module/pull/54 to use ^15.4.2 for react and react-dom.


doctr is not pushing to gh-pages due to a key error.

Looks like we may need to sort out the key stuff again.
See here: https://travis-ci.org/Quansight/lsst_dashboard/jobs/587496380
ERROR: The doctr command '/home/travis/miniconda/envs/lsst-panel/bin/doctr deploy . --built-docs ./docs/_build/html/' failed: RuntimeError("DOCTR_DEPLOY_ENCRYPTION_KEY_QUANSIGHT_LSST_DASHBOARD or DOCTR_DEPLOY_ENCRYPTION_KEY environment variable is not set. Make sure you followed the instructions from 'doctr configure' properly. You may need to re-run 'doctr configure' to fix this error.")
Error: DOCTR_DEPLOY_ENCRYPTION_KEY_QUANSIGHT_LSST_DASHBOARD or DOCTR_DEPLOY_ENCRYPTION_KEY environment variable is not set. Make sure you followed the instructions from 'doctr configure' properly. You may need to re-run 'doctr configure' to fix this error.```

I created a new branch to address this issue.  I created a new key and tested it and it is writing files to the gh-pages branch.  I'm not sure if something is happening when you try to pull it to the master?

Do a PR with the new key and I can merge to test.
On Sun, Sep 22, 2019, 11:19 PM ppwadhwa<EMAIL_ADDRESS>wrote:

I created a new branch to address this issue. I created a new key and
tested it and it is writing files to the gh-pages branch. I'm not sure if
something is happening when you try to pull it to the master?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Quansight/lsst_dashboard/issues/87?email_source=notifications&email_token=AAEJBFJJZAXXT6HRPBQMEBDQLAYUHA5CNFSM4IYXTGU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7JWTOA#issuecomment-533948856,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJBFJABHCODPLFZBQLAXTQLAYUHANCNFSM4IYXTGUQ
.

@dharhas @ppwadhwa What's the status of this?

@kcpevey I assume we broke documentation when we moved to github actions?

Yes, closing in favor of https://github.com/Quansight/lsst_dashboard/issues/120


Add coverage for subprocess specs

Per discussion in https://github.com/rubocop-hq/rubocop/pull/3705#issuecomment-460966422
Since PR https://github.com/rubocop-hq/rubocop/pull/6255 the coverage results from the different suites (one per tested Ruby version) are merged together, and the result is sent to Code Climate.
That means we no longer need to ignore coverage results from the specs that are run in a separate process. We just need to give these coverage results their own name (e.g. a namespace) and set a new PID (I think to stop SimpleCov's at_exit hook when the fork ends) and these subprocess results will be merged with the coverage results of the rest of the suite.
I tried running rm -fv coverage/.*  ; COVERAGE=true bundle exec rake spec before and after this change, and the “after” run gives me extra coverage for the lib/rubocop/rspec/host_environment_simulation_helper.rb file. The merging of each suite’s coverage results is handled elsewhere, and shouldn’t need to change: https://github.com/rubocop-hq/rubocop/blob/2269afce304dc349972a9d7f3062d353140e03b5/.circleci/config.yml#L261

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[ ] Added tests.
[ ] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Will defer review to someone who's more familiar with this. 🙂

/cc @maxjacobson

I'm not familiar with this one, it may take time to investigation and review.
@pocke Can you review this PR?

I'm not familiar also, sorry.
I think we can try merging this pull request. It is not a user-facing feature, and it looks working well. I guess this pull request is the right way that is mentioned by @maxjacobson in #3705 .

Since it's not user facing, let's try it out.
@bquorning How can we verify it works once merged?

Test coverage for host_environment_simulation_helper.rb should increase: https://codeclimate.com/github/bbatsov/rubocop/lib/rubocop/rspec/host_environment_simulation_helper.rb

@bquorning I tried queueing a refresh. Let's see if it works.

@bquorning

:raised_hands: thanks for figuring this out :)


Mouse without borders become crazy when the wifi signal lost and restore.

Microsoft PowerToys version
0.85.1
Installation method
PowerToys auto-update
Running as admin
None
Area(s) with issue?
Mouse Without Borders
Steps to reproduce
I have 4 machines with same version of powertoys. 2 of them by wire and 2 other by wifi.
With one wired machine being master, y use the other 3 machines as slaves.
When the wifi lost and reconect, the mouse without borders go crazy and if i move my mouse arround screen, the cursor clicks random and make keyboard pulsations. Seems to repeat my last actions but in random places.
I have MWB with service enabled and using it with shortcuts. Not use de move out cursor to change the machine.
✔ Expected Behavior
If i need to reload the service, or reload the config saved of PowerToys or similar action to restore de normal funcion of the cursor and keyboard, it's not problem... but... i don't know.
The only solution is restart the wifi PC to restore normal function.
❌ Actual Behavior
The cursor go crazy and do random clicks and keyboard strokes....
Other Software
No response

0.86.0 continue with the same problem. Only Wifi PCs go crazy with mouse without borders....


ci, Add junit.xml

This PR performs some refactoring in order to create a junit.xml from the test.tap already created by the coreos-ci

/hold
let's wait until the project-infra PR will be merged so we can see how it looks.

/test pull-rhcos-slb-ci-network-tests-main

closing and reopening a new PR to see if trigger is happening on new PRs


Add NSUserActivity conformance

Helped me to debug some issues related to NSUserActivity.
Current dump/description/debugDescription did not show any properties.

Thanks for the PR! Just one comment, since I think we want it on more than just iOS.

@tomaskraina We're going to close this out since it's been inactive for awhile. But if you (or someone else) would like to address the outstanding work and help us get it merged, let us know!


Support password protected keys via vars_prompt

Would you be open to adding support for password protected keys that get decrypted as they get installed?
e.g. some of the ideas in:

http://red-badger.com/blog/2014/02/28/deploying-ssl-keys-securely-with-ansible/
https://gist.github.com/charypar/9237634
https://blog.confirm.ch/deploying-ssl-private-keys-with-ansible/

Would you be open to that? If so let me know, either if its something you'd be open to doing or accepting a PR for.
Thanks,
Perry

My use of this role is a combination of storing the cert/key data via variables, not files, and use ansible-vault to encrypt the data.

@pstoll Thanks for the suggestion, I prefer suggest the usage of ansible-vaultand not complicate the role.


CachingReflector asks for Reflector instance but doesn't use it

The constructor of CachingReflector needs a Reflector instance, but it's not used anywhere in this class. I guess the idea for CachingReflector was to decorate another Reflector and cache its results, but it got lost somewhere in the way.
From what I see in StandardReflector, the CachingReflector could be rebuilt to act as actual decorator. What's your opinion on this?

PR #177 should cover it.

Merged, so closing.

Thank you for your contribution!


Build chat server with aiohttp

I'm trying to make chat server with aiohttp. I want to build this server with TLS/SSL.
My questions :

Is aiohttp good for big number of connections ? (Millions)
What should I use ? WebSocket ?

The Whole point is : Chat server that accept json packet like :
{"message": {"from"<EMAIL_ADDRESS>"to"<EMAIL_ADDRESS>"text": "Hi there"}}
Then send it to other users.
Thanks.

Yes, webocket is very widely used technology for building tools like chat.
aiohttp supports a lot of parallel connections but millions are probably is too high value for single node regardless of chosen framework and programming language. Such massive highload could be achieved by cluster of machines only I believe.


Implementing the model in Sequence to Sequence Learning with Neural Networks

Hello guys : )
I'm wondering is it possible to build encoder decoder model in the paper "Sequence to Sequence Learning with Neural Networks" with Keras ?
In the model, encoder's output (hidden),  is used to init the hidden(h_0) of decoder, instead of as input to decoder. Based on my current understanding, model in the examples/addition_rnn.py is not same with the one above.
It would be greatly appreciated if someone has suggestions & ideas,Cheers

I have been looking for good solutions to implement encoder/decoder with Keras.
It seems that it is not solved yet. ( else, please let me know )
I think @elanmart is the closest one ( who is eager to implement it and has some ideas on it ).
This is is closely related to 'stateful RNN', so please check following issues.
#443 for recurrent model and some API sketchy from @elanmart
#620  for new recurrent container idea from @EderSantana
#98  for real time rnn implementation discusstion
As far as I know, "stateful rnn" is planned as a next update item.
check #426 from @fchollet

Hi @sq6ra!
When I wrote the addition_rnn example I tried to stick as closely to the paper as possible but obviously there were shortcuts. Even with those limitations, the architecture described in the addition_rnn example should work quite well for many tasks.
The two differences from the paper come via:

Initializing a second RNN directly from the hidden state of the first RNN
The first RNN's output being fed into the second RNN as input instead of the second RNN's previous step's output  (see: stateful RNN)

For the first issue, it shouldn't be too difficult to modify the existing RNN code to "steal" the hidden state from a different model when they're done reading it but I've not attempted such a thing. Indeed, I intuitively feel that reading the output of the first RNN as repeated input to a second RNN isn't hurtful and might be quite useful, though I've no empirical evaluation to back that theory up^.
^ I remember reading in a paper where they performed average pooling of all the output steps of the first RNN and found that useful but can't recall the paper.
As mentioned in Sequence to Sequence Learning with Neural Networks, you could avoid having two seperate RNNs and instead use one (described in Figure 1) if you truly think that maintaining a single hidden state is beneficial.
Having two separate RNNs (a) increases the number model parameters at negligible computational cost and (b) allows them to easily train the LSTM on multiple language pairs. (a) can be fixed by increasing the model parameters and (b) is only if you've the time and data for multiple languages.

@sq6ra Currently there is no way to initialize the hidden state of an RNN becouse of the line: outputs_info=T.unbroadcast(alloc_zeros_matrix(X.shape[1], self.output_dim), 1),, which means that the hidden state only exists inside the scan.
@hugman I do indeed have some working code for the general form of recurrent models. I'm currently not at home though, so it'll have to wait.
One thing to mention is that "statefullness" is quite easy to implement, but there are other obstacles on the way.

Thanks for the feedback guys : D
I have decided to implement it with theano, Cheers

@Smerity
I'm working on the experiment that adopting machine translation "ways" to build auto-encoders
one paper published recently did the similar thing, maybe you're interesting
-- A Hierarchical Neural Autoencoder for Paragraphs and Documents
https://web.stanford.edu/~jurafsky/pubs/P15-1107.pdf

I think the only thing left for keras to fully support sequence to sequence learning is to fix an exception with output masking #693, since it seems to only learn to output the same thing for everything without output masking, but works when the sequences are all of the same length.

I do indeed have some working code for the general form of recurrent models. I'm currently not at home though, so it'll have to wait.

@elanmart Could you share the mentioned code?

Very interested in this as well. Simon really helped in me in this thread recently. We discussed encoding and decoding for sequence to sequence (variable length sequence). I'm testing a few things right now but I'll submit a PR if I get something useful working! This thread might help you as well (towards the bottom)
https://github.com/fchollet/keras/issues/395

Anyone had any success with this?

any updates for machine translation model  using keras ?

Hello guys when I run this code , I found this error :
Traceback (most recent call last):
File "seq2seq.py", line 30, in
X, X_vocab_len, X_word_to_ix, X_ix_to_word, y, y_vocab_len, y_word_to_ix, y_ix_to_word = load_data('ALG_trainig2.txt', 'MSA_training2.txt', MAX_LEN, VOCAB_SIZE)
ValueError: need more than 4 values to unpack
root@9cba51539a94:/home/laith/deeplearning_tests/mt_project/test_seq#
from future import print_function
from keras.preprocessing.sequence import pad_sequences
import numpy as np
import sys
import argparse
from seq2seq_utils import *
ap = argparse.ArgumentParser()
ap.add_argument('-max_len', type=int, default=200)
ap.add_argument('-vocab_size', type=int, default=20000)
ap.add_argument('-batch_size', type=int, default=100)
ap.add_argument('-layer_num', type=int, default=3)
ap.add_argument('-hidden_dim', type=int, default=1000)
ap.add_argument('-nb_epoch', type=int, default=2)
ap.add_argument('-mode', default='train')
args = vars(ap.parse_args())
MAX_LEN = args['max_len']
VOCAB_SIZE = args['vocab_size']
BATCH_SIZE = args['batch_size']
LAYER_NUM = args['layer_num']
HIDDEN_DIM = args['hidden_dim']
NB_EPOCH = args['nb_epoch']
MODE = args['mode']
if name == 'main':
# Loading input sequences, output sequences and the necessary mapping dictionaries
print('[INFO] Loading data...')
X, X_vocab_len, X_word_to_ix, X_ix_to_word, y, y_vocab_len, y_word_to_ix, y_ix_to_word = load_data('ALG_trainig2.txt', 'MSA_training2.txt', MAX_LEN, VOCAB_SIZE)
# Finding the length of the longest sequence
X_max_len = max([len(sentence) for sentence in X])
y_max_len = max([len(sentence) for sentence in y])

# Padding zeros to make all sequences have a same length with the longest one
print('[INFO] Zero padding...')
X = pad_sequences(X, maxlen=X_max_len, dtype='int32')
y = pad_sequences(y, maxlen=y_max_len, dtype='int32')

# Creating the network model
print('[INFO] Compiling model...')
model = create_model(X_vocab_len, X_max_len, y_vocab_len, y_max_len, HIDDEN_DIM, LAYER_NUM)

# Finding trained weights of previous epoch if any
saved_weights = find_checkpoint_file('/home/laith/deeplearning_tests/mt_project/test_seq/')

# Training only if we chose training mode
if MODE == 'train':
    k_start = 1

    # If any trained weight was found, then load them into the model
    if len(saved_weights) != 0:
        print('[INFO] Saved weights found, loading...')
        epoch = saved_weights[saved_weights.rfind('')+1:saved_weights.rfind('')]
        model.load_weights(saved_weights)
        k_start = int(epoch) + 1

    i_end = 0
    for k in range(k_start, NB_EPOCH+1):
        # Shuffling the training data every epoch to avoid local minima
        indices = np.arange(len(X))
        np.random.shuffle(indices)
        X = X[indices]
        y = y[indices]

        # Training 1000 sequences at a time
        for i in range(0, len(X), 1000):
            if i + 1000 >= len(X):
                i_end = len(X)
            else:
                i_end = i + 1000
            y_sequences = process_data(y[i:i_end], y_max_len, y_word_to_ix)

            print('[INFO] Training model: epoch {}th {}/{} samples'.format(k, i, len(X)))
            model.fit(X[i:i_end], y_sequences, batch_size=BATCH_SIZE, epochs=1, verbose=2)
        model.save_weights('checkpoint_epoch_{}.hdf5'.format(k))

# Performing test if we chose test mode
else:
    # Only performing test if there is any saved weights
    if len(saved_weights) == 0:
        print("The network hasn't been trained! Program will exit...")
        sys.exit()
    else:
        X_test = load_test_data(' ALG_testing2.txt', X_word_to_ix, MAX_LEN)
        X_test = pad_sequences(X_test, maxlen=X_max_len, dtype='int32')
        model.load_weights(saved_weights)

predictions = np.argmax(model.predict(X_test), axis=2)
sequences = []
for prediction in predictions:
sequence = ' '.join([y_ix_to_word(index) for index in prediction if index > 0])
print(sequence)
sequences.append(sequence)
np.savetxt('test_result.txt', sequences, fmt='%s')


Use image cache for tiles images

Every tiles are downloaded again after every restart the app, it will be good if we use CachedNetworkImageProvider instead NetworkImageWithRetry.
CachedNetworkImageProvider is part of:
https://pub.dartlang.org/packages/cached_network_image
I already have an example working with both options:

if(options.cachedTiles){
return new CachedNetworkImageProvider(url);
}else{
return new NetworkImageWithRetry(url);
}

Feature released in 0.5.0 : https://github.com/johnpryan/flutter_map/pull/204
This issue can be closed.


YouTube (official template) won't play in Codepen

Steps to reproduce

Open https://codepen.io/pen?template=GGqbbJ (is "minimal templates" for Youtube at https://github.com/sampotts/plyr)
Click "Play"

Expected behaviour
Start to play the video.
Actual behaviour
The video don't play, youtube's "play" button is visible. Click again just switch youtube's to plyr's "play" button.

Environment

Browser: Chrome
Version: 73
Operating System: Windows
Version: 10

Also, as I see, old versions of plyr can play videos from youtube: https://www.dyclassroom.com/reference-javascript/player-for-website-video-and-audio-using-plyr , https://github.com/sampotts/plyr/issues/714 works for me at plyr v. 2.0.15

It seems to be just an issue with the template and only happens sometimes. When I save, it works fine every time. The main page plays fine for me too in Chrome on Windows 10.

Doesn't seem to play for me, Chrome 73 on MacOS, even after anonymously saving the CodePen. It just toggles back and forth between the two play buttons.

I'm not sure if this is just an issue with CodePen?

Same issue is happening with me. If I am setting muted : true it works fine.But When I am going to volume up it pauses and not playing.

This problem is also there in chrome on Macs.. Works on windows/linux on chrome, but on mac the video's only play when you mute the video. As soon as you unmute the video it pauses the video and shows the following javascript warning:

Unmuting failed and the element was paused instead because the user didn't interact with the document before.

Facing the same issue

I can only reproduce the issue when browsing to any of the above links in Incognito mode only.

(MacOS 11.1, Chrome Version 87.0.4280.88 (Official Build) (x86_64))
No plugins enabled

I can only reproduce the issue when browsing to any of the above links in Incognito mode only.

(MacOS 11.1, Chrome Version 87.0.4280.88 (Official Build) (x86_64))
No plugins enabled

the same, seems the player does not work in chrome and incognito mode


git159 copy listener create time handling

#165 this change was suggested by sonar type for the listener; so I have changed the connector to be in line with the listener around create time handling


How to grant ES permission for this action

I want to grant permissions to beats ,which can setup the index-management.
but i always get a error below
2020-05-15T11:04:12.517+0800	INFO	[index-management]	idxmgmt/std.go:295	Loaded index template.
2020-05-15T11:04:12.517+0800	DEBUG	[elasticsearch]	elasticsearch/client.go:775	GET https://es-od-sec2.lab.local:9200/_alias/winlogbeat-7.6.2  <nil>
2020-05-15T11:04:12.518+0800	DEBUG	[elasticsearch]	elasticsearch/client.go:775	HEAD https://es-od-sec2.lab.local:9200/winlogbeat-7.6.2  <nil>
2020-05-15T11:04:13.907+0800	ERROR	pipeline/output.go:100	Failed to connect to backoff(elasticsearch(https://es-od-sec1.lab.local:9200)): Connection marked as failed because the onConnect callback failed: failed to check for alias 'winlogbeat-7.6.2': (status=403) : 403 Forbidden:

after i enable debug info for elasticsearch and using the admin user, i found it PUT,but how do i describe this permission in opendistro security ?
2020-05-15T11:08:20.640+0800	DEBUG	[elasticsearch]	elasticsearch/client.go:775	PUT https://es-od-sec2.lab.local:9200/%3Cwinlogbeat-7.6.2-%7Bnow%2Fd%7D-000001%3E  {"aliases":{"winlogbeat-7.6.2":{"is_write_index":true}}}

copy the default logstash role and add some permissions then it works
{
  "description": "Provide the minimum permissions for logstash and beats",
  "index_permissions": [
    {
      "index_patterns": [
        "*"
      ],
      "dls": "",
      "fls": [],
      "masked_fields": [],
      "allowed_actions": [
        "indices:admin/aliases/exists",
        "indices:admin/aliases/get"
      ]
    },
    {
      "index_patterns": [
        "logstash-*"
      ],
      "fls": [],
      "masked_fields": [],
      "allowed_actions": [
        "crud",
        "create_index",
        "manage",
        "manage_aliases"
      ]
    },
    {
      "index_patterns": [
        "*beat*"
      ],
      "fls": [],
      "masked_fields": [],
      "allowed_actions": [
        "crud",
        "create_index",
        "manage",
        "manage_aliases"
      ]
    }
  ],
  "tenant_permissions": [],
  "cluster_permissions": [
    "cluster_monitor",
    "cluster_composite_ops",
    "ODS_CLUSTER_READ_ILM",
    "indices:admin/template/get",
    "indices:admin/template/put",
    "cluster:admin/ingest/pipeline/put",
    "cluster:admin/ingest/pipeline/get",
    "cluster:admin/ilm/put"
  ]
}


New GCL implementation that handles multiple WT types, heights. New inflow definition. New plotting tools.

Thanks for the nice work. There were some issues with python3. These are fixed in #7.


Fix warning in push_to_hub

Fix warning:
FutureWarning: 'shard_size' was renamed to 'max_shard_size' in version 2.1.1 and will be removed in 2.4.0.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Does v-table need :id-fn  ?

What purpose does :id-fn serve in v-table?  The component itself seems to work principally with row-index and the elements of :model
Can we remove it?  Should we change it?
Gregg's initial answer:
It's primary purpose is to provide a unique id for React, just like in any of our other
re-com components that render lists of things.
Note that row-index was added later, so perhaps we can revisit using row-index as unique ids.
I would have to look further into it

My guess is that we can probably move to row_index. That would make scrolling efficient (aka adding and removing rows). However in apps where the underlying :model is changing (items are being added and removed often), then letting the programmer control this would be a good idea.  Its just that this is an edge case I think.
Proposal:

Gregg checks if :id-fn is only used in this one way (for React keys).
If so, we keep :id-fn but we:

document that's the purpose and change the name to reflect its real purpose:  :key-fn
make it optional
make it take two args row_index and row
provide a default which returns the row-index arg

Yep, only used for React keys
Done (except the code determines if :key-fn needs to be called, so no need to pass it two args)


No need for title attribute on cookie message.

How would you expect it to work?
It's an unnecessary title attribute because it is an exact match to the contents inside the link. So I would not expect it in this case.
Feel free to suggest a fix...
Remove the title attribute.
Explain the steps to reproduce the bug
https://govuk-elements.herokuapp.com
Tell us about your environment
N/A

Fixed by https://github.com/alphagov/govuk_elements/pull/389


Recommend that driver authors always write their own type implementations

Right now, the Neo4j::Types documentation actively encourages driver authors to inherit method implementations that depend upon a specific data structure, which is to be provided by the driver. This violates encapsulation.
As a result, Neo4j-related modules are coupled more tightly with each other than one might like. Making a change in the Neo4j::Types implementation to improve compatibility with one driver runs the risk of reducing compatibility with another. Following the principle of least knowledge can reduce that risk. Basically, you should make as few assumptions about another module’s internals as possible.
Driver authors should be strongly encouraged to write their own implementations of the methods defined in Neo4j::Types. This allows authors to better adapt to the peculiarities of each driver while limiting action at a distance.
Existing drivers can probably just copy the code from Neo4j::Types, so the impact of such a change should be relatively small.

Some methods could probably be inherited safely. For example, for a custom path class that already has an elements() method, the nodes() and relationships() methods could be mixed in from a role without needing to know the data structure – they would just call elements() and filter the return value. The same applies to get() and to many methods in the Point type.
But I think these are all refinements for a later date.


Listening for events

@kreynolds
Is it possible with this gem to listen to events that happen on the MIOS? I was thinking about removing all of my scenes in Vera and managing them all in my own app, but I'd have to be notified if I physically flip a switch somewhere, or poll the devices for status. Any thoughts on how you'd go about this?

I think the event system in mios requires that you use their goofy gateway service, I haven't played around with events much but when I did, I couldn't get anything to work. If you find anything, let me know because I'd be super interested in incorporating that .. going to close this for now .. reopen when you find something :)


sync: cherry picks from master for release 0.36.0

What type of PR is this?
/kind feature
Any specific area of the project related to this PR?
/area build
What this PR does / why we need it:
Cherry picks from master:

https://github.com/falcosecurity/falco/pull/2834
https://github.com/falcosecurity/falco/pull/2812
https://github.com/falcosecurity/falco/pull/2828
https://github.com/falcosecurity/falco/pull/2829
https://github.com/falcosecurity/falco/pull/2826
https://github.com/falcosecurity/falco/pull/2830
https://github.com/falcosecurity/falco/pull/2831
https://github.com/falcosecurity/falco/pull/2833

Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

update(cmake): bump libs to 0.13.1

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP,LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/hold

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This is the last sync needed ;)

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/unhold

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[AutoPR] eventgrid/resource-manager

Created to accumulate context: eventgrid/resource-manager

(message created by the CI based on PR content)

@AutorestCI rebase

Rebase done and pushed to the branch

@AutorestCI rebuild https://github.com/Azure/azure-rest-api-specs/blob/master/specification/eventgrid/resource-manager/readme.md

Working on generating https://github.com/azure/azure-rest-api-specs/blob/master/specification/eventgrid/resource-manager/readme.md for you!!!

Build SDK finished and cleaned

Codecov Report

Merging #2358 into master will decrease coverage by 1.3%.
The diff coverage is 57.92%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 85c2383...6e41a81. Read the comment docs.


Fix scanning list obj

Describe your changes
Fix issue for scanning list obj

@Moshe-Rappaport-CA The tests failed


Updated angular dependencies to 7.2.14

Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/HackedByChinese/ng2-idle/blob/master/CONTRIBUTING.md#pr
[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (check one with "x")
[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[x] Other... Please describe: Updated version of Angular dependencies to 7.2.14.

What is the current behavior? ( #119 )
Currently, version of Angular dependencies is 6.0.0.
After running npm install, it gives warning as:
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
What is the new behavior?
The version of Angular dependencies should be 7.x.x.
This warnings related to Angular version should disappear.
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Coverage remained the same at 99.372% when pulling f6c0bd3c3bfd0c0003e1d4959c271ff2e79f7105 on vishalhulawale:master into 6826db22d27cb9ff3e7ad84aa47c8ae63ee49a5d on HackedByChinese:master.

I took your commit and loosened the version to angular ^7.0.0. I'll have a build shortly. Thanks for the PR.


span: create 'substreams' from start to end bounds.

This feature was motivated by glTF .glb files. In them, JSON data is intermingled with binary data. A glTF parser like cl-gltf has a stream that is the binary data for the whole file, and knows the chunks or spans of JSON and binary. So we just use json:parse as usual with a new span/slice/substream of the original stream.

I'm so sorry, I believe I somehow missed this PR in the past.
Thank you for looking into this