Debug starts and stops a few seconds later with no log

Issue Type: Bug
I was able to debug as I have been for well over a year with this project, until I updated to Insiders 1.56 this afternoon.  Upon VSCode restarting, I started a debug session, the debugger stopped after a few seconds, and left no errors or logs.
I completely wiped vscode-insiders from my system, deleted all extension and trace of it existing, then installed again.  Same issue.
I noticed that the node-debug (ms-vscode.node-debug2) dependancy was updated, twice, yesterday.  I'm suspecting something isn't right with that extension, but it's built-in, so no rolling back without downgrading VSCode.
node-debug changes, yesterday:
https://github.com/microsoft/vscode/commit/4202acc9f765154b404d8287c0243a03b57a9e0b
https://github.com/microsoft/vscode/commit/accd84def608169a720b4e58ef30debf9490e653
VS Code version: Code - Insiders 1.56.0-insider (Universal) (bb90ba3d5a5f8d114045b045f3bbe1ebc404f54a, 2021-04-07T13:06:58.816Z)
OS version: Darwin x64 20.3.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
32.00GB (2.77GB free)

Process Argv
--crash-reporter-id 486ee961-8c9a-4d09-b0f1-e085c767cff1

Screen Reader
no

VM
0%

Extensions (17)

Extension
Author (truncated)
Version

vscode-toggle-quotes
Bri
0.3.4

vscode-eslint
dba
2.1.19

gitlens
eam
11.3.0

todo-tree
Gru
0.0.209

vue-snippets
hol
1.0.4

node-module-intellisense
lei
1.5.0

vscode-antlr4
mik
2.2.4

vscode-docker
ms-
1.11.0

remote-containers
ms-
0.166.1

remote-ssh
ms-
0.65.1

remote-ssh-edit
ms-
0.65.1

remote-wsl
ms-
0.54.6

vscode-remote-extensionpack
ms-
0.20.0

azure-account
ms-
0.9.8

vscode-typescript-next
ms-
4.3.20210406

vetur
oct
0.33.1

vscodeintellicode
Vis
1.2.12

A/B Experiments
vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstry244:30244315
pythonvsdeb440:30224570
pythonvsded773:30223139
vstes627:30244334
pythonvspyt875:30259475
pythonvsnew554:30265444
pythontb:30258533
wslfolderdoc:30278405
vscoresta800:30283263
vspre833:30267464
pythonptprofiler:30281269
vscnewfiletext:30280636
vshan820:30276952
pythondataviewer:30285072

I checked the extension host log and found this:
Log (Extension Host)
Notice the error "Cannot find module '/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/extensions/ms-vscode.node-debug/dist/extension.js'"

This issue only affects node-debug (legacy) for node.js legacy versions < 8.0.
For all other version of node.js the new js-debug is used, which works fine in Insiders 1.56.
node-debug (legacy) should be fixed in tomorrow's insiders.

Sounds good.  I am using legacy, so good catch.
Appreciate the quick mitigation.

I cannot find any reference to this.
Regards,
Tarek

Normally, the switching between node-debug and js-debug happens automatically behind the scenes.
But since "node-debug" was broken in Insiders, your approach of configuring "pwa-node" explicitly is the correct workaround.


Add map marker clustering to map on homepage

adds clustering functionality to map markers for entries that are not featured
render featured markers separately so that they do not get clustered and are therefore more visible

Nice! Leaving the featured pins out of the clusters is a great idea!
Let's run this by @amberfj during the design review today at 1pm. Thanks @ascott !


Detect dimensions/size of 3D model and scale accordingly?

Is there a way to detect the size (dimensions) of a loaded GLB file and calculate a scale factor so that the 3D model fits on the Android screen? Filament has a method called transformToUnitCube() which "tells the model viewer to transform the root node of the scene such that it fits into a 1x1x1 cube centered at the origin".
Could we build something like that for SceneView?
private fun loadGlb(name: String) {
   val buffer = readAsset("models/${name}.glb")
   modelViewer.loadModelGlb(buffer)
   modelViewer.transformToUnitCube()
}

Source: https://medium.com/@philiprideout/getting-started-with-filament-on-android-d10b16f0ec67

Here is what you are looking for:
https://github.com/SceneView/sceneview-android/blob/main/sceneview/src/main/kotlin/io/github/sceneview/node/ModelNode.kt#L332-L337
It will scale your model to a unit cube.

You could encounter some issue if your model is animated due to an actually wrong bounding box calculation returned by Filament.
If it's your case, please create an issue within the Filament repo and link it here.

BTW, that's the kind of thing that makes me love the kotlin-math integration within SceneView:
val halfExtent = asset.boundingBox.halfExtent.let { v -> Float3(v[0], v[1], v[2]) }
modelScale = Scale(units / max(halfExtent))

Really appreciate your help @ThomasGorisse Thank you!

Sorry. I won't be able to answer you on the deprecated/archived version of Google Sceneform within those repositories issues context.

No worries, I will be migrating over to SceneView.


Doing Image Processing on the RGB Image

I have read the examples and I can see that I will have the image loaded on _latestImageBytes.
Now, same as what SoylentGraham commented in https://github.com/VulcanTechnologies/HoloLensCameraStream/issues/10 I can understand of course that this is an 1D array. I suppose the BGRA is similar to the Format32bppArgb format.
I want to do some Image Processing on this image. (I have already the algorithms I want to implement so no problem there) Can anyone give some advice or pointers on how to proceed?

In the file MatrixUsageApp.cs or in VideoPanelApp.cs I see there is a OnFrameSampleAcquired function. I get that this is the function that gets called on every frame. Inside the function there is the UnityEngine.WSA.Application.InvokeOnAppThread call .
I don't understand this so well. Is this being called on another thread? Should I put all my processing inside this? (I do a lot of processing) . And if it is done on another thread, how can I finally get some results back to the Unity UI Thread (that I suppose is the one starting it)
Thousand thanks for any help. I really like this library.

Yes, you probably want to do your work within the InvokeOnAppThread callback. Here's why:
OnFrameSampleAcquired does not get called on Unity's thread, so InvokeOnAppThread is a way of returning your execution stack to Unity's thread. The reason why we want to return to Unity's thread is because calls to Unity's API will fail if they are called from a non-Unity thread.
However, you don't NEED to call InvokeOnAppThread if you won't be making any Unity API calls. However, I'm not sure if it's a good idea to do your processing on the same thread as the CameraStream library anyway. If you're doing expensive CPU-processing of the image bytes then it's probably best to do them on an independent thread anyway.

Thank you very much for your reply. I have been thinking about what you wrote, trying to understand it fully and I have come to a follow up question: Would it be a good idea (or good design) to do something like this:
void OnFrameSampleAcquired(VideoCaptureSample sample)
{  //the usual code here

//Now I have the data on _latestImageBytes

ProcessAsynchronously(_latestImageBytes);

UnityEngine.WSA.Application.InvokeOnAppThread( //....the rest here

);
}

void  async void ProcessAsynchronously(byte[] image)
{
await Process1(image);
await Process2(image);
//....
}

The idea would be to do the processing in a asynchronous process. I would like to call an await inside the OnFrameSampleAcquired but I don't think that is possible. So I put it inside the function.
I have not figure it out how to return any results, other than modify the byte[] array but something tells me this is not a good strategy.
Other concern I have (related to the previous) is that since the FrameSampleAcquiredCallback gets called every frame, it  will rewrite the _latestImageBytes every frame (sample.CopyRawImageDataIntoBuffer) and if my processing last too long it won't go well.
I read that instead of using this callback we can use RequestNextFrameSample() but I don't know if that is the correct way to do it or how to actually do it correctly.
Any recommendation or insight will be greatly appreciated

Hey @SoylentGraham
I would know if your system of thread that encode in h264 the pictures and then output it in an another queue has been posted on github somewhere? I am really interested to see how h264 encoding work because I may need to do something similar

I don't have an open source implementation I'm afraid, just the jpeg one is open sourced.
Implemented with MediaFoundation. But I may get it open sourced one day as a general cross platform unity h264-encoder. (Currently being used for commercial things, so only available for cash)


[Azure Search] Fix the definition of WebApiSkill

This SDK was generated based on this completed pull request in the swagger repository.

This pull request, intends to correct the definition of the WebApiSkill model in the SDK, by making headers a simple dictionary within the skill and not a nested property.

Added a test to ensure correctness

This is a breaking change, and is already preceeded by a version bump in this PR

Fixes #6468

I have a concern about an apparently unrelated test change.

@Careyjmac and I have added extra context in the thread. Let me know what you think

LGTM... preceding version bump looks good as well given this is a breaking change. Merging...


Adaptive loss thresholds to handle packet reorders

Implements:

[x] Adaptive loss threshold.  When receiving the first ACK carrying late ACKs, disable packet-based threshold. Then, for each successive ACK carrying late ACKs, double the time threshold until it reaches 1 RTT.
[x] Export loss threshold being used through quicly_stats_t.
[ ] Add tests.

The adaptive logic being implemented is dumb, but it would be sufficient as a stop-gap solution. The cost of calling out late acks depends on the sensitivity of the congestion controller to loss events - we can revisit the design of adaptive thresholds after we have more data (obtained through the added stats) or after we switch to a loss-resilient CC.

@janaiyengar Thank you for the review.

This might be a bit too aggressive. How about if in addition to doubling the time threshold, we increase the packet threshold by 1 on each late ack?

I'm not sure if I follow.
The strategy we have is to for the first late ACK, disable packet threshold. Then, for the following late ACKs, double the time threshold.
If we change the logic so that packet threshold would be retained (with increase of 1 per each late ACK), then I would assume that we would see more late ACKs that would in turn increase both our packet and time thresholds. Is that what you are suggesting?

Deployments with this change has been doing well.


Framing v1.0

Update the framing to version 1.0 and add tests to ensure correct framing.

Take an extra good look at canardHandleRxFrame, not the simples procedure to perform surgery on

@pavel-kirienko I added a cast in canard.c and fixed some types and constructions in test_framing to get CI compiling on the picky c/c++ compilers. Only minor stuff, and nothing that should require a new review.


GitLab in the style of GitHub

https://github.com/logos

The press logo assets aren't svg. There are ai and png


Improvement for k8s.io/docs/tasks/configure-pod-container/quality-service-pod/

I came to this page to understand the impact of QoS of pods on pod operations and on the nodes. This needs to be clarified.

We could add a concept or reference page about QoS for Pods
Some of the information in https://kubernetes.io/blog/2021/11/26/qos-memory-resources/ would be useful towards making that new page.
/kind feature
/language en


Changing TimePicker

#91605
I think we should make small visual changes to the TimePicker:

Change the "Period selector label text" to "On surface variant" for a non-selected color.

from specification

from flutter

In the "Time picker input" change the width of the period selection container, making it the same as in the time picker with the (12 hour) selector.

1
2

In the "Time picker dial" reduce the width of the period selector container, making it a width (114 dp), which is slightly larger than with the selector (12 hour).

example Flutter / Compose

M3 Timepicker UI is cropped.

See also Numeric values bouncing in the time picker container
https://github.com/flutter/flutter/assets/102767277/638637a0-2f1d-4d43-aed2-b0ebd6569f01

@guidezpl - what do you think of this proposal?
@MaxYablochkin - https://github.com/flutter/flutter/issues/127035#issuecomment-1560837171 looks like a bug. Does this occur because the app's text scale factor has been increased?

It is noticeable that this happens when switching the time format 24, 12 hours.

I've numbered the issues raised

This should be fixed
This is as designed in the spec
This should be fixed by growing the time picker height such that the dial keeps its size
This should be fixed. The source here might be the use of a custom font or a11y text size which renders it as such. Limiting the size and fading would be preferable to vertical overflow
This is a low priority fix, likely an artefact of the border drawing inside its bounds rather than outside. Might be easily fixable by aligning the text on the center line rather than the baseline.

@raghav042 I am unable to replicate the behavior you reported. I tried with official example using latest stable and master running on Pixel 6 api 33 emulator, but didn't see the cropped UI.

If this doesn't represent your case, I suggest you to file a new issue and provide relevant details (flutter doctor -v and complete minimal reproducible code sample) so that we can properly address the issue you are seeing.

Avobe ui issue is because of incorrect themedata.
return ThemeData.dark().copyWith(useMaterial3: true); --> Cropped Timepicker UI issue
return ThemeData.dark(useMaterial3: true).copyWith(); --> working fine.

Above ui issue is because of incorrect themedata.
return ThemeData.dark().copyWith(useMaterial3: true); --> Cropped Timepicker UI issue return ThemeData.dark(useMaterial3: true).copyWith(); --> working fine.

I can confirm the same behaviour: .copyWith(useMaterial3: true) breaks time picker, ThemeData.dark(useMaterial3: true) works OK.

Filed new issue https://github.com/flutter/flutter/issues/129941 based on above findings.

I faced thesame issue with material 3 time picker UI cropped and @darshankawar solution worked but I think there has to be a better way to prevent such from happening difficult to debug I spent a few hours wondering what's wrong 😂

Addressing issues mentioned here.
Under https://github.com/flutter/flutter/issues/127035#issue-1714203381

Fixed by https://github.com/flutter/flutter/pull/131253
Working as designed
Tracking in https://github.com/flutter/flutter/issues/82661

Under https://github.com/flutter/flutter/issues/127035#issuecomment-1560837171
4. See https://github.com/flutter/flutter/issues/129941
Under https://github.com/flutter/flutter/issues/127035#issuecomment-1560971982
5.  Fixed by https://github.com/flutter/flutter/pull/131253. The cause of the issue is explained in https://github.com/flutter/flutter/issues/131247


prepare for v0.2.0

I've verified SwiftPM and Carthage integration. I'm not familiar with CocoaPods, but everything should Just Works™.

tag?

ok~


Cannot import name "get" from voltage.utils

Summary
If you run your revolt bot, an error will occurs.
Reproduction Steps
I only coded a Revolt bot.
Minimal Reproducible Code
import voltage, os
rvl_cli = voltage.Client()
rvl_token = os.getenv('revolt')

@rvl_cli.listen('ready')
async def on_rdy():
    print('Revolt bot is ready !')

rvl_cli.run(rvl_token)

Expected Results
Run the bot
Actual Results
It crashed.
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    import voltage
  File "/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/__init__.py", line 32, in <module>
    from .client import Client
  File "/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/client.py", line 10, in <module>
    from .internals import CacheHandler, HTTPHandler, WebSocketHandler
  File "/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/internals/__init__.py", line 7, in <module>
    from .cache import CacheHandler
  File "/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/internals/cache.py", line 11, in <module>
    from ..server import Server
  File "/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/server.py", line 9, in <module>
    from .invites import Invite
  File "/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/invites.py", line 7, in <module>
    from .utils import get
ImportError: cannot import name 'get' from 'voltage.utils' (/home/runner/Discord2Revolt/venv/lib/python3.8/site-packages/voltage/utils/__init__.py)

Checklist

[x] I have searched the open issues for duplicates.
[x] I have shown the entire traceback, if possible.
[x] I have removed my token from display, if visible.

Additional Context
I have found a fix :

Copy all the content of the voltage/utils.py file
Go to the file voltage/utils/__init__.py.
Paste all the code in the file
Move the line from __future__ import annotations at the first line
Fixed ! (i think)

pip uninstall voltage && pip install git+https://github.com/enokiun/voltage is the current solution for this, why this occurs escapes me and I'm left to assume that it's merely PyPI's fault, if you have any ideas why the PyPI version of voltage is always borked like this besides the old utils submodule being cached then please do LMK


23833 Small text/style changes

Issue #: bcgov/entity#23833
Description of changes:

app version = 5.12.8
fixed share structure button styles (outlined)
added punctuation to some hint text
bolded and fixed layout of share structure instruction text (x3)
changed incorporation -> continuation in document delivery section
updated unit tests

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the bcrs-entities-create-ui license (Apache 2.0).

/gcbrun

Temporary Url for review: https://business-create-dev--pr-750-mrecsz68.web.app

Screenshots
Add Person buttons

Add Share Class buttons

Continuation Application "must include" text

Incorporation Application "must include" text

Amalgamation Application (regular) "must include text


Settings goes off the top of the screen on mobile

Using the provided theme and search options, opening settings by clicking on the (very small) cog centers the modal vertically on the first view height, which puts the top off the screen.  You can scroll down to see the bottom, but there's no way to scroll up past the top of the page.  The settings modal should start no higher than the top of the page.

The most recent commit reverts the fix for this.

Darn, didn't notice that. I'll push out a fix later today.

Thanks for fixing!


Images built since 11/01 introduced a bug from Docker Compose

Description
The bug prevents Docker Compose to parse any docker-compose.yml when the services contain a dot. This breaks all pipelines in such a case: https://github.com/docker/compose/issues/11336
It appears the runner doesn’t have the bug all the time, even on v2.311.0. However, Ubuntu containers versions 20240114.1.0 (tried on both Ubuntu 22.04 & 20.04) contain the faulty version of Docker Compose (2.24.0).
I believe managed runners are in a rolling upgrade since we have only few of our jobs running on 20240114.1.0. However, since we have a lot of jobs, all our pipelines started failing.
Platforms affected

[ ] Azure DevOps
[X] GitHub Actions - Standard Runners
[X] GitHub Actions - Larger Runners

Runner images affected

[X] Ubuntu 20.04
[X] Ubuntu 22.04
[ ] macOS 11
[ ] macOS 12
[ ] macOS 13
[ ] macOS 13 Arm64
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
v2.311.0 (builds 20240114.1.0 and laters)
Is it regression?
Yes, since builds 20240107.1.0 and earlier. Docker Compose stopped working completely in specific circumstances due to YAML parsing.
Expected behavior
Docker Compose should work with no issue on valid docker-compose.yml containing dots in service names.
Actual behavior
Docker Compose bug has been introduced from a failing Docker Compose dependency (https://github.com/docker/compose/issues/11336). The bug appears on a specific circumstances when parsing YAML files with one or multiple services containing a dot in their name.
Repro steps

Prepare a docker-compose.yml with services names including dots
Run docker compose in a pipeline using a runner v2.311.0 & build version 20240114.1.0
See error: 3 error(s) decoding: * 'services[dx-ref-java-quarkus-im-postgres.postgres-db].ports[0]' expected a map, got 'string'

closing as duplicate of https://github.com/actions/runner-images/issues/9172


docs: Ionic Variables and Theming

📚 Is your documentation request related to a problem?
I would like to change themes and css variables from ionic itself. On Ionic/Vue, it generate files ./src/theme/variables.css. But, i couldn't see the configuration in this (maybe, i missed something).
Actually, i would like to set base theme to force using light theme instead of dark by replacing with this @media (prefers-color-scheme: light)
🔍 Where should you find it?
Something in the feature page of the wiki, i guess.
ℹ Additional context
Here's the segment code from variables.css
 ....
  --ion-color-light-contrast: #000000;
  --ion-color-light-contrast-rgb: 0, 0, 0;
  --ion-color-light-shade: #d7d8da;
  --ion-color-light-tint: #f5f6f9;
}

@media (prefers-color-scheme: dark) {
  /*
   * -------------------------------------------
   */
...

@chiqors i would post this question in the ionic forum, you would probably get an answer much faster

@chiqors Yes, we should add a section in the docs about this. But basically if you create a CSS file (for example ~/assets/vars.css) and add this to your css array in Nuxt config, Ionic should pick it up.

@chiqors you can see example from test app I made some time ago here which basically works the way Daniel said. For forcing a light mode, maybe the easiest way would be to set dark theme variables to the same value as the light themed ones.

But basically if you create a CSS file (for example ~/assets/vars.css) and add this to your css array in Nuxt config, Ionic should pick it up.

Also curious what you think about a random thought that popped in my head @danielroe:
What if we could sync ionic css variables with let's say configuration from app.config.ts or theme.config.ts in nuxt project? It would allow for referencing those exact values programmatically in TS safe way

Ooh, interesting. Maybe a bit like https://github.com/Tahul/pinceau...

Kind of, but scoped only to Ionic variables. I guess the question would be then how far to take it?
We could also provide some utils, and maybe changing variables at runtime could update theme at the spot


GT06 timezone on traccar 3.5

hello Mr tananaev
i congratulate you for this wonderful software.
i installe traccar 3.5 and actually i use a GT06 device but there is some problem with the time;
it shows  wrong time :

this my configuration file:
<entry key='web.enable'>true</entry>
<entry key='web.port'>8082</entry>
<entry key='web.path'>/opt/traccar/web</entry>
 <entry key='gt06.enable'>true</entry>
<entry key='gt06.timezone'>28800</entry>
<entry key='geocoder.enable'>true</entry>
<entry key='geocoder.type'>google</entry>

<entry key='logger.enable'>true</entry>
<entry key='logger.level'>all</entry>
<entry key='logger.file'>/opt/traccar/logs/tracker-server.log</entry>

my device is set to GMT,
please what i am doing wrong?
thanks.

Are you talking about time in the "Devices" table?

no in the position table when i log into the platform the time of the last update is not correct it has 2 hours in plus and the time show by the report has 1 hour in plus.
i set   this  28800 in my configuration file and that is the value of the database
serveur time 2016-07-01 01:43:56
device time 2016-06-30 16:45:18
fix time 2016-06-30 16:45:18
my device is set to GMT+0

If your device reports GMT time, why do you need to specify timezone in the config? You don't.

Thanks very much  Mr i modify it now i i will get back to you for the result.

Mr tananaev i did it and i restarted the system but it still shows wrong time : this the picture

it is 09:32  am in my country now

i removed timezone from the config.

Please provide HEX samples from server log file.

2016-07-01 09:24:54  INFO: Shutting down server...
2016-07-01 09:24:56  INFO: Operating system name: Linux version: 3.13.0-74-generic architecture: amd64
2016-07-01 09:24:56  INFO: Java runtime name: Java HotSpot(TM) 64-Bit Server VM vendor: Oracle Corporation version: 25.77-b03
2016-07-01 09:24:56  INFO: Memory limit heap: 880mb non-heap: 0mb
2016-07-01 09:24:56  INFO: Character encoding: UTF-8 charset: UTF-8
2016-07-01 09:24:56  INFO: Version: 3.5-SNAPSHOT
2016-07-01 09:24:58  INFO: Starting server...
2016-07-01 09:27:03  INFO: [199AB833] connected
2016-07-01 09:27:03 DEBUG: [199AB833: 5023 < <IP_ADDRESS>] HEX: 78780d01035548802019966000bdc0640d0a
2016-07-01 09:27:03 DEBUG: [199AB833: 5023 > <IP_ADDRESS>] HEX:<PHONE_NUMBER>bda63b0d0a
2016-07-01 09:27:19 DEBUG: [199AB833: 5023 < <IP_ADDRESS>] HEX: 78781f12100701081c2bcb006a1808013bd6ad00340f02700100d5001feb00bd3f170d0a
2016-07-01 09:27:20 DEBUG: [199AB833: 5023 > <IP_ADDRESS>] HEX:<PHONE_NUMBER>bdccca0d0a
2016-07-01 09:27:20  INFO: [199AB833] id: 1, time: 2016-07-01 08:28:43, lat: 3.86276, lon: 11.49933, speed: 0.0, course: 15.0
2016-07-01 09:27:21  WARN: [199AB833] error - Connection reset by peer - IOException (...)
2016-07-01 09:27:21  INFO: [199AB833] disconnected

I highlighted date and time in the message:

78781f12100701081c2bcb006a1808013bd6ad00340f02700100d5001feb00bd3f170d0a

It's in HEX, so if you convert it into decimal, you get following results:

10 07 01 - (20)16-07-01
08 1c 2b - 08:28:43

Server prints exactly the same value, so I'm assuming that server is configured to GMT timezone. Is that correct? If yes, then your device must be sending GMT-1.

thank.
this my config file :
true
8082
/opt/traccar/web
true
google
true
all
/opt/traccar/logs/tracker-server.log
i don't see where we define the time zone of the server.
but if you talk about the timezone of the VPS where traccar is installed  it is set to (GMT+1) like my timezone.

sory it does'nt appear  this my config.

i will try to set  my device to GMT-1

hi Mr tananaev this the Hex obtained when i set the device to GMT-1
2016-07-01 16:01:32 DEBUG: [47B7E33D: 5023 < <IP_ADDRESS>] HEX: 78780d01035548802019966002e8f6fc0d0a
2016-07-01 16:01:32 DEBUG: [47B7E33D: 5023 > <IP_ADDRESS>] HEX:<PHONE_NUMBER>e890a30d0a
2016-07-01 16:01:50 DEBUG: [47B7E33D: 5023 < <IP_ADDRESS>] HEX: 78781f121007010e030ec9006a2993013bf8232134c202700100ca00087202e86c9a0d0a
2016-07-01 16:01:50 DEBUG: [47B7E33D: 5023 > <IP_ADDRESS>] HEX:<PHONE_NUMBER>e8fa520d0a
2016-07-01 16:01:51  INFO: [47B7E33D] id: 1, time: 2016-07-01 14:03:14, lat: 3.86526, lon: 11.50409, speed: 17.8, course: 194.0
2016-07-01 16:01:55  WARN: [47B7E33D] error - Connection reset by peer - IOException (...)
2016-07-01 16:01:55  INFO: [47B7E33D] disconnected
2016-07-01 16:11:43  INFO: [EDA6F76C] connected
2016-07-01 16:11:43 DEBUG: [EDA6F76C: 5023 < <IP_ADDRESS>] HEX: 78780d01035548802019966002f71e8a0d0a
2016-07-01 16:11:43 DEBUG: [EDA6F76C: 5023 > <IP_ADDRESS>] HEX:<PHONE_NUMBER>f778d50d0a
2016-07-01 16:12:00 DEBUG: [EDA6F76C: 5023 < <IP_ADDRESS>] HEX: 78781f121007010e0d17c8006a1711013bd96b00350c02700100d500151102f760ff0d0a
2016-07-01 16:12:00 DEBUG: [EDA6F76C: 5023 > <IP_ADDRESS>] HEX:<PHONE_NUMBER>f712240d0a
2016-07-01 16:12:00  INFO: [EDA6F76C] id: 1, time: 2016-07-01 14:13:23, lat: 3.86262, lon: 11.49972, speed: 0.0, course: 268.0
2016-07-01 16:12:04  WARN: [EDA6F76C] error - Connection reset by peer - IOException (...)
2016-07-01 16:12:04  INFO: [EDA6F76C] disconnected

M. i did what you asked i set the server timezone for GT06 to GMT+0 and i set the device to GMT-1 and now the time shows by the platform is right. but please what about the Last time update it is still wrong why did it goes at 2 hours in plus?
this the picture

thanks again for your  help

I didn't ask you to change anything. You need to set your device to send GMT+0.

Hi Mr tananaev
yes when i did it the time was already wrong. so
i set the gt06.tomezone to 0 and i set the device to GMT-1 and in this way
the time is correct now.
but in both configuration
the last time update is still wrong.
Joel TANZOUAK VAUMI Stagiaire Informaticien à mégasoft.
PhD student in Systems and Software in Distributed Environment
University of Ngaoundere Cameroun.
Tel: +237 695325716/ 672604905
Profil linkedIn.
https://www.linkedin.com/in/joel-tanzouak-vaumi-864740a6?trk=nav_responsive_tab_profile
2016-07-02 0:15 GMT+02:00 Anton Tananaev<EMAIL_ADDRESS>
I didn't ask you to change anything. You need to set your device to send
GMT+0.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tananaev/traccar/issues/2064#issuecomment-230058980,
or mute the thread
https://github.com/notifications/unsubscribe/ATRbXNVtgVDevge7Yds2Fb4pKFD8YeP6ks5qRZFygaJpZM4JCuLg
.

If you can set device to GMT+0, then you don't need to change anything in the config.

ok Mr tananaev thanks very much for this solution.
but what about the last time update it is still wrong?

Last update is your server time.

please the VPS time or the traccar server time?

i ask because in the database the server time is 13:43:50
but in the plateforme the last update is 15:43:50 i don't know why?

What's the difference between VPS time and Traccar server time? It should be the same thing.
Look, you need to tell me the timezone, not just time. Otherwise there is no way to know what's happening and what the problem is. Where do you see the database time? What's the timezone? What's the timezone in the browser that you are using to open web console?

the time zone of the server is UTC (traccar and the database are in the same server), the  time zone of the browser is UTC+1

That is not possible. You must be missing something. Did you change timezone on the server from the default one?

yes

Please read this thread: https://github.com/tananaev/traccar/issues/1522.

everything is ok now Mr tananaev thank you very much for your help


Thank you!

Hey, I hope you don't mind me opening an issue here (I wasn't sure how else to contact you)
Just wanted to say thanks for publishing such a small, effective package for polygon offsetting in JS - I've tried a few and they are usually HUGE, or don't work quite how I want them to.
Cheers :)

Hehe of course I don't mind, you made my day, thank you :)
I had the exactly same problem and once I found Hans' CodePen I thought - finally. I made a few changes and decided to release it on npm. I hope Hans won't mind 🙈

Ah! I'm glad to hear it! ❤

@georgedoescode I wasn't sure how to contact you either 🙈 so I sent you a linkedin request.
In other news, I found this library https://github.com/xaviergonz/js-angusj-clipper (a js clone of C clipper) which is amazing! It does all boolean operations and offsetting polygons. Documentation is a little bit hard to navigate, but the library is really good (and fast).


Fix active state in SingleSelect when clicking or using key navigation

Proposed Changes

When playing with the value selection in SingleSelect, we were not previously updating the activate state. It was only updated when using the navigation key.
I now keep track of the item index and change the active cells when selecting an item through a click or key navigation.
Potential Breaking Changes

Acceptance Criteria

[x] The proposed changes are covered by unit tests
[ ] The potential breaking changes are clearly identified
[x] README.md is adjusted to reflect the proposed changes (if relevant)

Codecov Report

Merging #1227 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1227      +/-   ##
==========================================
+ Coverage   99.65%   99.66%   +<.01%
==========================================
  Files         354      354
  Lines        8971     8927      -44
  Branches     2137     2112      -25
==========================================
- Hits         8940     8897      -43
+ Misses         31       30       -1

Impacted Files
Coverage Δ

...ges/react-vapor/src/components/itemBox/ItemBox.tsx
96.87% <ø> (ø)
:arrow_up:

...ges/react-vapor/src/components/listBox/ListBox.tsx
98.03% <100%> (+0.77%)
:arrow_up:

...ct-vapor/src/components/select/SelectConnected.tsx
100% <100%> (ø)
:arrow_up:

...-vapor/src/components/listBox/ListBoxConnected.tsx
100% <100%> (ø)
:arrow_up:

...act-vapor/src/components/listBox/ListBoxActions.ts
100% <100%> (ø)
:arrow_up:

...ct-vapor/src/components/listBox/ListBoxReducers.ts
100% <100%> (ø)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e89a21...8dfcda4. Read the comment docs.


cmake:fix add_user_library module archive issue

Summary

[!NOTE]
complete the issue for this discussion https://github.com/apache/nuttx/issues/10191

[x] correct the behavior of  nuttx_add_user_library
[x] make STATIC target export in BINDIR/staging/ directory by cmake --install

Impact
Testing
cmake build

How can i link this PR to the https://github.com/apache/nuttx/issues/10191 through right sidebar Development ？

How can i link this PR to the #10191 through right sidebar Development ？

Done.


Windows is not stable to run GPU Keras Tensorflow? & Memory Issue

I run the Keras with Tensorflow GPU backend.
I run two programs in two Jupyter notebooks. I tried to run two programs one after another without shutting down any notebooks, the PC stop running, thus I have to restart PC.
If I shutdown 1st notebook and run the other one, it won't have the same problem.
I tried the following code to release memory, but the problem is still there unless I shutdown the notebook.
from keras import backend as K K.clear_session()
So two questions:

Is that due to unstable status of Windows?
How to clean the memory of both Keras and Tensorflow? It seems both parameters and models take a lot of memories.

I am very new, but this is my understandings:
In ubuntu, with the nvidia-smicommand I can list the processes on GPU and their memory consumption. One instances of Keras (or TF) is getting almost all free memory on GPU. So I guess if you keep it running on jupyter notebook, No memory would left for the second instance.


Add enum map for property help descriptions

Description:

Currently, all help descriptions for a property are stored in XML files -- which means they are duplicated in every gen class that uses them. That complicates changing the help description for a property, since one must find every occurence. That issue is going to become worse with the addition of wxPython support, since descriptions will need to be updated if they need to add something specific about python (such as "Not supported by wxPython").
Long after the creation of the XML interface files, we added the property enum map that maps the prop_ enum to it's display and interface name. Adding a second map to map the prop_ enum to a help description would make it much easier to find the property help description so that it can be updated. It would also avoid all duplication, as well as cutting down on the size of the XML file to reduce the chance of overflowing the 16k limit.

Not everything can be moved from XML to the new property map introduced in PR #858 -- only true duplicates, or if the description can be made generic enough to fit everything.
Note that the generator can override any help description, so if most are duplicates, a special-case could be handled by a generator.
Since all property help descriptions now go through a single function, it would be relatively easy to add to the text replacement as long as the replacement text can be obtained via a Node* pointer.

I've verified that this is working as expected. However, going through all the properties to determine whether or not to move the descriptions is not worth delaying the 1.1 release for, so I'm removing the milestone for this issue.


metron agent does not recover from intermittent consul outage

In various of our CF deployments (version 245, both openstack and AWS), we sporadically see various VMs no longer reporting via metron agent.
As an example, on hm9000 the stderr logs of metron_agent have messages like
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:41:11 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
panic: No dopplers listening on [udp]

goroutine 54 [running]:
panic(0x960700, 0xc4204490d0)
        /var/vcap/data/packages/golang1.7/48ce7b3927e6f7e6576fff21a7dc9d22b7ba9c62.1-e04b1de537d7998b284ea1d9ad820d48b6ddf15a/src/runtime/panic.go:500 +0x1a1
metron/clientreader.Read(0xc420234030, 0xc420448ca0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /var/vcap/data/compile/metron_agent/loggregator/src/metron/clientreader/client_reader.go:17 +0x393
main.initializeDopplerPool.func1(0xc420234030, 0xc420096780, 0xc4200730e0, 0xc4200c5c20, 0xc4202f0140, 0xc420234060)
        /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:205 +0x169
created by main.initializeDopplerPool
        /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:209 +0x63d
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:42:02 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
panic: No dopplers listening on [udp]

goroutine 69 [running]:
panic(0x960700, 0xc420434970)
        /var/vcap/data/packages/golang1.7/48ce7b3927e6f7e6576fff21a7dc9d22b7ba9c62.1-e04b1de537d7998b284ea1d9ad820d48b6ddf15a/src/runtime/panic.go:500 +0x1a1
metron/clientreader.Read(0xc420246030, 0xc420434590, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /var/vcap/data/compile/metron_agent/loggregator/src/metron/clientreader/client_reader.go:17 +0x393
main.initializeDopplerPool.func1(0xc420246030, 0xc420096640, 0xc420073200, 0xc4200c5700, 0xc420304200, 0xc420246060)
        /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:205 +0x169
created by main.initializeDopplerPool
        /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:209 +0x63d
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)
2016/11/24 16:46:33 RuntimeStats: failed to emit: EventWriter: No envelope writer set (see SetWriter)

There were several monit restarts following the panic lines above. The last monit restart was unable to SetWriter,
but did not exit the process. As a result the stdout logs of metron_agent are full of messages like
{"timestamp":1480424571.683505297,"process_id":6298,"source":"metron","log_level":"warn","message":"EventMarshaller: Write called while byteWriter is nil","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/writers/eventmarshaller/event_marshaller.go","line":59,"method":"metron/writers/eventmarshaller.(*EventMarshaller).Write"}
{"timestamp":1480424572.683612823,"process_id":6298,"source":"metron","log_level":"warn","message":"EventMarshaller: Write called while byteWriter is nil","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/writers/eventmarshaller/event_marshaller.go","line":59,"method":"metron/writers/eventmarshaller.(*EventMarshaller).Write"}

Digging into the details, we found that just before the first metron_agent panic above, we find in the consul_agent logs of the same VM:
==> Log data will now stream in as it occurs:

    2016/11/24 16:40:59 [INFO] serf: EventMemberJoin: hm9000-z1-0 <REDACTED>
    2016/11/24 16:40:59 [WARN] manager: No servers available
    2016/11/24 16:40:59 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/11/24 16:41:00 [WARN] agent: Check 'service:hm9000' is now critical

    2016/11/24 16:41:03 [WARN] agent: Check 'service:hm9000' is now critical
    2016/11/24 16:41:06 [WARN] agent: Check 'service:hm9000' is now critical
    2016/11/24 16:41:09 [INFO] agent: Synced check 'service:hm9000'

The consul_agent recovered in the meantime, e.g. we can successfully dig cf-etcd.service.cf.internal
However metron_agent is still in error state but does not exit the process, so the problem can go unnoticed for a long time until you notice that logs or metrics are missing.
Workaround is to monit restart both consul_agent and metron_agent.

this looks similar to issue #169

It turned out that the root cause was a corrupted etcd cluster.
After following https://github.com/cloudfoundry-incubator/etcd-release#failed-deploys-upgrades-split-brain-scenarios-etc most metron agents recovered.
A few VMs still needed a manual monit restart metron_agent
Up to you if you want to keep this open to make metron_agent more robust against etcd failing and then recovering.

Thanks for confirming @jsievers. We have seen similar issues. We have plans to improve our service discovery and key value store that should address this in a future release.


Support more prefixes, according to official docs

Hello, and many thanks for your efforts
According to official Docs https://www.conventionalcommits.org/en/v1.0.0/, we are able to use these prefixes too

build:, chore:, ci:, docs:, style:,

but are not supported yet. Is is ease for you to add them?

These are already supported...

'Chore' at least, doesn't seem to be recognized
You may see the action ran

and the action.yml below

You need to define the patchList you want, as explained in https://github.com/ietf-tools/semver-action#inputs
chore is excluded by default.

This was the backup plan. :wink:  Thank you


[audio/simpleaudiosample]  devcon install failed

OS: win10 21h2
VS: 2020
ERROR: devcon.exe failed!

LOG: in C:\Windows\INF\setupapi.dev.log
     cmd: .\devcon-x64.exe  install SimpleAudioSample.inf Root\SimpleAudioSample
     ndv: INF path: C:\SimpleAudioSampleDriver\SimpleAudioSample.inf
     ndv: Install flags: 0x00000001
     ndv: {Update Device Driver - ROOT\MEDIA\0001}
     ndv:      Search options: 0x00000080
     ndv:      Searching single INF 'C:\SimpleAudioSampleDriver\SimpleAudioSample.inf'
     dvi:      {Build Driver List} 17:47:02.148
     dvi:           Searching for hardware ID(s):
     dvi:                root\simpleaudiosample
     dvi:      {Build Driver List - exit(0x00000000)} 17:47:02.216
!    ndv:      No matching drivers found in single INF
     dvi:      {DIF_SELECTBESTCOMPATDRV} 17:47:02.217
     dvi:           Default installer: Enter 17:47:02.219
     dvi:                {Select Best Driver}
!    dvi:                     Selecting driver failed(0xe0000228)
     dvi:                {Select Best Driver - exit(0xe0000228)}
!    dvi:           Default installer: failed
!    dvi:           Error 0xe0000228: There are no compatible drivers for this device.
     dvi:      {DIF_SELECTBESTCOMPATDRV - exit(0xe0000228)} 17:47:02.222
!    ndv:      Unable to select best compatible driver. Error = 0xe0000228
     ndv:      No drivers found for device.
     ndv: {Update Device Driver - exit(00000103)}
!    ndv: No better matching drivers found for device 'ROOT\MEDIA\0001'.
!    ndv: No devices were updated.

REASON: inf file (autogen by vs) has problems
FIX:
[Manufacturer]
;%MfgName%=SIMPLEAUDIOSAMPLE,NTamd64.10.0...22000
%MfgName%=SIMPLEAUDIOSAMPLE,NTamd64

;[SIMPLEAUDIOSAMPLE.NTamd64.10.0...22000]
[SIMPLEAUDIOSAMPLE.NTamd64]
%SIMPLEAUDIOSAMPLE_SA.DeviceDesc%=SIMPLEAUDIOSAMPLE_SA, ROOT\SimpleAudioSample

PS: delete sdk version number in section name, then you can install successfully!

@microsoft/windowsaudio can you take a look?


Uploader code refactored for handling one file upload at a time

r? @shankar2105

r? @ustulation
(maidsafe_highfive has picked a reviewer for you, use r? to override)

Reviewed 7 of 13 files at r1, 6 of 6 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from the review on Reviewable.io


Adding Paths, Folders to SettingType closes#1

Good)) But it also means support for forms for these types

For example: these are the MoviesRunner settings. You need to create one field that takes several values: like the PATH environment variable in Windows10

Ah alright. Will work on adding that as soon as I resolve a few dependency issues preventing me from building. Felt like it was too simple of a request. Thanks for the clarification.

The program is my personal development, so it would be interesting to know about these issues. Create a separate Issue for them, please.

Sure no problem. It is just a single nuget package missing for SharpShell, but having some permission issues on writing to the registry that I would like to read through first before I run with privileges. I can create a case for the nuget package if you would like for formality.

I also fixed one dependency problem in the last commit. I should have done it before

Regarding SharpShell - here you can get additional information. I fixed the problem of the existing version and use my version of the library

This library itself is needed to create this panel:

That merge was accidental. Forgot I had some commands aliased for work while trying to rebase with upstream locally. If I need to revert any changes from that let me know. Need to switch to my other git client.

https://github.com/HavenDV/HomeCenter.NET/wiki/Adding-a-new-module-to-the-program

https://github.com/HavenDV/HomeCenter.NET/issues/1


Replay continues after next page

If I begin a replay, then click the arrow -> key or Next button to accept it, the replay continues.  I expected the replay to halt.

Made the below UI improvements here

Disable review button when an audio plays
Disable next button when an audio plays
Avoid jumping to next prompt when a user clicks arrow -> key during replay


Error "tcp: too many colons in address :2376" when executing env command

I've successfully created a machine using openstack driver, but when I execute docker-machine env <my-machine> I get an error:
  › docker-machine -D env os01f

Docker Machine Version: 0.5.0 (04cfa58)
Found binary path at /usr/local/bin/docker-machine-driver-openstack
Launching plugin server for driver openstack
Plugin server listening at address <IP_ADDRESS>:61158
() Calling RpcServerDriver.GetVersion
Using API Version 1
() Calling RpcServerDriver.SetConfigRaw
() Calling RpcServerDriver.GetMachineName
(os01f) Calling RpcServerDriver.GetState
(os01f) DBG | Get status for OpenStack instance...		 MachineId=<machine-id>
(os01f) DBG | Authenticating...		 AuthUrl=<openstack-auth-url> DomainID=default DomainName= Insecure=false TenantID= TenantName=<tenant-name> Username=<username>
(os01f) DBG | State for OpenStack instance		 MachineId=<machine-id> State=ACTIVE
(os01f) Calling RpcServerDriver.GetURL
Reading CA certificate from /Users/varankinv/.docker/machine/certs/ca.pem
Reading server certificate from /Users/varankinv/.docker/machine/machines/os01f/server.pem
Reading server key from /Users/varankinv/.docker/machine/machines/os01f/server-key.pem
Error running connection boilerplate: Error checking and/or regenerating the certs: There was an error validating certificates for host "<my machines ipv6 address>:2376": dial tcp: too many colons in address <my machines ipv6 address>:2376
You can attempt to regenerate them using 'docker-machine regenerate-certs name'.
Be advised that this will trigger a Docker daemon restart which will stop running containers.

My machine is running inside IPv6-only network and looks like the only way to manage it with docker-machine is to specify $DOCKER_HOST env variable manualy using machine's fqdn.
At the same time I can ssh to machine with docker-machine ssh <my-machine> as well as run other commands.

I got this same issue when trying to create a machine on Hyper-V
(default) DBG | [executing ==>] : C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile (( Get-VM -Name default ).networkadapters[0]).ipaddresses[0]
(default) DBG | [stdout =====>] : fe80::215:5dff:fe20:b50f
(default) DBG |
(default) DBG | [stderr =====>] :
(default) Calling RpcServerDriver.GetSSHPort
(default) Calling RpcServerDriver.GetSSHKeyPath
(default) Calling RpcServerDriver.GetSSHUsername
SSH binary not found, using native Go implementation
About to run SSH command:
exit 0
Error dialing TCP: dial tcp: too many colons in address fe80::215:5dff:fe20:b50f:22


Gamemode Adjustments: Dynamic Just A Week Away Edition

About the PR
A decent amount of gamemode adjustments! - event and secret Weight changes, new midround antag events [Late Ling and Heretic], survival removal, new trifecta gamemode, traitors rolling at 30-60 delay, revs at 20 player ratio, and changeling dual antag gamemodes have received an increase in changelings.
Why / Balance
Survival Removal - Legitimately unfun. Better than kessler for the fact antags eventually exist- but the same exact problem: absolutely zero player created threat but extremely chaotic. Never really that fun of a gamemode and the weight's better shifted elsewhere, we have the antags to fill the "void" that's there. No, I'm not counting miss flukie "Here and Dead", and Weeb Antisocial "Never sees the crew ever" the Ninja as player created threat.
Weight Changes - Nukies have been decreased to 15% and from an overall 22% Spawnrate at 30+ Ready to 18% as Nukelings have been increased to 2%. Had enough of 5 nukler games in a row? Here's your fix :3 - Kinda. Barely a spawnrate decrease at 30+ Players ready, but a pretty major one at lowpop. As there's 9% Left from these gamemodes, 4% went to Lings [15%], 3% to Traitorlings [8%], and 2% for the new gamemode, Trifecta.
Trifecta - Traitors, Lings, and Heretics at the same time. 30 Players like other multi antag gamemodes. Generally just a way to let heretics roll w/ Traitorlings while not having an increased antag count.
Antag Event Weight Decreases & New Mid Antag Events- Makes games generally last longer than the current 1 hour average as ghost role antags have a 50% Decrease and Sleeper agents a 25% Decrease to their chances. Midround Antags are more common however by 50% due to the new Genome Awakening [Late Changeling] and Sudden Enlightenment [Late Heretic] events at half the weight of Sleeper Agent. Same announcement w/ a max of 2 Lings and 1 Heretic respectively. All Midround Antag events now start at 20 minutes, including sleeper agents.
CalmLing Changes - Calmling has only 2 Lings compared to CalmTraitor's 5 agents. This generally neutered the gamemodes that use it. Calmling now has a max of 4 but a player ratio of 20, as lings are generally stronger than agents. Traitorling now has a max of 9 antags. Dual antag games can't even roll sub gamemodes and Traitorling/Revling/Nukeling had less antags than normal games and their agent counterparts even before sub gamemodes. This fixes that.
Traitor Roll Delay Change - Every antag similar to traitor now uses 30-60 delay, and antag rolling may actually be easier when people know the times they roll at :godo: - So I made them match their counterparts at 30-60 delay. This generally places them at the same start as their counterparts, and prevents people from just cryoing after 8 minutes have passed to antag roll with barely any trace.
Revs at 20 Player Ratio - Revs generally have an easier time here than Wizard's den for example. also, having less at lower player counts and preventing them at player counts that normally barely ever have more than 1-2 command/sec helps them not crush the command staff that easily. That starting flash isn't worth an entire third of the crew anymore!
Technical details
0 C#, but the heretic gamerule was moved to the correct gamerules folder.
Requirements

[X] I have read and am following the Pull Request and Changelog Guidelines.
[X] I have added media to this PR or it does not require an ingame showcase.

Changelog
:cl:

add: The Trifecta gamemode with Traitors, Changelings and Heretics has been added at a 2% Chance.
add: You can now roll Changeling or Heretic during an Enemy Communication instead of just Sleeper Agent.
remove: The Survival gamemode has been removed. Survival's Death Party at MY PLACE!!!
tweak: Nukies have a decreased chance to spawn with their base gamemode at 15% from 20%, and Changelings an increased amount, with changelings at 15% from 10%, TraitorLing at 8% from 5%, and NukeLing at 2% from 1%.
tweak: Ghost role antags spawn at a halved chance, but mid-round antags at an increased chance. Generally, rounds shall be calmer later into the game, allowing longer rounds to be more common.
tweak: Traitors now roll at the 30-60 second mark, like Changelings and Heretics.
tweak: Revs need more players to start as a game, and have a higher player ratio per headrev at 20 from 15.
tweak: Changeling Dual Antag gamemodes now actually spawn a decent amount of changelings. No longer do they always spawn 2 of them!

Summary by CodeRabbit
Summary by CodeRabbit

New Features

Introduced new game events: "GenomeAwakening" and "SuddenEnlightenment" for enhanced gameplay dynamics.
Added a new game preset "Trifecta" combining elements of existing modes.

Gameplay Adjustments

Reduced occurrence rates of certain events to improve game balance.
Adjusted timing and player requirements for various game rules, allowing for quicker starts and more dynamic gameplay.

Bug Fixes

Recalibrated antagonist probabilities for better balance in gameplay.

I'm in support of this. I think it could help reduce the amount of people I see cryoing before 10 minutes, and encourage people to play the whole round.

Guh test fail

there goes my free ninja gamemode 😔
Nice changes though.

From the test it seems secret isn't picking a preset...
But it's excluding E V E R Y preset. I'm like 90% sure there's 0 reason for this. i'll test a couple of changes and if none including reverts work the check's just being dumb.

i'm currently working on some sort of dynamic gamemode which will work this way: one major antag will be selected (nukies, revs, or a decent amount of solo antags), and 0-2 minor antags or gamerules coming with them depending on the amount of points gathered during the round which will also affect other gamerules spawning during the round.
so trifecta and other dual antag gamemodes will be cut off.
leaving midround lings and heretics is a godo, changing gamerule timing on traitors is a godo.
you should also change cling and heretic midround message to something else. there's plenty of voicelines and messages you can pick from (in 13), such as "N anomaly detected near the station. There is no additional data."; "Level 5 biological hazard detected aboard the station. All personnel must contain the outbreak." (which is currently used in zombies and will be used in blob when i get to it)
overall good PR

i'm currently working on some sort of dynamic gamemode which will work this way: one major antag will be selected (nukies, revs, or a decent amount of solo antags), and 0-2 minor antags or gamerules coming with them depending on the amount of points gathered during the round which will also affect other gamerules spawning during the round.
so trifecta and other dual antag gamemodes will be cut off.
leaving midround lings and heretics is a godo, changing gamerule timing on traitors is a godo.
you should also change cling and heretic midround message to something else. there's plenty of voicelines and messages you can pick from (in 13), such as "N anomaly detected near the station. There is no additional data."; "Level 5 biological hazard detected aboard the station. All personnel must contain the outbreak." (which is currently used in zombies and will be used in blob when i get to it)
overall good PR

The point of using Enemy Communication is to alert the crew there's a new antag but not what exact antag. if desired I'm able to change them still, I'd use anomaly for heretics and find a fitting one for lings.
Also i'm gonna remove trifecta to test if that makes the tests not be dumb anyways, so that makes me care a bit less that I'm nuking it. Also maybe add a couple of roundstart minor/solo antags when you do dynamic to the major antags to do the role of their dual antag variants by the way. Great to hear dynamic's actually a real week away.

Test fail seems unrelated to my PR, how did adding trifecta kill secret as a gamemode? who knows :3

The point of using Enemy Communication is to alert the crew there's a new antag but not what exact antag.

also can you do it like in 13? when there's traitors regardless of if it's midround or roundstart the message will still play

The point of using Enemy Communication is to alert the crew there's a new antag but not what exact antag.

also can you do it like in 13? when there's traitors regardless of if it's midround or roundstart the message will still play

pretty sure it's any antag that's not "external" actually. external = nukies/wizard and there's a chance for it to not happen even when there's internal antags, and a chance for it to happen with no internal antags. it's a dynamic thing pretty sure.

the conflicts are because those files were moved to Resources/Prototypes/_Goobstation/

@TooSillyFennec

Dynamic was a week away after all!!!! :3


Followed the steps in README.md , But it doesn't works, seems to be incompatible with node v13

Can you upgrade node to >=13.7 version? It looks like some package isn't compatible only with version from 13.3.0 to 13.7 range.


scVI change corrected counts for tasks

Hi,
Thanks for putting the benchmark together. I'm happy to have a look why the results for integration are that different from e.g. OpenProblems https://openproblems.bio/results/batch_integration_embed/. It might help to use a more recent version (the one used in the environment is very outdated). I share your concerns with the presented results but am not certain why the performance is that terrible. The code looks correct.
However, I'm mainly posting as the use of corrected counts is misplaced for scVI. We would highly recommend using a posterior predictive sample for the DE tests performed here. Those are still imputed but are samples from the generative distribution instead of the mean of the generative distribution. If we recommend using get_normalized counts somewhere in our tutorials/documentation, it would be helpful to get that as feedback.
pp_counts = model.posterior_predictive_sample(                 model.adata,                 n_samples=1, # makes it most comparable with sparsity of input data.             )

@canergen I am keen to see what the outcome is. Could you keep this thread updated?
Am I right in my interpretation that the design seems to be quite different between the open problems and this benchmark?
The open problems benchmark seems to use actual dataset batches whereas this uses a single dataset split into artificial batches.
I agree with the authors that any integration should not introduce changes in cluster assignments if all “synthetic” batches are devoid of any true batch effects. The only caveat I see of using the datasets here is that the cell numbers per batch would be quite small so could scvi be underperforming due to tiny train/validation splits?
Also, curious to know if the authors considered using two identical cellxgene matrices as artificial batches? The random sampling process would still generate differences in cell composition that can be considered a true sampling “batch” effect. Therefore overly harsh integration methods such as seurat and harmony may be performing better purely in this scenario, but may perform less well conceptually when over integration isn’t desired. Eg batches representing different tissue sources etc.

@canergen Thank you for your comments. Is there any published material on any updates you have made in new versions of scVI?
On your points with the posterior predictive samples and fixing the seed. Those are very important points and we will update that in our workflow and update results to reflect changes, if any, that we find
@Nusob888 Thank you for your comments. Our mouse brain data has 20k cells for example. We think that the data we chose represents well the types of data that researchers are using these methods on. At one point in our process I did use two identical matrices. It introduced a lot of strange behaviour and could be criticised as a simulation strategy in various ways, so we didn't go further with the idea.

Hi @baroona . https://docs.scvi-tools.org/en/stable/release_notes/index.html are our release notes. One of the changes is that in scVI<1.0.0 the seed is fixed (so you did everything correct). I wouldn't expect strong variation of results when updating scvi-tools but it generally makes your and our life easier to stick to a recent version.
We made the point with varying seeds in our Colab notebook (https://colab.research.google.com/drive/15k5kahlT5qSdXNUdkqeEj6je8-nbR_Vh?usp=sharing) to put conservation of NN into perspective. Adding a random batch_key might have an impact (especially when using encode_covariates=True as used in some of our notebooks, e.g. in https://docs.scvi-tools.org/en/stable/tutorials/notebooks/scrna/scarches_scvi_tools.html). However, the effect of retraining with another random seed is larger. I tried encoding covariates out in our Colab notebook and found no strong effects (results not included in the notebook to keep it simple).
It's generally helpful to train scVI with large datasets and this increases conservation of NN with different random seeds (I tried out scvi.dataset.retina() ). However, I think it's fair to use 5k cells in the benchmarking. I also don't think that the process of adding a random batch key is producing biased results.
The major bias comes from the fact that the same PCA is used for e.g. harmony and the raw data. This favours results for harmony (when it does no correction you get perfect conservation). We are more in favor of validating conservation of biological entities like celltypes (with the caveat that ground-truth for biological data is difficult). However, to keep close to the current benchmarking you could compare all methods against projection to PCs computed on a reference dataset or use a factor model for the reference embedding of the dataset, e.g. NMF. Otherwise, the message reduces to: If you want to get a result close to PCA, use a method that relies on the same PCA.
For differential expression, it's more surprising that it works that well. Benchmarks influence how people are performing their analysis and we try to keep method use as sound as possible. You have our strong support that count data shouldn't be corrected for downstream analysis.
@Nusob888: We summarized our ideas in a Twitter style https://twitter.com/_canergen/status/1772190381871907122 as well as the accompanying notebook.
I'm closing here.


Can't train mamba2 from scratch with HF Trainer

I'm trying to train mamba2 130m from scratch.
config = Mamba2Config(
            vocab_size=len(tokenizer.vocab),
            n_positions=10,
            n_embd=768,
            n_layer=12,
            n_head=12,
            n_inner=3072,
        )
        model = Mamba2ForCausalLM(config)

    training_args = TrainingArguments(
    output_dir=args.output_dir,
    logging_dir='./logs',
    gradient_accumulation_steps=1,
    save_steps=2000,
    max_steps=1500000,
    evaluation_strategy="steps",
    eval_steps=2000,
    #prediction_loss_only=True,
    logging_strategy="epoch",
    #logging_steps=500,
    learning_rate=1e-4,
    # save_total_limit=999999,
    fp16=True,
    dataloader_num_workers=4,
    per_device_train_batch_size=512,
    per_device_eval_batch_size=512,
    lr_scheduler_type="constant_with_warmup",
    weight_decay=0.1,
    warmup_steps=2000,
    )

    trainer = Trainer(
        model=model,
        tokenizer=tokenizer,
        args=training_args,
        train_dataset=tokenized_train,
        eval_dataset=tokenized_eval
    )

trainer.train()

Error:
File "/users/PAS2581/kanaka/research/GrokkedTransformersarewang2024/trying_different_archs/mamba/main.py", line 542, in <module>
    main()
  File "/users/PAS2581/kanaka/research/GrokkedTransformersarewang2024/trying_different_archs/mamba/main.py", line 512, in main
    trainer.train()
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/trainer.py", line 1938, in train
    return inner_training_loop(
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/trainer.py", line 2279, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/trainer.py", line 3318, in training_step
    loss = self.compute_loss(model, inputs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/trainer.py", line 3363, in compute_loss
    outputs = model(**inputs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/accelerate/utils/operations.py", line 819, in forward
    return model_forward(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/accelerate/utils/operations.py", line 807, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 43, in decorate_autocast
    return func(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/models/mamba2/modeling_mamba2.py", line 1048, in forward
    mamba2_outputs = self.backbone(
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/models/mamba2/modeling_mamba2.py", line 900, in forward
    hidden_states = mixer_block(
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/models/mamba2/modeling_mamba2.py", line 648, in forward
    hidden_states = self.mixer(
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/models/mamba2/modeling_mamba2.py", line 607, in forward
    return self.torch_forward(hidden_states, cache_params, cache_position, attention_mask)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/models/mamba2/modeling_mamba2.py", line 531, in torch_forward
    L = torch.exp(segment_sum(A))
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/models/mamba2/modeling_mamba2.py", line 105, in segment_sum
    tensor_segsum = torch.cumsum(input_tensor, dim=-2)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB. GPU 0 has a total capacity of 79.15 GiB of which 3.70 GiB is free. Including non-PyTorch memory, this process has 75.44 GiB memory in use. Of the allocated memory 74.80 GiB is allocated by PyTorch, and 139.31 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.

where,
GPU 0: NVIDIA A100-SXM4-80GB
  GPU Utilization: 0%
  Memory Utilization: 0%
  Total Memory: 81920.00 MB
  Free Memory: 52246.00 MB
  Used Memory: 29674.00 MB

You're missing either the mamba-ssm package and/or causal-conv1d package:
return self.torch_forward(hidden_states, cache_params, cache_position, attention_mask) <-- this is the slow path, unoptimized and using a lot of memory
Hf only opts for the fast path, i.e. the kernels, if both are available (there should've been a warning). So consider installing the packages. For the future, these type of questions are more appropriate on the transformers repo ;)

Thanks for the response. I agree that it is more appropriate to post on the tranformers repo. I'll make sure to do that next time.

I've tried it with fast path too. I don't see this error when I work with Mamba with same hyperparams (instead of mamba2).
    out, ssm_state = mamba_split_conv1d_scan_combined(
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/ssd_combined.py", line 930, in mamba_split_conv1d_scan_combined
    return MambaSplitConv1dScanCombinedFn.apply(zxbcdt, conv1d_weight, conv1d_bias, dt_bias, A, D, chunk_size, initial_states, seq_idx, dt_limit, return_final_states, activation, rmsnorm_weight, rmsnorm_eps, outproj_weight, outproj_bias, headdim, ngroups, norm_before_gate)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/autograd/function.py", line 574, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 455, in decorate_fwd
    return fwd(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/ssd_combined.py", line 795, in forward
    out_x, _, dt_out, dA_cumsum, states, final_states = _mamba_chunk_scan_combined_fwd(x, dt, A, B, C, chunk_size=chunk_size, D=D, z=None, dt_bias=dt_bias, initial_states=initial_states, seq_idx=seq_idx, dt_softplus=True, dt_limit=dt_limit)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/ssd_combined.py", line 317, in _mamba_chunk_scan_combined_fwd
    states, final_states = _state_passing_fwd(rearrange(states, "... p n -> ... (p n)"), dA_cumsum[:, :, :, -1],
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/ssd_state_passing.py", line 205, in _state_passing_fwd
    out = torch.empty((batch, nchunks, nheads, dim), device=states.device, dtype=out_dtype)

This seems like a partial stack trace, no? Could you share the complete one?

/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/selective_scan_interface.py:164: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  def forward(ctx, xz, conv1d_weight, conv1d_bias, x_proj_weight, delta_proj_weight,
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/selective_scan_interface.py:240: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, dout):
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/layer_norm.py:986: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  def forward(
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/layer_norm.py:1045: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, dout, *args):
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/distributed/tensor_parallel.py:26: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  def forward(ctx, x, weight, bias, process_group=None, sequence_parallel=True):
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/distributed/tensor_parallel.py:62: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, grad_output):
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/ssd_combined.py:758: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  def forward(ctx, zxbcdt, conv1d_weight, conv1d_bias, dt_bias, A, D, chunk_size, initial_states=None, seq_idx=None, dt_limit=(0.0, float("inf")), return_final_states=False, activation="silu",
/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/mamba_ssm/ops/triton/ssd_combined.py:836: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  def backward(ctx, dout, *args):
Traceback (most recent call last):
  File "/users/PAS2581/kanaka/research/GrokkedTransformersarewang2024/trying_different_archs/mamba/main.py", line 575, in <module>
    main()
  File "/users/PAS2581/kanaka/research/GrokkedTransformersarewang2024/trying_different_archs/mamba/main.py", line 486, in main
    model.to(device)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2905, in to
    return super().to(*args, **kwargs)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1174, in to
    return self._apply(convert)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 780, in _apply
    module._apply(fn)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 780, in _apply
    module._apply(fn)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 780, in _apply
    module._apply(fn)
  [Previous line repeated 2 more times]
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 805, in _apply
    param_applied = fn(param)
  File "/users/PAS2581/kanaka/miniconda3/envs/grokk/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in convert
    return t.to(
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 128.00 MiB. GPU 0 has a total capacity of 15.77 GiB of which 125.12 MiB is free. Including non-PyTorch memory, this process has 15.65 GiB memory in use. Of the allocated memory 15.35 GiB is allocated by PyTorch, and 1.23 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

It seems to me that with your config initialization, you pass the wrong arguments:
config = Mamba2Config(
            vocab_size=len(tokenizer.vocab),
            n_positions=10,
            n_embd=768,
            n_layer=12,
            n_head=12,
            n_inner=3072,
        )

For example, there is no argument n_layer so you will get the defaults of codestral mamba (a 7b model) and then it doesn't surprise me that you won't have enough memory. See https://github.com/huggingface/transformers/blob/main/src/transformers/models/mamba2/configuration_mamba2.py for the config file. You could also get the defaults from the conversion script in source https://github.com/huggingface/transformers/blob/main/src/transformers/models/mamba2/convert_mamba2_ssm_checkpoint_to_pytorch.py (and inspect the values).
Also this time it's a different GPU? Looks like 16GB total RAM instead of 80.

Those configs are not compatible with hf transformers. For mamba(1) there exist compatible variants with "-hf" at the end, e.g. "state-spaces/mamba-130m-hf".
I'd assume that the default config values would be loaded instead then which are 2.7b and 7b models respectively for mamba(1) and mamba(2). Then you will more easily encounter RAM issues with mamba(2). You should be able to check a hf model parameter size with .num_parameters().
The previous link to the config was meant for the entries (that are possible), but since hf and here have different conventions it might be hard to see what to change. The conversion script might help there since you can use it to convert a model from here to hf format - this way you pretty much have the correct default values (for the config).

I understand that the configurations are not directly compatible with HuggingFace, and that using the conversion script can help obtain the correct default values for the configuration.
I’m having difficulty locating the Mamba2 130M checkpoint files necessary to use the conversion script. Could you please guide me on how to obtain these checkpoint files? Are they available for download, or is there a repository where I can access them?

Ah sure. You'll need git lfs, so potentially call git lfs install before. Afterwards, you could just clone the original repository at mamba2-130m, i.e. git clone https://huggingface.co/state-spaces/mamba2-130m.
If you want to wait a bit, I can upload the converted weights in fp32?

Sure, Thanks. Please give me an update when you did so.

Thanks for providing the Mamba2 130M configuration and weights.
Quick questions, I’m training the 130M model from scratch on an A100, with the same setup and parameters as GPT2. I’ve noticed that Mamba2 130M is taking about three times longer to train than GPT2 (150 hours vs. 50 hours).
Is this longer training time expected due to architectural differences, or is there something I might be missing to optimize training speed for Mamba2?

No problem.

This seems a bit unlikely tbh. Have you ensured that mamba-ssm and causal-conv1d are installed? Maybe set config.use_cache=False during training at least. Otherwise, I'm out of quick options. Mamba2 is also more suitable for bigger architectures and longer sequences so it might be faster in comparison when using 1b params + 4k seq lengths for example.
Can you pass safe_serialization=False when saving your model? Tbh, weight sharing is the standard in Mamba2 so you should not disable it. But, tied embeddings are a mixed thing; it might be beneficial or not.

@Jellymoon
Only thing I'm aware of would be this line: hf_config.num_heads = (hf_config.hidden_size * hf_config.expand) // hf_config.head_dim. expand is usually 2, so you need to consider that (2 * hidden_size) % head_dim == 0 .
Otherwise, this might also be relevant for the kernels: https://github.com/state-spaces/mamba/issues/352#issuecomment-2167093395.
And if you have a small reproducer or something, that would be ideal.

You also have to pass the correct number of heads, i.e.:
import torch
from transformers import AutoTokenizer, Mamba2Model, Mamba2Config

hidden_size = 2048
expand = 2
head_dim = 64
# this here is the key to pass too, not very nice to have it passed in the config imo but that's the current state now :(
num_heads = (hidden_size * expand) // head_dim

config = Mamba2Config(
    vocab_size=len(tokenizer.vocab),
    state_size=128,
    num_heads=num_heads,
    head_dim=head_dim,
    num_hidden_layers=4,
    hidden_size=hidden_size,
    expand=expand,
   # I'd suggest using this as it's default for all models in the mamba-ssm repo
   n_groups=1,
)

...

The scaling worked because the effective size didn't change @Jellymoon

I had a couple of questions regarding the model's default settings in the Huggingface configuration:
From the paper:

Is the no_bias_terms parameter set to True by default in the Huggingface version of Mamba2?
Does the configuration use RMSNorm instead of LayerNorm by default, or is there a specific way to set this manually?
I am trying to find the best hyperparameters that you've run experiments on the top of AutoConfig.from_pretrained('AntonV/mamba2-130m-hf').

@npkanaka There are two bias parameters:

Bias for the linear projections which is set to False by default ( https://github.com/huggingface/transformers/blob/d6ba1ac041ac0b07bc589dd82a67cfb76f75d0f9/src/transformers/models/mamba2/configuration_mamba2.py#L62 )
Bias for the convolution which is set to True by default ( https://github.com/huggingface/transformers/blob/d6ba1ac041ac0b07bc589dd82a67cfb76f75d0f9/src/transformers/models/mamba2/configuration_mamba2.py#L64 )

There is a parameter that suggests you could use layernorm instead of RMS norm in hf but it's sadly false / misleading. RMS norm will always be applied and tbh, it shouldn't matter too much.

No problem.

This seems a bit unlikely tbh. Have you ensured that mamba-ssm and causal-conv1d are installed? Maybe set config.use_cache=False during training at least. Otherwise, I'm out of quick options. Mamba2 is also more suitable for bigger architectures and longer sequences so it might be faster in comparison when using 1b params + 4k seq lengths for example.
Can you pass safe_serialization=False when saving your model? Tbh, weight sharing is the standard in Mamba2 so you should not disable it. But, tied embeddings are a mixed thing; it might be beneficial or not.

why set use_cache=False could speed up training though?

@Lynnzake At least in hf, you guarantee that inference is avoided 100% and in that case the code opts for the fused path, i.e. a kernel for conv combined with the mamba op. Anything "kernelized" is faster than the base ops one after the other (even if they are one kernel each).
Not sure what you mean with max_len but the mamba2 kernel at least is pretty unoptimized in regards to smaller seq lens. The speed only gets pretty good at lengths 2048<=

@Lynnzake At least in hf, you guarantee that inference is avoided 100% and in that case the code opts for the fused path, i.e. a kernel for conv combined with the mamba op. Anything "kernelized" is faster than the base ops one after the other (even if they are one kernel each).
Not sure what you mean with max_len but the mamba2 kernel at least is pretty unoptimized in regards to smaller seq lens. The speed only gets pretty good at lengths 2048<=

Appreciate your reply, max_len is the inputs length on my models.

Then the second part of my reply should be correct (?), i.e. max len should be 2048 or longer to be good on speed.

Then the second part of my reply should be correct (?), i.e. max len should be 2048 or longer to be good on speed.

Yes.


cargo vendor lacks compiler_builtins

Problem
Previously reported: https://github.com/alexcrichton/cargo-vendor/issues/177
cargo vendor's output /vendor/ omits compiler_builtins, which doesn't mix well with custom targets using xargo and/or cargo-xbuild.
Steps

use a custom target
use xargo/xbuild
use cargo vendor
attempt to build anything with the /vendor/ enabled

Possible Solution(s)
Include compiler_builtins.

I'm going to close this, as Cargo doesn't know anything about xargo/xbuild.  The intended path forward is the build-std feature, and supporting vendoring is tracked in https://github.com/rust-lang/wg-cargo-std-aware/issues/23.


yt-dlg User Id Issue

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

WARNING
All invalid issues will be rejected!!

Before going further

If your problem is a bug with youtube-dl or a request for new site support please report it here

Make sure you are using the latest yt-dlg version (Click the Settings icon and then About to view the current version)

Make sure you are using the latest youtube-dl version (Click the Settings icon and then Update to update to the latest youtube-dl version)

Make sure you searched the bugtracker for similar issues including closed ones

Make sure to read the FAQs file

[x] I think my problem is NOT with youtube-dl
[x] I've verified and i assure that I'm running yt-dlg 1.X.Y
[x] I assure that i am using the latest version of youtube-dl
[x] Searched bugtracker
[x] I've read the FAQs file

What is the purpose of your issue?

[x] Bug report
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Please remove any sections between (---) if they are not related to your issue

Bug report
If the problem occurs when downloading a URL please provide the full verbose output as follows:

Restart yt-dlg
Go to Options > Extra tab
Enable Debug youtube-dl
Go to Options > Advanced tab and Clear your log content
Try to download the URL
Copy the whole log content and insert it between the ``` part below

2023-02-21 23:25:02,693-DEBUG-Worker_1-ERROR: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
2023-02-21 23:25:02,694-DEBUG-Worker_1-Child process exited with non-zero code: 1
2023-02-21 23:57:54,174-DEBUG-Worker_1-ERROR: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
2023-02-21 23:57:54,175-DEBUG-Worker_1-Child process exited with non-zero code: 1

What operating system do you use ?
Windows 11
List of actions to perform to reproduce the problem:

paste url into top field
click add
click download

What is the expected behaviour ?
Should download the video in specified format
What happens instead ?
Nothing downloads. I have seen this error on the official yt-dlg git hub and they fixed it in a release but I'm not sure how to get the new version of it and connect it to this GUI. The issue comes from the the user id can now have an @ symbol. I think the issue is fixed but I updated the GUI to the lastest version and looks like the user id issue is still there.

Update to the latest yt-dlp CLI Backend. See https://github.com/yt-dlg/yt-dlg/issues/11#issuecomment-1436003375

I am on yt-dlg 1.8.4 and i clicked update, but it still shows up this error and I cannot download.

looks like it works when I changed the CLI backend from youtube-dlp.exe to yt-dlp.exe

If the issue is complete as fixed, please feel free to close it.

Ah ok, I think the issue is resolved now. Sorry about that I think I just had to play around with the settings with the new update. Thanks for the help.

Getting this issue on the most recent release now

@VaelophisNyx can share the URL , steps or logs the issue you having?

https://www.youtube.com/watch?v=ZBJA9lPi4gs URL in question
steps are just add, set to convert to mp3, download
log is just the exact same error in the original issue post

I think it may be that this video is trying to download as a WebM and convert from that

@VaelophisNyx for Post-Processing Video or Audio you need FFmpeg: https://yt-dlg.github.io/yt-dlg/Install-FFmpeg-on-Windows.html

I should state it's worked fine up to today, I have FFmpeg. It just died when I tried it today


Compacting copper

Modpack: StoneBlock 3
Mod version: functionalstorage-1.18.2-1.0.7.jar
Compacting drawer empty.
Put in a copper ingot - nugget does not display, and you can't put nuggets in
Take everything out of the drawer and put a nugget in first.
Now is will display and further ingots or nugget can be put in.

Can't replicate with just create, might be a conflicting recipe issue. When using a copper ingot it adds the nugget just fine.


synchronize the component name with skywalking project

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Pull Request Test Coverage Report for Build 9019

0 of 0   changed or added relevant lines in 0 files are covered.
58 unchanged lines in 7 files lost coverage.
Overall coverage increased (+0.2%) to 63.983%

Files with Coverage Reduction
New Missed Lines
%

sharding-jdbc/sharding-jdbc-core/src/main/java/org/apache/shardingsphere/shardingjdbc/jdbc/core/statement/EncryptPreparedStatement.java
1
74.14%

sharding-core/sharding-core-execute/src/main/java/org/apache/shardingsphere/core/execute/sql/execute/result/QueryResultMetaData.java
2
52.17%

sharding-orchestration/sharding-orchestration-core/src/main/java/org/apache/shardingsphere/orchestration/internal/util/IpUtils.java
3
76.0%

sharding-core/sharding-core-optimize/src/main/java/org/apache/shardingsphere/core/optimize/encrypt/segment/EncryptInsertColumns.java
6
51.52%

sharding-core/sharding-core-optimize/src/main/java/org/apache/shardingsphere/core/optimize/sharding/segment/insert/ShardingInsertColumns.java
9
52.38%

sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/encrypt/EncryptTable.java
11
0.0%

sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/rule/EncryptRule.java
26
6.1%

Totals

Change from base Build 186:
0.2%

Covered Lines:
8989

Relevant Lines:
14049

💛  - Coveralls


New feature: read persona from files

In main I've added an option to read persona information from a file.
Hopefully, this should support a few of the common formats,
though it will only read in the name and description, not
other fields.
This is exposed as a new setting in config.yml.
  # Path to a file containing a persona.
  # This can be just a single string, a json file in
  # the common "tavern" formats, or a yaml file in the
  # Oobabooga format.
  # With a single string, the persona will be set to that string.
  # Otherwise, the ai_name and persona will be overwritten with
  # the values in the file.  Also, the wakewords will be extended to
  # include the character's own name.
  #   default:
  persona_file: ./tests/test_data/test-char-1.yaml

If you're interested, please try this out and let me know if it works for you.  Also let me know if there's a burning desire to support character cards (images).  It would be possible to add but would add dependencies on external libraries, so I would rather not pull those in unless there's demand.
More details in commit.
https://github.com/chrisrude/oobabot/commit/6f95d7c9259831b9cc072058e9c7cd04d0c2d2f6

@Kozonak FYI.  Let me know if this works for you, or if you're using a different format.
I'm a little unsure about which fields exactly need to be read, so it's possible I need to try and incorporate more data from the card.  The bot will show you what it's using when it starts up, so let me know if you think that will work.

This sounds great, I will update to latest tomorrow and take it for a spin. I was happy with just txt but other formats might help keep things tidy.

pip install ruamel.yaml
now required

Yup!  In the dev environment after an update you can just run poetry install, and it will pull in all the needed dependencies.  That way you don't have to figure out what it is missing, and be assured that you're running against the same library versions that the python egg will install.

an odd behavior i've noticed with this is that the bot does only register the persona of the character file while ignoring other possibly relevant sections (e.g. the example dialogue)
though that may be intended

If you load the same persona png/json in ooba ui, it still only loads the persona itself (as far as i can tell). I think certain params like example history are only parsed in certain situations or maybe only in tavern.

This is intended, and probably not great.
The reason is that I haven't figured out yet how to prompt the AI with the sample dialog.  I know that other apps do use it, but I haven't dove into that yet, as I've been prioritizing the voice feature.
It's totally possible to add that in as well, if you think it would help.  It would make it easier if I knew what sort of prompts are effective for sending to oobabooga -- i.e. how is the example dialogue included in the prompt?

The reason is that I haven't figured out yet how to prompt the AI with the sample dialog.

I've been experimenting with that, and I've added a line to the "persona" part of the prompt that gets sent.
It's something along the lines of
Rose is a cat, blah blah blah blah.
Below is a past conversation between {{char}} and a user:
USER: example message
{{char}}: example response


Plotly configuration in Jupyter notebook cell metadata for subplots

Steps to reproduce
In the jupyter notebook, I have a chart widget with horizontal subplots. I tried to customize the plotly tickangle by modifying the cell's metadata:

However, only the first subplot inherits the plotly setting. Is this way of configuring the charts still legit going forward?
Environment

atoti: 0.5.2
Python: 3.9.1
Operating System: win32

Tracking this internally in this ticket.

One workaround would be to add the changes in xaxis2, xaxis3 etc as well

the issue is still here in 0.6.4