Fatal error when creating a chart from a saved query via API call

SavedQuery class seems to not have a name property, unlike other data sources.
Create chart workflow seems to expect that property on all data source types.
How to reproduce the bug
Assuming a database connection is available. I am using a BigQuery connection, but it's irrelevant to the issue.

Create a saved query request JSON (make sure to use the right database ID)

$ cat saved_query.json
{
    "db_id": 3,
    "label": "saved_query",
    "schema": "public",
    "sql": "SELECT 1"
}

Create the saved query

$ curl http://localhost:8088/api/v1/saved_query/ \
    -H "Content-Type: application/json" \
    -H "Cookie: session=..." \
    -H "X-CSRFToken: ..." \
<EMAIL_ADDRESS>{"id":1,"result":{"db_id":3,"description":null,"label":"saved_query","schema":"public","sql":"SELECT 1","template_parameters":null}}

Create a chart payload JSON (make sure to use the right datasource ID)

$ cat chart.json
{
    "datasource_id": 1,
    "datasource_type": "saved_query",
    "slice_name": "some_name",
    "viz_type": "table"
}

Create the chart

$ curl http://localhost:8088/api/v1/chart/ \
    -H "Content-Type: application/json" \
    -H "Cookie: session=..." \
    -H "X-CSRFToken: ..." \
<EMAIL_ADDRESS>{"message":"Fatal error"}

Expected results
Curl command to complete successfully with the creation of the chart from the saved query.
Actual results
Curl result is {"message":"Fatal error"}
superset app stacktrace:
superset_app          | 2023-02-24 12:15:23,974:ERROR:flask_appbuilder.api:'SavedQuery' object has no attribute 'name'
superset_app          | Traceback (most recent call last):
superset_app          |   File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/__init__.py", line 110, in wraps
superset_app          |     return f(self, *args, **kwargs)
superset_app          |   File "/app/superset/views/base_api.py", line 124, in wraps
superset_app          |     raise ex
superset_app          |   File "/app/superset/views/base_api.py", line 115, in wraps
superset_app          |     duration, response = time_function(f, self, *args, **kwargs)
superset_app          |   File "/app/superset/utils/core.py", line 1586, in time_function
superset_app          |     response = func(*args, **kwargs)
superset_app          |   File "/app/superset/utils/log.py", line 266, in wrapper
superset_app          |     value = f(*args, **kwargs)
superset_app          |   File "/app/superset/views/base_api.py", line 87, in wraps
superset_app          |     return f(self, *args, **kwargs)
superset_app          |   File "/app/superset/charts/api.py", line 316, in post
superset_app          |     new_model = CreateChartCommand(item).run()
superset_app          |   File "/app/superset/charts/commands/create.py", line 44, in run
superset_app          |     self.validate()
superset_app          |   File "/app/superset/charts/commands/create.py", line 64, in validate
superset_app          |     self._properties["datasource_name"] = datasource.name
superset_app          | AttributeError: 'SavedQuery' object has no attribute 'name'
superset_app          | <IP_ADDRESS> - - [24/Feb/2023:12:15:23 +0000] "POST /api/v1/chart/ HTTP/1.1" 500 26 "-" "curl/7.81.0"

Screenshots
none
Environment
(please complete the following information):

browser type and version: -
superset version: master, 2.1.0rc1 (1.5.3 does not support saved query as data source for charts)
python version: Python 3.10.6
node.js version: -
any feature flags active: -

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[X] I have reproduced the issue with at least the latest released version of superset.
[X] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
Ubuntu 22.04, ran locally using docker compose.
I was lazy and used the cookie authentication from the browser, but it shouldn't matter wrt the problem.

In 2.0.1 I get {"message":{"datasource_id":["Datasource does not exist"]}} even though the saved query exists and has the ID I pass to the create chart call.

Hello popescu,
Could you use datasource_type table instead saved_query then it should work properly while creating the chart.

Agreed, seems to be working as expected for a long while indeed. This is likely fixed by now, and is pretty out of date if not. If people are still encountering this in current versions (3.x) please open a new Issue or a PR to address the problem.


Fix Birthday change during Schedule operation

Coverage decreased (-0.04%) to 79.837% when pulling ada1db9ae11e322b622b2663281f4fa84c284fd9 on eldriclim:fix-schedule-bday into 1a7a13f2ab873759e4f06da42239ef16b7cd0562 on CS2103AUG2017-W09-B3:master.


Cesium3DTileset missing proxy option

All of our data providers (Imagery, Terrain, DataSources) provide the ability to specify a proxy option to work around CORS issues, but it looks like Cesium3DTileset is missing this capability.  I'm not sure how many different places create URLs during the loading process, but this should be easy enough to add to anyone that knows the code well enough already.

This is OBE and was fixed as part of the Resource refactor.


Fix truncate detection

The laset_line of the XML file returned is a byte object and previously it was compared to a string so the function always returns True and the XML is considered as truncated.
In practice, sometimes it is b'</modeling>\n' some times it is `b'', both cases are supported now. Not sure why! might be related to whether a file object or file path was used?
A test is added to make sure the truncation detection works as expected.

Awesome, thanks a lot for the quick turn around.

Awesome, thanks a lot for the quick turn around.


Add a hash to media files even in the devmode.

This will work correctly when there are duplicate filenames in different paths.
Fixes: #445

I tried the fix by ejecting the app and it worked pretty well. Do you know how can I write tests for this?

Thanks.


Recently listed coins fails to trade

[2021-06-07 05:59:01] pausebotmod: Market not looking too good, bot paused from buying 10/7 Waiting 1 minutes for next market checkup
[2021-06-07 06:00:02] pausebotmod: Market not looking too good, bot paused from buying 9/7 Waiting 1 minutes for next market checkup
[2021-06-07 06:01:02] pausebotmod: Market looks ok, bot is running 6/7 Waiting 1 minutes for next market checkup
[2021-06-07 06:01:12] Resuming buying due to change in market conditions, total sleep time: 0:03:05
[2021-06-07 06:01:12] Working...Session profit:15.62% Est:$3.12
Traceback (most recent call last):
File "Binance Detect Moonings.py", line 592, in
orders, last_price, volume = buy()
File "Binance Detect Moonings.py", line 292, in buy
volume, last_price = convert_volume()
File "Binance Detect Moonings.py", line 253, in convert_volume
volatile_coins, number_of_coins, last_price = wait_for_price()
File "Binance Detect Moonings.py", line 152, in wait_for_price
min_price = min(historical_prices, key = lambda x: float("inf") if x is None else float(x[coin]['price']))
File "Binance Detect Moonings.py", line 152, in
min_price = min(historical_prices, key = lambda x: float("inf") if x is None else float(x[coin]['price']))
KeyError: 'ATAUSDT'

It is a new coin that binance added, just edit tickers.txt and add it. Or blacklist it.

Why to blacklist rockets?! I want to catch it and fly to the moon, accordingly with name of the app "Binance Detect Moonings.py".
I use "CUSTOM_LIST: False" in config, but 'ATAUSDT' fiers ~4500% with few hours without me. ))

Why to blacklist rockets?! I want to catch them and fly to the moon, accordingly with name of the app "Binance Detect Moonings.py".
I use "CUSTOM_LIST: False" in config, but 'ATAUSDT' fiers ~4500% with few hours without me. ))

This isn't the place for this, but just so you know it shows 4500% because it launched today at 0 value, got a micro peak of 4.5$ (just few sells) then crashed to 1.6$. So, no, it didn't moon, that's what happen when new coins enter the exchanges and if you had put money on it you would have surely lose it.
This app need an option to ban new coins that enter the exchange for a certain of period, most bots have it so it prevents people from losing money on day 1.

@koshrf is right.

when Market looks ok, bot is running...
this error comes out:
Traceback (most recent call last):
File "c:/Users/pc/Desktop/TradingTry/Binance Detect Moonings.py", line 597, in
orders, last_price, volume = buy()
File "c:/Users/pc/Desktop/TradingTry/Binance Detect Moonings.py", line 294, in buy
volume, last_price = convert_volume()
File "c:/Users/pc/Desktop/TradingTry/Binance Detect Moonings.py", line 255, in convert_volume
volatile_coins, number_of_coins, last_price = wait_for_price()
File "c:/Users/pc/Desktop/TradingTry/Binance Detect Moonings.py", line 139, in wait_for_price
if historical_prices[hsp_head]['BNB' + PAIR_WITH]['time'] > datetime.now() - timedelta(minutes=float(TIME_DIFFERENCE / RECHECK_INTERVAL)):
KeyError: 'BNBUSDT'
what's the solution PLS ?


Update Code - Proof_Of_Contribution.md

Added hours for @jabo38 (Jeff McDonald) smart contract testing and tutorial rewrite.

Conflicts preventing merging. I'll merge it manually.


Add middleware

Please fix code coverage on sonar as exposed via secondary channel.

Please fix code coverage on sonar as exposed via secondary channel.

fixed thanks


add documentation to aegis go sdk

this can be part of aegis.z2h.dev too.
if these kinds of contents will increase, it might be time to switch to jeykll or some other static content generator too.

added some minimal docs; more will come, but the current one is good enough.


Trying to send Function Key (F6)

Using Appium and WinAppdriver to test drive our Microsoft Word Addin (using Addin-Express).
So far I have been able to do most things however hitting a bit of a brick wall with the .SendKeys() functionality.
I want to send a Function key to toggle a particular mode on the addin (keyboard driven mode) This is what I am currently doing:
var element = session.FindElementByName("Name of Document - WORD");
//setup
element.SendKeys(Keys.F6);

The result is that it switches the F6 mode on for a split-second then switches it off - (I can tell as a little keyboard will appear when this mode is active, and disappears when it is not).
Any ideas why it this is occurring?

What you actually want to check out, is generically telling the the server to not send anything to any element, but press the F6 key like so:
session.Keyboard.PressKey(OpenQA.Selenium.Keys.F6);

Hi @Rangyia thanks for the response. Yup that what I assumed as much.
So I dug a little deeper into what was actually being sent. So according to this Windows help documentation on the WindowsForm.Keys enum (https://docs.microsoft.com/en-us/dotnet/api/system.windows.forms.keys) the value that should be sent from pressing the F6 key is 117.
I put together a quick WinForm app to capture OnKeyDown event. Using the keyboard manual approach I received 117.
However driving the app from Appium using the code you suggested I receive 255 117 255.
With some further investigation it looks like the app that I am testing will disable the mode if it receives any other keys. So hence the reason why I am seeing the particular mode come on, then immediately switch off.
So the Selenium.Keys.F6 (and I imagine many more) is not sending the exact same KeyValue as the Window Forms Keys spec. Not sure if this is a problem for anyone else but it caused me issues.
As a work around I am now using SendKeys.SendWait("{F6}"); from the System.Windows.Forms namespace (https://docs.microsoft.com/en-us/dotnet/api/system.windows.forms.sendkeys). This works a treat and is significantly quicker.


Clarify whether Type Of Content in Data File Descriptions is correct

It is possible that this should be "Description Type", which would be consistent with Datacite.

That was explicitly documented in the text description for those fields, at least once upon a time.

After chatting offline with @graybeal.  This field should be the type for the description.
TODO: Add description to docs about how to fill out this field programmatically.


Bug: the switch doesn't compile

Bug description
Pretty self-explanatory, doesn't compile because of a slightly wrong OCaml version.
To reproduce
Steps to reproduce the behavior:

git clone the repo.
Attempt to start the build process with opam switch create . --deps-only --locked.
Wait for a bit.

Expected behavior
For the switch to compile.
Optional screenshots
I'd prefer to give you text which is probably easier to work with and will do so below.
Current configuration

Ubuntu 20.04.3 LTS ("focal")
Browser name is currently unnecessary.
Couldn't compile but CHANGELOG.md says version 0.13.1.

Additional context
Here is probably the "meat":
[ERROR] The compilation of ocaml.4.12.1 failed at "ocaml /home/readyready15728/src/learn-ocaml/_opam/share/ocaml-config/gen_ocaml_config.ml
        4.12.1 ocaml".

#=== ERROR while compiling ocaml.4.12.1 =======================================#
# context     2.1.0 | linux/x86_64 |  | https://opam.ocaml.org#1fc79772
# path        ~/src/learn-ocaml/_opam/.opam-switch/build/ocaml.4.12.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build ocaml /home/readyready15728/src/learn-ocaml/_opam/share/ocaml-config/gen_ocaml_config.ml 4.12.1 ocaml
# exit-code   1
# env-file    ~/.opam/log/ocaml-38639-e3db12.env
# output-file ~/.opam/log/ocaml-38639-e3db12.out
### output ###
# OCaml version mismatch: 4.12.2, expected 4.12.1

There's a little afterwards which I'll include as well:
<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build ocaml 4.12.1
└─
┌─ The following changes have been performed (the rest was aborted)
│ ∗ install base-bigarray         base
│ ∗ install base-threads          base
│ ∗ install base-unix             base
│ ∗ install conf-git              1.0
│ ∗ install conf-libssl           3
│ ∗ install conf-pkg-config       2
│ ∗ install conf-which            1
│ ∗ install ocaml-config          2
│ ∗ install ocaml-options-vanilla 1
│ ∗ install ocaml-variants        4.12.1+trunk
└─
# Run eval (opam env) to update the current shell environment
Switch initialisation failed: clean up? ('n' will leave the switch partially installed) [Y/n] Y

I noticed there are logs too although the .out file doesn't tell me anything that you haven't already seen re: the version mismatch. I'll give you the .env stuff though:
CAML_LD_LIBRARY_PATH=/home/readyready15728/src/learn-ocaml/_opam/lib/stublibs
CDPATH=
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
HOME=/home/readyready15728
LANG=C.UTF-8
LC_ALL=en_US.UTF-8
LOGNAME=readyready15728
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
MAKEFLAGS=
MAKELEVEL=
MANPATH=:/home/readyready15728/src/learn-ocaml/_opam/man
MOTD_SHOWN=pam
OCAML_TOPLEVEL_PATH=
OMF_CONFIG=/home/readyready15728/.config/omf
OMF_PATH=/home/readyready15728/.local/share/omf
OPAMCLI=2.0
OPAMROOT=/home/readyready15728/.opam
OPAMSWITCH=/home/readyready15728/src/learn-ocaml
OPAM_PACKAGE_NAME=ocaml
OPAM_PACKAGE_VERSION=4.12.1
OPAM_SWITCH_PREFIX=/home/readyready15728/src/learn-ocaml/_opam
PATH=/home/readyready15728/src/learn-ocaml/_opam/bin:/opt/rakudo-pkg/var/zef/bin:/opt/rakudo-pkg/bin:/opt/rakudo-pkg/share/perl6/bin:/home/readyready15728/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
PWD=/home/readyready15728/src/learn-ocaml
RAKULIB=/opt/rakudo-pkg/var/zef/lib
SHELL=/usr/bin/fish
SHLVL=1
SSH_CLIENT=<IP_ADDRESS> 44598 22
SSH_CONNECTION=<IP_ADDRESS> 44598 <IP_ADDRESS> 22
SSH_TTY=/dev/pts/0
TERM=xterm-256color
USER=readyready15728
XDG_RUNTIME_DIR=/run/user/1000
XDG_SESSION_CLASS=user
XDG_SESSION_ID=46
XDG_SESSION_TYPE=tty

eval (opam env) sans dollar sign is correct because I'm using fish FWIW. I wish I could offer more detail including steps towards the fix but I'm obviously just a learner. I attempted monkeying around with some of the opam configuration files attempting to force the slightly newer OCaml version rather than what appears to specifying "any 4.12 version" but that ended up going nowhere. Sorry about that.

ADDENDUM: I tried forcing 4.12.1 too in the two .opam files then tossing out the .locked files. No dice.

Hi @readyready15728, sorry for late reply, and thanks a lot for your thourough bug report!
My understanding is that:

learn-ocaml should be compatible with any point release of 4.12.*

indeed, see this line:
https://github.com/ocaml-sf/learn-ocaml/blob/6e3fa0f8731a383f70006abf127bbf3c8a6218ee/learn-ocaml.opam#L43

but in learn-ocaml.opam.locked, by definition of the locked feature, can only enforce one version of each specified dependency, so, only one point release.

cf. the line you had spotted:
https://github.com/ocaml-sf/learn-ocaml/blob/6e3fa0f8731a383f70006abf127bbf3c8a6218ee/learn-ocaml-client.opam.locked#L70
As a result:

I believe you found a solution, but I guess the optimal solution could have just been to replace
opam switch create . --deps-only --locked with opam switch create . --deps-only (without --locked)
following the feedback I had received from @AltGr in https://discuss.ocaml.org/t/are-there-best-practices-w-r-t-opam-install-locked/5741, it is advised to put --locked if the switch does not exist at all (and thereby, no conflicting ocaml), but to remove --locked if the switch already contains something…
so I'll label your issue as a documentation task;
I can't give an ETA to this but anyway, feel free to react anytime in this issue if you have other questions!
and BTW, thanks for the reminder that we should bump the point release of ocaml 4.12 :)

As of now, I have no idea about any of that my good dude. I'm just someone working through the CS3110 course from Cornell. Just glad I unearthed the error and made a little progress towards solving it.

This is an opam error, not a learn-ocaml issue. The ocaml package is only a virtual package depending on one of ocaml-base-compiler, ocaml-system, etc. ; here it seems you have ocaml-variants.4.12.1+trunk getting installed which is wrong, you want ocaml-base-compiler. Please check your opam version, run opam update to make sure package definitions are up-to-date, try again and report to ocaml/opam-repository if the issue persists.


MapReduceKeepHighDim::reduction dimension do not match

Hi,
I'm adding some modifications to the structure of a pretrained network; I added a skip connection and a 1x1 conv after the concat to make the final feature dimension unchanged. Here are the visualization of the modified part of the network:
original net

modified:

(I changed visualization.py a bit so it displays batch size, which is the first dimension)
since "shared_conv_reduce" is a new symbol and does not exist in the pretrained model, I initialized its weight using normal distribution with shape (1024, 1536, 1, 1); but during forwarding (I suppose), mxnet reports this error:
dmlc-core/include/dmlc/logging.h:235: [23:16:07] src/engine/./threaded_engine.h:306: [23:16:07] mshadow/mshadow/./tensor_gpu-inl.h:146: Check failed: (eshape[dimkeep]) == (dshape[0]) MapReduceKeepHighDim::reduction dimension do not match
and I don't know why?

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


2021-02-13 19:33:57 - ERROR: Unhandled rejection - Error: getaddrinfo ENOTFOUND core-mosquitto:1883

Hi, I am getting the following error which I am unable to fix.
2021-02-13 19:33:57 - ERROR: Unhandled rejection - Error: getaddrinfo ENOTFOUND core-mosquitto:1883

Change your config from host: core-mosquitto:1883 to simply host: core-mosquitto and it should fix the issue.

That has fixed it. Thank you and thanks for replying so quickly. Really appreciate your work on this.

I'm having the same problem but deleting the port number did not fix the problem.


Setting up PullApprove

Renamed .pullapprove.yaml -> .pullapprove.yml
I am so sorry..."baby steps"

:+1: from me

:+1:

:+1:

Approved


trailing comma and --mute

Used with mocha.
Trailing comma
to avoid this:
    ✓ should raise error if file does not exist[2016-05-11T09:03:34.958Z] INFO (app/: Send the file

and instead have this:
    ✓ should raise error if file does not exist
[2016-05-11T09:03:34.958Z] INFO (app/: Send the file

CLI option --mute
to have test results without logs

Can you please add the option to the README and the CLI help?

Any updates on this one?

Only mute.

Can you please make an example of how you would use this? I'm a bit lost, why you need to mute the pretty-printer instead of not having it in place?

There are some cases (i.e. using mocha reporters) where I don't want logs to be shown (i.e. because they are messing up the final mocha reporter output)...
But if you don't see the point I'll close the pr and keep a personal fork : )
My specific use case:
'use strict';

var expect = require('chai').expect;

function foo() {
  var pino = require('./')()
  pino.info('foo')
  return 'bar';
}

describe('A foo function', () => {
  it('should return bar', (done) => {
    expect(foo()).to.equals('bar')
    done()
  })
})

mocha test_mocha.js --reporter doc | ./pretty.js

A foo function

[2016-05-18T12:38:58.005Z] INFO (3613 on vt-macchip.valueteam.com): foo

should return bar

expect(foo()).to.equals('bar')done();

mocha test_mocha.js --reporter doc | ./pretty.js --mute

A foo function

should return bar

expect(foo()).to.equals('bar')done();

p.s. with bunyan I'm doing it with a workaround:
mocha test_mocha.js --reporter doc | bunyan -c '"mute" === "workaround"'

I'm 👍 on this. @davidmarkclements?

@mcollina  @nkint ... yeeeeah I think so
Although maybe it should be called --strip - because we're relying on parse errors to determine whether we pass the line through the stream, or strip the line out. I think that makes the use case clearer

I think "mute" makes it clearer, but it might be an Italian thing. It's not stripping anything, it's removing all logs from an output, and leaving only everything that is not a recognized log.

strip does imply remove - but I don't feel strongly against mute so either is fine
Also.. should we explain that mute skips logs in the stream by skipping parseable JSON because there is a difference. If you use this one stream that has other json, that will be skipped also.

@nkint can you please add a couple of checks so we don't mute valid JSON that are not pino logs?

Closing for now, feel free to reopen.


Tuya Alarm PG103 PG107 - Adding Alarm Zone of triggered alarm

When an Alarm is triggered for device gautone_pg103_alarm.yaml
From the log I also see DPS 26 received which gives details about which Zone of the alarm triggered.
This is a normal poll from the device.
[custom_components.tuya_local.device] myalarm received {"1": "arm", "2": 8, "3": 5, "4": false, "6": false, "10": true, "12": false, "13": false, "25": false, "27": true, "28": 4, "32": "alarm", "36": "remote_controller", "101": "normal", "full_poll": true}
When the alarm triggers - DPS 26 is received giving details of the zone in base64.
[custom_components.tuya_local.device] myalarm received {"26": "AFMAeQBzAHQAZQBtACAAQQBsAGEAcgBtAAoAWgBvAG4AZQA6ADAAMAAx", "full_poll": false}
This decodes to -
System Alarm
Zone:001
My experience with this is limited, but I tried to add a secondary entity.
I do get the Zone but it is still encoded in base64, any way to decode and display?
Can this sensor get reset after a disarm? as it would stay on whatever was received last.
  - entity: sensor
    class: none
    name: Alarm Zone
    category: diagnostic
    dps:
      - id: 26
        type: base64
        name: sensor
        endian: little
        optional: true

I think it makes sense to add this as an attribute on the Alarm sensor rather than as a separate sensor. It will then be available for further processing, without cluttering the UI with something that is not human readable.
base64 is used to encode binary data, and without a spec of the fields contained within, it is not guaranteed that it can be consistently decoded.  I also do not intend to complicate this integration by adding general text re-coding features, as this string appears to be encoded as UTF-16BE rather than the usual UTF-8, which is probably why the developers chose to use base64 to encode it.

Regarding the new Diagnostics
BatteryIsLow
powerDisconnect
I think it would be best to leave these out from the testing I encountered.
There is a switch on the back of the unit to enable/disable the alarm unit battery.
When enabled, and mains is cut off. it sets DPS "101": "powerDisconnect"
But nothing will reset this status so the attribute is always set.
I've tried everything, even deleting any alarm logs on the unit. The only way I could reset this flag was to turn the physical Battery switch to off on the unit.

The binary_sensor Alarm Zone message is working well.
I was able to use it as below. There is no gui setting to use it so I had to work this bit out. I'll post it here for reference as it may be helpful to others.
If you create a new Automation - Use the trigger - Alarm started detecting sound
I then set the action to either Notification / Email
In the message section - Edit as Yaml
You just have to trigger the zones and map the base64 string it outputs to your translated zone.
  message: >-
    {% set alarm_msg = state_attr('binary_sensor.alarm_pg107_alarm', 'message')
    %} {% if alarm_msg ==
    "AFMAeQBzAHQAZQBtACAAQQBsAGEAcgBtAAoAWgBvAG4AZQA6ADAAMAAx" %}
      Kitchen
    {% elif alarm_msg ==
    "AFMAeQBzAHQAZQBtACAAQQBsAGEAcgBtAAoAWgBvAG4AZQA6ADAAMAAy" %}
      Zone2
    {% elif alarm_msg ==
    "AFMAeQBzAHQAZQBtACAAQQBsAGEAcgBtAAoAWgBvAG4AZQA6ADAAMAAz" %}
      Zone3
    {% else %}
      Unknown Zone ({{ alarm_msg }})
    {% endif %}


preference doesn't save

code-server version: tot
OS Version: ubuntu 18.04 in docker and on host

Description
I tried to config the shortcut, and it doesn't save. if I refresh the browser, the old default preference comes back.
Steps to Reproduce

Does it work if you do something like docker run -t -p <IP_ADDRESS>:8443:8443 -v "$HOME/path/to/project:/root/project" -v "/$HOME/.coder:/root/.local"  codercom/code-server --allow-http --no-auth ? The preferences should be saved in /root/.local/ which will be stored in your /$HOME/.coder, and preserved between sessions.

Can't reproduce this issue. I don't think this persists anymore.

This is still happening to me. the specific config I need to change is keyboard binding. The result of the change should be in keybindings.json, which I verified is there with the right content.
but if I close the browser tab and reconnect, the keybindings won't change at all. it doesn't seem to load the json file.

So this is only for keybindings I presume? Other preferences saves fine

Same here.
Custom keybinding works properly when you first customize them.
When reload, default keybinding overwrites it.
If you open up custom keybindings again and just save it, custom keybinding works again.
Other preferences save just fine.

same issue here from me https://github.com/codercom/code-server/issues/150

This might be a confirmed bug, I can confirm this on CoreOS

Duplicate of https://github.com/codercom/code-server/issues/321


TIKTOK api uncompress video

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm requesting a feature unrelated to a specific site
[X] I've looked through the README
[X] I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
Anyone know how to obtain the same api as this website?
His api are able to download video uncompress from tiktok and they are not willing to share.
https://www.tikwm.com/en/
Provide verbose output that clearly demonstrates the problem

[ ] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
none

Duplicate of #4138
There used to be a known way of getting the source quality but it no longer works. As you noted, the people who know the current working method don't seem like they are willing to share


PCA SPMD python interfaces

Python interfaces for PCA SPMD

/intelci: run

/intelci: run

/intelci: run

/intelci: run

/intelci: run

Hi just add GPU device and librariry requirements for your examples here https://github.com/intel/scikit-learn-intelex/blob/master/tests/run_examples.py#L127:L156:
...
req_device['pca_spmd.py'] = ["gpu"]

...

req_library['pca_spmd.py''] = ['dpctl', 'mpi4py']

And also rename your examples and make spmd as suffix in the file names.
After that please run /intelci: run

/intelci: run

/intelci: run


Add a 'twoline' option for IP-only display

I just purchased a Stream Deck and loved your plugin idea, but the display was just too small for my eyes to see easily.
So I forked your code locally added a "two-line" option for the IP address for my own use but in the spirit of open-source I figured I would offer it back so you can add to your plugin, if you so choose.
Enjoy!

Thank you, will ask Elgato to publish it asap


Corrected subscriber_count to subscribers_count; added integration tests of user public/private status

I added integration tests of the user's ability to switch between public and private status and the user's visibility on a repo page.  (At the moment, private subscribers are not shown on the repo pages but public subscribers are.)
Many parts of the code call the subscribers_count method for the repo object, but it was called "subscriber_count" in the repo model and in a few other places.  I have replaced all instances of "subscriber_count" with "subscribers_count".  The tests of the visibility of a repo's subscribers were failing before I made this change.

I resolved the merge conflict.


Document how to test examples in user guide, add some more coverage

Which issue does this PR close?
Part of https://github.com/apache/datafusion/issues/11172 and https://github.com/apache/datafusion/issues/1813
Rationale for this change
I am trying to make the examples easier to find / navigate. Right now there are many files in datafusion-examples but they are somewhat overwhelming. There are also some examples in the user guide, but many are not tested
I tried to consolidate the examples in https://github.com/apache/datafusion/pull/11173 but @lewiszlw (rightly) pointed out that one giant .rs example file might be even harder to find / navigate
I think the ideal outcome from a users perspective is for the examples to be inline in the user guide so they are both discoverable and the context explained. However, it is critical that the examples be testable
What changes are included in this PR?

Document how to test examples in the user guide
Add test coverage for expressions.rs and fix the example there.

Are these changes tested?
Yes,
Are there any user-facing changes?
Documentation on tests.

As a newcomer, I can say that more guidance on the examples is good! I think it might also be helpful:

To perhaps provide more guidance on when on when an example should be included in the documentation?
To include links to the appropriate examples in the example directory when appropriate? Especially for more advanced use cases?

I'll also note that the for newcomers the hardest part of examples - and tests - is figuring out how to setup the appropriate pre-conditions. Maybe we could create some "best practices" around how to approach this. Or perhaps its just a matter of reading enough existing examples and tests!

Thanks @efredine, this is great feedback

To perhaps provide more guidance on when on when an example should be included in the documentation?

Yes, I agree this would be helpful. I'll think if I can come up with some summary that matches current reality (rather than what I would ideally liek)

To include links to the appropriate examples in the example directory when appropriate? Especially for more advanced use cases?

Yes, I agree -- I think this is the approach that @tshauck  took with the user guide (e.g. https://datafusion.apache.org/library-user-guide/working-with-exprs.html has links to various examples)

I'll also note that the for newcomers the hardest part of examples - and tests - is figuring out how to setup the appropriate pre-conditions. Maybe we could create some "best practices" around how to approach this. Or perhaps its just a matter of reading enough existing examples and tests!

When you say "preconditions" what do you mean? Is it the correct data directories / data files checked out?

By pre-conditions I meant that in order to test or illustrate something like an option when reading a file I need to first create a file and write to it with data that is meaningful for the thing being tested.

By pre-conditions I meant that in order to test or illustrate something like an option when reading a file I need to first create a file and write to it with data that is meaningful for the thing being tested.

🤔  Maybe we could add some examples of "how to create test data" (specifically we can now use the COPY command thanks to @devinjdangelo  so it is significantly easier)

Thank you for your review (as always @comphead ) 🙏


npm run docs error

cannot generate docs. this is due to an issue in tsconfig. if you remove the comment the docs generate. Im opening an issue for a paper trail. will add the fix to my pull request

now added to pull request #7

Merged branch.


startTimeStamp in Waters RAW file conversion

Hi,
When using MSConvert on Waters .raw files to mzML, the startTimeStamp field is populated using the HEADER.TXT acquisition date and time fields.
However it appears that if the acquired time contains '08' or '09' as the hour, minute or second; the value is replaced by '00' in the startTimeStamp. For example:
$$ Acquired Date: 05-Sep-2017
$$ Acquired Time: 08:09:08
Results in:
startTimeStamp="2017-09-05T00:00:00Z" instead of startTimeStamp="2017-09-05T08:09:08Z"
We experienced this issue with multiple datasets and it can be simulated by simply changing the acquisition time in HEADER.TXT in any raw data. (tested in today's build ProteoWizard 3.0.18182.5406cdbf0)
Thanks and apologies for cross-posting this in the support-mailing list
Best regards

I can confirm that this is an issue. Highly problematic when I use the startTimeStamp to do intensity drift correction.

Fixed with Boost upgrade in https://github.com/ProteoWizard/pwiz/pull/82 - so official downloads will have this fixed.

Is this supposed to have been fixed in the compiled version?
I am still seeing this issue unfortunately with:
ProteoWizard release: 3.0.19014 (f9d5b8a3b)
Build date: Jan 14 2019 20:20:10

If not, will we see the fix available soon?

Interesting. I'm not sure why I blamed this on Boost before but digging into it this time revealed that it's because of my overriding lexical_cast with strtol calls in optimized_lexical_cast.hpp. The time parser was looking at 09 and treating it like an octal (base 8) number. I'm updating the code to always treat such conversions as base 10, which appears to be what basic lexical_cast (and C++ istream) does.
Fixed in #439

Thanks!


PHP - Adding an image as an overlay

Resolves #79
sample url generated https://res.cloudinary.com/dozm52czt/image/upload/l_fetch:aHR0cHM6Ly9yZXMuY2xvdWRpbmFyeS5jb20vZGVtby9pbWFnZS91cGxvYWQvbG9nb3MvY2xvdWRpbmFyeV9mdWxsX2xvZ29fd2hpdGVfc21hbGwucG5n/fl_layer_apply/v1/samples/smile.jpg?_a=BAAAROBs
Also rolled my keys

looks great! thanks for adding the extra script. added a few tweaks so they were consistent


Add API management for issue/pull and comment attachments

Close #14601
Fix #3690
Revive of #14601.
Updated to current code, cleanup and added more read/write checks.

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@bf20786). Click here to learn what that means.
The diff coverage is 70.93%.

@@           Coverage Diff           @@
##             main   #21783   +/-   ##
=======================================
  Coverage        ?   48.35%
=======================================
  Files           ?     1034
  Lines           ?   141310
  Branches        ?        0
=======================================
  Hits            ?    68325
  Misses          ?    64813
  Partials        ?     8172

Impacted Files
Coverage Δ

modules/notification/webhook/webhook.go
58.28% <0.00%> (ø)

modules/structs/issue.go
0.00% <ø> (ø)

routers/api/v1/repo/issue_comment.go
58.23% <0.00%> (ø)

routers/api/v1/repo/release_attachment.go
0.00% <0.00%> (ø)

routers/web/repo/issue.go
36.37% <0.00%> (ø)

services/attachment/attachment.go
60.86% <66.66%> (ø)

routers/api/v1/repo/issue_comment_attachment.go
68.19% <68.19%> (ø)

routers/api/v1/repo/issue_attachment.go
72.43% <72.43%> (ø)

routers/api/v1/api.go
76.76% <82.60%> (ø)

models/issues/comment.go
49.72% <100.00%> (ø)

... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@KN4CK3R thanks for jumping onto this feature!

make L-G-T-M work


Add OnFrameAvailable callback

Add OnFrameAvailable callback to process each animation Frame.
There is a sample on how to blur the GIF animation

:+1: good stuff. Bumping version and pushing to central in a bit


Rvoievidko/tdi 40897 jdbc params field for snowflake

What is the current behavior? (You can also link to an open issue here)
https://jira.talendforge.org/browse/TDI-40897
What is the new behavior?
Additional params field was added
Please check if the PR fulfills these requirements

[x] The commit message follows Talend standard
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features) ?
[ ] The code coverage on new code >75%
[ ] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[x] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Pull request test result: success :star:
Sonar results

Pull request test result: success :star:
Sonar results

Pull request test result: success :star:
Sonar results


Implement columnspan for MathML’s mtd element

I see there has been some discussion, but no progress on this matter. A customer of ours intends to use MathJax for presentational MathML rendering. There are some aligned expressions in their content, currently expressed using mtd/@columnspan, since this works in Firefox’s native MathML implementation at least. <maligngroup/> does not seem to be supported by both implementations. I’m aware of #1104. Is there any chance that either alignment markers or column/row spanning attributes will be supported any time soon? Will it accelerate development if the customer provides some funding?

Column and row span have been on the hoped-for improvements list for some time.  There are some technical difficulties in handling all the cases using CSS tables, but it should be able to cover most of the important situations.
We are in the process of a total rewrite for MathJax v3 (first public beta released two weeks ago).  It would be possible to add column- and row-span support there.  Yes, I believe that support from a client could be used to push this development forward.  I will check into that.  Send me an email through the link in my profile and we can discuss it further.

This is a duplicate of #1102


An error occurred while installing json (1.8.2)

Hello,
I Looking for help because i can't install json (1.8.2)
An error occurred while installing json (1.8.2), and Bundler cannot continue. Make sure that gem install json -v '1.8.2' succeeds before bundling.
Thank's for your help !

Which version of bundler and what operating system are you using? Looks like others have had similar issues due to 'make' and/or 'gcc' missing from the machine, which are needed to build json (ie. https://github.com/bundler/bundler/issues/3650)

since the entire error message is missing, i'll take a guess (and assume you are on os x and use homebrew):
brew install libxml2
bundle config build.nokogiri "--use-system-libraries --with-xml2-include=/usr/local/opt/libxml2/include/libxml2"
bundle install

see this SO

I'm afraid I can't help with this issue through GitHub issues. These issues are just for making suggestions or submitting bug reports with Staytus. I cannot help individually with installing or running the application in the wild. I'd suggest trying posting your issues on a site like Stack Overflow.


x42-plugins: do not know how to unpack source archive

Issue description
The package x42-plugins alternates between building fine and "do not know how to unpack source archive".
It has been doing this for months, even for different versions of the pkg.
It has the same storepath for the .tar.xz, for both the good and bad builds.

This has indeed been noticed more often, but no solution was found. See also https://github.com/NixOS/nixpkgs/issues/18209#issuecomment-244542006.

@x42 Any idea what could cause this?

What does file x42-plugins-20170428.tar.xz say?  Do you have xz installed on the build-machine or a tar built with lzma support?
Does the file's sha1sum match the upstream release?
fd96c827aac170e22deb038966f3d2e55b5740c7 x42-plugins-20170428.tar.xz

@x42 The error is intermittent.
The build environment is the same each time, so the software to unpack is present.
The sha256  sum is checked each time, directly after the download, and never errors.
This error has been occurring on and of trough many versions of your plugins.
AFAIK it has never happened on other packages.

I thought maybe download fails on occasion but If the sha256 sum matches that cannot be the case.
I don't have an explanation for that: If an identical file only works sometimes, it must be something on  NixOS side.
https://github.com/NixOS/nixpkgs/issues/8694 has the same error message for a different package, file-extension being the issue there.

@x42 It indeed looks like the issue is on the NixOS side;I was just hoping you could provide some insight.
The issue you linked seems different, in that it tries to unpack something that shouldn't be unpacked, and it happens consistently.
Thanks for your time.

The same thing happens with newsboat: https://github.com/NixOS/nixpkgs/issues/60845

still important to me


Queue Patron Orders not working for 0 Quality Crafts

Blizzard erroneously is not supplying a crafting quality for this craft, which causes the crafting queue to break.

Noticing the same thing. After clicking the Claim button, the whole Patron tab becomes bugged for me.


New API similar to dartDeferredLibraryLoader

The ACX team is interested in a new API that is similar to dartDeferredLibraryLoader, except instead of a single URI being passed to the handler, a set of URIs is passed. This is useful in our MSS serving mode where we can batch part file loads. If it's easy to pass in a set of part file ids (as opposed to URIs), that would be beneficial as well.

The goal here would be to add a hook like dartDeferredLibraryMultiLoader with the following interface:
(Array<String> uris, Function successCallback, Function errorCallback, int loadId, int loadPriority) -> void
The user would be responsible for ensuring the successCallback is only invoked after all of the provided URIs are loaded (or invoking errorCallback is invoked if any one of the part files fails to load.
Otherwise this would act exactly as dartDeferredLibraryLoader does today. We want both these APIs so live alongside each other.
Longer term we want to allow users to implement custom library loading via a special Dart API but in the meantime this should be a reasonable solution.
This will expand the size of main bundles a small amount but I don't think it should be too significant.

Thanks for the new API!


KOGITO-4827: Reuse Datatypes across process IT tests

not stable enough, investigating couple of areas on CI.
Sleeps will be removed on final commit, before ready for review..

@domhanak Thanks for the initiative. I've been doing a lot of testing with the GitHub actions workflows as well, and as you can see, they're stuck in "Queued" for a while. One thing I noticed that helps a lot is to open a PR inside your own fork to test. Just a hint! :D

@jomarko ready for peer review, I missed some cleanup in last commit so I ll add it after CI proves green.

Just to summarize the changes:

Refactors BPMN related stuff to bpmn folder, same as DMN does
Introduces helper classes for properties panel, diagram navigator
Enhances BpmnEditorTester with new functions
Adds new test for ReuseDataTypes
Refactors the existing tests to use new functions and helper classes
Adds javadoc
Adds locators for use in testing

@tiagobento ready for merging, looks like mac-os has a failure that is unconnected. Mind rerunning it?

Rebased on latest main

@domhanak Conflicts with main after #553. Mind to check please?

@tiagobento rebased on main

@kiegroup/gatekeepers please merge.


Use Travis-CI Matrix to add more type of builds

This gives Recastnavigation coverage in the following areas:

MacOS (Xcode)
Ubuntu Xenial (GCC-5)
Ubuntu Xenial (GCC-8)
Ubuntu Xenial (Clang-7 w/ Static Analysis)
Coverity Scan (preparation work, not yet turned on)

All of this is done via CMake. I still need to add another build that uses premake5 so that there is coverage there as well.

Yeah, I want to have a seperate matrix build that uses only premake.
Question, why premake5 and not premake4? I mean, premake4 is in all distros while premake5 is still alpha (no one ships it).

I believe Premake4 does not support newer versions of VS and xcode. See #89 for more info.

OK, with cmake available is there a need then for premake?
No worries, I'll put in support for it but I'm curious as to why the need for two builds system... especially when one is still in development. :)

I can't speak for the other maintainers, but for myself it is a matter of preference. There was a time where I was optimistic about switching to pure cmake, but unfortunately I have never had a good experience with cmake since then.
I think the main reason is that cmake is very flexible and can accomplish anything. That combined with using a custom specification language makes it not very easy to use, IMO. Compare that to premake which is Lua, and the stuff it can accomplish it does very easily.
Anyway, this is just personal opinion. We have recognized that some users prefer cmake which is why I had no qualms about accepting it when it was contributed. But since the preferred way of integration is by simply copying the needed components, I have never found it to be super important. No build system is going to make integration simpler than that while still allowing the flexibility to modify bits as required.

What I'll do is add Premake5, just so that we have coverage using that build system as well.
Then I'll go about the appveyor side of things as well and do the same thing. That way we cover all the bases and leave the discussion for another time. :)
Sound good?

Sounds perfect!

Done and done. :)

Do you want to do the AppVeyor changes in a separate PR? This looks good to me now, so I can merge this as-is if you'd like.

AV in a separate PR. :)

Cool, thanks a lot for this!

Cheers!


GD32 timer initialization is not working properly

As per https://github.com/Candas1/Split_Hoverboard_SimpleFOC/issues/1#issuecomment-1596307195, there is a serious problem with the timer initialization
I'm opening this as a work item.

This is what I get when running your code.
My power supply was in cc mode because of the high duty cycle.

Your waveforms are smoothed due to the presence of  load on the GPIO pins. But you can clearly see the same spikes I was seeing in the CH2, and those will kill the MOSFETs over time
I'll look at the initialization code and see what's going on. The same code on STM32 produces absolutely clean signals. That could explain the noisy motor control you and @Robodurden are seeing
BTW: you might want to use something like https://www.eevblog.com/forum/testgear/rigol-screen-capture-vcl-windows-application/ it works either via USB or LAN, and it's lightweight and fast

One idea, I have never used the splitboard firmware so I assumed it was working well, but I saw something weird in the code but I haven't had time to investigate.
If I am not wrong on stm32 there is a shadow register were you save the new duty cycle, and it is applied only at the end of the period, on timer update.
I am wondering if this code is the problem.

My plan is to run the STM32 code, dump the register content, run the GD32 code dump as well, and compare the two. As much as the HAL is different from the SPL, the underlying registers are the same, so the values in the end must be the same

Hmm it could really be that, stm32duino enables it, but it was disabled in the splitboard firmware.
We can enable it and try again.

No forget it, I think it happens when you change the duty cycle. This doesn't happen in your case.
I don't think it will solve what you have seen.
But still worth checking.

I made some progress, not as much as I wanted.
There are 3 problems with the timer as is:

Phase G and B are reversed. @Candas1: I haven't looked at where these are swapped, since it's not as critical for now, but maybe you can spot the problem
The timer output is negated compared to the STM32 output for the same values. This is a big issue, probably need to enable the positive/negative edge of the timer, haven't looked into this yet
There is a recurring glitch which disappears when the processor is halted in debug mode. Which means that the glitches are generated by the GD32 Arduino code, given that in halt mode the timer keeps working. I'll open a bug in the GD32 project

I captured more scope images using the values 0, 5, 9 out of a max of 10V, i.e. 0%, 50%, 90%. The names of the files are processor-phase-percent and I have B, G and Y in order for both
First, the STM32

Now GD32 in same order. Note that the GPIO outputs are high when low for the STM32, and vice versa. So 90% is actually 10%, 0 is 100%. 50% looks identical, but it's reversed as well. This will obviously cause problem to the FOC motor control, since the voltages are wrong. The phase G and B being swapped is not as critical for motor control, as long as the motor connections are ok

the code I used in this for both projects

I almost figured out the glitch. It happens when a new PWM value is written to the timer registers (new PWM value, or in the case of my sample code simply re-writes the existing value). The sample code is setting PWM values as quickly as the loop time, which is why I see so many glitches with the loop containing only a single pwm command.
Probably in the STM32 there is a register setting to seamlessly update the register values without glitching the timer, and it's not set for GD32. I'll see if I can find it

It's 6am here, I can take over the shift 😜
Thanks a lot for the hard work.
I am going to work now, I have answers to some of your questions, I will reply when I am back on the computer.

The European shift tales over 😂
As you saw, I solved the glitch and reversed values. What is missing and critical is the wrong values for 0 pwm (and possibly 100%, I haven't checked). Then the swapped G and B phases, which I'm almost sure are simply a channel swap

I see you already enabled the shadow register as mentioned here, this is best practice that was missing in the original firmware. Are you sure this hasn't solved some of the glitches ?
About the PWM being inverted, that's also coming from the original firmware, and EFeru's firmware has the same. This is usually done to center the low side ON time for ADC sampling, see (1) on the picture:

I need to check how simpleFOC deals with that. There should be 2 update events per cycle, maybe they are using the second one, see (2) on the picture.
Regarding this:
void _setSinglePhaseState(PhaseState state, uint32_t timer_periph, uint16_t channel1,uint16_t channel2) {
  _UNUSED(channel2);
  switch (state) {
    case PhaseState::PHASE_OFF:
      timer_channel_output_state_config(timer_periph,channel1,TIMER_CCX_DISABLE);
      timer_channel_complementary_output_state_config(timer_periph,channel1,TIMER_CCXN_DISABLE);
      break;
    default:
      //timer_channel_output_state_config(timer_periph,channel1,TIMER_CCX_ENABLE);
      //timer_channel_complementary_output_state_config(timer_periph,channel1,TIMER_CCXN_ENABLE);
      break;
  }
}

SimpleFOC also seems to turn the phase OFF, that's probably why you see both high side and low side being low. It's called HIGH IMPEDANCE.
The STM32 code is pausing or resuming the channel, it could be the SPL I used is not doing exactly the same thing.
I also have some splitboards with STM32F103 chips, so I could try comparing how the firmware behaves on those boards.
If later you need the counter to stop during debug, I added this but it's commented.
I also noticed the old firmware was using this. I am not sure we want that. This will enable the timer output after each PWM cycle if it's disabled. We need to check if disabling the driver works as expected.
This is how STM is using it for motor control:

Well, my goal was to get the GD32 SimpleFOC driver work the same as the STM32. If we want SimpleFOC to work without surprises, we need to follow the same conventions and have the PWM timer work the same (phase voltage and all).
The 2 lines I commented out here https://github.com/Candas1/Arduino-FOC/blob/7cff1faa6acc47b04dabed47e230265bfb99a875/src/drivers/hardware_specific/gd32/gd32_mcu.cpp#L158 cause the glitch. The STM32 code for that function is much more complex and does a lot more, but I'm not sure that it writes to the same register as the GD32. I haven't fully followed that code yet and it's possible that we need more than just commenting out two lines. The "high impedance" phase_off code is still needed, but not yet called for now. That is definitely needed.
The inverted output must be changed. Otherwise SimpleFOC sends, say, a 10V out of 12V on a phase, but the GD32 driver will send ~2V instead.... not good :)
And we still have a problem with 0V and possibly Vmax, not sure yet. I'll work on that today

When you are saying G and B phases are reversed, are you talking about the  BLDCDriver6PWM function call ?

Just came across this, it can be useful.

When you are saying G and B phases are reversed, are you talking about the BLDCDriver6PWM function call ?

Yes.  On the STM32 with the same definitions, I get as expected  0% on pins A9/B14, 50% on A10/B15 and 90% on A8/B13.
The pins assigned to timer channels must have a problem. Incidentally, the current code has the pin names in the function parameters, but the code uses the macros defined in the settings instead

Yes I agree, I commented the same in the other thread. I tried changing PWM1 to PWM0, it just makes the motor spin in the other direction. I imagine that switching the 2 phases and changing from PWM1 to PWM0 will just make it run in the same direction again.
Touching any of this code, the motor stops spinning, I need to find how to make it work without the glitch.
[...]

I was afraid of this, thanks for checking. The simplest workaround is to check if the timer is already enabled and only call the function to enable it if not enabled. Unlike the PWM call I use for test, the motor control very likely turns off a phase in certain parts of the rotation, and with the current code commented out, the timer is never re-enabled. I will do it today, pretty trivial to check if the timer is already enabled or not

I wouldn't worry about that. On an STM32 or GD32, checking a register takes as many assembly instruction as checking a variable value. All registers are memory mapped and accessible with no delays :)
If you look at the STM32 code, that function is almost a hundred lines of code calling other function. On the GD32, it's going to be just a simple register read, binary and and a check, which is going to be super fast

I tried a lot of things but still it didn't improve.
As a workaround, using SinePWM or SVPWM instead of Trapezoidal, the High Impedance state is not used, and the channel is not enabled/disabled. So commenting the code in the function has no impact. And it is much more silent.

@Candas1 , @robcazzaro , you make me happy and i fear that i can contribute little to that low level library confsusion.
So only take my words as inspiration and do not waste time to explain things to me that are obvious to you !
I have simply deleted all files to let Github-Desktop re-clone his repo here.
I do not understand how the 6 gate driver pins are mapped to the pwm interrupts!
BLDCDriver6PWM driver = BLDCDriver6PWM( BLDC_BH_PIN,BLDC_BL_PIN,  BLDC_GH_PIN,BLDC_GL_PIN,  BLDC_YH_PIN,BLDC_YL_PIN );
only stores the 6 pins to the public (why not private?) element variables
  	int pwmA_h,pwmA_l; //!< phase A pwm pin number
  	int pwmB_h,pwmB_l; //!< phase B pwm pin number
  	int pwmC_h,pwmC_l; //!< phase C pwm pin number

BLDCDriver6PWM::init() sets them to OUTPUT and the only use of these 6 params are in the following _configure6PWM(pwm_frequency, dead_zone, pwmA_h,pwmA_l, pwmB_h,pwmB_l, pwmC_h,pwmC_l);
But when i look at the GD32 implementation of that function, the 6 params are never used at all:
void* _configure6PWM(long pwm_frequency, float dead_zone, const int pinA_h, const int pinA_l, const int pinB_h, const int pinB_l, const int pinC_h, const int pinC_l){
In the stm32_mcu.cpp they are stored to
 int pins[6] = { pinA_h, pinA_l, pinB_h, pinB_l, pinC_h, pinC_l };
Searching those public element variables like pwmA_h with strg+shift+F over the entire projects do also not show any references.
So i wonder how our gd32_mcu.cpp is able to drive the 6 mosfet pins at all ?

So the swapping of BLDCDriver6PWM( BLDC_BH_PIN,BLDC_BL_PIN,  BLDC_GH_PIN,BLDC_GL_PIN,  BLDC_YH_PIN,BLDC_YL_PIN ); of course had not effect :-(
But then i want to mention (like @robcazzaro ?) that HallSensor(HALL_A_PIN, HALL_B_PIN, HALL_C_PIN, 15); seems to init with Green,Blue,Yellow according to the color of the hall cables, but the driver is instantiated with blue,green,yellow.
Maybe i could beautify your "for the moment" code with the Arduino-Core helper functions that retrieve the port and pin from single values BLDC_BH_PIN, etc. :-)

I would rather have something that works and then beautify it.

I'll take over now for the Seattle shift :)
@RoboDurden to add to what @Candas1 says, the SimpleFOC is written for generic boards and a variety of processors. It tries to use every possible pin combination, and there is a lot of code devoted to ensure that the timer/pin combination is the right one. There are STM32 processors with multiple usable timers and pins, unlike in our case. Granted, if we wanted the GD32 code to be truly generic, we would need to use the pins passed in the call, but for now we are focusing on getting it at least usable, then make it more generic.
The reason why almost all the pins in the SimpleFOC code are global, is that the Arduino pin naming convention is crazy and some pins have multiple possible uses. In a case with the STM32G431, I needed to use a pin for the encoder (which uses a timer in encoder mode), and that pin was PB7. But I needed to use the PB_7_ALT1 version to properly set the pin in the timer. The Arduino code has no provision to identify which of the multiple functions a pin can use. So the workaround was to instantiate the encoder with PB7, then set encoder._pinB to PB_7_ALT1 (https://community.simplefoc.com/t/encoder-skips-pulses/2884/38)
It's very hard to write generic code that works on every possible processor family and processors inside a family. The SimpleFOC team did a pretty amazing job, hacks and all

For now channels (not pins), can be swapped here.

This code is taken from gd32/pinmap.c void pin_function(PinName pin, int function) , compilies successfully but i understand that for now you prefer hard coded port/pins and to not care for the higher abstraction levels:
  uint32_t port   = GD_PORT_GET(pinA_h);
  uint32_t gpio = gpio_clock_enable(port);
  uint32_t gd_pin = 1 << GD_PIN_GET(pinA_h);

  gpio_mode_set(gpio, GPIO_MODE_AF, GPIO_PUPD_NONE, gd_pin);
  //gpio_mode_set(TIMER_BLDC_GH_PORT, GPIO_MODE_AF, GPIO_PUPD_NONE, TIMER_BLDC_GH_PIN);

We need much more than that to get it to work that way.

One example
This gives you the timer channel for a pin, it doesn't exist in arduino-GD32.

Dear Candas, don't waste your prescious time to argue with a born loser. getTimerChannel is neither used in stm32_mcu.cpp nor in BLDCDriver6PWM.cpp so i guess it is okay to hard code the timer channel.
But when the constructor asks for the 6 pins, it will only cause "swapping" problems when not making use of them - in the long run.
Have a nice 14 day trip. Holiday or your job ?

Holidays. I will still be brainstorming from there as I cannot take this topic out of my mind.

Be happy to not have "bought" yourself holidays in order to free your mind from a job you hate.

I still think this is more readable and less error prone:
void* _configure6PWM(long pwm_frequency, float dead_zone, const int pinA_h, const int pinA_l, const int pinB_h, const int pinB_l, const int pinC_h, const int pinC_l){
  if( !pwm_frequency || !_isset(pwm_frequency) ) pwm_frequency = _PWM_FREQUENCY; // default frequency 16khz
  else pwm_frequency = _constrain(pwm_frequency, 0, _PWM_FREQUENCY_MAX); // constrain to 50kHz max
  // center-aligned frequency is uses two periods
  pwm_frequency *=2;

  // timeout timer parameter structs
  timer_parameter_struct timeoutTimer_paramter_struct;

  // PWM timer Parameter structs
  timer_parameter_struct timerBldc_paramter_struct;
  timer_break_parameter_struct timerBldc_break_parameter_struct;
  timer_oc_parameter_struct timerBldc_oc_parameter_struct;

  int aiPin[6] = { pinA_h, pinA_l, pinB_h, pinB_l, pinC_h, pinC_l };
  uint32_t aiGdPort[6];
  uint32_t aiGdPin[6];
  for (int i=0; i<6; i++)
  {
    aiGdPort[i] = gpio_clock_enable(GD_PORT_GET(aiPin[i]));
    aiGdPin[i] = 1 << GD_PIN_GET(aiPin[i]);
  }

  // Init PWM output Pins (Configure as alternate functions, push-pull, no pullup)
  for (int i=0; i<6; i++)
  {
    gpio_mode_set          (aiGdPort[i], GPIO_MODE_AF, GPIO_PUPD_NONE,    aiGdPin[i]);
    gpio_output_options_set(aiGdPort[i], GPIO_OTYPE_PP, GPIO_OSPEED_2MHZ, aiGdPin[i]);
    gpio_af_set            (aiGdPort[i], GPIO_AF_2,                       aiGdPin[i]);
  }

  // dead time is set in nanoseconds
  uint32_t dead_time_ns = (float)(1e9f/pwm_frequency)*dead_zone;

  // #todo this needs to be converted
  // uint32_t dead_time = __LL_TIM_CALC_DEADTIME(SystemCoreClock, LL_TIM_GetClockDivision(HT->getHandle()->Instance), dead_time_ns);

  // Enable timer clock
  rcu_periph_clock_enable(RCU_TIMER_BLDC);

  // Initial deinitialize of the timer
  timer_deinit(TIMER_BLDC);
  //dbg_periph_enable(DBG_TIMER0_HOLD); // Hold counter of timer 0 during debug

  // Set up the basic parameter struct for the timer
  timerBldc_paramter_struct.counterdirection  = TIMER_COUNTER_UP;
  timerBldc_paramter_struct.prescaler 		    = 0;
  timerBldc_paramter_struct.alignedmode 	    = TIMER_COUNTER_CENTER_DOWN;
  timerBldc_paramter_struct.period			      = SystemCoreClock / pwm_frequency;
  timerBldc_paramter_struct.clockdivision 	  = TIMER_CKDIV_DIV1;
  timerBldc_paramter_struct.repetitioncounter = 0;
  timer_auto_reload_shadow_disable(TIMER_BLDC);

  // Initialize timer with basic parameter struct
  timer_init(TIMER_BLDC, &timerBldc_paramter_struct);

  // Set up the output channel parameter struct
  timerBldc_oc_parameter_struct.ocpolarity 		= TIMER_OC_POLARITY_HIGH;
  timerBldc_oc_parameter_struct.ocnpolarity 	= TIMER_OCN_POLARITY_LOW;
  timerBldc_oc_parameter_struct.ocidlestate 	= TIMER_OC_IDLE_STATE_LOW;
  timerBldc_oc_parameter_struct.ocnidlestate 	= TIMER_OCN_IDLE_STATE_HIGH;

  // Set up the break parameter struct
  timerBldc_break_parameter_struct.runoffstate      = TIMER_ROS_STATE_ENABLE;
  timerBldc_break_parameter_struct.ideloffstate     = TIMER_IOS_STATE_DISABLE;
  timerBldc_break_parameter_struct.protectmode	    = TIMER_CCHP_PROT_OFF;
  timerBldc_break_parameter_struct.deadtime 	      = DEAD_TIME; // 0~255
  timerBldc_break_parameter_struct.breakstate	      = TIMER_BREAK_DISABLE;
  timerBldc_break_parameter_struct.breakpolarity	  = TIMER_BREAK_POLARITY_LOW;
  timerBldc_break_parameter_struct.outputautostate 	= TIMER_OUTAUTO_ENABLE;

  // Configure the timer with the break parameter struct
  timer_break_config(TIMER_BLDC, &timerBldc_break_parameter_struct);

  // Disable until all channels are set for PWM output
  timer_disable(TIMER_BLDC);

  uint16_t aiChannel[3] = {TIMER_BLDC_CHANNEL_G,TIMER_BLDC_CHANNEL_B,TIMER_BLDC_CHANNEL_Y};

  for (int i=0; i<3; i++)
  {
    timer_channel_output_fast_config(TIMER_BLDC, aiChannel[i], TIMER_OC_FAST_DISABLE);      // Deactivate output channel fastmode
    timer_channel_output_shadow_config(TIMER_BLDC, aiChannel[i], TIMER_OC_SHADOW_DISABLE);  // Deactivate output channel shadow function
    timer_channel_output_mode_config(TIMER_BLDC, aiChannel[i], TIMER_OC_MODE_PWM1);         // Set output channel PWM type to PWM1
    timer_channel_output_pulse_value_config(TIMER_BLDC, aiChannel[i], 0);    // Initialize pulse length with value 0 (pulse duty factor = zero)
    timer_channel_output_config(TIMER_BLDC, aiChannel[i], &timerBldc_oc_parameter_struct);  // Configure with the output channel parameter struct
    timer_channel_output_state_config(TIMER_BLDC, aiChannel[i], TIMER_CCX_ENABLE);          // Enable for PWM output
    timer_channel_complementary_output_state_config(TIMER_BLDC, aiChannel[i], TIMER_CCXN_ENABLE); // Enable complemenary channel for PWM output
  }

  // Enable TIMER_INT_UP interrupt and set priority
  //nvic_irq_enable(TIMER0_BRK_UP_TRG_COM_IRQn, 0, 0);
  //timer_interrupt_enable(TIMER_BLDC, TIMER_INT_UP);

  // Enable the timer and start PWM
  timer_enable(TIMER_BLDC);

  GD32DriverParams* params = new GD32DriverParams {
    .timers = { TIMER_BLDC, TIMER_BLDC, TIMER_BLDC, TIMER_BLDC, TIMER_BLDC, TIMER_BLDC },
    .channels = { aiChannel[0], aiChannel[0], aiChannel[1], aiChannel[1], aiChannel[2], aiChannel[2] },
    .pwm_frequency  = pwm_frequency,
    .range          = SystemCoreClock/pwm_frequency,
    .dead_zone      = dead_zone,
    .interface_type = _HARDWARE_6PWM
  };

  return params; // success
}

We can always improve the code.
This is mostly a copy paste of the timer setup from your splitboard repository into the simpleFOC driver code.

Ok, here's the working code (no glitch, can turn phases off). I needed to add a function _isChannelDisabled() before _setSinglePhaseState()
Just replace _setSinglePhaseState with the snippet below
uint32_t _isChannelDisabled(uint32_t timer_periph, uint16_t channel)
{
  switch (channel)
  {
  /* configure TIMER_CH_0 */
  case TIMER_CH_0:
    return (TIMER_CHCTL2(timer_periph) & (uint32_t)TIMER_CHCTL2_CH0EN);
    break;
  /* configure TIMER_CH_1 */
  case TIMER_CH_1:
    return (TIMER_CHCTL2(timer_periph) & (uint32_t)TIMER_CHCTL2_CH1EN);
    break;
  /* configure TIMER_CH_2 */
  case TIMER_CH_2:
    return (TIMER_CHCTL2(timer_periph) & (uint32_t)TIMER_CHCTL2_CH2EN);
    break;
  /* configure TIMER_CH_3 */
  case TIMER_CH_3:
    return (TIMER_CHCTL2(timer_periph) & (uint32_t)TIMER_CHCTL2_CH3EN);
    break;
  default:
    return 0xFFFF;    // should never get here. Added to avoid compile warning
    break;
  }
}

void _setSinglePhaseState(PhaseState state, uint32_t timer_periph, uint16_t channel1, uint16_t channel2)
{
  _UNUSED(channel2);
  switch (state) {
    case PhaseState::PHASE_OFF:
      timer_channel_output_state_config(timer_periph,channel1,TIMER_CCX_DISABLE);
      timer_channel_complementary_output_state_config(timer_periph,channel1,TIMER_CCXN_DISABLE);
      break;
    default:
      if (_isChannelDisabled(timer_periph, channel1) == 0) // enable timer channel only if disabled to avoid glitches
      {
        timer_channel_output_state_config(timer_periph,channel1,TIMER_CCX_ENABLE);
        timer_channel_complementary_output_state_config(timer_periph,channel1,TIMER_CCXN_ENABLE);
      }
      break;
  }
}

Thanks a lot.
I included it in the driver. I think I still see glitches from my side, but maybe we are not talking about the same glitch.
I also used your function but for when phase has to be disabled, and some small other changes.
It's published, you can try to update arduino-FOC from your side.

As a reminder, we have 2 remaining problems, one minor, one major
The minor one is the swapped G and B phases, which @Candas1 has a workaround for
The major is that 0 and 100% are still wrong. In the STM32, when the value is 0, a9 low, b14 high. when 100%  a8 high, b13 low
in the GD32 at 0 a10 high, b15 low, 100% a8 high, b13 high
I see also weirdness when using values like 98%, I'm investigating further

Tell me, I am still awake 😂

The complementary input is inverted, but at this point I don't know if it's because I messed something up with the many code changes. I need to download your code and start fresh. But I also need to take a break for another task, so it won't be for a couple of hours at least. Go to sleep 😉

OK no worries. I will still have some time tomorrow.
Platformio is a bit confusing me. The code was working, I published. Then I ran the update command but the wheel started cogging. I changed the zero angle offset again and it spins as least but it's a bit noisy. I definitely need to improve the calibration.
But SinePWM and SpaceVectorPWM are really smooth.

Ok, I think I have solved everything (minus the phase swap)
There were a couple of issues here.
The OC and OCN polarity should both be high, and the OC and OCN idle states must be low. It's counterintuitive, but they want to know if the active state of the complementary output is high when it should be high (which is what we want). One would think they want to know the phase of the complementary output (which is the opposite of the normal output), but that would be wrong 😉
Also, these calls use the full imerBldc_oc_parameter_struct content. When debugging, the structure could be initialized to values != 0, and when that happens, the register settings are completely messed up. It's safer to initialize it fully, including settings we change later.
So I changed the two definitions for N polarity and added the outputstate and outputnstate to be on the safe side
  // Set up the output channel parameter struct
  timerBldc_oc_parameter_struct.outputstate   = TIMER_CCX_DISABLE;
  timerBldc_oc_parameter_struct.outputnstate  = TIMER_CCXN_DISABLE;
  timerBldc_oc_parameter_struct.ocpolarity    = TIMER_OC_POLARITY_HIGH;
  timerBldc_oc_parameter_struct.ocnpolarity 	= TIMER_OCN_POLARITY_HIGH;
  timerBldc_oc_parameter_struct.ocidlestate 	= TIMER_OC_IDLE_STATE_LOW;
  timerBldc_oc_parameter_struct.ocnidlestate 	= TIMER_OCN_IDLE_STATE_LOW;

With those changes the PWM works as expected, all polarity are correct, including 0 and 100%

@Candas, I noticed a bug in your code here
The function _isChannelEnabled does not return a boolean, but the register value in the channel position (so 1, 16, 256 and 4096 depending on the channel). I did it to speed up the execution, there's no point shifting bits just to get a 0 or 1 result, all that matters is 0 or not zero
If you prefer to have an if statement, used
if (_isChannelEnabled(timer_periph, channel1) != 0) // disable timer channel only if enabled to avoid glitches
(I didn't put one because it was not needed: there's no glitch disabling an already disabled channel, unlike enabling it... but yours is safer, as long as you use != 0)

good morning from Germany :-)
@robcazzaro , your last comment is 4 hours ago and gd32_mcu.cpp changed 6 hours ago. But i do not see what you want to be changed. There is already if (_isChannelEnabled(timer_periph, channel1) != 0)
I tested with that latest code and motor indeed spins as smoothly as "my" gen2.x firmware with
  // choose FOC modulation (optional)
  //motor.foc_modulation = FOCModulationType::Trapezoid_150;
  //motor.foc_modulation = FOCModulationType::SpaceVectorPWM;
  motor.foc_modulation = FOCModulationType::SinePWM;

SpaceVectorPWM also works nicely and i do not see or hear a difference to SinePWM
Both modulations draw about the same current of 160 mA at 26 Volt just like the gen2.x binary.
Here my one-liner for testing backward/forward (5 seconds to go from zero to +-max):
  float fSpeed = 5.0 * (ABS(	(float)((millis()/50+100) % 400) - 200) - 100)/100;
  //fSpeed = 5;
  motor.move(fSpeed);

A target of 5 generates about the same speed as speed = -320; with TEST_SPEED of my gen2.x code.
The motor.foc_modulation = FOCModulationType::Trapezoid_150; is very loud for a target (speed?) of 2,
gets more silent for 3 but at 5, the motor blocks and i fearfully quickly override the binary to esacpe a blocked motor.

Thanks a lot @robcazzaro , I did the changes. But platformio really behaves in a weird way. It's not always using the driver code I changed locally.
When I published and refreshed, the polarity changes triggered shorts, so I reverted back.
While searching I notice this motor control code, it's using functions to init the structure properly, I did to be on the safer side.
I also remove the channel enabling code from the driver init, so we use driver.enable() for that.
@RoboDurden , I added your one liner but with a smaller speed.

I think having several environments for different boards was creating a mess.
I still had the C6 related dependencies although I had removed the board for platformio.ini.
I deleted those folders manually and simplified the platformio.ini.

Lastly I switched to sinePWM, trapezoidal is really too noisy.
The target goes from -5 to 5. Higher is possible if you increase the voltage limit.
I tried removing the pin initialization code here to see if the code here is good enough, but it didn't work anymore. The arduino-gd32 code probably has to be checked.

Thanks a lot @robcazzaro , I did the changes. But platformio really behaves in a weird way. It's not always using the driver code I changed locally. When I published and refreshed, the polarity changes triggered shorts, so I reverted back.

Now that I see "the polarity changes triggered shorts" I finally started realizing that the MOSFET circuit in the hoverboard might be different from the usual SimpleFOC drivers.
For reference, this is the waveforms I was trying to achieve for a PWM of 30%, with CH1 being the positive signal and CH2 the complementary one

But according to your note, that seems to cause shorts. Can one of you please send me the working waveform for 30% PWM (or any non 0 or 100% PWM), using the same CH1 CH2 assignments? I want to make sure I can check the signals properly going forward.
I'll also look at the pin initialization code and add a function or macro to properly calculate the dead time (which, incidentally, I verified and a value of 60 is indeed 1250 ns)

With working waveform you mean the full candas code not with my one Liner but a static target of something like 2.0 ?
I am not sure if we will see some nice static pwm for a rotating motor ?
I only have a 1 channel pocket dso.
I would like to probe phase a to phase b without the motor connected. Then I expect a positive voltage and a negative voltage depending on hall inputs and rotation direction..
For two dso channels of one timer channel with it's hiside and low side gate outputs I fear you must wait for candas to return from holidays.
I am still looking for a 2 channel pocket dso on AliExpress for $50 :-/

To my understanding, the 40+14V gate driver voltage of the high-side mosfets is "somehow" generated by a charge pumpe driven by the pwm of the low-side mosfets. So without a motor connected, i may not see a lot when probing phase A to phase B.
You can look at the schematics pdf of the gen1 board, found on the EFeru FOC repo. The gen2.0 boards without half-bridge driver chips seem to use the same analog circuit with 3 transistors for each mosfet.
But you are probing the GD32 output pins don't you ? On your bluepill maybe without pull-down resistors..

Sorry I checked on the scope but forgot to record it.
I was basically getting the same signal twice.
Now I am out but I will check tomorrow if I have time. I fly only on Friday.
I haven't swapped the phases because the motor was smooth in sinepwm, svpwm, and in simplefoc you have either to swap the pins or change this.
I guess next on the to-do list could be:

derive the phases from the pins in the call to BLDCDriver6PWM so people could swap the phases if needed, this could probably be used
use the deadtime parameter from simplefoc
check how the pins are set in BLDCDriver6PWM

I am not sure working on software6pwm would be really useful for now

I'm trying to get a signal on the scope using either the original firmware, or another firmware not using SimpleFOC, to check the polarity of the signal driving the MOSFET gates. Usually the signal used must be complementary (i.e. when the high side is high, the low side is low, and vice versa, but the hoverboard seems to use the same polarity for high and low side... I want to be sure that's the case)
So the idea would be to connect the scope probes to a pair of complementary GD32 pins, set the scope in single shot mode, and capture the signal. Even if the PWM keeps changing when the motor runs, the single shot will freeze a portion of the signal that is representative enough.
@Candas1, I will take on those 3 work items while you frolic on vacation 😁. I had already planned to look at those
And, yes, I agree, no value doing the software PWM. We might want to look at the other hardware PWMs just to have am ore complete GD32 solution, but would not work for the hoverboards, so low priority

The target goes from -5 to 5. Higher is possible if you increase the voltage limit.

5.0 is  way more slowly than the 1000 of the gen2.x firmware which i think is 100% pwm.
What should be the values to test max speed ?
Unfortunately, my one liner needs an offset because the setup() function takes so long that the motor no longer start from zero:
unsigned long iLoopStart = 0;
void setup()
{
  ......
  iLoopStart = millis();
}

void loop()
{
  float fSpeed = 5.0 * (ABS(	(float)(((millis()-iLoopStart)/50 + 100) % 400) - 200) - 100)/100;

Not very intuitive

Don't think that my 1-channel scans are helpful:
gd32 yellow hi side and lo side mcu outputs:  https://youtu.be/0w15tCuFyrE

gd32 phase difference blue yellow with 10x probe: https://youtu.be/uHkZAgpiEuQ

Thanks @RoboDurden, this is good. So that means that, indeed, the complementary signal is high when the output is high, which is the opposite of what I was trying to achieve (based on the standard STM32 for SimpleFOC). 🤦♂. Apologies about that, but I'm glad we solved it
Just to be clear: was this with the SimpleFOC version, or your old firmware? Would be good to see what your old firmware did. Also, where did you capture the phase difference on the board?
I tried with the original firmware, but unless everything is connected, it doesn't work, just beeps. And I can't reassemble everything, I don't have all the pieces anymore, so I won't be able to test. But if your motor is turning, and the board not burning, we must be on the right path  😁

I looked at the STM32 hoverboard schematic and the EFeru code, and indeed I was wrong on the polarity. The high phase goes thru an extra inverter transistor, so the outputs must be both high or low to properly drive the bridge, apologies for wasting your time on this.
Here's the STM32 timer init code from EFeru's repository
``
sConfigOC.OCMode       = TIM_OCMODE_PWM1;
sConfigOC.Pulse        = 0;
sConfigOC.OCPolarity   = TIM_OCPOLARITY_HIGH;
sConfigOC.OCNPolarity  = TIM_OCNPOLARITY_LOW;
sConfigOC.OCFastMode   = TIM_OCFAST_DISABLE;
sConfigOC.OCIdleState  = TIM_OCIDLESTATE_RESET;
sConfigOC.OCNIdleState = TIM_OCNIDLESTATE_SET;
Both the OC and OCN polarity and idle state must be set opposite of what I did (and like it's in the code now)

@Candas1 I noticed that there is still a [bug](https://github.com/Candas1/Arduino-FOC/blob/d9051562df093e10b6e30943e254dd5d758df1bd/src/drivers/hardware_specific/gd32/gd32_mcu.cpp#L172) in the PWM ON/OFF code.

That line must be "!=0" not "!=1"
``` if (_isChannelEnabled(timer_periph, channel1) != 0) // disable timer channel only if enabled to avoid glitches```

It does work anyway, given that the function never returns 1 😁 so the code is the same as without the if statement. And turning off the same phase multiple times has no negative side effect. But, to save execution time when a phase stays disabled, having "!=0" would be better.

@robcazzaro  i do not understand a lot about these low level timer functions. That is mainly a talk between you and candas.
Yes i will make the same dso test tomorrow with my gen2.x firmware. These two test up here have been made with this Candas1/Split_Hoverboard_SimpleFOC here.
The second test was putting gnd of the dso to the outlet of yellow phase, the probe of the dso was held to the middle phase cable. The motor was still connected and spinning +-5.0

@RoboDurden understood, and appreciate your help. The reason why I'm so focused on the low level timer, is because that initialization code is fundamental to everything else on top of it. SimpleFOC relies on a smooth PWM to be able to use the FOC algorithm properly. Especially at higher speeds or higher torque, proper control of the PWM with proper dead times is also critical to keep the MOSFETs cool and in good shape. And the motor running quietly. SimpleFOC has a lot of settings to ensure smooth running, smooth accelerations, etc, but it all relies on the PWM underneath. So I keep annoying everyone with it 😉
At this point we are looking really solid, and I'm happy. I'll work to improve the code for proper pin initialization to be more generic and to derive the dead time from the initialization settings (it's hardcoded in a constant now), but none of it is critical now for the rest of the project. So, please disregard my ramblings and keep working on the rest of the code. And if you run into any low level problem, open an issue and send it my way

Don't worry guys, we are all learning, and that's why we need to use power supplies.
@robcazzaro You are spending a lot of effort with this project and will not benefit if you don't have the real hardware. Let's do something about that

@Candas1 I do actually have the real hardware, a Layout 2.0 board with a C8. I just didn't want to burn the only board I have while messing around with low level timer code (and, given all the tests I did, I would have killed it almost for sure). I have a hard time finding other boards that I can work on (the other I have uses a weird MM32 processor which is almost impossible to support. Just buying random hoverboards gets expensive, unless you can find non-working ones (usually just a dead battery)
I can now use the actual board, but I will still prefer the Bluepill for any low level development: it's so much easier to probe pins on the Bluepill than trying to hold a probe on a tiny pad in the real board, not to mention that having nothing else connected I can verify waveforms and timing without other components influencing it. For example, the dead time timing is easier to measure on the Bluepill.

so here the dso comparison between the simpleFOC firmware and the gen2.x binary: https://youtu.be/T1nndQ1UpqM
The gen2.x firmware does a rect wave when not spinning the motor, which i think is needed for layouts without dedicated gate driver chips because the rectangle wave form for the low-side mosfet drives a charge pump to generate the 40+14V to drive the hi-side mosfets. But i may be wrong about that.
The simpleFOC code at least seem to have some 10% duty spikes when not spinning the motor and that could be enough to generate the 54V. But i may be wrong about this also.
Quite hot here in Germany and i have had little and bad sleep the last three days. Will return to my trainsation today.

Thanks for the additional screenshots
Methinks that both the gen2 firmware and SimpleFOC send power to the motor to keep it in place, powered, and prevent movement when speed is 0.
 When the motor is static, I'm sure that SimpleFOC keeps the phases energized to prevent movement when speed is 0.
The reason why the gen2 has a square wave when not moving, is probably because they apply a much stronger force to hold them motor, while with the settings we have now, SimpelFOC only applies a weak current to hold the motor. So gen2 uses roughly half the voltage for each phase, while SimpleFOC uses probably10-15% of the voltage. Changing the torque settings in SimpleFOC would increase that PWM value, and get to a square wave, too
Easy to verify: try moving the wheel by hand when powered on and not rotating, and you should feel a string resistance to movement, stronger in gen2 than SimpleFOC

Easy to verify: try moving the wheel by hand when powered on and not rotating, and you should feel a string resistance to movement, stronger in gen2 than SimpleFOC

No, not really. With SimpleFOC, the current only rises from 30 mA (@ 26V, setting the 6 gate ouputs back to INPUT to free the motor) to 70ma with motor.move(0.0);
The motor can still be moved quite freely.
Same for the gen2 firmware. To my experience, all hoverboard firmwares do not apply a constant torque on the wheels when only driven in pwm-mode. There is no closed loop to keep a certain speed or torque or position.
And for sure the not changing 50% duty square wave on hiside and loside of the gen2 with speed = 0 must cancel each other out, otherwise some uncontroled current would flow.
22:03 here in Germany now and i only have a 3W led lamp here in my solar powered train station. Too dark to probe the gate pins now.
I think the simpleFOC generates a 10% duty pwm on hi and low at motor.move(0.0); that also cancel each other out. And that 10% duty pwm might be sufficient to drive the charge pump.
Maybe you can fully understand how that C1 capacitor is driven to generate the 14+40V:

When the lo-side mosfet is closed (conducting), C1 is charged, when it is open, its voltage adds to the middle point of that half bridge (= source of hi-side mosfet) and the gate of the hi-side can be driven by that voltage.
But a pwm has to be applied to the lo-side mosfet at all time, i think.

Got it, thanks for the additional info!

SimpleFOC has a very good documentation, this can help understand how it works.
Sorry guys I couldn't check anything and I leave soon.

In my opinion, it's good enough to be used, and should already be as good as the motor control in gen2 firmware?
What do you think @robcazzaro ?
I haven't tried debugging after the recent changes, I would still not do it without a power supply.
We will still improve the GD32 code, but it shouldn't improve the pwm signal itself.
Now we need to make a better use of simplefoc.
Next release of simplefoc will also bring some improvements.
The current sensing for foc shouldn't be too complicated but not all the boards have that.
We could create a list of tasks, agree on the priorities, align on the dependencies not to impact each other. Like a real project 😂

I started to list some ideas here.

Just for the record, here the pwm for target 0 at the yellow lo-side gates for Gen2.x and this SimpleFOC repo here:

I guess the lower duty cycle of the simpleFOC library does not make a difference for the charge pump :-)

It could be this is happening because I kept the voltage_limit low to play it safe for the tests. limits.png

If we do not set the voltage limit, it will be equal to the power supply voltage, so the duty cycle should be 50% with target 0 as the modulation is centered
https://github.com/Candas1/Arduino-FOC/blob/d9051562df093e10b6e30943e254dd5d758df1bd/src/drivers/BLDCDriver6PWM.cpp#L61

Indeed yes :-)
  driver.voltage_power_supply = 26;
  driver.voltage_limit = 26;

produces the same 50% duty square rect at target speed 0 :-)
Will this then test max speed ?
float fSpeed = 26.0 * (ABS(	(float)(((millis()-iLoopStart)/50 + 100) % 400) - 200) - 100)/100;

I think so

But the speed is already reached at about 12.0. Setting it higher will only make the motor spin longer at the max of 12.0. So either that is the max or there is a clamp somewhere in the library.
Max current is then slightly above 0.5A so i increased it to 1.0 A CC.

That's weird. And the duty cycle is 100%, do you know the max speed you reach in rpm?

for that i would need to get the uart debug output working so i can log the rpm.
The Gen2.x at speed 1000 definitely spins faster an the current is still below 500mA (about 480 mA).
The simpleFOC code draws more current (about 550 mA) and spins more slowly :-/

You could try simplefocdebug and monitoring that I added recently

In my opinion, it's good enough to be used, and should already be as good as the motor control in gen2 firmware? What do you think @robcazzaro ?

Yes, the initialization code is robust and safe to use, I'll add the few missing things (as per your spreadsheet) soon, but none makes a real difference for now

We could create a list of tasks, agree on the priorities, align on the dependencies not to impact each other. Like a real project 😂

Yes, good idea. Please feel free to assign any low level tasks to me as a default. I'm much more useful working at that level, and you both are much more knowledgeable about the motors and their control

Here another dso test that compares the voltage differences of Gen2.x vs SimpleFOC between pahse blue and yellow at max speed: https://youtu.be/DFzkf_X7WHU
The dso output should reflect the current flowing through the blue-yellow windings and be proportional to the speed ?!
But i do not really see a difference between Gen2.x and SimpleFOC:

Both binaries produce these double pulses also at max speed. I guess that is okay because we have a vector addition of the three sets of coils, and even at max speed, one set must vary its current to achive a rotating vector.
The red horizontal lines is a voltage difference of zero = no current flowing.
I notice that the possible (i guess) max current of 90+% duty and 10% no current is never reached but that the 10% off must induce a sligtly opposite current that i guess cancle out some of the 90% current :-(
But this is only 1/3 of the motor coils, so the 90% duty cycle overlaps with green-blue and green-yellow anyway.
And again, i do not see a difference to the simpleFOC ouput.
But motor is definitely spinning more slowly.
When i click on Custom->Update_SimpleFOC that succeeds.
But Custom->Update_gd32 fails ,  because of my twowire.cpp changes from my other GD32_I2C_Slave project :-/
Don't know how to fetch the latested Arduino-GD32-Core :-(
Processing GD32F130C8 (framework: arduino; platform: https://github.com/CommunityGD32Cores/platform-gd32.git; board: genericGD32F130C8)
------------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/gd32/genericGD32F130C8.html
PLATFORM: GD GD32 (1.0.0+sha.3424e66) > GD32F130C8 (8k RAM, 64k Flash)
HARDWARE: GD32F130C8T6 48MHz, 8KB RAM, 64KB Flash
DEBUG: Current (stlink) External (blackmagic, cmsis-dap, jlink, sipeed-rv-debugger, stlink)
PACKAGES:
 - framework-arduinogd32 @ 4.20000.210603+sha.d0cf857
 - toolchain-gccarmnoneeabi @ 1.90201.191206 (9.2.1)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Library Manager: Installing Arduino-FOC @ 2.3.0+sha.d905156
Warning! Could not find the package with 'Arduino-FOC @ 2.3.0+sha.d905156' requirements for your system 'windows_amd64'
Found 14 compatible libraries
Scanning dependencies...
Dependency Graph
|-- RTT Stream @ 1.4.0
|-- Simple FOC @ 2.3.0+sha.d905156
Building in release mode
pio pkg update -g -p gd32
Platform Manager<EMAIL_ADDRESS>is already up-to-date
Tool Manager: Updating framework-arduinogd32 @ 4.20000.210603+sha.d0cf857
git version 2.39.2.windows.1
error: Your local changes to the following files would be overwritten by merge:
        libraries/Wire/src/Wire.cpp
Updating d0cf857..dfa7ccb
        libraries/Wire/src/utility/twi.cpp
Please commit your changes or stash them before you merge.
Aborting
Error: VCS: Could not process command ['git', 'pull', '--recurse-submodules']
*** [Update gd32] Error 1
================================================================ [FAILED] Took 5.94 seconds ================================================================
 *  The terminal process "C:\Users\PAN CF-LX6\.platformio\penv\Scripts\platformio.exe 'run', '--target', 'Update gd32', '--environment', 'GD32F130C8'" terminated with exit code: 1.
 *  Terminal will be reused by tasks, press any key to close it.

@RoboDurden to update only the GD32 Arduino framework To update, use a PlatformIO core CLI and execute
pio pkg update -g -p gd32

thanks. How do i stash on the cli ?
PS C:\GitHub\Candas1\Split_Hoverboard_SimpleFOC> pio pkg update -g -p gd32
Platform Manager<EMAIL_ADDRESS>is already up-to-date
Tool Manager: Updating framework-arduinogd32 @ 4.20000.210603+sha.d0cf857
git version 2.39.2.windows.1
error: Your local changes to the following files would be overwritten by merge:
        libraries/Wire/src/Wire.cpp
        libraries/Wire/src/utility/twi.cpp
Please commit your changes or stash them before you merge.
Aborting
Updating d0cf857..dfa7ccb
Error: VCS: Could not process command ['git', 'pull', '--recurs

No idea, sorry. In cases like these I simply copy the files to a temp location on my disk, discard them and update. If needed, I manually merge the files by copying the relevant lines in the IDE. Not very elegant, I admit 😁
I never took the time to learn how GIT commands work from Platformio. But it's all here https://docs.platformio.org/en/stable/integration/ci/github-actions.html

Thanks @robcazzaro , after removing C:\Users\PAN CF-LX6\.platformio\packages\framework-arduinogd32\libraries\Wire the update succeeded :-)

@RoboDurden if you want to discard changes, you can do so from the UI, too, much easier than chasing the right files on disk.

If you right click on the file name itself, it also offers more options (like discard changes)
From that UI it's easy to see what needs to be changed to unblock the updates, and decide what to do

the poor motor behavior of the SimpleFOC binary is still the same :-/
Maybe someone of you discovers a difference in the dso ouputs shown in my latest video.
Good night for today from a rainy and stormy Germany :-)

In the past, I had problems with SimpleFOC due to the limited speed of the encoder readings. The Arduino interrupt handling code has a lot of overhead, made worse by the various processor layers. If the code handling the hall sensors is not fast enough, speed is limited by that. And we are using a 48MHz processor, pretty slow all considered.
I don't have time today, though. Over the weekend, I'll look at ways to speed up the interrupt handler and see if that improves anything.

BTW: we should open a new issue every time we discover a problem like this one. Otherwise things become hard to follow. The motor performance is not a timer issue 😉

I am still not sure if the sensor alignment is good.
One way to check this is described here
Do you see/hear differences in how the motor spins in each direction with both binaries?
A wrong offset could lead to phase advance in one direction, and phase delay in the other direction.

In the past, I had problems with SimpleFOC due to the limited speed of the encoder readings. The Arduino interrupt handling code has a lot of overhead, made worse by the various processor layers.

There is some improvement in the next release https://github.com/simplefoc/Arduino-FOC/issues/270


Support type extension?

Some of our existing Cue schemas use similar concepts, and cuetsy supports generating Typescript code that relies on extending types.
Should we support it in cog as well?

Not needed