Trouble installing pyqt5 (Win 10 22H2 with Python 3.7.3)

Hi, bought the kit via Lectronz (4158) Any idea why the building is failing? What Python version are you using?
I have python 3.7.3 and pip 23.3 on W10 22H2
PS C:\dopplerradar\scripts> pip install pyqt5 chaco stm32loader
Collecting pyqt5
  Using cached PyQt5-5.15.10-cp37-abi3-win_amd64.whl.metadata (2.2 kB)
Requirement already satisfied: chaco in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (6.0.0)
Requirement already satisfied: stm32loader in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (0.5.1)
Collecting PyQt5-sip<13,>=12.13 (from pyqt5)
  Using cached PyQt5_sip-12.13.0.tar.gz (123 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyQt5-Qt5>=5.15.2 (from pyqt5)
  Using cached PyQt5_Qt5-5.15.2-py3-none-win_amd64.whl (50.1 MB)
Requirement already satisfied: traits>=6.2.0 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from chaco) (6.4.3)
Requirement already satisfied: traitsui in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from chaco) (8.0.0)
Requirement already satisfied: pyface>=7.2.0 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from chaco) (8.0.0)
Requirement already satisfied: numpy in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from chaco) (1.18.2)
Requirement already satisfied: enable>=5.2.0 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from enable[layout,svg]>=5.2.0->chaco) (6.0.0)
Requirement already satisfied: pyserial in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from stm32loader) (3.4)
Requirement already satisfied: progress in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from stm32loader) (1.6)
Requirement already satisfied: pillow in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from enable>=5.2.0->enable[layout,svg]>=5.2.0->chaco) (9.5.0)
Requirement already satisfied: fonttools in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from enable>=5.2.0->enable[layout,svg]>=5.2.0->chaco) (4.38.0)
Requirement already satisfied: kiwisolver in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from enable[layout,svg]>=5.2.0->chaco) (1.4.5)
Requirement already satisfied: pyparsing in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from enable[layout,svg]>=5.2.0->chaco) (2.4.6)
Requirement already satisfied: importlib-metadata>=3.6 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from pyface>=7.2.0->chaco) (6.7.0)
Requirement already satisfied: importlib-resources>=1.1.0 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from pyface>=7.2.0->chaco) (5.12.0)
Requirement already satisfied: zipp>=0.5 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from importlib-metadata>=3.6->pyface>=7.2.0->chaco) (3.1.0)
Requirement already satisfied: typing-extensions>=3.6.4 in c:\users\philippe\appdata\local\programs\python\python37\lib\site-packages (from importlib-metadata>=3.6->pyface>=7.2.0->chaco) (4.7.1)
Using cached PyQt5-5.15.10-cp37-abi3-win_amd64.whl (6.8 MB)
Building wheels for collected packages: PyQt5-sip
  Building wheel for PyQt5-sip (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for PyQt5-sip (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [42 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'PyQt5.sip' extension
      creating build
      creating build\temp.win-amd64-cpython-37
      creating build\temp.win-amd64-cpython-37\Release
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /Tcapiversions.c /Fobuild\temp.win-amd64-cpython-37\Release\apiversions.obj
      apiversions.c
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tpbool.cpp /Fobuild\temp.win-amd64-cpython-37\Release\bool.obj
      bool.cpp
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcdescriptors.c /Fobuild\temp.win-amd64-cpython-37\Release\descriptors.obj
      descriptors.c
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcint_convertors.c /Fobuild\temp.win-amd64-cpython-37\Release\int_convertors.obj
      int_convertors.c
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcobjmap.c /Fobuild\temp.win-amd64-cpython-37\Release\objmap.obj
      objmap.c
      objmap.c(99): warning C4311: 'type cast': pointer truncation from 'void *' to 'unsigned long'
      objmap.c(100): warning C4311: 'type cast': pointer truncation from 'void *' to 'unsigned long'
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcqtlib.c /Fobuild\temp.win-amd64-cpython-37\Release\qtlib.obj
      qtlib.c
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcsip_array.c /Fobuild\temp.win-amd64-cpython-37\Release\sip_array.obj
      sip_array.c
      sip_array.c(322): warning C4090: '=': different 'const' qualifiers
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcsiplib.c /Fobuild\temp.win-amd64-cpython-37\Release\siplib.obj
      siplib.c
      siplib.c(2604): warning C4267: 'function': conversion from 'size_t' to 'unsigned long', possible loss of data
      siplib.c(2901): warning C4013: 'sipSetBool' undefined; assuming extern returning int
      siplib.c(3905): warning C4133: '=': incompatible types - from 'PyObject *' to 'sipSimpleWrapper *'
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcthreads.c /Fobuild\temp.win-amd64-cpython-37\Release\threads.obj
      threads.c
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\include -IC:\Users\Philippe\AppData\Local\Programs\Python\Python37\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\winrt" /EHsc /Tcvoidptr.c /Fobuild\temp.win-amd64-cpython-37\Release\voidptr.obj
      voidptr.c
      creating C:\Users\Philippe\AppData\Local\Temp\pip-install-4c0wkmp7\pyqt5-sip_7547275e66eb4e758b17007ad84cc49e\build\lib.win-amd64-cpython-37

      creating C:\Users\Philippe\AppData\Local\Temp\pip-install-4c0wkmp7\pyqt5-sip_7547275e66eb4e758b17007ad84cc49e\build\lib.win-amd64-cpython-37\PyQt5
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Philippe\AppData\Local\Programs\Python\Python37\libs /LIBPATH:C:\Users\Philippe\AppData\Local\Programs\Python\Python37 /LIBPATH:C:\Users\Philippe\AppData\Local\Programs\Python\Python37\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.20348.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.20348.0\um\x64" /EXPORT:PyInit_sip build\temp.win-amd64-cpython-37\Release\apiversions.obj build\temp.win-amd64-cpython-37\Release\bool.obj build\temp.win-amd64-cpython-37\Release\descriptors.obj build\temp.win-amd64-cpython-37\Release\int_convertors.obj build\temp.win-amd64-cpython-37\Release\objmap.obj build\temp.win-amd64-cpython-37\Release\qtlib.obj build\temp.win-amd64-cpython-37\Release\sip_array.obj build\temp.win-amd64-cpython-37\Release\siplib.obj build\temp.win-amd64-cpython-37\Release\threads.obj build\temp.win-amd64-cpython-37\Release\voidptr.obj /OUT:build\lib.win-amd64-cpython-37\PyQt5\sip.cp37-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-37\Release\sip.cp37-win_amd64.lib
      siplib.obj : warning LNK4197: export 'PyInit_sip' specified multiple times; using first specification
         Creating library build\temp.win-amd64-cpython-37\Release\sip.cp37-win_amd64.lib and object build\temp.win-amd64-cpython-37\Release\sip.cp37-win_amd64.exp
      Generating code
      Finished generating code
      LINK : fatal error LNK1158: cannot run 'rc.exe'
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit code 1158
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for PyQt5-sip
Failed to build PyQt5-sip
ERROR: Could not build wheels for PyQt5-sip, which is required to install pyproject.toml-based projects
WARNING: There was an error checking the latest version of pip.

Hello!
Did you correctly install the C++ build tools?
see https://github.com/limpkin/cdm324_fft/tree/main/scripts

can you remove the VS2015?

One step closer, it build, but running _start.py does not work :).
PS C:\dopplerradar\scripts> python _start.py
Traceback (most recent call last):
  File "_start.py", line 244, in <module>
    app = CDM324MonitorApp()
  File "_start.py", line 237, in __init__
    self.com_port_monitor()
  File "_start.py", line 134, in com_port_monitor
    (self.connected_to_cdm, cdm_version) = self.device.connect(com_port)
  File "C:\dopplerradar\scripts\cdm324_device.py", line 29, in connect
    self.serial = serial.Serial(self.com_port, 1000000)
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python37\lib\site-packages\serial\serialwin32.py", line 31, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python37\lib\site-packages\serial\serialutil.py", line 240, in __init__
    self.open()
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python37\lib\site-packages\serial\serialwin32.py", line 78, in open
    self._reconfigure_port()
  File "C:\Users\Philippe\AppData\Local\Programs\Python\Python37\lib\site-packages\serial\serialwin32.py", line 222, in _reconfigure_port
    'Original message: {!r}'.format(ctypes.WinError()))
serial.serialutil.SerialException: Cannot configure port, something went wrong. Original message: OSError(22, 'De parameter is onjuist.', None, 87)
PS C:\dopplerradar\scripts>

The device is assigned COM2 in the device manager. I have the complete kit, with USB isolator and expansion board.

how many COM ports do you have in the device manager? can you share a screen capture?

And the CH340 COM2 is from your board

that's a fun one... you're correctly selecting COM2 in the GUI I assume?
Maybe update your python install? :)

What version are you running 3.12?
To start the scripts, I tried running all 3 py files. Anything I missed?
Is "python _start.py" the way to go?
PS C:\dopplerradar\scripts> python stream_visualizer.py
Traceback (most recent call last):
  File "stream_visualizer.py", line 269, in <module>
    s = serial.Serial(port)
AttributeError: module 'serial' has no attribute 'Serial'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "stream_visualizer.py", line 272, in <module>
    except (OSError, serial.SerialException):
AttributeError: module 'serial' has no attribute 'SerialException'
PS C:\dopplerradar\scripts> python cdm324_device.py
PS C:\dopplerradar\scripts> python _start.py
Traceback (most recent call last):
  File "_start.py", line 31, in scan_available_com_ports
    s = serial.Serial(port)
AttributeError: module 'serial' has no attribute 'Serial'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "_start.py", line 244, in <module>
    app = CDM324MonitorApp()
  File "_start.py", line 237, in __init__
    self.com_port_monitor()
  File "_start.py", line 118, in com_port_monitor
    self.current_com_ports = self.scan_available_com_ports()
  File "_start.py", line 34, in scan_available_com_ports
    except (OSError, serial.SerialException):
AttributeError: module 'serial' has no attribute 'SerialException'

3.12 should work indeed.
python _start.py is indeed the way to go
I think you have the wrong serial module installed.
can you do:

pip uninstall serial
pip install pyserial

I'm uninstalling all old Python versions and will re-install 3.12 and hopefully that will work. I had 5 different versions installed over the years.

still struggling. Could the reason by that I cannot select 1 000 000 baud from the dropdown? Cannot go above 128 000 baud.

With Python 3.12 I'm unable to install/compile chaco. With Python 3.11 chaco installs/compiles fine.
After installing 311 I tried to "python _start.py"
Had to "pip install scipy" and "pip install keyboard"
Then I get again this error:

PS C:\dopplerradar\scripts> python _start.py
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\_start.py", line 244, in <module>
    app = CDM324MonitorApp()
          ^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 237, in __init__
    self.com_port_monitor()
  File "C:\dopplerradar\scripts\_start.py", line 134, in com_port_monitor
    (self.connected_to_cdm, cdm_version) = self.device.connect(com_port)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\cdm324_device.py", line 29, in connect
    self.serial = serial.Serial(self.com_port, 1000000)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 33, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\serial\serialutil.py", line 244, in __init__
    self.open()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 80, in open
    self._reconfigure_port()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 222, in _reconfigure_port
    raise SerialException(
serial.serialutil.SerialException: Cannot configure port, something went wrong. Original message: OSError(22, 'De parameter is onjuist.', None, 87)
PS C:\dopplerradar\scripts>

did you

pip uninstall pyserial
pip install serial
?

yes
PS C:\dopplerradar\scripts> pip uninstall pyserial
WARNING: Skipping pyserial as it is not installed.
PS C:\dopplerradar\scripts> pip uninstall serial
WARNING: Skipping serial as it is not installed.
PS C:\dopplerradar\scripts> pip install serial
Collecting serial
  Using cached serial-0.0.97-py2.py3-none-any.whl (40 kB)
Requirement already satisfied: future>=0.17.1 in c:\python311\lib\site-packages (from serial) (0.18.3)
Requirement already satisfied: pyyaml>=3.13 in c:\python311\lib\site-packages (from serial) (6.0.1)
Requirement already satisfied: iso8601>=0.1.12 in c:\python311\lib\site-packages (from serial) (2.1.0)
Installing collected packages: serial
Successfully installed serial-0.0.97
PS C:\dopplerradar\scripts> python _start.py
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\_start.py", line 31, in scan_available_com_ports
    s = serial.Serial(port)
        ^^^^^^^^^^^^^
AttributeError: module 'serial' has no attribute 'Serial'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\dopplerradar\scripts\_start.py", line 244, in <module>
    app = CDM324MonitorApp()
          ^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 237, in __init__
    self.com_port_monitor()
  File "C:\dopplerradar\scripts\_start.py", line 118, in com_port_monitor
    self.current_com_ports = self.scan_available_com_ports()
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 34, in scan_available_com_ports
    except (OSError, serial.SerialException):
                     ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'serial' has no attribute 'SerialException'
PS C:\dopplerradar\scripts>

sorry i made a mistake.... could you instead:

pip uninstall serial
pip install pyserial

I already tried that :), I saw an earlier post of you was edited.
Do you think it has to do with the 1 Mbit / 1 000 000 baud speed?
Could you make the script we can do something like: "python _start.py -p COM2" ? Forcing the script to use the right/specific COM port.
PS C:\dopplerradar\scripts> pip uninstall serial
Found existing installation: serial 0.0.97
Uninstalling serial-0.0.97:
  Would remove:
    c:\python311\lib\site-packages\serial-0.0.97.dist-info\*
    c:\python311\lib\site-packages\serial\*
Proceed (Y/n)? y
  Successfully uninstalled serial-0.0.97
PS C:\dopplerradar\scripts> pip uninstall pyserial
WARNING: Skipping pyserial as it is not installed.
PS C:\dopplerradar\scripts> pip install pyserial
Collecting pyserial
  Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
Installing collected packages: pyserial
Successfully installed pyserial-3.5
PS C:\dopplerradar\scripts> python _start.py
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\_start.py", line 244, in <module>
    app = CDM324MonitorApp()
          ^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 237, in __init__
    self.com_port_monitor()
  File "C:\dopplerradar\scripts\_start.py", line 134, in com_port_monitor
    (self.connected_to_cdm, cdm_version) = self.device.connect(com_port)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\cdm324_device.py", line 29, in connect
    self.serial = serial.Serial(self.com_port, 1000000)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 33, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\serial\serialutil.py", line 244, in __init__
    self.open()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 80, in open
    self._reconfigure_port()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 222, in _reconfigure_port
    raise SerialException(
serial.serialutil.SerialException: Cannot configure port, something went wrong. Original message: OSError(22, 'De parameter is onjuist.', None, 87)
PS C:\dopplerradar\scripts>

well it does look like the program tries to open the port... it looks like something else now.
maybe try these suggestions? https://stackoverflow.com/questions/20879939/pyserial-error-cannot-open-port

This simple code does not return any errors

import serial

serialPort = serial.Serial(port = "COM2", baudrate=1000000,
                           bytesize=8, timeout=2, stopbits=serial.STOPBITS_ONE)

serialPort.close()

Testing the code on the other COM ports in my system does return some errors now and then.

PS C:\dopplerradar\scripts> py comtestcloseCOM2.py
PS C:\dopplerradar\scripts> py comtestcloseCOM3.py
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\comtestcloseCOM3.py", line 3, in <module>
    serialPort = serial.Serial(port = "COM3", baudrate=1000000,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 33, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\serial\serialutil.py", line 244, in __init__
    self.open()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 80, in open
    self._reconfigure_port()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 222, in _reconfigure_port
    raise SerialException(
serial.serialutil.SerialException: Cannot configure port, something went wrong. Original message: OSError(22, 'De parameter is onjuist.', None, 87)
PS C:\dopplerradar\scripts> py comtestcloseCOM4.py
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\comtestcloseCOM4.py", line 3, in <module>
    serialPort = serial.Serial(port = "COM4", baudrate=1000000,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 33, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\serial\serialutil.py", line 244, in __init__
    self.open()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 64, in open
    raise SerialException("could not open port {!r}: {!r}".format(self.portstr, ctypes.WinError()))
serial.serialutil.SerialException: could not open port 'COM4': OSError(22, 'De time-outperiode van de semafoor is verlopen.', None, 121)
PS C:\dopplerradar\scripts> py comtestcloseCOM5.py
PS C:\dopplerradar\scripts>

I then tried to change line 25 in _start.py to:
		ports = ['COM%s' % (i + 1) for i in range(7,256)]
So the first COM port it can find is COM8, changed the CH340 from COM2 to COM8 and voila, tiny step further:

Now we need to know why it cannot open the com port. Is there a way to force the script to use COM8 and not do any com port scan?

I'm afraid not, as your error happens when python tries to talk with COM2: https://github.com/limpkin/cdm324_fft/blob/main/scripts/_start.py#L133
at this line, could you add a print(com_port) ? it should report COM2.
worst case, could you add here: https://github.com/limpkin/cdm324_fft/blob/main/scripts/_start.py#L133 the following line: return ["COM2"]

Because of line 25 https://github.com/limpkin/cdm324_fft/blob/main/scripts/_start.py#L25
ports = ['COM%s' % (i + 1) for i in range(2,256)]
COM3 is the earliest com port that can be found, so that's why I changed the com port in the W10 device manager from com2 to com 8 and changed the detection range to (5,256).
Now it finds COM8 (this is correct, the cdm324 device) and that gives me this:

no button needs to be pressed indeed.
just to make sure the serial link is working, could you try the reflashing procedure ( https://github.com/limpkin/cdm324_fft#reflashing-the-device ) with https://github.com/limpkin/cdm324_fft/releases/download/v1.00/cdm324_1000000.bin ?

I can also make the debug tool work when hardcoding the com port
		# Launch debug tool
		self.com_port = "COM8"
		stream_vis_start(self.com_port)

But I need to reset the device to re-run the script after closing the window. Probably because the com_port isn't closed properly with this hardcoded COM8.
PS C:\dopplerradar\scripts> python _start.py
COM8
2023-11-19 21:10:17.086388: skipping one frame
2023-11-19 21:10:26.237943: skipping one frame
2023-11-19 21:10:26.407199: skipping one frame
2023-11-19 21:10:26.718888: skipping one frame
2023-11-19 21:10:28.005532: skipping one frame
2023-11-19 21:10:28.637009: skipping one frame
2023-11-19 21:10:30.348279: skipping one frame
2023-11-19 21:10:31.979112: skipping one frame
2023-11-19 21:10:32.059897: skipping one frame
2023-11-19 21:10:34.920247: skipping one frame
2023-11-19 21:10:37.286116: skipping one frame
2023-11-19 21:10:38.058548: skipping one frame
2023-11-19 21:10:38.266316: skipping one frame
2023-11-19 21:10:38.742160: skipping one frame
2023-11-19 21:10:38.973134: skipping one frame
2023-11-19 21:10:39.115380: skipping one frame
2023-11-19 21:10:40.830078: skipping one frame
2023-11-19 21:10:41.542726: skipping one frame
2023-11-19 21:10:41.805361: skipping one frame
COM8
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python311\Lib\tkinter\__init__.py", line 1948, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\tkinter\__init__.py", line 861, in callit
    func(*args)
  File "C:\dopplerradar\scripts\_start.py", line 137, in com_port_monitor
    (self.connected_to_cdm, cdm_version) = self.device.connect(com_port)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\cdm324_device.py", line 44, in connect
    self.cdm324_version = self.cdm324_version[i:-2].decode('utf-8')
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in position 8: invalid start byte
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\_start.py", line 249, in <module>
    app.mainloop()
  File "C:\Python311\Lib\tkinter\__init__.py", line 1485, in mainloop
    self.tk.mainloop(n)
  File "C:\Python311\Lib\tkinter\__init__.py", line 1943, in __call__
    def __call__(self, *args):

KeyboardInterrupt
PS C:\dopplerradar\scripts> python _start.py
COM8
Traceback (most recent call last):
  File "C:\dopplerradar\scripts\_start.py", line 248, in <module>
    app = CDM324MonitorApp()
          ^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 241, in __init__
    self.com_port_monitor()
  File "C:\dopplerradar\scripts\_start.py", line 137, in com_port_monitor
    (self.connected_to_cdm, cdm_version) = self.device.connect(com_port)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\cdm324_device.py", line 44, in connect
    self.cdm324_version = self.cdm324_version[i:-2].decode('utf-8')
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 1-2: invalid continuation byte
PS C:\dopplerradar\scripts> python _start.py
COM8

just to make sure we're on the same page:
if you modify https://github.com/limpkin/cdm324_fft/blob/main/scripts/_start.py#L133 to add the following line: return ["COM8"] you do get the "No COM ports detected" message?

Yes, see video :).
https://github.com/limpkin/cdm324_fft/assets/8402106/3962455d-9ed1-4066-b95d-8ccc725d4c0f

something odd is happening... the "no com ports detected!" only gets displayed if scan_available_com_ports return no com ports ( scan_available_com_ports ).
However, you modified scan_available_com_ports to return one com port....
maybe change self.current_com_ports = self.scan_available_com_ports()to self.current_com_ports = ["COM8"] ?

Then it doesn't work anymore, sadly :)
PS C:\dopplerradar\scripts> python _start.py
COM8
COM8
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python311\Lib\tkinter\__init__.py", line 1948, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\tkinter\__init__.py", line 861, in callit
    func(*args)
  File "C:\dopplerradar\scripts\_start.py", line 138, in com_port_monitor
    (self.connected_to_cdm, cdm_version) = self.device.connect(com_port)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\cdm324_device.py", line 29, in connect
    self.serial = serial.Serial(self.com_port, 1000000)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 33, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\serial\serialutil.py", line 244, in __init__
    self.open()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 64, in open
    raise SerialException("could not open port {!r}: {!r}".format(self.portstr, ctypes.WinError()))
serial.serialutil.SerialException: could not open port 'COM8': PermissionError(13, 'Toegang geweigerd.', None, 5)
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python311\Lib\tkinter\__init__.py", line 1948, in __call__
    return self.func(*args)
           ^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 231, in <lambda>
    self.update_fw_button = tk.Button(self, text="Start Debug Tool", font=tkfont.Font(family='Helvetica', size=9), width="18", command=lambda:[self.start_debug_tool()])
                                                                                ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dopplerradar\scripts\_start.py", line 100, in start_debug_tool
    stream_vis_start(self.com_port)
  File "C:\dopplerradar\scripts\stream_visualizer.py", line 253, in stream_vis_start
    platform_serial = serial.Serial(com_port, 1000000)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 33, in __init__
    super(Serial, self).__init__(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\serial\serialutil.py", line 244, in __init__
    self.open()
  File "C:\Python311\Lib\site-packages\serial\serialwin32.py", line 64, in open
    raise SerialException("could not open port {!r}: {!r}".format(self.portstr, ctypes.WinError()))
serial.serialutil.SerialException: could not open port '': FileNotFoundError(2, 'Het systeem kan het opgegeven pad niet vinden.', None, 3)
PS C:\dopplerradar\scripts>

"access denied".... any reason why that would be? any software running in the background?

maybe the script is opening it and not closing it somewhere and then trying to open but gets an error trying to open an already open port? I'll see if I can run it on my work laptop, but it's pretty closed by IT. So I don't think I can install Python on it.

any other tool running on your system maybe?

I got the same on my worklaptop, only the com port of the module, no other com ports on that machine. Script without any modifications opens, but gives red combo and no com ports detected. Clicking the debug tools gives some similar permission error.

@fluppie ?

@fluppie any update's i have been able to come to the same point, meaning: i am able to open streamvisualizer, nevertheless the debug tool trows errors.
If i use Arduino Serial Monitor: on my com port i am able to see data passing.
Also have not been able to use my esp8266 or esp8266 esphome to read the uart to be able to display this on my 7 segment display. the default baud_rate 1000000 is also causing me challenges. python is definity not my strong suit hence i am severly blocked.
Using windows 11 btw, hence keep in mind you would require to remove some blockings like downloaded files


Missing Audios in Batch Generation

When I try to generate audio using batch generation on long text, it drops some words/sentences in audio. like they are never there.
the code I am using is given below
def sentense_tokenize(text):
    text = text.replace('\n',' ')
    sentences = nltk.sent_tokenize(text)
    chunks = ['']
    max_token = 300
    token_counter = 0
    for sentence in sentences:
        current_tokens = len(nltk.Text(sentence))
        if token_counter + current_tokens <= max_token:
            token_counter = token_counter + current_tokens
            chunks[-1] = chunks[-1] + " " + sentence
        else:
            chunks.append(sentence)
            token_counter = current_tokens
    return chunks

processor = AutoProcessor.from_pretrained("suno/bark-small")
model = BarkModel.from_pretrained("suno/bark-small").to(CONSTS.DEVICE)
model = self.model.to_bettertransformer()

sentences = sentense_tokenize(text)

inputs = processor(sentences, voice_preset=voice_preset).to(CONSTS.DEVICE)
pieces = model.generate(**inputs, pad_token_id=10000, return_output_lengths=True)
audio_array = np.concatenate([i.cpu().numpy()[:l] for i,l in zip(pieces[0],pieces[1])])

write_wav(dst, rate, audio_array)

this is the best generated audio so far: https://1drv.ms/b/s!AutU-ly4nMgqZtJKr6btZFxgsVc?e=Sm5Yvy
the text used was
AlphaZero is a computer program developed by artificial intelligence research company DeepMind to master the games of chess, shogi and go. This algorithm uses an approach similar to AlphaGo Zero. On December 5, 2017, the DeepMind team released a preprint paper introducing AlphaZero, which within 24 hours of training achieved a superhuman level of play in these three games by defeating world-champion programs Stockfish, Elmo, and the three-day version of AlphaGo Zero. In each case it made use of custom tensor processing units (TPUs) that the Google programs were optimized to use. AlphaZero was trained solely via self-play using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks, all in parallel, with no access to opening books or endgame tables. After four hours of training, DeepMind estimated AlphaZero was playing chess at a higher Elo rating than Stockfish 8; after nine hours of training, the algorithm defeated Stockfish 8 in a time-controlled 100-game tournament (28 wins, 0 losses, and 72 draws). The trained algorithm played on a single machine with four TPUs. DeepMind's paper on AlphaZero was published in the journal Science on 7 December 2018; however, the AlphaZero program itself has not been made available to the public. In 2019, DeepMind published a new paper detailing MuZero, a new algorithm able to generalise AlphaZero's work, playing both Atari and board games without knowledge of the rules or representations of the game.


Ignore lockfiles while building Terraform docs

what

Ignore lockfiles while building Terraform docs

why

By default, terraform-docs takes into account any version pinning present in the automatically generated .terraform.lock.hcl. However, Cloud Posse modules do not use the lock files for version pinning, which means that different people can get different results when creating the documentation, which causes problems. By ignoring the lock files, we make the documentation easier to reproduce.

as per https://terraform-docs.io/user-guide/configuration/settings/#lockfile

@Nuru needs cloudposse/engineering, i'm not yet part of that elite team ...


mediathekview: 13.8.0 -> 13.9.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/mediathekview/versions.
meta.description for mediathekview is: "Offers access to the Mediathek of different tv stations (ARD, ZDF, Arte, etc.)"
meta.homepage for mediathekview is: "https://mediathekview.de/"
meta.changelog for mediathekview is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
Warning: no invocation of /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1/bin/MediathekView_ipv4 had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1/bin/mediathek had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1/bin/MediathekView had a zero exit code or showed the expected version
0 of 3 passed binary check by having a zero exit code.
0 of 3 passed binary check by having the new version present in output.
found 13.8.1 with grep in /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1
found 13.8.1 in filename of file in /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
docbookrx
mediathekview

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A mediathekview https://github.com/r-ryantm/nixpkgs/archive/fa77590109cbbdf2c7df497dd9d1437a20cbab07.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1
ls -la /nix/store/87xjpll2vyxfs22xbrgfyzbw1nnrx0a5-mediathekview-13.8.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

mediathekview

Maintainer pings
cc @moredread for testing.

@ofborg eval

Likely needs a Backport to 21.05 as log4j was updated https://mediathekview.de/changelog/13-8-1/

We need to wait for https://github.com/NixOS/nixpkgs/pull/140364

Are we still waiting for https://github.com/NixOS/nixpkgs/pull/140364 or can this be merged?

I haven't checked recently, but I don't think it works with openjdk17, so we still have to wait

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/mediathekview/versions.
meta.description for mediathekview is: "Offers access to the Mediathek of different tv stations (ARD, ZDF, Arte, etc.)"
meta.homepage for mediathekview is: "https://mediathekview.de/"
meta.changelog for mediathekview is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 13.9.1 with grep in /nix/store/zhppcvwr1454rv2cw0gs0xy9gpqc5l31-mediathekview-13.9.1
found 13.9.1 in filename of file in /nix/store/zhppcvwr1454rv2cw0gs0xy9gpqc5l31-mediathekview-13.9.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
mediathekview

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/zhppcvwr1454rv2cw0gs0xy9gpqc5l31-mediathekview-13.9.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A mediathekview https://github.com/r-ryantm/nixpkgs/archive/508d305d4580364d563724738855e29d048e5c3b.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/zhppcvwr1454rv2cw0gs0xy9gpqc5l31-mediathekview-13.9.1
ls -la /nix/store/zhppcvwr1454rv2cw0gs0xy9gpqc5l31-mediathekview-13.9.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

mediathekview

Maintainer pings
cc @Moredread for testing.

Last time I checked (about 3 weeks ago) we still didn't have a Java version packaged that works for newer Mediathekview releases

FYI, https://github.com/NixOS/nixpkgs/pull/140364 has been merged.

See https://github.com/NixOS/nixpkgs/pull/194125

This PR can be closed :)


taxes2022

Docs Build status updates of commit 675edd4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

dynamics-gp/payroll/media/BC2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/FED2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/MB2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/NB2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/NL2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/NT2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/NU2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/ON2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/SK2022.JPG
:white_check_mark:Succeeded
View

dynamics-gp/payroll/media/YT2022.JPG
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Why Rasa installation have so much versions for dependencies

It is too much frustrating whenever i am running any RASA Project it always give at least 1 or 2 version errors.

Hi @sudhir512kj are you installing rasa into a fresh (i.e. no packages installed) environment? If so can you please post the error messages you are seeing?

Please reopen if this still an issue. Questions like this are best asked on the Rasa Forum in the future


Validator Client: Track skipped slots

Is your feature request related to a problem? Please describe.
At validator side, sometimes we skip slots (mainly due to the I/O lag issue) without noticing it
Describe the solution you'd like

Track it metrics
Add some warn logs

What do mean with validator client skipping a slot?

@dapplion in runAtMostEvery() of validator clock, it's designed that if a task spans more than a slot, next slot is skipped and it really happened on mainnet

Got it! There is an argument to be made if skipping the slot is the previous runs for too long is actually a good idea. Would be good to have metrics to understand how often that happens and for how much does it overrun the slot

related discussion https://github.com/ChainSafe/lodestar/issues/5314

Closed in favour of #5314


Unable to retrieve child resources

Storage Explorer Version: 1.14.2
Build Number: 20200715.2
Platform/OS: Windows 10
Architecture: x64
Regression From:  It was working earlier but I don't know in which version
Bug Description
I cannot use local emulator. When I'm trying to expand Tables or Blob Containers it gives me next error:

Unable to retrieve child resources.
Details:
Server encountered an internal error. Please try again after some time.
RequestId:ab1ce6c8-3252-4e48-99f6-ac72ac9f1b4d
Time:2020-07-21T08:29:57.9614500Z

I believe it's also a reason why I can't run my azure functions locally. It's very annoying.
I double checked that Azure Storage Emulator is initialized and running.

@koljada, are you using the .Net storage emulator or Azurite? which version are you using? From the error details, it says that server encountered an internal error so it is possible that the problem is in your local emulator's configuration.

I'm using the latest version of Azure Storage Emulator. And it seems like the issue was related to emulator: for some reason schema in LocalDB was no created. After I reinstalled SqlServer it's working. If it's not related to Azure Storage Explorer then this issue can be closed. The error message is not very helpful anyway.

Thanks!


VSL-43: Add Events to VESSEL Contracts

Adds events to DAOTreasury.cdc
Adds events to MyMultiSig.cdc
Adds events to TreasuryActions.cdc

NOTE:
Some of these events are a little redundant, because I wasn't 100% sure which events would be the most useful.
Some of these events also might not contain all the context the front-end needs to query them properly. We will need to play around with the Events API to determine whether we need to enrich any of these events further to serve the FE needs.

Closing this PR because these changes will be integrated as part of the Quantstamp audit response


storybook/no-uninstalled-addons fails with locally references file addons

Describe the bug
The latest version 0.6.0 introduces storybook/no-uninstalled-addons which will fail if you use file path addons.

[eslint] storybook/main.js
[eslint]   91:5  error
The ./my-addon/preset.cjs is not installed. Did you forget to install it?  storybook/no-uninstalled-addons

To Reproduce
Steps to reproduce the behavior:

Add an addon to your main.js array using a relative file path, e.g. addons: ['./my-addon/preset.cjs']
Run eslint
See error

Expected behavior
Local file references should be ignored since there is no need to install them.

Hey @alopix thanks for surfacing this! We'll take that into account and come up with a fix. cc @andrelas1

Hey @alopix I just released v0.6.1 containing a fix for this. Thank you so much for helping make this happen!

Thank you for using this plugin and more so for creating this issue!


Some problems to install

I try to install this oj in my server, but I have some unexpected errors.
When I went to step 8, after I add * * * * * php /path-to-noj/artisan schedule:run in the file edited, I run php artisan queue:work --queue=noj,codeforces,contesthunter,poj,vijos,pta,uva. It shows error.
   Predis\Connection\ConnectionException  : Connection refused [tcp://<IP_ADDRESS>:6379]

  at /var/www/html/oj/vendor/predis/predis/src/Connection/AbstractConnection.php:155
    151|      */
    152|     protected function onConnectionError($message, $code = null)
    153|     {
    154|         CommunicationException::handle(
  > 155|             new ConnectionException($this, static::createExceptionMessage($message), $code)
    156|         );
    157|     }
    158|
    159|     /**

  Exception trace:

  1   Predis\Connection\AbstractConnection::onConnectionError("Connection refused")
      /var/www/html/oj/vendor/predis/predis/src/Connection/StreamConnection.php:128

  2   Predis\Connection\StreamConnection::createStreamSocket(Object(Predis\Connection\Parameters), "tcp://<IP_ADDRESS>:6379")
      /var/www/html/oj/vendor/predis/predis/src/Connection/StreamConnection.php:178

  Please use the argument -v to see more details.

I wanna know what problem may I have. Thanks!
I think maybe I have problems when I edit .env. I just changed the database user and password. Should I change others?

Hi, it looks like you haven't install Redis, NOJ now requires Redis support. See .env.example , our cache system uses them.

Updated Dependencies List

I try again this morning but failed again.
This time when I I run php artisan queue:work --queue=noj,codeforces,contesthunter,poj,vijos,pta,uva, it shows nothing and it got stuck. Do you know what may problems are? Or where can I use the logs. Thank you!

artisan queue is a daemon, it listens the submission and push it to the remote judge, supervisor is recommended for lasting monitoring.

Please reopen the issue beforehand or we would not be able to receive notifications.


Write Unit tests

Write some unit tests for the extension so no matter if I add features, the extension should still act as expected.

b2cdfd6 has the groundwork for writing unit tests

Here's a list of things that will need tests:

[ ] Function call with no params
[x] Function call with params
[ ] Function call with more arguments than actual params
[ ] Function call with rest parameters
[ ] Function call that spreads multiple lines
[ ] Function call inside another function call
[ ] Function call with params but no definition

Finished in 62b2432


Reuse trees

Use document registry to get sourceFiles in SyntaxTreeCache to share trees between semantic and syntactic operations
@vladima can you take a look

:+1:


Clean up user data on logout

Description
MA-2515
The PR does 4 main things:

Nuke user data except videos and downloads database.
Runs migrations of the database to hash the usernames to their SHA1 counterparts.
Runs migrations to change the legacy directory structure of how we store videos and their subtitles.
Silently fetch and cache course structure of the courses we have downloaded videos of.

Notes

To properly test this PR, first make a build out of current master and then overwrite the build from this PR to view the migrations in action.
Results of migrations are more visible if testing is being done on the emulator (cuz we can see whatever files we want to see from Android Device Monitor).

Testing

[x] Manual tests this PR should pass

Following tests should pass on this PR:

Make a build from master, download some videos and then make a build from this branch. The directory that stores videos should be updated i.e. previously the videos were stored like this sdcard/org.edx.mobile/staff/(SHA1_hash_of_video_name), now it should appear as sdcard/org.edx.mobile/videos/(SHA1_of_username)/(SHA1_hash_of_video_name).
srtFiles has been renamed to subtitles and has been moved inside videos folder.
The username field in the database should now have SHA1 representation of the username.
After logout, in the app's directory only databases/download.db should be there and in the app's external directory only the videos folder should remain.

[x] Database migrations are backwards-compatible

Database has been upgraded from version 5 to version 6.
Reviewers
If you've been tagged for review, please check your corresponding box once you've given the :+1:.

[x] Code review: @1zaman
[x] Code review: @BenjiLee
[x] Code review: @mdinino
[ ] Code review: @bguertin

Hmmm, If we are migrating the location of all the video files, it might be a good time to move them to the correct folder (our app's data folder) instead of an arbitrary place on the SD card, so that they are properly deleted when uninstalling the edX app.
Not necessary for this PR of course, just thinking we might want to do one migration and not two :confused:

Hmmm, If we are migrating the location of all the video files, it might be a good time to move them to the correct folder (our app's data folder) instead of an arbitrary place on the SD card, so that they are properly deleted when uninstalling the edX app.

Well we could certainly do that, but uninstalling or clearing the data, cleans up the app's whole directory on SD Card so every single video and subtitles are properly deleted.

uninstalling or clearing the data, cleans up the app's whole directory on SD Card so every single video and subtitles are properly deleted.

Hm, right, my bad. The actual issue, now I'm remember, is that we require the storage permission because we are technically modifying files outside our apps files directory. If we migrated all our users to use the "files" directory, we can remove the requirement for having the "files" permission.

If we migrated all our users to use the "files" directory, we can remove the requirement for having the storage permission.

Wouldn't that migration also require the write and read permission. Plus, in our future plans we might allow user to store a video on a custom location, bam! write permission again 😉

Done with pass. I had minor suggestions, but other than those it looks great.

Plus, in our future plans we might allow user to store a video on a custom location, bam! write permission again

But, we could at least save them in the default location without asking for permission. Anyway, forget I said anything, we can just move them again if we have to :package:
I agree about the null/nonnull annotations, but I wouldn't worry about annotating all the existing methods that weren't really affected by this PR, unless it is straightforward (no need to increase scope for that).

@mdinino most of the refactoring reviews that you gave are related to this commit 763026f, which isn't actually what the PR is addressing at core, so, counting them as out of scope.

I think you need to rebase before tests can run on travis.
👍

👍

@@             master       #797   diff @@
==========================================
  Files           485        487     +2
  Lines         19839      19874    +35
  Methods           0          0
  Messages          0          0
  Branches       2821       2843    +22
==========================================
- Hits           3710       3708     -2
- Misses        15699      15736    +37
  Partials        430        430

Review #797 on Codecov

Powered by Codecov. Last update 29ba087...8f88d85

All review comments have been addressed and branch has been rebased on master. Just need you review @1zaman before I merge this PR.

@1zaman do notify by an ending comment when ur pass finishes, so that I can make the said changes and make this PR ready for another/final pass.

OK, my review is completed.

@1zaman I've addressed your review comments.
Ready for another review @1zaman, @mdinino, @bguertin.
These are the commits that need review: 07ae69d & c6b3478.

Ok then, @1zaman I've fixed all the review points u mentioned or I had left previously. Hopefully we'll get them approved this time 😉
The commits to review are as follows:

Missed review points 5065d40
Final review changes 12cf6fe
Separate function for manual logout on button tap f3afb2b
Singleton issues fix for CacheManager 6ad5186 (This was one pesky issue I had real hard time finding, causing everything to break upon relogin)

@1zaman changes done, hopefully the last time 🙏 🙏 😝

👍 All my comments seem to be addressed, but I just made another one 😜 If you would address that as well, then I'll finally approve this.

@1zaman ok, np. If thats the last thing, should i just do that and squash the commits ?

Sure, you can squash the commits now.


html2pdf failed with too many pages

Hi @eKoopmans,
Firstly, I want to thank you for writing this great file. It saves me a lot of time dealing with the quality of content and layout. Recently, I was playing around with this file and vue js and found an issue:
https://jsfiddle.net/rover1234/woa5ucyx/
In two tests in the link, I created a simple for loop which shows 5 texts on the screen, and it is fine to output to a pdf file with 5 pages which show these texts since I apply your page break. However, in the second test, it failed to show the content of more than 20 texts which are all blank pages even though the page break still works fine.
I am not sure about this case. Is it because the picture generated from canvas is too large? and how could I find a way to get away from it (like creates multiple elements to generate different pdf content and combine everything into one file)?
Thanks

It seems to me that everytime I increase the number of the height of the margin, I could get more pdf pages with the text. On the other hand, when I decrease the height, there are blank pages with the same amount of contexts I generated. Increasing the height is not a solution for me since if there are more pages generated, it will still fail at some point.
https://jsfiddle.net/rover1234/woa5ucyx/

Hi @CSENoni, sorry I didn't respond sooner. Yes, I see the problem. I've reproduced it here using only html2canvas, which html2pdf relies on.
The problem is in fact deeper than that - it's a problem directly in how canvases work. Canvases have a maximum height/width, and once you exceed that the canvas becomes unusable! That's what's happening here.
I wasn't aware of that problem, so thank you for bringing it up. It's a serious limitation and I don't have any immediate solution. As you mentioned, changing the margins and page size will change how many pages you can get (also changing the resolution/scale), but that's just because it changes the size of the underlying canvas.

I ran this problem as well and @MaNuGitH's workaround didn't work for me. I wrote a function that might be helpful for someone. Lucky for me the new branch new-api exposes a lot more functions. I notice that I can output my html as an image created from being drawn onto a canvas which is pretty much the last step before turning it into a PDF. So, I decided to create my own instance of jsPDF and only pass in a page at a time to html2pdf. Then I add the image created by html2pdf to my instance of jsPDF.  I hope this is helpful.
import html2pdf from 'html2pdf.js';
import jsPDF from 'jspdf';

const exportHTMLToPDF = async (pages, outputType='blob') => {
  const opt = {
    margin:       [0,0],
    filename:     'myfile.pdf',
    image:        { type: 'jpeg', quality: 0.98 },
    html2canvas:  { dpi: 192, letterRendering: true },
    jsPDF:        { unit: 'in', format: 'letter', orientation: 'portrait' }
  };
  const doc = new jsPDF(opt.jsPDF);
  const pageSize = jsPDF.getPageSize(opt.jsPDF);
  for(let i = 0; i < pages.length; i++){
    const page = pages[i];
    const pageImage = await html2pdf().from(page).set(opt).outputImg();
    if(i != 0) {
      doc.addPage();
    }
    doc.addImage(pageImage.src, 'jpeg', opt.margin[0], opt.margin[1], pageSize.width, pageSize.height);
  }
  // This can be whatever output you want. I prefer blob.
  const pdf = doc.output(outputType);
  return pdf;
}

You can also write it using recursion if you can't use async-await.

Hi @GGross5213 ,
I am still learning JS and I am running into the same problem and I was wondering how can I implement your workaround in my case.
Currently, I have a dynamic content and I am using @eKoopmans plugin to generate a PDF report for it.
So this is my current function:
$("#download").click(function(){
filename = this.value + '.pdf'
var element = document.getElementById('report');

html2pdf(element, {
		margin:       0.25,
		filename:     filename,
		image:        { type: 'jpeg', quality: 0.98 },
		html2canvas:  { dpi: 96, letterRendering: true },
		jsPDF:        { unit: 'in', format: 'letter', orientation: 'portrait' }
	  });

});
and I would like to know where should I put my "element" variable into your function in order to have it working?
I would much appreciate if you could give me hand with it.

@nevergiveup777 Does your element have the html2pdf page-break classes in it?
The way I did it was by breaking my element up into the pages that I wanted, and then passing only one page at a time into html2pdf. Then I convert that page into an image using html2pdf and add it to an instance of jsPDF that I control (what html2pdf does underneath the hood).  I would also recommend using the new-api branch of html2pdf has it exposes an outputImg function. I guess you could implement it using the callback branch, but it will be much easier with the new-api. I hope this helps.

@GGross5213  in your example I got error jsPDF.getPageSize is not a function. and pages arg is the html element that we are going to export, right? I have very large tables and images that i needed to convert into pdf but as you know if goes blank.

@waqar-imtiaz The pages arg is a list containing strings of HTML. In my case each element in the list is a single page that is ended with the

. Can you post the code where you are calling jsPDF.getPageSize()? Also what format are your images in? I know I had problems with images showing up in the PDF. If they were coming from an external url such as AWS S3, they wouldn't render. I think it has something to do with CORS. I couldn't figure out how to allow CORS/set up a proxy. So, I just converted them into base64 data URI's and included those.

My HTML is as follow:
`

True Monitor Report
{{monitorName}}

PDF Created Date: {{currentDate}}
Last Run:{{getDateTime(monitorDetails.lastRun)}}
Search Sources:

Facebook,
 Twitter,
 Google,
 YouTube,
 Instagram,
 Documents

Search Terms:{{term}},
Result Counts:{{totalMonitorResultsCount.counts}}

{{monitorName}}

`

and js code is:
` var pages = document.getElementById('pdf-export')
const exportHTMLToPDF = async (pages, outputType='blob') => {
console.log('checking now the pdf');
const opt = {
margin:       [0,0],
filename:     'myfile.pdf',
image:        { type: 'jpeg', quality: 0.98 },
html2canvas:  { dpi: 192, letterRendering: true },
jsPDF:        { unit: 'mm', format: 'a4', orientation: 'landscape' }
};
const doc = new jsPDF(opt.jsPDF);
const pageSize = jsPDF.getPageSize(opt.jsPDF);
for(let i = 0; i < pages.length; i++){
const page = pages[i];
const pageImage = await html2pdf().from(page).set(opt).outputImg();
if(i != 0) {
doc.addPage();
}
doc.addImage(pageImage.src, 'jpeg', opt.margin[0], opt.margin[1], pageSize.width, pageSize.height);
}
// This can be whatever output you want. I prefer blob.
const pdf = doc.output(outputType);
return pdf;
}
                  exportHTMLToPDF(element, 'outputPdf')`

PS: I think my knowledge of js is very limited, sorry in  advance if this is wrong in any way.

Hi @GGross5213,
How do you create the array of pages that you pass to the exportHTMLToPDF function?
Thanks

Hi @GGross5213,
I tried using
html2pdf().from(element).toPdf().get('pdf').then(pdf => {
$scope.pages = pdf.internal.pages;
});
but the result is another blank pdf... I tried toContainer, toCanvas, toImg, but none of those produce an array of pages. Do I have to send the html for each page in an array somehow?

@waqar-imtiaz  sorry for the delay. Life has been crazy lately. It looks like you don't need to loop through that html element. You should just be able to pass it into the html2pdf function as is. If you do have too many rows in the table that it is causing the Canvas height limit to be reached then maybe you need to periodically split up the table with

. If you add the page breaks then create a list of html strings where each element in the list is a page ending with the html2pdf page break. Also the output type needs to be one of save, blob, arraybuffer, bloburi, bloburl, datauristring, dataurlstring, dataurlnewwindow, datauri, or dataurl. This comes from jsPDF's documentation: https://rawgit.com/MrRio/jsPDF/master/docs/jspdf.js.html#line992
Let me know if you have anymore questions. I hope this helps.

@audra415 sorry for the delay. Life has been crazy. So the the array of pages that I pass into theexportHTMLToPDF funciton is a mixture of html strings that I hard coded which I add data to on run time and html elements that I selected using jquery/javascript selectors.
What are you trying to do with the pdf after you generate it? It looks like you are using angular and I am not too familiar with that. It also looks like you should using outputImg or outputPdf instead of toPdf or toImg.
Let me know if that helps.

@GGross5213,
Ah ha, array of strings for each page... Got it. I'm building a report that will have different lengths depending on the data used to generate it. So I'll have to break up the pages, use toPdf, then stitch them together in one pdf.

@eKoopmans Sounds great. I will say that I think you want to use outputImg then add the img to your own instance of jsPDF. I think that is the easiest way to stitch the pages together if your total page size exceeds the canvas height limit. (I believe that is how html2pdf.js generates the pdf underneath the hood)

Hi,
Reading through the comments I see it mentions canvas height limit.
If the problem is with the canvas height limit can the canvas height limit be increased?
If this is the case do you know how can this be done.

Hi @eKoopmans  ,
I have very large tables and images that i needed to convert into pdf.
lots of table row with different height of column content in it.
may be attachment can explain my issue

Hi @CSENoni, sorry I didn't respond sooner. Yes, I see the problem. I've reproduced it here using only html2canvas, which html2pdf relies on.
The problem is in fact deeper than that - it's a problem directly in how canvases work. Canvases have a maximum height/width, and once you exceed that the canvas becomes unusable! That's what's happening here.
I wasn't aware of that problem, so thank you for bringing it up. It's a serious limitation and I don't have any immediate solution. As you mentioned, changing the margins and page size will change how many pages you can get (also changing the resolution/scale), but that's just because it changes the size of the underlying canvas.

Have you tried to generate one canvas per page break and them add all to the PDF?

@eKoopmans first of all thank you for your library ... we are completely blocked for two days: the "limitation of the canvas" is driving us crazy .... we have tried many solutions but none seems to solve the problem in a reliable way (one is too slow, another produces a "not nice" layout", and so on)
Do you have in your plan to address this problem in the next future? It would be really really really appreciated :)

@sohilfynd really thanks... I will try it...
I didn't read anything yet.. there is some utility, "on the shelve",  for converting an existing HTML template to PDF?

Is this issue fixed? It is generating blank pages when number of pages increase more than 15 or 16.

@charutiwari04 no news on that :(! Since the problem is not the library itself but "the way" it uses for generating the pdf (see [https://stackoverflow.com/a/11585939/4080966]) i solved generating more than one pdf and, then, merging it server side. For sure it's not the cleaner possible solution... but it works :)

Hi @ScottStevenson, the cancelling is a cool idea! There's nothing supported now but I could see incorporating a cancel feature into the .then() implementation. Could you open a separate issue with that suggestion? Thanks.

@eKoopmans 	Thanks everyone for the workarounds! It prints all the pages for me but somehow there is a blank page between every meaningful page in my pdf. I was using margin-top: -1px !important; before to avoid this. However, with the new approach of adding pages one by one, it throws Uncaught (in promise) Error: Supplied Data is not a valid base64-String jsPDF.convertStringToImageData. Any ideas? Thanks!

@eKoopmans using your example here:
javascript
// Assuming "pages" is an array of HTML elements or strings that are separate pages:
var worker = html2pdf().from(pages[0]).toPdf();
pages.slice(1).forEach(function (page) {
    worker = worker.get('pdf').then(function (pdf) {
        pdf.addPage();
    }).from(page).toContainer().toCanvas().toPdf();
});
worker = worker.save();

where should we be passing the options used similar to @GGross5213's example? I assume we use .set(opt) somewhere. I'm just confused as to where.
Also, can anyone tell me whether the fragments of HTML that constitute the pages need to each be a complete HTML page (open and close HTML, HEAD and BODY tags), or if they can simply be chunks from the original page? I assume you can't split open/close tags between pages, so how would one handle a large table that spans many pages? Or can I just shove what is between each page-break into each page? That would be ideal, I think.

@eKoopmans I am using above solutions to generate pdfs with lot of pages. However, most of them will contain links and using that solution, they're all drawn at the first page. (due to page always being 1 in your hyperlink plugin)
Right now, I've 'fixed' this by altering Worker.prototype.toContainer by allowing to pass a pagenumber to the function based on the index of the loop.
Worker.prototype.toContainer = function toContainer(page) {}.
var worker = html2pdf()
  .from(elements[0])
  .set(opt)
  .toContainer(pagenumber)
  .toPdf()

Is there a way to do this without altering some files?

To answer my own questions, above, for anyone else who might come across this...
// Assuming "pages" is an array of HTML elements or strings that are separate pages:
var worker = html2pdf().set(opt).from(pages[0]).toPdf();
pages.slice(1).forEach(function (page) {
    worker = worker.get('pdf').then(function (pdf) {
        pdf.addPage();
    }).from(page).toContainer().toCanvas().toPdf();
});
worker = worker.save();

As for the pages, they can be any HTML elements. No need to use whole pages.
Also, I ended up using html2pdf.js to generate the pages, rather than figure them out myself because I have data elements that vary in height.
I started with this:
let worker = this.$html2pdf().set(opt).from(element).toContainer()
worker.get('container').then((some) => {

then looped through the children and since mine all had innerHTML, and the DIVs added by html2pdf.js didn't, I'd add my DIVs to a page (using appendChild), then start a new page when I encountered one of html2pdf.js's DIVs. This doesn't exactly make single pages. If you have elements that already break correctly, then one 'page' might actually be more than one page. However, in my case, I don't expect that I'll ever have a case where this will exceed the canvas size.
I then take the total pages generated and add headers/footers.
I'm finding that html2canvas is VERY SLOW!!! All other steps are fast. Some of my reports will be 100+ pages. Big ones could be 300+ pages. It's too slow for that. I'll have to figure out something else.
I may scrap this work and go with something that could generate the PDF on a server. But I do want it to be identical to the PDF on the page, and I don't know whether there are server-side options for this. Anybody know any options?

Hope the html2pdf.js could detect the excess automatically !

Have anyone got a solution for empty pages? I am getting randomly empty pages from top then i got some of the content printed out in pdf.
@eKoopmans Please Help !!

Hi all! First off, huge thanks to @GGross5213 for being all over the comments here, I really appreciate your help.
Second, the central issue here is that HTML canvases are just broken, and have a maximum size that I can't fix. I've investigated creating a "fake" canvas that stitches together multiple canvases, and it might be worth the effort to try to create such a thing, but I haven't had the chance to yet and I don't think it's a realistic goal.
However, there's a recent development that may fix this issue as well as a few other big problems. jsPDF has a built-in canvas "engine", which means we could draw straight into jsPDF! This would also mean vector graphics (highlightable text, small files, all that jazz). It's very promising and it's high on my to-do list for this project.
That said, I have very little time to maintain this project, so I can't make any promises. For now, this approach from @GGross5213 is your best bet (splitting your page into smaller chunks and individually adding them into the PDF). You could cut out a bit of the work with something like this:
// Assuming "pages" is an array of HTML elements or strings that are separate pages:
var worker = html2pdf().from(pages[0]).toPdf();
pages.slice(1).forEach(function (page) {
    worker = worker.get('pdf').then(function (pdf) {
        pdf.addPage();
    }).from(page).toContainer().toCanvas().toPdf();
});
worker = worker.save();

Notice you don't even need any async/await, the code above will build a "then" chain that should bring you right through to the end. Good luck!
Edit: Modified code snippet to remove the "extra" page at the end per @ScottStevenson's comment.

Hi, Thanks for suggesting this workaround, however, the Page-breaks settings is not working.
Consider this
// Page-breaks settings still working here
let worker = html2pdf().set(opt).from(pages[0]).toContainer().get('container')

// Page-breaks settings is not working at all
pages.slice(1).forEach(function (page) {
   worker = worker.get('pdf').then(function (pdf) {
        pdf.addPage();
    }).from(page).toContainer().toCanvas().toPdf();
});

Anyone have any idea can help me out with this? Thanks in advance

In case someone needs an alternative workaround, I was able to do the following since my page sizes are fixed.  Essentially, in the HTML file I sectioned out each page and gave them name (eg 'page1', 'page2' 'page3', etc.).  Then I strung together the HTML2PDF promises and created a series of pages which I stringified and sent via ajax to a PHP script which stitches them back together into a single doc.
Here is some pseudo code:
function senddata(pagearray) {
parameters = {pdfstr1: pagearray[0],
pdfstr2: pagearray[1],
pdfstr3: pagearray[2],
};
data: JSON.stringify(parameters);
send via ajax;
};
function page3(pagearray) {
var element = document.getElementById("page3");
HTML2PDF promise here and return to senddata;
};
function page2(pagearray) {
var element = document.getElementById("page2");
HTML2PDF promise here and return to page3;
};
var element = document.getElementById("page1");
pagearray= new Array();
html2pdf().set(opt).from(element).outputPdf('datauristring').then((result) => {
pagearray[0]= result;
return page2(pagearray);
});
Hope this helps someone.
Cheers,
Steven

Hi everyone! I has same problem. In my case, I can't add page-break for each page, my mean I can't split my div to pages. So, what I should to do for resolve it. I need your help :( ! Pls!

Hi all! First off, huge thanks to @GGross5213 for being all over the comments here, I really appreciate your help.
Second, the central issue here is that HTML canvases are just broken, and have a maximum size that I can't fix. I've investigated creating a "fake" canvas that stitches together multiple canvases, and it might be worth the effort to try to create such a thing, but I haven't had the chance to yet and I don't think it's a realistic goal.
However, there's a recent development that may fix this issue as well as a few other big problems. jsPDF has a built-in canvas "engine", which means we could draw straight into jsPDF! This would also mean vector graphics (highlightable text, small files, all that jazz). It's very promising and it's high on my to-do list for this project.
That said, I have very little time to maintain this project, so I can't make any promises. For now, this approach from @GGross5213 is your best bet (splitting your page into smaller chunks and individually adding them into the PDF). You could cut out a bit of the work with something like this:
// Assuming "pages" is an array of HTML elements or strings that are separate pages:
var worker = html2pdf().from(pages[0]).toPdf();
pages.slice(1).forEach(function (page) {
    worker = worker.get('pdf').then(function (pdf) {
        pdf.addPage();
    }).from(page).toContainer().toCanvas().toPdf();
});
worker = worker.save();

Notice you don't even need any async/await, the code above will build a "then" chain that should bring you right through to the end. Good luck!
Edit: Modified code snippet to remove the "extra" page at the end per @ScottStevenson's comment.

Hi, I have a page containing 170 images, every 4 images is contained inside a div with the class image-with-4. I am trying to do 4 images in each page and stitch them together.
So in typescript i am doing:
let pages = Array.from(window.document.getElementsByClassName('image-with-4')); let worker = html2pdf().set(opt).from(pages[0]).toContainer().get('container');      pages.slice(1).forEach(page=>{         worker = worker.get('pdf').then(pdf=>pdf.addPage())         .from(page).toContainer().get('container')         .toCanvas().toPdf();       })
When i inspect my network, it download 170 images and in every loop, so a total of 170170/4 = 7310 images will be downloaded.
given that each images is 2MB, 2MB7310 images = 14620MB (14.27GB)
of images will be downloaded. So i need at least 14.27GB of free ram to simply download the page as pdf, not mentioning the time to download too. I have 16gb ram in my laptop, and my chrome snapped when it's 48% done.
Idk why it keeps downloading the one that's not selected, is there anything i did wrongly? Please enlight me! Thanks in advance.

Created a PR
https://github.com/eKoopmans/html2pdf.js/pull/314

I'm using html2pdf.js to generate PDF from html obtained from within a WYSIWYG editor, I've been getting an error when generating PDFs when the generated PDF had only one page with content, a second page was generated in white. I found out how to remove the blank page during the transformation process as follows.
const opt = {
margin: [0, 0],
filename: myPdf + ".pdf",
html2canvas: {
scale: 4,
dpi: 192,
letterRendering: true,
ignoreElements: e => {
return e.classList.contains("cke_pagebreak") || e.classList.contains("html2pdf__page-break") ? true : false;
}
},
jsPDF: {
unit: "pt",
format: "A4",
orientation: "portrait",
putOnlyUsedFonts: true,
pagesplit: true,
},
pagebreak: {mode: ["avoid-all"], after: ".cke_pagebreak"}
};
const worker = html2pdf().set(opt).from(document.body).toPdf().get("pdf").then(pdf => {
    const e = pdf.internal.collections.addImage_images;
    for (let i in e) {
        e[i].h <= 133 ? pdf.deletePage(+i + 1) : null;
    }
});

/* download the PDF */
worker.save();

/* save to blob format */
worker.output('blob');

I ran this problem as well and @MaNuGitH's workaround didn't work for me. I wrote a function that might be helpful for someone. Lucky for me the new branch new-api exposes more functions and breaks the process of turning html into a pdf into several steps. I notice that I can output my html as an image created from being drawn onto a canvas which is pretty much the last step before turning it into a PDF. So, I decided to create my own instance of jsPDF and only pass in a page at a time to html2pdf. Then I add the image created by html2pdf to my instance of jsPDF. I hope this is helpful.
import html2pdf from 'html2pdf.js';
import jsPDF from 'jspdf';

const exportHTMLToPDF = async (pages, outputType='blob') => {
  const opt = {
    margin:       [0,0],
    filename:     'myfile.pdf',
    image:        { type: 'jpeg', quality: 0.98 },
    html2canvas:  { dpi: 192, letterRendering: true },
    jsPDF:        { unit: 'in', format: 'letter', orientation: 'portrait' }
  };
  const doc = new jsPDF(opt.jsPDF);
  const pageSize = jsPDF.getPageSize(opt.jsPDF);
  for(let i = 0; i < pages.length; i++){
    const page = pages[i];
    const pageImage = await html2pdf().from(page).set(opt).outputImg();
    if(i != 0) {
      doc.addPage();
    }
    doc.addImage(pageImage.src, 'jpeg', opt.margin[0], opt.margin[1], pageSize.width, pageSize.height);
  }
  // This can be whatever output you want. I prefer blob.
  const pdf = doc.output(outputType);
  return pdf;
}

You can also write it using recursion if you can't use async-await.

Hi @GGross5213,
I've encountered the same issue in a project. I've got an issue as I've innerHTML so How can I convert that into pages?

I ran this problem as well and @MaNuGitH's workaround didn't work for me. I wrote a function that might be helpful for someone. Lucky for me the new branch new-api exposes more functions and breaks the process of turning html into a pdf into several steps. I notice that I can output my html as an image created from being drawn onto a canvas which is pretty much the last step before turning it into a PDF. So, I decided to create my own instance of jsPDF and only pass in a page at a time to html2pdf. Then I add the image created by html2pdf to my instance of jsPDF. I hope this is helpful.
import html2pdf from 'html2pdf.js';
import jsPDF from 'jspdf';

const exportHTMLToPDF = async (pages, outputType='blob') => {
  const opt = {
    margin:       [0,0],
    filename:     'myfile.pdf',
    image:        { type: 'jpeg', quality: 0.98 },
    html2canvas:  { dpi: 192, letterRendering: true },
    jsPDF:        { unit: 'in', format: 'letter', orientation: 'portrait' }
  };
  const doc = new jsPDF(opt.jsPDF);
  const pageSize = jsPDF.getPageSize(opt.jsPDF);
  for(let i = 0; i < pages.length; i++){
    const page = pages[i];
    const pageImage = await html2pdf().from(page).set(opt).outputImg();
    if(i != 0) {
      doc.addPage();
    }
    doc.addImage(pageImage.src, 'jpeg', opt.margin[0], opt.margin[1], pageSize.width, pageSize.height);
  }
  // This can be whatever output you want. I prefer blob.
  const pdf = doc.output(outputType);
  return pdf;
}

You can also write it using recursion if you can't use async-await.

Could you help me understand how you take "pages" from HTML? I just try to understand how to take my page (HTML) and transfer it as an array as it is required in your solution.
Many thanks!

Hello guys,
Facing the same issue. Its okay when using Windows and Android browsers. Does not work using Safari for iPhone users.
I will try some of the code here. Hope it works.

@eKoopmans I am using above solutions to generate pdfs with lot of pages. However, most of them will contain links and using that solution, they're all drawn at the first page. (due to page always being 1 in your hyperlink plugin)
Right now, I've 'fixed' this by altering Worker.prototype.toContainer by allowing to pass a pagenumber to the function based on the index of the loop. Worker.prototype.toContainer = function toContainer(page) {}.
var worker = html2pdf()
  .from(elements[0])
  .set(opt)
  .toContainer(pagenumber)
  .toPdf()

Is there a way to do this without altering some files?

I'm also facing the same issue but didn't get your workaround can you help this out

Hi @eKoopmans
I have use html2pdf:  "html2pdf.js": "^0.10.1".
I have created html to pdf with 40-50 pages.
In generated pdf pages with 1 to 10 its working fine but when pages more than 20 pages it failed to show the full page content and color disappear issue is coming.
Here is my code:
htmlToPdfOptions: {
margin: [1,0.5],
html2canvas: {
scale:2,
dpi: 300,
removeContainer: true,
imageTimeout: 0,
letterRendering: true,
useCORS: true,
},
jsPDF: {
unit: "in",
format: 'letter',
orientation: "portrait",
compress:true,
pagesplit: true,
},
image: {
type: "jpg",
quality: 0.98
},
pagebreak: { avoid: ['.divsec' , 'img'] },
enableLinks: false
}
        When we decrease scale equal to "0.5" then pdf color issue solve but text blurry issue coming and when I increase scale
         equal to"1 or 2" then text blurry issue solve but color disappear issue coming.

        Could you please help!

        Thank you!

@eKoopmans  and anyone else please help.
I have hit a blocker when trying to convert a page to pdf.
Apparently the issue is with the canvas size limitation that we have with respect to browser. The permissible limit for canvas height is 32,767px for chrome/edge/firefox
My code basically grabs the current page which is suppose to be a report and then adds a defined header footer to each page while adding the grabbed html instance.
The problem arrives when we have a longer report whose canvas size exceeds the defined canvas size of browser then it renders the whole pdf blank apart from header and footer.
Sharing the code for reference:
==========================
html2pdf().from(rootElement).set(opt).toImg().toPdf().get('pdf').then(function(pdf){
var totalPages = pdf.internal.getNumberOfPages();
for (var i = 1; i <= totalPages; i++) {
pdf.setPage(i);
let definedHeight = pdf.internal.pageSize.getHeight();
renderFooter(pdf, i, definedHeight)
}
}).then(function(){
if (callback) {
callback(true);
}
}).save();
Note:
renderFooter its is function to populate header and footer and is working fine for any number of pages.
callback its just a value which updates the waiting status at the web page.
Kindly suggest and alternate solution which would be able to accomodate any number of pages in pdf.
Currently its is able to create a maximum of pa pdf 16 pages, beyond that it breaks.
I need to go uptill 29-40 pages if possible.

@tmislos-cgi Becauase I am appending header and footer on each page explicitly

@GAVTIN here is how i resolved my issue with the canvas height limitation:
I added aria-label to my divs dynamically, where i can logically divide them.

then upon generating the pdf, I broke the html elements to an array and then combined them to a single pdf file.
const pages = Array.from(input.querySelectorAll('div[aria-label^="pdf-page-"]'))
let worker = html2pdf()
    .set(pdfOptions)
    .from(pages[0])

  worker = worker.toPdf() // worker is now a jsPDF instance

  if (pages.length > 1) {

    // add each element/page individually to the PDF render process
    pages.slice(1).forEach((page, index) => {
      worker = worker
      //worker = worker.toPdf() // worker is now a jsPDF instance
        .get('pdf')
        .then(pdf => {
          pdf.addPage()
        })
        .from(page)
        .toContainer()
        .toCanvas()
        .toPdf()
    })
  }

got these code from above or another issue i think.

@GAVTIN here is how i resolved my issue with the canvas height limitation:
I added aria-label to my divs dynamically, where i can logically divide them.
<div aria-label={`pdf-page-` + ++i}>

then upon generating the pdf, I broke the html elements to an array and then combined them to a single pdf file.
const pages = Array.from(input.querySelectorAll('div[aria-label^="pdf-page-"]'))
let worker = html2pdf()
    .set(pdfOptions)
    .from(pages[0])

  worker = worker.toPdf() // worker is now a jsPDF instance

  if (pages.length > 1) {

    // add each element/page individually to the PDF render process
    pages.slice(1).forEach((page, index) => {
      worker = worker
      //worker = worker.toPdf() // worker is now a jsPDF instance
        .get('pdf')
        .then(pdf => {
          pdf.addPage()
        })
        .from(page)
        .toContainer()
        .toCanvas()
        .toPdf()
    })
  }

got these code from above or another issue i think.

This fixes the problem yes!

Hola y una pregunta y como prodria hacer para que el contenido se agregue a continuación del anterior ejemlplo quiero hacer el metodo que tienen agregar las paginas pero deseo agregar justo donde terminar el primero ya que como mi contenido es dinamico no se cuando este termine.

Any update? still facing the issue when pages are more than 70

I ran this problem as well and @MaNuGitH's workaround didn't work for me. I wrote a function that might be helpful for someone. Lucky for me the new branch new-api exposes more functions and breaks the process of turning html into a pdf into several steps. I notice that I can output my html as an image created from being drawn onto a canvas which is pretty much the last step before turning it into a PDF. So, I decided to create my own instance of jsPDF and only pass in a page at a time to html2pdf. Then I add the image created by html2pdf to my instance of jsPDF. I hope this is helpful.
import html2pdf from 'html2pdf.js';
import jsPDF from 'jspdf';

const exportHTMLToPDF = async (pages, outputType='blob') => {
  const opt = {
    margin:       [0,0],
    filename:     'myfile.pdf',
    image:        { type: 'jpeg', quality: 0.98 },
    html2canvas:  { dpi: 192, letterRendering: true },
    jsPDF:        { unit: 'in', format: 'letter', orientation: 'portrait' }
  };
  const doc = new jsPDF(opt.jsPDF);
  const pageSize = jsPDF.getPageSize(opt.jsPDF);
  for(let i = 0; i < pages.length; i++){
    const page = pages[i];
    const pageImage = await html2pdf().from(page).set(opt).outputImg();
    if(i != 0) {
      doc.addPage();
    }
    doc.addImage(pageImage.src, 'jpeg', opt.margin[0], opt.margin[1], pageSize.width, pageSize.height);
  }
  // This can be whatever output you want. I prefer blob.
  const pdf = doc.output(outputType);
  return pdf;
}

You can also write it using recursion if you can't use async-await.

Hey , thanks for the solution as it is working and I am able to download pages with around 230 pages. But there is a new issue which I encountered the pdf css is not as in original page.
Anyone encountered the same problem?
@eKoopmans

@GAVTIN here is how i resolved my issue with the canvas height limitation:
I added aria-label to my divs dynamically, where i can logically divide them.
<div aria-label={`pdf-page-` + ++i}>

then upon generating the pdf, I broke the html elements to an array and then combined them to a single pdf file.
const pages = Array.from(input.querySelectorAll('div[aria-label^="pdf-page-"]'))
let worker = html2pdf()
    .set(pdfOptions)
    .from(pages[0])

  worker = worker.toPdf() // worker is now a jsPDF instance

  if (pages.length > 1) {

    // add each element/page individually to the PDF render process
    pages.slice(1).forEach((page, index) => {
      worker = worker
      //worker = worker.toPdf() // worker is now a jsPDF instance
        .get('pdf')
        .then(pdf => {
          pdf.addPage()
        })
        .from(page)
        .toContainer()
        .toCanvas()
        .toPdf()
    })
  }

got these code from above or another issue i think.

I am using the same code in my project, but when there are multiple pages(more than 20) my application becomes static/hanging . I am not able to interact with my webpage, not able to click on the buttons etc..
To solve this I passed the whole html at-once to html2Pdf() instead of the loop. But in that case if there are multiple pages(more than 20), then pages are blank in the pdf

version<EMAIL_ADDRESS>I hope this is helpful
my html code like this:

xxxx

xxxx

xxxx

xxxx

.sep is a empty css class name, it is used to manual split large content into small chunk
js code
const pages = document.querySelectorAll('.sep');
const worker = html2pdf();
for (let i = 0; i < pages.length; i++){
  const page = pages[I];
  await worker.from(page).toPdf();
}
worker.save()

version<EMAIL_ADDRESS>I hope this is helpful
my html code like this:

xxxx

xxxx

xxxx

xxxx

.sep is a empty css class name, it is used to manual split large content into small chunk
js code
const pages = document.querySelectorAll('.sep');
const worker = html2pdf();
for (let i = 0; i < pages.length; i++){
  const page = pages[I];
  await worker.from(page).toPdf();
}
worker.save()

Hi @GGross5213 ,
I am still learning JS and I am running into the same problem and I was wondering how can I implement your workaround in my case.
Currently, I have a dynamic content and I am using @eKoopmans plugin to generate a PDF report for it.
So this is my current function:
$("#download").click(function(){
filename = this.value + '.pdf'
var element = document.getElementById('report');

html2pdf(element, {
		margin:       0.25,
		filename:     filename,
		image:        { type: 'jpeg', quality: 0.98 },
		html2canvas:  { dpi: 96, letterRendering: true },
		jsPDF:        { unit: 'in', format: 'letter', orientation: 'portrait' }
	  });

});

and I would like to know where should I put my "element" variable into your function in order to have it working?
I would much appreciate if you could give me hand with it.

I have this same implementation and issue, did you find any concrete solution for the same.

For those who are having issues exporting documents with many pages and find that they export blank, here is a solution I’ve found. When the document has more than 30 pages, I’ve noticed it exports empty. However, by separating the sections and processing them in batches of 30, I can avoid this problem. Then, I simply merge the results and download the final file. This method is faster than other solutions that process each section individually.
const exportFunction = async (idElement: string, nameFile: string, pageOrientation = 'portrait') => {
await new Promise(resolve => setTimeout(resolve, 0));
const element = document.getElementById(idElement);
if (!element) return;
const opt = {
    margin: [0, 0, 0, 0],
    filename: nameFile,
    image: { type: 'jpeg', quality: 0.99 },
    html2canvas: { scale: 2 },
    jsPDF: { unit: 'in', format: 'letter', orientation: pageOrientation },
    pagebreak: { mode: ['avoid-all', 'css', 'legacy'] },
};
const sections = Array.from(element.children);

if (sections.length > 30) {
    let worker = html2pdf().set(opt);

    const processInBatches = async () => {
        const batchSize = 30;
        for (let i = 0; i < sections.length; i += batchSize) {
            const batchElement = document.createElement('div');
            const batchSections = sections.slice(i, i + batchSize);
            batchSections.forEach(section => batchElement.appendChild(section.cloneNode(true)));
            await worker.from(batchElement).toContainer().toCanvas().toPdf().get('pdf').then((pdf: any) => {
                if (i + batchSize < sections.length) {
                    pdf.addPage();
                }
            });
        }
        await worker.save();
    };
    await processInBatches();

} else {
    await html2pdf().set(opt).from(element).save();
}

};