Aborting/quitting client is very broken

Trying to Ctrl-C results in
deduplicating getty artists.  This may take a while...
Parsed 1 entries, 1 unique artistsRPC error inserting entity: RemoteError(code=StatusCode.UNKNOWN, details="")

done deduplicating artists
^CTraceback (most recent call last):
  File "/Users/arkadiy/mine/mediachain-client/venv/bin/mediachain", line 11, in <module>
    sys.exit(main())
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/mediachain/cli/main.py", line 114, in main
    fn(ns)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/mediachain/cli/main.py", line 107, in <lambda>
    ns.download_thumbs)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/mediachain/getty/ingest.py", line 23, in ingest
    transactor, datastore, dir_root, max_num, download_thumbs
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/mediachain/getty/ingest.py", line 96, in getty_artefacts
    thumbnail_ref=thumbnail_ref
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/mediachain/getty/ingest.py", line 45, in getty_to_mediachain_objects
    entity_ref = transactor.insert_canonical(entity)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/mediachain/transactor/client.py", line 29, in insert_canonical
    ref = self.client.InsertCanonical(req, timeout)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/grpc/framework/crust/implementations.py", line 75, in __call__
    protocol_options, metadata, request)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/grpc/framework/crust/_calls.py", line 109, in blocking_unary_unary
    return next(rendezvous)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/grpc/framework/crust/_control.py", line 425, in next
    self._condition.wait()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 340, in wait
    waiter.acquire()
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/Users/arkadiy/mine/mediachain-client/venv/lib/python2.7/site-packages/concurrent/futures/thread.py", line 39, in _python_exit
    t.join(sys.maxint)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 960, in join
    self.__block.wait(delay)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 359, in wait
    _sleep(delay)
KeyboardInterrupt
Error in sys.exitfunc:

(and it still doesn't exit)
Seems like you need to assign an interrupt handler maybe?

Here was the temporary "fix" used by the Indexer ingester...
        try:
           #code here
        except KeyboardInterrupt:
            import traceback, sys, os
            for line in traceback.format_exception(*sys.exc_info()):
                print line,
            sleep(1)
            os._exit(-1)

k; I'll wrap main with that for now.  Hopefully we can find a real fix eventually :)

I think the real issue is an exception handler in gRPC catching the SystemExit exception.  I dug into that a while ago, and I think that a fix might have made it into master, but isn't released yet.  I'll see if I can find that again so we can track it

Closing for now, can reopen with real fix as needed


Organize source code and documentation

Organize source code and documentation
Reorganize source code and documentation

Archive old source, testing results, references
Simplify active (v2) source code tree

Add documentation
(In Sphinx)

First attempt: https://github.com/jj1bdx/avrhwrng/commit/69104e28c8e8659cc31587732fe85ba81fb3a46d


azure-functions-maven-plugin:1.0.0-beta-6:deploy is not working

Plugin name and version
azure-functions-maven-plugin:1.0.0-beta-6:deploy
Plugin configuration in your pom.xml
    <dependency>
        <groupId>com.microsoft.azure.functions</groupId>
        <artifactId>azure-functions-java-library</artifactId>
        <version>1.0.0-beta-5</version>
    </dependency>

....

com.microsoft.azure
azure-functions-maven-plugin
1.0.0-beta-6

....

FUNCTIONS_EXTENSION_VERSION
2.0.12095.0

Expected behavior
Function successfully deployed and running
Actual behavior
Function deployment takes significantly longer then when using 1.0.0-beta-4.
Function is only partially deployed (see the attached screenshots); when 1.0.0-beta-4 is deployed correctly so this is a regression.
Steps to reproduce the problem
Build a simple Java function with a time trigger and an EventHub output. For actual code see (https://github.com/Microsoft/inventory-hub-java-on-azure/tree/master/function-apps/Point-Of-Transaction)

@milismsft
Thanks for your report, could you please offer more details infos to reproduce the problem? I tried to clone the Point-Of-Transaction project and deploy it by plugin beta6, it seems works well. Besides, we change the default deploy method to zip deploy, so infos printed in console may differs from beta4 to beta6.

@Flanker32 can you please attach a screenshot of your console and the pom.xml settings you used? Might be good to setup a skype call and see if we can straight these things right.

Ok, it seems to be working now by just changing the version of the plugin version in the pom.xml. The deployment takes a bit longer than when using beta-4 but it's definitely shorter than the 9 minutes.

@Flanker32 The deployment failure when using beta-6 plugin repro'ed constantly last Saturday so I'm a bit puzzled at the moment what might have caused the failure (see the console screenshot I attached with 9 min deployment time). I'm running couple more tests over the day and if all goes well I'll close the issue.


Open file/folder crashes VS Code in Ubuntu 24.04

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.89.1
OS Version: Ubuntu 24.04

Steps to Reproduce:

Launch VS Code with an open folder
Select any of the following from the File menu and VS Code crashes immediately

Open File
Open Folder
Open Workspace from File

The following work:

File -> Open Recent
File -> New Window

The following causes a crash:

File -> New Window -> Open File -> select a folder to open
File -> New Window -> Open Folder -> select a folder to open

There could be an issue with the open file due to changes in GNOME 46 Nautilus.

Can you confirm if the issue is present with latest insiders https://code.visualstudio.com/insiders

Can you confirm if the issue is present with latest insiders https://code.visualstudio.com/insiders

There is an error when trying to install.

Can someone suggest a workaround for the time being?

Have the same issue with vscode installed through Ubuntu Snap.
Tried the insiders and regular .deb packages and they work without the above mentioned issue.

@papanireal Which version of Ubuntu are you using?

24.04

I tried again with the latest .deb and there is still an error.

Does this issue occur when all extensions are disabled?: Yes
* VS Code Version: 1.89.1

* OS Version: Ubuntu 24.04

Steps to Reproduce:
1. Launch VS Code with an open folder

2. Select any of the following from the File menu and VS Code crashes immediately

* Open File

* Open Folder

* Open Workspace from File

The following work:
* File -> Open Recent

* File -> New Window

The following causes a crash:
* File -> New Window -> Open File -> select a folder to open

* File -> New Window -> Open Folder -> select a folder to open

There could be an issue with the open file due to changes in GNOME 46 Nautilus.

I had the same problem since I updated VSCode today, it was instant closing, not even opening the main menu. What worked for me was uninstalling all vscode and autoremove after. Then, installing it from the official webpage .deb file, with that, I solve it

@papanireal @Lexslogic Are you using Gnome 46 Wayland desktop?

I had the same problem since I updated VSCode today, it was instant closing, not even opening the main menu.

Same here after updating the VSCode from snap.
Next ....

Uninstalled VS Code from App Center
autoremove --purge
Re-installed from App Center
Same problem

Next ...

Uninstalled VS Code from App Center
autoremove --purge
Downloaded code_1.90.0-1717531825_amd64.deb from https://code.visualstudio.com/Download
Right-click -> Open With... -> GDebi Package Installer
"Error: Dependency is not satisfiable: libglib 2.0-0 (>= 2.37.3)"

Error: Dependency is not satisfiable: libglib 2.0-0 (>= 2.37.3)

MySQL Workbench - Dependency Issue
You want the deb for ubuntu 22.04, not 23.10.

See also:

#212430
#211762
#204159
#212494
#212633
VS Code Jupyter Crash Ubuntu 24.04

I had the same issue for three days, but now the Vscode doesnt even start anymore
Please fix ASAP

Does the following have any relevance?
If so, how can it be fixed?
~/.config/Code/logs/20240607T005226/main.log

2024-06-07 00:52:27.002 [info] update#setState idle
2024-06-07 00:52:27.935 [error] vscode-file: Refused to load resource /snap/code/159/usr/share/code/resources/app/extensions/theme-seti/icons/seti.woff from vscode-file: protocol (original URL: vscode-file://vscode-app/snap/code/159/usr/share/code/resources/app/extensions/theme-seti/icons/seti.woff)

However, there is no /snap/code/159

From Visual Studio Code crashes on Ubuntu 22.04.4 LTS

This also works (No need to reinstall or downgrade)
code --in-process-gpu

Above works for me too, although ...
$ code --in-process-gpu
Warning: 'in-process-gpu' is not in the list of known options, but still passed to Electron/Chromium.

I got same issues today and have been unable to resolve this.
Vs code closes almost immediately I attempt to open it. I haven't installed any new extension since I last used it a few hours ago.

I just resolved mine now. I downloaded the deb file from vscode's site, installed it using:
sudo dpkg -i [file_path/file_name.deb]
This solved my isses:
Note: remember to uninstall the previous vscode you have before or after doing this avoid confusions

I can also confirm that uninstalling via snap & manually installing code_1.90.0-1717531825_amd64.deb fixes the issue.
sudo dpkg -i code_1.90.0-1717531825_amd64.deb
It seems there is an issue with the snap code installation.
No errors in ~/.config/Code/logs/20240607T130517/main.log
Note: Although I exported my settings and extensions before uninstal, snap uninstall didn't remove the extensions from ~/.vscode and everything was there after manually installing above (which was great).

I also had VSCode crash but right now, after installing all the updates from apt-get and snap, VSCode works fine.
I didn't do any hacks to fix it myself (like disabling wayland, etc).

nope, snap code (160) is a total failure.

Same problem here:
vscode-file: Refused to load resource /snap/code/159/usr/share/code/resources/app/extensions/theme-seti/icons/seti.woff from vscode-file: protocol (original URL: vscode-file://vscode-app/snap/code/159/usr/share/code/resources/app/extensions/theme-seti/icons/seti.woff)

Same here, but VS Code crashes immediately when starting it.
ubuntu 24.04
vs code 1.90

code --in-process-gpu

I had same issue and it also works for me.

If your VSCode crashes immediately you open it. Perhaps you have installed it through the App Center. To solve this issue, Uninstall VSCode from App Center and follow the following steps;

First, download and install the GPG key to trust packages from the Microsoft repository:
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > packages.microsoft.gpg sudo install -o root -g root -m 644 packages.microsoft.gpg /etc/apt/trusted.gpg.d/ rm packages.microsoft.gpg

Next, add the Visual Studio Code repository to your system's software sources list:
echo "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main" | sudo tee /etc/apt/sources.list.d/vscode.list

Finally, update your package lists and install Visual Studio Code:
sudo apt-get update
sudo apt-get install code

Following these 3 simple steps should definitely fix your issue. Have a great time coding with VSCode!

just updated the snap version, got 1.90.1 and it doesn't crash anymore

It's still crashed on Fedora even with the latest version :/

If your VSCode crashes immediately you open it. Perhaps you have installed it through the App Center. To solve this issue, Uninstall VSCode from App Center and follow the following steps;

First, download and install the GPG key to trust packages from the Microsoft repository:
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > packages.microsoft.gpg sudo install -o root -g root -m 644 packages.microsoft.gpg /etc/apt/trusted.gpg.d/ rm packages.microsoft.gpg
Next, add the Visual Studio Code repository to your system's software sources list:
echo "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main" | sudo tee /etc/apt/sources.list.d/vscode.list
Finally, update your package lists and install Visual Studio Code:
sudo apt-get update
sudo apt-get install code

Following these 3 simple steps should definitely fix your issue. Have a great time coding with VSCode!

Thanks @Olorunnisola01 , it works perfect to me

The new veraion seems fixed problem

This is such a laughable problem to have for Microsoft when they can't stop bragging about how much they love open-source.

This is happening to me also with the latest snap version on Ubuntu 24.04, but it doesn't happen with all folders and of note it doesn't happen at all in Kubuntu 24.04 with plasma.

 I am anti-snap person, it is a bug from VScode. Now they seems fix it

I just resolved mine now. I downloaded the deb file from vscode's site, installed it using: sudo dpkg -i [file_path/file_name.deb]
This solved my isses:
Note: remember to uninstall the previous vscode you have before or after doing this avoid confusions

It solved for me. Thanks!

None of those worked for me.  I removed via apt remove, reinstalled via dpkg -i with the latest deb, removed my recent folder, removed all vs folders from the home directory, rebooted, and nothing has worked.
             /////////////                cuzimbob@pop-os
         /////////////////////            ---------------
      ///////*767////////////////         OS: Pop!_OS 22.04 LTS x86_64
    //////7676767676*//////////////       Host: Thelio thelio-b3
   /////76767//7676767//////////////      Kernel: 6.9.3-76060903-generic
  /////767676///*76767///////////////     Uptime: 15 mins
 ///////767676///76767.///7676*///////    Packages: 2062 (dpkg), 11 (flatpak)
/////////767676//76767///767676////////   Shell: bash 5.1.16
//////////76767676767////76767/////////   Resolution: 2560x1080, 2560x1080
///////////76767676//////7676//////////   DE: GNOME 42.9
////////////,7676,///////767///////////   WM: Mutter
/////////////*7676///////76////////////   WM Theme: Pop
///////////////7676////////////////////   Theme: Pop-dark [GTK2/3]
 ///////////////7676///767////////////    Icons: Pop [GTK2/3]
  //////////////////////'////////////     Terminal: gnome-terminal
   //////.7676767676767676767,//////      CPU: 12th Gen Intel i7-12700K (20) @ 4.900GHz
    /////767676767676767676767/////       GPU: Intel AlderLake-S GT1
      ///////////////////////////         Memory: 10261MiB / 64058MiB
         /////////////////////
             /////////////

[1]+  Done                    code-insiders

It crashes as soon as I click the file menu.

Reproducing the issue is pretty easy... i uninstalled all extensions, nothing happened, i changed some settings in VScode nothing happened.
Gonna use for the moment different IDE

hitting this issue again on 1.94.0 on ubuntu 24.04.1

for me v1.94.0 freezes because opening kdialog will eat all memory

Manually installing the deb file using: sudo dpkg -i FILE_NAME solved the problem!

This works perfectly i suggest to uninstall it from snap and install it from this official page https://code.visualstudio.com/.
No need to downgrade it just click on .deb install and then use the command what @piedpipr wrote down. The case is closed

APT auto-upgraded me to 1.95.0-insider today.  When I opened it up, I clicked the file menu, and bam! crash.  I ran sudo apt install code-insiders=1.93.0-1725459622, and it downgraded to that version; when I opened it and clicked the File menu, nothing happened except the file menu opening.  So, it looks like anything greater than 1.93.0-1725xxx crashes.

Some problem here. I have solved opening VS Code from the terminal (code .) on the right folder.

My most recent sudo apt upgrade brought xwayland as an uninvited guest. vscode terminal window started to be laggy, and there is lots of similar reports out there. I was trying to resolve an issue with laggy terminal following this advice. It may or may not have improved things, but I still notice the "File..." menu crash described in the present issue.
Environment
sourced from deb [arch=amd64,arm64,armhf] https://packages.microsoft.com/repos/code stable main
code --version with alias code='code-insiders --enable-features=UseOzonePlatform --ozone-platform=wayland'
Warning: 'enable-features' is not in the list of known options, but still passed to Electron/Chromium.
1.95.0-insider
a016ec9b66ffdd3ff0f831768b8e75be008a54e4
x64

lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 12 (bookworm)
Release:	12
Codename:	bookworm

echo $XDG_SESSION_TYPE
wayland

downgrading to one of the 1.94 appears to resolve the "File..." crash issues:
apt-cache show code-insiders | grep Version
sudo apt install code-insiders=1.94.0-1727878717

Still occuring with latest snap release on 24.04 when opening files on host machine.   Does not happen on remote servers.

It keeps happening to me too with the latest version of VS 1.94.2 on my release: Ubuntu 24.04.1 LTS

Workaround until the fix comes in
Create a workspace file in the directory you want to open. In VScode,
File -> Open Workspace from file...
Browse to the workspace file and open it.
To make a blank workspace file, open a VScode directory that already works for you. Save it as a workspace. Then copy the workspace file to the directory you want to open and rename the file to the name of the directory.

Workaround until the fix comes in
Create a workspace file in the directory you want to open. In VScode, File -> Open Workspace from file...
Browse to the workspace file and open it.
To make a blank workspace file, open a VScode directory that already works for you. Save it as a workspace. Then copy the workspace file to the directory you want to open and rename the file to the name of the directory.

I don't follow.  We can't use File->Open Workspace without crashing vscode, and a number of us doing certain types of development are not using workspaces at all.  And it still doesn't avoid the situation where we need to do things like File|SaveAs/NewFile/RevertFile/Preferences/OpenFolder/etc... and we will crash the app before we even think about it (I did this a half dozen times before downgrading).  The only workaround is to run an old version and that will quickly be a problem if you use certain extensions.

I have the same issue

Faced the same problem (ubuntu 24). The reason was an installation way (from snap store) fixed by installing from official apt repository

Not sure if this will help anyone else. I ran into this issue today. Every time I tried to click "File" -> "Open Folder..." it would cause vscode to crash.
I opened the terminal, navigated to the folder that I wanted to open in vscode and simply typed "code insert_filename". Once it opened, I was able to then use "File" -> "Open Folder..." as I could in the past.
I know this doesn't solve the issue per-say, but it seemed to fix the crash issue and allowed me to open folders again.
Ubuntu Version Info:
> lsb_release -a

Distributor ID:	Ubuntu

Description:	Ubuntu 24.04.1 LTS

Release:	24.04

Codename:	noble

sudo apt install code-insiders=1.94.0-1727878717

This worked for me. It also seems to have something to do with the Copilot Chat extension but I'm not sure. I'm happy for now

arch linux, same happens for me

FYI there is another related issue where clicking "File" -> "Open Folder..." to any folder containing a .json file crashes VScode with different workarounds: https://github.com/MicrosoftDocs/intellicode/issues/520

Ubuntu 24.04 - suddenly doing the same for me today.
Version: 1.94.2
Commit: 384ff7382de624fb94dbaf6da11977bba1ecd427
Date: 2024-10-09T16:08:44.566Z
Electron: 30.5.1
ElectronBuildId: 10262041
Chromium: 124.0.6367.243
Node.js: 20.16.0
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 6.8.0-47-generic snap

Looks like a snap issue. Did
snap remove code

Then follow the instructions here to use apt: https://code.visualstudio.com/docs/setup/linux
Now all is well.

Ubuntu 24.04 - suddenly doing the same for me today.
Version: 1.94.2
Commit: 384ff7382de624fb94dbaf6da11977bba1ecd427
Date: 2024-10-09T16:08:44.566Z
Electron: 30.5.1
ElectronBuildId: 10262041
Chromium: 124.0.6367.243
Node.js: 20.16.0
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 6.8.0-47-generic snap

Looks like a snap issue. Did
snap remove code

Then follow the instructions here to use apt: https://code.visualstudio.com/docs/setup/linux
Now all is well.

My Problem Solved by this method.
Thanks a lot..

An alternative to open a folder in VSC, is using the Terminal. Go to the project path you need to open and use:
code -n .

installing with deb works for me

An alternative to open a folder in VSC, is using the Terminal. Go to the project path you need to open and use:
code -n .

was facing crash issue while using open folder option and code porject-folder seems to work.
Version: 1.94.2
Commit: 384ff7382de624fb94dbaf6da11977bba1ecd427
Date: 2024-10-09T16:08:44.566Z
Electron: 30.5.1
ElectronBuildId: 10262041
Chromium: 124.0.6367.243
Node.js: 20.16.0
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 6.8.0-47-generic snap

Ubuntu 24.04 - suddenly doing the same for me today.
Version: 1.94.2
Commit: 384ff7382de624fb94dbaf6da11977bba1ecd427
Date: 2024-10-09T16:08:44.566Z
Electron: 30.5.1
ElectronBuildId: 10262041
Chromium: 124.0.6367.243
Node.js: 20.16.0
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 6.8.0-47-generic snap

Looks like a snap issue. Did
snap remove code

Then follow the instructions here to use apt: https://code.visualstudio.com/docs/setup/linux
Now all is well.

Ubuntu 24.04.1 .. This one solved the problem for me.

This is for vscode insiders, I install via apt.  Have the file menu crashes every time.  Ctrl-k ctrl-o works.
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
apt list code-insiders
Listing... Done
code-insiders/stable,now 1.95.0-1729577238 amd64 [installed]
Version: 1.95.0-insider
Commit: defe52dc43e6f5d861f55600ff01f7553fac7eaf
Date: 2024-10-22T05:04:01.628Z
Electron: 32.1.2
ElectronBuildId: 10323173
Chromium: 128.0.6613.162
Node.js: 20.17.0
V8: 12.8.374.33-electron.0
OS: Linux x64 6.5.0-44-generic

Running in verbose displays the error.
[101075:1023/142333.932239:FATAL:check.cc(376)] Check failed: false. NOTREACHED log messages are omitted in official builds. Sorry!
[1023/141716.552067:ERROR:process_memory_range.cc(75)] read out of range
[1023/141716.553880:ERROR:directory_reader_posix.cc(43)] opendir /home/<user>/.config/Code - Insiders/Crashpad/attachments/7e9b5e85-d176-4b66-974d-278e49f5fcaf: No such file or directory (2)

I had this problem yesterday, couldn't touch "File".  However, for some reason, reinstalled it today and it is working fine
code-insiders client running
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
Codename:	noble

1.95.0-insider
fe997185b5e6db94693ed6ef5456cfa4e8211edf
x64

connects to server also running code-insiders
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.5 LTS
Release:	22.04
Codename:	jammy

1.95.0-insider
defe52dc43e6f5d861f55600ff01f7553fac7eaf
x64

I too installed today(apt xubuntu)
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
Version: 1.95.0-insider
Commit: fe997185b5e6db94693ed6ef5456cfa4e8211edf
Date: 2024-10-23T05:06:13.568Z
Electron: 32.2.1
ElectronBuildId: 10427718
Chromium: 128.0.6613.186
Node.js: 20.18.0
V8: 12.8.374.38-electron.0
OS: Linux x64 6.5.0-44-generic
and the issue is no longer present - was it fixed by a different bug?

Potential duplicate of https://github.com/microsoft/vscode/issues/230604, as merl-dev mentioned.

I got a full disk yesterday and started doing some desperate cleanup, and if I remember correctly I deleted contents of /home/<user>/.config/Code - Insiders/User/workspaceStorage and
/home/<user>/.config/Code - Insiders/User/History
I found those folders to be ridiculously huge using the Disk Usage Analytics tool.
I'm not sure if this is a coincidence but after that, clicking file worked without crashing.

installing deb helped me, it seems like VSC is bugged in ubuntu app centre

I did not mention that in both cases (client and server), code-insiders  was installed using snap and not apt/deb...

Reports here had a mix of scenarios from two different bugs that affected snap packages  as well as deb. Given both of them have been addressed in latest insiders, I am closing this one. Please report if you are seeing otherwise.


Crash while installing

Trying to install ClassicUO on Windows 11 but something is not working.
I have full access to the folder I'm working on  (Complete Control for the user), tried both with and without admin rights, also tried changing folder installation (EG: Documents\CUO), same issue.
ClassicUOLauncher.exe got renamed to ClassicUOLauncher.exe_
Here's the crashlog.
Datetime: 08/08/2024 17:14:09
Environment PATH: D:\Games\Electronic Arts
.NET Environment PATH: D:\Games\Electronic Arts
args: D:\Games\Electronic Arts\ClassicUOLauncher.exe
System.ComponentModel.Win32Exception (5): An error occurred trying to start process 'D:\Games\Electronic Arts\launcher_update_temp\ClassicUOLauncher.exe' with working directory 'D:\Games\Electronic Arts\launcher_update_temp'. Accesso negato.
at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo) + 0xe00
at System.Diagnostics.Process.Start(ProcessStartInfo) + 0x38
at ClassicUOLauncher.Patcher.LauncherFilePatcher.d__4.MoveNext() + 0x7cf
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
at ClassicUOLauncher.Patcher.LauncherFilePatcher.d__3.MoveNext() + 0x1e0
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
at System.Threading.Tasks.Task.<>c.b__128_0(Object state) + 0x28
at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[TState](ExecutionContext, Action`1, TState&) + 0x60
at System.Threading.ThreadPoolWorkQueue.Dispatch() + 0x2a0
at System.Threading.WindowsThreadPool.DispatchCallback(IntPtr, IntPtr, IntPtr) + 0x45

probably related to some fix.
Should be ok now


chore: add myself to usernames

Could you resolve conflicts, please?

Oops, I caused a second conflict by merging another one, I'll get that one!


Remove jquery from organisation-list-filter

Trello: https://trello.com/c/nXatVRcM
What's changed?
Removes jQuery from organisation-list-filter.js and make it a conventional GOV.UK module.
Also converts start methods into init methods.
Why?
We're removing jQuery from GOV.UK and this file is a little unusual.
Testing
Used on: https://www.gov.uk/government/organisations
Paired with: @andysellick
⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠

@owenatgov Thanks for your review. I've made the changes to the for loop and fixed up, and responded to the other comments.


Preflight CORS with Lambda Proxy Integration

So -  it looks like I can add the headers to the POST integration for my Lambda Proxy Integration -  but I'm not sure how I should be doing this for the OPTIONS requests for endpoints.  From what I can tell, I've configured things correctly, but I'm still getting {"message": "Internal server error"} back from the OPTIONS endpoint specifically.

@TimothyDalbey are you using Serverless >= 1.0? This plugin is only for older versions without native support for CORS, so in that case this might be not the best place to ask. Anyway, you should have a look at the Cloudwatch logs to see the actual error message.


Support for code_verifier and code_challenge

Hi,
How do we extend this framework to support code_verifier and code_challenge functionality specified in RFC 7636 (Proof Key for Code Exchange) PCKE?
The code_challege needs to be persisted along with the auth code.
Can you let us know extension points? (if any) through which I can read the code_challege through the request.
Reference : https://www.authlete.com/documents/article/pkce/index
Thanks
Kunal d.

Hi @codestorycooked . I've implemented PKCE in #452 .
Is that what you need?

Thanks @visvk


Remove delegation role fallback when applying targets changes

Given the behavior proposed in https://github.com/docker/docker/pull/21046, the fallback behavior is no longer that necessary -- this PR removes the fallback logic when applying changes on targets entirely.
Docker will be able to determine the roles it will sign for from TUF metadata and private keys in its trust directory, so there is no need to silently fallback on the notary side (I'm especially worried about any potential role confusion)
Signed-off-by: Riyaz Faizullabhoy<EMAIL_ADDRESS>

LGTM!  Thank you so much for fixing all the misspellings of "delegation" in the test case names and comments :D!

Just to confirm my understanding: it will be up to docker (or a user of the notary CLI) to explicitly specify "targets" vs "targets/a_delegation" and if the top level "targets" role is provided, we'll happily add the entry to that role?
Assuming I'm understanding that correctly, LGTM.

@endophage: yes -- we'll attempt to add to whichever role is specified, and it will only succeed if docker/notary has the appropriate private keys to sign


fix for upcoming use_build_context_synchronously changes

A recent fix to use_build_context_synchronously (https://github.com/dart-lang/linter/pull/3679) is flagging a few places that need addressing before publishing the new linter.
Fixes dart-lang/linter#3680

Awesome.  Thanks a million, @a14n!

test-exempt: fixes for lint


Makyen: Watch maryabbate128

Makyen requests the watch of the watch_keyword maryabbate128. See the Metasmoke search here and the Stack Exchange search here.

maryabbate128 has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Coverage remained the same at 65.091% when pulling 492a04bbd5218523f6a455de414dace65bcf107a on auto-blacklist-1524275997.5514219 into caa5f863ca666f887c6090b04f0bce243a4a96a7 on master.


Relative image paths not working?...

Hi,
I tried these links below:
from goose3 import Goose
g = Goose({'browser_user_agent': 'Mozilla'})

article = g.extract(url='https://www.cryptocompare.com/') #........[1]

article = g.extract(url='https://www.google.com/admob/businesskit/from-an-idea-to-an-app/') #........[2]

I get the following errors:
(for [1])
MissingSchema: Invalid URL 'https:/media/12318336/mnz.png?width=350': No schema supplied. Perhaps you meant http://https:/media/12318336/mnz.png?width=350?

(for [2])
MissingSchema: Invalid URL 'https:/admob/images/businesskit/interviews/marvinpaul.jpg': No schema supplied. Perhaps you meant http://https:/admob/images/businesskit/interviews/marvinpaul.jpg?

Perhaps the output should have been the following?...
(for[1])
https://www.cryptocompare.com/media/12318336/mnz.png?width=350

(for [2])
https://www.google.com/admob/images/businesskit/interviews/marvinpaul.jpg?

System info:
Windows 10
Python 3.5
goose3 installed from here: https://github.com/goose3/goose3
(The one given in the instructions is an old repo I guess?...)
Full traceback for [2]:
Traceback (most recent call last):

  File "<ipython-input-15-94ebba58016d>", line 3, in <module>
    article = g.extract(url='https://www.google.com/admob/businesskit/from-an-idea-to-an-app/')

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\__init__.py", line 81, in extract
    return self.crawl(cc)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\__init__.py", line 97, in crawl
    raise e

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\__init__.py", line 91, in crawl
    article = crawler.crawl(crawl_candidate)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\crawler.py", line 120, in crawl
    return self.process(raw_html, parse_candidate.url, parse_candidate.link_hash)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\crawler.py", line 190, in process
    self.get_image()

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\crawler.py", line 212, in get_image
    self.article.top_image = self.image_extractor.get_best_image(doc, top_node)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\extractors\images.py", line 82, in get_best_image
    image = self.check_large_images(topNode, 0, 0)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\extractors\images.py", line 135, in check_large_images
    depth_obj.parent_depth, depth_obj.sibling_depth)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\extractors\images.py", line 116, in check_large_images
    good_images = self.get_image_candidates(node)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\extractors\images.py", line 278, in get_image_candidates
    good_images = self.get_images_bytesize_match(filtered_images)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\extractors\images.py", line 295, in get_images_bytesize_match
    local_image = self.get_local_image(src)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\extractors\images.py", line 338, in get_local_image
    return ImageUtils.store_image(self.fetcher, self.link_hash, src, self.config)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\utils\images.py", line 66, in store_image
    data = http_client.fetch(src)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\goose3-3.0.6-py3.5.egg\goose3\network.py", line 54, in fetch
    response = self._connection.get(url, timeout=self.config.http_timeout, headers=self.headers)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\requests\sessions.py", line 488, in get
    return self.request('GET', url, **kwargs)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\requests\sessions.py", line 461, in request
    prep = self.prepare_request(req)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\requests\sessions.py", line 394, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\requests\models.py", line 294, in prepare
    self.prepare_url(url, params)

  File "D:\Applications\WinPython-64bit-<IP_ADDRESS>Qt5\python-3.5.3.amd64\lib\site-packages\requests\models.py", line 354, in prepare_url
    raise MissingSchema(error)

MissingSchema: Invalid URL 'https:/admob/images/businesskit/interviews/marvinpaul.jpg': No schema supplied. Perhaps you meant http://https:/admob/images/businesskit/interviews/marvinpaul.jpg?

@pavitrakumar78 I had a hunch that relative image URLs were not working but hadn't run into the issue myself. As such, I was already working on tracking down the root of the problem. Your examples helped immensely in tracking it down!

@barrust hey! Thanks for quick reply and fix! Tested your changes on my system and images are fetched fine for the links I posted. Should I leave this issue open until the changes have been merged?

I'm glad that it worked! I would leave this open until it gets merged so that it doesn't get lost!


Header blink white when rotate

Current Behavior
When you change backgroundColor of header headerStyle: { backgroundColor: '#000' } this blink white background when rotate.

Expected Behavior
Change backgroundColor of header should change all animated views Background.
How to reproduce
Change backgroundColor of the header to dark color and rotate.
Link to repository of example
Your Environment

software
version

react-navigation-stack
1.7.2

react-native
https://github.com/expo/react-native/archive/sdk-33.0.0.tar.gz

node
v8.16.0

npm or yarn
1.13.0

It was fixed by 2.0, waiting for release, thanks!


Cut new release

It'd be great if we could cut a new gimli version soon. addr2line needs #164 and #165 in order to compile, so it's currently required to use gimli as a git dependency. Once a new release is cut, it can move both gimli and owning-ref (following the release of 0.2.4) to regular, versioned dependencies, which will allow us to start pushing to crates.io again.
/cc @philipc @fitzgen

I published 0.11.0.


Feature/zenodo error fix

I talk more about the solution in the issue linked, but the gist is that we generate the zenodo token on every call, though on error case we generated a new token but didn't store it. This means you have to relink the account to fix it.
This PR adds two things:

on token generation, save to the db right away so that new token is saved even on error
only regenerate token after 1 hour has passed (expiry time of token)

This is a draft because it has no tests yet. Waiting for Walt to implement basic zenodo testing so that I can build off that.
https://github.com/dockstore/dockstore/issues/2787

@denis-yuen @coverbeck @garyluu @wshands It looks like it is only using one connection locally. I've made some updates based on feedback here.


ViAppendLine crashes when the cursor is at the end of the input, on an empty line.

As discussed in #796, the ViAppendLine method crashes when the cursor is at the very end of the input, on an empty line.

Thanks again for the contribution.


Better error message on The value of X has a wrong structure error

Is your feature request related to a problem? Please describe.
When writing client queries or mutations potential mistakes can be made, but there is no way to debug the issues, because the HotChocolate server always returns The value of $input has a wrong structure..
Describe the solution you'd like
It would be great if an exception would be raised with additional information
(located in DictionaryToObjectValueConverter#80). The exception or message should atleast containt field.Key or the whole ConverterContext

The input paser has been completly rewritten for this in v12.
This is now reslolved


Improve code coverage of LDAP* and AnyMap

Addresses part of the umbrella issue #235
For the following sources the coverage is expanded to all the reachable lines of the source files, which results in the total coverage percentage indicated by the numbers next to the files.
AnyMap.cpp : 93.81%
LDAPExpr.cpp : 97.77%
LDAPProp.cpp : 100.00 %
LDAPFilter.cpp : 98.36 %
Coverage delta for the framework/ dir : 85.83 (+2.89%)

Codecov Report

Merging #236 into development will increase coverage by 2.85%.
The diff coverage is 98.97%.

@@               Coverage Diff               @@
##           development     #236      +/-   ##
===============================================
+ Coverage        82.89%   85.75%   +2.85%
===============================================
  Files              187      189       +2
  Lines            13185    13498     +313
===============================================
+ Hits             10930    11575     +645
+ Misses            2255     1923     -332

Impacted Files
Coverage Δ

framework/src/util/LDAPExpr.cpp
97.77% <ø> (+25.65%)
:arrow_up:

framework/test/gtest/LDAPFilterTest.cpp
100% <100%> (ø)
:arrow_up:

framework/test/gtest/AnyMapTest.cpp
100% <100%> (ø)

framework/test/gtest/LDAPExprTest.cpp
98.1% <98.08%> (ø)

...k/include/cppmicroservices/detail/TrackedService.h
20% <0%> (-60%)
:arrow_down:

framework/include/cppmicroservices/ServiceEvent.h
80% <0%> (-20%)
:arrow_down:

...include/cppmicroservices/detail/TrackedService.tpp
64.28% <0%> (-14.29%)
:arrow_down:

...nclude/cppmicroservices/ServiceTrackerCustomizer.h
66.66% <0%> (-6.07%)
:arrow_down:

...include/cppmicroservices/detail/ServiceTracker.tpp
72% <0%> (-5.65%)
:arrow_down:

.../cppmicroservices/detail/BundleAbstractTracked.tpp
69.38% <0%> (-4.77%)
:arrow_down:

... and 19 more

Because the branch history is not that nice to look at, I recommend to squash it this time before merging.


Configuration Directory doesn't seem to work on Terraform CLI task

Hi,
In my Pipeline deployment service, I'm trying to do the following:

Extract the terraform files from a published build in in the destination folder : $(Agent.ReleaseDirectory)/Copernic/tfplan/Payload
Use the Terraform CLI task to run the terraform init command in that directory.

I'm assuming that the "Configuration Directory" setting is the directory where you want to run the terraform command. However, I keep getting the "terraform initialized in an empty directory!" error.
Command is ran in /usr/local/bin/terraform when I don't use the terraform version task on Ubuntu agent

Can you please advise whether I'm making a good use of this task or not?
Cheers

@Thitou can you rerun the release stage with the debug flag enabled. Please attach the log here.

Closing as there has been no response on this for several months


New plugins: first PR logstash-output-neo4j

This PR add the first code base for a prospect logstash-output-neo4j plugins that is basically storing events in a time based index within the database.
The code base support the ability to query the time based index in different ways.
As with the logstash-input-neo4j this project is in an early basis stage so some more feature can be added in the feature.

@suyograo @jsvd looking for a review so we can merge this new plugins.

Ran the tests, the code looks good but I'm not familiar enough with Neo4j to say that all the timetree stuff does what it's supposed to do.
1/2 LGTM. get another one :P

Since I did a small check on the input I have also look at the output.
I havent test them locally just did the core review.

Looking good @purbon ! :)

After some reviews I will land this in and work on extending it with new features :-) thanks @jsvd and @ph for the reviews.


Decide how to compute exact Hessians

We should decide how to compute exact Hessians. Hessians may be computed either 1) manually, 2) automatically, by JuMP/ReverseDiffSparse, or 3) automatically, by forking JuMP and modifying it to parse a BUGS-like specification of our model. To decide, it'd helpful to understand whether our objective function can be written in the current JuMP modeling language.

I think as a first step it might be worth trying BFGS preconditioning with only the diagonal of the true Hessian to get the problem "scale" right.  That will be easy to calculate by hand.

Great idea!

On second thought, if you manually compute the diagonal of the Hessian, convert and bounded problem to an unbounded one, and call BFGS with preconditioning, you've actually done a good chunk of the work needed to manually compute the exact Hessian and use Newton's method. If it's only going to take a week or so to get diagonally preconditioned BFGS running, you might as well just go for it. But if it's a bigger undertaking, you might want to spend like a day looking at what's possible with JuMP first. Because if JuMP can work for us and we use it, we'd presumably end up with really concise code, that would be easy to modify in the future...it takes the derivatives, it finds the Hessian, and then it calls a really mature optimization library (IpOpt) that naively support all the kinds of constraints we might want. We'd be able to delete OptimizeElbo.jl almost entirely, ElboDeriv.jl might become half its current length, and we wouldn't need a lot of the stuff in CelesteTypes.jl either.

I think right now it might take me a week just because I'm still getting
used to the basics of Julia, which won't be mitigated by looking into JuMP.
On Mon, Feb 23, 2015 at 11:28 AM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

On second thought, if you manually compute the diagonal of the Hessian,
convert and bounded problem to an unbounded one, and call BFGS with
preconditioning, you've actually done a good chunk of the work needed to
manually compute the exact Hessian and use Newton's method. If it's only
going to take a week or so to get diagonally preconditioned BFGS running,
you might as well just go for it. But if it's a bigger undertaking, you
might want to spend like a day looking at what's possible with JuMP first.
Because if JuMP can work for us and we use it, we'd presumably end up with
really concise code, that would be easy to modify in the future...it takes
the derivatives, it finds the Hessian, and then it calls a really mature
optimization library (IpOpt) that naively support all the kinds of
constraints we might want. We'd be able to delete OptimizeElbo.jl almost
entirely, ElboDeriv.jl might become half its current length, and we
wouldn't need a lot of the stuf f in CelesteTypes.jl either.
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-75613285
.

That's fine, it sounds to me like more than a week of work, even if you were familiar with Julia. My point was just that JuMP + IpOpt and manually computed second derivatives + BFGS are alternatives...makes sense to at least consider whether JuMP could work before spending a lot of time on a competing (and likely tedious) approach. I could spend some time later this week or next week evaluating whether JuMP might work for our problem.

Learning about JuMP sounds like more fun than computing second derivatives,
so if you think it's best to start there I'll check it out before doing any
calculations of my own.
On Tue, Feb 24, 2015 at 6:27 PM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

That's fine, it sounds to me like more than a week of work, even if you
were familiar with Julia. My point was just that JuMP + IpOpt and manually
computed second derivatives + BFGS are alternatives...makes sense to at
least consider whether JuMP could work before spending a lot of time on a
competing (and likely tedious) approach. I could spend some time later this
week or next week evaluating whether JuMP might work for our problem.
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-75894118
.

I spent some time learning about JuMP (and how to debug Julia package install errors) today.  I am tentatively optimistic that it would be possible to do Celeste in JuMP, but based on the basic documentation at least I suspect it would not be a quick re-write of the existing code.  But maybe I've missed something or you can think of some meta-programming tricks.
The problem as I see it is how to keep the structure of ElboDeriv.jl but have everything assign JuMP expression objects rather than return values.  As far as I can tell, you can't do something like
function MyFun(x)
   x + 5
end
@defNLExpr(x_plus_5, MyFun(x))

However, you can chain together the result of defNLExpr.  So you could do something like
function AddFive(i)
  @defNLExpr(x_plus_5[i], x[i] + 5)
end

function Square(i)
  @defNLExpr(x_plus_5_squared[i], x_plus_5[i]^2
end

It would be nice to do that without having to index everything, i.e. to have something like the imaginary
function AddFiveAndStore(x, result_name)
  @def(result_name, x + 5)
end

Maybe there's some metaprogramming thing that would let you do that.  Though as Miles mentioned, Julia doesn't handle flat sums very well, so that might not be advisable anyway.
Finally, even though this seems possible in principle, I couldn't really get even pretty simple things like this to actually work.  (This is partly, I suspect, because NLOpt doesn't play very nicely with JuMP and I have been unable to get Ipopt to work on my mac due to persistent fortran compiler errors.)  For example, the very simple optimal control example from the website does not run with NLOpt, and the error messages are pretty cryptic.
So in short, I think JuMP is doable, but not faster than doing the Hessian diagonal by hand.  How about I try some basic stuff on linux when I go to Berkeley tomorrow, and if the above ideas are actually possible I email Miles, but in the meantime focus on trying to get unconstrained pre-conditioned BFGS working so you can make quicker progress on the galaxy model?

I spent a bit of time looking at JuMP too...don't want to step on your toes here....just wanted to make sure we're going the right direction while you're getting started. Check out this toy program:
using JuMP
using Ipopt

const N = 5000000
const A = randn(N);

m = Model(solver=IpoptSolver())
@defVar(m, -10 <= x <= 10)
@setNLObjective(m, Min, sum{exp(x*A[i]), i=1:N})   #This is like the sum over pixels in our objective.
solve(m)

It's amazing...like 5--15 iterations until it converges, all in less than 1 minute on a very old laptop. The toy problem above is simple, but it's not unlike our actual objective function: a sum over ~5 million pixels, expensive calls to exp. I also tried  calling Ipopt directly, with manually computed gradients/hessian for this toy problem---it's no faster.
If you exclude all the derivatives and comments from ElboDeriv.jl our objective is only like 100-200 lines of code. Should be a pretty straightforward mapping from it to JuMP syntax (fingers crossed), making heavy use of JuMP's @defExpr macro. Do you want to try working on that tomorrow?

That Fortran compiler issue was a nightmare for me too. On linux (and probably on mac too), it's solved by deleting lib/julia/libgfortran* from your julia directory. Compiling Julia from source, rather than downloading the binaries also solves the problem.

Hm, I just tried deleting libgfortran.dylib and that didn't fix the
problem.  (I'm pretty sure it's using the library in /usr/lib anyway
because one of the earlier problems was it being unable to find that
library.)
If your'e curious, I opened an issue
https://github.com/JuliaOpt/Ipopt.jl/issues/30#issuecomment-76127699 with
more information.
On Wed, Feb 25, 2015 at 11:35 PM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

That Fortran compiler issue was a nightmare for me too. On linux (and
probably on mac too), it's solved by deleting lib/julia/libgfortran* from
your julia directory. Compiling Julia from source, rather than downloading
the binaries also solves the problem.
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76134342
.

Ok, cool.  I like the idea of JuMP if we can get it to work.  I'm hamstrung
until I can get Ipopt installed (though I'm optimistic that my linux laptop
will be easier).
I made a JuMP branch on the repo with some screwing around.  Maybe we can
both use that branch.  I'll be more convinced if we can do something like
least squares in the following way.
@defVar(m, beta[1:k])
function SetResidual(i)
@defExpr(eps[i], y[i] - dot(beta, vec(x[i, :]))
end
for i = 1:n
SetResidual(i)
end
@setNLObjective(m, Min, sum{eps^2, i=1:n})
solve(m)
(This is nothing you'd ever actually do, of course, it's just a kind of
proof of concept.)
On Wed, Feb 25, 2015 at 11:32 PM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

I spent a bit of time looking at JuMP too...don't want to step on your
toes here....just wanted to make sure we're going the right direction while
you're getting started. Check out this toy program:
using JuMP
using Ipopt
const N = 5000000
const A = randn(N);
m = Model(solver=IpoptSolver())
@defVar(m, -10 <= x <= 10)
@setNLObjective(m, Min, sum{exp(x*A[i]), i=1:N})   #This is like the sum over pixels in our objective.
solve(m)
It's amazing...like 5--15 iterations until it converges, all in less than
1 minute on a very old laptop. The toy problem above is simple, but it's
not unlike our actual objective function: a sum over ~5 million pixels,
expensive calls to exp. I also tried calling Ipopt directly, with
manually computed gradients/hessian for this toy problem---it's no faster.
If you exclude all the derivatives and comments from ElboDeriv.jl our
objective is only like 100-200 lines of code. Should be a pretty
straightforward mapping from it to JuMP syntax (fingers crossed), making
heavy use of JuMP's @defExpr macro. Do you want to try working on that
tomorrow?
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76134083
.

With Ipopt in hand, I got some working examples.  I'd like you to take a look, though.  Consider the code below, which doesn't work.  The error is shown -- eps_squared is not actually being defined inside the function.  Note that if this had been done in the main body of the code, i.e. not inside the function, this would have worked fine.  (See other examples in the haphazardly organized JuMPExample.jl.)
Some scoping thing?  Any thoughts?
using JuMP
n = 100
k = 3
x = randn(n, k)
true_beta = randn(k)
y = x * true_beta + randn(n)

function CalculateEpsilon(index)
  @defNLExpr(eps_squared[index], (y[index] - sum{x[index, j] * beta[j], j=1:k})^2)
end
m = Model()
@defVar(m, -10 <= beta[1:k] <= 10)
for i = [n]
  CalculateEpsilon(i)
end
@setNLObjective(m, Min, sum{eps_squared[i], i=1:n})
# ERROR: eps_squared not defined
status = solve(m)

One more concern is whether JuMP objects make a deep copy of the data that is used to construct them.  Consider this, for example:
m = Model()
big_data = randn(100)
@defVar(m, big_parameter)
@defNLExpr(big_transform[i=1:100], big_parameter * big_data[i]^2)
big_data = -5
print(big_transform)
print(big_data)

You can see that big_transform still contains all the data from big_data.  If you were to define another expression that used big_transform, maybe it, too, would contain a copy of big_data.  My Julia chops are not strong enough to check this directly, though, and maybe I'm missing something.  Any ideas for how to check?

Unlike @defVar, etc., @defNLExp doesn't take model m as it's first argument. So if you put a call to @defNLExpr in a for loop, it won't mutate anything outside of the body of the for loop.
Here's working code:
using JuMP
using Ipopt

n = 100
k = 3
x = randn(n, k)
true_beta = randn(k)
y = x * true_beta + randn(n)

m = Model()
@defVar(m, -10 <= beta[1:k] <= 10)
@defNLExpr(eps_squared[index=1:n], (y[index] - sum{x[index, j] * beta[j], j=1:k})^2)
@setNLObjective(m, Min, sum{eps_squared[i], i=1:n})
status = solve(m)

Everything else worked once I fixed that. Nicely done!

I was concerned that JuMP wouldn't do things in efficient manner too, but I'm much less so now that I've run that toy example with 5,000,000 pixels that I posted yesterday. It finished as fast as my hand coded derivatives, on a laptop with only 2 GB of RAM. 2x or even 10x too much memory isn't so bad....it's the O(n) memory inefficiencies, where n is the number of pixels, that I worry most about. (For speed on the other hand 2x overhead is ok, but 10x isn't.) As long as we can process like 10-megapixels worth of images in at most few minutes, I think we're in business!

So if you cant define an NLExpr in a function, do you think the ELBO can be
cleanly written as a sequence of sums without using any functions?
-- Ryan.  Sent from a phone.  Nevertheless, all typos are deliberate.
On Feb 26, 2015 5:21 PM, "Jeffrey Regier"<EMAIL_ADDRESS>wrote:

Unlike @defVar, etc., @defNLExp doesn't take model m as it's first
argument. So if you put a call to @defNLExpr in a for loop, it won't
mutate anything outside of the body of the for loop.
Here's working code:
using JuMP
using Ipopt
n = 100
k = 3
x = randn(n, k)
true_beta = randn(k)
y = x * true_beta + randn(n)
m = Model()
@defVar(m, -10 <= beta[1:k] <= 10)
@defNLExpr(eps_squared[index=1:n], (y[index] - sum{x[index, j] * beta[j], j=1:k})^2)
@setNLObjective(m, Min, sum{eps_squared[i], i=1:n})
status = solve(m)
Everything else worked once I fixed that. Nicely done!
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76313355
.

I think the code above is cleaner without the one line function, because it only gets called once. I'm not sure about the ELBO, but less nesting could be better. A lot of quantities are just sums of something else: the elbo is the sum a likelihood and a kl. The likelihood is a sum over 5 band-specific likelihood. Each band specific likelihood is a sum over tiles. Each tile is a sum over pixels. All that might work pretty well as a flat list of commands. You might try working backwards. You can also use constraints when necessary rather than expressions. One constraint per tile even should be ok, but one constraint per pixel would likely be a problem.

So if it's making a deep copy (and it very well may not be) it seems to
me that the memory problem is on the order of number of pixels * amount of
prior data * # of intermediate expressions, and that would be a problem.
For example, each pixel's color ELBO term is a function of the pre-computed
color priors.  If you put that ELBO term in an intermediate expression, and
JuMP stores that expression with a deep copy, then instead of one integer
per pixel (the raw poisson counts) you now have a JuMP expression
containing a copy of the whole color prior dataset, containing many floats,
for each pixel.  The same is true for the PSF prior and galaxy prior.
A test for this would be to take your 5M pixel problem and generate a bunch
of intermediate expressions that are functions of the raw pixel sizes and
see what happens.  For example, do the min of something like
@setNLObjective(m, Min, sum{alpha_x[i] + beta_x[i] + gamma_x[i], i=1:n})
...where each inner term is a JuMP NLExpr, like
@defNLExpr(alpha_x[i=1:5000000], alpha * x[i])
@defNLExpr(beta_x[i=1:5000000], beta^2 * x[i]^2)
@defNLExpr(gamma_x[i=1:5000000], gamma^3 * x[i]^3)
If that uses the same amount of RAM or only marginally more than the first
version, I'll be convinced.
On Thu, Feb 26, 2015 at 5:32 PM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

I was concerned that JuMP wouldn't do things in efficient manner too, but
I'm much less so now that I've run that toy example with 5,000,000 pixels
that I posted yesterday. It finished as fast as my hand coded derivatives,
on a laptop with only 2 GB of RAM. 2x or even 10x too much memory isn't so
bad....it's the O(n) memory inefficiencies, where n is the number of
pixels, that I worry most about. (For speed on the other hand 2x overhead
is ok, but 10x isn't.) As long as we can process like 10-megapixels worth
of images in at most few minutes, I think we're in business!
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76314544
.

I think the code above is cleaner without the one line function, because
it only gets called once.

Sure, this was just an example to see if you could use functions at all in
this way.  If you could, then we could maybe move to JuMP with something
that is much like the code as-is.
Without functions the ELBO in JuMP would look much different than it does
now.  Of course, you're much more familiar with it than I am so your
intuition about what's easy and what's not is surely better than mine.

Good news, it looks like JuMP is not being as wasteful as I was worried
about.  The amount of memory allocated by JuMP (above and beyond the
original dataset) increases linearly with the amount of data, but adding
extra @NLExpr definitions doesn't increase that memory usage.
The graph shows two models optimizing sum(exp(alpha * x[i])), one of which
has two NLExpr definitions (one for alpha * x[i] and one for exp(of that)),
and one with only one  (the whole thing in one expression).  The x-axis is
the log10 of the data size and the y the log10 of the memory JuMP uses.
The two models are pretty much the same.  (NB I couldn't figure out how to
plot multiple colors yet.)   Code is in the JuMP branch.
So that's promising!
On Thu, Feb 26, 2015 at 6:03 PM, Ryan Giordano<EMAIL_ADDRESS>wrote:

So if it's making a deep copy (and it very well may not be) it seems to
me that the memory problem is on the order of number of pixels * amount of
prior data * # of intermediate expressions, and that would be a problem.
For example, each pixel's color ELBO term is a function of the
pre-computed color priors.  If you put that ELBO term in an intermediate
expression, and JuMP stores that expression with a deep copy, then instead
of one integer per pixel (the raw poisson counts) you now have a JuMP
expression containing a copy of the whole color prior dataset, containing
many floats, for each pixel.  The same is true for the PSF prior and galaxy
prior.
A test for this would be to take your 5M pixel problem and generate a
bunch of intermediate expressions that are functions of the raw pixel sizes
and see what happens.  For example, do the min of something like
@setNLObjective(m, Min, sum{alpha_x[i] + beta_x[i] + gamma_x[i], i=1:n})
...where each inner term is a JuMP NLExpr, like
@defNLExpr(alpha_x[i=1:5000000], alpha * x[i])
@defNLExpr(beta_x[i=1:5000000], beta^2 * x[i]^2)
@defNLExpr(gamma_x[i=1:5000000], gamma^3 * x[i]^3)
If that uses the same amount of RAM or only marginally more than the first
version, I'll be convinced.
On Thu, Feb 26, 2015 at 5:32 PM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

I was concerned that JuMP wouldn't do things in efficient manner too, but
I'm much less so now that I've run that toy example with 5,000,000 pixels
that I posted yesterday. It finished as fast as my hand coded derivatives,
on a laptop with only 2 GB of RAM. 2x or even 10x too much memory isn't so
bad....it's the O(n) memory inefficiencies, where n is the number of
pixels, that I worry most about. (For speed on the other hand 2x overhead
is ok, but 10x isn't.) As long as we can process like 10-megapixels worth
of images in at most few minutes, I think we're in business!
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76314544
.

I wrote and checked in a JuMP implementation of a toy version of the model (1-d, point galaxies, one band, single gaussian psf).  I do think it looks promising.
I came across two issues.  One is the lgamma() function doesn't seem to be supported or at least have an error in it.  The other is stranger, and I have posted an issue about it:
https://github.com/JuliaOpt/JuMP.jl/issues/406

This is great progress! And I'm glad the issue you posted about got fixed so quickly...the JuMP maintainers have been really helpful.
I think I see how to get the lgamma function working. JuMP loads differentiation rules from Calculus.jl. Calculus.jl lists digamma(x) as the derivative of lgamma(x), but it doesn't list a derivative for digamma(x), so JuMP doesn't know that polygamma(1, x) is the second derivative of lgamma(x). The following addition of one line to Calculus/src/differentiate.jl remedies this:
diff --git a/src/differentiate.jl b/src/differentiate.jl
index 49c9270..4fe3abe 100644
--- a/src/differentiate.jl
+++ b/src/differentiate.jl
@@ -166,6 +166,7 @@ symbolic_derivative_1arg_list = [
     ( :erfi,        :(  2 * exp(square(x)) / sqrt(pi)           ))
     ( :gamma,       :(  digamma(x) * gamma(x)                   ))
     ( :lgamma,      :(  digamma(x)                              ))
+    ( :digamma,     :(  polygamma(1, x)                         ))
     ( :airy,        :(  airyprime(x)                            ))  # note: only covers the 1-arg version
     ( :airyprime,   :(  airy(2, x)                              ))
     ( :airyai,      :(  airyaiprime(x)                          ))

Oh, awesome.  Do you mind if I post your response to the issue in JuMP.jl
https://github.com/JuliaOpt/JuMP.jl/issues/407?
I guess this should be a pull request to Calculus.jl.  Do you want to do
that or let the JuMP guys do it?  Or should I give it a shot?  (I may have
to tap you if there are questions but I'd be happy to give it a go.)
On Sat, Feb 28, 2015 at 11:59 AM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

This is great progress! And I'm glad the issue you posted about got fixed
so quickly...the JuMP maintainers have been really helpful.
I think I see how to get the lgamma function working. JuMP loads
differentiation rules from Calculus.jl. Calculus.jl lists digamma(x) as
the derivative of lgamma(x), but it doesn't list a derivative for
digamma(x), so JuMP doesn't know that polygamma(1, x) is the second
derivative of lgamma(x). The following addition of one line to
Calculus/src/differentiate.jl remedies this:
diff --git a/src/differentiate.jl b/src/differentiate.jl
index 49c9270..4fe3abe 100644
--- a/src/differentiate.jl
+++ b/src/differentiate.jl
@@ -166,6 +166,7 @@ symbolic_derivative_1arg_list = [
( :erfi,        :(  2 * exp(square(x)) / sqrt(pi)           ))
( :gamma,       :(  digamma(x) * gamma(x)                   ))
( :lgamma,      :(  digamma(x)                              ))

( :digamma,     :(  polygamma(1, x)                         ))
( :airy,        :(  airyprime(x)                            ))  # note: only covers the 1-arg version
( :airyprime,   :(  airy(2, x)                              ))
( :airyai,      :(  airyaiprime(x)                          ))

—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76543042
.

In the meantime, thanks to JuMP's quick fix, a rudimentary likelihood
maximizer on this toy problem (sans priors or entropy due to the lgamma
problem) is working JuMPGammaPoisson.jl in the JuMP branch of the repo.
Want to check it out?  I don't yet have an intuitive sense of how fast
something like this should be, but on my laptop it can do 5k points in
about 10 minutes.
I think I need to work on my NIPS paper for the rest of the weekend, so I
may have to pick this up again next week.
Yeah, it's very encouraging how responsive the JuMP team was.
On Sat, Feb 28, 2015 at 12:07 PM, Ryan Giordano<EMAIL_ADDRESS>wrote:

Oh, awesome.  Do you mind if I post your response to the issue in JuMP.jl
https://github.com/JuliaOpt/JuMP.jl/issues/407?
I guess this should be a pull request to Calculus.jl.  Do you want to do
that or let the JuMP guys do it?  Or should I give it a shot?  (I may have
to tap you if there are questions but I'd be happy to give it a go.)
On Sat, Feb 28, 2015 at 11:59 AM, Jeffrey Regier<EMAIL_ADDRESS>
wrote:

This is great progress! And I'm glad the issue you posted about got fixed
so quickly...the JuMP maintainers have been really helpful.
I think I see how to get the lgamma function working. JuMP loads
differentiation rules from Calculus.jl. Calculus.jl lists digamma(x) as
the derivative of lgamma(x), but it doesn't list a derivative for
digamma(x), so JuMP doesn't know that polygamma(1, x) is the second
derivative of lgamma(x). The following addition of one line to
Calculus/src/differentiate.jl remedies this:
diff --git a/src/differentiate.jl b/src/differentiate.jl
index 49c9270..4fe3abe 100644
--- a/src/differentiate.jl
+++ b/src/differentiate.jl
@@ -166,6 +166,7 @@ symbolic_derivative_1arg_list = [
( :erfi,        :(  2 * exp(square(x)) / sqrt(pi)           ))
( :gamma,       :(  digamma(x) * gamma(x)                   ))
( :lgamma,      :(  digamma(x)                              ))

( :digamma,     :(  polygamma(1, x)                         ))
( :airy,        :(  airyprime(x)                            ))  # note: only covers the 1-arg version
( :airyprime,   :(  airy(2, x)                              ))
( :airyai,      :(  airyaiprime(x)                          ))

—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/7#issuecomment-76543042
.


2x LED Strips with same amount LEDs - Color Problem

Hi,
i got two LED stripes paralell with the same amount of LEDs. One is working fine, the spectrum effect is purple but on the other LED stripe it uses all colors... The spectrum effect is working
What can cause the problem?

My guess is that one of the strips is RGB (or BGR or whatever) and the other one is RGBW, i.e. it expects four values per pixel. Sending RGB data to such a strip means that it takes the R value of the second pixel as the white value of the first, the GB of the second and the RG of the third for its second pixel... Those two strips cannot be used in parallel.

Sorry to mention but the second strip is just the left over from the first one.
But somehow it fixed itself after executing  the led.py


Logout button component

Implement a logout button component to use across all pages.

Implemented and available as <SignOutButton />


Added files has no reference in the project

The following files aren't referenced in the project and causing error on compiling:
ASN1DistinguishedNames.swift
X509Extension.swift
X509PublicKey.swift

Thanks for the hint. It should be fixed now.


Text not appearing on AFRAME 1.3.0

The a-gui-button text not appearing, only shows this error above.
It only works on AFRAME 1.0.3.

Hello,
The solution that worked for me is to remove the build the code without 'third-party' folder in index.js. Then load the latest of aframe-troika
   <script src="https://unpkg.com/aframe-troika-text/dist/aframe-troika-text.min.js"></script>

You can check this pull request #65


Strange utf-8 error

Describe the bug
I've imported a large Parquet file. In Rust I'm getting this error:
FromUtf8Error(FromUtf8Error { bytes: [48, 48, 205, 204, 204, 204, 204, 204, 244, 63, 1, 110, 154, 153, 153, 153, 153, 153, 40, 64, 51, 51, 51, 51, 51, 179, 75, 64, 0, 0, 0, 0, 0, 0, 36, 64, 3, 3, 111, 119, 110, 5, 49, 50, 50, 51, 53, 205, 204, 204, 204, 204, 204, 244], error: Utf8Error { valid_up_to: 2, error_len: Some(1) } })

I have a struct type with field Vec<u8> and upon calling String::from_utf8(field) on that field, I get this error.
I tried String type for the field, but then I get a cryptic message about invalid encoding coming deep from clickhouse-rs. Anyways, the clickhouse-cli has no problems with this field, same for the JS clickhouse client.
Steps to reproduce
I'm not able to provide detailed data, however there are no non-ascii encodings in that field.
When I invoke SELECT 'x' AS myfield I get correct "x" string, but when I do SELECT field (which is of String type), I get that utf-8 error.
Expected behaviour
No error.
Configuration
Environment

Client version: 0.13.1
OS: Linux (docker)

ClickHouse server

ClickHouse Server version: 24.10.2

CREATE TABLE statements for tables involved: well, DESCRIBE mytable returns myfield as String

Can you provide your struct and client.query() call?
Then, I don't understand what you mean by "I've imported a large Parquet file" and how it's related to the problem described.
Also, if you're sure that that column contains valid utf8 you should prefer myfield: String instead of myfield: Vec<u8> (or use serde_bytes as described in the documentation to avoid performance overhead).

My struct is like this:
#[derive(Debug, Row, Deserialize)]
struct CHObservation {
    // station_id: String,
    station_id: Vec<u8>,
    value: f64,
    // name: String,
    name: Vec<u8>,
    latitude: f64,
    longitude: f64,
    height: f64,
    score: f64,
    // owner_id: String,
    owner_id: Vec<u8>,
}

I tried String but it didn't work as well. The advantage of Vec<u8> seems that I can postpone to calling String::from_utf8 in my code and so I have an insight at the bytes that were received by the client. These bytes don't make sense for me though, the stationId strings are short ASCII codes.
My query is of this form:
 let q = r#"
       SELECT observations_ts.stationId AS station_id,
              1.3 AS value,
              'n' AS name,
              12.3 AS latitude,
              55.4 AS longitude,
              10.0 AS height,
              3 AS score,
              'own' AS owner_id
          FROM observations_ts
            WHERE ts BETWEEN (toDateTime(?) - toIntervalSecond(?)) AND (toDateTime(?) + toIntervalSecond(?))
       "#;
    let mut rows = db
        .query(&q)
        // .bind(Identifier(param_name))
        .bind(ti.t.to_string())
        .bind(ti.d.num_seconds())
        .bind(ti.t.to_string())
        .bind(ti.d.num_seconds())
        // .fetch::<(Vec<u8>, f64, Vec<u8>, f64, f64, f64, f64, Vec<u8>)>()?;
        .fetch::<CHObservation>()?;

When I specify some dummy string 'x' instead of observations_ts.stationId in the query, this works.
Mentioning Parquet file isn't directly related, but I had a thought that there might be some issues encoding/decoding strings (the Parquet file itself is generated as a dump from a DuckDB table; I can read that Parquet file in duckdb just fine).

Also, as you see from commented code above, I tried to just use a tuple instead of custom Row derive, as all examples I've seen have the Row and SELECT ?fetch and I use some more complex query which doesn't directly map field names to sql columns and I had a thought that the fields being deserialized might be in wrong places. But it didn't work as well.

duplicate of #173

OK I created a side project with just the clickhouse and that query and I played a bit with it, adding fields one by one. It seems that it works now but if you mistype some field (e.g. specify f64 as u8), you'll get utf8 encoding errors. This is quite misleading...


Look into JS library for supporting various devices like phones

e.g., bootstrap.js?

Duplicate of #26

Duplicate of #26


Remove Jetty as a runtime dependency of Javalin

Make Javalin run without Jetty, to be able to be embedded into other servlet containers. Fixes https://github.com/tipsy/javalin/issues/402 .

Looking at this, I'm wondering if it would be better to extend Javalin. You could probably avoid these checks that way. Might need to make the Jetty stuff protected rather than private.

With this approach you could also override irrelevant methods:
public class ServletExposingJavalin extends Javalin {

    public JavalinServlet createServlet() {
        return new JavalinServlet(
            this,
            pathMatcher,
            exceptionMapper,
            errorMapper,
            debugLogging,
            requestLogger,
            dynamicGzipEnabled,
            autogeneratedEtagsEnabled,
            defaultContentType,
            maxRequestCacheBodySize,
            prefer405over404,
            singlePageHandler,
            null
        );
    }

    @Override
    public Javalin enableStaticFiles(@NotNull String classpathPath) {
        throw new RuntimeException("Cannot use static files in standalone mode");
    }

    @Override
    public Javalin contextPath(@NotNull String contextPath) {
        throw new RuntimeException("Cannot use context-path standalone mode");
    }

    @Override
    public Javalin start() {
        throw new RuntimeException("Cannot start server in standalone mode");
    }

}

🤔

Can be made a lot less verbose if you use Kotlin:
class ServletExposingJavalin : Javalin() {

    fun createServlet() = JavalinServlet(
        this,
        pathMatcher,
        exceptionMapper,
        errorMapper,
        debugLogging,
        requestLogger,
        dynamicGzipEnabled,
        autogeneratedEtagsEnabled,
        defaultContentType,
        maxRequestCacheBodySize,
        prefer405over404,
        singlePageHandler,
        null
    )

    override fun enableStaticFiles(classpathPath: String) = notAvailable("enableStaticFiles()")
    override fun contextPath(contextPath: String) = notAvailable("contextPath()")
    override fun start() = notAvailable("start()")
    private fun notAvailable(action: String): Nothing = throw RuntimeException("$action is not available in standalone mode")

}

I'm not sure if Java will try loading the classes of the parent though, could you test?

I believe that extending class shouldn't remove a functionality from its superclass. Ideally Javalin's code should be moved to e.g. JavalinBase; but then we need generics to make all JavalinBase methods eventually return Javalin. So I guess your approach is simpler.

It's better than having methods that silently fail, IMO. I know it's not clean, I'm just trying to make the smallest change we can get away with in order to enable this feature. We'll have to rewrite it for 3.0, so I'm trying to keep things simple now.

I propose to name the class EmbeddedJavalin instead of ServletExposingJavalin.

This won't be a widely used feature, so I don't have strong feelings on the name. I'm fine with EmbeddedJavalin or EmbeddableJavalin, or even things like JavalinServletBuilder.

@mvysny Do you think you can finish this before the weekend? I'm considering releasing 2.4.0 on Saturday or Sunday.

@tipsy I think the current state of the PR should be according to your requirements. I can't remove the null assignment to the jetty server unfortunately, since filling it up with an actual Jetty server instance would make it crash with NoClassDefFoundError outright when running from Tomcat.
Please let me know if there is anything else I can improve.

@mvysny the methods I used in my code snippets were just examples, there are a lot more of them. Almost everything from https://javalin.io/documentation#configuration needs to be overridden. I really want to avoid the null values in Javalin, let me play around with it a bit locally one of the next few days.

I've been unexpectedly busy this week, but I'll try to have a look in the weekend.

I was wondering if we could create a factory method EmbeddedJavalin.create() which cleared the jetty-fields of Javalin using reflection, before creating an instance. I couldn't quite get there though.

Sorry, I've been quite busy myself too. But I got this idea that we could pass jettyServer and jettySessionHandler as constructor parameters! And indeed it seem to work pretty nicely, so I was able to remove the now-useless null check. What do you think?

Great, this feels a lot cleaner than the previous solution!

The (un)helpful message override should be moved into the constructor of EmbeddedJavalin, users shouldn't have to do that themselves.

You're right - please review :)

I've been playing around a bit with this. My suggested overrides are pretty useless since any method on Javalin will return the a Javalin object, which doesn't have these overrides. Brain fart.
What we can do instead of build on ensureActionIsPerformedBeforeServerStart and create something like ensureHasJettyServer ? We could also just skip this, and let users figure this out on their own, as it's a pretty niche use case.. In which case you can just remove the overrides and I'll merge the PR.

Hi, actually in my case overrides work:
    val javalin = EmbeddedJavalin()
            .get("/rest") { ctx -> ctx.result("Hello!") }
            .start()
            .createServlet()

This code correctly fails with:
 java.lang.RuntimeException: start() is not available in standalone mode
	at io.javalin.EmbeddedJavalin.notAvailable(EmbeddedJavalin.kt:53)
	at io.javalin.EmbeddedJavalin.start(EmbeddedJavalin.kt:52)

The reason is that get() returns this which is EmbeddedJavalin.
With that being said, I would advise against investing a lot of time into this. The reason is that this is just a hack (if done properly it should be the other way around, there should be JettyJavalin adding support for Jetty). Since you're going to refactor this anyway in 3.0.0, maybe we could keep it as is (or maybe add a few more overrides - I can do that yet please list the methods to override).
The reason why I'm pushing for this: I'm planning to ditch RESTEasy in Vaadin-on-Kotlin and replace it with Javalin; VoK apps are primarily WARs which run in servlet environment.

Do you also get the depreciations? I couldn't see them when checking out your code locally, the type was inferred to just Javalin.

I added these at least:
@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun contextPath(contextPath: String) = notAvailable("contextPath()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun enableStaticFiles(classpathPath: String) = notAvailable("enableStaticFiles()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun enableWebJars() = notAvailable("enableWebJars()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun port() = notAvailable("port()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun port(port: Int) = notAvailable("port(port)")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun server(server: Supplier<Server>) = notAvailable("server()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun sessionHandler(sessionHandler: Supplier<SessionHandler>) = notAvailable("sessionHandler()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun start() = notAvailable("start()")

@Deprecated("Not available in standalone mode", level = DeprecationLevel.ERROR)
override fun stop() = notAvailable("stop()")

Ah, yes, the @Deprecated annotations are useless, we can remove them. Let me create a PR

Done - @tipsy can you please review?

Added a few changes in https://github.com/tipsy/javalin/commit/848169cb3d03e128f4cc85efadb960eb6334e8a2, mostly formatting.

Awesome, thank you very much :+1:

Yes, it will

@mvysny This change is out as part of 2.4.0 now: https://javalin.io/news/2018/11/04/javalin-2.4.0-released.html

@tipsy Awesome, thank you for letting me know :+1:

It's an interesting feature, thanks for working on it!

@mvysny Have you taken this into use yet? Running into any problems? Would you be interested in writing a tutorial for https://javalin.io at some point?

@tipsy I have. In fact I've replaced RESTEasy with Javalin and it has been working really nicely. I've already updated the documentation in my project here: https://github.com/mvysny/vaadin-on-kotlin/tree/master/vok-rest . Regarding the tutorial - sure, what do you have in mind?

@mvysny @tipsy I have. In fact I've replaced RESTEasy with Javalin and it has been working really nicely. I've already updated the documentation in my project here: https://github.com/mvysny/vaadin-on-kotlin/tree/master/vok-rest .

Happy to hear that, seems like a good use-case.

Regarding the tutorial - sure, what do you have in mind?

Nothing fancy, just a simple guide showing how to pull in tomcat, exclude jetty, and set up the Servlet. Preferably with an example repo people can clone. There were a few people asking about this before you implemented the functionality, so I think it would be good to show it off in a tutorial.

Sure, that I can do. I can use this example repo: https://gitlab.com/mvysny/javalin-tomcat-embed-test ; I can write a simple guide no worries. Could you point me out to Javalin docs sources, so that I can create a PR for those please?

Great! Source is https://github.com/javalin/javalin.github.io.

Thanks @tipsy created a PR with the first draft: https://github.com/javalin/javalin.github.io/pull/20


Fix reading bad channels

Reference issue
Fixes: 11832
What does this implement/fix?
Fixes bug where read_raw_cnt() does not read bad channels

Actually, I think I need to do some more investigating. The sample data set already includes a test to check for bad channels and it uses byte 5 instead of 15 and it passes. However, the documentation cites it is located at byte 15. http://paulbourke.net/dataformats/eeg/sethead.h

Looks like this is where we look for bad channel names
https://github.com/mne-tools/mne-python/blob/74f3f18c82d3548e12833712ac6fd6074f1a3005/mne/io/cnt/cnt.py#L345-L351

Is @jaeilepp still around?

No and this was implemented a long time ago IIRC so I doubt there would be much memory remaining. Maybe older vs newer versions of the file spec or something?

I thought so too, but the strange thing was that a comment written at that
time cites documentation with the new file format.
On Tue, Jul 25, 2023 at 8:30 AM Eric Larson @.***>
wrote:

Is @jaeilepp https://github.com/jaeilepp still around?
No and this was implemented a long time ago IIRC so I doubt there would be
much memory remaining. Maybe older vs newer versions of the file spec or
something?
—
Reply to this email directly, view it on GitHub
https://github.com/mne-tools/mne-python/pull/11833#issuecomment-1649852547,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE33KTM2BRMK5HPULTNE77TXR7DAJANCNFSM6AAAAAA2WHTLIU
.
You are receiving this because you authored the thread.Message ID:
@.***>

We find bugs in our code all the time :) If the documentation and the results on files where you know what to expect in terms of good/bad channels say we're doing something wrong currently, then we probably are

My change will fail reading that sample data. Should I remove it?

Have you checked to see if that file has any version information in it? That way we could triage which bad-channel-reading method to use. Otherwise we could use a try/except but that's a bit uglier / brittle.

It does not have any version info. Another discrepancy is that the file has 128 channels, but when I try reading it into Neuroscan it states that it does not support 128 channels.

Probably we have an "old" format file that Neuroscan no longer supports then. It would be good to be able to support both. Maybe try/except is the best we can do... or we add a header='old' | 'new' kwarg or something

Probably we have an "old" format file that Neuroscan no longer supports then. It would be good to be able to support both. Maybe try/except is the best we can do... or we add a header='old' | 'new' kwarg or something

Okay, I can do that!
I just find it strange that the documentation references a website that describes the 'new' file format. If it's based on the old format why does it reference new documentation? Maybe the website changed?

For some reason a bunch of the tools/*.sh got chmod -x'ed

For some reason a bunch of the tools/*.sh got chmod -x'ed

Yes, sorry about that @larsoner I am using wsl2, and apparently there is a problem with configuring git line endings that causes this. I reverted the push. I ended up adding a keyword, does the documentation on the website get updated automatically?

Okay, everything should be fixed now @larsoner. I think the only thing that needs to be done is the testing data needs to be updated. I already merged the new file in:
https://github.com/mne-tools/mne-testing-data/blob/master/CNT/test_CNT_events_mne_JWoess_clipped.cnt

That PR has been merged and in MNE-Python main it should exist, but the CIs are not happy:
________________________________ test_new_data _________________________________
mne/io/cnt/tests/test_cnt.py:46: in test_new_data
    assert raw.info["bads"] == ["F8"]  # test bads
E   AssertionError: assert [] == ['F8']
E     Right contains one more item: 'F8'
E     Full diff:
E     - ['F8']
E     + []

Okay! Thanks! @larsoner. I found the problem and fixed it.

Does the test pass locally for you? Remotely it doesn't:
https://github.com/mne-tools/mne-python/actions/runs/6385705361/job/17331050081?pr=11833#step:17:4172
Can you try:
pytest mne/io/cnt/tests/test_cnt.py -k new_data

? It's best to iterate locally like this first before pushing to make sure things are green

Yes @larsoner, I just ran the test and it passed. I will be sure to be more mindful about looking over my code if I make another contribution. Thanks for the help!

Okay last thing, can you add a line to doc/changes/devel.rst?


improve: increase maximum pool size.

By default, Slonik limits the database connection pool to 10 connections. However, 10 seems to be on the low side. Right now if a request is unable to acquire a connection, there is usually an unhandled Slonik ConnectionError and a 500 response.
We have also observed that in some cases, trying to create a connection when the pool is full results in a connection leak. We're going to try to solve that issue more systematically, but for now, increasing the pool size should make it less likely that that issue comes up.
There has been some discussion about what number to choose instead of 10. In one Slack discussion, @issa-tseng wrote:

10 is lowish. but for a 1GB server it's probably about right. 20-25 is probably a reasonable default. maybe even 30.

@yanokwa wrote:

AWS max_db_connection is 83 on DO […] its 100.

We decided on 32 partly with the thought that it's the JDBC default.
I have tried this change on the QA server. When maximumPoolSize was 10, I observed that there could be up to 9 concurrent requests for a large OData feed. After changing the max to 32, I observed that there could be up to 31 concurrent requests. So it looks like Central is able to utilize the new max.
This commit branches off of v1.4.2 in case we want to try to release it as a patch.

I did a little bit more quick reading and recollection on pool sizes and I'm feeling less excited about this change. I believe that this would increase the likelihood of Central/the db running out of memory, right? A host with 1Gb of RAM is unlikely to be able to have more than 10 connections. What's the failure mode when there's an attempt to add another connection but there's no available memory?

on DO […] its 100.

Might be good to get a reference there. I see:

Clusters provide 25 connections per 1 GB of RAM. 3 connections per cluster are reserved for maintenance, and all remaining connections can be allocated to connection pools.

https://docs.digitalocean.com/products/databases/postgresql/how-to/manage-connection-pools/
Also:

Additionally, each active connection uses about 10 MB of RAM.

https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing has some good thoughts on tuning.

I believe that this would increase the likelihood of Central/the db running out of memory, right?

I don't have a deep understanding of this, but that sounds plausible to me. I feel a little reassured that the QA server (with 2 GB of RAM I believe) was able to support some heavy lifting: 31 concurrent requests for a large OData feed.

What's the failure mode when there's an attempt to add another connection but there's no available memory?

I'm not sure. Hopefully the connection attempt would be rejected, but maybe things would grind to a halt and/or be killed.

Clusters provide 25 connections per 1 GB of RAM.

If that's what Digital Ocean has landed on, would it be reasonable for us to make our max 25? That'd still be a sizable increase over the current 10.
I also still think it wouldn't be hard to make this configurable, if we want to keep the default on the lower side, but then have the ability to increase it for more robust servers.

A lot of this is new to me as well! I've been excited to learn more about this area while working on this issue.

I'm going to go ahead and close this PR in favor of a new one that allows maximumPoolSize to be configured.


Hidden Sheet - needs to make clearer that comments & formatting will be lost

Hi,
I think if you turn the Hidden Sheet option off, there should be a warning that says all the comments and formula formatting will be lost. In my case even some of the formulas broke when I turned this option off (and closed and opened Excel), but then I had to upgrade to the latest version of Excel. Then only the comments and formula formatting were lost.
If users rely on the Hidden Sheet, and then it's turned off by accident, it can easily break the whole workbook. Whilst I know you do not consider Hidden Sheet as a long term solution, there should be a clearer warning of what will happen when this option is turned off, so the user properly understands its affect.

Hi Jack, I am using the latest Excel Build 16.59 (22020901) on Mac
I'm thinking from the perspective of sharing the workbook, and other users who (i) might not have the latest Excel build (ii) might have the hidden sheet setting turned off.... it would break the whole workbook... : (
Hope you Supersmart people and Supersmart engineers can solve it!
Sachin

Closing this because hidden sheets are now deprecated in the new version of AFE. The code files are always stored with the document and are manager separately from names defining using the name manager. As a result, there is no hidden-sheet option.