Vedo Points becomes White or Transparent

We encountered problems when plotting point cloud points by Vedo in a pyside2 environment. It was good before however, without any notice of system change, points become white or transparent when run the same code recently. We tried the explore5D sample code on our system and got the same issue. Are there any suggestions to fix it?
Attached the explore5D output as FYI
Python 3.9.7
vedo: 2023.5.0
matplotlib: 3.7.3

Thanks for reporting, this is a weird bug in VTK which happens only on some machines.
Try
mypoints.render_points_as_spheres(False)

Thank you Marcomusy! The code above did work initially.
Unfortunately, we passed the first plot but the later plot causes fatal errors. Due to biz reason we can't provide the  code here. The working logic is simple: we first plot the raw point cloud, then do different works against those points and passed the new points, numpy array, to ploter to replot it.  at the plotter we do:
mypoints = Points(self.xyz,c ="rainbow")
mypoints.render_points_as_spheres(False)
plt.show(mypoints,zoom=1.3,axes=1, title = plot_type)
This was running in a multithreading framework of pyside2. That means a worker to do the data process and the main thread deal with plotting. Data passed between them are numpy arrays.
The error codes are :
Windows fatal exception: access violation
Thread 0x00002af4 (most recent call first):
File "our_env\lib\concurrent\futures\thread.py", line 75 in _worker
File "our_env\lib\threading.py", line 910 in run
File "our_env\lib\threading.py", line 973 in _bootstrap_inner
File "our_env\lib\threading.py", line 930 in _bootstr
Appreciate if you can provide further assistance.
Regards
DR.

BTW, we did  plt.clear() before the plotting

The code was running in a spyder console, The final error msg following above are :
Main thread:
Current thread 0x00002ac0 (most recent call first):
File "our_env\lib\site-packages\vedo\plotter.py", line 3235 in show
....
File "our_env\lib\site-packages\spyder_kernels\py3compat.py", line 356 in compat_exec
File "our_env\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 469 in exec_code
File "our_env\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 611 in _exec_file
File "our_env\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 524 in runfile
File "C:\Users\Complie-WIN10\AppData\Local\Temp\ipykernel_10088\3320116497.py", line 1 in

This seems related to the qt widget line 3235 is
        if self.qt_widget is not None:
            self.qt_widget.GetRenderWindow().AddRenderer(self.renderer)

can you try it on the dev version:
pip install -U git+https://github.com/marcomusy/vedo.git

PS: also add
settings.enable_default_mouse_callbacks = False
settings.enable_default_keyboard_callbacks = False
settings.enable_pipeline = False

Hello,2024.5.0+dev31 installed. where to pu the following settings? in my working py or vedo setting files?
settings.enable_default_mouse_callbacks = False
settings.enable_default_keyboard_callbacks = False
settings.enable_pipeline = False

Installed dev31 and put settings into our py code. The result2 are:

The initial plot disappared from plot window but move to somewhere outside view. We need to zoom to find it

The flowup plot got errors as following:
self.plt.show(mypoints,zoom=1.3,axes=1, title= plot_type)  ## axes= 1/7/11

File our_env\site-packages\vedo\plotter.py:3288 in show
bns = self.renderer.ComputeVisiblePropBounds()
AttributeError: 'NoneType' object has no attribute 'ComputeVisiblePropBounds'

We did further tests ,here are some information:

comment out the mouse and keyboard call back settting (=False), the inital plotter remains at the central of window without errors
remove the pipline settings cause plt.clear() errors as following:

File out_envs\lib\site-packages\vedo\plotter.py:3489 in clear
self.get_meshes()
File our_env\lib\site-packages\vedo\plotter.py:1211 in get_meshes
at = self.renderers.index(renderer)
ValueError: <vtkmodules.vtkRenderingOpenGL2.vtkOpenGLRenderer(0x000001DFF0442770) at 0x000001DFF7CCB460> is not in list
PS
If we want  to clean up older points' show before plotting new points, are there any better way than plt.clear()? It looks like the clear() causes item 2 above.

There are problems connecting to git. Just a quick question: where to put those settings?
发自我的荣耀手机
-------- 原始邮件 --------
发件人： Marco Musy @.>
日期： 2024年1月30日周二 半夜11:03
收件人： marcomusy/vedo @.>
抄送： DRLing2021 @.>, Author @.>
主 题： Re: [marcomusy/vedo] Vedo Points becomes White or Transparent (Issue #1039)
PS: also add
settings.enable_default_mouse_callbacks = False
settings.enable_default_keyboard_callbacks = False
settings.enable_pipeline = False
―
Reply to this email directly, view it on GitHubhttps://github.com/marcomusy/vedo/issues/1039#issuecomment-1917088230, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2XDDDVVCVUDGO5KB7FWWK3YREDVDAVCNFSM6AAAAABCQSMKQGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJXGA4DQMRTGA.
You are receiving this because you authored the thread.Message ID: @.***>

Hi Marco, would you please advise what to do to deal with the new error messages?
File our_env\site-packages\vedo\plotter.py:3288 in show
bns = self.renderer.ComputeVisiblePropBounds()
AttributeError: 'NoneType' object has no attribute 'ComputeVisiblePropBounds'

Sorry, i was away, you put the senttings in your script on top E.g.:
import vedo

vedo.settings.enable_default_mouse_callbacks = False
vedo.settings.enable_default_keyboard_callbacks = False

to add and remove object do not use clear() rather give individual objects a name and add/remove them individualy:
def mycallback(event):
    # in some callback:
    obj1 = vedo.Points(...)
    obj1.name = "mypoints"
    # remove some other previous Points and add the new one:
    plotter.remove("mypoints").add(obj1, obj2, ...)
    # plotter.reset_camera()  # if needed
    plotter.render()

plotter = vedo.Plotter()
plotter.add_callback("key press", mycallback)
plotter.show(some objects, ....)

there are many examples showing this mechanism e.g.:
examples/basic/mousehighlight.py
examples/volumetric/slicer2.py
...
make sure you can reproduce those.
vedo --search remove

No problem Macro. We did what you suggested with some changes as the plots' show use different parms as following:
The first plot use no remove command as there is nothing yet but we give it a name
self.Show_PC = Points(self.xyz,c="green")  ### actors of points
self.Show_PC.name = "last"   ##
self.Show_PC.render_points_as_spheres(False)
print("Points Gened")
self.Show_PC.cmap('viridis', Z)
self.plt.show(self.Show_PC,zoom=1.3,axes=1, title= plot_type)
The following re-plots with new data we use the following codes:
self.Show_PC = Points(self.xyz,c="green")  ### actors of points
self.Show_PC.name = "last"   ##
self.Show_PC.render_points_as_spheres(False)
print("Tiered Gened")
self.Show_PC.cmap('viridis', Z) #,name="mydata")
        self.plt.remove("last")  ##.add(self.Show_PC)  we use show below but not
        self.plt.show(self.Show_PC,zoom=2.35axes=1, title= plot_type)

Unfortunately we got following errors:
self.plt.remove("last")  ##.add(self.Show_PC)
File our_env\lib\site-packages\vedo\plotter.py:995 in remove
for a in self.get_actors(include_non_pickables=True):
File our_env\lib\site-packages\vedo\plotter.py:1287 in get_actors
at = self.renderers.index(renderer)
ValueError: None is not in list

No problem Macro. We did what you suggested with some changes as the plots' show use different parms as following:
The first plot use no remove command as there is nothing yet but we give it a name
self.Show_PC = Points(self.xyz,c="green")  ### actors of points
self.Show_PC.name = "last"   ##
self.Show_PC.render_points_as_spheres(False)
print("Points Gened")
self.Show_PC.cmap('viridis', Z)
self.plt.show(self.Show_PC,zoom=1.3,axes=1, title= plot_type)
The following re-plots with new data we use the following codes:
self.Show_PC = Points(self.xyz,c="green")  ### actors of points
self.Show_PC.name = "last"   ##
self.Show_PC.render_points_as_spheres(False)
print("Tiered Gened")
self.Show_PC.cmap('viridis', Z) #,name="mydata")
        self.plt.remove("last")  ##.add(self.Show_PC)  we use show below but not
        self.plt.show(self.Show_PC,zoom=2.35axes=1, title= plot_type)

Unfortunately we got following errors:
self.plt.remove("last")  ##.add(self.Show_PC)
File our_env\lib\site-packages\vedo\plotter.py:995 in remove
for a in self.get_actors(include_non_pickables=True):
File our_env\lib\site-packages\vedo\plotter.py:1287 in get_actors
at = self.renderers.index(renderer)
ValueError: None is not in list

Hi Macro，
We‘ve tried serval ways but the "remove " seems never work. Basically what we do are as:
Step1： Plot the raw data
        self.Show_PC = Points(self.xyz,c="green")  <=== convert numpy to Points
        self.Show_PC.name = "last"   <=== give Points a name "last"
        self.Show_PC.render_points_as_spheres(False)    <=== To fix the White/Transparent issue
        self.Show_PC.cmap('viridis', Z)   <=== assign a cmap
        self.plt.add(self.Show_PC)   <=== add "last " to Plotter
        self.plt.show(zoom=1.3,axes=1, title= plot_type)   <=== render with parms (this is different from what you
                             said and from samples )

Step2: Processed the raw data and get new data
Setp2: Plot the  new data
        self.Show_PC = Points(self.xyz,c="green")  <=== assign new data to Points
        self.Show_PC.name = "last"  <=== give a name, same as before, to new Points
        self.Show_PC.render_points_as_spheres(False)  <=== To fix the White/Transparent issue
        self.Show_PC.cmap('viridis', Z)    <=== assign a cmap
        self.plt.remove("last").add(self.Show_PC)  <=== this is where ERROR reported
        self.plt.show(zoom=1.3,axes=1, title= plot_type)  ## axes= 1/7/11

The error msg are:
 File our_env\lib\site-packages\vedo\plotter.py:995 in remove
   for a in self.get_actors(include_non_pickables=True):

 File our_env\lib\site-packages\vedo\plotter.py:1287 in get_actors
   at = self.renderers.index(renderer)

  ValueError: None is not in list

Would you mind to advice what to do to fix it? Do we do it in a right way or are there something missing here to make it run? We need to fix it to move the work ahead. Your help is highly appreciated.

The definition of self.plt in our code is
self.plt = Plotter(qt_widget=self.vtkWidget,axes=1)
This is different from the sample codes we got from vedo --search xxxx command.

The definition of self.plt in our code is
self.vtkWidget = QVTKRenderWindowInteractor(self.Plot_Frame_3d)
self.plt = Plotter(qt_widget=self.vtkWidget,axes=1)

This is different from the sample codes we got from vedo --search xxxx. It this reason why we encountered those errors?

Hi Macro, we did further investigation and got the latest results as following:

Vedo sample code, examples/volumetric/slicer2.py, runs well with the desired result

Our Vedo scrip still stuck with the errors:
File our_env\lib\site-packages\vedo\plotter.py:995 in remove
for a in self.get_actors(include_non_pickables=True):
File our_env\lib\site-packages\vedo\plotter.py:1287 in get_actors
at = self.renderers.index(renderer)
ValueError: None is not in list

Our running enviroment:
VTK: 9.26.
Qt interrface: PySide2 <===== suspect this is the error causer?
We checked the Plotter code line 995 and later. It seems that the renderer is not picked correctly when running our working code. But it is ok for the slicer2.py sample code. One major difference between the codes is that ours runs in a qt framework through pyside2 interface.
self.Plot_Frame_3d = QFrame(self.centralwidget)
...
self.vtkWidget = QVTKRenderWindowInteractor(self.Plot_Frame_3d)
self.plt = Plotter(qt_widget=self.vtkWidget,axes=1)

We suspect the error relates to pysideX version. After checking Vedo Github we found that in vtkmodules_9.2.6_hierarchy.txt there is no pyside* information rather in  vtkmodules_9.3.0_hierarchy.txt there is one place says:
vtkmodules.qt.QVTKRenderWindowInteractor.PySide6
Does it mean that we should upgrade to pyside6 and vtk 9.3.0? Since upgrade from pyside2 to pyside6 needs lots script changes we haven't done this step yet.

Would you mind to kindely help us to move it forward as our work has been suspended for days waiting to see the visual result to descide the next step. Thanks in advance.! @marcomusy

Hi Marcro, Just want to let you know that after various experiments, we figured out that if Show() is BEFORE add(), things are ok. This is weired. But add/remove issues is fixed !

Hi @DRLing2021 sorry for the delay - i'm happy that you managed to fix the problem, just to confirm that the general structure is
def somecallback():
  # here you call add(), remove(), render()
  # or plt += some_object
  # but NOT show()

# in your main script you define the Plotter and call show() only once, which does a number of inittializations
plt = Plotter()
plt += Sphere()
...
plt.add_callback("key press", somecallback)
plt.show()

Obviously you can have plt as a member of a class, but the general idea is the same.
Please look at some examples, especially the ones in examples/other/qt_*.py

Hi Marco, thanks for you coments. Sorry for late update as I have been on leave last weeks. The code we run on experimental env is different from what suggested above in that the plotter.show() has to be BEFORE add(). The sample code is as following:
Vedo_Show.txt
Gen_3D_Data.txt
The questions regarding above codes are :

Are there anything wrong, or double wrong, in the codes that cause the show() has to be BEFORE add()?
The code is suppose to increase the Z value scale after each button press. it results in the points zoom out of view. Is there anyway to set "autoscale" so that all points fit into the show window?

PS we use pyside2 that is different from the sample examples/other/qt_*.py, We guess it should not be a problem
Regards.
DR, Ling


Stable pre release

Once completed, reviewed, tested and verified, this will go on to become Stable version 2.0.7.
If you think any important fixes are missing, tell me the PR and commit numbers.
Changes since 2.0.6:

Fix CVE-2022-37434 #1328
Fix chunkmemset #1196
Fix deflateBound too small #1236
Fix Z_SOLO #1263
Fix ACLE variant of crc32 #1274
Fix inflateBack #1311
Fix deflate_quick windowsize #1431
Fix DFLTCC bugs related to adler32 #1349 and #1390
Fix warnings #1194 #1312 #1362
MacOS build fix #1198
Add invalid windowBits handling #1293
Support for Force TZCNT #1186
Support for aligned_alloc() #1360
Minideflate improvements #1175 #1238
Dont use unaligned access for memcpy #1309
Build system #1209 #1233 #1267 #1273 #1278 #1292 #1316 #1318 #1365
Test improvements #1208 #1227 #1241 #1353
Cleanup #1266
Documentation #1205 #1359
Misc improvements #1294 #1297 #1306 #1344 #1348
The relevant CI workflows have been backported from Develop branch

Missing commit:
https://github.com/zlib-ng/zlib-ng/pull/1267 - fix for bug in https://github.com/zlib-ng/zlib-ng/pull/1393/commits/3a5a2c436c7dc3d05b041b8f1dd99302b910ff8f

Missing commit: #1267 - fix for bug in 3a5a2c4

What do you mean? The commit is in this PR already, and there is no other commit in the source PR.

Oh! I am sorry!
For some unknown reason, not all commits and changes (on "Files changed" tab) were loaded for me.

Build breaks initially at https://github.com/zlib-ng/zlib-ng/pull/1393/commits/9bb926a89178e83dce733e0312a21d1daced4c99 ... After that everything is noise.

@mtl1979 Thanks for the assist 😄.
The offending commits have now been edited and repushed.

@mtl1979 Thanks for the assist 😄. The offending commits have now been edited and repushed.

I've said many times that we need a new stable release soon... I will do anything within the limits of my knowledge...

There still seems to be issues finding clang-11 and Visual C++ compiler...

Those commits would have to be re-made from scratch, as the CMake code has diverged quite a lot since then.
PS: Also updated the PR description.

There still seems to be issues finding clang-11 and Visual C++ compiler...

Those commits would have to be re-made from scratch, as the CMake code has diverged quite a lot since then.
PS: Also updated the PR description.

I think we should force the failing runs to use "ubuntu-20.04" and "windows-2019" so the fix stays simple... Trying to update the build system to work with latest versions might take too long.

Could you please pull https://github.com/iii-i/zlib-ng/tree/stable-pre-release?
https://github.com/iii-i/zlib-ng/commit/f0a02c0f9055350bb6cc498ba32fb3ac73340aac is a combination of https://github.com/zlib-ng/zlib-ng/pull/1349 and https://github.com/zlib-ng/zlib-ng/pull/1390 (not merged yet) minus tests.

I tested the prerelease against the test suite of python-zlib-ng. The following issue remains: https://github.com/zlib-ng/zlib-ng/issues/1417
It is fixed on the develop branch.

Getting close to having the CI passing now.
I  am working on getting the abicheck files updated, but the sanitizers are complaining about a memory leak.
Ex:
https://github.com/zlib-ng/zlib-ng/actions/runs/4126285777/jobs/7128014363
682/692 Test #668: CVE-2004-0797 ..............................................................................................................***Failed    0.69 sec
/home/runner/work/zlib-ng/zlib-ng/minigzip: <fd:0>: invalid bit length repeat

=================================================================
==13496==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 16384 byte(s) in 1 object(s) allocated from:
    #0 0x7fa5b37da55c in __interceptor_posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:226
    #1 0x558e2ac36b5d in gz_look (/home/runner/work/zlib-ng/zlib-ng/minigzip+0x20b5d)

Indirect leak of 8192 byte(s) in 1 object(s) allocated from:
    #0 0x7fa5b37da55c in __interceptor_posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:226
    #1 0x558e2ac36a8c in gz_look (/home/runner/work/zlib-ng/zlib-ng/minigzip+0x20a8c)

Indirect leak of 7160 byte(s) in 1 object(s) allocated from:
    #0 0x7fa5b37da55c in __interceptor_posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:226
    #1 0x558e2ac61135 in zng_calloc (/home/runner/work/zlib-ng/zlib-ng/minigzip+0x4b135)
    #2 0x558e2ac7ab21  (/home/runner/work/zlib-ng/zlib-ng/minigzip+0x64b21)

Indirect leak of 232 byte(s) in 1 object(s) allocated from:
    #0 0x7fa5b37da55c in __interceptor_posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:226
    #1 0x558e2ac3417e in gz_open (/home/runner/work/zlib-ng/zlib-ng/minigzip+0x1e17e)

Indirect leak of 7 byte(s) in 1 object(s) allocated from:
    #0 0x7fa5b37d9867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x558e2ac34723 in gz_open (/home/runner/work/zlib-ng/zlib-ng/minigzip+0x1e723)

SUMMARY: AddressSanitizer: 31975 byte(s) leaked in 5 allocation(s).
CMake Error at cmake/run-and-redirect.cmake:53 (message):
  /home/runner/work/zlib-ng/zlib-ng/minigzip;-d failed: Subprocess aborted

Weirdly, only CVE tests fail..
The following tests FAILED:
	667 - CVE-2002-0059 (Failed)
	668 - CVE-2004-0797 (Failed)
	669 - CVE-2005-1849 (Failed)
	670 - CVE-2005-2096 (Failed)

Any ideas?
I tried to have a look at the commit log for gzlib.c and gzguts.h, but the two commits missing should not matter.
Coverity flags no problems.

Perhaps #1422 should go in since some other related changes went in too..

Perhaps #1422 should go in since some other related changes went in too..

This branch predates even the zmemcmp macros, so all the changes would have to be remade from scratch.

Hmm I think you are right, I guess I was mistaken.

Regarding the memory leaks, I am at a loss.
I am unable to reproduce any of them locally.
gz_look() allocates state->in and state->out, and gzclose_r() is supposed to deallocate them.
Those two variables fit well well as they are 8KiB and 16KiB, matching the first two leaks.
But gzlib.c and gzread.c are practically the same as they are in develop, where it does not leak.
Minigzip calls those functions from its gz_uncompress() function, nothing different from develop there either.
The rest of the leaks I have not looked at.
I am coming down with the flu or something, been getting steadily worse since this morning, so I might not be around much or at all the next few days. Trying to button up what I can while I am still able to look at the monitor.

@mtl1979 Package check fails one of the runs, it looks like something went completely wrong and it is missing output from one of the runs, or something like that?
https://github.com/zlib-ng/zlib-ng/actions/runs/4129678522/jobs/7135548390

@iii-i One of the DFLTCC runs is failing:
Seems to succeed on GCC but fail with Clang.
https://github.com/zlib-ng/zlib-ng/actions/runs/4129678532/jobs/7135553202
==4648==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x2aa2b95bd9d in dfltcc_can_deflate (/home/actions-runner/_work/zlib-ng/zlib-ng/switchlevels+0xdbd9d)
    #1 0x2aa2b941867 in zng_deflateBound (/home/actions-runner/_work/zlib-ng/zlib-ng/switchlevels+0xc1867)
    #2 0x2aa2b934811 in main (/home/actions-runner/_work/zlib-ng/zlib-ng/switchlevels+0xb4811)
    #3 0x3ff90f24409 in __libc_start_main (/lib/s390x-linux-gnu/libc.so.6+0x24409)
    #4 0x2aa2b8a11d3 in _start (/home/actions-runner/_work/zlib-ng/zlib-ng/switchlevels+0x211d3)

@mtl1979 Package check fails one of the runs, it looks like something went completely wrong and it is missing output from one of the runs, or something like that? https://github.com/zlib-ng/zlib-ng/actions/runs/4129678522/jobs/7135548390

I think one of the builds is targeting wrong processor...

Only things blocking the release now are the ASAN and MSAN CI failures.

I did a BeyondCompare between the two branches and didn't notice very much difference in the gz source files. It would be good to get in all the Mark Adler fixes: e713f2f4, 4af454bf, 33e671b9, 0e49aa38, 5393c219, 6581fb0e, 24d7d0d7.

ASAN leak seems to be coming from line error(PREFIX(gzerror)(in, &err)); in gz_uncompress() of test/minigzip.c. in is not closed properly.

For MSan, please take https://github.com/iii-i/zlib-ng/commit/2050ed59b8697d1b9258c473ed12f2398409648b and https://github.com/iii-i/zlib-ng/commit/4b32f7c5c489e2f49a08c1a03272f18aca3e5182 from https://github.com/iii-i/zlib-ng/commits/stable-msan.

Sorry, I just noticed that I introduced a duplicate CI job (https://github.com/iii-i/zlib-ng/commit/4b32f7c5c489e2f49a08c1a03272f18aca3e5182#diff-cdd48abbd3eb8d1c54077449fc74a8de1f29805d2be5d8e5232b7aab76ea7a6fR238), and had to respin the branch. Please take https://github.com/iii-i/zlib-ng/commit/2050ed59b8697d1b9258c473ed12f2398409648b and https://github.com/iii-i/zlib-ng/commit/bba6bf1f8acdc009df7743492035a97e4496a42c instead.

All checks pass!

All checks pass!

We're getting closer and closer to next stable release... Still need some testing before we can do official release...

I don't know if it is very helpful but for the python bindings I test the compatibility with proper zlib. I simply take all possible compression levels, memory levels and windowbits settings and run them through zlib-ng's deflate and see if zlib's inflate can properly handle them. Then I do the same thing the other way round. This results in thousands of compatibility tests.
I just checked this branch and on Windows x86/64 (msvc), Linux x86/64(gcc) and MacOS x86/64(clang) everything seems to be fine.

Heads up: This is getting released later today, unless something comes up in last minute testing.
I'd like it if people would do a quick review of this PR. (You don't need to read every line, just say what you did do; cursory review, tested on x86-64, agree with the proposed changes, etc).
(Merge protection is on, so reviews are also required before merging, and I'd like to avoid having to bypass those protections)
I meant to release this late last week, but I have been out with the flu for a week now, and I am finally able to spend a little time in front of a monitor again.
I have been running this on a production web server for a while now without problems, unfortunately I no longer have access to big server deployments since I am no longer employed (I became 100% disabled recently).
But confidence is high that there are no glaring bugs in this, but corner cases are always the hardest ones to test.

GCC 12.2 on x86-64 i9-9900K (no AVX-512)
2.0.6
 Tool: minigzip   Levels: 1-9
 Runs: 45         Trim worst: 5

 Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
 1     47.618%      1.852/1.881/1.900/0.011        0.897/0.921/0.936/0.010      100,930,174
 2     35.518%      2.716/2.745/2.766/0.013        0.990/1.006/1.021/0.007       75,282,961
 3     34.198%      3.106/3.157/3.206/0.025        0.953/0.979/0.990/0.008       72,485,726
 4     32.928%      3.562/3.622/3.670/0.031        0.922/0.935/0.947/0.007       69,793,995
 5     32.662%      3.817/3.869/3.938/0.034        0.908/0.927/0.938/0.007       69,228,858
 6     32.507%      4.308/4.412/4.473/0.047        0.908/0.927/0.938/0.007       68,901,209
 7     32.255%      5.925/6.066/6.185/0.072        0.887/0.908/0.923/0.009       68,366,800
 8     32.167%      8.655/8.787/9.027/0.092        0.894/0.908/0.920/0.007       68,180,776
 9     32.156%   11.603/11.802/12.225/0.151        0.876/0.907/0.921/0.010       68,156,160

 avg1  34.668%                        5.149                          0.935
 tot                               1853.663                        336.716      661,326,659

   text    data     bss     dec     hex filename
 112085    1296      32  113413   1bb05 libz-ng.so.2

2.0.7
 Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
 1     44.407%      1.777/1.791/1.811/0.009        0.975/0.994/1.008/0.010       94,124,801
 2     35.518%      2.753/2.777/2.798/0.012        1.013/1.036/1.047/0.007       75,282,961
 3     34.198%      3.118/3.180/3.227/0.027        0.989/1.008/1.020/0.008       72,485,726
 4     32.928%      3.585/3.659/3.715/0.034        0.947/0.963/0.975/0.007       69,793,995
 5     32.662%      3.813/3.891/3.959/0.037        0.945/0.958/0.970/0.007       69,228,858
 6     32.507%      4.321/4.452/4.566/0.059        0.928/0.954/0.964/0.007       68,901,209
 7     32.255%      5.977/6.092/6.207/0.062        0.901/0.935/0.951/0.010       68,366,800
 8     32.167%      8.646/8.793/9.029/0.096        0.910/0.928/0.945/0.010       68,180,776
 9     32.156%   11.625/11.800/12.054/0.106        0.903/0.927/0.939/0.008       68,156,160

 avg1  34.311%                        5.159                          0.967
 tot                               1857.406                        348.108      654,521,286

 Level   Comp   Comptime min/avg/max/stddev  Decomptime min/avg/max/stddev  Compressed size
 1     44.407%      1.779/1.799/1.815/0.010        0.976/0.993/1.008/0.009       94,124,801
 2     35.518%      2.731/2.780/2.799/0.012        1.023/1.038/1.052/0.008       75,282,961
 3     34.198%      3.116/3.179/3.225/0.025        0.982/1.005/1.017/0.009       72,485,726
 4     32.928%      3.599/3.656/3.705/0.028        0.943/0.963/0.976/0.008       69,793,995
 5     32.662%      3.816/3.897/3.961/0.039        0.940/0.956/0.968/0.008       69,228,858
 6     32.507%      4.336/4.436/4.512/0.054        0.928/0.955/0.966/0.008       68,901,209
 7     32.255%      5.949/6.085/6.202/0.067        0.903/0.932/0.945/0.010       68,366,800
 8     32.167%      8.658/8.817/9.079/0.124        0.909/0.928/0.940/0.009       68,180,776
 9     32.156%   11.649/11.815/12.112/0.121        0.908/0.928/0.942/0.009       68,156,160

 avg1  34.311%                        5.163                          0.966
 tot                               1858.591                        347.925      654,521,286

   text    data     bss     dec     hex filename
 111613    1296      32  112941   1b92d libz-ng.so.2

Code size is slightly smaller.
Compression and Decompression is slightly slower overall, but not by much.
Level 1 compression has improved by quite a bit, but it is also 4.5% faster.

Thank you very much for the release! FWIW, I ran some additional tests on s390x, and all is looking good.

Thanks very much for releasing. I updated the python bindings to include the new version. Since the wbits bug is fixed these bindings can be used as a one-to-one replacement for python's zlib module.


Add task polling data to task responses

What changed?
Add some additional information to polling responses about the task queue
Outcome of discussion w/ @ShahabT and @dnr
Initially I would've liked TaskQueueStats to be returned on every poll response. This appears to be too much from both a performance and complexity perspective initially because regularly updating and distributing aggregated data across partitions would require more frequent internal RPCs, and would require them to go in a direction they don't currently (from sub-partition -> root rather than just the other way around).
The alternative is to have SDK acquire that data via DescribeTaskQueue. Calling that very frequently would also be problematic, since there is an RPS limit per-namespace.
The landed on middle ground, which is low complexity but ought to be reasonably efficient, is to include TaskQueueStats in the response whenever a poll hits the root partition, where it is readily available. If the SDK has not seen such a response in some reasonable TTL (~seconds), then it will call DescribeTaskQueue in order to have an updated view. In practice, it should be fairly unlikely for a worker with a reasonable number of pollers to somehow miss root, and low-traffic queues are also more likely to forward to the root to see if there is a task. Since the desired outcome SDK side is to only have a low number of pollers when there's low traffic, that should work out.

Why?
This will allow poller auto-tuning to have an idea of if it should be increasing the number of pollers because most matches are not sync (possibly in conjunction w/ a backlog hint)
If we are making sync matches, then we can reduce the number of pollers once the wait duration starts to exceed some threshold.
It can also inform the SDK what the minimum latency is when polling, since if we didn't make a sync match, then we were pulling something out of the backlog and delivered a task roughly as quickly as is possible.
The task queue stats info is useful to help to decide how quickly pollers should be ramped up, if at all.

Breaking changes
nope

Server PR
Will make one assuming we're OK with this add.

Leaving as an FYI here: I started the server work but it's likely I won't get back to it for a bit b/c of other Replay-related priorities, so this might have to sit for a bit.


update the translation guide

@bobdos

Was already fixed in one PR a few daysago, is there something more missing?

I may double-check and confirm not missing anything, many thanks @qlpqlp


Get repeated answers of no more than three words while deploy LLaMA3-Instruct-8B with Triton server使用Triton部署LLaMA3模型时遇到重复生成问题

Environment
CPU architecture:  x86_64
CPU/Host memory size: 32G
GPU properties: SM86
GPU name: NVIDIA A10
GPU memory size: 24G
Clock frequencies used: 1695MHz
Libraries
TensorRT-LLM: v0.9.0
TensorRT: 9.3.0.post12.dev1 (display 8.6.3 while input "dpkg -l | grep nvinfer" in cmd),
CUDA: 12.3,
Container used : Y
NVIDIA driver version: 535.161.08
OS : Ubuntu 22.04
Reproduction Steps
docker exec -it trtllm1 /bin/bash
mamba deactivate
mamba deactivate

# git from correct branch
git clone -b v0.9.0 https://github.com/NVIDIA/TensorRT-LLM.git
git clone -b v0.9.0 https://github.com/triton-inference-server/tensorrtllm_backend.git

# build trt engines
cd TensorRT-LLM
trtllm-build --checkpoint_dir ../Work/TensorRT-LLM/examples/llama/tllm_checkpoint_1gpu_tp1 \
            --output_dir ./tmp/llama/8B/trt_engines/fp16/1-gpu/ \
            --remove_input_padding enable \
            --gpt_attention_plugin float16 --gemm_plugin float16 \
            --context_fmha enable --paged_kv_cache enable \
            --streamingllm enable \
            --use_paged_context_fmha enable --enable_chunked_context\
            --use_context_fmha_for_generation enable \
            --max_input_len 512 --max_output_len 512 \
            --max_batch_size 64

# copy rank0.engine & config.json
cd ../tensorrtllm_backend
cp ../TensorRT-LLM/tmp/llama/8B/trt_engines/fp16/1-gpu/* all_models/inflight_batcher_llm/tensorrt_llm/1/

# model configuration
export HF_LLAMA_MODEL=/path/to/llama3-8B-Instruct-hf
export ENGINE_PATH=/path/to/tensorrtllm_backend/all_models/inflight_batcher_llm/tensorrt_llm/1/
python3 tools/fill_template.py -i all_models/inflight_batcher_llm/preprocessing/config.pbtxt tokenizer_dir:${HF_LLAMA_MODEL},tokenizer_type:auto,triton_max_batch_size:64,preprocessing_instance_count:1
python3 tools/fill_template.py -i all_models/inflight_batcher_llm/postprocessing/config.pbtxt tokenizer_dir:${HF_LLAMA_MODEL},tokenizer_type:auto,triton_max_batch_size:64,postprocessing_instance_count:1
python3 tools/fill_template.py -i all_models/inflight_batcher_llm/tensorrt_llm_bls/config.pbtxt triton_max_batch_size:64,decoupled_mode:True,repetition_penalty:0.9,bls_instance_count:1,accumulate_tokens:False
python3 tools/fill_template.py -i all_models/inflight_batcher_llm/ensemble/config.pbtxt triton_max_batch_size:64
python3 tools/fill_template.py -i all_models/inflight_batcher_llm/tensorrt_llm/config.pbtxt triton_max_batch_size:64,decoupled_mode:False,decoding_mode:top_p,enable_chunked_context:True,batch_scheduler_policy:max_utilization,engine_dir:${ENGINE_PATH},max_tokens_in_paged_kv_cache:2560,max_attention_window_size:2560,kv_cache_free_gpu_mem_fraction:0.9,exclude_input_in_output:True,enable_kv_cache_reuse:True,batching_strategy:v1,enable_trt_overlap:True,max_queue_delay_microseconds:0

# launch triton-server
python3 scripts/launch_triton_server.py --model_repo=all_models/inflight_batcher_llm --world_size 1

# send request via curl
curl -X POST localhost:8000/v2/models/ensemble/generate -d '{"text_input": "what are flowers","max_tokens": 100,"bad_words":[""],"stop_words":["<|eot_id|>"]}'

Expected Behavior
normal answer
Actual Behavior

Additional Notes
When I simply run TensorRT-LLM locally, via
python3 ../run.py --tokenizer_dir ./tmp/llama/8B/ \
                  --engine_dir=./tmp/llama/8B/trt_engines/fp16/1-gpu/ \
                  --input_text "How to build tensorrt engine?" \
                  --max_output_len 100

The model can answer normally.

Can you try turning off inflightbatching? This may be a known bug (of inflightbatching) that will be fixed in the latest commit.

I already turned off in-flight batching by setting"batching_strategy:v1" in tensorrt_llm/config.pbtxt. Or do you mean change the parameters while building TensorRT engine? By the way, I saw someone else got all-well answers using almost the same command  lines as me, except he used TensorRT-LLM&tensorllm_backend repository version 0.8.0 while mine are 0.9.0.

Thanks.
Please try disabling these options. One of them may be causing the problem:
--context_fmha enable --paged_kv_cache enable \
            --streamingllm enable \
            --use_paged_context_fmha enable --enable_chunked_context\
            --use_context_fmha_for_generation enable \
``

Thanks. Please try disabling these options. I think one of them may be causing the problem,
--context_fmha enable --paged_kv_cache enable \
            --streamingllm enable \
            --use_paged_context_fmha enable --enable_chunked_context\
            --use_context_fmha_for_generation enable \

In addition, I recommend using the main branch to compile TRT-LLM because it contains the latest bug fixes

Thank you for your guidance! After disabling all the options you mentioned, I can use it under v1 mode normally. However, I still have two questions: 1. Which specific parameter caused the bug, and has it been reported? 2. Does everyone else encounter this particular bug when using it? If not, what should I do in the current version to enable the in-flight mode?
Many thanks.

Thanks. Please try disabling these options. I think one of them may be causing the problem,
--context_fmha enable --paged_kv_cache enable \
            --streamingllm enable \
            --use_paged_context_fmha enable --enable_chunked_context\
            --use_context_fmha_for_generation enable \

In addition, I recommend using the main branch to compile TRT-LLM because it contains the latest bug fixes

Thank you for your guidance! After disabling all the options you mentioned, I can use it under v1 mode normally. However, I still have two questions: 1. Which specific parameter caused the bug, and has it been reported? 2. Does everyone else encounter this particular bug when using it? If not, what should I do in the 0.9.0 version to enable the in-flight mode? Because I encountered lots of version adaptation problems when deploying the latest version. Many thanks.

We will update the main branch code soon, including the triton server so that you can try the new version at that time, and please give feedback if you have the same issue.
Try turning off these options one by one to reduce the performance regression.
Older versions of v0.9 are not recommended.

Issues about streamingllm feature.


test(scrapers): Add new Twitter scraper test suite

Description
This pull request introduces a new test suite for the Twitter scraper functionality. The suite is designed to ensure that our Twitter scraping operations are reliable, secure, and function as expected under various conditions. The tests cover authentication, session reuse, and specific scraping capabilities.
Changes

Added scrapers_suite_test.go
Sets up the test suite for all scraper-related tests, ensuring proper environment variables are loaded and the testing environment is correctly configured.

Added twitter_scraper_test.go
Contains tests specifically for the Twitter scraper, focusing on authentication and scraping functionalities.

Tests Included

Authenticates and Logs In Successfully

Purpose: Verifies that the scraper can authenticate with Twitter using valid credentials.
What it tests:

Ensures that the cookie file does not exist before authentication.
Authenticates with Twitter and checks that a new cookie file is created.
Checks that the scraper is in a logged-in state.
Performs a simple profile scrape to confirm the session is valid.

Reuses Session from Cookies

Purpose: Confirms that the scraper can reuse an existing session from stored cookies.
What it tests:

Performs an initial authentication to generate a cookie file.
Clears the scraper instance to simulate a fresh start.
Authenticates again, expecting it to use the existing cookies.
Verifies that the scraper is still in a logged-in state.
Confirms the session by performing a profile scrape.

Scrapes Profile and Recent Tweets Using Saved Cookies

Purpose: Tests the scraper's ability to perform complex scraping tasks using an authenticated session.
What it tests:

Uses the saved cookies to authenticate without re-entering credentials.
Scrapes the Twitter profile of 'god' and checks the retrieved data.
Scrapes recent tweets containing the #Bitcoin hashtag.
Verifies that the correct number of tweets are retrieved and outputs their content.

Notes

Environment Variables: The tests rely on environment variables for Twitter credentials. Ensure that TWITTER_USERNAME, TWITTER_PASSWORD, and optionally TWITTER_2FA_CODE are set in a .env file at the project root.
Temporary Directories: Tests create temporary directories to isolate cookie files and other data, preventing interference with existing sessions.
Cleanup: After each test, the temporary directories and any generated files are cleaned up to maintain a clean testing environment.

How to Test

Set up the required environment variables in your .env file.

From root run the tests using the Ginkgo testing framework:
ginkgo -v ./pkg/tests/scrapers

Review the output to ensure all tests pass successfully.

https://www.loom.com/share/da177d585ab941edbfbf6fe5236878fe

@mudler how do we configure CI to use environment variables to run tests?
Also I am having issues loading .env for some reason. Maybe you can take a look and help?
brendanplayford@Brendans-MacBook-Pro masa-oracle % ginkgo -v ./pkg/tests/scrapers
time="2024-10-01T14:21:23-07:00" level=warning msg="Error loading .env file" error="open .env: no such file or directory"
time="2024-10-01T14:21:23-07:00" level=warning msg="USER_AGENTS environment variable is not set. Using default user agent."
time="2024-10-01T14:21:23-07:00" level=info msg="Loaded .env from /Users/brendanplayford/masa/masa-oracle/.env"
Running Suite: Scrapers Suite - /Users/brendanplayford/masa/masa-oracle/pkg/tests/scrapers
==========================================================================================
Random Seed:<PHONE_NUMBER>

@mudler how do we configure CI to use environment variables to run tests?
Also I am having issues loading .env for some reason. Maybe you can take a look and help?
brendanplayford@Brendans-MacBook-Pro masa-oracle % ginkgo -v ./pkg/tests/scrapers
time="2024-10-01T14:21:23-07:00" level=warning msg="Error loading .env file" error="open .env: no such file or directory"
time="2024-10-01T14:21:23-07:00" level=warning msg="USER_AGENTS environment variable is not set. Using default user agent."
time="2024-10-01T14:21:23-07:00" level=info msg="Loaded .env from /Users/brendanplayford/masa/masa-oracle/.env"
Running Suite: Scrapers Suite - /Users/brendanplayford/masa/masa-oracle/pkg/tests/scrapers
==========================================================================================
Random Seed:<PHONE_NUMBER>

Other than this tests are working great!

that just shows how the code is too much tied on the configuration and viper generally. When building the first integration tests what I did is trying to move away from depending on a file first and instrument it from the code. (edited)
Ideally the code exercised in the integration tests shouldn't depend on a file on the system, because as you can see now things gets hairy really quickly with tests
My reasoning usually in this case is:

e2e tests shoud prepare all the necessary (e.g. .env files. configuration in the system, etc.) as similar as you would run the software via CLI
Integration tests and unit tests shouldn't depend on system settings such configuration files - if the code does it should be refactored to NOT depend on specific files, and make it load configuration files only from specific entrypoints (e.g. in packages dedicated to configure and then pass the configuration down the line to the components which are under test)

Why: it forces a clear separation of domains in the code which tends to help into writing code which is more "pluggable" and testable
As we are not yet at building e2e test suites I'd recommend to go ahead and refactor the code and avoiding config singletons all the way down to the methods until you figure out which lines are calling the singleton and then loading the config/env files.
This is basically what I did when untie-ing the Masa node from the config instance, if you look closer after my refactoring the masa node doesn't call anymore the config instance which in turns load the .env file and expects all the configurations. Now there is a node options which is instrumented entirely via code: https://github.com/masa-finance/masa-oracle/blob/main/node/options.go#L12 and for instance the configuration is read and translated to node options here: https://github.com/masa-finance/masa-oracle/blob/7e749047932985d54c1911ae4f8e032b0e7d67bc/cmd/masa-node/config.go#L10 https://github.com/masa-finance/masa-oracle/blob/main/node/options.go
it's still misplaced - that function shouldn't be there but more closer to the data that needs to generate the options (in this case AppConfig), but that's still in WIP as I didn't wanted to move too many things in one PR

@mudler I need to re-write this test now. I will do this tomorrow probably


Solving Issue Request!!

Please if you can, solve this issue it will help us a lot, how you said that, clone this and unzip and add it to the extensions in chrome i did all that but it is not working in leetcode i dont know why, i did refresh multiple times but still does'nt work !!

#99


🌱 Use catalog availability

Description

Reviewer Checklist

[ ] API Go Documentation
[ ] Tests: Unit Tests (and E2E Tests, if appropriate)
[ ] Comprehensive Commit Messages
[ ] Links to related GitHub Issue(s)

Going to leave this as draft with test failed until https://github.com/operator-framework/catalogd/pull/421 (its companion) merges.

/hold Adding a temp (for few hours) hold to make sure we do not merge before I get a chance to review it.

/hold cancel


fix: avoid scan errors from deleted images

Avoid these recurrent email (despite tag and release removed from repo)


Fix Azure Client /openai/deployments route override to be used more w…

…idely

[ ] I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested
Additional context & links
Hello, there seems to be a problem with the openai python client regarding the AzureOpenAI client.
While the Python client operates seamlessly with most endpoints, such as chat/completions (as detailed in the API reference: https://platform.openai.com/docs/api-reference/chat/create), I've identified a discrepancy when it comes to training-related routes.
It appears that the AzureOpenAI Client incorrectly appends /openai to the request URLs for training operations. This modification results in erroneous routes that my Azure deployment does not recognize, as it does not support URLs containing the openai segment.
The routes that get affected are:

https://platform.openai.com/docs/api-reference/fine-tuning/create
https://platform.openai.com/docs/api-reference/fine-tuning/list-events
https://platform.openai.com/docs/api-reference/fine-tuning/list
https://platform.openai.com/docs/api-reference/files/create
etc

My fix resolves these issues. Would like someone to take a look.
Thanks

cc @kristapratico

Hi @isebasus - can you share an example of how you're building the Azure client and calling the fine-tuning operations? Fine tuning requests should include the /openai/ segment according to docs so I'm curious what could be happening to create the erroneous routes.

Here is my client setup:
class OpenAIService:
    def __init__(self, token_manager: TokenManager):
        self.token_manager = token_manager
        self.token_url = "<token_url"
        self.scope = "azureopenai-readwrite"
        self.client = self.__create_client()

    def __create_client(self) -> AzureOpenAI:
        token = self.token_manager.get_oauth_token(self.token_url, CLIENT_ID, CLIENT_SECRET, self.scope)

        openai.api_type = "azure"
        openai.api_base = "<base_url>"
        openai.api_version = "2024-05-01-preview"
        openai.api_key = token

        return AzureOpenAI(
            api_version="2024-05-01-preview",
            api_key=token,
            azure_endpoint="<base_url>",
        )

Could it be something wrong with the AzureOpenAI deployment itself?

@isebasus can you share what you're passing for azure_endpoint? Is this just the endpoint to your Azure OpenAI resource, like "https://{resource-name}.openai.azure.com/"?
In terms of not recognizing deployments, another thing to keep in mind is that only a subset of models / model versions support fine-tuning: https://learn.microsoft.com/azure/ai-services/openai/concepts/models?tabs=python-secure%2Cglobal-standard%2Cstandard-chat-completions#fine-tuning-models
When calling fine-tune create, you need to pass the model name (not deployment name) with its model version. So for gpt-4o (2024-08-06), you would pass model=gpt-4o-2024-08-06.

@kristapratico the Azure deployment is an Internal NVIDIA instance, so I cannot share the azure_endpoint. However yes, it is something like "https://{domain_name}/llm/v1/azure/".
Here is the function I am using to create a training job:
def create_training_job(self, training_path: str) -> str:
        """
        Creates a training job with the OpenAI API using the content of the specified training file.

        :param training_path: The file path to the training data to be uploaded for fine-tuning.
        :return: The ID of the created fine-tuning job.
        """
        try:
            # Upload training file
            file = self.client.files.create( # FAILS AT THIS CALL
                file=open(training_path, "rb"),
                purpose="fine-tune"
            )

            while True:
                file_status = self.client.files.retrieve(file.id) # FAILS AT THIS CALL TOO
                if file_status.status == 'processed':
                    break
                print(f"File: {file.filename} is still uploading...")
                time.sleep(1)

            print(f"File: {file.id} has been uploaded, creating fine_tuning job")

            # Create fine tuned model
            fine_tune_job = self.client.fine_tuning.jobs.create( # FAILS AT THIS CALL TOO
                training_file=file.id,
                model="gpt-4o-2024-08-06"
            )

            return fine_tune_job.id

It looks like we are using gpt-4o-2024-08-06

@isebasus gotcha, thanks for the info. The azure_endpoint keyword argument is really meant to work with an Azure OpenAI resource endpoint in the shape of "https://{resource-name}.openai.azure.com/". If you are using a custom endpoint, then you want to pass base_url into AzureOpenAI instead of azure_endpoint. base_url won't append the /openai/ segment. Can you give that a try and let me know if it works for you?

Hi @kristapratico, sorry for the late response - I just tried out replacing the base_url with the azure_endpoint input parameters and it worked! Thanks for all the help. I think we can close this PR :)


Upgrade http, rest and openapi3 library to 0.59.1 fix

Waiting for another hotfix to the compiler as well

I don't have any context on this. Why ask me to review it?

I don't have any context on this. Why ask me to review it?

I didn't explicitly it just did automatically, not sure you need to review here, just upgrading typespec versions

Because the PR touches the Swagger in openai data-plane lib, hence it looped in 2 reviewers for Swagger change on data-plane.


Categorize and Filter Symptoms by ICPC Categories

ICPC Categorization is now part of standard response, so we can move this issue forward


Question about  component.

Hello, I return my data as json with fetch, I need to view the user and pull the data into it as a table, and I use your table  component which is ready when pulling the data,
For example model user, notes from user model will come into the table.
How can I do this in the fastest way? I am currently using fetch but I think this is not the right method.
If you say compile on the homepage, let's say there is 100K+ data, if I index it to the homepage, this would not be very correct, I need to use paginate in any way, how should I follow the path?

Even if I send data directly to the page with inertia, when I do an automatic search, for example http://<IP_ADDRESS>:8000/admin/players/view/10 returns to this address because I have this table as a compontent in the tab.

Hello, the Listing component handles pagination, you just need to send paginated data from the backend. This means using ->paginate() in your Eloquent query. You can check the implementation in the index method of the generated controllers. With this approach, the page should not reload when a search is applied.

Yes, but this is a problem if you use the table in , the search part does not work.

Are you using the search in the same way as it is used in the generated Controllers and Requests?
In the controller you need to have:
QueryBuilder::for(YOUR_MODEL_CLASS)
->allowedFilters([
            AllowedFilter::custom('search', new FuzzyFilter(
                YOUR_FIELDS_TO_SEARCH_IN
            )),
        ])
->paginate();

In request you should have this rule:
'search' => ['sometimes', 'string'],

If I understand correctly, you're essentially trying to do the same thing as in the generated listings, so you should follow the implementation used there.


upcoming: [DI-20555] - Added Alert tab and General Information components for Create Alert Definition form

Description 📝
Added Alert tab under Monitor for the alerting services features. Added General Information components for Create Alert Definition flow
Changes  🔄
List any change relevant to the reviewer.

Added Alerts tab under Monitor which is visible when at least one flag of the sub-tabs is accessible(via feature flag)
Added Recent activity, Alert Definitions, Notification channels tabs under Alert and the tabs visible only when they're accessible through feature flags
Added a Create button for the Create Alert Definition form under Alert Definitions
Added the General Information components under the create alert definition flow

Target release date 🗓
Please specify a release date to guarantee timely review of this PR. If exact date is not known, please approximate and update it as needed.
Preview 📷
Include a screenshot or screen recording of the change
:bulb: Use <video src="" /> tag when including recordings in table.

Before
After

How to test 🧪

Switch to mock user as APIs are not live and for the feature flags(Legacy MSW handlers)
The Create Alert definition form is rendered after clicking on the Create button under Alert definition

As an Author I have considered 🤔
Check all that apply

[x] 👀 Doing a self review
[x] ❔ Our contribution guidelines
[x] 🤏 Splitting feature into small PRs
[x] ➕ Adding a changeset
[x] 🧪 Providing/Improving test coverage
[x] 🔐 Removing all sensitive information from the code and PR description
[x] 🚩 Using a feature flag to protect the release
[ ] 👣 Providing comprehensive reproduction steps
[ ] 📑 Providing or updating our documentation
[ ] 🕛 Scheduling a pair reviewing session
[x] 📱 Providing mobile support
[ ] ♿  Providing accessibility support

@santoshp210 This PR is very large with a diff of over 1200 lines. It includes multiple new components, new queries and types, and validation changes. Can you please break this down before we give it a more thorough review?

@mjac0bs Understood, but this is already a divided portion of the feature. Can I get an idea of how smaller to split this part ? Should we handle validation, types and requests, components separately or logical chunks of the code with few components and their relevant types, requests and validation.

Can I get an idea of how smaller to split this part ? Should we handle validation, types and requests, components separately or logical chunks of the code with few components and their relevant types, requests and validation.

@santoshp210 Sure - either are reasonable approaches, but the second option, breaking things down into logical chunks, tends to provide more context surrounding the changes. In turn, this makes it easier to test and review. A reasonable-sized diff is a max of about 500 lines.

Okay sure, I will discuss this with the team and will take action at the earliest. Thank you!

@mjac0bs , Yeah we will split it into two. So we'll raise a new pr with the appropriate changes . This pr then will be closed ?

@santoshp210 We can close this one since it doesn't have any in progress reviews. 👍🏼 Thanks for breaking the changes down.


fix(file-tree): refresh promise never resolve

Types

[x] 🐛 Bug Fixes

Background or solution
Changelog
refresh promise never resolve while multiple calls

/publish

Codecov Report

Merging #709 (1c03811) into main (513bf75) will decrease coverage by 0.64%.
The diff coverage is 90.00%.

@@            Coverage Diff             @@
##             main     #709      +/-   ##
==========================================
- Coverage   59.29%   58.64%   -0.65%
==========================================
  Files        1209     1209
  Lines       76770    74801    -1969
  Branches    16256    15530     -726
==========================================
- Hits        45518    43867    -1651
+ Misses      28418    28197     -221
+ Partials     2834     2737      -97

Impacted Files
Coverage Δ

...es/file-tree-next/src/browser/file-tree.service.ts
62.02% <90.00%> (+0.11%)
:arrow_up:

packages/addons/src/index.ts
0.00% <0.00%> (-100.00%)
:arrow_down:

packages/extension/src/node/index.ts
0.00% <0.00%> (-57.90%)
:arrow_down:

packages/core-node/src/connection.ts
11.59% <0.00%> (-53.63%)
:arrow_down:

packages/editor/src/browser/breadcrumb/default.ts
52.50% <0.00%> (-27.50%)
:arrow_down:

...s/extension/src/node/extension.profiler.service.ts
0.00% <0.00%> (-26.67%)
:arrow_down:

...owser/src/authentication/authentication.service.ts
28.23% <0.00%> (-23.79%)
:arrow_down:

...c/browser/vscode/api/main.thread.authentication.ts
32.72% <0.00%> (-20.72%)
:arrow_down:

...n/src/hosted/api/vscode/ext.host.authentication.ts
51.63% <0.00%> (-18.86%)
:arrow_down:

packages/core-node/src/bootstrap/app.ts
57.00% <0.00%> (-10.00%)
:arrow_down:

... and 59 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 513bf75...1c03811. Read the comment docs.


Add linter to Canary and fix go 1.23 linting issues

This adds a lint validation to the Canary target.
Currently done on windows (because of the go install on the host), but will move to an independent stage after cleaning up the go_version detection routine.
Once green, we should merge ASAP to help with #3320

Thanks, in the commit message could you mention the fixed errors such as SA4032: due to the file's build constraints, runtime.GOOS will never equal "windows" (staticcheck)?
The commits can be also split for the each of the SA%d fixes.

go 1.23 linting issues

IIUC this is the matter of golangci-lint rather than go itself?

go 1.23 linting issues

IIUC this is the matter of golangci-lint rather than go itself?

I think it is a combination of both? (eg: golangci-lint with that version of go)
Anyhow, thanks for the comments.
Will integrate all and split in different commits.
Right now struggling with windows (as usual), will re-request review when done.

@AkihiroSuda canary lint is green. All comments addressed.


Explore using parallel maven build to speed up CI builds

Explore maven option -T that will parallelize the build and reduce the CI pipeline build time. Finding the right argument to pass to -T might be the tricky part.
https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3

// cc @mitchdenny

I think that -T 1C looks like the correct option since it will not over commit based on the number of cores on the VM (since that varies between environments).

It's possible 1C is correct, but I would like to see if 1.5C or 2C is even better. We will investigate and see. As Srikanta notes, the work is mostly I/O bound, so CPU cores will be sitting idle and we might benefit from multiple threads per core.

I tried this at https://github.com/Azure/azure-sdk-for-java/pull/4983, but there was no discernible difference. I may have made a mistake (please double-check @srnagar), but I think we should just close this issue.

Closing this issue based on Jonathan's comments above.


Not able to change configuration of the integration

The problem
I installed the integration after update of afvalwijzer 2024.12.1.
My provider is mijnafvalwijzer
I I do want to change the configuration it is not possible.
The configuration keeps as it was

Version of afvalwijzer integration having the issue?
afvalwijzer - 2024.12.1
Version of Home Assistant Core having the issue?
core - 2024.12.0
Anything in the logs that might be useful for us?
no errors

Additional information
No response

manualy deleted all  sensor sensor items, deleted the app, re installed is and now it works again

I tried also, but not working for me

I missed the release note saying that you have to delete your current configuration of Afvalwijzer, and re-add it via Add integration flow. Info is here https://github.com/xirixiz/homeassistant-afvalwijzer/releases/tag/2024.12.01. Maybe you have missed it too?
Working for me again now.

I missed the release note saying that you have to delete your current configuration of Afvalwijzer, and re-add it via Add integration flow. Info is here https://github.com/xirixiz/homeassistant-afvalwijzer/releases/tag/2024.12.01. Maybe you have missed it too? Working for me again now.

No. The integration itself is working fine.
I removed the old sensor and added the settings by the configuration itself
Except I can not change the configuration settings as shown in the picture above

I mean you need to delete the previous configuration, not the sensors. Is suppose when there is an old configuration file or code in HA configuration.yaml, things get tripped up. If that is all in order, I'm afraid I don't know what the problem is.

I did remove all before installing the new integration

I should have done this before, but I tested it today. Indeed I can also not change the configuration of the integration. I can change all the value, but when I click Submit, things do not get saved.
So that's a +1 from me.

Hi, this is odd. Somehow I missed all open issues. I just got a new laptop and suddenly a lot of open issues are visible. I`ll have a look and come back to it asap. Thanks!

Fixed in the upcoming release (end of this week)


RuntimeError: output with shape [77, 12, 1, 1] doesn't match the broadcast shape [77, 12, 77, 77]

It's hard to make the file diffsynth/models/attention.py run.
Please help,Compared to comfyui,i really like your code.
Thanks for your harding work.
output_paths = sample(
File "/code/./diffsynth_w.py", line 148, in sample
output_video = pipe(
File "/opt/conda/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/code/./diffsynth/pipelines/stable_diffusion_video.py", line 195, in call
prompt_emb_posi = self.prompter.encode_prompt(self.text_encoder, prompt, clip_skip=clip_skip, device=self.device, positive=True).cpu()
File "/code/./diffsynth/prompts/init.py", line 84, in encode_prompt
prompt_emb = text_encoder(input_ids, clip_skip=clip_skip)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/code/./diffsynth/models/sd_text_encoder.py", line 68, in forward
embeds = encoder(embeds, attn_mask=attn_mask)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/code/./diffsynth/models/sd_text_encoder.py", line 23, in forward
hidden_states = self.attn(hidden_states, attn_mask=attn_mask)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/code/./diffsynth/models/attention.py", line 76, in forward
return self.torch_forward(hidden_states[0], encoder_hidden_states=encoder_hidden_states, attn_mask=attn_mask)
File "/code/./diffsynth/models/attention.py", line 43, in torch_forward
hidden_states = torch.nn.functional._scaled_dot_product_attention(q, k, v, attn_mask=attn_mask)
RuntimeError: output with shape [77, 12, 1, 1] doesn't match the broadcast shape [77, 12, 77, 77]

I don't know how you run the code, but I noticed that the code in line 43 is modified by yourself. The original function used is scaled_dot_product_attention, which is a function in torch>=2.0.0. The function starting with _ is not suitable. If you are using an old version, please upgrade it. See the official document for more details about how to install torch.
This function is very important because it provides us a memory-efficient attention implementation. Therefore, we will not support the old version of torch anymore.

I don't know how you run the code, but I noticed that the code in line 43 is modified by yourself. The original function used is scaled_dot_product_attention, which is a function in torch>=2.0.0. The function starting with _ is not suitable. If you are using an old version, please upgrade it. See the official document for more details about how to install torch.
This function is very important because it provides us a memory-efficient attention implementation. Therefore, we will not support the old version of torch anymore.

Very thanks for your help!!!Let me try it.


[LevelUp] Multiple Issues with levelup

1st Issue:
The issue is after the recent update, hash being - 57e48dd100eb7ae03ee3d6bba87645304314b2ff it seems the Total Exp part got removed from the embed.
e.g.

Before 👇

After 👇

Update: Discord fixed the issue with embed (being pushed out to the left) & code block (that was broken before)
So, the code block can come back :)

2nd Issue
It's a well-known issue at this point I suppose, and you already know it, -
The mypf bg with attachment, still doesn't seem to be recognizing the attachment atall and always replies with "Uh Oh, looks like that is not a valid URL"

I tried in multiple bots yet same result.

e.g.

Test Instance

On autto

A complete new instance JUST for this test - where the cog has just been freshly installed

Sometimes it also removes the previous bg (which was set before e.g. card01) gets removed when it returns "Uh Oh, looks like that is not a valid URL"

Same story with passing a valid image link

Honestly, it's been an issue for sooo long, sometimes it works sometimes it doesn't, highly unpredictable behavior, not to mention - nothing on the console, no error.
That's all for today thank you :)

Fixed with v3.15.6


Submiting link to camper news results in 404 flash message

Links still post successfully. Seems like the logic tries to redirect to the newly created news story with an incorrect url. May be related to dasherizing.
cc @terakilobyte

Cool, I'll look into what exactly is going on.

Nathan: Man of Action!

I'm pretty sure this is because of your dasherization changes. We just need to update the save() to redirect to the story name dasherized by the new system instead of the old one.


Ll get agent info flying sitting fix

Applies a workaround to an issue that caused llGetAgentInfo to indicate that an avatar was flying when they were sitting, triggered by the avatar trying to sit while they are flying, and then llGetAgentInfo being called upon the CHANGED_ANIMATION trigger firing in the changed event.

This change looks good. It's an improvement to some pretty ugly code in there. But we can improve that in a separate step from this fix. Merging.


Pod job are not shown in pod section

I have octant 0.22.0 .  I created a job that is completed.  I don't see the pods of that job in Octant
root@test-pcl4004:~# kubectl -n test get pods,jobs
NAME                                    READY   STATUS      RESTARTS   AGE
pod/gateway-6c45f8c87d-nhg4g            0/1     Running     0          21h
pod/job-test-2p6gx                      0/1     Completed   0          47s
pod/kubernetes-api-7bf56456-v9cvx       1/1     Running     0          3m25s
pod/twin-api-service-5778d99c8c-7kmsz   1/1     Running     0          21h
pod/twin-neo4j-core-0                   1/1     Running     0          21h

NAME                 COMPLETIONS   DURATION   AGE
job.batch/job-test   1/1           2s         47s

You can select the Phase, by default we only show Pending and Running

thanks.


When using PDF.js in node, need for worker seems overkill and many example uses of pdf.js fail.

Configuration:

Using node, not in the browser
Operating system and its version: Windows 8.1
PDF.js version: 2.9.359
Is a browser extension: No

Steps to reproduce the problem:
Example 1.
Code from Folder example getinfo.js

Works fine with version 2.8.335
Breaks with version 2.9.359

Example 2.
Code from this video (https://www.youtube.com/watch?v=V_v5vqXbnCA&t=185s)

As require("pdfjs-dist/es5/build/pdf") example uses version 2.7.570 and works with until that version
Will work with require("pdfjs-dist/es5/build/pdf") when using version 2.8.335
Will break with version 2.9.359

What went wrong? (add screenshot)

What is the expected behavior? (add screenshot)
Expected behaviour is for node example to work straight after npm install pdfjs-dist and not need to install anything else as, to be honest, steps to add worker unclear and use of worker in Node probably unnecessary for most use cases.

Code from this video (https://www.youtube.com/watch?v=V_v5vqXbnCA&t=185s)

Sorry, but that's not an official example an as such we cannot provide support for it here.

As require("pdfjs-dist/es5/build/pdf") example uses version 2.7.570 and works with until that version
Will work with require("pdfjs-dist/es5/build/pdf") when using version 2.8.335
Will break with version 2.9.359

In the latest versions of the PDF.js library, you'll need to change the import path to require("pdfjs-dist/legacy/build/pdf") in Node.js environments; please refer to issue #12976 and PR #12978.

Expected behaviour is for node example to work straight after npm install pdfjs-dist and not need to install anything else as,

The current version of that file already contains the the correct import path, so it should just work without the need to install anything else. Possibly what's causing the issue is a change in how npm handles peerDependencies, which will be worked-around in a future PDF.js release; please refer to PR #13833 for additional details.

steps to add worker unclear and use of worker in Node probably unnecessary for most use cases.

Note that while Node.js environments doesn't support Web Workers, the pdf.worker.js file is still necessary since that's where all parsing actually happens (although the file should be loaded automatically in Node.js).

Closing for now since this should be working with the correct import paths (see above), and that we cannot provide additional assistance without more information; please see https://github.com/mozilla/pdf.js/blob/master/.github/CONTRIBUTING.md (emphasis mine):

If you are developing a custom solution, first check the examples at https://github.com/mozilla/pdf.js#learning and search existing issues. If this does not help, please prepare a short well-documented example that demonstrates the problem and make it accessible online on your website, JS Bin, GitHub, etc. before opening a new issue or contacting us in the Matrix room -- keep in mind that just code snippets won't help us troubleshoot the problem.


Where is https://ice1000.org/lagda/ now

https://ice1000.org/lagda/ is linked from the Agda documentation, but seems to have been moved.
This is the remaining broken link in https://github.com/agda/agda/issues/3976.

I think we shall remove it.

Fixed via https://github.com/agda/agda/commit/9a938969b35901afb865f532993350f3dc612825


Revert "Make doxygen run on a the doxygen image instead"

Reverts project-chip/connectedhomeip#11537

Hotfix to fix oxygen


Pull Up Pull down register is the wrong way around

The enum in io.c (part14-spi-ethernet) of pull up pull down register is the wrong way around.
Current implementation:
enum {
Pull_None = 0,
Pull_Down = 1, // Are down and up the right way around?
Pull_Up = 2
};
Correct implementation:
enum {
Pull_None = 0,
Pull_Down = 2,
Pull_Up = 1
};
source:

page 74 of bcm2711 ARM Peripherals

Will be solved by pullrequest #51


mumble

Package Request
Information
Name:
mumble
Description:
already known scoop app
Homepage:
https://dl.mumble.info/stable/

| mumble_client-1.4.230.x64.msi | 2022-01-16 17:34 | 31M |
-- | -- | -- | -- | --

Download link(s):
https://dl.mumble.info/latest/stable/client-windows-x64.msi
Some indication of popularity/repute:
PS C:\Users\dutch> scoop list mumble
Installed apps matching 'mumble':
mumble 1.3.4 global [extras]
It is already in scoop, but not updating since a month now.

Oh, I meant Pull Request, as in a contribution, not a package-request 😅

Sorry, I don't know how to do that (I've never did a pull request ever)...

It's not very difficult - you can read our Contributing guide to get started :)


Optional attribute with specific format

class Schema < Dry::Validation::Schema
  key(:id) { |id| id.nil? | (id.str? & id.format?(UUID)) }
end

I want to check if data hash doesn't have id or when it has then id should be uuid, so id should be in UUID format. I used str? because I checked situation when id was number too. format? method with number param raises exception.
Unfortunately when I had written above code I received:
undefined method `to_ary' for true:TrueClass

I checked backtrace and it looks:
undefined method `to_ary' for true:TrueClass
/Users/wafcio/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/dry-validation-0.1.0/lib/dry/validation/schema/key.rb:26:in `method_missing'
/Users/wafcio/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/dry-validation-0.1.0/lib/dry/validation/schema/definition.rb:6:in `key'

Checked this problem deeply and there is problem with simple nil? predicate
class Schema < Dry::Validation::Schema
  key(:id) { |id| id.nil? }
end

it returns
NoMethodError: undefined method `to_ary' for false:FalseClass
from /Users/wafcio/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/dry-validation-0.1.0/lib/dry/validation/schema/key.rb:31:in `method_missing'


Port over Tap-to-Place feature from HoloToolkit

Describe the problem
HoloToolkit has a Tap-to-Place feature that is missing in MRTK:
https://github.com/microsoft/MixedRealityToolkit-Unity/blob/htk_release/Assets/HoloToolkit/SpatialMapping/Scripts/TapToPlace.cs
Describe the solution you'd like
Reimplement the Tap-to-Place feature in MRTK.

Not sure if this is helpful to anyone, but I figured out a decent work around until this script is available. I used the "Interactable" script and the "Orbital" solver components.
I then did a little bit of configuring to make it work how I want it to:

I set up the interactable component to be a toggle button by changing the dimensions to equal 2.
I then selected the "InteractableOnToggleReceiver" Event Type.
Then I created a script that has two methods, one that turns on the solver and one that turns off the solver. (This is pretty easy, just get a reference to the "SolverHandler" and then change the boolean "UpdateSolvers" to true or false.)
On the "OnSelect()", I have it set up to call the method that turns on the solver.
On the "On Deselect()", I have it set up to call the method that turns off the solver.


Changed python to python3

I usually use alias py="python3", so there may be a mix of python and python3 command in the documentation.
I think it is not good about that.
In venv environment, python is an alias for python3, and in conda environment, python3 command does not exist, so I think python is not a bad choice in that example commands.

See also the discussion in cpython.
Prefer "python" over "python3" for command line examples in docs
python seems to be preferred.


cmdline mappings mess up history

With
call plug#begin('~/.vim/plugged')
  Plug 'https://github.com/mhinz/vim-grepper'
  Plug 'https://github.com/vim-scripts/cmdline-completion'
call plug#end()

filetype plugin indent on
syntax enable

cnoremap <SID>(up)   <up><home>
cmap     <expr>   <C-n> getcmdpos() == 1 ? '<SID>(up)' : '<Plug>CmdlineCompletionForward'

as .vimrc, hitting <c-p> after :Grepper does not yield any matches, even if the same string in the same directory with the same searchtool and paramters did so before. Also, say foo was searched. Then $$$cr###foo appears in the history.

What is <sid>(up)?
Can you reproduce it without those mappings and the cmdline-completion plugin?

It is the mapping cnoremap <SID>(up)   <up><home>
Yes, by

cnoremap <SID>(up)      <up><home>
cnoremap <SID>(CtrlK)   <c-k>
cmap     <expr>   <C-k> getcmdpos() == 1 ? '<SID>(up)' : '<SID>(CtrlK)'

and hitting <ctrl-k> after :Grepper.

Here's what I tried (note that I added set nocompatible to the config):

In the end I use <c-k> multiple times to get the previous commands.

The issue is about

first :Grepper, searching for  foo, and
then :Grepper<cr> and hitting <c-k> to search for foo again.

Suddenly there are no matches anymore because foo turned into $$$cr###foo.

Here's the exact config I use:
" vimrc

set nocompatible

call plug#begin('~/.vim/plugged')
  Plug 'https://github.com/junegunn/vim-plug'
  Plug 'https://github.com/mhinz/vim-grepper'
  Plug 'https://github.com/vim-scripts/cmdline-completion'
call plug#end()

filetype plugin indent on
syntax enable

cnoremap <SID>(up)      <up><home>
cnoremap <SID>(CtrlK)   <c-k>
cmap     <expr>   <C-k> getcmdpos() == 1 ? '<SID>(up)' : '<SID>(CtrlK)'

And I run it via vim -i NONE -u vimrc vimrc. Can you try the exact same steps? It all seems to work for me:

For reference, the code that is supposed to handle this is here and here.

Followed by :Grepper<cr>, foo<cr>, :Grepper<cr> and <c-k><cr>

I also commented out /etc/vimrc which didn't make a difference. The Vim version is VIM - Vi IMproved 8.0 (2016 Sep 12), Included patches: 1-566

Does the same happenw with :Grepper<cr>, <up>?

Looking at your gif, something is off: After :Grepper<cr>, foo<cr> and :Grepper<cr>, now <c-k> should give foo but bar it is ? Also, you already see flashing up that foo is saved as ###foo$$$.

No, with <up> everything is fine as it is not remapped. Only <c-k> is.

Looking at your gif, something is off: After :Grepper, foo and :Grepper, now  should give foo but bar it is ?

No, I do :Grepper<cr>, foo<cr>, :Grepper<cr>, bar<cr>, :Grepper<cr>, <c-k>, <c-k>.

Also, you already see flashing up that foo is saved as ###foo$$$.

Yes, that magic sequence is an unavoidable detail. When you hit <cr>, this code gets executed. Later the original query is added to the input history: https://github.com/mhinz/vim-grepper/blob/master/autoload/grepper.vim#L540

No, with  everything is fine as it is not remapped. Only  is.

But when you do :Grepper<cr>, <c-k>, your mapping is supposed to do <up><home>. (As it does in my gif.) When <up> is working for you, but not <c-k>, it suggests that something else than grepper is the issue here.

Hmm, perhaps a wrong terminal sequence for <home> but the error is the same in gvim.

The input line history in .viminfo is
# Input Line History (newest to oldest):
@$$$cr###foo
|2,3,1494416570,,"$$$cr###foo"
@$$$cr###helpful
|2,3,1494415941,,"$$$cr###helpful"
@''
|2,3,1494415930,,"''"
@foo
|2,3,1494415913,,"foo"
@$$$cr###$$$cr###helpful
|2,3,1494347323,,"$$$cr###$$$cr###helpful"
@helpful
|2,3,1494347290,,"helpful"
@help
|2,3,1494322064,,"help"

Somehow the magic sequence is not removed.

The line  execute 'cnoremap <cr>' s:magic.cr .'<cr>' seems suspect.

As you do it,
:Grepper<cr>, foo<cr>, :Grepper<cr>, bar<cr>, :Grepper<cr>, <c-k>, <c-k> it is not reproducible. You have to hit <cr> instead of <c-k> the second time. That is:
:Grepper<cr>, bar<cr>, :Grepper<cr>, <c-k>, <cr>.

I get it now. It's because of your <home>. Remapping <cr> is needed for adding the magic sequence, but the code expects that sequence at the end of the input.
If you do <c-k> in the above example, bar will appear but your cursor will be in the front. When you now hit <cr>, the entry $$$cr####bar will be added to the input history.
People and their crazy workflows. :-P Let me see if there's a good way to handle that from grepper.

Please see if the latest commit is working for you.

Thanks! I found those <c-j/k> remappings to combine history and cmd-line completion quite elegant, so  I grew attached to them. Thanks for making Grepper work with 'em!

:+1: And thanks for your quick feedback!


Add Github auth provider

Backend is in and ready for UI integration.

Version - 2.0 master 2/23
Verified added - bug will be entered separately from this issue.

@tfiduccia I am playing around with this feature and found some issues. Do you want any reports for this and if that's the case should I create separate issues?
Issues found where:

Not being able to remove users/org once added.
Organizations does not load properly (tried added "saiaku-gaming" ended up blank).

@PhroZenOne Can you create any new issues that you've found?


[8pt] Source column has None values in usgs_gage_crosswalk.py

-----------------------------------------------------------------
USGS Crosswalk 13080002 0
Tue Sep  5 15:52:31 UTC 2023
5 gages in branch 0
Traceback (most recent call last):
  File "/foss_fim/src/usgs_gage_crosswalk.py", line 150, in <module>
    gage_crosswalk.run_crosswalk(input_catchment_filename, input_flows_filename, dem_filename, dem_adj_filename, output_directory)
  File "/foss_fim/src/usgs_gage_crosswalk.py", line 62, in run_crosswalk
    self.write(output_directory)
  File "/foss_fim/src/usgs_gage_crosswalk.py", line 104, in write
    elev_table.source = elev_table.source.apply(str.lower)
  File "/usr/local/lib/python3.8/dist-packages/pandas/core/series.py", line 4630, in apply
    return SeriesApply(self, func, convert_dtype, args, kwargs).apply()
  File "/usr/local/lib/python3.8/dist-packages/pandas/core/apply.py", line 1025, in apply
    return self.apply_standard()
  File "/usr/local/lib/python3.8/dist-packages/pandas/core/apply.py", line 1076, in apply_standard
    mapped = lib.map_infer(
  File "pandas/_libs/lib.pyx", line 2834, in pandas._libs.lib.map_infer
TypeError: descriptor 'lower' for 'str' objects doesn't apply to a 'NoneType' object
Command exited with non-zero status 1
        Command being timed: "/foss_fim/src/run_unit_wb.sh"
        User time (seconds): 3141.24
        System time (seconds): 277.93
        Percent of CPU this job got: 106%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 53:16.14
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 12847372
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 3997
        Minor (reclaiming a frame) page faults: 86896418
        Voluntary context switches: 148132
        Involuntary context switches: 54234
        Swaps: 0
        File system inputs: 5230816
        File system outputs: 53523544
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 1

***** An error has occured  *****

The source is defined here in src/usgs_gage_unit_setup.py. For some reason, it appears that AHPS sites that are not USGS gages are not getting loaded correctly. The location_id is being filled with bogus lids, which I think is causing a bad WRDS call.