Error while running a program

What Operating System(s) are you seeing this problem on?
macOS (Apple Silicon)
dlib version
19.24.2
Python version
3.9.6
Compiler
I did pip install
Expected Behavior
I am making a face recognition program so it should work according .
Current Behavior
It showed me an error.
(.venv) anshtyagi@MacBook-Air facereco % /Users/anshtyagi/Documents/facereco/.venv/bin/python /Users/anshtyagi/Documents/facereco/main.py
Traceback (most recent call last):
  File "/Users/anshtyagi/Documents/facereco/main.py", line 2, in <module>
    import face_recognition
  File "/Users/anshtyagi/Documents/facereco/.venv/lib/python3.9/site-packages/face_recognition/__init__.py", line 7, in <module>
    from .api import load_image_file, face_locations, batch_face_locations, face_landmarks, face_encodings, compare_faces, face_distance
  File "/Users/anshtyagi/Documents/facereco/.venv/lib/python3.9/site-packages/face_recognition/api.py", line 4, in <module>
    import dlib
  File "/Users/anshtyagi/Documents/facereco/.venv/lib/python3.9/site-packages/dlib/__init__.py", line 19, in <module>
    from _dlib_pybind11 import *
ImportError: dlopen(/Users/anshtyagi/Documents/facereco/.venv/lib/python3.9/site-packages/_dlib_pybind11.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_DGifCloseFile'

Steps to Reproduce
Don't know
Anything else?
No response

Can you provide a reproducible example?

This is a problem some people run into, it's not related to dlib, but rather there is a broken copy of libgif on your system and dlib is trying to use it. Try grabbing the code from https://github.com/davisking/dlib/pull/2912 and installing from that with python setup.py install.  That new cmake setup might realize to avoid the broken libgif.  Let us know if that doesn't work and I'll tell you another way to definitely avoid it :)

it says libgif not found while removing

I'm not sure what you mean by that. Post the output text of what you did.

@anshtyagi0 isn't it called giflib, at least that's what's called on Linux.

No I mean, get a copy of dlib from the git branch in https://github.com/davisking/dlib/pull/2912.  Then do python setup.py install after first removing whatever copy of dlib you have.  And then paste in what happens.

ok so while installing using GitHub I got this:
[100%] Linking CXX shared module /Users/anshtyagi/Documents/facereco/dlib/build/lib.macosx-10.9-universal2-cpython-311/_dlib_pybind11.cpython-311-darwin.so
[100%] Built target _dlib_pybind11
creating build/bdist.macosx-10.9-universal2/egg
copying build/lib.macosx-10.9-universal2-cpython-311/_dlib_pybind11.cpython-311-darwin.so -> build/bdist.macosx-10.9-universal2/egg
creating build/bdist.macosx-10.9-universal2/egg/dlib
copying build/lib.macosx-10.9-universal2-cpython-311/dlib/__init__.py -> build/bdist.macosx-10.9-universal2/egg/dlib
byte-compiling build/bdist.macosx-10.9-universal2/egg/dlib/__init__.py to __init__.cpython-311.pyc
creating stub loader for _dlib_pybind11.cpython-311-darwin.so
byte-compiling build/bdist.macosx-10.9-universal2/egg/_dlib_pybind11.py to _dlib_pybind11.cpython-311.pyc
creating build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/PKG-INFO -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/SOURCES.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/dependency_links.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/not-zip-safe -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/top_level.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
writing build/bdist.macosx-10.9-universal2/egg/EGG-INFO/native_libs.txt
creating 'dist/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg' and adding 'build/bdist.macosx-10.9-universal2/egg' to it
removing 'build/bdist.macosx-10.9-universal2/egg' (and everything under it)
Processing dlib-19.24.99-py3.11-macosx-10.9-universal2.egg
removing '/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg' (and everything under it)
creating /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg
Extracting dlib-19.24.99-py3.11-macosx-10.9-universal2.egg to /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
dlib 19.24.99 is already the active version in easy-install.pth

Installed /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg
Processing dependencies for dlib==19.24.99
Searching for dlib==19.24.99
Reading https://pypi.org/simple/dlib/
Download error on https://pypi.org/simple/dlib/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) -- Some packages may not be found!
Couldn't retrieve index page for 'dlib'
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) -- Some packages may not be found!
No local packages or working download links found for dlib==19.24.99
error: Could not find suitable distribution for Requirement.parse('dlib==19.24.99')

Looks like you already have dlib installed. Try uninstalling it first.  pip uninstall dlib.

I already did that:
anshtyagi@MacBook-Air dlib % python setup.py install
running install
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing tools/python/dlib.egg-info/PKG-INFO
writing dependency_links to tools/python/dlib.egg-info/dependency_links.txt
writing top-level names to tools/python/dlib.egg-info/top_level.txt
reading manifest file 'tools/python/dlib.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'tools/python/build*'
no previously-included directories found matching 'dlib/cmake_utils/*/build*'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
adding license file 'LICENSE.txt'
writing manifest file 'tools/python/dlib.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.9-universal2/egg
running install_lib
running build_ext
Building extension for Python 3.11.4 (v3.11.4:d2340ef257, Jun  6 2023, 19:15:51) [Clang 13.0.0 (clang-13<IP_ADDRESS>)]
Invoking CMake setup: 'cmake /Users/anshtyagi/Documents/facereco/dlib/tools/python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/Users/anshtyagi/Documents/facereco/dlib/build/lib.macosx-10.9-universal2-cpython-311 -DPYTHON_EXECUTABLE=/usr/local/bin/python3 -DCMAKE_BUILD_TYPE=Release'
CMake Deprecation Warning at /Users/anshtyagi/Documents/facereco/dlib/dlib/external/pybind11/CMakeLists.txt:8 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- pybind11 v2.10.0
CMake Warning (dev) at /Users/anshtyagi/Documents/facereco/dlib/dlib/external/pybind11/tools/FindPythonLibsNew.cmake:98 (find_package):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

Call Stack (most recent call first):
  /Users/anshtyagi/Documents/facereco/dlib/dlib/external/pybind11/tools/pybind11Tools.cmake:50 (find_package)
  /Users/anshtyagi/Documents/facereco/dlib/dlib/external/pybind11/tools/pybind11Common.cmake:180 (include)
  /Users/anshtyagi/Documents/facereco/dlib/dlib/external/pybind11/CMakeLists.txt:200 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Using CMake version: 3.28.3
-- Compiling dlib version: 19.24.99
-- SSE4 instructions can be executed by the host processor.
-- Found system copy of libpng: /usr/local/lib/libpng.dylib;/Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr/lib/libz.tbd
-- Found system copy of libjpeg: /usr/local/lib/libjpeg.dylib
-- Searching for BLAS and LAPACK
-- Searching for BLAS and LAPACK
-- Checking for module 'cblas'
--   No package 'cblas' found
-- Checking for module 'lapack'
--   No package 'lapack' found
-- Found OpenBLAS library
-- Found LAPACK library
-- Configuring done (9.3s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/anshtyagi/Documents/facereco/dlib/build/temp.macosx-10.9-universal2-cpython-311
Invoking CMake build: 'cmake --build . --config Release -- -j4'
[ 69%] Built target dlib
[100%] Built target _dlib_pybind11
creating build/bdist.macosx-10.9-universal2/egg
copying build/lib.macosx-10.9-universal2-cpython-311/_dlib_pybind11.cpython-311-darwin.so -> build/bdist.macosx-10.9-universal2/egg
creating build/bdist.macosx-10.9-universal2/egg/dlib
copying build/lib.macosx-10.9-universal2-cpython-311/dlib/__init__.py -> build/bdist.macosx-10.9-universal2/egg/dlib
byte-compiling build/bdist.macosx-10.9-universal2/egg/dlib/__init__.py to __init__.cpython-311.pyc
creating stub loader for _dlib_pybind11.cpython-311-darwin.so
byte-compiling build/bdist.macosx-10.9-universal2/egg/_dlib_pybind11.py to _dlib_pybind11.cpython-311.pyc
creating build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/PKG-INFO -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/SOURCES.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/dependency_links.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/not-zip-safe -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
copying tools/python/dlib.egg-info/top_level.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO
writing build/bdist.macosx-10.9-universal2/egg/EGG-INFO/native_libs.txt
creating 'dist/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg' and adding 'build/bdist.macosx-10.9-universal2/egg' to it
removing 'build/bdist.macosx-10.9-universal2/egg' (and everything under it)
Processing dlib-19.24.99-py3.11-macosx-10.9-universal2.egg
removing '/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg' (and everything under it)
creating /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg
Extracting dlib-19.24.99-py3.11-macosx-10.9-universal2.egg to /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
dlib 19.24.99 is already the active version in easy-install.pth

Installed /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dlib-19.24.99-py3.11-macosx-10.9-universal2.egg
Processing dependencies for dlib==19.24.99
Searching for dlib==19.24.99
Reading https://pypi.org/simple/dlib/
Download error on https://pypi.org/simple/dlib/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) -- Some packages may not be found!
Couldn't retrieve index page for 'dlib'
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) -- Some packages may not be found!
No local packages or working download links found for dlib==19.24.99
error: Could not find suitable distribution for Requirement.parse('dlib==19.24.99')

is this issue only for MacOS if yes then I can get someones windows laptop for this project as I have to submit the project day after tomorrow

@davisking

I don't know what that last part is about:
Searching for dlib==19.24.99
Reading https://pypi.org/simple/dlib/
Download error on https://pypi.org/simple/dlib/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) -- Some packages may not be found!
Couldn't retrieve index page for 'dlib'
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) -- Some packages may not be found!
No local packages or working download links found for dlib==19.24.99
error: Could not find suitable distribution for Requirement.parse('dlib==19.24.99')

It shouldn't be talking to pypi.org if you are just using setup.py.  That's a purely local thing and doesn't have any business talking to pypi about dlib.
So I don't know what to tell you.  The other thing you could have done is say --no DLIB_GIF_SUPPORT as an option when installing.  That will disable trying to use libgif at all and avoid that other problem.  But I don't know why you are unable to install a python package like you are showing now.

can you tell me about windows? will it work or not?

@davisking just tell me this

It works on all platforms. I don't know why it's not working on your computer. Windows is fine too.

actually I am trying to install face_recognition in that dlib is a dependency so I think that is causing some problem

No solution found. If anyone finds then please reopen and ping me.

any fix found?


Implement native threads

Our port uses pthreads and it generally seems to work ok. There are still some issues:

configure + makefile refuses to build shared object, apparently this is related to libtool and -no-undefined parameter.

an attempt to use libSDL2.so from a Hollywood plugin crashes due to some pthread-related issue.

an attempt to build native sdl2.library crashes due to some pthread-related issue.

Having native threads would probably get rid of all these issues, or at least enable us to debug the crashes. As a reference, SDL1 uses native threads.

This would be useful for a future an AROS and OS3.x port as well. I did look at the OS4 SDL 1.2 threading implementation, but it looks needlessly complicated compared to the other platforms. A native timer implementation will probably be necessary as well.

@BSzili
I remove myself as an assignee since I'm not working at this thing currently. I agree about 1.2 complexity, any improvement on that area would be great.


Update _index.md

Aggiunto link al CV.

👷 Deploy Preview for gladia processing.

Name
Link

🔨 Latest commit
d9f2ebac08f5330d74cfe3ea82d7b4940811250b

🔍 Latest deploy log
https://app.netlify.com/sites/gladia/deploys/642ab145e7585f00085ff070


Something about training time

Hi all.
I train the pool45_crops_training.py, but it almost 4 days to get the mature model. I figure out, when the training stage, the GPU utilization is almost between 0 and 20%. So is there any wrong when i am training?
I environment is Keras 1.1 theano 0.8.2 CUDA7.5 CUDNN 5.0 python 2.7

Hi!
It also took some days for me. The GPU utilization is low because there is processing each time an action is performed, which is not performed on GPU. If you optimize it, let us know!
Míriam


Small steps keywords

The problem you're addressing (if any)
No response
Describe the solution you'd like
Keywords should be as simple as possible, remember that Robot Framework is used to easily read what a given step does. We should look at it as if we were writing manual instructions and the keywords should correspond to the next steps of such instructions.
This will allow for easier reuse of such keywords, they will not be strongly related to a given case, which may ultimately help to optimize the number of keywords.
https://github.com/Dasharo/open-source-firmware-validation/pull/45/files#diff-6ad7c5fb97cb1099c7ecd42f15dde3caf06d38b9fe5244afb96394454b6d6bfaL40-R80
Enter Dasharo System Features Submenu
Enter Dasharo System Features Submenu Snapshot
these are examples of keywords that, due to their extensiveness, are not clear and universal.
That's why I created keywords
Enter Dasharo Submenu
Enter Dasharo Submenu Snapshot
they are less complicated, and therefore more readable, and more universal.
When reading the test, we must immediately know how to perform it manually. In the future, this will help in automatically generating manual instructions.
https://github.com/Dasharo/open-source-firmware-validation/pull/45/files#diff-3c157e9d4762250f95224f014fa3acb1f979e1aca0c9fe3808a4156f980d7cc7R69-R73
For example, I took the liberty of creating a keyword that calls another keyword to make it easier to understand why such a step was included here. This will also help in generating documentation.
Where is the value to a user, and who might that user be?
No response
Describe alternatives you've considered
No response
Additional context
No response

https://github.com/Dasharo/open-source-firmware-validation/pull/45/files#diff-3c157e9d4762250f95224f014fa3acb1f979e1aca0c9fe3808a4156f980d7cc7R69-R73
For example, I took the liberty of creating a keyword that calls another keyword to make it easier to understand why such a step was included here. This will also help in generating documentation.

This does not make much sense. The Power On keyword simply should never be used here. Save Changes and Reset should be used, as here: https://github.com/Dasharo/open-source-firmware-validation/pull/55/files#diff-3c157e9d4762250f95224f014fa3acb1f979e1aca0c9fe3808a4156f980d7cc7R39

This should be simple enough to reproduce manually: https://github.com/Dasharo/open-source-firmware-validation/pull/55/files#diff-3c157e9d4762250f95224f014fa3acb1f979e1aca0c9fe3808a4156f980d7cc7R28
Let me know what do you think about that.

@macpijan

Save Changes and Reset should be used

You're right. I followed the example of reset-to-defaults.robot, which unfortunately does not contain such a keyword.

Let me know what do you think about that.

The test is understandable what it does step by step.
Enter Setup Menu Tianocore And Return Construction
Enter Submenu From Snapshot And Return Construction
This is a great example that a good keyword name can replace keyword documentation.


@OptionValues annotation doesn't support lazy values as a return type

When there is an accessor annotated with @OptionValues the only supported types is Collection.
We should support also lazy types for consistency, e.g. Provider<List<T>>.
One example is https://github.com/gradle/gradle/blob/13e8991cac07673969541534e45bed9e1788155b/platforms/software/build-init/src/main/java/org/gradle/api/tasks/wrapper/Wrapper.java#L337-L340
If you try to use a lazy type, you get:
@OptionValues annotation not supported on method 'getAvailableDistributionTypes' in class 'org.gradle.api.tasks.wrapper.Wrapper'.
Supported method must be non-static, return a Collection<String> and take no parameters.

I don't think this is release worthy. It just fixes a small missing Provider API support.


Out-ObfuscatedCommandArgumentTokenLevel3 should treat filter names like function names

Problem
The Out-ObfuscatedCommandArgumentTokenLevel3 treats function names as special CommandArgument tokens that can't be concatenated. I believe that filters should be treated the same way.
I discovered this issue while attempting to obfuscate modules in the Empire project.
This error specifically was found while attempting to obfuscate powerview.ps1
Steps to reproduce
PS> git clone https://github.com/danielbohannon/Invoke-Obfuscation.git
PS> wget https://github.com/adaptivethreat/Empire/raw/master/data/module_source/situational_awareness/network/powerview.ps1
PS> Import-Module .\Invoke-Obfuscation\Invoke-Obfuscation.psm1

[*] Validating necessary commands are loaded into current PowerShell session.

[*] Function Loaded :: Out-ObfuscatedTokenCommand
[*] Function Loaded :: Out-ObfuscatedStringCommand
[*] Function Loaded :: Out-EncodedAsciiCommand
[*] Function Loaded :: Out-EncodedHexCommand
[*] Function Loaded :: Out-EncodedOctalCommand
[*] Function Loaded :: Out-EncodedBinaryCommand
[*] Function Loaded :: Out-SecureStringCommand
[*] Function Loaded :: Out-EncodedBXORCommand
[*] Function Loaded :: Out-PowerShellLauncher
[*] Function Loaded :: Invoke-Obfuscation

[*] All modules loaded and ready to run Invoke-Obfuscation

PS> Out-ObfuscatedTokenCommand -Path .\powerview.ps1 | Out-File out
Obfuscating powerview.ps1

[*] Obfuscating 747 Comment tokens.
[*]             600 Comment tokens remaining to obfuscate.
[*]             500 Comment tokens remaining to obfuscate.
[*]             400 Comment tokens remaining to obfuscate.
[*]             300 Comment tokens remaining to obfuscate.
[*]             200 Comment tokens remaining to obfuscate.
[*]             100 Comment tokens remaining to obfuscate.

[*] Obfuscating 1990 String tokens.
[*]             1800 String tokens remaining to obfuscate.
[*]             1700 String tokens remaining to obfuscate.
[*]             1600 String tokens remaining to obfuscate.
[*]             1500 String tokens remaining to obfuscate.
[*]             1400 String tokens remaining to obfuscate.
[*]             1300 String tokens remaining to obfuscate.
[*]             1200 String tokens remaining to obfuscate.
[*]             1100 String tokens remaining to obfuscate.
[*]             1000 String tokens remaining to obfuscate.
[*]              900 String tokens remaining to obfuscate.
[*]              800 String tokens remaining to obfuscate.
[*]              700 String tokens remaining to obfuscate.
[*]              600 String tokens remaining to obfuscate.
[*]              500 String tokens remaining to obfuscate.
[*]              400 String tokens remaining to obfuscate.
[*]              300 String tokens remaining to obfuscate.
[*]              200 String tokens remaining to obfuscate.
[*]              100 String tokens remaining to obfuscate.

[*] Obfuscating 531 Argument tokens.
[*]             400 Argument tokens remaining to obfuscate.
[*]             300 Argument tokens remaining to obfuscate.
[*]             200 Argument tokens remaining to obfuscate.
[*]             100 Argument tokens remaining to obfuscate.
Exception calling "Create" with "1" argument(s): "At line:445 char:7
+ filter ("{0}{2}{1}{3}{4}"-f 'E','t-','xpor','PowerViewCS','V') {
+       ~
Missing name after filter keyword.
(errors continue)

The error message makes it pretty clear that there is a problem with the obfuscated filter name.
The powerview.ps1 file makes use of filters extensively.
Solution
It seems that the issue is that filter names should be treated as a special case, just like function names.
The following code:
# Function name declarations are CommandArgument tokens that cannot be obfuscated with concatenations.
# For these we will obfuscated them with ticks because this changes the string from AMSI's perspective but not the final functionality.
If($ScriptString.SubString(0,$Token.Start-1).Trim().ToLower().EndsWith('function'))
{
	$ScriptString = Out-ObfuscatedWithTicks $ScriptString $Token
	Return $ScriptString
}

becomes:
# Function name declarations are CommandArgument tokens that cannot be obfuscated with concatenations.
# For these we will obfuscated them with ticks because this changes the string from AMSI's perspective but not the final functionality.
# This is also true of the 'filter' keyword
If($ScriptString.SubString(0,$Token.Start-1).Trim().ToLower().EndsWith('function') -or $ScriptString.SubString(0,$Token.Start-1).Trim().ToLower().EndsWith('filter'))
{
	$ScriptString = Out-ObfuscatedWithTicks $ScriptString $Token
	Return $ScriptString
}

This fixed the issue for me.

Issue fixed in d419d0b4a0592c0cd48d7a22d462477f4b976970 release.


Verify REST upload image id JSON response

$ curl -c cjar -b cjar -k -H "Content-Type: application/octet-stream" -X POST -T obmc-phosphor-image-witherspoon.ubi.mtd.tar https://$BMC_IP/upload/image
{
  "data": "7c2c9537",   <---  image id
  "message": "200 OK",
  "status": "ok"
}

Duplicate
https://github.com/openbmc/openbmc-test-automation/issues/1356


Flaky dask test test_boost_from_prediction

https://travis-ci.org/github/dmlc/xgboost/jobs/774230442

Closing.  Happens on osx once for some reason.


IllegalAccessError: WAL not empty; commit first

So I'm not doing any puts/writes before my compact thread kicks in,  but
I'm getting this error, what is the WAL ? This happens right after a make.
Caused by: java.lang.IllegalAccessError: WAL not empty; commit first, than
compact
at org.mapdb.StoreWAL.compact(StoreWAL.java:742)
at org.mapdb.EngineWrapper.compact(EngineWrapper.java:111)
Should I be reading the MapDb source code to solve my issues or is this an
appropriate forum to post questions?
credit: Jay Houghton

Hi,
I put "localDb.commit();" before every "localDb.compact();" but still get the exception:
Exception in thread "Thread-0" java.lang.IllegalAccessError: WAL not empty; commit first, than compact
at org.mapdb.StoreWAL.compactPreUnderLock(StoreWAL.java:1100)
at org.mapdb.StoreDirect.compact(StoreDirect.java:810)
at org.mapdb.EngineWrapper.compact(EngineWrapper.java:130)
at org.mapdb.EngineWrapper.compact(EngineWrapper.java:130)
at org.mapdb.DB.compact(DB.java:1626)
My commit(), compact() are located in Thread.run(), and my DBMaker configuration is:
DB localDb = DBMaker.newFileDB(new File(filename)).closeOnJvmShutdown().make();
store size: 471M, including *.compact, *.compact.p, *.p, *.t
java -version: java version "1.8.0_11"
uname -r: 2.6.32-279.el6.x86_64
Thanks!


[IDAS ERROR]  IDACalcIC   Newton/Linesearch algorithm failed to converge.

When running the reset() and get_next_step() functions I very rarely get the error:
[IDAS ERROR]  IDACalcIC
Newton/Linesearch algorithm failed to converge.
This is happening while Reinforcement Learning. So I run the reset once, and then run get_next_step until the kite crashes (around 285 times). That is one episode, and this is repeated x times.
I get the error for the first time after around 76 episodes (each running reset once, and get_next_step 285 times on average).
One episode pseudocode:
reset()
while not crashed:
    get_next_step(action)

Output.txt containing the rollout of one episode and the first occurrance error:
---------------------------------
| rollout/           |          |
|    ep_len_mean     | 285      |
|    ep_rew_mean     | -200     |
| time/              |          |
|    episodes        | 76       |
|    fps             | 456      |
|    time_elapsed    | 84       |
|    total_timesteps | 38448    |
| train/             |          |
|    actor_loss      | 4.3e+11  |
|    critic_loss     | 3.32e+23 |
|    ent_coef        | 2.06e+10 |
|    ent_coef_loss   | -437     |
|    learning_rate   | 0.0456   |
|    n_updates       | 768      |
---------------------------------

[IDAS ERROR]  IDACalcIC
  Newton/Linesearch algorithm failed to converge.

Environment.jl:
module Environment

using Timers; tic()
using KiteModels
using KiteUtils
# using PyCall #removed pycall!!
# using Plots

const Model = KPS4

set_data_path(joinpath(@__DIR__, "../../Simulator/data"))
kcu = KCU(se());
kps4 = Model(kcu);
dt = 1/se().sample_freq
steps = 1000
step = 0
logger = Logger(se().segments + 5, steps)

GC.gc();
toc();

integrator = KiteModels.init_sim!(kps4, stiffness_factor=0.04);

function get_next_step(depower, steering)
    global step
    depower = Float32(depower)
    steering = Float32(steering)

    v_ro = 0.0

    if depower < 0.22; depower = 0.22; end
    set_depower_steering(kps4.kcu, depower, steering)

    t_sim = 0.0
    open("next_step_io.txt", "w") do io
        redirect_stdout(io) do
            t_sim = @elapsed KiteModels.next_step!(kps4, integrator, v_ro=v_ro, dt=dt)
        end
    end

    GC.gc(false)

    sys_state = SysState(kps4)
    step += 1

    return sys_state.orient[1], sys_state.orient[2], sys_state.orient[3], sys_state.orient[4], sys_state.force
end

function reset()
    global kcu
    global kps4
    global integrator
    global step
    global sys_state
    update_settings()
    save_log(logger)
    kcu = KCU(se());
    kps4 = Model(kcu);
    integrator = KiteModels.init_sim!(kps4, stiffness_factor=0.04)
    step = 0
    sys_state = SysState(kps4)
    GC.gc();
    return sys_state.orient[1], sys_state.orient[2], sys_state.orient[3], sys_state.orient[4], sys_state.force
end

function render()
    global sys_state, logger, step, steps
    if(step < steps)
        log!(logger, SysState(kps4))
    end
end

end

System:
I am running the code on IDUN High Performance Computing: https://www.hpc.ntnu.no/idun/
inside an apptainer ubuntu container.
I made a system image with Environment as a precompiled package.

Yes, thank you!
On Tue, Mar 5, 2024 at 3:27 PM Uwe Fechner @.***> wrote:

There are many possible reasons why the solver can fail.
The first thing I would try is to change the
solver settings:
``
solver:
abs_tol: 0.0006 # absolute tolerance of the DAE solver [m, m/s]
rel_tol: 0.001 # relative tolerance of the DAE solver [-]
linear_solver: "GMRES" # can be GMRES or Dense
max_order: 4 # maximal order, usually between 3 and 5
max_iter: 200 # max number of iterations of the steady-state-solver
This can be changed globally in settings.yaml, but also in a case-by-case way
e.g. by doing:
se().abs_tol=0.000006
se().rel_tol=0.0000001
The second thing to try is to reduce the stiffness of the tether
tether:
c_spring
At the beginning of a simulation I always use a low stiffness and increase it to the nominal
value when an equilibrium is reached.
Does this answers your question?
—
Reply to this email directly, view it on GitHub
https://github.com/ufechner7/KiteModels.jl/issues/44#issuecomment-1978897649,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIBR55GL3ZTQJXDJAUXW723YWXI53AVCNFSM6AAAAABCI3ONSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYHA4TONRUHE
.
You are receiving this because you authored the thread.Message ID:
@.***>

I just added the option to use the DFBDF solver, which - in general - works much better, much more stable, in average 4 times faster and half the memory usage. Please try it out and tell me if this fixes your problem.

Thanks, it fixed the problem!
On Wed, Apr 3, 2024 at 4:34 PM Uwe Fechner @.***> wrote:

I just added the option to use the DFBDF solver, which - in general -
works much better, much more stable, in average 4 times faster and half the
memory usage. Please try it out and tell me if this fixes your problem.
—
Reply to this email directly, view it on GitHub
https://github.com/ufechner7/KiteModels.jl/issues/44#issuecomment-2034801002,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIBR55HH6OB3YJ6AJQ43QB3Y3QHRFAVCNFSM6AAAAABCI3ONSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZUHAYDCMBQGI
.
You are receiving this because you authored the thread.Message ID:
@.***>


XP/level progress bar

Short description
I would like some way of viewing my progress in levelling, it doesn't need to be precise at all, just a hint for how far into the level I am.
The main reason for this would be in the future when people are in really high levels and don't have any way of knowing if your half there, almost next level or anything.
Detailed description
My idea is that the progress bar should consist of steps of some kind, not many at all, thus making it to be less likely to result in people spamming for levelling up.
For example, having 3 "steps" per level. When you are more than 33% into reaching the next level, the first step is marked, over 66% the next step is marked etc. This makes you have at least a clue of how far you are to the next level and at the same time, hopefully not making people spamming to get more levels.
Another feature that could be added to this one, is that you get more of these "steps" by leveling, for example, one step more for every 10 levels or so. I think this would be more fun for the people that are at really high levels.
I don't really know where or how this may be done, but maybe in the .me command or something. The example of values or amounts of steps etc. are just examples, I haven't thought them through that much.
Additional context:
I would like to think about how this could be done, but I sadly don't really have time for that so I will just write my thoughts when I get them and let you decide and think about how it may be implemented.

Great idea!

Good idea!
I haven't tested it, but could something like this work to make the progress bar, if placed in the .me command? To show it, I assume making a new field with the value progressMessage could work?
		let xpPercentage = profileData.xp / Math.pow(profileData.level + configData.xp.levelBaseOffset, configData.xp.levelExponent); 		var progressMessage; 		if (xpPercentage > 0 && xpPercentage <= 0.3) { 			progressMessage = "░░░░░░░░░"; 		} else if (xpPercentage > 0.3 && xpPercentage <= 0.6) { 			progressMessage = "███░░░░░░"; 		} else if (xpPercentage > 0.6 && xpPercentage < 1) { 			progressMessage = "██████░░░"; 		}


Misc MIME fixes

media: Add PDF MIME type
media: Add missing BMP and GIF to the default MediaTypes list
media: Also consider extension in FromContent

OK, there is a test failure ... will fix.

OK, now the tests should pass. As to the MIME resolution in GetRemote, the adjustment here is shortly:

Determine MIME from content
If found:

If the same as the extensionHint, all OK
If both are text formats (e.g. plain/text and application/json), use the MIME from the extension.
If not resolved from extensionHint, return zero.

The above should handle:

fake.png which contains JS (is plain text)
fake.js which contains image (or non-plain-text).

@jmooring good catch, I have pushed an update where I consider all extensions from mime.ExtensionsByType (for non text/plain).


Refactor Model/Storage and Add List Command

Resolves #65
Some important changes to note.

Appointment has been refactored to also have an appointment ID
Appointment has been refactored to hold a Person object instead of just a person ID
Storage has been refactored to also load Person objects into Appointments upon invoking toModelType() in JsonAdaptedAppointment.
UI manager has been refactored to swap between different list panels
Bug fix with edit person command where not specifying all parameters would throw error
Bug fix with add appointment command where multiple "t/" prefixes throws error. AppointmentType prefix thus changed to "ty"

LGTM from the model side!


Remove xmlns:xlink in removeXMLNS plugin

xmlns:xlink is also not required in SVGs embedded in HTML

Hi @devongovett. Thanks for PR. I think removeUnusedNS.js removes xmlns:xlink.

That only removes it if it's unused though. In this case, it's always safe to remove xlink:href in HTML. Since this plugin is opt-in, I think it makes sense to do it here as well.

I think it's better to create a separate plugin which removes xlink namespace from xlink:herf as well.
I can take care of it in a few days.


No API mode with askgd?

The runapi mode mentioned in the README doesn't seem to be available with the version of askgd that gets installed via go install github.com/mosajjal/askg/cmd/askgd@latest:
jmac@idun:~$ go/bin/askgd runapi
Error: unknown command "runapi" for "askgd"
Run 'askgd --help' for usage.
2024-03-05T23:16:44-05:00 ERR failed to execute command: unknown command "runapi" for "askgd"

jmac@idun:~$ go/bin/askgd
Use Google's AI model. This is a reverse engineered API of Gemini Web.
		in order to use this, you first need to run the browser command to get the cookies from the browser.
		note that all cookies will be stored in PLAINTEXT on this machine in the ~/.askg.yaml file.
		make sure that file is only readable by the user running this daemon.

Usage:
  askgd [command]

Available Commands:
  browser     Get cookies from browser
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  run         Run the askgd daemon

Flags:
  -c, --config string   path to YAML configuration file (default "$HOME/.askg.yaml")
  -h, --help            help for askgd
  -v, --version         show version info and exit

Use "askgd [command] --help" for more information about a command.
jmac@idun:~$

just pushed the 0.4.0 release which hopefully helps with this. Still shows as 0.3.7 in my box but that could very well be caching.


Possibly high Resource usage of Pylance

Reported by @ale-dg here https://github.com/microsoft/vscode/issues/206119#issuecomment-1996165888
Yes, but consider I did not use Pylance. Would you like me to test with it? I just activated it and opened the log trace for python and just for indexing it finishes its memory.
2024-03-13 17:51:01.092 [info] [Info - 17:51:01] (4192) Heap stats: total_heap_size=1220MB, used_heap_size=1156MB, total_physical_size=1218MB, total_available_size=2900MB, heap_size_limit=4096MB
2024-03-13 17:51:01.108 [info] [Warn - 17:51:01] (4192) Workspace indexing has hit its upper limit: 2000 files

Hi @rchiodo
I honestly don't know. Everything has been a mystery as it happens with a lot of users, especially with large notebooks and lot of markdowns.
These are my Mac specs:

System Info

Item
Value

CPUs
Apple M1 Pro (10 x 24)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
3, 3, 3

Memory (System)
16.00GB (5.96GB free)

Process Argv

Screen Reader
no

VM
0%

These are the Jupyter logs I have retrieved for @DonJayamanne disabling Pylance, with and without markdown cells.
1-Jupyter-no-MD.log
1-Jupyter-with-MD.log
With MD it takes a bit longer to run, the computer warms up a little bit and it was done with Pylance deactivated. What most people has been reporting is Python/Jupyter being slow and only a couple of us have actually found out that Pylance was using a lot of resources (see https://github.com/microsoft/vscode/issues/206119#issuecomment-1965478455, https://github.com/microsoft/vscode/issues/206119#issuecomment-1987025052 and https://github.com/microsoft/pylance-release/issues/5614). So yet again, no idea which one is it.
I also tried to retrieve the logs from Pylance, although when just opening VSCode, the log files something around 30,000 lines and some of the final lines are the ones above. Below you can find the log file.
Pylance.log
Let me know if I can be of any help (although I only know Python for DS 😜)
Best

@rchiodo I assumed that the numbers were off, and based on the comments from @ale-dg disabling Pylance improved things.
However there are other things that slow Jupyter as well, but Pylance was reported as being one of them, e.g. if CPU/Memory is used, then VS Code tends to slow down, which slows down Jupyter extensions & others.
Feel free to close this issue if there's nothing to be done here, we're already looking into other issues reported by the user.

Hi,
Before closing the issue, I was about to comment that I just ran again the large notebook with markdowns and these extensions:
Extensions (4)

Extension
Author (truncated)
Version

python
ms-
2024.2.1

vscode-pylance
ms-
2024.3.1

jupyter
ms-
2024.2.0

jupyter-renderers
ms-
1.0.17

After getting to where I wanted it (around coding cell 152), when typing in a function (i.e. OrdinalEncoder(), or any other from a library), the mini-window that pop-ups with hints loads VERY slow compared with other the previous versions when I downgraded, and it begins to lag. Also, when just beginning to type in the first parenthesis, it kind of stops everything else for loading the window (it stops around 3-5s). Then if we are lucky, it doesn't crash or lags or anything.
The same happens for the "auto-complete" function (not sure what its name is... but is the one that loads below the code for choosing a previous function or variable or something else).
Best

@ale-dg Please can you confirm that disabling Pylance extension makes it faster.
I.e. the following issue no longer exists

e mini-window that pop-ups with hints loads VERY slow compared with other the previous versions when I

Yes, you will no longer get completions, thats a different matter, however if it is still NOT faster, then we know for a fact that Pylance is not causing any delays and we can close this issue;

@DonJayamanne it ran with the same speed, but the lag with typing just went away. Would it be worth it to close this issue as it is actually a bug with Pylance lagging VSCode?
Best

but the lag with typing just went away.

lets leave this issue open, basically completions is slow

Well... just for the record, when I turned on Pylance again, I tested restarting the kernel to see what happens and all VSCode got stuck for some seconds (couldn't even scroll) and then it began to lag. In one of the issues was reported as well (and sometimes it crashes).
I also tried it without Pylance and it didn't crash. It just took its time to start running the notebook again.
Best

@ale-dg can you try this? https://github.com/microsoft/pylance-release/wiki/Collecting-data-for-an-investigation.#collecting-cpuprofiles
and provide us with *.cpuprofiles?
by the way, to make things simpler first, try it with python.analysis.indexing: false and try to repro slow completion. it will help us to see whether it is actually pylance completion slow and why it is slow.
thank you

ah, one more thing, if you are seeing the memory pressure log, your workspace might be too big for our default setup (using vscode as node)
Heap stats: total_heap_size=1220MB, used_heap_size=1156MB, total_physical_size=1218MB, total_available_size=2900MB, heap_size_limit=4096MB

Try this new setting (https://github.com/microsoft/pylance-release/pull/5602/files) we just added in 2024.3.100. pylance will get a lot bigger memory space than the default one.

My guess is this is a duplicate of the completion problem we found with Jupyter completions:
https://github.com/microsoft/pyrx/issues/4663

My guess is this is a duplicate of the completion problem we found with Jupyter completions: https://github.com/microsoft/pyrx/issues/4663

I'd gladly confirm if it's the same, but it shows me a 404 error when opening.

@ale-dg can you try this? https://github.com/microsoft/pylance-release/wiki/Collecting-data-for-an-investigation.#collecting-cpuprofiles
and provide us with *.cpuprofiles?
by the way, to make things simpler first, try it with python.analysis.indexing: false and try to repro slow completion. it will help us to see whether it is actually pylance completion slow and why it is slow.
thank you

I'll try it after a request of @bschnurr here https://github.com/microsoft/pylance-release/issues/5614#issuecomment-1998007556

Sorry... this might be quite a simple question, but I am not getting the "Pylance: start profiling" option. How do I get it working?

Hi,
I just completed the cpuprofile. I forgot to do the python.analysis.indexing: false because I have been doing so many changes and tests that I no longer know what I have changed or not, so my apologies for it. You will see a long time-gap somewhere between 14:00 and 14:12 (or somewhere around there), that is how long the notebook takes to run. After that it is all just typing to add some functions and it lagged.
Just for the fun, I tried to open the profiles with VSCode, and it crashed 🤣🫠, so maybe something you'd like to check as well.
Best
profiles.zip

The profiles don't show completions taking any time. Was it slow when you were typing? It shows like 150ms for completions to work:

This likely means the slowdown is not in pylance but somewhere else. I'm guessing it's this code here:
https://github.com/microsoft/vscode-jupyter/blob/df25cd4ba2d39227ff186c36b01e1a629e7dee88/src/standalone/intellisense/kernelCompletionProvider.ts#L96
That code is getting completions from us too but also combining them with ones from the kernel. If the kernel is slow, (which I believe you said somewhere it was slow running cells?) then completions over all would be slow.

They become slow after executing the cells. So, if I type without executing anything, they feel normal, like in old versions. When I begin executing cells, then the lags begins, something like this:

type ord = OrdinalEncoder(
lag lag lag lag lag lag lag
3 ) hint window pops
lag lag lag lag
type han
6 ) lag lag lag lag lag lag lag
auto complete appears
lag lag lag lag
allows to select or finishes the sentence
lag lag lag lag lag lag

.. and so on for the rest of the time you are working on the file
Hope it makes sense

It makes it feel not normal, like there is a complete disconnection between the keyboard, the screen, VsCode, etc

That sounds to me like it's the kernel completions then. If it works fine without running any cells, then just Pylance is involved. If it slows down only when the kernel is involved, then it's both Pylance and the Jupyter Kernel providing the completions. Given the cpu profile you sent it, Pylance isn't taking any time to compute completions, so I'm going to transfer this issue to the Jupyter extension.


[DotNetCoreCLI] Using different library for searching files

Hi @desattir, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Deepak Sattiraju). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;


add this to vscode?

Hey there!
This is definitely one of my favorite atom packages. So well executed!
I have been using vscode recently and was wondering if you knew if it were even possible to implement it as an extension there. If it is possible, it would be awesome if you made it!
Lemme know!

I think it should be possible assuming I can get D3 working in their electron environment.  I was pleasantly surprised to see MSFT using electron and npm.   Several people on my team have switched from sublime or atom to vscode, maybe one of them would be willing to do the fork and port.
Let me get v2 for Atom released and I'll take another look.

@littlebee any update on this one? I only ask because I really love your atom package and want it oh so badly for vscode.

@littlebee anything we can do to help get this ported?

@littlebee it looks like v2 is out for atom :) I second @TrevorSayre in asking if there is anything that we can do to help this module get ported over?

@tnrich,  or....    What if git-time-machine could handle directories and provide the same kinda functionality by directory, author....   What would that look like?   ;) I have some thoughts, but yeah I would love some help with it:
https://git-temporal.github.io/git-temporal/docs/design/git-temporal-teaser.mp4
The idea is to host this in a WebView in VSCode.  I have most of the functionality done and a few bugs  for the webview content in the form of a react + redux UI that gets data from a REST service that scrapes git commit information.
The rest service is crude, slow and could use some loving.  The code editor integrations still need to be done.  I'll invite everyone on the thread to the github org.
Instuctions for running the demo above locally are on the mono repo's README.md.  Oh yeah, documentation as always is lagging and could use some more work. :)

This is not mine but this is a great idea of how it may look like:
http://www.gittimemachine.com/


Can't find include file in an umbrella project

I tried this on an umbrella codebase and it struggled due to modules with an explicit include path to another application. For example:
Project
+--apps
   +--app1
   |  +--include
   |  |  +--app1.hrl
   |  +--src
   |     +--app1.erl
   +--app2
      +--src
         +--app2.erl

-module(app2).
-include("apps/app1/include/app1.hrl").

This is fine for compile, ct, xref, dialyzer, elvis, hank - the usual suite. Also fine for edoc. However, with rebar3 ex_doc I get:
can't find include file "apps/app1/include/app1.hrl"
edoc: error in doclet 'edoc_doclet_chunks': {'EXIT',error}.

Looks like this might be a problem with edoc when generating chunks. I get the same error with edoc using the following config :
{edoc_opts, [
    {preprocess, true},
    {doclet, edoc_doclet_chunks},
    {layout, edoc_layout_chunks},
    {dir, "_build/docs"},
    private,
    hidden
]}.

It could also be there's some extra opts we need to set when calling edoc , as another possibility.

A few thoughts :

As a quick fix, you could use relative paths (i.e., "../../app1/include/app1.hrl"). I believe I've seen that in OTP, regardless I can say that works.

It's not clear who's responsibility this is. I can think a patch that could go for epp, if OTP would accept it. Basically, for any include file you get the abs path for it, likewise you transform include paths provided via includes option to edoc into abs paths. I think that would work, but wouldn't know unless it was tried. I've never poked around in edoc or epp in anger.

I could also see where possibly the rebar3 edoc provider could handle this.
Regardless of the outcome of 2, we will have to support merging edoc opts I believe, or allow specifying them within ex_doc opts, the latter might favorable if there's a case where you want ex_doc but you maybe also need edoc and need different sets of options. You could probably use profiles to solve that problem too though.
In the end, in regards to 2. . we shall have to reach out to a few folks and figure out where the responsibility lies.

@starbelly, I'm experiencing the same issue. Using your 1. workaround (../stuff.hrl) I was able to bypass the issue while maintaining the lib. compilable.

Hi, @altaica!
It works fine if you use -include_lib to include the header files across apps. Please see https://github.com/erszcz/learning/blob/482c46d20a907cc541c6cd4a7648721d6d5fdbf1/rebar3-umbrella-test/apps/test2/src/test2_app.erl#L13 for an example umbrella repo.
The problem with -include and include path handling is that each app parsing source code (so Rebar3, edoc, compiler, Dialyzer, ...) have to deal with -I flags and include paths separately, which is a lot of redundant work and lot of space for inconsistencies to sneak in. With -include_lib we reuse the code path handling infrastructure from stdlib, so do away with all this redundancy. It's better to stick with -include_lib if only possible.
I hope this helps!

@erszcz, my issue is a bit different.
My app is like
+--my_app
   +--src
   |  +--drivers
   |     +--my_app_driver.erl
   +--my_app.hrl

Compilation is fine with just -include("my_app.erl"). (from my_app_driver.erl), but rebar3_ex_doc doesn't like it.
Using -include(".../my_app.hrl") the problem is somewhat solved, though I'm sure this'll break if I move my_app_driver.erl to a different location.
Thoughts?
Note: I'm using erl_opts with {i, "src"}, in rebar.config.

@paulo-ferraz-oliveira Got it, include_lib doesn't work with that. Why put the header file in the root dir of the app instead of include, though?
I'm not sure about rebar3_ex_doc, but with regard to EDoc itself it's possible to pass {includes, [Dir1, Dir1, ...]} as documented at https://www.erlang.org/doc/man/edoc.html#read_source-2. It could be added to edoc_opts in rebar.config:
{edoc_opts, [
    {preprocess, true},
    {includes, Dirs},
    ...
]}.

I think it's sensible to leave setting includes properly to the user as it's not possible to foresee every header file placement for every project out there. The standard and advised way, though, is to use include_lib.

Why put the header file in the root dir of the app instead of include, though?

I don't want the file to be importable by other lib.s.
Generating the doc. with edoc I didn't have that issue. And I did use includes.
I see, though, it's possible (https://github.com/miniclip/gen_rpc/pull/7/files#diff-427d8ddce21ccc4fae24fb547c7a82c2a793ed57208d8343e519e0aec8119ff7L122) the removal of edoc's options caused the issue, but I'll have to check that again.

BTW, this is how the option is populated in edoc_cli.erl:
https://github.com/erlang/otp/blob/56240f084d80de80d5f1c7fef94db68a6f1b81cc/lib/edoc/src/edoc_cli.erl#L105

Would it be possible to read it from the config. and use it in

I think we could even default to adding the $APP/include and $APP/src to EDoc's {include, Dirs}. It shouldn't hurt and would likely solve 95% of header placement issues.

I think I read, somewhere else, that there's a goal of merging {edoc_opts, _} (I'm not sure if it's for a transitional period or permanent) into this, but I'm not sure, so am leaving the suggestion.

Do you mean in rebar3 edoc provider? That could possibly work. It's been a minute since I've tried, but I seem to remember having trouble with edoc devoid of this plugin anyway and it was only when preprocess was set to true. That said, we should be merging the opts anyway.

I think we could even default to adding the $APP/include and $APP/src to EDoc's {include, Dirs}. It shouldn't hurt and would likely solve 90% of header placement issues.

Worth trying.

I mean: edoc_opts already supports it and it works fine. If we somehow can import that (or create a new place for the option) in ex_doc that'd be nice, since we fix the bug I mention.

That said, we should be merging the opts anyway.

If this is the case, it'll even be simpler, not only for a migration period (since consumers don't have to move stuff away from ex_doc). In any case, I'm wondering if allowing the includes option directly inside ex_doc isn't an option, for you (?)
Also, I like @erszcz's suggestion of having the default includes include ["include", "src"] already (I tend to repeat this in all my lib.s, but I'm not sure it's a generic use case).
I'll probably get to pull request and we can go from there 😄


Realtime socket communication is not normal.

I brought a new error.
await table.On(ChannelEventType.Insert, (sen, args) =>
{
Userworklist.Add(args.Response.Model<WORKS>());
});

I am checking to define a table called WORKS as Realtime as below and import it.
And when I uploaded the file, I could see this message in the debug.
[0:] 0%
[0:] 0%
[0:] 0%
[0:] 0%
[0:] 0%
[0:] 0%
[0:] 0%
[0:] 100%
[TrafficStats] tagSocket(115) with statsTag=0xffffffff, statsUid=-1

The problem is that you are receiving Realtime, but you cannot get anything.

When you receive a Realtime event, the first one feels ignored.
Do you happen to know about the content? I'm a MAUI developer, but I don't think it's an Android problem because I did the same test in the Windows environment.

I checked it out.
I made a channel with a SupabaseClient.Auth.CurrentSession.
And I wrote to monitor the event for the change and subscribed() the channel.
And it turns out that Realtime's item was anon permission.
To change this, I gave the new client an AccessKey, but the Subscribe method did not work this time. (i.e. not registered in the subscription table)
Judging from this, I thought it only works for the anon key as described in the description in the supabase, but as I told you in the last article, it works from the second time.
When I uploaded the file, the anon permission of the linked account in the Subscribe table was changed to authenticated.

In fact, I have a question about this.

Hi. After a lot of tests, I checked what the problem is.
await SupabaseClient.Realtime.ConnectAsync();
await SupabaseClient.From<WORKS>().On(ChannelEventType.All, (sen, ea) =>
{

});

Since I wrote it like this, I checked that when I called the Subscribe() method, a row was added to the realtime->subscription table and claims_role became anon.
After 10 seconds, I experienced that the claims_role changes to authenticated when I refresh.
First of all, there's a problem that I don't know why there's a 10-second delay, and if I can update this quickly, I think I can solve the fundamental problem.
The problem is that I didn't use On, I used SupabaseClient.Realtime.Channel("realtime", "public", "WORKS").
WORKS is my table here.
If I make a channel with this, it's fixed as anon and it didn't change. And when I looked at the code, I found something that I had to send parameters when I made the channel.
It's a shame there wasn't such an explanation. I don't think anyone knows that we have to send user_token to the parameter.
As I did so, I confirmed that anon changed to authenticated after about 10 seconds like the On written above.
But I can't receive any changes in real time because I'm anon for 10 seconds anyway. If I could check the changes in climes_role as an event, I would block other movements or turn the busyIndicator, but I don't know at the moment.
If there is a user Access Token key in "Authorization" by looking up the header inside the client when creating the channel, I think the code will generate user_token.
Of course, depending on the developer's intention, some people want to do it with anon.
And I told you that I uploaded the file and the permission changed?
At this time, there seems to be something updating claims_role on the server, so if  call taht, wouldn't we be able to reduce the delay of 10 seconds?
I'm sorry I only found weird things :)

Oh, I think I found it.
Define Subscribe() to subscribe to events coming from the Channel.
Developers must issue commands or write code.
1)
SupabaseClient.Auth.SetAuth(SupabaseClient.Auth.CurrentSession.AccessToken);

2)
channel.Push("access_token", payload: new Dictionary<string, string>
{
{ { "access_token", SupabaseClient.Auth.CurrentSession.AccessToken }
});

Actually, the contents of No. 2 are included in SetAuth.
The developer only needs to operate code 1, but I think it can be customized in some cases.
When I wrote it like this, I confirmed that the claims_role was changed from anon to authenticated within a term of 2 seconds.
It would be more convenient to automatically check SetAuth() when the Subscribe() method works, but I'm proud to find the answer anyway.
Thank you for making a good service!
I hope it helps someone.

I experienced the same. Calling SetAuth is not enough. Pushing the access token did the trick for me, even without the SetAuth call. It did not matter whether I had AutoConnectRealtime on or off.
Thanks for your digging @honeyhead !
I still think this is a bug or at least needs better documentation.

@honeyhead thanks so much for digging into this. I was trying to debug last night and was running to the same issue. I think the current client has fallen behind on the proper handshake from the current deployed supabase realtime version! So this is very helpful. The intended implementation would be that the developer shouldn't have to call SetAuth or push an access token, it should be handled automatically by the library.
I'll look into getting your suggestions pushed into the next version (with credit of course)!

v5.0.2 will automatically push the access token to any channel on subscription - the timing issue you mentioned should also be fixed. Your name is included in the changelog. Thanks for your help @honeyhead!

@acupofjose
Thanks! I checked that Claim changes to Authenticated without SetAuth() or channel.Push("access_token") after subscribe().
However, it seems inevitable that there will be a two-second term. If so, is there a way to check if Claim_Role is changed?

I suppose we could make it to where the async Subscribe() only returns after an acknowledgement from the server? Which should be after the claims role has been given. Thoughts?

@acupofjose
I think each event should be defined before the Subscribe() method.
However, in the process of initializing the supabase client, autorealtime may be given as true.
What is certain is that the client must be able to recognize the state of the claim_role being changed before or after Subscribe() so that developers can deal with issues that they cannot receive due to lack of permission.


fix: accumulate discriminator mapping keys

Status

READY
Description
Mappings which share the same schema are now correctly generated with all mapping keys
Fixes #1287

Looks like the formatter is failing?

seems like formatting in master wasn't up to date for some reason, it should be fixed now

Running build now.


get_site_root_paths is not respecting the current default site

and thus it results in the wrong site (base url) being used although we have a defined default page.
Its even getting more weird, because the last saved site will be used instead of the defined default.
The function should query all sites the following way:
Site.objects.select_related('root_page').order_by('-is_default_site', '-root_page__url_path')

Hi @kakulukia,
You seem to have jumped straight to the diagnosis without explaining what the problem is :-) Please can you give specific steps to reproduce the bug you're seeing?

Hehe .. we have multiple sites setup referring to the same root page because of staging and production systems having the "same" database. So when syncing the production database to the stage DB the system hast to respect the other default page. The problem is that its not respecting the current default page but instead takes the last saved one.

Its easily adjustable by just saving the right site last, but saving any other site after that will cause a lot of trouble. :)
BTW: we are using Page.url which is giving us the wrong url in this case

Thanks for the clarification!
Having two Site records pointing to the same page is not a well-defined setup, because it means there are two possible URLs for a given page. Giving priority to the default site would be a possible way of resolving this, but:

that's not what the 'default' flag was designed for (it only means "use this site to handle any requests to hostnames which aren't recognised")
it doesn't solve the general problem, because we can still have two non-default sites pointing to the same place.

Please see #1282 and #1283 for previous discussion about the "correct" behaviour for two sites with the same root page - it's still an open question.


No errors are shown when using undefined component

No warning is shown when I'm using undeclared component:
// ... is defined but never used
import {
    LoadingIndicator,
    // View
} from 'components';

...

render() {
    // no warnings..
    return (
        <View>
            <LoadingIndicator />
        </View>
    );
}

ESLint removed JSX support in no-undef since v0.18.0, you must now use the react/jsx-no-undef rule.

@yannickcr cool, that works!


Save session expiration time in credentials file

Similar to https://github.com/cevoaustralia/aws-google-auth, it would be nice if the expiration time of the session is saved in the credentials files to be used for automatic or manual inspection.

Fixed by #6


Describe end-to-end processes in Flux

It would be good to

[..] have clarification on end-to-end processes in
Flux, similar to how Envoy Proxy has an “life of an event” documentation:
https://www.envoyproxy.io/docs/envoy/v1.19.1/intro/life_of_a_request
From a security perspective such an overview would highly improve the understanding of
what trust boundaries Flux assumes and also describe the threat model of Flux. In such
end-to-end documentation, it would be of high value to make it clear how the individual
components relate to each other as well as describe where authentication and hardening
procedures are in place.

This seems like something I could take care of.


Set builddate in our .PKGINFO control data.

This sets the builddate on our APKs according to the SourceDateEpoch configured in the pipeline context.
Fixes: https://github.com/chainguard-dev/melange/issues/421

cc @deitch


Local CDK Lambda Layer asset path not respected

Description
When using CDK to create a layer, SAM local does not respect the aws:asset:path metadata.
Steps to reproduce
Generated template snippet:
      "Type": "AWS::Lambda::LayerVersion",
      "Properties": {
        "Content": {
          "S3Bucket": {
            "Ref": "LayerCodeS3Bucket01B5996C"
          },
          "S3Key": {
            "Fn::Join": [
              "",
              [
                {
                  "Fn::Select": [
                    0,
                    {
                      "Fn::Split": [
                        "||",
                        {
                          "Ref": "LayerCodeS3VersionKey1E2A5824"
                        }
                      ]
                    }
                  ]
                },
                {
                  "Fn::Select": [
                    1,
                    {
                      "Fn::Split": [
                        "||",
                        {
                          "Ref": "LayerCodeS3VersionKey1E2A5824"
                        }
                      ]
                    }
                  ]
                }
              ]
            ]
          }
        }
      },
      "Metadata": {
        "aws:cdk:path": "Stack/Layer/Resource",
        "aws:asset:path": "/tmp/aws-lambda-layer-kubectl/layer.zip",
        "aws:asset:property": "Code"
      }
    },

Modified lambda_downloader.py to add logging:
        if layer.is_defined_within_template:
            LOG.info("%s is a local Layer in the template", layer.name)
            LOG.info("HELLO %s", layer.codeuri)
            layer.codeuri = resolve_code_path(self.cwd, layer.codeuri)
            LOG.info("FUN %s", layer.codeuri)
            return layer

Observed result
Logs:
[sam] Layer129049A2 is a local Layer in the template
[sam] HELLO .
[sam] FUN /Users/..../dist/cdk
[sam] Image was not found.
[sam] Building image...

Expected result
HELLO should say /tmp/aws-lambda-layer-kubectl/layer.zip
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: OS X
sam --version: 0.21.0

Add --debug flag to command you are running

@abrooksv Looks like a CDK bug. A Layer does not have a property 'Code', it should be 'Content'. LayerVersion docs: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-layerversion.html

Overwrote the template:
      "Metadata": {
        "aws:cdk:path": "Stack/Layer/Resource",
        "aws:asset:path": "/tmp/aws-lambda-layer-kubectl/layer.zip",
        "aws:asset:property": "Content"
      }

Leads to:
HELLO /tmp/aws-lambda-layer-kubectl/layer.zip
Resolving code path. Cwd=/Users/.../dist/cdk, CodeUri=/tmp/aws-lambda-layer-kubectl/layer.zip
FUN /tmp/aws-lambda-layer-kubectl/layer.zip

CDK has the 'aws:asset:property` set incorrectly.
As far as the second issue (not allowing zip), we have an open issue for that here #947 but really the path should be pointed to the source (none-zipped asset).
Closing as an issue is open with CDK and we already have an tracking issue for the other issue brought up.


chore: bdd-spring-1585160719 to 0.0.1

chore: Promote bdd-spring-1585160719 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Add Windows support

Need to try building on Windows after running source_cleaner.sh and domain_patcher.sh on the source code.

Complete with 51.0.2704.106-2


Disable link animations

Hi,
Is it possible to disable link hover without modifying the /src & rebuilding?
Thanks!

Which link?
I assume you are talking about the link in post content and you want to remove the underscore.
Add the following code to your custom css
.post .post-content a:hover {
    border-bottom: none;
}

I mean that underline animation, when you hover the mouse over a link. There is that bottom border extending from center to the left and right.
Thanks I will try your solution!

Actually my goal is to make it a usual underline border on hover, just without that animation.

You can reset the transition-duration, see also https://developer.mozilla.org/en-US/docs/Web/CSS/transition-duration
.post .post-header .post-link:before {
  -webkit-transition-duration: initial;
  transition-duration: initial;
}

.header .site-navbar .menu .menu-item:before {
  -webkit-transition-duration: initial;
  transition-duration: initial;
}

Worked, thank you!


Update directory links

This PR updates directory link from external to internal. This will make sure the links still works for local repository.
I also update the BASE_URL for the github action.

@darwinz done removing the unused variable


concepts/services-networking/service: removed duplicate proxy term

I have removed the duplicate ProxyTerminatingEndpoints term found on the following page: https://kubernetes.io/docs/concepts/services-networking/service/#external-traffic-policy.
This fix addresses the issue found in the following Open Issue: https://github.com/kubernetes/website/issues/29827.

Thanks.
/lgtm
/approve


Codespaces limit is outdated

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
https://docs.github.com/codespaces/developing-in-codespaces/creating-a-codespace
What part(s) of the article would you like to see updated?
The "About codespace creation" section says that maximum number of codespaces that can be created is 10.
But I was able to create more than 10, so maybe this thing is outdated.
Additional information
NA

@thispsj Thanks so much for opening an issue! I'll triage this for the team to take a look :eyes:


⚠ Turnos 24x7 has degraded performance

In ccb1f87, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 982 ms

Resolved: Turnos 24x7 performance has improved in 4f269b1.


Integrate 2023 LBNL data

Success criteria: LBNL 2023 data in dev, create a brief narrative about what has changed.

[x] Find previous notebook for updating LBNL.

[x] Do columns differ?
[x] queue id and region still unique id?

[x] Compare raw LBNL 23 to GS. we should expect all LBNL 23 projects to be in raw GS (except for old MISO and non active ERCOT)
[x] Run the raw data through existing code
[x] Add raw data to dgm-archive instead of git lfs
[x] Use LBNL CAISO information. How many projects are in LBNL 23 that aren't in GS?
[x] Align ISONE and ISO-NE
[x] Compare LBNL 2022 to LBNL 2023 + Latest gridstatus.
[x] Compare raw LBNL with data mart table for 2022 and 2023. Are the pct changes between raw and data marts reasonable? Save the raw and data mart tables for 2022 and 2023.
[ ] Why are so many NYISO projects being removed?
[ ] Why are aggregate capacity numbers for each region loweriso_projects_long_format then in the LBNL study?

NYISO issue
The raw GS data contains 147 GW of active NYISO projects but iso_projects_long_format only has 66 GW. This is because 47% of NYISO active capacity is in offshore wind and transmission projects (45 and 8 respectively). The transmission projects are intentionally being dropped. The offshore wind projects are dropped and replaced by the manually currated offshore wind data. The offshore wind data has not been updated since April 2023. Which Asana sheet does this CSV come from? Is it still being updated?


This documentation contains a confusing typo

Updated to improve readability

@RyanDrx : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Thanks for the clarifications, @RyanDrx. I guess some of the technical language slipped past the devs. This will be published in the next few days.


How should I set the color of played progress?

I need to set a special color to the played progress but I can't  find the option.
Actual behaviour
Here is my code of porgress:

                                <input data-plyr="seek" type="range" min="0" max="100" step="0.01" value="0" id="plyr-seek-{id}">
                                <progress class="plyr__progress--played" max="100" value="0" role="presentation"></progress>
                                <progress class="plyr__progress--buffer" max="100" value="0">0%</progress>

I found that there is not a progress bar for played in default controls options, not like the old version. So is there any chance I can set the color of the played  progress within the plyr of newest version

Using the SASS variable as per the previous version:
https://github.com/sampotts/plyr/blob/master/src/sass/settings/colors.scss#L5


The environment variable is not imported when the JSON file is upload.

Description:
The environment variable is not imported when the JSON file is uploaded.
TM Version  - 3.2.3, TM-CLI Version- 1.4.4
Priority: High
Expected behavior:
The environment variable should be successfully imported when uploading the JSON file.

Issue Fixed
Tested Version: Version 3.2.5
https://jam.dev/c/7bb12e79-689e-439e-ab11-aa2baf7d1b40


[New Package]: Mozilla Firefox

List the PackageIdentifier of the package to be added to automation
Mozilla.Firefox
Details about the application
This request is for regular Mozilla Firefox. There's already a discussion for Mozilla Firefox Nightly, although it's not in the latest version right now:
https://github.com/vedantmgoyal2009/vedantmgoyal2009/issues/188

https://github.com/microsoft/winget-pkgs/pull/82512

I already have it in the works, but will commit the changes after all installers are added to WinGet (a PR is already open for it).

https://github.com/microsoft/winget-pkgs/pull/60151

There's a PR in progress for beta, which cannot pass due to a bug in pipelines, which is to be fixed by MS team. After the bug-fix, PRs will be opened for adding all installers to every Firefox pacakge. BTW, I'm merging this request with Nightly.

https://github.com/vedantmgoyal2009/vedantmgoyal2009/issues/188


[Crash] Multi Mod Conflict?

I'll be totally honest I'm not sure what's going on here but I'm getting a crash with a multitude of mods when enabled in tandem. Removing any one of the following mods seems to fix the crash (some are dependencies however). I'm not really sure where to submit this but Forgero was the first conflict I identified so it's your lucky day I guess.
Minecraft 1.19.2
Quilt 18.2
BCLib 2.1.6
Bedrockify 1.5
Bedspreads 6.0.0
Better Nether 7.1.3
Better Taskbar 1.5.1
Bookshelf 16.2.17
Botania 437
CorgiLib <IP_ADDRESS>
EMI 0.6.5
Enchant the Rainbow 1.1.0
Enchanting Infuser 4.2.2
Enhanced Celestials <IP_ADDRESS>
Entity Texture Features 4.3.1
Forge Config API Port 4.2.11
Forgero for Fabric 0.10.6
Patchouli 77
Puzzles Lib 4.3.12
Quilted Fabric API 4.0.0 beta 28 & QSL 3.0.0 beta 29
Starlight 1.1.1
Trinkets 3.4.2
Log: https://pastebin.com/h3cnFPDj
Prism Launcher Instance: https://drive.google.com/file/d/1COmoenH9bkAUhim_UHm5wCupqM-hT1NP/view?usp=share_link

So it seems like mixins are failing for Botania  in a worldrenerer class. I do believe I also have a mixin there's so It's possible our mods are conflicting on the mixin injection. I think this can be fixed easily, but I'll have to have another look at it when I have some more time.
Also, not familiar with how quilt handles things, but hopefully that's not the source of the error. Thanks for the report!

ahhh yw, okay, I'll post on the botania issue tracker too

Alright, I had another look at this issue, and I can't really find any connection to Forgero.
The crash comes from this mixin:
org.spongepowered.asm.mixin.injection.throwables.InvalidInjectionException: Variable modifier target for net/minecraft/class_761::makeMoonBigger was removed by another injector [INJECT Applicator Phase -> #botania:botania_xplat.mixins.json:client.LevelRendererMixin from mod botania -> Apply Injections ->  -> Inject -> #botania:botania_xplat.mixins.json:client.LevelRendererMixin from mod botania->@ModifyVariable::makeMoonBigger(Lnet/minecraft/class_1159;)Lnet/minecraft/class_1159;]

So the issues likely stems from mods that inject something into the same function as botania does to make the moon bigger. I tried recreating your setup of mods, but I could not recreate the crash so I can't really do any more from my end.

It appears Enhanced Celestials <IP_ADDRESS> causes this issue as I got the same Crash error when attempting to load these mods together

I'll check that version to see if I can reproduce the issue!


Webpack error: export was not foud

While compiling for production webpack is throwing an error related to the import of vue2-leaflet
245:23-34 "export 'default' (imported as 'Vue2Leaflet') was not found in 'vue2-leaflet'

 warning  in ./assets_src/js/marketplaceDirectory.js

247:29-40 "export 'default' (imported as 'Vue2Leaflet') was not found in 'vue2-leaflet'

 warning  in ./assets_src/js/marketplaceDirectory.js

249:26-37 "export 'default' (imported as 'Vue2Leaflet') was not found in 'vue2-leaflet'

 warning  in ./assets_src/js/marketplaceDirectory.js

251:25-36 "export 'default' (imported as 'Vue2Leaflet') was not found in 'vue2-leaflet'

caused by
import Vue2Leaflet from 'vue2-leaflet';

Vue.component('v-map', Vue2Leaflet.Map);
Vue.component('v-tilelayer', Vue2Leaflet.TileLayer);
Vue.component('v-marker', Vue2Leaflet.Marker);
Vue.component('v-popup', Vue2Leaflet.Popup);

Did anyone else get this error?

Forget it, it works using a single import for each component.
    import Map from 'vue2-leaflet/src/components/Map';
    import TileLayer from 'vue2-leaflet/src/components/TileLayer';
    import Marker from 'vue2-leaflet/src/components/Marker';
    import Popup from 'vue2-leaflet/src/components/Popup';


seas:connectedTo -> Property chain axiom commented

Dear  @maximelefrancois86,
the seas system ontology is a valuable asset when describing technical systems. Looking into it again I came across the commented property chain axiom for seas:conectedTo in v1.1. and v1.0.
 # seas:connectedTo owl:propertyChainAxiom ( seas:connectedThrough seas:connectsSystem ) .

Why has this been commented? It would be helpful to have the same chain rule applied as for
seas:connectedThrough owl:propertyChainAxiom ( seas:connectsAt seas:connectsSystemThrough ) .
seas:connectsSystem owl:propertyChainAxiom ( seas:connectsSystemAt seas:connectionPointOf ) .

best
Georg

Hello Georg,
This axiom would make the ontology violate the OWL 2 DL profile

OK thanks @maximelefrancois86 for this clarification. I was wondering if it makes sense to omit the commented line in the file? or add to the commented line a comment ^^
# Logically applies but resulting ontology is outside of OWL 2 DL
# seas:connectedTo owl:propertyChainAxiom ( seas:connectedThrough seas:connectsSystem ) .


API: Move Mainfest to root

Update the add_datqa function
Relates to #80 #84

Add a new function in dataset class:  def add_data(self, source_path, data_type, subject, sample, sds_parent_dir=None, copy=True, overwrite=False, sample_metadata={}, subject_metadata={}):

use dataset.add_data( data_type='primary') to call add_primary_data
use  dataset.add_data( data_type='derivative') to call add_derivative_data

Modified derivative_folder variable in add_primary_data and add_derivative_data:

use  derivative_folder_list to instead.
before derivative_folder = '/tem/data/primary/subject/sample'
now derivative_folder_list = ['tem/data', 'primary', 'subject', 'sample' ]
the benefits for this is we can easily to combine different paths in other functions, in order to we can save manifest.xlsx in root folder.

all test on dataset version 2.0.0

Resolved by #89


Inline zarr.Array in da.from_zarr

Inlining means that we take a graph like
{
  "zarr-arrray": (zarr.Array, ...),
  "arr-0-0": (getitem, "zarr-array", 0, 0),
  "arr-0-1": (getitem, "zarr-array", 0, 0),
}

and replace it with something like
zarr_array = zarr.Array(...)
{
  "arr-0-0": (getitem, zarr_array, 0, 0),
  "arr-0-1": (getitem, zarr_array, 0, 1),
}

In this case, we want to inline the zarr.Array to ensure that we get good task fusion
later on. For reasons currently beyond me, the presence of a root node confuses our
fusion algorithms. Compare the two
# setup
import dask.array as da
original = "original.zarr"

a = da.random.random(size=(4, 4000, 4000), chunks=(1, 4000, 4000))
a.to_zarr(original, overwrite=True)
b = da.from_zarr(original)
c = b.rechunk({1: 2000, 2: 2000})
with dask.config.set(**{"optimization.fuse.ave-width": 4}):
    display(c.visualize(optimize_graph=True))

On master

On this PR

Failure to fuse tends to cause excess communication and memory issues on the distributed scheduler. Compare the to task graph panes in these reports.
c.to_zarr() on master:
https://gistcdn.rawgit.org/TomAugspurger/1ebffe84db72e630229efb3a4b9cd7d2/19f63c5b972332708ffec041927f0cd764a302a6/master.html
c.to_zarr() on this PR:
https://gistcdn.rawgit.org/TomAugspurger/1fae16981f23a8ff892f120da7dbdba4/c1101da0e41759d2f81642369ed48e1c910147f7/pr.html
xref https://github.com/dask/dask/issues/5105. There's one more issue with to_zarr that I want to track down. I'm seeing what looks like unnecessary data moving around, but need to verify.
cc @alimanfoo, @rabernat, and @rsignell-usgs since this has affected you.
Setting the fuse_ave_width to be greater or equal the rechunk factor (4 in this case) is quite important right now. I'm hopeful we'll be able to do this automatically in the future.

This functionality has flopped back and forth a couple of times.  I don't recall the PR/issue where it was brought up before, but I think the problem was that people were using data stores that were not as trivial to move around as Zarr files.  Perhaps git blame can point to the previous conversation.

Thanks for investigating this Tom. It seems like you are on the right track to solving some long-standing performance issues related to large distributed reads.

Sorry, I think the issue I'm thinking of focused on from_array.
In general though, I think that this will break if someone points a Dask
array to a Zarr array that uses in-memory storage.
Maybe there was also an issue with storing or creating the object many
times?  Perhaps authentication as taking a while or the serialized form of
the task was large enough that people wanted to store it once rather than n
times?
In general treating the Zarr array separately seems somewhat cleaner.  It
would be good to see if there is a deeper issue that can be solved here.
Especially if this is funded work, it would be good to frontload cost
rather than have this change pop up later and have to be handled by
volunteers.
On Wed, May 13, 2020 at 7:58 AM Tom Augspurger<EMAIL_ADDRESS>wrote:

I didn't see much in the git blame. In the original implementation
https://github.com/dask/dask/pull/3561/files @jakirkham
https://github.com/jakirkham didn't inline the Array. Perhaps it was
HDF?
I'll this try out with some zarr datasets in cloud storage later today.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/6203#issuecomment-628047689, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACKZTG6IBP3LW65AYJ7MR3RRKYQJANCNFSM4M7ZHVOA
.

Hmmm unfortunately this doesn't look like an unambiguous win. I whether or not it's good to inline the getitem depends on the downstream operations.
I'm going to explore why the presence of the root node causes issues with fusion in the first place.

I didn't see much in the git blame. In the original implementation https://github.com/dask/dask/pull/3561/files @jakirkham didn't inline the Array. Perhaps it was HDF?

My guess is I just did whatever was easiest to get us started :)

@TomAugspurger, I'm confused by what you say you want to do and what you did in this PR.  To me, it looks like the Zarr array wasn't inlined.  The getters were inlined.  Please let me know if I misunderstood something.
Also, I wouldn't expect fuse to do anything for this example.  fuse works on reductions or linear chains.  The given example is the opposite of that.
Looking at the example from https://github.com/dask/dask/issues/5105#issuecomment-625282112
import dask
import dask.array as da
a = da.random.random((4, 4000, 4000), chunks=(1, 4000, 4000))
da.to_zarr(a, "/tmp/data/original.zarr", overwrite=True)
data = da.from_zarr("/tmp/data/original.zarr")
desired = data.rechunk({0: 4, 1: 2000, 2: 2000})
data2 = data.rechunk({1: 2000, 2: 2000})
data2.to_zarr("/tmp/data/temp.zarr", overwrite=True)
data3 = da.from_zarr("/tmp/data/temp.zarr")
data4 = data3.rechunk({0: 4})
with dask.config.set(**{"optimization.fuse.ave-width": 4}):
    display(data4.visualize(optimize_graph=True))

Master gives

This PR gives (due to inlining, not fuse, so {"optimization.fuse.ave-width": 1} also gives this graph)

For this example, fuse refuses to fuse on master because hold_keys specifically tells fuse not to fuse the getitems.  We can trick hold_keys with a hack:
diff --git a/dask/array/core.py b/dask/array/core.py
index 4d38c63c..c4dda7b7 100644
--- a/dask/array/core.py
+++ b/dask/array/core.py
@@ -2861,7 +2861,11 @@ def from_zarr(
     chunks = chunks if chunks is not None else z.chunks
     if name is None:
         name = "from-zarr-" + tokenize(z, component, storage_options, chunks, **kwargs)
-    return from_array(z, chunks, name=name)
+    arr = from_array(z, chunks, name=name)
+    alias = 'alias-' + name
+    arr.dask.layers[name][alias] = arr.dask.layers[name][name]
+    arr.dask.layers[name][name] = alias
+    return arr

 def to_zarr(

which gives

As an observation, inlining is much more overzealous than fusing and more difficult to control.  Is there a getter that isn't held by hold_keys (so is always fuse-able) but also doesn't get inlined?

Is there a getter that isn't held by hold_keys (so is always fuse-able) but also doesn't get inlined?

Answering my own question... here's a less hacky way (learned from this PR) to avoid both hold_keys and inline_functions for Zarr getters.  This may have more reasonable and controllable behavior:
diff --git a/dask/array/core.py b/dask/array/core.py
index 4d38c63c..02b91d6a 100644
--- a/dask/array/core.py
+++ b/dask/array/core.py
@@ -135,6 +135,10 @@ def getter_inline(a, b, asarray=True, lock=None):
     return getter(a, b, asarray=asarray, lock=lock)

+def getter_nohold(a, b, asarray=True, lock=None):
+    return getter(a, b, asarray=asarray, lock=lock)
+
+
 from .optimization import optimize, fuse_slice

@@ -2861,7 +2865,7 @@ def from_zarr(
     chunks = chunks if chunks is not None else z.chunks
     if name is None:
         name = "from-zarr-" + tokenize(z, component, storage_options, chunks, **kwargs)
-    return from_array(z, chunks, name=name)
+    return from_array(z, chunks, name=name, getitem=getter_nohold)

 def to_zarr(

FWIW, I may be abandoning this PR. It's not obviously better on all workloads.

To me, it looks like the Zarr array wasn't inlined. The getters were inlined.

Yes, I think that you're correct.

Which PR is that?

Must be  https://github.com/dask/dask/pull/6222, and I'm curious how you expect that to help here.
What workloads perform poorly with this PR?

Thanks Erik!
Would also be curious to know. Though guessing these are some of the issues.

FWIW, I may be abandoning this PR. It's not obviously better on all workloads.

I'm a bit curious about this point, @TomAugspurger. Is there a workload where this would behave worse? If so, by how much?

Sorry for the delay in responding here. https://github.com/dask/dask/pull/6773 is making this a user-facing keyword. da.from_zarr(..., inline_array=True). Taking my original example:
import dask
import dask.array as da
original = "original.zarr"

a = da.random.random(size=(4, 4000, 4000), chunks=(1, 4000, 4000))
a.to_zarr(original, overwrite=True)
b = da.from_zarr(original)
c = b.rechunk({1: 2000, 2: 2000})

with dask.config.set(**{"optimization.fuse.ave-width": 4}):
    c.visualize(optimize_graph=True, color="order", cmap="autumn", filename="bad")

order doesn't do well there. We finish the first chain, and then jump between the remaining three thanks to just using the tie-breaker of the key name to decide which to work on next. We can hopefully fix that in order, but for now, with the array inlined, we finish off each chain before moving onto the next:
b = da.from_zarr(original, inline_array=True)
c = b.rechunk({1: 2000, 2: 2000})

with dask.config.set(**{"optimization.fuse.ave-width": 4}):
    c.visualize(optimize_graph=True, color="order", cmap="autumn", filename="good")

Should from_zarr provide sane defaults based on the source of the Zarr
array?  Do we know what those sane defaults would be?
On Wed, Oct 28, 2020 at 8:55 AM Tom Augspurger<EMAIL_ADDRESS>wrote:

Closed #6203 https://github.com/dask/dask/pull/6203.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/6203#event-3932051597, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACKZTCWCPLPJJ5OIXQ4VS3SNA5IFANCNFSM4M7ZHVOA
.

I might not be following the question. Could you please provide an example? Think I'm just missing context 😅

I'm asking what the default value for inlining should be for the from_zarr
function.  Should from_zarr inline or not inline?  Are there things about a
zarr array that could help us better choose a default value?
For example, if the zarr array holds data in memory, then it should
probably not inline.  If it points to data on-disk then maybe it's ok?  (Or
maybe not, and we should consider caching like Xarray does)
On Wed, Oct 28, 2020 at 2:52 PM jakirkham<EMAIL_ADDRESS>wrote:

I might not be following the question. Could you please provide an
example? Think I'm just missing context 😅
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/6203#issuecomment-718230014, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACKZTCBJPXG75YQC42QKSTSNCHB7ANCNFSM4M7ZHVOA
.


There is an identification issue in publish microservice function.

When the function is writing source code to algoritima, then there is an issue in that source code.

Indentation issue in publish micro service is solved and the changes has been pushed to PyPi.


Autoencoder shape

What's the reasoning behind having the first layer smaller than the middle layers, unlike what the picture shows? Is it to reduce the number of parameters and overfitting, or simply the best configuration from the experiments?

Hoping @okuchaiev and @borisgin can weigh in as well...
My understanding is that while numerous architectures were explored (different activation types, adding hidden layers, trying different numbers of nodes per layer, different dropout rates, different learning rates, dense re-feeding off and on, etc.), this confirmation had the best out-of-sample performance.
As to why this configuration performed the best in empirical experiments, I hypothesize that having a wide bottleneck layer helps the neural network learn a large number of "features" from the previous layer. Additionally, having a high dropout rate forces the model to learn robust features; e.g. with only 20% of the neurons active (80% dropout rate), the model must be extra careful when learning which features are most useful for the task at hand.
Thus, this configuration likely had the best out-of-sample performance because the wide bottleneck layer with the high dropout rate allowed the model to learn a large number of very robust features.

Yes, I think @paulhendricks  is right - wide middle layer with large dropout allows it to learn robust representations.
Regarding first layers (e.g. first encoder layer) and last layer (e.g. last decoder layer) - those are actually huge in terms of weights because number data (x) is high dimensional. This, if x is around 17,000 and first layer has only 128 activations, then it means that there are 17,000x128 weights in the first layer.


Fix yanked 2.5.1

Why 2.5.1 was yanked on crates.io? How can it be fixed?

Oh, I though you've seen my comment. https://github.com/slog-rs/slog/pull/215/files#diff-b4aea3e418ccdb71239b96952d9cddb6L2186 broke backward compatibility. Previously any prefix of a logging level would get parsed correctly. It was reported on gitter channel.

Hmm, I think I can restore this feature

The issue is fixed. Could you release 2.5.2 to make these changes public?

Done. Thanks!


Impossible to add (multiple) GPG key(s)

It's currently impossible in the user form to add one (or many) GPG keys.

A little bit more complicated than we thought. Client-side solution needs to be found.

@abompard @relrod are there any examples of what exactly a GPG key ID will look like for this field?
is it a multi-line thing? or a simple one-line ID?

@sfinn85 complicated in what way?
@ryanlerch Each is a simple string maxlength="16", but there can be multiple of them, so we just need a button to add extra fields for more keys.

Sprint 3 planning update: Keep it simple - ( S )


add total suffix in counter metrics

Signed-off-by: yeya24<EMAIL_ADDRESS>I am wondering if this pr will cause compatibility problem. But I think we should follow the metrics naming . All counter metrics need add suffix total.

This change is

@krishnadurai  Please take a look. :smile:

Coverage remained the same at 76.744% when pulling cce01e7faefcb51305e10da43e7f37f2c668d854 on yeya24:rename-metrics into 7ccfeafcd3305b7aaf63f7567f5877244f09b704 on kubeflow:master.

Do you mean, compatibility problems with existing deployments?

@johnugeorge  Exactly. But add this suffix in important and we can know what the metric type is from its name.

I think so. The PR LGTM.
/lgtm
Thanks for your contribution! :tada: :+1:

/ok-to-test

Forget to update the docs I will update it tomorrow.

Thanks @yeya24 Can you add the docs in the same PR

@johnugeorge Done. PTAL

@krishnadurai I' m not sure if this note is enough. PTAL. Thanks!

This note is fine.
/lgtm

/approve


Fix #6

Codecov Report

Merging #7 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master     #7   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           8      8
  Lines         154    181   +27
  Branches       32     38    +6
=====================================
+ Hits          154    181   +27

Impacted Files
Coverage Δ

packages/vue-teible/src/DataTablePagination.vue
100% <100%> (ø)
:arrow_up:

packages/vue-teible/src/helpers.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 74bd585...74ce95a. Read the comment docs.


Page won't display after oAuth callback...

Trying to go through the quickstart guide to get the Google Analytics API working via PHP (https://developers.google.com/analytics/devguides/reporting/core/v3/quickstart/web-php). While the page loads and jumps over to Google for authorization, the callback page won't load when it comes back from Google: oauth2callback.php?code=4/xvilrDCNpY6MpFd6cqe2OoWNINiodiZCC0iyX8ZtNtM#
I just get the following from Chrome and no PHP errors or anything:

No data received
ERR_EMPTY_RESPONSE

I don't think it's library's bug


Feature request: Inline replies

When trying to write text below the quoted text, I hit RET on the last line of the quoted text, and I'm only given the option to extend the quote, not type my own response (the left-margin line follows me wherever I go). This is crippling.

Hi @artagnon — unfortunately this is a known issue. N1 does not yet support inline responses (where you mix your response into quoted text.) I've added this to our open source backlog–feel free to track it here or upvote if you have a Trello account: https://trello.com/c/TvjRaV7W/49-inline-replies-mix-reply-into-quoted-text

Not having an functioning inline response option is a show stopper for me when it comes be using N1 for work. It needs to be put very high on the feature request list!

this was fixed with https://github.com/nylas/N1/commit/1de3a4707739adfbcac2df04de74b92f82a4a135 and will be available in the next release!

this was fixed with 1de3a47 and will be available in the next release!

That's fantastic — thank you!

Fix is not working for me.
Commit 1de3a47 says "You can now break up blockquotes (as in quoted text areas) by pressing
'delete' at the start of a line. This allows you to reply inline."
However, it's not working for me in version 0.4.40 on Ubuntu.

I'm having the same problem as @jackdesert. I cannot reply inline in Nylas N1 on Ubuntu. This is a pretty nasty bug for me because I need to reply inline to many of the email threads I have.


Mediator still has IOHK reference

Please remove this reference
https://github.com/search?q=repo%3Ainput-output-hk%2Fatala-prism-mediator IOHK&type=code

The only reference we need to rename for now is the html title <title>IOHK Mediator</title>.
The emails will be done later

New link
https://github.com/search?q=repo%3Ahyperledger/identus-mediator IOHK&type=code

Done


WFLY-11239 Rewriting SAR subsytem to don't use deprecated MSC ServiceBuilder methods

https://issues.jboss.org/browse/WFLY-11239

@ropalka This needs squashing - you have more commits than changed files :)

Squashed @kabir - for those who will review this commit here's the unsquashed version of this commit:
https://github.com/ropalka/wildfly/tree/WFLY-11239-for-review