Quick link open.

Please select if your request is either something new or an enhancement

[ ] Enhancement of an existing Feature.
[x] Request of a new feature.

Please select the area your request applies to. (Multiple selections are Possible. You can leave blank if you're not sure.)

[ ] Workspace - VSCode workspace, vaults, Intellisense/autocomplete, Dendron settings
[ ] Lookup - Dendron's Lookup Command
[ ] Views - Dendron Preview, Tree View, Side Panels in the UI
[ ] Schema - Dendron Schemas
[ ] Pod - Data import from / export to Dendron
[ ] Publish - External Site Publish
[x] Markdown - Markdown features, such as syntax support and features

Is your feature request related to a problem? Please describe
New shortcut to quick open the first link in the note without having to reach for the mouse.
Describe the solution you'd like
A new command along the lines of Dendron: Quick open web link Which will pick the first web link of the note and open it.
Describe alternatives you've considered
Reaching for the mouse to open the first link.
Additional context
This is very useful for link proxy notes. Notes with just a single link that give a name that is easy to remember and query instead of some weird URL. However, without this shortcut the Flow is broken and you have to leave the keyboard entirely to reach for the mouse to click on the link.
Implementation
Can be a simple regex that is activated when the shortcut is used and looks for something along the lines of http.* and opens that link in the default browser (same as clicking on the link within the markdown).

Was thinking about this as well. We want to enable this using note traits since we are using proxy notes more internally as well.
Hoping to have this implemented soon-ish (want to say next week but I learned from  @aleksey-rowan that I need to stop setting dates in gh issues 😅 )

As a workaround, you can use Ctrl + F12 to open an external link under the cursor without reaching for the mouse. That doesn't help with opening the first link in the note, but if you are using Vim jumping to places is rather easy.

The [goto](https://wiki.dendron.so/notes/vikda112zw2sss8lbit4qey/) command will now open the uri of the proxy note when executed over its link, but you need to have a link to the proxy note in some other note--you can't open the uri of the proxy note by simply locating it with the lookup-like command.


Autocompletion

Some autocompletion would be very nice for the text editors (e.g., to type "PREFIX foaf:" and be suggested the IRI, or to type "ex:Alice foaf:k" and be suggested foaf:knows).

A starting point would be https://github.com/TriplyDB/Yasgui
See also https://www.emse.fr/~zimmermann/Teaching/SemWeb/turtle-ed.html


There are option for focus point when scan ?

I want to focus one point on camera when I tap on it ?

I'll need to study it when I'm free, which will probably take a few days


REX-Ray 0.4.x Volume Creation incompatible with VirtualBox 5.1

I'm using REX-Ray 0.3.3 version with Virtualbox (for local dev). After updating to Virtualbox 5.1, only first rexray VM "owns" all Virtualbox created volumes.
When trying to mount volume from second machine, I've got the following error:
docker: Error response from daemon: VolumeDriver.Mount: {"Error":"no device name returned"} VolumeDriver.Mount: {"Error":"no device name returned"}

In Virtualbox "Virtual Media Manager" UI, it's possible to see that volume indeed attached to the first machine.
Currently the only way to work is to rollback to Virtualbox 5.0.24.

confirmed on v0.3.3. 0.4.3 has more servere issues where a new volume cannot even be created. steps to recreate:
0.3.3

create 2 machines using virtualbox & install REX-Ray v0.3.3

docker-machine create -d virtualbox rrtest01
docker-machine create -d virtualbox rrtest02
install REX-Ray

on machine 1 create a new volume

docker volume create -d rexray --name mytest01 --opt=size=5

on machine 2, try to run a container with the volume and return the error

docker run -ti --volume-driver=rexray -v mytest01:/mytest01 busybox
returned error is docker: Error response from daemon: VolumeDriver.Mount: {"Error":"no device name returned"}.

0.4.3

upgrade both machines to 0.4.3 and update config files if necessary
restart RR
rexray volume create --volumename=mytest --size=5 will return the following error:

ERRO[0001] error creating new volume                     host=unix:///home/docker/.libstorage/var/run/libstorage/687358817.sock inner=failed to create medium instanceID=virtualbox=af734df9-2904-4cf7-bdbe-9e3c4e760213 provider=virtualbox route=volumeCreate server=split-master-ch service=virtualbox size=5 task=0 time=1468859145593 tls=false txCR=1468859145 txID=5f08fa4d-3284-4ee0-7367-865f05190758 volumeName=mytest
ERRO[0001] error creating new volume                     host=unix:///home/docker/.libstorage/var/run/libstorage/687358817.sock inner=failed to create medium provider=virtualbox route=volumeCreate server=split-master-ch size=5 time=1468859145594 tls=false txCR=1468859145 txID=5f08fa4d-3284-4ee0-7367-865f05190758 volumeName=mytest
FATA[0001] error creating new volume

@akutz @alexei-led I just ran a test again because I realized that I jumped the gun and my 0.4.3 configuration file was incorrect.
I tested with version 0.4.3 on VirtualBox 5.1 and I have no issues with pre-emption or using a disk created on one host and using it on another

@alexei-led have you tested with 0.4.3 yet? No issues have been indentified. We are probably not going to go back and fix older versions of code with 0.3.3. Please close if this works


Implement Collection.Compact for PostgreSQL

What should be done?
PostgreSQL's backend Collection.Compact method should call VACUUM ANALYZE on the given collection/table.
When Full is true, VACUUM FULL ANALYZE should be run instead.
If the database or collection does not exist, ErrorCodeDatabaseDoesNotExist or ErrorCodeCollectionDoesNotExist should be returned (that's already checked by the contact and backend test).
Where?

https://github.com/FerretDB/FerretDB/blob/main/internal/backends/postgresql/collection.go
https://github.com/FerretDB/FerretDB/blob/main/internal/backends/collection_test.go

Definition of Done

PostgreSQL backend updated;
Skip removed from the test.

Hi @AlekSi i would like to work on that. Please assign me for that

@alhridoy Done! We just merged #3485 that added stubs for this method

@AlekSi sounds good.

@alhridoy any news? Anything we could help you with?

@alhridoy any news? Anything we could help you with?

Hi @AlekSi, I have created a PR for issue #3484
Could you please review the PR and provide any feedback? I'm ready to make any necessary changes. Thank you!


Distinct list values for  CoverageSqliteData.arcs() and CoverageSqliteData.lines()

Now the return values properly reflect the data from the JSON data implementation.
(It is also a huge performance improvement!)

Thanks!


Lock Required Tables in 2.6 migrations

Just wondering, how would existing users who applied the 2.6.0 migration get this update? Or is this purely for net new users?


Benchmark ... "compression" aspect unclear.

Describe the bug (feature?)
When running a benchmark (weed 1.15) the storage result is clearly "compressed" data. But the documentation claims:
[root@korora-01 v_1_15]# ./weed help benchmark
Usage: weed benchmark -server=localhost:9333 -c=10 -n=100000

  benchmark on an empty SeaweedFS file system.

  Two tests during benchmark:
  1) write lots of small files to the system
  2) read the files out

  The file content is mostly zero, but no compression is done.
..

So either

the code is not creating "uncompressible" data
the help text is ambiguous about the meaning of "no compression is done".

System Setup
I start a master:
./weed master -volumeSizeLimitMB=10 -defaultReplication=000 -ip <IP_ADDRESS> -mdir /var/zones/seaweed-m01/data

a volume
./weed volume -mserver=<IP_ADDRESS>:9333 -max=150 -port=8080 -dataCenter=dc1 -rack=rackA -ip <IP_ADDRESS> -dir /var/zones/seaweed-v01/data

and then run the benchmark
./weed benchmark -master="<IP_ADDRESS>:9333" -c=16 -size=1024000 -n=10000

Which runs as expected.
Expected behavior
If I want to benchmark the system read/write IO performance I want to have "no compression is done" feature .. otherwise I would be testing the CPUs compression/decompression speed.
With the above settings I would expect ... 10000 * 1 MB = 10 GB of data in the storage.
But I see:
[root@korora-01 data]# ls -la
total 10720
drwxr-xr-x. 2 root root    4096 Dec 27 17:24 .
drwxr-xr-x. 3 root root      18 Dec 21 01:03 ..
-rw-r--r--. 1 root root 1545048 Dec 27 17:26 benchmark_1.dat
-rw-r--r--. 1 root root   22912 Dec 27 17:26 benchmark_1.idx
-rw-r--r--. 1 root root 1521288 Dec 27 17:26 benchmark_2.dat
-rw-r--r--. 1 root root   22560 Dec 27 17:26 benchmark_2.idx
-rw-r--r--. 1 root root 1493424 Dec 27 17:26 benchmark_3.dat
-rw-r--r--. 1 root root   22144 Dec 27 17:26 benchmark_3.idx
-rw-r--r--. 1 root root 1600376 Dec 27 17:26 benchmark_4.dat
-rw-r--r--. 1 root root   23728 Dec 27 17:26 benchmark_4.idx
-rw-r--r--. 1 root root 1559296 Dec 27 17:26 benchmark_5.dat
-rw-r--r--. 1 root root   23120 Dec 27 17:26 benchmark_5.idx
-rw-r--r--. 1 root root 1503064 Dec 27 17:26 benchmark_6.dat
-rw-r--r--. 1 root root   22288 Dec 27 17:26 benchmark_6.idx
-rw-r--r--. 1 root root 1567888 Dec 27 17:26 benchmark_7.dat
-rw-r--r--. 1 root root   23248 Dec 27 17:26 benchmark_7.idx
[root@korora-01 data]# du -sk .
10720	.

only about 10 MB of stored data.
Since all 10000 files have been written (according to the ID log file) the files are compressed .. which can be confirmed by checking the data:
[root@korora-01 v_1_15]# ./weed export -limit 10 -dir /var/zones/seaweed-v01/data -volumeId 1 -collection benchmark
key	name	size	gzip	mime	modified	ttl	deleted
1,0f2f6dae19	.	1031	true		2018-12-27T17:24:56		false
1,19196d8240	.	1031	true		2018-12-27T17:24:56		false
1,2a375165dd	.	1031	true		2018-12-27T17:24:56		false
1,23806f4708	.	1031	true		2018-12-27T17:24:56		false
1,2d37824cd4	.	1031	true		2018-12-27T17:24:56		false
1,386297bd30	.	1029	true		2018-12-27T17:24:56		false
..

Additional context
Perhaps it makes sense to allow both options for the benchmark ... compressed and uncompressed storage ... in order to allow testing both behaviors. But if CPU compression should be the test target then the test data needs to represent the "final" test data way better than "all zero" or "all random".
So in the current form it seems to me more plausible to have "no compression is done" apply to the volume servers storage behavior.
I would assume that this should be achievable if the data uploaded was marked as "gzip" content.

This is due to some smarter gzip detection. https://github.com/chrislusf/seaweedfs/commit/c043fd17cb8f9753e284c41a627a269ad951b37d
Need a way to optionally skip this.

So then maybe a "quick hack" could be to use some info that allows the benchmakr data to "fail early" in the gzippable test.
E.g. a MIME type ala
image/benchmark.fakeRaw

to meet the test
"if strings.HasPrefix(mtype, "image/")
..

Actually found a bug in the previous diff, which think files without file name extensions are gzippable. Fixed it now.

Correct! I did not see that one.
But in that case the benchmark would still run down to the test:
isMostlyText := util.IsText(data)

.. which might get smarter in future releases and at some point change the behavior again.
Maybe a modification of the benchmark line:
fp := &operation.FilePart{Reader: &FakeReader{id: uint64(id), size: fileSize}, FileSize: fileSize}

with either a well defined FileName or MimeType might be useful as well.
Faking a FileName might be an interesting addition since it seems to me like this would enable benchmarking the "filer" subsystem too.
As I have never written a line of Go I an not sure what the proper creating of a "FileName" from the id: uint64(id) would look like.
Faking the MimeType: "image/benchmark.noCompression"  seem like the easier code part ... and it would serve the purpose of explicitly documenting the "idea" behind the data set.
But maybe both could useful modifications?

fixed accordingly: https://github.com/chrislusf/seaweedfs/commit/2a19511ba848913402b77c2f120ef583b7530a79
btw: golang is easy to code.


Follow up on the TODOs in PR #375

resolve the TODOs added by #375.
refactor agent package as suggested by @alexmwu.

resolved by #419


django.core.exceptions.ImproperlyConfigured: 'chatterbot.ext.django_chatterbot' must supply a name attribute.

After I integrate with Django, I got this exception when I execute "python manager.py runserver":

Traceback (most recent call last):
File "C:\Users\mamajingjie\AppData\Local\Programs\Python\Python36\lib\site-packages\django\apps\config.py", line 149, in create
app_name = cls.name
AttributeError: type object 'AppConfig' has no attribute 'name'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 21, in
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "C:\Users\mamajingjie\AppData\Local\Programs\Python\Python36\lib\site-packages\django\core\management_init_.py", line 381, in execute_from_command_line
utility.execute()
File "C:\Users\mamajingjie\AppData\Local\Programs\Python\Python36\lib\site-packages\django\core\management_init_.py", line 357, in execute
django.setup()
File "C:\Users\mamajingjie\AppData\Local\Programs\Python\Python36\lib\site-packages\django_init_.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "C:\Users\mamajingjie\AppData\Local\Programs\Python\Python36\lib\site-packages\django\apps\registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "C:\Users\mamajingjie\AppData\Local\Programs\Python\Python36\lib\site-packages\django\apps\config.py", line 152, in create
"'%s' must supply a name attribute." % entry)
django.core.exceptions.ImproperlyConfigured: 'chatterbot.ext.django_chatterbot' must supply a name attribute.

You confused class name with it's attribute
in your settings.py then INSTALLED_APPS
change AppConfig to ApiConfig

You confused class name with it's attribute
in your settings.py then INSTALLED_APPS
change AppConfig to ApiConfig

I don't know if answering this after one year is against the rules of github, but you are an anonimous hero. Thanks.


Hbase 24987 Rest Client throws AbstractMethodError about javax.ws.rs.core.UriBuilder.uri

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 24s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
shelldocs
0m  0s
Shelldocs was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 24s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  8s
Maven dependency ordering for patch

-0 :warning:
shellcheck
0m  1s
The patch generated 2 new + 34 unchanged - 0 fixed = 36 total (was 34)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  1s
The patch has no ill-formed XML file.

+1 :green_heart:
hadoopcheck
13m 24s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

_ Other Tests _

+1 :green_heart:
asflicense
0m 27s
The patch does not generate ASF License warnings.

22m 21s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2362

Optional Tests
dupname asflicense shellcheck shelldocs hadoopcheck xml

uname
Linux c749f5c10e55 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / d48c732851

shellcheck
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-general-check/output/diff-patch-shellcheck.txt

Max. process+thread count
63 (vs. ulimit of 12500)

modules
C: . hbase-rest U: .

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 14s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m  0s
the patch passed

+1 :green_heart:
compile
2m 51s
the patch passed

+1 :green_heart:
javac
2m 51s
the patch passed

+1 :green_heart:
shadedjars
6m 32s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 19s
hbase-rest in the patch failed.

-0 :warning:
javadoc
0m 15s
root in the patch failed.

_ Other Tests _

-1 :x:
unit
145m  5s
root in the patch failed.

176m 30s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2362

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 37a17f1e5f6f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / d48c732851

Default Java
2020-01-14

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rest.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt

unit
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/testReport/

Max. process+thread count
4070 (vs. ulimit of 12500)

modules
C: hbase-rest . U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 36s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 24s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 37s
master passed

+1 :green_heart:
compile
2m 24s
master passed

+1 :green_heart:
shadedjars
6m 30s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
2m 22s
master passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 27s
the patch passed

+1 :green_heart:
compile
2m 22s
the patch passed

+1 :green_heart:
javac
2m 22s
the patch passed

+1 :green_heart:
shadedjars
6m 39s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
2m 21s
the patch passed

_ Other Tests _

-1 :x:
unit
324m 42s
root in the patch failed.

359m 21s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2362

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux fe5f2b288d3d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / d48c732851

Default Java
1.8.0_232

unit
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/testReport/

Max. process+thread count
6329 (vs. ulimit of 12500)

modules
C: hbase-rest . U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2362/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


Base64.btoa TypeError: Illegal invocation

This is exactly the same issue as #6 with atob. Will make a PR to fix this.

Oh, wow, I created this pull request 4 years ago. Time flies!


Creating a specimen builder for a SByte* type

Hi,
I'm in a situation where I am using a thirdparty library (Fluent validation fyi). Having removed the ThrowingRecursionBehavior and added the OmitOnRecursionBehavior, I'm told that AutoFixture carn't create a value for SByte*. What is the best way of creating a specemin builder for this situation, or, is there another way to handle the situation, given it's fairly deep in the object graph?
Any help appreciated.
Kind regards
Sean.

How about Fixture.Register<T> or Fixture.Customize<T>?

Closing due to inactivity. If you need further help, then write here. We can always re-open the issue.


Navigating away from .ipynb file and lost all changes :(

Azure Data Studio Version:
1.4.4 insider

Steps to Reproduce:

Open 1.ipynb file
Make some edits
Click another 2.ipynb file to view
1.ipynb file no long in tabs above
Click 1.ipynb file
all changes made to file in step 1 gone

Wait it gets better. I closed it and opened it several times. Then gave up and started over, making modifications similar to that in step #1. Then, try to save the file (doesn't work). Then, navigate to a .sql file. Return to 1.ipynb and all the old edits are there!

@amthomas46 I'm seeing an issue where when tabbing we lose the scroll position. Is this what you mean? I'm struggling to reproduce where on editing you lose visibility on the file, since as soon as you edit this should mark file as dirty and not close it.
Note there is a related issue from what you mentioned:

Ctrl+S isn't saving right now. @rajmusuku  is working on that and it's tracked in another bug. Workaround is to use the "Save" button in the notebook

No, this was something else, it was super weird. Let me see if I can repro it

OK please let me know, looking forward to trying to solve as many of these as possible (you should see a bunch of your bugs getting closed)

I'm going to close, please reopen once we have a repro


VHDL lexer improvements

(Original issue 1177 created by tspiteri on 2015-11-30T23:34:31.502069+00:00)
I am attaching a patch with some VHDL fixes and improvements:

Fix an issue where two strings on one line, together with anything in between, are marked as one single string.
In use library.package.all, highlight all as a keyword.
Highlight std.* and ieee.* as namespaces.
Highlight the words std, ieee and work as namespaces.
Highlight the labels before processes or for loops as class names.
Highlight signed and unsigned as type names.
Highlight rem as a keyword.
Fix an issue where sli is highlighted as a keyword instead of sll.

(Original comment by birkenfeld on 2016-02-02T11:43:16.658043+00:00)
VHDL updates (closes #1177)
→ <>

(Original comment by birkenfeld on 2016-02-02T11:43:20.515831+00:00)
Thanks for the patch!


編譯問題

sudo apt install cmake libproj-dev libpcap-dev libboost-all-dev libyaml-cpp-dev libjpeg-dev libgdal-dev libpq-dev libvtk6-dev libvtk6-qt-dev libpcl-dev
error:
E: Unable to locate package libvtk6-qt-dev
E: Unable to locate package libpcl-dev

What's the OS version?
We tested it on Ubuntu 16.04.

ubuntu 14.04，使用ubuntu 18.04可以了。


Overcome machine limitation issue for VS Code tunneling

If you tunnel from many gitpod workspaces, eventually you'll run into this error message:

You've exceeded the 10 machine limit for the port fowarding service. Please remove other machines before trying to add this machine.

You can unregister a remote using the VS Code UI (https://www.github.com/microsoft/vscode-remote-release/issues/7683 and https://www.github.com/microsoft/vscode-remote-release/issues/8544#issuecomment-1570741832):


Add dynamic port

Fix static port issue and select box issue.

Refer to this link for build results (access rights to CI server needed):
http://guh.guru:8080/job/guh-webinterface-pull-request/60/


Cannot Connect to Shark Vaccum

Describe Your Problem:

I'm trying to connect to Shark Clean vacuum using this home bridge plugin but does not work me. I'm using the email and password method.
Logs:
Show the Homebridge logs here, remove any sensitive information.
[11/8/2024, 10:47:40 AM] [SharkIQ] Loaded homebridge-sharkiq v1.3.2 child bridge successfully
[11/8/2024, 10:47:40 AM] Loaded 0 cached accessories from cachedAccessories.0E9BE50235A5.
[11/8/2024, 10:47:40 AM] Homebridge v1.8.5 (HAP v0.12.3) (SharkIQ) is running on port 55108.
[11/8/2024, 10:47:40 AM]

NOTICE TO USERS AND PLUGIN DEVELOPERS
> Homebridge 2.0 is on the way and brings some breaking changes to existing plugins.
> Please visit the following link to learn more about the changes and how to prepare:
> https://github.com/homebridge/homebridge/wiki/Updating-To-Homebridge-v2.0

[11/8/2024, 10:47:40 AM] [SharkIQ] None of the DSNs provided matched the vacuum(s) on your account.

Plugin Config:
Show your Homebridge config.json here, remove any sensitive information.
{
    "name": "SharkIQ",
    "email"<EMAIL_ADDRESS>    "password": "*******",
    "vacuums": [
        "AC000W030800785"
    ],
    "europe": false,
    "invertDockedStatus": false,
    "dockedUpdateInterval": 30000,
    "_bridge": {
        "username": "0E:9B:E5:02:35:A5",
        "port": 55108
    },
    "platform": "SharkIQ"
}

Screenshots:

Environment:

Plugin Version: v1.3.2
Homebridge Version:  1.8.5
Node.js Version:  v20.9.0
NPM Version:  10.1.0
Operating System:  macOS

@abraich77 If there are no login errors, double check the vacuum DSN your inputted


TW is not defined

I have a Next.js app setup with TS, Tailwind and Styled-Components. Storybook and the application work fine in displaying my components however my tests are failing:
my component:
export const HamburgerIconWrapper = tw.div flex flex-1 justify-start sm:hidden
my test:
test("renders HamburgerIconWrapper snapshot", async () => {
    let renderedComp
    await act(async () => {
        renderedComp = render(<HamburgerIconWrapper />)
    })
    expect(renderedComp.asFragment()).toMatchSnapshot()
})

Result:

Same scenario


Fix AfterWatermark Early and Late javadoc

The docs were reversed - the late trigger is only considered after the
watermark has passed the end of the window, and the early trigger only
before the watermark has passed the end of the window.

R: @bjchambers


Confusion regarding db read performance after the switch fo fly_postgres_elixir.

Hello,
I have a test application that is essentially based on Hello-Phoenix (https://github.com/fly-apps/hello-phoenix ).
I initially had this test application including postgres db deployed exclusively in the fra region. Using the classic Repo and running a simple read request (Repo.all(User)), the response time was several hours at an average of 30ms. After I switched the test application to use fly_postgres_elixir and deployed the application including the postgres volumes to the fra and lax regions, the application's read performance deteriorated from 30ms to an average of 300ms. Unfortunately, I cannot explain this performance degradation.
App-Status
vscode ➜ /workspace (main ✗) $ fly status
App
  Name     = billowing-star-1538
  Owner    = personal
  Version  = 13
  Status   = running
  Hostname = billowing-star-1538.fly.dev

Deployment Status
  ID          = 2ab52089-3db7-6cc9-5ef8-77c266ffa7fa
  Version     = v13
  Status      = successful
  Description = Deployment completed successfully
  Instances   = 2 desired, 2 placed, 2 healthy, 0 unhealthy

Instances
ID       PROCESS VERSION REGION DESIRED STATUS  HEALTH CHECKS      RESTARTS CREATED
54bf2e53 app     13      fra    run     running 1 total, 1 passing 0        2h26m ago
1b8ff1ed app     13      lax    run     running 1 total, 1 passing 0        2h27m ago

Database Status
vscode ➜ /workspace (main ✗) $ fly status -a hello-phoenix-database
App
Name     = hello-phoenix-database
Owner    = personal
Version  = 3
Status   = running
Hostname = hello-phoenix-database.fly.dev
Instances
ID       PROCESS VERSION REGION DESIRED STATUS            HEALTH CHECKS      RESTARTS CREATED
3e6d8469 app     3       lax    run     running (replica) 3 total, 3 passing 0        3h13m ago
ebaa9d9a app     3       fra    run     running (replica) 3 total, 3 passing 0        2021-11-01T18:17:40Z
ae18bd04 app     3       fra    run     running (leader)  3 total, 3 passing 0        2021-11-01T18:17:30Z
log Region fra
2021-11-03T11:47:40.774 app[54bf2e53] fra [info] 11:47:40.774 request_id=FrQG0VRHT3tGIS4AAJkB [info] GET /
2021-11-03T11:47:40.775 app[54bf2e53] fra [info] 11:47:40.775 request_id=FrQG0VRHT3tGIS4AAJkB [info] Sent 200 in 808µs
2021-11-03T11:47:49.533 app[54bf2e53] fra [info] 11:47:49.532 request_id=FrQG015Tpj4V_H8AAJkR [info] GET /users
2021-11-03T11:47:49.838 app[54bf2e53] fra [info] 11:47:49.838 request_id=FrQG015Tpj4V_H8AAJkR [info] Sent 200 in 305ms
2021-11-03T11:47:49.953 app[54bf2e53] fra [info] 11:47:49.953 request_id=FrQG03dit6HuymYAAJkh [info] GET /users
2021-11-03T11:47:50.258 app[54bf2e53] fra [info] 11:47:50.258 request_id=FrQG03dit6HuymYAAJkh [info] Sent 200 in 304ms
2021-11-03T11:47:52.420 app[54bf2e53] fra [info] 11:47:52.419 request_id=FrQG1ApnEQCQShcAAJkx [info] GET /users
2021-11-03T11:47:52.725 app[54bf2e53] fra [info] 11:47:52.725 request_id=FrQG1ApnEQCQShcAAJkx [info] Sent 200 in 305ms
2021-11-03T11:47:54.173 app[54bf2e53] fra [info] 11:47:54.173 request_id=FrQG1HLsVgC4PokAAJlB [info] GET /
2021-11-03T11:47:54.174 app[54bf2e53] fra [info] 11:47:54.174 request_id=FrQG1HLsVgC4PokAAJlB [info] Sent 200 in 871µs
Region lax
2021-11-03T11:44:17.573 app[1b8ff1ed] lax [info] 11:44:17.571 request_id=FrQGogRxGvi3RxQAAFhB [info] GET /
2021-11-03T11:44:17.574 app[1b8ff1ed] lax [info] 11:44:17.572 request_id=FrQGogRxGvi3RxQAAFhB [info] Sent 200 in 1ms
2021-11-03T11:44:17.838 app[1b8ff1ed] lax [info] 11:44:17.836 request_id=FrQGohQ_qQBdPLgAAFhR [info] GET /users
2021-11-03T11:44:18.009 app[1b8ff1ed] lax [info] 11:44:18.006 request_id=FrQGohQ_qQBdPLgAAFhR [info] Sent 200 in 170ms

Hi @jeissing! Thanks for trying out the library and giving feedback!
What is your fly regions list? And can you show the status of your database app? fly status -a your-db-app

Also, since this is a very recently opened issue, I assume you are using version 0.1.9. If not, you should update to that first.

Perhaps also interesting is that the instance in fra uses the following connection strings:
Primary Fly.Postgres.primary_db_url()
postgres://billowing_star_1538_kzor184w07xemvj5:@fra.hello-phoenix-database.internal:5432/billowing_star_1538?sslmode=disable
Replica: Fly.Postgres.replica_db_url()
postgres://billowing_star_1538_kzor184w07xemvj5:@fra.hello-phoenix-database.internal:5433/billowing_star_1538?sslmode=disable

Ok. Thanks for the region listing. While this isn't your specific issue, you will hit it eventually. You want to disable backup regions that aren't an intended region.
See this issue for details on how to address it. This is the case for multi-region Elixir applications that use postgres replicas (and fly_postgres).
Are you physically closer to fra or lax? I noticed that your primary is in fra.
For the DB connections, that's expected for fra.
You can get an IEx console into the lax instance like this:
> fly ssh console --select

Then use the keyboard to select the instance in lax.
I'd be interested to know what you get for the:
> Fly.is_primary?
false

> Fly.Postgres.database_url
"postgres://app_95gd38z3loxnwq2v:password@lax.app-db.internal:5433/app?sslmode=disable"

The important part here is does it have @lax.? It should be connecting to the local replica database.

I am here in Germany and therefore several thousand miles closer to Frankfurt (fra).
 Assumed DEV or TEST environment
The expected lax prefix is not present.

That's what I was wondering about. When it's not specified, DNS will just pick one. So you potentially end up connecting to a DB that is far away from you. Then you get the latency you're seeing.
I'm guessing the dockerfile doesn't set the MIX_ENV value the library is expecting. That also needs to be documented!
https://github.com/superfly/fly_postgres_elixir/blob/main/lib/fly_postgres.ex#L54
So it's falling through to use the DATABASE_URL which doesn't specify a region.
The challenge is dealing with local dev/test environments and production ones.
Just a heads up, I'm currently updating the hello_elixir repo and the Fly Elixir guide for Phoenix 1.6 and other changes. The hello_elixir repo has already been updated. You can use it's Dockerfile as an example.

I used the Dockerfile from hello_elixir repository in my project and did a deployment. Without success. In another step, I added MIX_ENV=prod again in the Dockerfile in front of the last line. This time with success. Why the redundant declaration of the environment variable MIX_ENV=prod leads to success is not clear to me.
.....
RUN set -eux;
ln -nfs $(basename *)/bin/$(basename *) /app/entry
ENV MIX_ENV=prod
CMD /app/entry start

@jeissing thank you for your multiple experiments to find the fix. Based on your feedback I updated the Dockerfile again. There was a problem with the 2nd setting of the MIX_ENV. There are two Dockefile stages and fly_postgres really needs the 2nd stage one.
I'm looking into better ways to solve the need so this issue can be avoided.
Hope that helps!

Yes, now I see it. It is a multistage Dockerfile. The MIX_ENV setting is needed not only at compile time but also at runtime for Fly_postgres_elixir.
Thanks for the quick and professional support. And by the way. I love the massively reduced latency.

@jeissing I released v0.1.10. This includes a CHANGELOG and I updated the README.
This removes the need for the 2nd MIX_ENV setting in the Dockerfile. So you'd still have the first stage ENV MIX_ENV="prod" but it doesn't need it for the 2nd release section.
This is a breaking change, in that you need to add a new config for it work properly in production.
In either config/prod.exs or config/runtime.exs you set the following:
config :fly_postgres, rewrite_db_url: true

This works for the production to connect to the primary or replica DB appropriately. In dev and test it leave the DB config alone. Give it a try! Hope this cleans up some of that for you!

I have migrated to v.0.1.10. It works as expected. Thank you very much.


Not working in Ubuntu

I am running the command npm i from root user in an ubuntu server and getting following error:

Have you noticed

Error: EACCES: permission denied...

Looks like the OS user doesn't have enough privileges to create a directory.
All packages except the optional ed25519 were installed. This package dependency is not critical and can be safely skipped. The app is ready for the launch.

@orbitlens Yes I did notice that. However I doubt because I am running the commands using root user.


[sombrero] Simplify classes used in benchmarks

Address https://github.com/ukri-excalibur/excalibur-tests/issues/237#issue-2010042384.  @mirenradia Can you please check this solves the issues you reported in the first post?

This resolves all of the issues I mentioned in #237 except for the num_tasks problem and I can now run the ITT-sn benchmark (after manually setting num_tasks appropriately). However, I do get the following warning:
WARNING: skipping test 'SombreroBenchmarkScaling': test has one or more undefined parameters

I can reproduce that, as far as I understand it comes from the fact case_filter.generate(scaling_config) in https://github.com/ukri-excalibur/excalibur-tests/blob/40b4f4a874010ebfd822a46d08fcf1975020496d/benchmarks/apps/sombrero/sombrero.py#L103 is an empty list [], and so the parameter params is undefined.  I'm not entirely sure how that benchmark is supposed to work, it was contributed by @mmesiti, maybe he can shed some light on this?
In any case I'm going ahead and merge this PR, as it addresses the previously reported issues.


Update dns.md

Replaced old promo with new implementation services promo
Description of the change:
Reason for the change:
Link to original source:

Closes #

Hello @negarzipan,Thanks again for the PR! We appreciate your contribution and look forward to                                              continued collaboration. Thanks!                                             Team SendGrid DX


Updated margins on logo

Logo's that aren't the size of the sidebar are left-aligned.
Changed logo-margin variable to be 10px and set the left and right margins to auto to properly center it.
Originally found on the Shins fork

Hi @uncle-scooter, really sorry for the slow reply on this! Definitely like the idea of centering narrow logos, but do we have a good reason to add margin to the top and changing the semantics of $logo-margin? Glad to hear arguments, but my initial reaction is to avoid breaking people's custom variable setups, and just set margin-left: auto and margin-right: auto on .logo without the other margin changes? Thoughts?

Closing since it's been a little while, but if somebody wants to resubmit, would be happy to take another look!


Add decoder initialization error to the server's error cache

Add decoder initialization error to server's error cache so that it can show up in the server's debug API.
A follow-up can be to surface this error in the UI itself.

Codecov Report

Merging #10773 (c3017ff) into master (a9c97e3) will decrease coverage by 54.95%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             master   #10773       +/-   ##
=============================================
- Coverage     68.60%   13.66%   -54.95%
+ Complexity     6457      439     -6018
=============================================
  Files          2141     2103       -38
  Lines        115137   113532     -1605
  Branches      17347    17255       -92
=============================================
- Hits          78993    15515    -63478
- Misses        30561    96746    +66185
+ Partials       5583     1271     -4312

Flag
Coverage Δ

integration1
?

unittests1
?

unittests2
13.66% <0.00%> (-0.07%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...manager/realtime/LLRealtimeSegmentDataManager.java
0.00% <0.00%> (-69.25%)
:arrow_down:

... and 1679 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@npawar / @Jackie-Jiang / @mayankshriv : review pls!


CheriBSD renamed VMEM SW perm bit

CheriBSD 00d71bd4d11af448871d196f987c2ded474f3039 changes
"CHERI_PERM_CHERIABI_VMMAP" to be spelled "CHERI_PERM_SW_VMEM" and deprecated
the old form.  Follow along.

The change looks trivial but I wonder if we could ease the transition a bit with something like:
#ifndef CHERI_PERM_SW_VMEM
#define CHERI_PERM_SW_VMEM CHERI_PERM_CHERIABI_VMMAP
#endif

Which would allow you to build snmalloc against an old or new kernel.

I suppose if it is deprecated not removed that is probably not necessary.

Actually, it would be useful to have the define for building snmalloc against an old FreeBSD that doesn't define CHERI_PERM_SW_VMEM.


Training and Dataset preparation

Hey,
I'm currently trying to train also a net with the LSP Dataset. Is it maybe possible to share your code how you prepared the dataset and trained it? In this repository I just found the model definition but not how to train.
Would love to hear back!

@dluvizon would love to hear from you.

Hi @SaifAlDilaimi ,
I did not provide the training code here because this project is still under development (in a private repo), but it will be available soon.
Despite that, on the paper you can find all the details for training.
The procedure for LSP and MPII are very close, basically all you need to do it to crop the images using ground truth bounding boxes an do some data augmentation on it (see section 4.1 from the paper).
All the best,

hey @dluvizon ,
thank you for the answer!
I hope I'm not rude if I ask some specific things:

The MPII Dataset has 16 Joints but the LSP Dataset has 14 Joints. In your paper you wrote that you have trained the LSP starting with the weights of MPII but how is that possible? Would this work without any issues?

The MPII Dataset has a bounding box annotated but as far as I know the LSP don't. How did you cropped the images on that dataset?

lastly, when augmenting the data, how did you recalculated the joints when rotating the image?

Currently I'm working on my bsc thesis and I would love to cite your work if its ok..

Hi, my answers:

For training on LSP, you can use its 14 joints with two more "invalid" joints, on which you always set zero loss error.
In LSP, images are already cropped, so just use it as it is (at most, crop it squared).
This is a affine transformation: https://en.wikipedia.org/wiki/File:2D_affine_transformation_matrix.svg
Just do the same for images and 2D points.

Of course it is OK citing.
BTW, in this paper I extended the work, if you are interested: https://arxiv.org/abs/1802.09232

Hey @dluvizon ,
Sorry for my last post.. I had a wrong implementation. Still, I got a new error which I can't fix. I have prepared the MPII Dataset as you said. I have copied your Network into a new file called DeepPoseNetwork and used it as follows:
def train(x_train, y_train, x_val, y_val, x_test, y_test, model_type=PreTrainedModel.CUSTOM):
    model = build_model()

    print("Shape of x_train: ", x_train.shape, ", of type: ", x_train.dtype)
    print("Shape of y_train: ", y_train[0].shape, ", of type: ", y_train.dtype)
    print("Shape of x_val: ", x_val.shape, ", of type: ", x_val.dtype)
    print("Shape of y_val: ", y_val.shape, ", of type: ", y_val.dtype)
    print("Shape of x_test: ", x_test.shape, ", of type: ", x_test.dtype)
    print("Shape of y_test: ", y_test.shape, ", of type: ", y_test.dtype)

    # checkpoint
    filepath="weights_"+model_type.name+".best.hdf5"
    reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.4, patience=5, min_lr=0.001)
    checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')
    tensorboard = TensorBoard(log_dir="logs/{}".format(time()), histogram_freq=0, write_graph=True, write_images=False)
    csv_logger = CSVLogger('log_'+model_type.name+'.csv', append=True, separator=';')

    callbacks_list = [reduce_lr, checkpoint, tensorboard, csv_logger]

    model.fit(x_train, y_train,
            batch_size=PARAMS.ML_BATCH_SIZE,
            epochs=PARAMS.ML_EPOCHS,
            verbose=1,
            validation_data=(x_val, y_val), callbacks=callbacks_list)
    score = model.evaluate(x_test, y_test, verbose=0)
    print('Test loss:', score[0])
    print('Test accuracy:', score[1])

def build_model():
    if K.image_data_format() == 'channels_first':
        input_shape = (3, PARAMS.ML_INPUT_IMAGE_HEIGHT, PARAMS.ML_INPUT_IMAGE_WIDTH)
    else:
        input_shape = (PARAMS.ML_INPUT_IMAGE_WIDTH, PARAMS.ML_INPUT_IMAGE_HEIGHT, 3)

    m_input = Input(shape=input_shape)
    m_outputs = []

    num_heatmaps = (PARAMS.ML_LABEL_SIZE + 1) * PARAMS.ML_LABEL_CLASSES

    x = DeepPoseNetwork.stem(m_input)

    num_rows, num_cols, num_filters = K.int_shape(x)[1:]

    # Build the soft-argmax models (no parameters) for specialized and
    # contextual maps.
    sams_input_shape = (num_rows, num_cols, PARAMS.ML_LABEL_CLASSES)
    sam_s_model = DeepPoseNetwork.build_softargmax_2d(sams_input_shape, name='sSAM')
    jprob_s_model = DeepPoseNetwork.build_joints_probability(sams_input_shape, name='sjProb')

    # Build the aggregation model (no parameters)
    if PARAMS.ML_LABEL_SIZE > 0:
        samc_input_shape = (num_rows, num_cols, num_heatmaps - PARAMS.ML_LABEL_CLASSES)
        sam_c_model = DeepPoseNetwork.build_softargmax_2d(samc_input_shape, name='cSAM')
        jprob_c_model = DeepPoseNetwork.build_joints_probability(samc_input_shape, name='cjProb')
        agg_model = DeepPoseNetwork.build_context_aggregation(PARAMS.ML_LABEL_CLASSES, PARAMS.ML_LABEL_SIZE, PARAMS.ML_DEEPPOSE_ALPHA, name='Agg')

    for bidx in range(PARAMS.ML_DEEPPOSE_STAGES):
        block_shape = K.int_shape(x)[1:]
        x = DeepPoseNetwork.build_reception_block(x, name='rBlock%d' % (bidx + 1), ksize=PARAMS.ML_DEEPPOSE_KERNEL_SIZE)

        ident_map = x
        x = DeepPoseNetwork.build_sconv_block(x, name='SepConv%d' % (bidx + 1), ksize=PARAMS.ML_DEEPPOSE_KERNEL_SIZE)
        h = DeepPoseNetwork.build_regmap_block(x, num_heatmaps, name='RegMap%d' % (bidx + 1))

        if PARAMS.ML_LABEL_SIZE > 0:
            pose, visible, hm = DeepPoseNetwork.pose_regression_context(h, PARAMS.ML_LABEL_CLASSES, sam_s_model, sam_c_model, jprob_c_model, agg_model, jprob_s_model)
        else:
            pose, visible, hm = DeepPoseNetwork.pose_regression(h, sam_s_model, jprob_s_model)

        m_outputs.append(pose)
        m_outputs.append(visible)
        if PARAMS.ML_DEEPPOSE_EXPORT_HEATMAPS:
            m_outputs.append(hm)

        if bidx < PARAMS.ML_DEEPPOSE_STAGES - 1:
            h = DeepPoseNetwork.build_fremap_block(h, block_shape[-1], name='fReMap%d' % (bidx + 1))
            x = add([ident_map, x, h])

    model = Model(inputs=m_input, outputs=m_outputs)

    rmsprop = RMSprop(lr=PARAMS.ML_LEARN_RATE)

    model.compile(rmsprop, loss='binary_crossentropy', metrics=['accuracy'])

    model.summary()

    return model

def main():
    types = [PreTrainedModel.DEEPPOSE]

    (x_train, y_train), (x_val, y_val), (x_test, y_test) = Dataset(dstype=DatasetType.MPII).prepare_sets()

    print('Starting training...')
    train(x_train, y_train, x_val, y_val, x_test, y_val)

Calling this class generates the following numpy arrays with shape:
Shape of x_train:  (15935, 256, 256, 3) , of type:  int8
Shape of y_train:  (16, 2) , of type:  int32
Shape of x_val:  (1992, 256, 256, 3) , of type:  int8
Shape of y_val:  (1992, 16, 2) , of type:  int32
Shape of x_test:  (1992, 256, 256, 3) , of type:  int8
Shape of y_test:  (1992, 16, 2) , of type:  int32

After Keras displaying the networks summary it throws following error:
ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 16 array(s), but instead got the following list of 1 arrays: [array([[[219, 147],
        [157, 131],
        [103, 156],
        ...,
        [ 59, 134],
        [106, 125],
        [145, 102]],

       [[228, 123],
        [187, 123],
        [146, 120],
    ...

The summary of the Keras Model prints this:
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            (None, 256, 256, 3)  0
__________________________________________________________________________________________________
Stem (Model)                    (None, 32, 32,<PHONE_NUMBER>     input_1[0][0]
__________________________________________________________________________________________________
rBlock1 (Model)                 (None, 32, 32,<PHONE_NUMBER>     Stem[1][0]
__________________________________________________________________________________________________
SepConv1 (Model)                (None, 32, 32, 576)  347904      rBlock1[1][0]
__________________________________________________________________________________________________
RegMap1 (Model)                 (None, 32, 32, 48)   27648       SepConv1[1][0]
__________________________________________________________________________________________________
fReMap1 (Model)                 (None, 32, 32, 576)  29376       RegMap1[1][0]
__________________________________________________________________________________________________
add_12 (Add)                    (None, 32, 32, 576)  0           rBlock1[1][0]
                                                                 SepConv1[1][0]
                                                                 fReMap1[1][0]
__________________________________________________________________________________________________
rBlock2 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_12[0][0]
__________________________________________________________________________________________________
SepConv2 (Model)                (None, 32, 32, 576)  347904      rBlock2[1][0]
__________________________________________________________________________________________________
RegMap2 (Model)                 (None, 32, 32, 48)   27648       SepConv2[1][0]
__________________________________________________________________________________________________
fReMap2 (Model)                 (None, 32, 32, 576)  29376       RegMap2[1][0]
__________________________________________________________________________________________________
add_22 (Add)                    (None, 32, 32, 576)  0           rBlock2[1][0]
                                                                 SepConv2[1][0]
                                                                 fReMap2[1][0]
__________________________________________________________________________________________________
rBlock3 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_22[0][0]
__________________________________________________________________________________________________
SepConv3 (Model)                (None, 32, 32, 576)  347904      rBlock3[1][0]
__________________________________________________________________________________________________
RegMap3 (Model)                 (None, 32, 32, 48)   27648       SepConv3[1][0]
__________________________________________________________________________________________________
fReMap3 (Model)                 (None, 32, 32, 576)  29376       RegMap3[1][0]
__________________________________________________________________________________________________
add_32 (Add)                    (None, 32, 32, 576)  0           rBlock3[1][0]
                                                                 SepConv3[1][0]
                                                                 fReMap3[1][0]
__________________________________________________________________________________________________
rBlock4 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_32[0][0]
__________________________________________________________________________________________________
SepConv4 (Model)                (None, 32, 32, 576)  347904      rBlock4[1][0]
__________________________________________________________________________________________________
RegMap4 (Model)                 (None, 32, 32, 48)   27648       SepConv4[1][0]
__________________________________________________________________________________________________
fReMap4 (Model)                 (None, 32, 32, 576)  29376       RegMap4[1][0]
__________________________________________________________________________________________________
add_42 (Add)                    (None, 32, 32, 576)  0           rBlock4[1][0]
                                                                 SepConv4[1][0]
                                                                 fReMap4[1][0]
__________________________________________________________________________________________________
rBlock5 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_42[0][0]
__________________________________________________________________________________________________
SepConv5 (Model)                (None, 32, 32, 576)  347904      rBlock5[1][0]
__________________________________________________________________________________________________
RegMap5 (Model)                 (None, 32, 32, 48)   27648       SepConv5[1][0]
__________________________________________________________________________________________________
fReMap5 (Model)                 (None, 32, 32, 576)  29376       RegMap5[1][0]
__________________________________________________________________________________________________
add_52 (Add)                    (None, 32, 32, 576)  0           rBlock5[1][0]
                                                                 SepConv5[1][0]
                                                                 fReMap5[1][0]
__________________________________________________________________________________________________
rBlock6 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_52[0][0]
__________________________________________________________________________________________________
SepConv6 (Model)                (None, 32, 32, 576)  347904      rBlock6[1][0]
__________________________________________________________________________________________________
RegMap6 (Model)                 (None, 32, 32, 48)   27648       SepConv6[1][0]
__________________________________________________________________________________________________
fReMap6 (Model)                 (None, 32, 32, 576)  29376       RegMap6[1][0]
__________________________________________________________________________________________________
add_62 (Add)                    (None, 32, 32, 576)  0           rBlock6[1][0]
                                                                 SepConv6[1][0]
                                                                 fReMap6[1][0]
__________________________________________________________________________________________________
rBlock7 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_62[0][0]
__________________________________________________________________________________________________
SepConv7 (Model)                (None, 32, 32, 576)  347904      rBlock7[1][0]
__________________________________________________________________________________________________
RegMap7 (Model)                 (None, 32, 32, 48)   27648       SepConv7[1][0]
__________________________________________________________________________________________________
fReMap7 (Model)                 (None, 32, 32, 576)  29376       RegMap7[1][0]
__________________________________________________________________________________________________
add_72 (Add)                    (None, 32, 32, 576)  0           rBlock7[1][0]
                                                                 SepConv7[1][0]
                                                                 fReMap7[1][0]
__________________________________________________________________________________________________
rBlock8 (Model)                 (None, 32, 32,<PHONE_NUMBER>     add_72[0][0]
__________________________________________________________________________________________________
SepConv8 (Model)                (None, 32, 32, 576)  347904      rBlock8[1][0]
__________________________________________________________________________________________________
RegMap8 (Model)                 (None, 32, 32, 48)   27648       SepConv8[1][0]
__________________________________________________________________________________________________
lambda_27 (Lambda)              (None, 32, 32, 16)   0           RegMap1[1][0]
__________________________________________________________________________________________________
lambda_28 (Lambda)              (None, 32, 32, 32)   0           RegMap1[1][0]
__________________________________________________________________________________________________
lambda_29 (Lambda)              (None, 32, 32, 16)   0           RegMap2[1][0]
__________________________________________________________________________________________________
lambda_30 (Lambda)              (None, 32, 32, 32)   0           RegMap2[1][0]
__________________________________________________________________________________________________
lambda_31 (Lambda)              (None, 32, 32, 16)   0           RegMap3[1][0]
__________________________________________________________________________________________________
lambda_32 (Lambda)              (None, 32, 32, 32)   0           RegMap3[1][0]
__________________________________________________________________________________________________
lambda_33 (Lambda)              (None, 32, 32, 16)   0           RegMap4[1][0]
__________________________________________________________________________________________________
lambda_34 (Lambda)              (None, 32, 32, 32)   0           RegMap4[1][0]
__________________________________________________________________________________________________
lambda_35 (Lambda)              (None, 32, 32, 16)   0           RegMap5[1][0]
__________________________________________________________________________________________________
lambda_36 (Lambda)              (None, 32, 32, 32)   0           RegMap5[1][0]
__________________________________________________________________________________________________
lambda_37 (Lambda)              (None, 32, 32, 16)   0           RegMap6[1][0]
__________________________________________________________________________________________________
lambda_38 (Lambda)              (None, 32, 32, 32)   0           RegMap6[1][0]
__________________________________________________________________________________________________
lambda_39 (Lambda)              (None, 32, 32, 16)   0           RegMap7[1][0]
__________________________________________________________________________________________________
lambda_40 (Lambda)              (None, 32, 32, 32)   0           RegMap7[1][0]
__________________________________________________________________________________________________
lambda_41 (Lambda)              (None, 32, 32, 16)   0           RegMap8[1][0]
__________________________________________________________________________________________________
lambda_42 (Lambda)              (None, 32, 32, 32)   0           RegMap8[1][0]
__________________________________________________________________________________________________
sSAM (Model)                    (None, 16, 2)        33280       lambda_27[0][0]
                                                                 lambda_29[0][0]
                                                                 lambda_31[0][0]
                                                                 lambda_33[0][0]
                                                                 lambda_35[0][0]
                                                                 lambda_37[0][0]
                                                                 lambda_39[0][0]
                                                                 lambda_41[0][0]
__________________________________________________________________________________________________
cSAM (Model)                    (None, 32, 2)        67584       lambda_28[0][0]
                                                                 lambda_30[0][0]
                                                                 lambda_32[0][0]
                                                                 lambda_34[0][0]
                                                                 lambda_36[0][0]
                                                                 lambda_38[0][0]
                                                                 lambda_40[0][0]
                                                                 lambda_42[0][0]
__________________________________________________________________________________________________
cjProb (Model)                  (None, 32, 1)        0           lambda_28[0][0]
                                                                 lambda_30[0][0]
                                                                 lambda_32[0][0]
                                                                 lambda_34[0][0]
                                                                 lambda_36[0][0]
                                                                 lambda_38[0][0]
                                                                 lambda_40[0][0]
                                                                 lambda_42[0][0]
__________________________________________________________________________________________________
Agg (Model)                     (None, 16, 2)        512         sSAM[1][0]
                                                                 cSAM[1][0]
                                                                 cjProb[1][0]
                                                                 sSAM[2][0]
                                                                 cSAM[2][0]
                                                                 cjProb[2][0]
                                                                 sSAM[3][0]
                                                                 cSAM[3][0]
                                                                 cjProb[3][0]
                                                                 sSAM[4][0]
                                                                 cSAM[4][0]
                                                                 cjProb[4][0]
                                                                 sSAM[5][0]
                                                                 cSAM[5][0]
                                                                 cjProb[5][0]
                                                                 sSAM[6][0]
                                                                 cSAM[6][0]
                                                                 cjProb[6][0]
                                                                 sSAM[7][0]
                                                                 cSAM[7][0]
                                                                 cjProb[7][0]
                                                                 sSAM[8][0]
                                                                 cSAM[8][0]
                                                                 cjProb[8][0]
__________________________________________________________________________________________________
sjProb (Model)                  (None, 16, 1)        0           lambda_27[0][0]
                                                                 lambda_29[0][0]
                                                                 lambda_31[0][0]
                                                                 lambda_33[0][0]
                                                                 lambda_35[0][0]
                                                                 lambda_37[0][0]
                                                                 lambda_39[0][0]
                                                                 lambda_41[0][0]
==================================================================================================
Total params: 14,847,936
Trainable params: 14,669,952
Non-trainable params: 177,984
__________________________________________________________________________________________________

Do you have any idea what may be the issue? Is the label shape wrong?

@dluvizon I have tried every possible shape but its not working. Do you have an idea?

Hi @SaifAlDilaimi ,
It seems to me that you designed your model with 16 outputs, but you only passed 1 array for training.
In other words, if your model has intermediate supervisions (which is the case of my original model), you'll need to replicate the labels by the same amount of outputs that you have.

Hey @dluvizon ,
thanks for the hint! I have reshaped my labels to correlate with the output length of your model. Unfornatley it still dont work :( Using stages/supervisions of 6 (with export heatmaps set to False) I'm feeding the model following data:
Input shape:  (?, 256, 256, 3)
Output length:  12
Shape of x_train:  (15935, 256, 256, 3) , of type:  int8
Shape of y_train:  (15935, 16, 2) , of type:  int32
Shape of x_val:  (1992, 256, 256, 3) , of type:  int8
Shape of y_val:  (1992, 16, 2) , of type:  int32
Shape of x_test:  (1992, 256, 256, 3) , of type:  int8
Shape of y_test:  (1992, 16, 2) , of type:  int32
Reshaping label to correlate with deep supervision of stages
Shape of y_train:  (12, 15935, 16, 2) , of type:  int32
Shape of y_val:  (12, 1992, 16, 2) , of type:  int32
Shape of y_test:  (12, 1992, 16, 2) , of type:  int32

I'm reading the Model.output property and building a np.array containing the same y_label Model.output times. Still I'm getting the same error...
ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 12 array(s), but instead got the following list of 1 arrays: [array([[[[219, 147],
         [157, 131],
         [103, 156],
         ...,
         [ 59, 134],
         [106, 125],
         [145, 102]],

        [[228, 123],
         [187, 123],
         [146, ...

I'm feeding arrays containing labels of length 12. Why does it still says it receives array of length 1??

@dluvizon I think I found something: following the traceback I found this line in ther training.py in keras (https://github.com/keras-team/keras/blob/af804d0a5db9a8f20fbb083b48655b2687ce89d9/keras/engine/training.py#L78).
It compares if the input names are as many as the label arrays..  "names: List of expected array names."
Idk why that error is even thrown.. Do you have an idea?

I still think that the problem is related to the labels that you are passing for training:
Expected to see 12 array(s), but instead got the following list of 1 arrays:

You should pass 12 arrays for training, not one array of size 12.

Well sure I'm passing an array containing 12 arrays where each elements are my labels...
Reshaping label to correlate with deep supervision of stages
Shape of y_train:  (12, 15935, 16, 2) , of type:  int32
Shape of y_val:  (12, 1992, 16, 2) , of type:  int32
Shape of y_test:  (12, 1992, 16, 2) , of type:  int32

How can I pass 12 arrays directly? Currently I'm doing it like this:
def train(x_train, y_train, x_val, y_val, x_test, y_test, model_type=PreTrainedModel.CUSTOM):
    model = build_model()

    print("Shape of x_train: ", x_train.shape, ", of type: ", x_train.dtype)
    print("Shape of y_train: ", y_train.shape, ", of type: ", y_train.dtype)
    print("Shape of x_val: ", x_val.shape, ", of type: ", x_val.dtype)
    print("Shape of y_val: ", y_val.shape, ", of type: ", y_val.dtype)
    print("Shape of x_test: ", x_test.shape, ", of type: ", x_test.dtype)
    print("Shape of y_test: ", y_test.shape, ", of type: ", y_test.dtype)

    # checkpoint
    filepath="weights_"+model_type.name+".best.hdf5"
    reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.4, patience=5, min_lr=0.001)
    checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')
    tensorboard = TensorBoard(log_dir="logs/{}".format(time()), histogram_freq=0, write_graph=True, write_images=False)
    csv_logger = CSVLogger('log_'+model_type.name+'.csv', append=True, separator=';')

    callbacks_list = [reduce_lr, checkpoint, tensorboard, csv_logger]

    # Modify output to correlate with stages
    print("Reshaping label to correlate with deep supervision of stages")
    out_y_train, out_y_val, out_y_test = [], [], []

    for i in range(0, len(model.output)):
        out_y_train.append(y_train)
        out_y_val.append(y_val)
        out_y_test.append(y_test)

    y_train = np.array(out_y_train)
    y_val = np.array(out_y_val)
    y_test = np.array(out_y_test)

    print("Types: ", all(isinstance(x, np.ndarray) for x in y_train[0][0][0]))

    print("Shape of y_train: ", y_train.shape, ", of type: ", y_train.dtype)
    print("Shape of y_val: ", y_val.shape, ", of type: ", y_val.dtype)
    print("Shape of y_test: ", y_test.shape, ", of type: ", y_test.dtype)

    model.fit(x_train, y_train,
            batch_size=PARAMS.ML_BATCH_SIZE,
            epochs=PARAMS.ML_EPOCHS,
            verbose=1,
            validation_data=(x_val, y_val), callbacks=callbacks_list)
    score = model.evaluate(x_test, y_test, verbose=0)
    print('Test loss:', score[0])
    print('Test accuracy:', score[1])

@dluvizon I'm one step further! You were right that my arrays were not formatted correctly.. Although now the model throws a error that seems to be linked to the joints probability layer sjProb:
Traceback (most recent call last):
  File "Train.py", line 154, in <module>
    main()
  File "Train.py", line 151, in main
    train(x_train, y_train, x_val, y_val, x_test, y_val)
  File "Train.py", line 76, in train
    validation_data=(x_val, out_y_val), callbacks=callbacks_list)
  File "C:\Users\saifa\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1630, in fit
    batch_size=batch_size)
  File "C:\Users\saifa\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1480, in _standardize_user_data
    exception_prefix='target')
  File "C:\Users\saifa\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 123, in _standardize_input_data
    str(data_shape))
ValueError: Error when checking target: expected sjProb to have shape (16, 1) but got array with shape (16, 2)

Where do I add this array and what should be its content? Thank you so much for the help!

@dluvizon can you give me any hints?

Hi,
ValueError: Error when checking target: expected sjProb to have shape (16, 1) but got array with shape (16, 2)

You have to provide joint probabilities in the right shape (16,1).

Can you elloborate this more? where do I get the joint probabilities? @dluvizon

If you want to supervise joint probabilities (section 3.2.2 in the paper), you need to provide the labels for that in the shape (16,1). Basically, it is 1 if the joint if available, 0 otherwise.

okay ty, but I'm calling keras fit with a list containing 12x arrays of my joints location of shape (15935, 16, 2) + 1 array that contains 16x 1 [[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]]. I build those like this:
def train(x_train, y_train, x_val, y_val, x_test, y_test):
    # x,y are in following shapes passed
    # Input shape:  (?, 256, 256, 3)
    # Output length:  18
    # Shape of x_train:  (15935, 256, 256, 3) , of type:  int8
    # Shape of y_train:  (15935, 16, 2) , of type:  int32
    # Shape of x_val:  (1992, 256, 256, 3) , of type:  int8
    # Shape of y_val:  (1992, 16, 2) , of type:  int32
    # Shape of x_test:  (1992, 256, 256, 3) , of type:  int8
    # Shape of y_test:  (1992, 16, 2) , of type:  int32

    model = build_model()
    out_y_train, out_y_val, out_y_test = [], [], []

    for i in range(0, len(model.output) - 1):
        out_y_train.append(y_train)
        out_y_val.append(y_val)
        out_y_test.append(y_test)

    # append label array for joint probability
    y_sjprob_train = np.array([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]])
    out_y_train.append(y_sjprob_train)
    y_sjprob_test = np.array([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]])
    out_y_test.append(y_sjprob_test)
    y_sjprob_val = np.array([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]])
    out_y_val.append(y_sjprob_val)

    model.fit(x_train, out_y_train,
            batch_size=PARAMS.ML_BATCH_SIZE,
            epochs=PARAMS.ML_EPOCHS,
            verbose=1,
            validation_data=(x_val, out_y_val), callbacks=callbacks_list)

I'm still getting the same error @dluvizon :( Can you give me an example how the training data should look and how I pass the training data for the sjProb layer? Any help is highly apreciated

hey @dluvizon ,
after checking which outputs the network generates and specialy in which order:
Output:  Tensor("Agg/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_1/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_1/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_2/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_2/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_3/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_3/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_4/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_4/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_5/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_5/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)

So I have build my labels to contain always the joint probability array:
    # Modify output to correlate with stages
    print("Reshaping label to correlate with deep supervision of stages")
    out_y_train, out_y_val, out_y_test = [], [], []
    sjprob_init = [[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]]

    for i in range(0, len(model.output)):
        # append label array for joint probability
        out_y_train.append(np.array([y_train, sjprob_init]))
        out_y_test.append(np.array([y_test, sjprob_init]))
        out_y_val.append(np.array([y_val, sjprob_init]))

unfornatley I have now another error that belongs to the context aggregation layer. I'm getting now this error:
Traceback (most recent call last):
  File "Train.py", line 146, in <module>
    main()
  File "Train.py", line 143, in main
    train(x_train, y_train, x_val, y_val, x_test, y_val)
  File "Train.py", line 68, in train
    validation_data=(x_val, out_y_val), callbacks=callbacks_list)
  File "C:\Users\saifa\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1630, in fit
    batch_size=batch_size)
  File "C:\Users\saifa\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1480, in _standardize_user_data
    exception_prefix='target')
  File "C:\Users\saifa\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 113, in _standardize_input_data
    'with shape ' + str(data_shape))
ValueError: Error when checking target: expected Agg to have 3 dimensions, but got array with shape (2, 1)

Now I'm not sure what is needed here. My labels are of shape (16,2) for joints x,y and the joints probability (16,1) where 1 returns a probability (right?), 0 not. So what does the layer Agg need here? 3 dimension are mostly images... but I have read now your paper (section 3.2.3?) but I can't seem to find a reference to this.. Can you give me a hint to what data that layer expect and why this error is thrown?

@dluvizon the problem is I'm not able to figure out what labels your models needs. If I know what data your models expects as labels I can provide that. It has not really something todo with my implementation. Its a general question.

But you already know that, as you posted here:
Output:  Tensor("Agg/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_1/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_1/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_2/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_2/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_3/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_3/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_4/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_4/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)
Output:  Tensor("Agg_5/add_2/add:0", shape=(?, 16, 2), dtype=float32)
Output:  Tensor("sjProb_5/lambda_9/ExpandDims:0", shape=(?, 16, 1), dtype=float32)

The model expects a tensor of shape (Batch_size, 16, 2) for poses and (Batch_size, 16, 1) for joint probabilities, which you can use a vector of ones, like you did. Then you have to provide the same amount of labels (in the right order) as the model is expecting.
Finally, if you can produce these outputs by passing an image through the model you should be able to train it, right?

@dluvizon sorry for the late response but its working and I'm able to train the model! I'm just curious because after 47/120 epochs the accuracy of all aggregation layers are the same.. its starts at acc of 0.65 and goes down to 0.47 for each epoch... here are some epochs results:
Epoch 44/120
 - 1782s - loss: 4634412.5771 - Agg_loss:<PHONE_NUMBER> - sjProb_loss: 1.1921e-07 - Agg_acc: 0.4487 - sjProb_acc: 1.0000 - Agg_acc_1: 0.4487 - sjProb_acc_1: 1.0000 - Agg_acc_2: 0.4487 - sjProb_acc_2: 1.00
00 - Agg_acc_3: 0.4487 - sjProb_acc_3: 1.0000 - Agg_acc_4: 0.4487 - sjProb_acc_4: 1.0000 - Agg_acc_5: 0.4487 - sjProb_acc_5: 1.0000 - Agg_acc_6: 0.4487 - sjProb_acc_6: 1.0000 - Agg_acc_7: 0.4487 - sjPro
b_acc_7: 1.0000 - val_loss: 4631173.0040 - val_Agg_loss:<PHONE_NUMBER> - val_sjProb_loss: 1.1921e-07 - val_Agg_acc: 0.4519 - val_sjProb_acc: 1.0000 - val_Agg_acc_1: 0.4519 - val_sjProb_acc_1: 1.0000 - val
_Agg_acc_2: 0.4519 - val_sjProb_acc_2: 1.0000 - val_Agg_acc_3: 0.4519 - val_sjProb_acc_3: 1.0000 - val_Agg_acc_4: 0.4519 - val_sjProb_acc_4: 1.0000 - val_Agg_acc_5: 0.4519 - val_sjProb_acc_5: 1.0000 - v
al_Agg_acc_6: 0.4519 - val_sjProb_acc_6: 1.0000 - val_Agg_acc_7: 0.4519 - val_sjProb_acc_7: 1.0000

Epoch 00044: val_Agg_acc did not improve
Epoch 45/120
 - 1781s - loss: 4634412.5756 - Agg_loss:<PHONE_NUMBER> - sjProb_loss: 1.1921e-07 - Agg_acc: 0.4487 - sjProb_acc: 1.0000 - Agg_acc_1: 0.4487 - sjProb_acc_1: 1.0000 - Agg_acc_2: 0.4487 - sjProb_acc_2: 1.00
00 - Agg_acc_3: 0.4487 - sjProb_acc_3: 1.0000 - Agg_acc_4: 0.4487 - sjProb_acc_4: 1.0000 - Agg_acc_5: 0.4487 - sjProb_acc_5: 1.0000 - Agg_acc_6: 0.4487 - sjProb_acc_6: 1.0000 - Agg_acc_7: 0.4487 - sjPro
b_acc_7: 1.0000 - val_loss: 4631173.0040 - val_Agg_loss:<PHONE_NUMBER> - val_sjProb_loss: 1.1921e-07 - val_Agg_acc: 0.4519 - val_sjProb_acc: 1.0000 - val_Agg_acc_1: 0.4519 - val_sjProb_acc_1: 1.0000 - val
_Agg_acc_2: 0.4519 - val_sjProb_acc_2: 1.0000 - val_Agg_acc_3: 0.4519 - val_sjProb_acc_3: 1.0000 - val_Agg_acc_4: 0.4519 - val_sjProb_acc_4: 1.0000 - val_Agg_acc_5: 0.4519 - val_sjProb_acc_5: 1.0000 - v
al_Agg_acc_6: 0.4519 - val_sjProb_acc_6: 1.0000 - val_Agg_acc_7: 0.4519 - val_sjProb_acc_7: 1.0000

Epoch 00045: val_Agg_acc did not improve
Epoch 46/120
 - 1782s - loss: 4634412.5766 - Agg_loss:<PHONE_NUMBER> - sjProb_loss: 1.1921e-07 - Agg_acc: 0.4487 - sjProb_acc: 1.0000 - Agg_acc_1: 0.4487 - sjProb_acc_1: 1.0000 - Agg_acc_2: 0.4487 - sjProb_acc_2: 1.00
00 - Agg_acc_3: 0.4487 - sjProb_acc_3: 1.0000 - Agg_acc_4: 0.4487 - sjProb_acc_4: 1.0000 - Agg_acc_5: 0.4487 - sjProb_acc_5: 1.0000 - Agg_acc_6: 0.4487 - sjProb_acc_6: 1.0000 - Agg_acc_7: 0.4487 - sjPro
b_acc_7: 1.0000 - val_loss: 4631173.0040 - val_Agg_loss:<PHONE_NUMBER> - val_sjProb_loss: 1.1921e-07 - val_Agg_acc: 0.4519 - val_sjProb_acc: 1.0000 - val_Agg_acc_1: 0.4519 - val_sjProb_acc_1: 1.0000 - val
_Agg_acc_2: 0.4519 - val_sjProb_acc_2: 1.0000 - val_Agg_acc_3: 0.4519 - val_sjProb_acc_3: 1.0000 - val_Agg_acc_4: 0.4519 - val_sjProb_acc_4: 1.0000 - val_Agg_acc_5: 0.4519 - val_sjProb_acc_5: 1.0000 - v
al_Agg_acc_6: 0.4519 - val_sjProb_acc_6: 1.0000 - val_Agg_acc_7: 0.4519 - val_sjProb_acc_7: 1.0000

Epoch 00046: val_Agg_acc did not improve
Epoch 47/120
 - 1780s - loss: 4634412.5786 - Agg_loss:<PHONE_NUMBER> - sjProb_loss: 1.1921e-07 - Agg_acc: 0.4487 - sjProb_acc: 1.0000 - Agg_acc_1: 0.4487 - sjProb_acc_1: 1.0000 - Agg_acc_2: 0.4487 - sjProb_acc_2: 1.00
00 - Agg_acc_3: 0.4487 - sjProb_acc_3: 1.0000 - Agg_acc_4: 0.4487 - sjProb_acc_4: 1.0000 - Agg_acc_5: 0.4487 - sjProb_acc_5: 1.0000 - Agg_acc_6: 0.4487 - sjProb_acc_6: 1.0000 - Agg_acc_7: 0.4487 - sjPro
b_acc_7: 1.0000 - val_loss: 4631173.0040 - val_Agg_loss:<PHONE_NUMBER> - val_sjProb_loss: 1.1921e-07 - val_Agg_acc: 0.4519 - val_sjProb_acc: 1.0000 - val_Agg_acc_1: 0.4519 - val_sjProb_acc_1: 1.0000 - val
_Agg_acc_2: 0.4519 - val_sjProb_acc_2: 1.0000 - val_Agg_acc_3: 0.4519 - val_sjProb_acc_3: 1.0000 - val_Agg_acc_4: 0.4519 - val_sjProb_acc_4: 1.0000 - val_Agg_acc_5: 0.4519 - val_sjProb_acc_5: 1.0000 - v
al_Agg_acc_6: 0.4519 - val_sjProb_acc_6: 1.0000 - val_Agg_acc_7: 0.4519 - val_sjProb_acc_7: 1.0000

Do you have any idea what may be the issue here? As you described in section 4.1 I have setup an array that contains the loss functions depending on the number of generated outputs. I will let it run till the end but it feels wrong...

Hello @dluvizon , in your code you set a Flag to export heatmaps. On my training I didn't export heatmaps. Is this maybe the problem?
To test it I have tried to export heatmaps but with K = 8 I have to provide 8 labels in following shapes:
Tensor("lambda_27/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_29/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_31/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_33/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_35/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_37/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_39/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)
Tensor("lambda_41/strided_slice:0", shape=(?, 32, 32, 16), dtype=float32)

What do I need to provide here? Images? I know that I'm really stubborn but I really want to understand your work..

Hi,
The option to export heatmaps is only to visualize them after training.
BTW, there is something really wrong with your loss here:
loss: 4634412.5786

That value is too high.

Hi, I don't know how you are getting the acc. because there is no function on Keras that implements PCK nor PCP.
The function pckh(...) in my code do that. The references are in the paper.
In practical case, you train for x epochs, compute the predictions, and finally compute the PCKh.

@dluvizon can you just tell me on which metric of the validation set you adjust the learning rate?
The learning rate begins at 10^3 and decreases by a factor of 0.4 when accuracy on validation plateaus.
Which accuracy do you mean?

Hi,
As it is written in the paper, the decrease factor is applied when accuracy on validation plateaus and the prediction taken into account is the last one.
There is only one accuracy per model, which is the PCKh metric for MPII.

So with model in that context you also mean the Blocks K_i?

Also: When using tensorboard I have several layers that I can inspect (every layer in the model summary see  above) but which layer represent or displays the weights? As I know the conv2d layers but the weights seem as if they don't change:

In your paper you wrote:

The implementation of Soft-argmax can be easily done
with recent frameworks, such as TensorFlow, just by concatenating
a spatial softmax followed by one convolutional
layer with 2 filters of size W x H, with fixed parameters
according to equation (3).

Do I need todo here something or is it all done in def build_softargmax_2d(input_shape, name=None)  function?

I mean, there is only one type o accuracy. If the model has K supervisions, then it has K accuracy scores, but all of them are the same type (PCKh for example) and only the last is used to decrease the lr.

I don't know what you want to see with these pictures, but what I said in the paper is implemented in build_softargmax_2d.

If the model has K supervisions, then it has K accuracy scores, but all of them are the same type (PCKh for example) and only the last is used to decrease the lr.

Well thats exactly what irritates me. You wrote before:

In practical case, you train for x epochs, compute the predictions, and finally compute the PCKh.

So how do I on one side train, predict and then calculate the PCKh and on the other hand reduce the lr based on PCKh while training? OR should calculate custom metric (PCKh) WHILE training and use that instead of the accuracy of the aggregation layers that keras outputs?

I'm sorry if my last message sounded in any kind rude. This wasn't my intention! I'm very glad that you are helping me!
 Thanks again for your time :)


Fix incorrect stat reporting

With the default Logger, mutilate was reporting the stddev as the
minimum and the minimum as the 1st percentile.

bump @leverich


fix error of add suffix

2^0 should add to bin suffixes
Signed-off-by: duguhaotian<EMAIL_ADDRESS>Sorry, we do not accept changes directly against this repository. Please see
CONTRIBUTING.md for information on where and how to contribute instead.

hi, PRs should go here instead:
https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/apimachinery
also please sign the CLA, thanks.
/close


Implement mutli tenant tokens

Hi, I currently have a use case with multi tenant tokens. In the use case I want to create a virtual machine inside one tenant with an image reference from another tenant. I have successfully been able to do this with the corresponding Go library by using the following struct https://github.com/Azure/go-autorest/blob/v14.2.0/autorest/authorization.go#L300
Suggestively the same feature could be implemented in this library by appending bearer tokens to the x-ms-authorization-auxiliary (https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/authenticate-multi-tenant) header when signRequest is invoked (however, the scope is most likely much broader than this).
As a workaround outside the library, you can implement the ServiceClientCredentials interface and sign the request headers according to the above. However, it would be better to do this directly in the library. As a reference:
class MultiTokenServiceCredentials {
  constructor(primaryToken, auxiliaryToken) {
      this.primaryToken = primaryToken
      this.auxiliaryToken = auxiliaryToken
  }
  async signRequest(webResource) {
    const primaryTokenResponse = await this.primaryToken.getToken();
    const auxiliaryTokenResponse = await this.auxiliaryToken.getToken();
    webResource.headers.set(`authorization`, `${primaryTokenResponse.tokenType} ${primaryTokenResponse.accessToken}`);
    webResource.headers.set(`x-ms-authorization-auxiliary`, `${auxiliaryTokenResponse.tokenType} ${auxiliaryTokenResponse.accessToken}`);
    return webResource;
  }
}
...
const primaryToken = await msRestNodeAuth.loginWithServicePrincipalSecretWithAuthResponse(appID, secret, clientTenant);
const auxiliaryToken = await msRestNodeAuth.loginWithServicePrincipalSecretWithAuthResponse(appID, secret, auxiliaryTenant);
const serviceCredentials = new MultiTokenServiceCredentials(primaryToken.credentials, auxiliaryToken.credentials);
const vmClient = new compute.ComputeManagementClient(serviceCredentials, subID)

Thanks for reporting @lovew-lc and our apologies for not replying sooner.
We are moving our efforts on authentication with Azure Active Directory to a new project @azure/identity. I'll be transferring this issue to the repo for the new package to continue this discussion.


feat: add and export wallet connectors

Build and publish @fuels/connectors package
Basic example
import { Fuel } from "@fuels";
import { FuelWalletConnector } from '@fuels/connectors';

const fuel = new Fuel({
  connectors: [
    new FuelWalletConnector(),
  ],
});

Can we include the CI for release latest and preview

https://github.com/FuelLabs/fuel-connectors/pull/1/commits/0c56a524e70cc670b8c2651b9e299e5ab2c1e1c4

Hey @helciofranco great work 👍
I have two questions:
1 - Do the tests for these connectors exist on another repo?
2 - I am having a Typescript error when trying to use any of the connectors implemented here within the Fuel connector manager:
 What am I missing?

1 - nice, I missed those tests, let me get them back.
2 - I'll try to replicate here to find the issue.


Scipy.sparse.csgraph has no atribute reduce_cython

Im bulding a tkinter app who is working fine, but when i try to run the executable version builded with cx_freeze, i get the following line
Error: AtrributeError:Type object 'scipy.sparse.csgraphy' has no attribute 'reduce_cython'
Error message:

Python Version: 3.7.3
Scipy: 1.3.2
Cython: 0.29.13

Looks similar to https://github.com/cython/cython/issues/1953, and likely not an issue with scipy itself. You may want to check if the installed scipy version in your site-packages was built with an older Cython verison (see https://github.com/cython/cython/issues/1953#issuecomment-398128940).

Looks similar to cython/cython#1953, and likely not an issue with scipy itself. You may want to check if the installed scipy version in your site-packages was built with an older Cython verison (see cython/cython#1953 (comment)).

following the traceback the problem start with the importk of nltk and then with scipy library but nltk doesnt have any information about cython version.

This was an issue in Cython that has long since been fixed, so closing.


Support Ascii function for ascii and latin-1 [databricks]

Closes #9585
This PR part supported ascii function for strings starting with ASCII and Latin-1 supplement, returning results from 0 to 255. The function is disabled by default.
This PR uses cudf::code_points to get the utf-8 code points of the first letter in a string, and converts to ASCII using the following rules:

ascii
utf-8 dec
workaround

0~127
0~127
cudf::code_points

128~191
49792~49855
code_points - 49,664

192~255
50048~50111
code_points -49,856

It is good enough for customer, we can file another issue to fully support it when needed.
perf test results
data: 50000000 string from big datagen:
spark.time(df.selectExpr("COUNT(ascii(a)) as asc0", "COUNT(ascii(a)) as asc1", "COUNT(ascii(a)) as asc2", "COUNT(ascii(a)) as asc3", "COUNT(ascii(a)) as asc4", "COUNT(ascii(a)) as asc5", "COUNT(ascii(a)) as asc6", "COUNT(ascii(a)) as asc7", "COUNT(ascii(a)) as asc8", "COUNT(ascii(a)) as asc9").show())

GPU Time (ms)
CPU Time (ms)
Speed up

1,653
2,787
69%

Did some perf tests, and gpu is slightly faster than cpu (12%). I plan to refine my implementation to reduce times of cuDF API calling, but the undefined values will be more undefined, which doesn't matter for now.

Ok, I refined the implementation and it now runs faster, about 69% speedup over cpu.

build

Test failed in CI because Latin-1 support in Spark is from 3.3.1, will add shim for it so ascii under 3.3.1 will be faster and (maybe) fully supported.

build

build

build

Looks like we need to fix this for 3.5.1 too

build

perf test results on the byte-based solution (in lower version shims):

GPU Time (ms)
CPU Time (ms)
Speed up

541.4
2,020.6
3.73x

The previous tests timed out trying to run on databricks.  Trying again.

build


Lean very slow after importing large theories

On my notebook, lean becomes very slow after importing certain theories from the library, for example flycheck already takes around 5 seconds after each keystroke in this file:
import homotopy.sphere

open unit

example : unit := ⋆

On my office computer this is not the case. Both are on the current master branch. Does anyone have an idea why that is or pointers to possible troubleshooting?
It's not a flycheck problem, directly compiling it in a terminal also produces the big difference. It also slows down the compilation time of the library to ~30 min.

Have you compiled in debug mode?

No, release mode.

I can't reproduce this problem. On my machine it takes 0.1 sec to process the file, and it consumes 20Mb. I would rebuild everything from scratch (delete build directory and .olean files and rebuild).
I would also check if there are other processes running in the background and consuming all resources (e.g., memory).

I deleted the entire repository clone actually. There's no memory consuming background processes but I noticed that at least the library build starts three lean threads even if I've only two processor cores. But I don't think that's the cause...

I don't really know how I managed to do this but there was a ninja process owned by root? Which might be the cause? It didn't consume much CPU time, but when I killed it, Lean became faster again...


VS Code shows pending changes in freshly cloned repository

Issue Type: Bug
When i clone a repository in WSL (Ubuntu) and then open the repository with VS Code, the Source Control shows pending changes, although I didn't change anything.
git status tells me that my branch is up-to -date, as it should be.
VS Code version: Code 1.40.1 (8795a9889db74563ddd43eb0a897a2384129a619, 2019-11-13T16:49:35.976Z)
OS version: Windows_NT x64 10.0.18362

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (8 x 3392)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_offsurface_control: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.89GB (9.92GB free)

Process Argv
.

Screen Reader
no

VM
0%

Extensions (6)

Extension
Author (truncated)
Version

xml
Dot
2.5.0

python
ms-
2019.10.44104

cpptools
ms-
0.26.1

powershell
ms-
2019.11.0

java
red
0.53.1

cmake
twx
0.0.17

"git config core.filemode false" fixed my problem

Thank you CR82. That worked for me, too.


I open the app but it just a blank windows

I'm using Macbook OS Sierra (MQLT2)
I just download Caption version 2 and when I open it is just a blank windows and I can not do anything with it.
I had record the reproduce here
http://recordit.co/0MYv0rgL5Z

There seems to be a bug when you rename the app. I think the app works if you just call is Caption.app We're working on a fix.

thanks you, I rename the application and it work now


Skip static columns paging test 10476

As per a comment from Sylvain on CASSANDRA-10476:

We do expect 2.2.3 -> 3.0 upgrade to succeed. We don't expect
static_columns_paging_test to pass however since that test don't even
pass for non-upgrade test on 2.2.3, it's a bug of 2.2.3 that has been
fixed in 2.2.4 (CASSANDRA-10381). If we can make so that the test
doesn't run on 2.2.3 (but does run once we'll update to 2.2.4), that
would be perfect.

@pcmanus is this what you had in mind? This will skip the entire test if any node is version 2.2.3 or lower. Another option is to still run the test, but only query nodes running higher versions. I'm not sure what configurations we expect errors in.

Looks good on principle. I haven't tested it myself, but if you have, +1.

Great, thank you. Merging.


Feat: add getTopChampionMasteries

lockfile autoupdated
Endpoint docs: https://developer.riotgames.com/apis#champion-mastery-v4/GET_getTopChampionMasteries

This should be done by https://github.com/fightmegg/riot-rate-limiter/pull/394 apologies for taking so long


C3: Refactor dns polling to improve experience for WARP users

What this PR solves / how to test:
This gives WARP users a smoother experience by doing a more custom lookup of DNS that avoid potential caching issues.
Associated docs issue(s)/PR(s):

[insert associated docs issue(s)/PR(s)]

Author has included the following, where applicable:

[ ] Tests
[ x ] Changeset (Changeset guidelines)

Reviewer is to perform the following, as applicable:

Checked for inclusion of relevant tests
Checked for inclusion of a relevant changeset
Checked for creation of associated docs updates
Manually pulled down the changes and spot-tested

Note for PR author:
We want to celebrate and highlight awesome PR review! If you think this PR received a particularly high-caliber review, please assign it the label highlight pr review so future reviewers can take inspiration and learn from it.

oops.. just noticed that the test is failing.. due to a flake? please take a look

oops.. just noticed that the test is failing.. due to a flake? please take a look

Yep, dealing with some flakiness, working through it now.


AVRO-2603: Clean Up Imports

This change moves the imports embedded in test functions to the top of the test module. It also standardizes the imports and removes some unnecessary ones.
Jira

[x] My PR addresses AVRO-2603.
[x] My PR title references the ticket.
[x] My PR doesn’t add any dependencies.

Tests

[x] My PR improves an existing test.

Commits

[x] My commits all reference Jira issues in their subject lines.
[x] My commits follow the guidelines from "How to write a good git commit message"

Documentation

[x] My PR doesn’t add functionality and doesn’t require documentation.

Thanks @kojiromike
Shouldn't this be caught by isort? https://github.com/apache/avro/blob/master/lang/py3/build.sh#L32
Would be great to have the CI check the import order, and fail if it doesn't pass isort.

Unfortunately isort can't catch imports that are embedded within a class or function.

It also doesn't catch unused imports. This change removes some unused imports as well.

Ok, let's maybe introduce mypy in a different ticket :)

@Fokko oh... I'm confused. I didn't introduce mypy in this ticket. I definitely want to introduce it though.

@Fokko Even though mypy can do typechecking on a python2 codebase, it has to run using python3. (3.5 or up, I think.) I am trying to devise a way to run more versions of Python in our docker/travis/yetus setup. I'd appreciate any advice you have.


Can this be used outside of Android projects?

This looks like the only complete AspectJ plug-in for Gradle that supports compiling *.aj separately from Java. I was wondering if there is any problem or drawback to using this project outside of an Android Gradle project (for example, Spring).

Nowdays the plugin depends on android's compile flow and Transform API. I'm planning to write a java/groovy/kotlin/scala separate implementation in future. So stay tuned or contribute :)
In general, needs to unbound compiler task from android structure and write a weaver-task on pure gradle mechanics. Also I have a little expirience with pure java projects so dunno about some specific stuff that could broke this idea.

No time to write a pure java wrapper :(


Integrate Coveralls

In pr.yaml or push.yaml (whichever is appropriate, please integrate Coveralls (I've already configured things at coveralls.io). An example: https://github.com/open-contracting/ocds-merge/blob/main/.github/workflows/ci.yml#L27-L29

https://coveralls.io/github/open-contracting/spoonbill-web