Remove jQuery dependency

The library now works without jQuery using native Javascript.
jQuery is now only a development dependency for tests using
jasmine-jquery matchers.
However, the jQuery plugin is still added if jQuery is defined.
Some caveats:
1- I am not very confident of src/js/wrapper.js.ejs correctness.
2- The minimized version produces errors when jQuery is not defined
while the non-minfied version works correctly. This is probably
because of some hacky behavior in the tooling. Most likely in
grunt/replace.js. In addition, appearantly, replace tasks aren't
applied to all files (e.g: some files spec/). Anyways, I personally
think that grunt/replace.js should be replaced by a more
conventional solution; if library size is so important then all
scripts that are always used should be concatenated, uglified and
gzipped.
Please, review and let me know what I should do to make it easy to merge the PR.

You can find the demo here: http://ahimta.github.io/intl-tel-input/

Wow this is super cool, thank you! I had wanted to look into this anyway, but not had the time.
My only concern is filesize, and it looks like there's a lot of green additions in this pull request! Do you have an idea of filesize of the built minified file before and after this change?

Before: 23 KB
After: 30 KB
Unfortunately, for now the minified version doesn't work without. Most likely grunt/replace.js has something to do with it.
I didn't have filesize in mind when I was working on this. If you really care about filesize that much, I can make it a little bit smaller.
However, I think the most important part is to concatinate all files that are always used, uglify them and then gzip them. In general trying to make it smaller, more using tools, than manual effort.

Hmm have you read this doc? https://twitter.com/paul_irish/status/431584056883429376

Seems like a complete nightmare, keeping up with all the different bugs for the different browsers, and updating all of the extra checks/hacks as they are fixed, or as we drop support for old browsers etc...

I would almost never use the native Javascript API for web apps. I would use (e.g: Angular, React).
Even writing a web app using jQuery is time consuming.
However, for Javascript libraries it's best to not use jQuery (and many Javascript libs don't use jQuery).
In reality, not everyone is using jQuery and if they do, they are using different versions that could produce different behavior and break the library code.
I read the doc.
jQuery has many many features and intl-tel-input is using only a small subset of them, that why we can without much effort make it work without jQuery.
Additionally, jQuery browser support isn't as impressive as most people would think.
And I am happy that it isn't. Otherwise the development would be crippled and we would end up with a worse product.
In my opinion, browser support is a tradeoff. If you want to support older browsers (e.g: IE7, IE8) there's a big price (even jQuery2 doesn't support them). It's okay to do it as long as you understand the tradeoff.

In reality, not everyone is using jQuery and if they do, they are using different versions that could produce different behavior and break the library code.

We work with >=1.7 which was released in 2011, so unlikely to be a problem.

jQuery has many many features and intl-tel-input is using only a small subset of them

I disagree. We're using:

selectors
element creation
DOM manipulation
event listening
$.extend
$.Deferred
$.inArray
$.isNumeric
$.trim
$.ajax
$.data
$.when
$.each

All of these contain their own fixes for various browsers/versions (even modern browsers), which we would now have to manually keep up-to-date. We also lose out on the millions of people using jQuery providing bug reports, and the fixes that get written in response.

Additionally, jQuery browser support isn't as impressive as most people would think.

I'm not sure what your point is - if you look at that doc it is clear they have fixes for hundreds of browser bugs. If we don't use jQuery we have to do all of that ourselves.

If you want to support older browsers (e.g: IE7, IE8) there's a big price

I hate supporting these old browsers, but it's fairly easy to do in this case - just use jQuery v1.
I'm sorry but I don't think we should drop jQuery for two reasons: stability (in terms of browser support), and filesize.

Thanks for your honesty and fast reply, seriously!
I think it's worth it for Javascript libs to work without jQuery.
So I will work on my fork for the long term; that's the beauty of open source.
What I plan to do (mostly):
1- Maintain my own fork.
2- Incorporate all new intl-tel-input features into my fork.
3- Make sure it has a good browser support, probably using [https://saucelabs.com/](Sauce Labs).
4- Move jQuery plugin to its own repo (using my fork through package mangers: npm, bower, component).
5- Write different wrappers, each in its own repo for Angular, React, Ember (eventually).

Sounds good. Thanks for your interest in the plugin!

Thanks, for work, it's perfect widget.
But, you know, jQuery really make sense for project based on other framework (Angularjs, for me as example)
If I want use your perfect lib, i should make 3 steps:

Include angular wrapper (like https://github.com/mareczek/international-phone-number);
Include your lib;
Include jQuery;

It's a bit overkill for me, cause angularjs already has jqlite inside and no other libs required for jquery.
I want to ask - is there any known forks without jQuery dependency?
@Ahimta How works going on? Is it your repo stable?

@se-panfilov you can use https://github.com/Ahimta/bc-phone-number.
It depends only on Angular, Angular UI, Bootstrap CSS files (not js files) and some other libs I made
specifically for bc-phone-number.

Update: I'm now looking to try to move away from jQuery incrementally - help appreciated! https://github.com/jackocnr/intl-tel-input/issues/405

You can start by refactoring. For example, factor out the logic that doesn't depend on the DOM into its own functions, and have the parts that deal with the DOM, events, etc... call these pure functions.


Beta distribution

I would love to be able to use GitHub Actions for sharing beta builds since it's a trustworthy source and not just some random jar off my computer, BUT it's impossible for people without GitHub accounts to download it, I've looked around and found this hacky workaround service for getting direct links, but it's adding another service we need to trust to it...
https://nightly.link/
As an alternative, I think it's viable to keep sending files directly in the Discord to be publicly available, but I think we should modify the CI to include the hash in the zip name (or somewhere else easily accessible) in order to verify downloaded files either from the service mentioned above, most people won't make use of this, but I think it'd be worthwhile to ensure nobody is sneaking stuff in
We could also use GitHub releases for this but it would really flood the GitHub with garbage releases and would be very limited
This could be relevant to the auto-updater in the future for downloading beta/nightly versions and validating the integrity of the builds
Discord forum post for relevant discussion: https://discord.com/channels/817184208525983775/1044111027542970428

We are now using Github pre-releases for release candidates and Discord forums for all betas.


include missing vector in detail/mmap.ipp

I was using mio in a project compiled with mingw-w64 toolchain on arch and I noticed it was missing
In file included from /build/MIOProject/include/mio/mmap.hpp:490,
                 from /src/pack.cpp:6:
/build/MIOProject/include/mio/detail/mmap.ipp: In function ‘std::wstring mio::detail::win::s_2_ws(const string&)’:
/build/MIOProject/include/mio/detail/mmap.ipp:60:21: error: ‘vector’ is not a member of ‘std’
   60 |     auto buf = std::vector<wchar_t>(s_length);
      |                     ^~~~~~
/build/MIOProject/include/mio/detail/mmap.ipp:29:1: note: ‘std::vector’ is defined in header ‘<vector>’; did you forget to ‘#include <vector>’?
   28 | #include <algorithm>
  +++ |+#include <vector>
   29 |
/build/MIOProject/include/mio/detail/mmap.ipp:60:28: error: expected primary-expression before ‘wchar_t’
   60 |     auto buf = std::vector<wchar_t>(s_length);

I also ran amalgamate.py for the single-include file, please correct me if I did it wrong.

Same issue here.

FYI, https://github.com/mandreyel/mio/pull/88 fixes this problem as well as linking errors

Thanks for the fix!


Documentation/op-guide: fix failed RPC rate, leader election metrics

This fixes failed RPC rate query, where we do not need
subtraction because we already query by the status code.
Also adds grpc_method to make it more specific. Most of the
time, the failure recovers within 10-second, which is our
Prometheus scrap interval, so 'rate' query might not cover
that time window, showing as 0s, but still shows up in the graph.
Before

After

Also fixes RPC rate query.

Codecov Report

Merging #8093 into master will decrease coverage by 0.14%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #8093      +/-   ##
==========================================
- Coverage   76.65%   76.51%   -0.15%
==========================================
  Files         342      342
  Lines       26592    26568      -24
==========================================
- Hits        20384    20328      -56
- Misses       4759     4785      +26
- Partials     1449     1455       +6

Impacted Files
Coverage Δ

clientv3/namespace/watch.go
87.87% <0%> (-12.13%)
:arrow_down:

proxy/grpcproxy/register.go
83.33% <0%> (-11.12%)
:arrow_down:

pkg/fileutil/purge.go
73.68% <0%> (-7.9%)
:arrow_down:

etcdserver/api/v3rpc/lease.go
92.85% <0%> (-7.15%)
:arrow_down:

clientv3/txn.go
96.36% <0%> (-3.64%)
:arrow_down:

proxy/grpcproxy/watch.go
90.97% <0%> (-3.48%)
:arrow_down:

rafthttp/peer.go
90.22% <0%> (-3.01%)
:arrow_down:

clientv3/concurrency/election.go
80% <0%> (-2.41%)
:arrow_down:

raft/node.go
90.17% <0%> (-2.24%)
:arrow_down:

pkg/adt/interval_tree.go
89.63% <0%> (-2.14%)
:arrow_down:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 750dc7f...5158542. Read the comment docs.

@brancz can you do a double check on the rules? thanks!

@xiang90 Removed election query change, instead changed the title.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@e6d2667). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #8093   +/-   ##
=========================================
  Coverage          ?   76.64%
=========================================
  Files             ?      342
  Lines             ?    26568
  Branches          ?        0
=========================================
  Hits              ?    20363
  Misses            ?     4753
  Partials          ?     1452

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6d2667...02e7257. Read the comment docs.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@e6d2667). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #8093   +/-   ##
=========================================
  Coverage          ?   76.71%
=========================================
  Files             ?      342
  Lines             ?    26568
  Branches          ?        0
=========================================
  Hits              ?    20381
  Misses            ?     4739
  Partials          ?     1448

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6d2667...02e7257. Read the comment docs.

@brancz thanks for the review.

@xiang90 always happy to help :slightly_smiling_face:

lgtm

@xiang90 PTAL.

@gyuho Hmm... After second thought, I prefer an aggregated view for both RPC and failed RPC. As discussed with @heyitsanthony before, the dashboard should be as clear and easy as possible. If users want to see detailed information, they can create a perf/debugging page themselves.

All fixed.

lgtm


Assertion failure in esp_bignum (IDFGH-11507)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.0
Espressif SoC revision.
ESP32-D0WDR2-V3-V3 (revision v3.0)
Operating System used.
Windows
How did you build your project?
Command line with idf.py
If you are using Windows, please specify command line type.
CMD
Development Kit.
Custom Board
Power Supply used.
Battery
What is the expected behavior?
No panic
What is the actual behavior?
We encountered panic described in https://github.com/espressif/esp-idf/issues/11850, so backported https://github.com/espressif/esp-idf/commit/8e25eb1e08dcb5a346248451ea0b75d64295ddfe. We start to observe new assertion failure that seems related.
system panic with
assert failed: mpi_mult_mpi_failover_mod_mult esp_bignum.c:662 ((z_words >= mpi_words(Z)) && (z_words - mpi_words(Z) <= (size_t)1))

Steps to reproduce.
This could happen when esp32 communicates with an https server, sending a request every 0.5s. And/or when a https client sends request to esp32 every 3-5s
Debug Logs.
assert failed: mpi_mult_mpi_failover_mod_mult esp_bignum.c:662 ((z_words >= mpi_words(Z)) && (z_words - mpi_words(Z) <= (size_t)1))

Backtrace: 0x40082726:0x3ffcadd0 0x4008cef5:0x3ffcadf0 0x40096bba:0x3ffcae10 0x40133e83:0x3ffcaf30 0x40133f61:0x3ffcaf60 0x401287b4:0x3ffcaf90 0x40128972:0x3ffcb030 0x40133a41:0x3ffcb060 0x40133b62:0x3ffcb090 0x40131dd7:0x3ffcb120 0x40132ca0:0x3ffcb150 0x40132d0a:0x3ffcb180 0x4012fe63:0x3ffcb1b0 0x4012fb45:0x3ffcb1e0 0x40125400:0x3ffcb200 0x40123567:0x3ffcb2a0 0x401235c0:0x3ffcb2c0 0x4011a711:0x3ffcb2e0 0x4011a1e1:0x3ffcb300 0x4011a2db:0x3ffcb350 0x40171b8b:0x3ffcb380 0x40197275:0x3ffcb3b0 0x4011eb9e:0x3ffcb3d0 0x4011efc5:0x3ffcb3f00x40104be7:0x3ffcb420 0x401053d3:0x3ffcb450 0x401060e4:0x3ffcb6e0 0x400d9ade:0x3ffcb7a0

0x40082726: panic_abort at /esp-idf/components/esp_system/panic.c:412

0x4008cef5: esp_system_abort at /esp-idf/components/esp_system/esp_system.c:135

0x40096bba: __assert_func at /esp-idf/components/newlib/assert.c:78

0x40133e83: mpi_mult_mpi_failover_mod_mult at /esp-idf/components/mbedtls/port/esp_bignum.c:662
 (inlined by) mbedtls_mpi_mul_mpi at /esp-idf/components/mbedtls/port/esp_bignum.c:522

0x40133f61: mbedtls_mpi_mul_int at /esp-idf/components/mbedtls/port/esp_bignum.c:557

0x401287b4: mbedtls_mpi_div_mpi at /esp-idf/components/mbedtls/mbedtls/library/bignum.c:1690

0x40128972: mbedtls_mpi_mod_mpi at /esp-idf/components/mbedtls/mbedtls/library/bignum.c:1760

0x40133a41: calculate_rinv at /esp-idf/components/mbedtls/port/esp_bignum.c:200 (discriminator 2)

0x40133b62: esp_mpi_exp_mod at /esp-idf/components/mbedtls/port/esp_bignum.c:388
 (inlined by) mbedtls_mpi_exp_mod at /esp-idf/components/mbedtls/port/esp_bignum.c:460

0x40131dd7: mbedtls_rsa_public at /esp-idf/components/mbedtls/mbedtls/library/rsa.c:763

0x40132ca0: mbedtls_rsa_rsassa_pkcs1_v15_verify at /esp-idf/components/mbedtls/mbedtls/library/rsa.c:2154

0x40132d0a: mbedtls_rsa_pkcs1_verify at /esp-idf/components/mbedtls/mbedtls/library/rsa.c:2206

0x4012fe63: rsa_verify_wrap at /esp-idf/components/mbedtls/mbedtls/library/pk_wrap.c:247

0x4012fb45: mbedtls_pk_verify_restartable at /esp-idf/components/mbedtls/mbedtls/library/pk.c:441

0x40125400: ssl_parse_server_key_exchange at /esp-idf/components/mbedtls/mbedtls/library/ssl_tls12_client.c:2512
 (inlined by) mbedtls_ssl_handshake_client_step at /esp-idf/components/mbedtls/mbedtls/library/ssl_tls12_client.c:3663

0x40123567: mbedtls_ssl_handshake_step at /esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:3027
 (inlined by) mbedtls_ssl_handshake_step at /esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:2984

0x401235c0: mbedtls_ssl_handshake at /esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:3092
 (inlined by) mbedtls_ssl_handshake at /esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:3068

0x4011a711: esp_mbedtls_handshake at /esp-idf/components/esp-tls/esp_tls_mbedtls.c:192

0x4011a1e1: esp_tls_handshake at /esp-idf/components/esp-tls/esp_tls.c:83 (discriminator 15)
 (inlined by) esp_tls_low_level_conn at /esp-idf/components/esp-tls/esp_tls.c:437 (discriminator 15)

0x4011a2db: esp_tls_conn_new_sync at /esp-idf/components/esp-tls/esp_tls.c:464

0x40171b8b: ssl_connect at /esp-idf/components/tcp_transport/transport_ssl.c:109

0x40197275: esp_transport_connect at /esp-idf/components/tcp_transport/transport.c:123

0x4011eb9e: esp_http_client_connect at /esp-idf/components/esp_http_client/esp_http_client.c:1283

0x4011efc5: esp_http_client_perform at /esp-idf/components/esp_http_client/esp_http_client.c:1130

0x40104be7: cad::HttpClient::perform() at /tmp/build/80754af9/source/managed_components/rest_client/include/HttpClient.hpp:110
 (inlined by) cad::RestClient::odu_send(char const*, esp_http_client_method_t) at /tmp/build/80754af9/source/managed_components/rest_client/RestClient.cpp:266

0x401053d3: cad::RestClient::odu_hello() at /tmp/build/80754af9/source/managed_components/rest_client/include/RestClient.hpp:73
 (inlined by) cad::RestClient::odu_sync(unsigned long) at /tmp/build/80754af9/source/managed_components/rest_client/RestClient.cpp:189

0x401060e4: cad::RestClient::odu_sync(unsigned long) at /tmp/build/80754af9/source/managed_components/rest_client/RestClient.cpp:174
 (inlined by) cad::RestClient::run() at /tmp/build/80754af9/source/managed_components/rest_client/RestClient.cpp:616

0x400d9ade: cad::TaskComponent::run_task(void*) at /tmp/build/80754af9/source/managed_components/component_base/include/TaskComponent.hpp:70

assert failed: mpi_mult_mpi_failover_mod_mult esp_bignum.c:662 ((z_words >= mpi_words(Z)) && (z_words - mpi_words(Z) <= (size_t)1))

Backtrace: 0x40082726:0x3ffd9a80 0x4008cef5:0x3ffd9aa0 0x40096bba:0x3ffd9ac0 0x4012c97b:0x3ffd9be0 0x4012ca59:0x3ffd9c10 0x401212f4:0x3ffd9c40 0x401214b2:0x3ffd9ce0 0x4012aa60:0x3ffd9d10 0x4012b480:0x3ffd9de0 0x4012b529:0x3ffd9e00 0x40128979:0x3ffd9e30 0x40128805:0x3ffd9e60 0x4011f1f1:0x3ffd9e90 0x4011c0cb:0x3ffd9fc0 0x4011c108:0x3ffd9fe0 0x40113a4f:0x3ffda000 0x40111e35:0x3ffda030 0x400f16cc:0x3ffda050 0x40119c52:0x3ffda080 0x40118771:0x3ffda0e0
0x40082726: panic_abort at C:/Users/emily.xing/esp/esp-idf/components/esp_system/panic.c:412

0x4008cef5: esp_system_abort at C:/Users/emily.xing/esp/esp-idf/components/esp_system/esp_system.c:135

0x40096bba: __assert_func at C:/Users/emily.xing/esp/esp-idf/components/newlib/assert.c:78

0x4012c97b: mpi_mult_mpi_failover_mod_mult at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/port/esp_bignum.c:662
 (inlined by) mbedtls_mpi_mul_mpi at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/port/esp_bignum.c:522

0x4012ca59: mbedtls_mpi_mul_int at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/port/esp_bignum.c:557

0x401212f4: mbedtls_mpi_div_mpi at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/bignum.c:1690

0x401214b2: mbedtls_mpi_mod_mpi at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/bignum.c:1760

0x4012aa60: rsa_prepare_blinding at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/rsa.c:818
 (inlined by) mbedtls_rsa_private at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/rsa.c:964

0x4012b480: mbedtls_rsa_rsassa_pkcs1_v15_sign at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/rsa.c:1883
 (inlined by) mbedtls_rsa_rsassa_pkcs1_v15_sign at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/rsa.c:1838

0x4012b529: mbedtls_rsa_pkcs1_sign at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/rsa.c:1927

0x40128979: rsa_sign_wrap at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/pk_wrap.c:357

0x40128805: mbedtls_pk_sign_restartable at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/pk.c:627
 (inlined by) mbedtls_pk_sign at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/pk.c:641

0x4011f1f1: ssl_prepare_server_key_exchange at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls12_server.c:3177
 (inlined by) ssl_write_server_key_exchange at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls12_server.c:3251
 (inlined by) mbedtls_ssl_handshake_server_step at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls12_server.c:4358

0x4011c0cb: mbedtls_ssl_handshake_step at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:3044
 (inlined by) mbedtls_ssl_handshake_step at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:2984

0x4011c108: mbedtls_ssl_handshake at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:3092
 (inlined by) mbedtls_ssl_handshake at C:/Users/emily.xing/esp/esp-idf/components/mbedtls/mbedtls/library/ssl_tls.c:3068

0x40113a4f: esp_mbedtls_server_session_create at C:/Users/emily.xing/esp/esp-idf/components/esp-tls/esp_tls_mbedtls.c:796

0x40111e35: esp_tls_server_session_create at C:/Users/emily.xing/esp/esp-idf/components/esp-tls/esp_tls.c:611

0x400f16cc: httpd_ssl_open at C:/Users/emily.xing/esp/esp-idf/components/esp_https_server/src/https_server.c:126 (discriminator 13)

0x40119c52: httpd_sess_new at C:/Users/emily.xing/esp/esp-idf/components/esp_http_server/src/httpd_sess.c:212

0x40118771: httpd_accept_conn at C:/Users/emily.xing/esp/esp-idf/components/esp_http_server/src/httpd_main.c:65
 (inlined by) httpd_server at C:/Users/emily.xing/esp/esp-idf/components/esp_http_server/src/httpd_main.c:256
 (inlined by) httpd_thread at C:/Users/emily.xing/esp/esp-idf/components/esp_http_server/src/httpd_main.c:272

More Information.
could disable CONFIG_MBEDTLS_HARDWARE_MPI as a workaround

Hello, @xing35!
Thank you for the issue report.
Sorry for the late response - could you please attach the following -

The sdkconfig file for your example
Minimal example that produces the issue
Could you also test this minimal example with the v5.0.4 tag?

Also, try the example after cherry-picking the commit https://github.com/espressif/esp-idf/commit/521dd35f2f974746a3e2b1561371e8b4b8e33f09 on the v5.0.4 tag (this commit just updates the assert but does not change its function)

Also, try the example after cherry-picking the commit 521dd35 on the v5.0.4 tag (this commit just updates the assert but does not change its function)

That does not apply to v5.0.4.
The file components/mbedtls/port/bignum/esp_bignum.c does not exist in v5.0 and v5.1 branches.

Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates.


Building packages on OS X silently fails during ncp call

Using electron-packager v5.1.0 on OS X 10.10.5 and Node v4.1.1
➜ electron-packager . MyApp --overwrite --platform=darwin --arch=x64 --version=0.33.6

Packaging app for platform darwin x64 using electron v0.33.6

No errors, it just exits about 10 seconds after displaying the message "Packaging app for platform darwin x64 using electron v0.33.6" and there are no files written to my CWD.
I realise that this is very sparse on info but I'm not sure what other details I can give. When I last ran this command about three weeks ago it worked fine.

Can you bisect which released version of electron-packager is the last one that works for you?

I believe I was using 5.1.0 when it worked before I tried moving back to an earlier version but it still didn't work. Could this be something related to something in my app code base (I see symlink issues are mentioned a lot, I have symlinked modules).

Could you try specifying an --out directory?
Otherwise, I think we're going to need reproduction steps on a minimal Electron app.

Still isn't working with --out.
I'm doing a bit of debugging, seems to be failing here the callback function (cb) is never fired.

Hmm. Very odd. I noticed that Travis CI only tests under Node 0.12. Let me add some more Node versions, and see if that surfaces anything.

Possibly related to https://github.com/AvianFlu/ncp/issues/98?

Possibly, but that's a really vague bug report with no follow-up.

Yup.
FYI, tried a couple of other node versions, I'm getting the same issue with Node-v0.12.2 and iojs-v2.0.1.

The actual copying of files works fine by the way, all the files are copied correctly, it's just that the callback isn't fired.
I put the callback in a setTimeout just to get it working and the app packaged itself correctly and runs as expected.
  var ncpCallbackFired = false
  ncp(opts.dir, appPath, {filter: userIgnoreFilter(opts), dereference: true}, function(err) {
    ncpCallbackFired = true;
    cb(err);
  });
  // callback above isn't being fired by ncp for some reason,
  // so call it synthetically after 20 secs if it hasn't been fired.
  // https://github.com/maxogden/electron-packager/issues/150
  setTimeout(function() {
    if (!ncpCallbackFired) {
      cb(null);
    }
  }, 20000);

Just want to mention that a number people have found ncp inconsistent and buggy and it isn't being actively maintained anymore: https://github.com/AvianFlu/ncp/issues/100
I also noticed that the fs-extra module dropped ncp because of bugs surrounding non-firing/double-firing of callbacks.

we should switch to using https://www.npmjs.com/package/tar-fs: tarfs.pack().pipe(tarfs.extract()), its like JSON.parse(JSON.stringify(obj)) but for filesystems :)

We should open a new issue for that.

Looks like this was intended to be closed in #150.


Docker build failing at step 30

After cloning I ran docker build -t 3dseg .. Build is failing at step 30 with following error.
Step 30/36 : RUN python3 setup.py bdist_wheel
 ---> Running in a575717e6e68
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/spconv
copying spconv/test_utils.py -> build/lib.linux-x86_64-3.6/spconv
copying spconv/__init__.py -> build/lib.linux-x86_64-3.6/spconv
copying spconv/modules.py -> build/lib.linux-x86_64-3.6/spconv
copying spconv/pool.py -> build/lib.linux-x86_64-3.6/spconv
copying spconv/functional.py -> build/lib.linux-x86_64-3.6/spconv
copying spconv/conv.py -> build/lib.linux-x86_64-3.6/spconv
copying spconv/ops.py -> build/lib.linux-x86_64-3.6/spconv
creating build/lib.linux-x86_64-3.6/spconv/utils
copying spconv/utils/__init__.py -> build/lib.linux-x86_64-3.6/spconv/utils
running build_ext
-- The CXX compiler identification is GNU 7.5.0
-- The CUDA compiler identification is NVIDIA 10.2.89
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "10.2")
-- Caffe2: CUDA detected: 10.2
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 10.2
-- Found CUDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so
-- Found cuDNN: v8.2.4  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Automatic GPU detection failed. Building for common architectures.
-- Autodetected CUDA architecture(s): 3.5;5.0;5.2;6.0;6.1;7.0;7.0+PTX;7.5;7.5+PTX
-- Added CUDA NVCC flags for: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_70,code=compute_70;-gencode;arch=compute_75,code=compute_75
-- Found Torch: /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
CMake Error at CMakeLists.txt:28 (add_subdirectory):
  The source directory

    /spconv/third_party/pybind11

  does not contain a CMakeLists.txt file.

-- Configuring incomplete, errors occurred!
See also "/spconv/build/temp.linux-x86_64-3.6/CMakeFiles/CMakeOutput.log".
See also "/spconv/build/temp.linux-x86_64-3.6/CMakeFiles/CMakeError.log".
/spconv/build/lib.linux-x86_64-3.6
Release
Traceback (most recent call last):
  File "setup.py", line 103, in <module>
    zip_safe=False,
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 204, in run
    self.run_command('build')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 39, in run
    self.build_extension(ext)
  File "setup.py", line 71, in build_extension
    subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/spconv', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/spconv/build/lib.linux-x86_64-3.6/spconv', '-DCMAKE_PREFIX_PATH=/usr/local/lib/python3.6/dist-packages/torch', '-DPYBIND11_PYTHON_VERSION=3.6', '-DSPCONV_BuildTests=OFF', '-DCMAKE_CUDA_FLAGS="--expt-relaxed-constexpr"', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_CUDA_COMPILER:FILEPATH=/usr/local/cuda/bin/nvcc']' returned non-zero exit status 1.
The command '/bin/sh -c python3 setup.py bdist_wheel' returned a non-zero code: 1

Here is partial output of nvidia-smi
$ nvidia-smi
Tue Nov 30 16:51:26 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.91.03    Driver Version: 460.91.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+

Please help me build the image and run inference on it. Thanks

It appears /spconv/third_party/pybind11 directory is empty. Executing git clone https://github.com/pybind/pybind11.git at /spconv/third_party directory and letting remaining stuff run leads to successful build. Please fix the Dockerfile accordingly if this is the problem.


[AutoPR datafactory/resource-manager] Change type name and add timeout property

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/6717

This PR has been merged into https://github.com/Azure/azure-sdk-for-go/pull/5341


IE11 cannot spread non-iterable object

I get this error using Traceur 0.0.88 in IE11, but not in Chrome.
let mySet = new Set();
mySet.add('foo').add('bar);
let myArr = [...mySet];
Why doesn't Traceur think mySet is iterable in IE11?

Duplicate of #1810


Update dependencies & CI

Update dependencies (incl. JHipster so it should fix Windows issues) and added CI matrix so it tests across Windows, Mac & Linux and Node.js versions.

Not sure why, but the tests seems flaky after the JHipster update.
It pass locally on Windows/Mac, but sometimes get stuck on timeout.


Set default Node.js version to 6.9.0 LTS

It's me again, Ionut-Cristian. The Node.js 6.9.0 LTS was just released.

This change is

Thanks again! :-)

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


uv doesn't respect pyproject.toml dependencies

I'm trying out a very basic example, and it simply ignores what is specified in the PEP 621 dependencies section of a pyproject.toml. I get that there should be a lock file requirements.txt in the final environment, but in a lockless scenario it has to install everything, right?
uv version 0.1.1
user@hostname ~/w/e/o/example> uv pip install pyproject.toml
Resolved 9 packages in 1.04s
Installed 9 packages in 4ms
 + attrs==23.2.0
 + jsonschema==4.21.1
 + jsonschema-specifications==2023.12.1
 + pyproject-toml==0.0.10
 + referencing==0.33.0
 + rpds-py==0.18.0
 + setuptools==69.1.0
 + toml==0.10.2
 + wheel==0.42.0

user@hostname ~/w/e/o/example> cat pyproject.toml
[project]
name = "example"
version = "0.0.1"
description = ""
authors = [
    {name = "Mikhail Krassavin", email =<EMAIL_ADDRESS>]
dependencies = [
    "litestar[standard]>=2.0.1",
]
requires-python = ">=3.11"
readme = "README.md"
license = {text = "MIT"}

user@hostname ~/w/e/o/example> uv pip freeze
attrs==23.2.0
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
pyproject-toml==0.0.10
referencing==0.33.0
rpds-py==0.18.0
setuptools==69.1.0
toml==0.10.2
wheel==0.42.0

Oh, sorry, there's actually a -r flag, which I forgot.


[Docs] Building a RustyVault Documentation Website

In order to make RustyVault better, we decided to build a separate documentation site for RustyVault alone to display and maintain the documentation. The documentation site will be maintained in a separate /docs directory and can be run independently.
TODO

[ ] build site
[ ] CI pipeline
[ ] Page Preview
[ ] i18n
[ ] logo design

I am going to add some badges in the README.md of RustyVault

A crates.io + version badge is added.


[v3] Separator for columns that support arrays

Hierarchy

Global config
Export config
Column config

Currently this is solved on a global level by overriding ArrayFormatter, which I think is enough for now.


Update biom import examples

Hi! I have updated the examples on biom import. Checking for related things to do.

Im wondering if this works also in system that does not have biomformat installed (especially if this works in Bioc system; sometimes they work in GHA but not in Bioc). biomformat is not dependency of mia, it is just dependency of loadFromBiom function

I see! Then should it work if I put this back?
#' if(requireNamespace("biomformat")) {
#'   library(biomformat)

I see! Then should it work if I put this back?
#' if(requireNamespace("biomformat")) {
#'   library(biomformat)

Well, in practice they could be removed, I think... https://pecanproject.github.io/pecan-documentation/master/declaring-dependencies-depends-suggests-imports.html


[stable/metallb] Fix helm warnings

What this PR does / why we need it:
Fixed helm warnings about configInline not being a table (when set).
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

Mentioning @danderson as he is listed as a maintainer.
This fixes CI/CD pipelines that break due to helm warnings.

/ok-to-test
/assign


Update azure-active-directory-b2c-deployment-plans.md

Replaced "." with ":".

@Freenity : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Cannot find blasfeo headers because CMake does not include them despite finding blasfeo

Dear developers,
I downloaded hpipm and blasfeo from their respective git repositories here on Github and built/installed them on my machine. When compiling the Control Toolbox with the optimal control addon, I encounter the problem that blasfeo_target.h cannot be found. The problem could be solved when changing line 66 in ct_optcon/CMakeLists.txt from:
list(APPEND HPIPM_LIBS ${hpipm_LIBRARIES} ${blasfeo_LIBRARIES})
to:
list(APPEND HPIPM_LIBS hpipm blasfeo)
CMake does not require you to add ${<name>_LIBRARIES} and ${<name>_INCLUDE_DIRS} to link against dependencies or include their headers. Just link the target created by find_package(), this will also include all headers. I believe that this way is less error prone and should be preferred.
Edit: I did not encounter any problem with this, but int ct_core/CMakeLists.txt you added the Python libraries with ${Python_LIBRARIES} and ${PYTHON_INCLUDE_DIRS}. You can simply link against Python::Python according to this page in the CMake manual.

I agree with this. The _LIBRARIES and _INCLUDE_DIRS variables are legacy methods of handling deps. I'll see if I can reproduce the error with the latest hpipm and blasfeo and come up with a solution.

re: python, we want to stay on a lower minimum cmake version to support legacy operating systems which is why we cannot link against the Python:Python target. Once we raise our min cmake version we will be sure to update that as well.

@schulz0r if you could test out the linked PR and confirm that it resolves the issue for you that would be great. Thx for reporting.

I just want to tell that I have seen your PR and I will check it out this week.

I checked it out and it worked for me. Thanks a lot!

Glad to hear!


Merge video & audio file metadata when merging files

Checklist

[X] I'm reporting a feature request
[X] I've looked through the README
[X] I've verified that I'm running yt-dlp version 2<IP_ADDRESS>. (update instructions)
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Description
Currently when separate video & audio files are merged, any pieces of non-matching metadata are lost; the only thing that remains is the truly common data (like length). So while the video has info on frame width/height, and audio has info on the bit rate, the output contains neither of these fields.
It'd be great if these could be stitched together so metadata isn't lost. Admittedly I'm not familiar with the merging process at all, so I'm not sure how big/small of an undertaking this would be (assuming it's possible, of course).
Below is a picture of the 3 file properties side-by-side: on the left is separate video, middle is separate audio, and the right is the merged file. As you can see, some data has been lost in the merge.

This "metadata" is not written by yt-dlp/ffmpeg, but parsed by windows explorer. Since you have not provided logs, I can only assume that the files are being merged to mkv. Windows does not support parsing much metadata out of mkv files. If you use mediainfo/ffprobe to look at the metadata, you can see the height width etc

Sorry, your template didn't ask for logs so I didn't think to provide them. It only asked for a description. If that's an expectation you might want to update the template?

Here's the verbose logs from the process
PS C:\Users\user\Downloads> ./yt-dlp.exe -f bestvideo*+bestaudio/best https://www.youtube.com/watch?v=5OWOQF3dWi0 -k --verbose
[debug] Command-line config: ['-f', 'bestvideo*+bestaudio/best', 'https://www.youtube.com/watch?v=5OWOQF3dWi0', '-k', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg 2022-03-31-git-e301a24fa1-full_build-www.gyan.dev (setts), ffprobe 2022-03-31-git-e301a24fa1-full_build-www.gyan.dev
[debug] Optional libraries: brotli, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=5OWOQF3dWi0
[youtube] 5OWOQF3dWi0: Downloading webpage
[youtube] 5OWOQF3dWi0: Downloading android player API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[info] 5OWOQF3dWi0: Downloading 1 format(s): 248+251
[debug] Invoking downloader on "https://rr1---sn-vgqsrne6.googlevideo.com/videoplayback?expire=1648810634&ei=KoZGYsCuFIySiwTM9p2oDA&ip=<IP_ADDRESS>&id=o-AFcyuvlTAh9y746MgSo6uP3xj5AlV8IZZpeLWhtoG5-G&itag=248&source=youtube&requiressl=yes&mh=de&mm=31%2C29&mn=sn-vgqsrne6%2Csn-vgqskns7&ms=au%2Crdu&mv=m&mvi=1&pl=19&initcwndbps=2005000&vprv=1&mime=video%2Fwebm&gir=yes&clen=22188315&dur=67.239&lmt=1509677466227961&mt=1648788781&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAK6bPUbHSbENg-gwo4HpVGj_4LLYL84uJ98D7kToCOwtAiEAwxwJTdRg2rmedPxOqDTo6InWYh7bzdhKVb8myplyO48%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgPGWVsUVS9fu7ubx6joxV7NPwt838gSV09XzZKoB349cCIBmz9AyHI5DrWzzlGQRG75JFptqLuDhOJ_dnxJrychpV"
[download] Destination: A very happy wolf [5OWOQF3dWi0].f248.webm
[download] 100% of 21.16MiB in 00:15
[debug] Invoking downloader on "https://rr1---sn-vgqsrne6.googlevideo.com/videoplayback?expire=1648810634&ei=KoZGYsCuFIySiwTM9p2oDA&ip=<IP_ADDRESS>&id=o-AFcyuvlTAh9y746MgSo6uP3xj5AlV8IZZpeLWhtoG5-G&itag=251&source=youtube&requiressl=yes&mh=de&mm=31%2C29&mn=sn-vgqsrne6%2Csn-vgqskns7&ms=au%2Crdu&mv=m&mvi=1&pl=19&initcwndbps=2005000&vprv=1&mime=audio%2Fwebm&gir=yes&clen=774088&dur=67.261&lmt=1509676899282283&mt=1648788781&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgIwwC6t_RLgEoocS7Ng-DSSEReThXV0-4NscTtEEo_QACIQCKTIruwZK-Yn7GlTAVv0QUMfY7plu3HmZKxWZnelKfdw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgPGWVsUVS9fu7ubx6joxV7NPwt838gSV09XzZKoB349cCIBmz9AyHI5DrWzzlGQRG75JFptqLuDhOJ_dnxJrychpV"
[download] Destination: A very happy wolf [5OWOQF3dWi0].f251.webm
[download] 100% of 755.95KiB in 00:00
[Merger] Merging formats into "A very happy wolf [5OWOQF3dWi0].webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:A very happy wolf [5OWOQF3dWi0].f248.webm" -i "file:A very happy wolf [5OWOQF3dWi0].f251.webm" -c copy -map "0:v:0" -map "1:a:0" -movflags "+faststart" "file:A very happy wolf [5OWOQF3dWi0].temp.webm"

The files are being downloaded as webm and that's also what they're automatically being merged to. It's a consistent format throughout (separate video, separate audio, and merged are all webm), but you're right, I can see the missing metadata on the output file with ffprobe, though Windows Explorer seems to be missing it.
I'll certainly go digging for a solution, but have any tips/references (off the top of your head of course, I don't want to waste your time) for fixing that (if it's possible)?
Really appreciate the info!

"Feature requests" don't generally need logs. And I understand why you thought to file this as one. But if you are correct that this is yt-dlp's fault (I still think it's windows' fault, but I'll try to check what is going on), this would be actually a bug report

Ah, gotcha! Sorry about that mix up then.
It totally could be Windows, and it looks like you're right, as the metadata seems to be indistinguishable (to my eye anyway) compared from the source(s) to the merged output:

video alone

PS C:\Users\Ryun\Downloads> ffprobe ./wolf_video.webm

...

Input #0, matroska,webm, from './wolf_video.webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:01:07.24, start: 0.000000, bitrate: 2639 kb/s
  Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709/unknown/unknown), 1920x1080, SAR 1:1 DAR 16:9, 28.97 fps, 28.97 tbr, 1k tbn (default)

merged output

PS C:\Users\Ryun\Downloads> ffprobe ./wolf_merged.webm

...

Input #0, matroska,webm, from './wolf_merged.webm':
  Metadata:
    ENCODER         : Lavf59.20.101
  Duration: 00:01:07.26, start: -0.007000, bitrate: 2731 kb/s
  Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709/unknown/unknown), 1920x1080, SAR 1:1 DAR 16:9, 28.97 fps, 28.97 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:01:07.239000000
  Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:01:07.261000000

If we're playing "Spot the Difference", I guess the only thing is that the merged output has the extra MetaData: DURATION? Would be odd if that were the issue.

Ah, gotcha! Sorry about that mix up then.

No worries. While the issue templates are generally helpful, it's not always obvious where something should go. It's definitely not your fault


[Formatting Bug]: custom directives support?

Version
latest
Template before formatting
@hasfield('list')

    @fields('list')

@sub('item')

    @endfields

@endfield

Template after formatting
Error formatting document.

Expected Behaviour
hi,
is it possible to somehow increase the number of supported directives?
it seems that the https://github.com/log1x/sage-directives composer package is not supported.
it says an error when formatting: Error formatting document.
thanks for the help!
Relevant log output
No response

Currently supported custom directive format is @~~~ with @end~~~. (~~~ must matched with start and end directive)
If sage-directive's @endfield was @endhasfield, it works as expected.
@hasfield('list')

    @fields('list')

@sub('item')

    @endfields

@endhasfield


Improve the number formatting with input validation  and customizable separators

Hey @UTSAVS26 I made the following changes in number formatting and test number formatting files:

Added input validation to check if the input is an integer or a float, raising a ValueError if not
Introduced parameters thousands_sep and decimal_sep, allowing customization of thousands and decimal separators.
Added logic to replace the default thousands and decimal separators with user-defined ones, ensuring that the function can format numbers according to different localization standards and included the additional examples

And for test file:

Added a test case to check if the function correctly formats numbers with custom thousands and decimal separators.
Added a test to ensure that the function raises a ValueError when the input is invalid (not an integer or float).
Proper formatting of integers and floats and handling of negative numbers.

Issue number #29

Also @UTSAVS26 if everything is fine please provide the labels to my pr, thank you


Meaning of weights in cineast.json file

Dear devs,
What do the weights mean for each feature in the cineast.json file? How do the weights affect the results?
Furthermore, is it possible to query for multiple features with the AND operator? So for example, query all images with a certain color and a score between 4 and 5.
Best,
Ribin

The weights are used for the score fusion per category. When you have multiple feature modules, you need a way to aggregate these scores into one list. The weights enable you to change the influence of one feature's score with respect to another. The weights found in the config which are used for the low-level visual features are set empirically.

So how can I achieve the AND condition when querying with multiple categories? Currently in the UI, when I search for an image with a certain color and add the boolean condition for my score, the shown results are using the OR condition.

The categories are evaluated independently and their scores are fused in the front end. Since these are not sets but ordered lists, it's not exactly equivalent to a Boolean 'or' since elements which are present in both lists tend to have a higher combined score than those which are only present in one. If you explicitly want to limit the results of one category to those of another one, you can use query stages. This will cause cineast to consider only the results retrieved in the first stage to be part of the second and so on. Stages are based on query modalities and are present in the UI. You can simply push your second modality to a later stage to force it to only consider results from the first.

Ok, so can I do this directly on the webpage or do I need to configure something in the config file of vitrivr-ng? Currently, when I add an additional query container by clicking the green plus button, they are all on stage 0.

You can do this directly in the UI, there are little arrow buttons next to the stage to move the individual input types up and down.

I'm probably missing something, but which button are you referring to?

In this particular use case, you do not use the green + button, but instead click on the query-by-sketch button and use the
magnifying glass icon to move the query-by-sketch to the second stage

Ah, now I get it. Thanks a lot! Maybe this could be made a bit more intuitive?

Yes, a re-design of this entire thing is in the works, it's however in rather early stages and not a high priority activity, so there's no estimate on when it might be available yet.


Add HTML5.1 dialog element

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/dialog

I don't think it makes sense to add experimental tags before they get standardized. Use Html.node "dialog" for now, and we can revisit this when all browsers implement it the same way.

That's fair, I guess, but there are already some experimental tags inside this package.

Like which ones? I thought it was only HTML5 things. Which ones are you referring to?

menu and menuitem are the 2 standouts.
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/menu and
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/menuitem
but even details, summary, datalist, main and (in a subsequent PR, picture) are not super well-supported across browsers.
I guess we'd need to define "experimental". Do we mean at least Candidate Recommendation?

The following features are at-risk, and may be dropped during the CR period:

The use of text tracks exposing in-band metadata
label as a reassociatable element
datetime and datetime-local values of the type attribute
keygen element
inputmode attribute
autofill mechanism
menu element, and each enumerated value of the type attribute
menuitem element
contextmenu attribute
dialog element

https://www.w3.org/TR/html51/#status

@evancz What do you think?

Not sure what action to take based on this. I think it is a good policy to not add stuff that is not widely supported.
 That's possible, but I'd want to get community discussion on this, so I'd recommend talking about it on elm-dev.

Not exactly. What I'm saying is that we have to establish some kind of guideline as "experimental" and "supported" are ambiguous terms. If anything, I'm suggesting adding any element that has reached candidate recommendation status. Not removing anything.

Stuff like vendor-specific attributes like mozactionhint or autocorrect I could see being kept out until they're standardized, though there is a definite case to be made for progressive enhancement.

Gotcha, makes sense to have a rule like this. Need to hear from folks on elm-dev though. Need more perspectives to know when is "the right time"

Sounds good. Thanks Evan.

Here you go: https://groups.google.com/forum/#!topic/elm-dev/Fpx0ba19o1w


Improve onboarding experience for new contributors

Make videos
Change docs
Architecture how to make it as easy as possible to contribute

put me in coach


emacs25pre: 25.0.50-1b5630e -> 25.0.92

Things done:

[x] Tested using sandboxing (nix-build --option build-use-chroot true or nix.useChroot on NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

More
cc @chaoflow @lovek323 @peti @the-kenny @jwiegley @NicolasPetton

If this pull request is ok, I propose to prepare another one that refactors emacs 24 and 25 derivations so duplication of code is avoided.

@peti done

@DamienCassou thanks!


Run from binary

In all examples the code is always executed programatically using #run method. Example:
MyClass.run %w(hello)

This is useful for examples and testing, but in real world I would like to do:
./binary hello

I couldn't find any example or documentation explaining how can I "initialize" my CLI in order to my binary run it instead of calling it from inside my code with #run methods.
How can I achieve that?

Hi. You can use the standard global variable ARGV for getting your command-line argument strings. Try it!


TypeError: Cannot redefine property: $$el     at Function.defineProperty ()     at Object.e.consoleReference (chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:83265)     at chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:81185     at j (chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:78471)

Augury: 1.19.1
Date: Tue, 31 Jul 2018 03:29:51 GMT
OS: Win64
Description of issue:
-- When selecting a component it throws error :+1:
TypeError: Cannot redefine property: $$el
at Function.defineProperty ()
at Object.e.consoleReference (chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:83265)
at chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:81185
at j (chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:78471)
Angular version (required): 5
Steps to reproduce:

Open Angular app
Go to Developer too;
Go to Augury select any component.ts file throws error :+1:

ypeError: Cannot redefine property: $$el
at Function.defineProperty ()
at Object.e.consoleReference (chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:83265)
at chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:81185
at j (chrome-extension://elgalmkoelokbchhkhacckoklkejnhcd/build/backend.js:31:78471)

Hello, thanks for letting us know.
It looks like the$$el property in your window has been set elsewhere as non-configurable. Can you confirm this by running Object.getOwnPropertyDescriptor(window, '$$el') from your console?

In chrome when I ran this it returned undefined.

Please advise with the further course of action, in case I need to change anything.
Thanks !!

Now after clicking to a component which are present in my app it is throwing error but when I ran -
Object.getOwnPropertyDescriptor(window, '$$el')
it says {value:"There is no component selected", configurable:false, enumerable:false, writable:false}
kindly suggest.
Thanks !

one more observation it is specifically breaking for Angular 5 for Angular 6 it is working fine.

Same error here. Any ideas on a fix?

It happens on clean install of Firefox (MacOS) with only Augury extension being installed.

Can we fix this by:
diff --git a/src/backend/backend.ts b/src/backend/backend.ts
index a6817f3..8f2bebb 100644
--- a/src/backend/backend.ts
+++ b/src/backend/backend.ts
@@ -444,14 +444,16 @@ export const routerTree = (): Array<any> => {
 };

 export const consoleReference = (node: Node) => {
-  Object.defineProperty(window, selectedComponentPropertyKey, {
-    get: () => {
-      if (node) {
-        return ng.probe(node.nativeElement());
+  if (!Object.getOwnPropertyDescriptor(window, selectedComponentPropertyKey)) {
+    Object.defineProperty(window, selectedComponentPropertyKey, {
+      get: () => {
+        if (node) {
+          return ng.probe(node.nativeElement());
+        }
+        return null;
       }
-      return null;
-    }
-  });
+    });
+  }
 };

Is it going to be fixed in any upcoming release ??

Hello ,
Is it going to be fixed in any of future release.
Please advise.

Still having issue on this with latest Angular and Augury canary build.
Augury version (required): Augury Canary Build
Angular version (required): 8.2.6
Date: 2019-09-15T01:23:09.458Z
OS: MacOS 10.14.6
Description of issue:
Clicking on any component in the component tree will throw the following error. Reproduced in both Chrome and Firefox.

Steps to reproduce:

Start Angular Application: Port 4200
Open Browser Developer Tools > Augury
Click on an Angular component in Augury's Component Tree

I have just started to use this tool but unable to explore it further because of this issue. Is there any quick fix ?

I can also add that it's also happening in Angular 8, i get the same error.

And this is the result of Object.getOwnPropertyDescriptor(window, '$$el').

I'm using Angular 8(accurately 8.2.3) and got the same error.

I fixed this issue in my PR. You can wait till it's merged or build the extension yourself.


Loading appears to 'remember' directory, opening a file doesn't

If I have saved an Elan program in a directory, then go to load it, the directory where I saved it is the default in the Open dialog. If I load again then the same default is used.
When I open a file for reading, then open another file, the directory I've navigated to for the first file isn't the default for the second Open dialog, so I may have to navigate the filesystem again to find it. The default seems always to be My Documents.
I realise, having done some research, that all the file input/output is hedged around with security concerns and hence restrictions, but I'm worried that always showing a dialog when a student does a file open (whether for reading or writing) is going to be another 'problem' for teachers, and anything that we can do to alleviate this would help Elan take-up.
I did have a thought about an emulated filesystem where a single file could keep track of multiple files. You'd still have to select the file system file though, both for reading and writing that file.

We really have very little control over this process - it is managed by the browser.
I also think that school-level use of data files is generally quite simple - in a single session you are likely to be dealing with one file only, typically.

I'm curious as to why the Load 'remembers' and the file open doesn't.
Presumably it's much the same Javascript code.
All the best.
John
Sent from (one of):
one of my Raspberry Pi computers
the Android phone or tablet
the Windows desktop
one of the Windows laptops/notebooks/tablets
one of the Macbook Airs
the iPad Pro,
the LG TV, or
the new iMac (mid-2014)!
On Fri, 8 Nov 2024 at 11:36, Richard Pawson @.***>
wrote:

We really have very little control over this process - it is managed by
the browser.
I also think that school-level use of data files is generally quite simple

in a single session you are likely to be dealing with one file only,
typically.

—
Reply to this email directly, view it on GitHub
https://github.com/elan-language/LanguageAndIDE/issues/836#issuecomment-2464492359,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQQAB67KUQZSLNTUY3GBULZ7SO2VAVCNFSM6AAAAABRNIMQW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRUGQ4TEMZVHE
.
You are receiving this because you authored the thread.Message ID:
@.***>

Fixed as part of #841


calculating Morisita index isn't working

Hi! I am facing an issue while trying to calculate repertoire overlap, Morisita index in particular.
Not sure whether this is truly a typo/a bug and not myself doing something wrong.
Here it is, using the example repseq dataset from immunarch:

immunarch::repOverlap(immdata$data, 'morisita', 'nuc', .verbose = T)
|==
|   1%Error in tbl_dt(.x) : could not find function "tbl_dt"

sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default
locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252    LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                            LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
other attached packages:
[1] dtplyr_1.1.0 dplyr_1.0.7  tibble_3.1.3
loaded via a namespace (and not attached):
[1] modeltools_0.2-22   tidyselect_1.1.1    purrr_0.3.2         kernlab_0.9-27      reshape2_1.4.3      lattice_0.20-38
[7] colorspace_1.4-1    vctrs_0.3.8         generics_0.0.2      htmltools_<IP_ADDRESS>   stats4_3.5.3        yaml_2.2.0
[13] utf8_1.1.4          rlang_0.4.11        later_1.2.0         pillar_1.6.1        glue_1.4.2          DBI_1.0.0
[19] prabclus_2.2-7      treemap_2.4-2       RColorBrewer_1.1-2  fpc_2.1-11.1        factoextra_1.0.5    lifecycle_1.0.0
[25] plyr_1.8.4          robustbase_0.93-4   stringr_1.4.0       munsell_0.5.0       gtable_0.3.0        mvtnorm_1.0-10
[31] fastmap_1.1.0       httpuv_1.6.1        flexmix_2.3-15      class_7.3-15        fansi_0.4.0         DEoptimR_1.0-8
[37] trimcluster_0.1-2.1 Rcpp_1.0.1          xtable_1.8-3        readr_1.3.1         promises_<IP_ADDRESS>    scales_1.0.0
[43] diptest_0.75-7      mime_0.6            digest_0.6.27       ggplot2_3.1.0       hms_0.4.2           stringi_1.4.3
[49] shiny_1.6.0         ggrepel_0.8.0       grid_3.5.3          tools_3.5.3         magrittr_1.5        lazyeval_0.2.2
[55] cluster_2.0.7-1     crayon_1.3.4        pkgconfig_2.0.2     gridBase_0.4-7      MASS_7.3-51.1       ellipsis_0.3.2
[61] data.table_1.14.0   assertthat_0.2.1    rstudioapi_0.10     immunarch_0.3.0     R6_2.4.0            mclust_5.4.3
[67] igraph_1.2.4        nnet_7.3-12         compiler_3.5.3
Can it be a typo somewhere in the function? Or am I missing a package?
Thank you for help!

Hi @AnnaVyborova, you need to upgrade to a newer immunarch release to solve it :) most recommended is, generally, the latest (or even the developer version).


Lightgbm CPU learner error - lightgbm.basic.LightGBMError: Check failed: (best_split_info.left_count) > (0)

How you are using LightGBM?

LightGBM component:
Environment info

Operating System:
Ubuntu 20.04
CPU/GPU model:
Threadripper 1920x
C++ compiler version:
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
CMake version:
3.16.3
Java version:
Python version:
3.8.5
R version:
Other:

LightGBM version or commit hash:
Version: 3.0.0
Error message and / or logs

@guolinke Have just built it from the latest master branch, still fails. I'll try to separate a minimum reproducible example and create an issue then.
Originally posted by @diditforlulz273 in https://github.com/microsoft/LightGBM/issues/2793#issuecomment-716483788
[LightGBM] [Fatal] Check failed: (best_split_info.left_count) > (0) at /__w/1/s/python-package/compile/src/treelearner/serial_tree_learner.cpp, line 630 .
Traceback (most recent call last):
File "/home/seva/PycharmProjects/ECOM_demand/lgbm_mre.py", line 41, in
model = lgb.train(lgb_params, train_dat, valid_sets=test_dat, verbose_eval=20)
File "/home/seva/PycharmProjects/ECOM_demand/venv/lib/python3.8/site-packages/lightgbm/engine.py", line 252, in train
booster.update(fobj=fobj)
File "/home/seva/PycharmProjects/ECOM_demand/venv/lib/python3.8/site-packages/lightgbm/basic.py", line 2370, in update
_safe_call(_LIB.LGBM_BoosterUpdateOneIter(
File "/home/seva/PycharmProjects/ECOM_demand/venv/lib/python3.8/site-packages/lightgbm/basic.py", line 55, in _safe_call
raise LightGBMError(decode_string(_LIB.LGBM_GetLastError()))
lightgbm.basic.LightGBMError: Check failed: (best_split_info.left_count) > (0) at /__w/1/s/python-package/compile/src/treelearner/serial_tree_learner.cpp, line 630 .
Reproducible example(s)

Archive with code and pickled datasets, 4.5 kb total (lol really a MINIMUM reproducible example)
https://drive.google.com/file/d/1y04Z_11Ce-sETRZPfwjBEBY-aSOMBcHK/view?usp=sharing
Steps to reproduce
1.Run it on lgbm=03.0.0
2. ?????
3. NO PROFIT!
4. Run on lgbm==2.3.1
5.?????
6. PROFIT!

Thanks @diditforlulz273
It seems I met the error when loading data.
>>> train_x = pd.read_pickle('train_x.pkl')
>>> test_x = pd.read_pickle('test_x.pkl')
>>> train_y = np.loadtxt('train_y.txt')
>>> test_y = np.loadtxt('test_y.txt')
>>> train_x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\guoke\Anaconda3\lib\site-packages\pandas\core\frame.py", line 680, in __repr__
    self.to_string(
  File "C:\Users\guoke\Anaconda3\lib\site-packages\pandas\core\frame.py", line 801, in to_string
    formatter = fmt.DataFrameFormatter(
  File "C:\Users\guoke\Anaconda3\lib\site-packages\pandas\io\formats\format.py", line 593, in __init__
    self.max_rows_displayed = min(max_rows or len(self.frame), len(self.frame))
  File "C:\Users\guoke\Anaconda3\lib\site-packages\pandas\core\frame.py", line 1041, in __len__
    return len(self.index)
  File "C:\Users\guoke\Anaconda3\lib\site-packages\pandas\core\generic.py", line 5270, in __getattr__
    return object.__getattribute__(self, name)
  File "pandas\_libs\properties.pyx", line 63, in pandas._libs.properties.AxisProperty.__get__
  File "C:\Users\guoke\Anaconda3\lib\site-packages\pandas\core\generic.py", line 5270, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute '_data'

could you share the raw data or NumPy format data?

I guess the problem could be in different versions of Pandas and python's underlying pickle part.
I used Pandas 1.1.3 and Python 3.8.5
Sharing numpy arrays is nearly impossible - train dataframe has ~54 columns.
If versions matching won't help, I could share the data in .csv, although saving to .csv sometimes transforms ints like 7 to floats like 7.000000039 in an unpredictable manner, which can affect reproductability.

@diditforlulz273 I found the root cause you cannot set both min_data_in_leaf and min_child_weight to 0.
The leaf should be at least have one sample, otherwise, it cannot be a leaf.
We will fix the parameter checking, and throw errors before training.

@guolinke Well, indeed, this combination looks stupid, my bad. I used this tiny dataset in some of my integration tests, and, I guess, set all the possible constraints to 0 to make LightGBM build at least some trees and predict something reproducible. Anyway, it worked out well in 2.3.1 version :)


Cleanup: remove ts::ConstBuffer from ink_inet.

This turns out to have a lot of tentacles all over so I may have removed ConstBuffer from places that aren't strictly required. It is certainly not the case that all uses of ConstBuffer were removed, the remnants can go in other PRs. This is all driven by cleaning up ink_inet and then dealing with the fall out.
Also, ts::string_view constants!  AFAICT this is eleventy compliant.
I've got another branch I haven't pushed yet that has ts::TextView which is a reworking of StringView based on ts::string_view. Once that is it place we'll cull all of ConstBuffer and StringView. I'm still exploring how much pain it's worth to use string_view over TextView. You can see places here where noticeably more awkward, but we have no control over the string_view API.

[approve ci clang-format]

[approve ci autest]

[approve ci autest]

[approve ci icc]

[approve ci autest]

[approve ci autest]

[approve ci autest]

Does this mean we can / should get rid of MemView.h etc. ?

Not yet, but eventually. This is a first step, although one of the bigger ones to getting rid of ConstBuffer which will let us nuke TsBuffer.h.

[approve ci autest]

[approve ci autest]

[approve ci autest]


Portal

404 Page
Portal redesign (font tbd)
Deadline not show

Background issue still doesn't seem to be fixed

It'd also be nice if you could makes the widths a little bit smaller! (Refer to MongoDBs one – imo with not a lot of content, it makes more sense to squish it for UX purposes)

I have two postings but only one shows in the portal – is this an issue on my side?

Ye sadly I cant find any good ways to compare time. It is possible but it will be more string comparisons. I think for now we always make the deadline to be 23:59 for that day there is no need to compare the time.


Use pirates for interoperability with other require hooks

Per https://github.com/standard-things/esm/issues/119, this PR uses pirates to register the require hook instead of patching it in directly.
Advantage: better interoperability with other require hooks
Disadvantage: adds a (small) dependency

Ah, looks like pirates uses some es2015 features that aren't supported in node 0.10/0.12 🤷♂

Node 0.10 and 0.12 are no longer supported by Node and haven't been since 2016.

We should definitely have this. Seems like it would be pretty safe to drop Node 0.10 and 0.12 support as part of the v2 release I'm planning. Thanks @anandthakker! Apologies for how long it's taken us to look at this PR.


Turkish Language Support (tr.json)

After creating your pull request, tick these boxes if they are applicable to you.

[x] I have tested my changes against the review branch (the latest developmental version), and this pull request is targeting that branch as a base
[x] I have tested my changes on Python 3.5.3 or higher

Description
New Turkish language support for MusicBot.
Related issues (if applicable)
tr.zip

PR's target review, not master.


feat: effective path

:gift: adds an effectivePath function which allows use of --path value during command instantiation
:gift: sets the --builder flag default of the build command to the value of the function at --path (if it exist)

The effectivePath function evaluates arguments prior to Cobra/Viper instantiation such that the value of --path|-p|FUNC_PATH can be used during the creation of the commands.
The use-case used to illustrate its usefulness is to set the default of the build command's --builder flag to the function at --path (if it exists).  This feature will be used in the Global Config: Function Contxt, where it will be part of the config precidence flow:  Static Defaults -> Global Config -> Function State -> Flag Defaults -> Environment Variables -> Flag Values -> API invocation.
/kind enhancement
adds an `effectivePath` function which allows use of `--path` value during command instantiation
sets the `--builder` flag default of the `build` command to the value of the function at `--path` (if it exist)

Related: #901

Thanks very much for the review @lance

Also needed a rebase and update to use NewTestClient in the tests.  Should be GTG now.

/override "On Cluster RT Test (1.18.x, ubuntu-latest)"


Made it works with python2.7

super() in python 3 is equivalent to super(klass, self) in python 2.7

I have also removed some non ascii char from the docstrings.

Thanks for the PR, I'll take time to look into it over the week-end and come back to you.
Tangentially, if we are going to support multiple versions, I think we should automate testing with all supported versions (via Travis/Tox).
Also please note that although I have been testing it with a project of mine currently in development, I don’t think the library is tested enough to be used in production yet. I’ll get in touch with the minio developers soon to ask their opinion on the library.

Sorry for the delay. I'm going to merge.
I contacted the minio devs and it turns out there is another library that does the same as this one, by @shapoglyk. We discussed it and we are going to join the efforts on a common repo. So I'm going to work on making it production-ready.
As for testing, I looked into dockerized travis builds and it seems it would be pretty easy to test on both python 2.7 and the latest.

Thank you for creating this backend and working toward making it production ready. I tried the other library  but I had no luck with it. I have not spent a ton of time debuguing the issue but for some reason uploading a Media file was entering into an "infinite" loop.
Having a good test suite run on multiple python and django versions would be amazing.


topic (DOCS-CODE-016): [readme] apply styling guidelines Part 2

continuation for #12

@ferantivero -- Just one more comment to resolve from #12 -- do you want to include it in this PR so we only have to merge once?


Suport for php-parser v5

Otherwise... this package can't be used in apps with PHPUnit 11.
If this package didn't go totally over my head, I'd contribute a PR 😆

To help set some expectations, this will likely be a new major version/bump on this one due to some under-the-hood changes 👍

Available with major version 4. This version only supports php-parser v5. For installations that need to use php-parser v4, please continue to version 3 for those.

Thanks John! 🎉


[Regression | OMPT]  missing ompt_scope_end for target enter/exit data

Description:
The OMPT interface offers different callbacks to track the execution of an OpenMP program. For regions on target devices the callback ompt_callback_target is certainly the most important.
The target region callback is defined like this:
typedef void (*ompt_callback_target_t) (
    ompt_target_t kind,
    ompt_scope_endpoint_t endpoint,
    int device_num,
    ompt_data_t *task_data,
    ompt_id_t target_id,
    const void *codeptr_ra
);

A target region can have different kinds, which describe what type of target directive was used. This issue will focus on two kinds, which were working fine in aomp 16.x and ROCm 5.4.x. We are looking at #pragma omp target enter data and #pragma omp target exit data.
Target region callbacks will get called twice with a different endpoint. When entering a target region, an endpoint of ompt_scope_begin is used. When exiting the region, the endpoint ompt_scope_end is chosen. There's also ompt_scope_beginend but I'm not sure when this is ever used.
Previously, we would get both endpoints.
With this issue present, we are unable to track any target enter data or target exit data regions at all as we require ompt_scope_end to know when the regions did finish! For Score-P, we would run into error messages because we cannot match the locations and wouldn't be able to produce any tracing data.
Reproduce the issue:
We can take the following source code as an example:
#include <omp.h>
#include <stdio.h>
#include "callbacks.h"

int main( void )
{
    int M[10];
#pragma omp target enter data map(to: M[:10])
#pragma omp target
    {
#pragma omp teams distribute parallel for simd
        for(int i = 0; i < 10; ++i)
        {
            M[i] = i;
        }
    }
#pragma omp target exit data map(from: M[:10])
    return 0;
}

callbacks.h is the header file used in the veccopy-ompt-target-tracing example
With ROCm 5.4.x, everything seems fine. However, target_id=6 is missing ompt_scope_end in the device tracing interface:
> ./error_missing_scope_end | grep endpoint
Callback Target: target_id=1 kind=2 endpoint=1 device_num=0 code=0x55da7977084c
Callback Target: target_id=1 kind=2 endpoint=2 device_num=0 code=0x55da7977084c
Callback Target: target_id=4 kind=1 endpoint=1 device_num=0 code=0x55da797708cf
	Record Target: kind=2 endpoint=1 device=0 task_id=0 target_id=1 codeptr=0x55da7977084c
	Record Target: kind=2 endpoint=2 device=0 task_id=0 target_id=1 codeptr=0x55da7977084c
Callback Target: target_id=4 kind=1 endpoint=2 device_num=0 code=0x55da797708cf
Callback Target: target_id=6 kind=3 endpoint=1 device_num=0 code=0x55da7977093a
	Record Target: kind=1 endpoint=1 device=0 task_id=0 target_id=4 codeptr=0x55da797708cf
Callback Target: target_id=6 kind=3 endpoint=2 device_num=0 code=0x55da7977093a
	Record Target: kind=1 endpoint=2 device=0 task_id=0 target_id=4 codeptr=0x55da797708cf
	Record Target: kind=3 endpoint=1 device=0 task_id=0 target_id=6 codeptr=0x55da7977093a

With aomp 17.0, the situation is worse. If we check for the endpoints, all target enter data and target exit data are missing ompt_scope_end
> ./error_missing_scope_end | grep endpoint
Callback Target: target_id=1 kind=2 endpoint=1 device_num=0 code=0x55803e209be0
Callback Target: target_id=4 kind=1 endpoint=1 device_num=0 code=0x55803e209c6c
	Record Target: kind=2 endpoint=1 device=0 task_id=0 target_id=1 codeptr=0x55803e209be0
Callback Target: target_id=4 kind=1 endpoint=2 device_num=0 code=0x55803e209c6c
Callback Target: target_id=6 kind=2 endpoint=1 device_num=0 code=0x55803e209cd8
	Record Target: kind=1 endpoint=1 device=0 task_id=0 target_id=4 codeptr=0x55803e209c6c
	Record Target: kind=1 endpoint=2 device=0 task_id=0 target_id=4 codeptr=0x55803e209c6c
	Record Target: kind=3 endpoint=1 device=0 task_id=0 target_id=6 codeptr=0x55803e209cd8

The bug is now also present in ROCm 5.6, which is quite unfortunate, since it is a pain / near impossible for us to deal with this bug at configure time. I will check the most recent aomp release..

Agreed. This needs to be fixed.

This is what we get on trunk with the just-upstreamed implementation.
Callback Target: target_id=0 kind=2 endpoint=1 device_num=-1 code=0x55e4a69245a0
Callback Init: device_num=0 type=gfx90a device=0x55e4a8dad440 lookup=0x7f38ae500c50 doc=(nil)
Callback Load: device_num:0 module_id:0 filename:(null) host_adddr:0x55e4a6920378 device_addr:(nil) bytes:7688
Callback DataOp: target_id=0 host_op_id=1 optype=1 src=0x7ffe0428ee40 src_device_num=1 dest=(nil) dest_device_num=0 bytes=40 code=0x7f38ae3d957f
Callback DataOp: target_id=0 host_op_id=2 optype=2 src=0x7ffe0428ee40 src_device_num=1 dest=0x7f37a1000000 dest_device_num=0 bytes=40 code=0x7f38ae3cfc80
Callback Target: target_id=0 kind=2 endpoint=2 device_num=-1 code=0x55e4a69245a0
Callback Target: target_id=3 kind=1 endpoint=1 device_num=0 code=0x55e4a692462c
Callback Submit: target_id=3 host_op_id=4 req_num_teams=0
Callback Target: target_id=3 kind=1 endpoint=2 device_num=0 code=0x55e4a692462c
Callback Target: target_id=5 kind=3 endpoint=1 device_num=-1 code=0x55e4a6924698
Callback DataOp: target_id=5 host_op_id=6 optype=3 src=0x7f37a1000000 src_device_num=0 dest=0x7ffe0428ee40 dest_device_num=1 bytes=40 code=0x7f38ae3da146
Callback DataOp: target_id=5 host_op_id=7 optype=4 src=0x7f37a1000000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7f38ae3de904
Callback Target: target_id=5 kind=3 endpoint=2 device_num=-1 code=0x55e4a6924698
Callback Fini: device_num=0
The target callback endpoints are correct. However, some device numbers are not set.

First things first: I can reproduce the output from above.
I was very interested in the ordering issue, as Callback Init is printed after(!) Callback Target.
Second, I found the root cause of this in LLVM/main and am further looking into this.
In ASO there's a fix for this very behavior in place, which was commented as temporary.
FYI: when __tgt_target_data_begin_mapper is called the DeviceId is in fact -1.
That's because the CTOR will be called in targetDataMapper but we create the RAII object just before that call.
Unfortunately with the "wrong" information, but there's no other available at this particular point.
The corresponding CTOR call will also trigger OMPT callbacks for Init and Load; then the correct info would be available.

I've also noticed that the device initialize callback is called right after the first target callback. I don't know if it's still compliant with the OpenMP specification, since it only notes that the initialize callback will be dispatched before any OpenMP construct is executed on the target device.
I've changed how we handle this in Score-P and this shouldn't be an issue for us.
It's good to know, that the callbacks are now all dispatched again. Getting the correct device numbers as well would be great, since we require that knowledge for proper profiling and tracing, at least during the begin-event.

This took a bit more effort than anticipated, sorry for the delay.
I used your reproducer and added #pragma omp target update from(M) just before the main's return statement to have an additional example of the target update callback case.
This is the output after applying my fix:
Callback Init: device_num=0 type=gfx90a device=0x992310 lookup=0x7fe1059c0a90 doc=(nil)
Callback Load: device_num:0 module_id:0 filename:(null) host_adddr:0x200378 device_addr:(nil) bytes:7952
Callback Target EMI: kind=2 endpoint=1 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000001) code=0x204a80
  Callback DataOp EMI: endpoint=1 optype=1 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000001) host_op_id=0x7fe1043ed380 (0x8000000000000002) src=0x7ffe7e356a30 src_device_num=8 dest=(nil) dest_device_num=0 bytes=40 code=0x7fe1058c974b
  Callback DataOp EMI: endpoint=2 optype=1 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000001) host_op_id=0x7fe1043ed380 (0x8000000000000002) src=0x7ffe7e356a30 src_device_num=8 dest=(nil) dest_device_num=0 bytes=40 code=0x7fe1058c974b
  Callback DataOp EMI: endpoint=1 optype=2 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000001) host_op_id=0x7fe1043ed380 (0x8000000000000003) src=0x7ffe7e356a30 src_device_num=8 dest=0x7fe0fd000000 dest_device_num=0 bytes=40 code=0x7fe1058b9af4
  Callback DataOp EMI: endpoint=2 optype=2 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000001) host_op_id=0x7fe1043ed380 (0x8000000000000003) src=0x7ffe7e356a30 src_device_num=8 dest=0x7fe0fd000000 dest_device_num=0 bytes=40 code=0x7fe1058b9af4
Callback Target EMI: kind=2 endpoint=2 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000001) code=0x204a80
Callback Target EMI: kind=1 endpoint=1 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000004) code=0x204b12
  Callback Submit: target_id=9223372036854775812 (0x8000000000000004) host_op_id=1 req_num_teams=0
Callback Target EMI: kind=1 endpoint=2 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000004) code=0x204b12
Callback Target EMI: kind=3 endpoint=1 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000005) code=0x204b74
  Callback DataOp EMI: endpoint=1 optype=3 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000005) host_op_id=0x7fe1043ed380 (0x8000000000000006) src=0x7fe0fd000000 src_device_num=0 dest=0x7ffe7e356a30 dest_device_num=8 bytes=40 code=0x7fe1058cadba
  Callback DataOp EMI: endpoint=2 optype=3 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000005) host_op_id=0x7fe1043ed380 (0x8000000000000006) src=0x7fe0fd000000 src_device_num=0 dest=0x7ffe7e356a30 dest_device_num=8 bytes=40 code=0x7fe1058cadba
  Callback DataOp EMI: endpoint=1 optype=4 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000005) host_op_id=0x7fe1043ed380 (0x8000000000000007) src=0x7fe0fd000000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7fe1058d1fb9
  Callback DataOp EMI: endpoint=2 optype=4 target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000005) host_op_id=0x7fe1043ed380 (0x8000000000000007) src=0x7fe0fd000000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7fe1058d1fb9
Callback Target EMI: kind=3 endpoint=2 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000005) code=0x204b74
Callback Target EMI: kind=4 endpoint=1 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000008) code=0x204bb4
Callback Target EMI: kind=4 endpoint=2 device_num=0 task_data=0x4ed480 (0x0) target_task_data=0x50d658 (0x0) target_data=0x7fe1043ed368 (0x8000000000000008) code=0x204bb4
Callback Fini: device_num=0

Previously kind=2/3/4 did not generate endpoint=2 callbacks, now they are called the same number of times (as expected).
@Thyre Please, if you find the time, check this output and if applicable confirm that this is the desired output.
I will then open a Phabricator review, so this is properly fixed upstream as well as on ASO, and report back here.
Just want to make sure everything's in order before I put it up for review.
Thank you!

Accidentally put up the mixed EMI (all EMI but Submit, for some testing) output but maybe it is interesting as well, so I will leave it in the previous comment.
Here the non-EMI output:
Callback Init: device_num=0 type=gfx90a device=0x1c95310 lookup=0x7f33d0a90a90 doc=(nil)
Callback Load: device_num:0 module_id:0 filename:(null) host_adddr:0x200378 device_addr:(nil) bytes:7952
Callback Target: target_id=0 kind=2 endpoint=1 device_num=0 code=0x2044d0
  Callback DataOp: target_id=0 host_op_id=1 optype=1 src=0x7ffe350389c0 src_device_num=8 dest=(nil) dest_device_num=0 bytes=40 code=0x7f33d099974b
  Callback DataOp: target_id=0 host_op_id=2 optype=2 src=0x7ffe350389c0 src_device_num=8 dest=0x7f33c3c00000 dest_device_num=0 bytes=40 code=0x7f33d0989af4
Callback Target: target_id=0 kind=2 endpoint=2 device_num=0 code=0x2044d0
Callback Target: target_id=3 kind=1 endpoint=1 device_num=0 code=0x204562
  Callback Submit: target_id=3 host_op_id=4 req_num_teams=0
Callback Target: target_id=3 kind=1 endpoint=2 device_num=0 code=0x204562
Callback Target: target_id=5 kind=3 endpoint=1 device_num=0 code=0x2045c4
  Callback DataOp: target_id=5 host_op_id=6 optype=3 src=0x7f33c3c00000 src_device_num=0 dest=0x7ffe350389c0 dest_device_num=8 bytes=40 code=0x7f33d099adba
  Callback DataOp: target_id=5 host_op_id=7 optype=4 src=0x7f33c3c00000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7f33d09a1fb9
Callback Target: target_id=5 kind=3 endpoint=2 device_num=0 code=0x2045c4
Callback Target: target_id=8 kind=4 endpoint=1 device_num=0 code=0x204604
Callback Target: target_id=8 kind=4 endpoint=2 device_num=0 code=0x204604
Callback Fini: device_num=0

Looks good at first glance. Thanks for looking and fixing the issue.

Sure & thanks for the quick response!
Phab review opened:
https://reviews.llvm.org/D157605

@Thyre If it's not too much to ask for, could you please open a corresponding issue for LLVM?
Maybe this would help to bump the priority / visibility -- also, then I could add sth. like fixes issue X.
I've just revisited the Phabricator review, since Dhruva has posted some feedback.

@Thyre If it's not too much to ask for, could you please open a corresponding issue for LLVM? Maybe this would help to bump the priority / visibility -- also, then I could add sth. like fixes issue X. I've just revisited the Phabricator review, since Dhruva has posted some feedback.

Sure, I'll work on that after my next meeting. I'll also link to this issue to preserve the discussion.

Thank you very much!

I'm currently not able to reproduce the issue on my notebook with LLVM Trunk from today. [...]

Interesting. The issue is still reproducible for me.
If you find the time, please look at the testcase I added in the Phabricator review:
https://reviews.llvm.org/D157605#change-tdrKICtoWCc5
This will even segfault with EMI(*) callbacks when using current LLVM (without the patch) and also print Callback Target EMI before Callback Init -- taht is, at least during my testing.
(*) This is due to target_task_data=(nil)

I'm currently not able to reproduce the issue on my notebook with LLVM Trunk from today. [...]

Interesting. The issue is still reproducible for me.
If you find the time, please look at the testcase I added in the Phabricator review: https://reviews.llvm.org/D157605#change-tdrKICtoWCc5
This will even segfault with EMI(*) callbacks when using current LLVM (without the patch) and also print Callback Target EMI before Callback Init -- taht is, at least during my testing.
(*) This is due to target_task_data=(nil)

I found out that I was using the wrong test case. The issue should be up on the LLVM repository in a moment.

I'm currently not able to reproduce the issue on my notebook with LLVM Trunk from today. [...]

Interesting. The issue is still reproducible for me.
If you find the time, please take a look at the testcase I added in the Phabricator review: https://reviews.llvm.org/D157605#change-tdrKICtoWCc5
This will even segfault with EMI(*) callbacks when using current LLVM (without the patch) and also print Callback Target EMI before Callback Init -- that is, at least during my testing.
(*) This is due to target_task_data=(nil)

Yeah, I can reproduce the same segmentation fault. This means that the issue should be completely present in the current trunk. For further discussion, we should move to the LLVM issue.

fix added to ROCm 6.0 RC2

fix added to ROCm 6.0 RC2

The issue doesn't seem to be fixed in version:
AMD clang version 17.0.0 (https://github.com/RadeonOpenCompute/llvm-project roc-6.0.0 23445 285893de04054931eb5e1ba6d215a01b4ade3d84)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/apps/software/ROCm/6.0.0/llvm/bin
Configuration file: /opt/apps/software/ROCm/6.0.0/llvm/bin/clang.cfg

Here, the following code results in these callbacks:
#include <assert.h>
#include <omp.h>
#include <stdio.h>

#include "callbacks.h"
#include "register_emi.h"

void foo()
{

}

int main(void)
{
    int a[10];
#pragma omp target enter data map(to: a[:10])
    foo();
#pragma omp target exit data map(from: a[:10])
    foo();
    return 0;
}

$ amdclang -fopenmp --offload-arch=gfx1101 test.c
$ ./a.out
Callback Init: device_num=0 type=gfx1101 device=0xce4710 lookup=0x7f5a088c03d0 doc=(nil)
Callback Target EMI: kind=2 endpoint=1 device_num=0 task_data=0xb7ea00 (0x0) target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000001) code=0x2035ca
  Callback DataOp EMI: endpoint=1 optype=1 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000001) host_op_id=0x7f5a11a2b760 (0x8000000000000002) src=0x7ffd4ca1c680 src_device_num=1 dest=(nil) dest_device_num=0 bytes=40 code=0x7f5a11b5266b
  Callback DataOp EMI: endpoint=2 optype=1 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000001) host_op_id=0x7f5a11a2b760 (0x8000000000000002) src=0x7ffd4ca1c680 src_device_num=1 dest=0x7f5905a20000 dest_device_num=0 bytes=40 code=0x7f5a11b5266b
  Callback DataOp EMI: endpoint=1 optype=2 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000001) host_op_id=0x7f5a11a2b760 (0x8000000000000003) src=0x7ffd4ca1c680 src_device_num=1 dest=0x7f5905a20000 dest_device_num=0 bytes=40 code=0x7f5a11b534fb
  Callback DataOp EMI: endpoint=2 optype=2 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000001) host_op_id=0x7f5a11a2b760 (0x8000000000000003) src=0x7ffd4ca1c680 src_device_num=1 dest=0x7f5905a20000 dest_device_num=0 bytes=40 code=0x7f5a11b534fb
Callback Target EMI: kind=2 endpoint=1 device_num=0 task_data=0xb7ea00 (0x0) target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000004) code=0x203637
  Callback DataOp EMI: endpoint=1 optype=3 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000004) host_op_id=0x7f5a11a2b760 (0x8000000000000005) src=0x7f5905a20000 src_device_num=0 dest=0x7ffd4ca1c680 dest_device_num=1 bytes=40 code=0x7f5a11b54923
  Callback DataOp EMI: endpoint=2 optype=3 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000004) host_op_id=0x7f5a11a2b760 (0x8000000000000005) src=0x7f5905a20000 src_device_num=0 dest=0x7ffd4ca1c680 dest_device_num=1 bytes=40 code=0x7f5a11b54923
  Callback DataOp EMI: endpoint=1 optype=4 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000004) host_op_id=0x7f5a11a2b760 (0x8000000000000006) src=0x7f5905a20000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7f5a11b54140
  Callback DataOp EMI: endpoint=2 optype=4 target_task_data=0xb80398 (0x0) target_data=0x7f5a11a2b7b8 (0x8000000000000004) host_op_id=0x7f5a11a2b760 (0x8000000000000006) src=0x7f5905a20000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7f5a11b54140
Callback Fini: device_num=0

Here, we get two enter events for target_enter_data (which is wrong since we have only one) and no exit event for both. Clang 17.0.4 works perfectly fine. Is this fixed in a newer RC I just haven't installed?

the version you referenced appears to be ROCm 6.0 RC1.
Our commit logs show it is im ROCm 6.0 RC2

the version you referenced appears to be ROCm 6.0 RC1. Our commit logs show it is im ROCm 6.0 RC2

Yup, that seemed to be the main issue:
$ amdclang -fopenmp --offload-arch=gfx1101 test.c
$ ./a.out
Callback Init: device_num=0 type=gfx1101 device=0x5d4850 lookup=0x7f3330ec0b50 doc=(nil)
Callback Target EMI: kind=2 endpoint=1 device_num=0 task_data=0x46ea00 (0x0) target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000001) code=0x20339a
  Callback DataOp EMI: endpoint=1 optype=1 target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000001) host_op_id=0x7f333a041760 (0x8000000000000002) src=0x7fff4b4ce190 src_device_num=1 dest=(nil) dest_device_num=0 bytes=40 code=0x7f333a0d060b
  Callback DataOp EMI: endpoint=2 optype=1 target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000001) host_op_id=0x7f333a041760 (0x8000000000000002) src=0x7fff4b4ce190 src_device_num=1 dest=0x7f322e020000 dest_device_num=0 bytes=40 code=0x7f333a0d060b
  Callback DataOp EMI: endpoint=1 optype=2 target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000001) host_op_id=0x7f333a041760 (0x8000000000000003) src=0x7fff4b4ce190 src_device_num=1 dest=0x7f322e020000 dest_device_num=0 bytes=40 code=0x7f333a0d149b
  Callback DataOp EMI: endpoint=2 optype=2 target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000001) host_op_id=0x7f333a041760 (0x8000000000000003) src=0x7fff4b4ce190 src_device_num=1 dest=0x7f322e020000 dest_device_num=0 bytes=40 code=0x7f333a0d149b
Callback Target EMI: kind=2 endpoint=2 device_num=0 task_data=0x46ea00 (0x0) target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000001) code=0x20339a
Callback Target EMI: kind=3 endpoint=1 device_num=0 task_data=0x46ea00 (0x0) target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000004) code=0x2033dd
  Callback DataOp EMI: endpoint=1 optype=3 target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000004) host_op_id=0x7f333a041760 (0x8000000000000005) src=0x7f322e020000 src_device_num=0 dest=0x7fff4b4ce190 dest_device_num=1 bytes=40 code=0x7f333a0d28c3
  Callback DataOp EMI: endpoint=2 optype=3 target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000004) host_op_id=0x7f333a041760 (0x8000000000000005) src=0x7f322e020000 src_device_num=0 dest=0x7fff4b4ce190 dest_device_num=1 bytes=40 code=0x7f333a0d28c3
Callback Target EMI: kind=3 endpoint=2 device_num=0 task_data=0x46ea00 (0x0) target_task_data=0x4777d8 (0x0) target_data=0x7f333a0417b8 (0x8000000000000004) code=0x2033dd
  Callback DataOp EMI: endpoint=1 optype=4 target_task_data=(nil) (0x0) target_data=0x7f333a0417b8 (0x0) host_op_id=0x7f333a041760 (0x8000000000000006) src=0x7f322e020000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7f333a0d20e0
  Callback DataOp EMI: endpoint=2 optype=4 target_task_data=(nil) (0x0) target_data=0x7f333a0417b8 (0x0) host_op_id=0x7f333a041760 (0x8000000000000006) src=0x7f322e020000 src_device_num=0 dest=(nil) dest_device_num=-1 bytes=0 code=0x7f333a0d20e0
Callback Fini: device_num=0

We now see the deletions after the second target kernel has finished, but that makes sense since we do not use map(delete: a[:10]).


Added Url validation for on bookmark save

Fix: #124

Thanks for tackling this, but I don't understan how this change would address the issue. The problem in #124  was that it seems to be possible to save a Bookmark instance that has no URL or an empty URL. In that case I would assume that the validation for the url field is not set up correctly? Or maybe the validation is not triggered as part of the import?

self.full_clean() is called on .save() to validate fields.
But as discussed here - https://stackoverflow.com/questions/4441539/why-doesnt-djangos-model-save-call-full-clean ,
in our case, url validation is not happending.
In this commit, I'm overriding .save() to validate.
One other way will be to setup pre-save signal like this -
@receiver(pre_save)
def pre_save_handler(sender, instance, *args, **kwargs):
    instance.full_clean()

Thanks for the explanation. To summarize, full_clean needs to be called manually when not using Model Forms. I think in that case I would prefer to do just that - call full_clean manually before saving a bookmark in the import. Changing the way the save method works seems like a common approach as well, but I think it has downsides:

it's different from how Django models work out of the box
according to the SO thread linked above it can have side effects
it seems overblown for this problem, which only seems to occur in the import ATM (I tested the REST API, which runs validation automatically as well)

As such I would suggest a different solution:

call full_clean before saving in the bookmark in this line: https://github.com/sissbruecker/linkding/blob/master/bookmarks/services/importer.py#L60
add a test case that verifies that bookmarks with empty URLs are not imported - it seems there are no test cases for the importer itself yet 😬. However there are some integration tests for the import in https://github.com/sissbruecker/linkding/blob/master/bookmarks/tests/test_settings_import_view.py. I think for now it would be OK to add another test for bookmark validation to those.

Feel free to leave this as is. With the solution outlined, I should be able to make the change myself rather quickly.

Closing in favor of https://github.com/sissbruecker/linkding/pull/149