Perform initial indexing and synchronization

Motivation
This PR performs the initial indexing of files and maintains the index synchronized upon modifications.
Implementation
There were a few issues in the code structure I had to address.

Because the Ruby LSP runs in a separate process from the application, we need some sort of DSL file to allow people to configure the index. Otherwise, there's nowhere to put the configuration. I created the concept of .indexrc for this, which is a Ruby DSL file that can be used to configure the index
The API for indexing at the RubyIndexer level was a bit weird, since we need to control the index from the outside. I moved index_all and index_single inside the Index class itself, which feels like a much nicer interface
Having people configure files_to_index is a bit too much. Instead, we can just provide folks with an array of excluded gems and excluded file patterns, which we will use to tune the files indexed. This makes the index easier to configure

Finally, after addressing those, I started:

Pushing a progress notification before starting indexing
Indexing all files on the initialized notification and then closing the progress report
Dynamically registering the editor to listen to Ruby file modifications
Handling the modifications to synchronize the index

Everything is marked as experimental for now, so that we can test it first.
Automated Tests
Added a few tests.
Manual Tests
For now, the only behaviour change is seeing the progress notification while indexing is happening.

Switch to this branch
Reload the VS Code window
Verify that a status shows up in the status bar saying that we're indexing files

@st0012 The reason I think it's useful to use a Ruby DSL is because you can access the $LOAD_PATH or even bundler methods if necessary, which provides more flexibility for users.
I'm curious to hear other opinions. Do people prefer YAML?

Okay, I've migrated the index configuration file to use YAML. Let me know if you've got more feedback.

Let's see if that becomes necessary. For now, I'll keep the existing error classes. We're not rescuing them anywhere just yet.


openssl 3.0.7 unable to establish connection to TLS 1.0 and 1.1 Apache service with openssl 1.1.1l

I'm in the process of migrating from OpenSSL 1.1.1 to 3.0.7 and we have tests that connect to an Apache service where four port numbers (9010, 9011, 9012, and 9013) are configured to only accept a single protocol, TLS 1.0 for 9010, TLS 1.1 for 9011, TLS 1.2 for 9012, and TLS 1.3 for 9013.  The Apache service is 2.4.53 with mod_ssl;OpenSSL 1.1.1l
A sample of the Apache conf file would look like this (with suitable VirtualHost sections updating SSLProtocol and SSLCipherSuite for each of the other three ports, 9011, 9012, and 9013):
<VirtualHost *:9010>
	SSLEngine On
	SSLCertificateFile conf/ssl/ses-server.crt
	SSLCertificateKeyFile conf/ssl/ses-server.key

	SSLHonorCipherOrder		on
	SSLCompression          off
	SSLSessionTickets       off

	SSLProtocol             TLSv1
	SSLCipherSuite			ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA
</VirtualHost>

With the client running OpenSSL version 1.1.1, connections are successful.
However, after migrating the client to OpenSSL version 3.0.7, connections to 9010 (TLS 1.0) and 9011 (TLS 1.1) both fail.
I ran the openssl s_client -trace :9010 and found that the failure appears to be occurring during key exchange with an internal error 603A0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl\statem\statem_clnt.c:2254:
The error is identical when connecting to :9011.
connections to :9012 and :9013 both succeed.
Connections to all four ports are successful when connecting with openssl s_client from 1.1.1
CONNECTED(000001B4)
Sent Record
Header:
  Version = TLS 1.0 (0x301)
  Content Type = Handshake (22)
  Length = 323
    ClientHello, Length=319
      client_version=0x303 (TLS 1.2)
      Random:
        gmt_unix_time=0x83F8E4D5
        random_bytes (len=28): 0566F2EBB8DD10A3943BAA6AE9A077FC49C7C4C2CC91CCCE7DE30D31
      session_id (len=32): 55C19CEF92ECBD39F5B8C13E132333E898D4590E771990BAC1A3F52F91DDE3DE
      cipher_suites (len=62)
        {0x13, 0x02} TLS_AES_256_GCM_SHA384
        {0x13, 0x03} TLS_CHACHA20_POLY1305_SHA256
        {0x13, 0x01} TLS_AES_128_GCM_SHA256
        {0xC0, 0x2C} TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        {0xC0, 0x30} TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        {0x00, 0x9F} TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
        {0xCC, 0xA9} TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
        {0xCC, 0xA8} TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
        {0xCC, 0xAA} TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256
        {0xC0, 0x2B} TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        {0xC0, 0x2F} TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        {0x00, 0x9E} TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
        {0xC0, 0x24} TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
        {0xC0, 0x28} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
        {0x00, 0x6B} TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
        {0xC0, 0x23} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        {0xC0, 0x27} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        {0x00, 0x67} TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
        {0xC0, 0x0A} TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
        {0xC0, 0x14} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
        {0x00, 0x39} TLS_DHE_RSA_WITH_AES_256_CBC_SHA
        {0xC0, 0x09} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
        {0xC0, 0x13} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0x33} TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0x9D} TLS_RSA_WITH_AES_256_GCM_SHA384
        {0x00, 0x9C} TLS_RSA_WITH_AES_128_GCM_SHA256
        {0x00, 0x3D} TLS_RSA_WITH_AES_256_CBC_SHA256
        {0x00, 0x3C} TLS_RSA_WITH_AES_128_CBC_SHA256
        {0x00, 0x35} TLS_RSA_WITH_AES_256_CBC_SHA
        {0x00, 0x2F} TLS_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0xFF} TLS_EMPTY_RENEGOTIATION_INFO_SCSV
      compression_methods (len=1)
        No Compression (0x00)
      extensions, length = 184
        extension_type=server_name(0), length=27
          0000 - 00 19 00 00 16 76 6a 61-6d 62 61 32 32 2e 64   .....vjamba22.d
          000f - 73 6f 6e 65 2e 33 64 73-2e 63 6f 6d            sone.3ds.com
        extension_type=ec_point_formats(11), length=4
          uncompressed (0)
          ansiX962_compressed_prime (1)
          ansiX962_compressed_char2 (2)
        extension_type=supported_groups(10), length=22
          ecdh_x25519 (29)
          secp256r1 (P-256) (23)
          ecdh_x448 (30)
          secp521r1 (P-521) (25)
          secp384r1 (P-384) (24)
          ffdhe2048 (256)
          ffdhe3072 (257)
          ffdhe4096 (258)
          ffdhe6144 (259)
          ffdhe8192 (260)
        extension_type=session_ticket(35), length=0
        extension_type=encrypt_then_mac(22), length=0
        extension_type=extended_master_secret(23), length=0
        extension_type=signature_algorithms(13), length=42
          ecdsa_secp256r1_sha256 (0x0403)
          ecdsa_secp384r1_sha384 (0x0503)
          ecdsa_secp521r1_sha512 (0x0603)
          ed25519 (0x0807)
          ed448 (0x0808)
          rsa_pss_pss_sha256 (0x0809)
          rsa_pss_pss_sha384 (0x080a)
          rsa_pss_pss_sha512 (0x080b)
          rsa_pss_rsae_sha256 (0x0804)
          rsa_pss_rsae_sha384 (0x0805)
          rsa_pss_rsae_sha512 (0x0806)
          rsa_pkcs1_sha256 (0x0401)
          rsa_pkcs1_sha384 (0x0501)
          rsa_pkcs1_sha512 (0x0601)
          ecdsa_sha224 (0x0303)
          rsa_pkcs1_sha224 (0x0301)
          dsa_sha224 (0x0302)
          dsa_sha256 (0x0402)
          dsa_sha384 (0x0502)
          dsa_sha512 (0x0602)
        extension_type=supported_versions(43), length=9
          TLS 1.3 (772)
          TLS 1.2 (771)
          TLS 1.1 (770)
          TLS 1.0 (769)
        extension_type=psk_key_exchange_modes(45), length=2
          psk_dhe_ke (1)
        extension_type=key_share(51), length=38
            NamedGroup: ecdh_x25519 (29)
            key_exchange:  (len=32): 2658F5426D3B9C89D22671D94E9A9901AF7838881C0BDA160838F37949968541

Received Record
Header:
  Version = TLS 1.0 (0x301)
  Content Type = Handshake (22)
  Length = 101
    ServerHello, Length=97
      server_version=0x301 (TLS 1.0)
      Random:
        gmt_unix_time=0x89075B1B
        random_bytes (len=28): 1EE301C35ABEB842AA67851FF14CCDD127DF78C9CBFBA57E0FC60F76
      session_id (len=32): 00AB66D0D4A897C5141A4723F712CAEB0B654627DA240C25B3B4D52967DEB286
      cipher_suite {0xC0, 0x14} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
      compression_method: No Compression (0x00)
      extensions, length = 25
        extension_type=renegotiate(65281), length=1
            <EMPTY>
        extension_type=server_name(0), length=0
        extension_type=ec_point_formats(11), length=4
          uncompressed (0)
          ansiX962_compressed_prime (1)
          ansiX962_compressed_char2 (2)
        extension_type=encrypt_then_mac(22), length=0
        extension_type=extended_master_secret(23), length=0

SNIP  -- excluding the certificate verification which appears to be working fine

Received Record
Header:
  Version = TLS 1.0 (0x301)
  Content Type = Handshake (22)
  Length = 298
    ServerKeyExchange, Length=294
      KeyExchangeAlgorithm=ECDHE
        named_curve: ecdh_x25519 (29)
        point (len=32): 8CBCAEBEABDB43A75903B7742D564B97F2D6FEC5550E1EBC9508D6378E5D792C
      Signature (len=256): 4CDD3317DBE9ECE03E2E664D5D26B5CD62C6CEC9C790FADD26C37F35DE61D060F2AEC12258E409B9C929376A8804442CC02B8E33C831CFF64159ED2F8E9B8FAC8C295E39EE3A1903FF8C4FA524E258CD0F1786EF3777994360AB00342ACAD405267328166B7D3B54BF57E528A1C742711ABBD7C597CD733B51087284F3E592F3A95D05C2FC28FB19CE085213EC8788C0FD27FFA68EB0A429662160D0BC42DA5D78E90CB9E2AE210D9B2ACF8BCC9F69A03B9EDE0FBCC8B5C7BE1DBE2F518CE174B77F96DB0150D2A5B22B7BCF328C45693113EED12B35140FBE398C6A64FBAB62340303221C3BD183ADC75B6661A306A12C6B3DD53112F06F053A320338DB77B1

Sent Record
Header:
  Version = TLS 1.0 (0x301)
  Content Type = Alert (21)
  Length = 2
    Level=fatal(2), description=internal error(80)

603A0000:error:0A0C0103:SSL routines:tls_process_key_exchange:internal error:ssl\statem\statem_clnt.c:2254:

There does not appear to be an equivalent to s_client -trace in OpenSSL 1.1.1, so I used "-debug" instead and the connection was established setting up Cipher ECDHE-RSA-AES256-SHA:
New, TLSv1.0, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : ECDHE-RSA-AES256-SHA
    Session-ID: F599F3140954DCF5E6E46CCF9436F0EB082ACE6592DC8B33B3ACF69AEF2F65FC
    Session-ID-ctx:
    Master-Key: 7BF95BD915A8F580253F16101E728BA88FC5FC54BAC82C1ECABC72937426DC0443E341315FB84534E25EF254773AD229
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time:<PHONE_NUMBER>
    Timeout   : 7200 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
    Extended master secret: yes

Follow up:  I just found #15335 which indicates that TLS 1.0 and 1.1 were disabled in the default OpenSSL 3.0.  So, this becomes a documentation and error message request instead.
The information on the site was very opaque and the error message of "internal error" was not helpful.
One other note.
The description of #15335 states:
  BCP 195 has been formally updated by RFC 8996 to deprecate TLS 1.0/1.1 and DTLS 1.0

  Ref: https://www.rfc-editor.org/info/bcp195

  In RFC parlance "TLS 1.0/1.1/DTLS 1.0 MUST NOT be used. Negotiation of TLS 1.0/1.1/DTLS 1.0 from any version of
TLS MUST NOT be permitted"

However, that doesn't appear to be accurate or has been since changed.  Section 3.1 of bcp195 https://www.ietf.org/rfc/bcp/bcp195.html#section-3.1.1 states:
  Implementations SHOULD NOT negotiate TLS version 1.0 [[RFC2246](https://www.rfc-editor.org/rfc/rfc2246)];
      the only exception is when no higher version is available in the
      negotiation.

      Rationale: TLS 1.0 (published in 1999) does not support many
      modern, strong cipher suites.  In addition, TLS 1.0 lacks a per-
      record Initialization Vector (IV) for CBC-based cipher suites and
      does not warn against common padding errors.

   o  Implementations SHOULD NOT negotiate TLS version 1.1 [[RFC4346](https://www.rfc-editor.org/rfc/rfc4346)];
      the only exception is when no higher version is available in the
      negotiation.

      Rationale: TLS 1.1 (published in 2006) is a security improvement
      over TLS 1.0 but still does not support certain stronger cipher
      suites.

This looks the same as #19867. From one of my comments in that other issue:

The problem is because we are trying to negotiate TLSv1 we are falling back to the legacy TLSv1 signature algorithm which is based on SHA1. That doesn't have enough security strength for our default security level.

The error message from OpenSSL is particularly unhelpful here. That's been fixed in 97b8db1af2f7 so should be better from OpenSSL 3.0.8 when that's released.

Closing as duplicate of #19867


Fix docstring line length

This is a follow through on #136.
Fixed docstring line length of preprocessing, representation, and visualization modules to respect numpy docstring guide of 75 characters max length. TODOs in docstrings were left untouched.

That's soo cool Cedric! 🎉 , thank you very much!
I just added a comment regarding a best-practice we might want to follow (also, we should add this same best-practice under CONTRIBUTING for informing new contributors, would you like to do it?)

Hi, Jonathan. Done updating the docstrings to follow best practice.

That's soo cool Cedric!  , thank you very much!
I just added a comment regarding a best-practice we might want to follow (also, we should add this same best-practice under CONTRIBUTING for informing new contributors, would you like to do it?)

Yup, I'd like to do it! So far these are the things that I want to add:

Manually limit docstring line length to 75 characters due to blacks's limitation
Multiline code best practice as discussed above

I'll open a PR for this and maybe you can add more.

Amazing, thank you!! 🎉 🎉


Elasticsearch storage backend

Restucture the crawing and parsing to store structured data in Elasticsearch indices instead of in the file system.
Also store WARCs in S3 instead of raw files.
The new storage backends should be flexible enough to allow for re-parsing parts of the dataset without having to delete anything.
The second key requirement is to be able to scale up massively by only interacting with standard ES/S3 APIs instead of having to mount a shared file system on all nodes.

Closes #9

Fixes #6


Masking Error

There is an error when using fit_region and a  .npy file. The problem is that the mask is not binned! We need to simply add binning to the mask BEFORE passing it to the fitting function inside of fit_region.

Added binning for mask function


Differences in phasedAutodetect based on different Nextflow versions

Hi! Thanks for developing this plugin. We are using it in nf-core/phaseimpute.
Recently, we found a difference in behavior between different versions of Nextflow and nft-vcf when submitting a PR.

Nextflow Version: latest-everything (24.07.0-edge)
Successful Run Log

Nextflow Version: 24.04.4
Failed Run Log

The failure occurs in the following part of the log:
!  99     "VcfFile [chromosomes=[chr21, chr22], sampleCount=1, variantCount=1739, phased=true, phasedAutodetect=false]"

When using Nextflow version 24.04.4, there is a difference where phasedAutodetect is set to false instead of true.

Interesting. Does this issue still persist? phasedAutodetect only checks if the first line is phased or not, with the default value being true. Do you have a test file I could use to investigate this further?


CMS Enabled / Requests for MicrosoftAjax.de.js / Multilanguage

We noticed that our 9.4.1 DNN makes requests to MicrosoftAjax.de.js if the locale is set to German and CMS is enabled. While our page doesn't require Ajax and works with Java Script disabled, it adds a request that fails during page load, because localized versions of MicrosoftAjax.js do not exist afaik. It resulted in some flickering when switching between pages.
The fix was to change this in AJAX.cs:
using (var scriptManager = new ScriptManager //RadScriptManager
{
ID = "ScriptManager",
EnableScriptGlobalization = true,
SupportsPartialRendering = true
})
into that:
using (var scriptManager = new ScriptManager //RadScriptManager
{
ID = "ScriptManager",
EnableScriptGlobalization = true,
SupportsPartialRendering = true,
EnableScriptLocalization = false
})

I would have a patch for this issue that I could submit in case anyone is interested in this.
Can anyone, running multiple languages, using modules that include Ajax and having CDN enabled for Ajax, confirm that they have the same problem? You should be able to see the failed request, due to the wrong file name generated having the language code added, in a Browser network trace.
I would be glad if I could submit this so I don't have to use a custom build anymore.
Best regards & happy DNNing
Andy9999

I will test it one more time as a lot of time has passed, but if I remember correctly it also would occur if CDN is disabled. The fix would then be, to simply copy the existing original version and give the copy the expected file name. But I could be wrong. The problem with CDN being enabled is clearly that you cannot create that additional copy on Microsoft's CDN server.
Best regards & happy DNNing
Andy9999

Well, if it doesn't work at all, then we can just always set EnableScriptLocalization to false, right?

True. :)

A tiny update.
This is the URL to Ajax being used: https://ajax.aspnetcdn.com/ajax/4.6/1/MicrosoftAjax.js
There is a folder below with the name globalization and inside there is for example a file "fr-FR.js" or "de-DE.js". I am not sure if this is loaded automatically by the main script.
So this is for example a valid URL: https://ajax.aspnetcdn.com/ajax/4.6/1/globalization/fr-FR.js or https://ajax.aspnetcdn.com/ajax/4.6/1/globalization/de-DE.js
I have opened an issue over here and hope someone can clarify: https://github.com/dotnet/AspNetDocs/issues/594
Best regards & happy DNNing
Andy9999

Here is some interesting documentation about globalization support in ASP.NET AJAX applications: https://docs.microsoft.com/en-us/archive/msdn-magazine/2008/january/globalization-support-in-asp-net-ajax-applications
According to that article, setting EnableScriptLocalization  to true (the default) should cause the following to be included (which resembles of the fr-FR.js content for example):
var __cultureInfo = '{"name":"en-GB",
"numberFormat":{ "CurrencyDecimalDigits":2, "CurrencyDecimalSeparator":".","IsReadOnly":true,
"CurrencyGroupSizes":[3],"NumberGroupSizes":[3], "PercentGroupSizes":[3],"CurrencyGroupSeparator":", ",
"CurrencySymbol":"£","NaNSymbol":"NaN", "CurrencyNegativePattern":1,"NumberNegativePattern":1,
"PercentPositivePattern":0,"PercentNegativePattern":0, "NegativeInfinitySymbol":"-Infinity","NegativeSign":"- ...
Maybe this problem occurs in conjunction with EnableMsAjaxCDN being enabled only.
Here is one other person complaining, in this case about MicrosoftAjax.zh-Hant.js:
https://github.com/dotnet/AspNetCore.Docs/issues/10133
Am still wondering what is going on there and how the js files inside of that globalization folder should get loaded.
Best regards & happy DNNing
Andy9999

LOL. Case closed -> https://github.com/dotnet/AspNetDocs/issues/594

Maybe referring me to StackOverflow was helpful:
https://stackoverflow.com/questions/817046/what-about-script-globalization-of-microsoftajax-js-in-asp-net-mvc
Documenting it properly would have been helpful in the first place. Looks like a bug to me. But anyways. At least something to work with.
Best regards & happy DNNing
Andy9999


Getting mime error ['']

When deploying a flutter app, I get mime error for the main.dart.js file.
It happens with both the wrangler and manual deployment, in different browsers.
I tried disabling cache, redeploying... Nothing seems to work.
This looks like to happen with flutter web projects in such file.

@petebacondarwin So magically it works now, from both the CLI and direct upload.
I think a recent update fixed the bug or similar?
From my side i didn't have upgraded or modified anything...
Closing this as solved

Very strange. I haven't heard anything about this from the Pages team. But I see that this appears to have occurred occasionally in the past for various reasons. Please do reach out if it happens again.

I got something similar again.
Example:
https://xdonde.dh-sw.net/virgendeaguassantas/

@petebacondarwin

Are you using Firefox and the Dashboard Drag & Drop? If so, there's a known bug there that causes this.
If you're using Wrangler then ideally we get logs with WRANGLER_LOG=debug wrangler pages deploy ...

Yes, I am

Okay, tried with chrome. Same is happening

Going to try with wragler

Tried with wrangler, also mime error. I'm getting the plain text HTML instead of the actual website

have you got a repro you can share?

Unfortunately not, but here you have a ZIP with it:

xdonde.zip

@WalshyDev
Is that enough? Were you able to reproduce the issue?

@DH-555 it doesn't appear that you attached anything. can you please provide a repro?

Looks like this issue is specific to the deployed page.
I created a new page and it works fine.
I'm closing this consequently, but I don't understand why it happens only with that project/page deployment.


PR: Use community-based python-lsp-server instead of Palantir's python-language-server

Description of Changes
This PR migrates Spyder to use the community maintained python-lsp-server instead of Palantir's python-language-server, which stopped being maintained at the end of last year. This PR also updates the versions of pyls-spyder and replaces pyls-black by python-lsp-black.

[ ] Wrote at least one-line docstrings (for any new functions)
[x] Added unit test(s) covering the changes (if testable)

[ ] Included a screenshot or animation (if affecting the UI, see Licecap)

Issue(s) Resolved

Fixes #
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: @andfoy

Hello @andfoy! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file external-deps/python-lsp-server/pylsp/config/config.py:

Line 54:80: E501 line too long (97 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/hookspecs.py:

Line 108:80: E501 line too long (81 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/plugins/autopep8_format.py:

Line 19:80: E501 line too long (85 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/plugins/flake8_lint.py:

Line 71:80: E501 line too long (99 > 79 characters)
Line 76:80: E501 line too long (99 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/plugins/jedi_completion.py:

Line 14:80: E501 line too long (82 > 79 characters)
Line 16:80: E501 line too long (97 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/plugins/jedi_rename.py:

Line 12:80: E501 line too long (101 > 79 characters)
Line 20:80: E501 line too long (81 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/plugins/pydocstyle_lint.py:

Line 63:80: E501 line too long (82 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/plugins/pylint_lint.py:

Line 239:80: E501 line too long (99 > 79 characters)
Line 245:80: E501 line too long (99 > 79 characters)

In the file external-deps/python-lsp-server/pylsp/python_lsp.py:

Line 53:80: E501 line too long (80 > 79 characters)
Line 87:80: E501 line too long (80 > 79 characters)
Line 234:80: E501 line too long (95 > 79 characters)
Line 240:80: E501 line too long (81 > 79 characters)
Line 247:80: E501 line too long (83 > 79 characters)
Line 253:80: E501 line too long (88 > 79 characters)
Line 262:80: E501 line too long (98 > 79 characters)
Line 265:80: E501 line too long (88 > 79 characters)
Line 284:80: E501 line too long (88 > 79 characters)

In the file external-deps/python-lsp-server/test/plugins/test_autopep8_format.py:

Line 5:80: E501 line too long (83 > 79 characters)

In the file external-deps/python-lsp-server/test/plugins/test_completion.py:

Line 14:80: E501 line too long (85 > 79 characters)
Line 15:80: E501 line too long (85 > 79 characters)
Line 116:80: E501 line too long (88 > 79 characters)
Line 119:80: E501 line too long (92 > 79 characters)

In the file external-deps/python-lsp-server/test/plugins/test_definitions.py:

Line 36:80: E501 line too long (95 > 79 characters)
Line 59:80: E501 line too long (95 > 79 characters)

In the file external-deps/python-lsp-server/test/plugins/test_jedi_rename.py:

Line 43:80: E501 line too long (82 > 79 characters)

In the file external-deps/python-lsp-server/test/plugins/test_rope_rename.py:

Line 30:80: E501 line too long (82 > 79 characters)

In the file external-deps/python-lsp-server/test/plugins/test_yapf_format.py:

Line 60:80: E501 line too long (109 > 79 characters)

In the file external-deps/python-lsp-server/test/test_language_server.py:

Line 41:80: E501 line too long (93 > 79 characters)
Line 45:80: E501 line too long (103 > 79 characters)

In the file external-deps/python-lsp-server/test/test_workspace.py:

Line 218:80: E501 line too long (84 > 79 characters)
Line 245:80: E501 line too long (91 > 79 characters)
Line 256:80: E501 line too long (82 > 79 characters)

@spyder-ide/core-developers, @spyder-ide/junior-developers, after this PR you need to install our new, community supported Python language server, called python-lsp-server, pyls-spyder 0.4.0 and python-lsp-black.
I recommend you to use Conda-forge these days because the defaults channel is usually outdated and has problems. If you're already using it, please run the following commands:
conda remove -c conda-forge python-language-server
conda install -c conda-forge pyls-spyder=0.4.0 python-lsp-black

If you're using pip, please run
pip uninstall python-language-server
pip install -U pyls-spyder python-lsp-black

Sorry for the disruption in your workflows. @andfoy and I have been working on this for the last three months because I lost maintainer permissions in the previous language server and we decided that the best moving forward was to fork it and maintain it in an organization of our own with other interested developers.


[iOS] SWIFT_STRICT_CONCURRENCY Complete - can't build

I can't build my projects with set SWIFT_STRICT_CONCURRENCY Complete
In Swift 6.0 this property will be set by default. This bug should have high priority because projects with ModernRIBs included can't be build.
Details:
Issue #633 in RIBs

Thanks!
I will take a look into it.

@programming086
I was preparing a patch for Swift 6, but I found out that there was a comment in the RIBs repo.
I think it would be good to watch the changes in RIBs.

If there is a need to change in ModernRIBs, I will revisit again.

@DevYeom https://github.com/uber/RIBs/issues/640

@DevYeom UBER created repo https://github.com/uber/RIBs-iOS


Using actingAs with call function in testing (authenticate user with call function)

Laravel Version: 5.2
PHP Version: 5.5
Database Driver & Version:

Description:
I need to test a form submit of an authenticated user.
I need to use assertSessionHasErrors function, but it only works when you use call function.
If I use call function I'm always redirected to login page.
If I use actingAs function I never see validation errors.
Steps To Reproduce:
$user = factory(App\User::class)->create();
$this
        ->actingAs($user)
        ->visit('/offers/create')
        ->select('1', 'client_id')
        ->select('2', 'network_id')
        ->type('50', 'royalty')
        ->press('SAVE')
        ->seePageIs('/offers');

$errors = session('errors');
$this->assertSessionHasErrors();

// REDIRECTED TO LOGIN PAGE
$response = $this->call('POST', 'offers', array(
    '_token' => csrf_token(),
    'client_id' => 1,
    'network_id' => 2,
    'royalty' => 50
));

Laravel 5.2 is no longer supported.
Try a forum like Laracasts or StackOverflow.


Catch an exception that is able to provide a request

Unfortunately I was not able to add tests to handle this scenario. There seems to be no unit tests present, and the existing integration tests are not covering this logic (it's hard to follow them).

Thanks!


feat: add ew chains

Why is the ewc chain in a different directory than the volta chain?

Both should be in same directory i.e. under src, I fixed this.


clangd find references does not work for macros under extern "C"

I find that clangd is unable to "go to references" on TEST_THIS ("YcmCompleter GoToReferences" in my case), if I simply enclose #define TEST_THIS within extern "C" {}.
// header.h

#pragma once

extern "C" {
#define TEST_THIS
}

// main.cpp

#include <iostream>
#include "header.h"

int main()
{
#ifdef TEST_THIS
        std::cout << "Test this\n";
#endif
        std::cout << "Hello, world!\n";
}

If I remove extern "C" {} everything behaves as expected.
I generate the compilation database with the help of:
# CMakeLists.txt

project(hello)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
add_executable(hello main.cpp)

The compilation database looks as follows:
[
{
  "directory": "/path/test/build",
  "command": "/usr/bin/c++      -o CMakeFiles/hello.dir/main.cpp.o -c /path/test/main.cpp",
  "file": "/path/test/main.cpp"
}
]

The built program behaves as expected in both cases.
Note: extern "C" {} in the case above has no value for the enclosed code, but I guess I am not the only one working with code bases which mix C and C++, in which the C header files systematically include an enclosing extern "C" {} under #ifdef __cplusplus for the entire file.
If this is a bug, it looks surprising. If I am missing something, please tell me and I will happily close this issue.
Logs
clangd_stderros87pcri.log
System information
Output of clangd --version:
clangd version 11.0.0 (https://github.com/llvm/llvm-project d6875948aaade1cd39e5d9b373d02749dd1e58f2)
Editor/LSP plugin: Vim/YouCompleteMe
Operating system: Xubuntu Linux 20.04

I don't think you're missing anything, it looks like a bug to me.

Here is a further reduced example:
extern "C" {
#define MACRO 42
}

int main() {
  int x = MACRO;
}

Find-refs on the macro definition does not find any results. If you remove the extern "C", it works.

Here's a way to trigger the same bug without extern "C":
int magic;

#define MACRO 42

int main() {
  int x = MACRO;
}

Once again, find-refs of the macro definition does not find any results. However, if you comment out int magic;, it works.

The presence of the bug appears to hinge on whether the macro definition occurs in the "preamble" part of the source file (find-refs works) vs. the "main-file" part (find-refs does not work).

It's also worth nothing that find-refs works when the search is initiated from a reference:
int magic;

#define MACRO 42

int main() {
  int x = MACRO;  // <--
  int y = MACRO;
}

Find-refs on the macro use on the indicated line finds both uses and the definition. It's only searching from the definition that fails, in the case where the definition occurs outside the preamble.

In case of a definition outside the preamble, find-refs fails in the locateMacroAt() operation, which tries to find a macro definition corresponding to the token under the cursor.
Since macros can be undefined and redefined, there can be multiple macro definitions associated with a single macro name in a translation unit. Therefore, to determine whether a particular macro definition is the correct one to return, we check that the location of its definition comes before the token under the cursor, and if there is an #undef, that the #undef comes after. These checks are implemented in MacroDirective::findDirectiveAtLoc(), which clangd calls into via Preprocessor::getMacroDefinitionAtLoc().
According to these checks, the location of the macro name itself in both a #define and an #undef are (just) outside the "scope" of the macro. However, we want locateMacroAt() to find the macro definition in both of these cases.
We have an existing workaround for #undef: we adjust the token location, moving it forward by one token. The modified location now passes the "the #undef comes after" check.
We do not have a similar workaround for #define; for that, the original location already fails the "the #define comes before" check by one token (and the adjustment makes it fail by two tokens).
So, why does it work for macro definitions that are in the preamble?
For a non-preamble definition, the location of the definition is something like {FileID=1, Offset=26}, and the location of the token under the cursor is the same (and then gets adjusted to {FileID=1, Offset=25}) and fails the check.
For a preamble definition, the location of the token under the cursor is something like {FileID=1, Offset=26}, but the location of the macro definition is something like {FileID=-3, Offset=26}. The differing FileID (presumably, a "preamble FIleID") seems to cause it to pass the "comes before" check regardless of the offset.
(Why doesn't the location of the token under the cursor also have a "preamble FileID"? This part I don't understand. This location comes from a syntax::Token which is found by searching ParsedAST::getTokens(). The documentation for this function says it does not contain preamble tokens, so I'm not sure how we get a result here at all.)

Here is a proposed fix which takes the approach of checking two different locations, just before the token after the cursor (to handle the #undef case) and just after (to handle the #define) case.


Include documentation files in the sdist.

In particular I'm interested in getting LICENSE into the sdist tarball, since that would make things easier for anyone redistributing the code to satisfy the
(a) You must give any other recipients of the Work or
          Derivative Works a copy of this License;

condition.

Thanks !


Third party libraries need update on casper

#3110 upgraded several of our third party libraries, built in the conventional /usr/local/VAPOR-Deps/2019-Aug directory.
They need to additionally be built on casper, which holds the libraries in /glade/p/cisl/vast/vapor/third-party/Apr-2022.

Ugh. Is there some way we can avoid this? It's valuable to be able to build on casper because all of our larger test data sets that we often have to use for debugging are there. But having to build on casper simply because we don't have write access to /usr/local is a pain (the builds never seem to go smoothly). Thoughts anyone?

Maybe explore using patchelf on the 2019-Aug libraries?

That might address shared library issues, but there are application-specific resources (such as the UDUnits XML data base) that I don't think patchelf would help with.


Predict for tabular: Cannot convert non-finite values (NA or inf) to integer

Got an error when preds = model.predict(test_data) on a popular kaggle datasets (may due to this dataset has a bunch of NAs?)
File delimiter for ../input/house-prices-advanced-regression-techniques/test.csv inferred as ',' (comma). If this is incorrect, please manually load the data as a pandas DataFrame.
Loaded data from: ../input/house-prices-advanced-regression-techniques/test.csv | Columns = 80 / 80 | Rows = 1459 -> 1459
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-48-91455d7b9940> in <module>
      1 test_data = task.Dataset(file_path=os.path.join(in_dir, job, 'test.csv'))
      2
----> 3 preds = model.predict(test_data)

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/autogluon/task/tabular_prediction/predictor.py in predict(self, dataset, model, as_pandas, use_pred_cache, add_to_pred_cache)
    130         """
    131         dataset = self.__get_dataset(dataset)
--> 132         return self._learner.predict(X=dataset, model=model, as_pandas=as_pandas, use_pred_cache=use_pred_cache, add_to_pred_cache=add_to_pred_cache)
    133
    134     def predict_proba(self, dataset, model=None, as_pandas=False, as_multiclass=False):

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/autogluon/utils/tabular/ml/learner/abstract_learner.py in predict(self, X, model, as_pandas, use_pred_cache, add_to_pred_cache)
    127
    128         if len(X_cache_miss) > 0:
--> 129             y_pred_proba = self.predict_proba(X=X_cache_miss, model=model, inverse_transform=False)
    130             problem_type = self.trainer_problem_type or self.problem_type
    131             y_pred = get_pred_from_proba(y_pred_proba=y_pred_proba, problem_type=problem_type)

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/autogluon/utils/tabular/ml/learner/abstract_learner.py in predict_proba(self, X, model, as_pandas, as_multiclass, inverse_transform)
     92     # TODO: Add pred_proba_cache functionality as in predict()
     93     def predict_proba(self, X: DataFrame, model=None, as_pandas=False, as_multiclass=False, inverse_transform=True):
---> 94         X = self.transform_features(X)
     95         y_pred_proba = self.load_trainer().predict_proba(X, model=model)
     96         if inverse_transform:

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/autogluon/utils/tabular/ml/learner/abstract_learner.py in transform_features(self, X)
    216     def transform_features(self, X):
    217         for feature_generator in self.feature_generators:
--> 218             X = feature_generator.transform(X)
    219         return X
    220

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/autogluon/utils/tabular/utils/decorators.py in inner1(*args, **kwargs)
     12         begin = time.time()
     13
---> 14         output = func(*args, **kwargs)
     15
     16         # storing time after function execution

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/autogluon/utils/tabular/features/abstract_feature_generator.py in transform(self, X)
    183             raise ValueError(f'Required columns are missing from the provided dataset. Missing columns: {missing_cols}')
    184
--> 185         X = X.astype(self.features_init_types)
    186         X.reset_index(drop=True, inplace=True)
    187         X_features = self.generate_features(X)

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/generic.py in astype(self, dtype, copy, errors, **kwargs)
   5863                     results.append(
   5864                         col.astype(
-> 5865                             dtype=dtype[col_name], copy=copy, errors=errors, **kwargs
   5866                         )
   5867                     )

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/generic.py in astype(self, dtype, copy, errors, **kwargs)
   5880             # else, only a single dtype is given
   5881             new_data = self._data.astype(
-> 5882                 dtype=dtype, copy=copy, errors=errors, **kwargs
   5883             )
   5884             return self._constructor(new_data).__finalize__(self)

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/internals/managers.py in astype(self, dtype, **kwargs)
    579
    580     def astype(self, dtype, **kwargs):
--> 581         return self.apply("astype", dtype=dtype, **kwargs)
    582
    583     def convert(self, **kwargs):

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/internals/managers.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
    436                     kwargs[k] = obj.reindex(b_items, axis=axis, copy=align_copy)
    437
--> 438             applied = getattr(b, f)(**kwargs)
    439             result_blocks = _extend_blocks(applied, result_blocks)
    440

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/internals/blocks.py in astype(self, dtype, copy, errors, values, **kwargs)
    557
    558     def astype(self, dtype, copy=False, errors="raise", values=None, **kwargs):
--> 559         return self._astype(dtype, copy=copy, errors=errors, values=values, **kwargs)
    560
    561     def _astype(self, dtype, copy=False, errors="raise", values=None, **kwargs):

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/internals/blocks.py in _astype(self, dtype, copy, errors, values, **kwargs)
    641                     # _astype_nansafe works fine with 1-d only
    642                     vals1d = values.ravel()
--> 643                     values = astype_nansafe(vals1d, dtype, copy=True, **kwargs)
    644
    645                 # TODO(extension)

~/miniconda3/envs/autogluon/lib/python3.7/site-packages/pandas/core/dtypes/cast.py in astype_nansafe(arr, dtype, copy, skipna)
    698         if not np.isfinite(arr).all():
    699             raise ValueError(
--> 700                 "Cannot convert non-finite values (NA or inf) to " "integer"
    701             )
    702

ValueError: Cannot convert non-finite values (NA or inf) to integer

Here is the code
import os
import glob

def run_cmd(cmd):
    assert os.system(cmd) == 0, cmd

job = 'house-prices-advanced-regression-techniques'
in_dir = os.path.join('..', 'input')

run_cmd(f'kaggle c download -c {job} -p {in_dir}')
run_cmd(f'cd {in_dir}; unzip -n {job}.zip -d {job}')
glob.glob(os.path.join(in_dir, job, '*'))

from autogluon import TabularPrediction as task

train_data = task.Dataset(file_path=os.path.join(in_dir, job, 'train.csv'))
model = task.fit(train_data=train_data.drop(columns='Id'), label='SalePrice', verbosity=4)
test_data = task.Dataset(file_path=os.path.join(in_dir, job, 'test.csv'))
preds = model.predict(test_data.drop(columns='Id'))

@Innixma  Seems like a new bug perhaps introduced in the feature-generator refactor?
I've run this dataset multiple times in the past without issue.

Fixed in #584 (https://github.com/awslabs/autogluon/pull/584/commits/11b4f5abf9e29f773be074ddc0ccfe685a0b6c51)
When the functionality to convert dtypes to the original train dtypes at inference time was added, this bug was created, because integer types can't have NaN. So during training, if an int type has no NaN's, then training succeeds, but if the same column has NaN's at test time, pandas actually loads the column as float. When asked to convert to int by our code, it crashes with the error described. It appears that none of our benchmark datasets have this property unfortunately, so it wasn't caught.
For now, I've patched this by filling NaN's to 0 for such columns, although this may not be ideal in the long term.
Another way to avoid the crash (Without needing the fix) is to pass test_data as tuning_data=test_data.drop(columns='Id') in task.fit, since when concatenating train and test dataframes for feature generation, the int columns of train are converted to float if test type is float. This guarantees test_data will not crash in predict, but not new data test_data_2 for example.

@Innixma
Whenever we cast columns into new types, what if we just to try-except and keep the original-type if the column can't be casted?
I'd imagine all models will still work if trained on int column but then tested with float column (not 100% sure about this though).

@Innixma
Whenever we cast columns into new types, what if we just to try-except and keep the original-type if the column can't be casted?
I'd imagine all models will still work if trained on int column but then tested with float column (not 100% sure about this though).

That will open the gates to many defects down the road I think. This will also mean anyone who makes models will have to know that the input data could be of a different type during inference, which is really confusing.

I agree having unexpected types during inference could be weird if people are using models that can't handle the type-change (I'm not sure what models those are though). For now we can just do this on case-by-case basis, although I guess then we need to wait for bug notifications and then. For these ints specifically, we should probably be imputing via something like: int(nanmean(column_vals). Introducing 0s at test-time may be bad if say the training ints are all > 1000.

Imputation method should likely be a parameter to the generator. The main reason I am converting to 0 at present is because that is how our downstream models handle it generally (RF/KNN), so making it something other than 0 would also mean we probably want to change it for RF/KNN as well.


feat(landing): implement landing screen ui

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#46  👈
#45
#44
#43
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @libhide and the rest of your teammates on  Graphite

Merge activity

Jun 6, 8:55 AM EDT: @libhide started a stack merge that includes this pull request via Graphite.


Add some error handling and retries when creating a new connection

If connection to DB fails before we hand it off to the application, retry connecting to the DB a few times before failing

+1 Looks reasonable, are there any drawbacks for this PR?

if MYSQLPOOL_RETRY = 5, it actually retried 6 times.

I think it should be refactored into:
while(retry and not connection):
    try:
        connection = get_pool().connect(**kwargs)
    except:
        if not retry:
            raise
        connection = None
        retry -= 1
return connection


refactor: synchronize version numbers for all libraries

Fixes #3615, fixes #3537, and fixes #3535

This change is

Created #3712 to cleanup the excess Version() functions


vm continues 5 minites, then err: "dirty virt-launcher shutdown"

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

What happened:
vm continues 5 minites, then err: "dirty virt-launcher shutdown", anyone knows why?
{"component":"virt-launcher","level":"info","msg":"Collected all requested hook sidecar sockets","pos":"manager.go:68","timestamp":"2020-12-31T07:12:11.458211Z"}
{"component":"virt-launcher","level":"info","msg":"Sorted all collected sidecar sockets per hook point based on their priority and name: map[]","pos":"manager.go:71","timestamp":"2020-12-31T07:12:11.458326Z"}
{"component":"virt-launcher","level":"info","msg":"Connecting to libvirt daemon: qemu:///system","pos":"libvirt.go:374","timestamp":"2020-12-31T07:12:11.460801Z"}
{"component":"virt-launcher","level":"info","msg":"Connecting to libvirt daemon failed: virError(Code=38, Domain=7, Message='Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory')","pos":"libvirt.go:382","timestamp":"2020-12-31T07:12:11.461297Z"}
{"component":"virt-launcher","level":"info","msg":"libvirt version: 6.0.0, package: 16.fc31 (Unknown, 2020-04-07-15:55:55, )","subcomponent":"libvirt","thread":"98","timestamp":"2020-12-31T07:12:11.498000Z"}
{"component":"virt-launcher","level":"info","msg":"hostname: ubuntu-image","subcomponent":"libvirt","thread":"98","timestamp":"2020-12-31T07:12:11.498000Z"}
{"component":"virt-launcher","level":"error","msg":"internal error: Child process (/usr/sbin/dmidecode -q -t 0,1,2,3,4,17) unexpected exit status 1: /dev/mem: No such file or directory","pos":"virCommandWait:2709","subcomponent":"libvirt","thread":"98","timestamp":"2020-12-31T07:12:11.498000Z"}
{"component":"virt-launcher","level":"info","msg":"Connected to libvirt daemon","pos":"libvirt.go:390","timestamp":"2020-12-31T07:12:11.962866Z"}
{"component":"virt-launcher","level":"info","msg":"Registered libvirt event notify callback","pos":"client.go:382","timestamp":"2020-12-31T07:12:11.965756Z"}
{"component":"virt-launcher","level":"info","msg":"Marked as ready","pos":"virt-launcher.go:71","timestamp":"2020-12-31T07:12:11.965952Z"}
{"component":"virt-launcher","level":"error","msg":"dirty virt-launcher shutdown","pos":"virt-launcher.go:505","reason":"exit status 2","timestamp":"2020-12-31T07:17:11.976847Z"}
What you expected to happen:
vm runs normally.
How to reproduce it (as minimally and precisely as possible):
Deploy vm.yaml
Anything else we need to know?:
no
Environment:ubuntu

KubeVirt version (use virtctl version):0.33.0
Kubernetes version (use kubectl version):1.17
VM or VMI specifications:
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

/assign @rmohr

/assign @rmohr

@lixiaobing1 does your VM support ACPI triggered shutdown?

@lixiaobing1 does your VM support ACPI triggered shutdown?

In a normal virt-launcher creation flow, a new container is created, being configured and marked as ready.
Then it waits for virt-handler to start a VM.
It seems, in this case, virt-launcher was Marked as ready, but either there was no command from virt-handler to start a vm or the vm just crashed immediately.
We'll probably need to look at the virt-handler logs, from the node where this virt-launcher attempted to start, to better understand the issue.

In a normal virt-launcher creation flow, a new container is created, being configured and marked as ready.
Then it waits for virt-handler to start a VM.
It seems, in this case, virt-launcher was Marked as ready, but either there was no command from virt-handler to start a vm or the vm just crashed immediately.
We'll probably need to look at the virt-handler logs, from the node where this virt-launcher attempted to start, to better understand the issue.

@lixiaobing1 could you provide the logs @vladikr requested? Thanks.


Apparent Memory Leak

I'm back!
When recording with wf-recorder memory usage just keeps going up, at a fairly rapid pace. I believe this is a new issue. Haven't noticed it until the most recent update, at least (though wf-recorder was broken for me recently, until #251's last commits).
Of note, the output file does still increase in side on disk while the recording is happening.
This issue is reproducible on my machine with:
wf-recorder -f /tmp/$(date +'%Y-%m-%d_%H-%M-%S.mp4') -o DP-1

Here is a video (recorded with `wf-recorder!) of my memory ticking up until I hit my machine's limit (3x speed):
https://github.com/ammen99/wf-recorder/assets/29154459/0bbc57a1-e0bb-48d7-b537-96df1972d016
And a snapshot shortly after killing wf-recorder:

here are some logs:
command
WAYLAND_DEBUG=1 wf-recorder -f /tmp/$(date +'%Y-%m-%d_%H-%M-%S.mp4') -o DP-1 &> /tmp/vid.log

logs: https://gist.github.com/John-Paul-R/f09f0904240c1cb9f24f0b6237f6d3e9

You are aware that /tmp/ is likely to be tmpfs, and therefore you're recording to RAM, right?

I was not aware of that! However I also encounter the issue when recording to other directories. e.g.
wf-recorder -f ~/Videos/a-real-path.mp4 -o DP-1

yields similar memory graphs

Hmm. I'll see if I can reproduce it to help out debugging it, in case the developer can't reproduce it.

I restarted my PC, and suddenly can no longer reproduce the issue. I suspect I had some broken local state. Closing!


safe list

Sometimes we need to filter resources on public networks to prevent erroneous references. For example, the inappropriate license. When we downloaded resources from the public network, can we provide the function of configuring the security list?

@Jeffrey1029 I think you will need to elaborate much better what you want to achieve. Examples, etc etc.

I'm very sorry,my English a little poor.I'll try my best to make me understood.
I have a request like this:
we can only use MIT license package, others is not allowed.
I want my private server can help me to filter which npm package is conform to my request
I remember that npm package often own a file like License.md
I don't know if it's possible to implement this function?

anyway  thank you very much


Website down?

I'm trying to get eta/etlas setup to compile a pretty straightforward cabal project to the jvm using eta on wsl, and it appears certain parts of the eta-lang website nescesary to get etlas to work are down.
Whenever I try to setup the eta-init project using gradle, I get `Execution failed for task ':setupEnvironmentEta'.

Failed to fetch file from url: http://cdnverify.eta-lang.org/eta-binaries/etlas-<IP_ADDRESS>/binaries/x86_64-linux/etlas`

Similarly, whenever I try to etlas update, I get: curl: (6) Could not resolve host: metrics.eta-lang.org

I am afraid that it is down since some time ago see: https://github.com/typelead/etlas/issues/109

I am afraid that it is down since some time ago see: https://github.com/typelead/etlas/issues/109

The same here, cdnverify.eta-lang.org refers to etabinaries.azureedge.net but it does not exists and sadly fails

Sad to hear.
It is also sad that it is not on Archive.org or something
and I am not sure how to build it myself, so I don;t even know if I can use it or not.
And if I can be honest, if a migration takes this long, there is something wrong in the infrastructure.
@rahulmutt Can you post any news on it?

I'm not in the position of stating this, but I believe that the project has been abandoned 😭

I tried building it before. Didn't work
If you want to use it, use the Docker container, it works. But fair warning, it is not easy to get files in or out of the container (unless I did something wrong, which is likely, since I do not use Docker)


Shard strategy for Kafka Sink

Use case

As a user, i would like to shard the data to my Kafka topic according to some sharding expression when sink to it.
Describe the solution you'd like

CREATE EXTERNAL STREAM sharded_external_stream
(raw string, id string)
SETTINGS sharding_expr=hash(id);

https://github.com/timeplus-io/proton/blob/develop/src/Storages/MergeTree/MergeTreeSettings.h#L171
https://github.com/timeplus-io/proton/blob/develop/src/Storages/Streaming/StreamSink.cpp#L55-L84
Describe alternatives you've considered

Additional context

We calculate an integer and pass the integer as partition key to librdkafka

There are two things for sharding/paritioning:

just use a message key. Kafka will make sure messages with the same key will be stored in the same partition.
calculate the parition ID directly. We can assign a partitioner function and let it to calculate the partition ID for the messages.

For now, I think using message key should be enough for most cases. I would like to separate the implementation for 2 in another ticket (when it's needed, and in fact, in the example of this ticket hash(id), it does not look like it will generate a partition ID, it just says it wants to use id as the message key).
And for speicifying message key, I suggest we use a setting called message_key ( or sharding_key sounds fine to me too ).
@chenziliang any objection?

sharding could depend on multiple columns or an expression like a function. If we need leverage kafka message key , we will need allocate memory and copy over the key columns and for expression, we still need evaluate it first and then copy over.
So above sharding expression doesn't calculate a partition ID or shard ID, it just produce an integer. We can pass the integer to Kafka as the partition key i think things will just work.

Message key is a string, so the expression should generate a string, not an integer, thus the expression should return a string, and we should not do any implicite converstion.

the expression shall return an integer

we can cast the integer before passing it to libradkafka or just use the integer to assign a partition by ourselves which is also easy.

There are two things for sharding/paritioning:

just use a message key. Kafka will make sure messages with the same key will be stored in the same partition.
calculate the parition ID directly. We can assign a partitioner function and let it to calculate the partition ID for the messages.

For now, I think using message key should be enough for most cases. I would like to separate the implementation for 2 in another ticket (when it's needed, and in fact, in the example of this ticket hash(id), it does not look like it will generate a partition ID, it just says it wants to use id as the message key).
And for speicifying message key, I suggest we use a setting called message_key ( or sharding_key sounds fine to me too ).
@chenziliang any objection?

I think step 1 that proton provides a message key is enough for now. Let us try to implement this first.

(Jove Github Bot) assuming it is not done, deferred this ticket to the next sprint.


[PDI-8935] - PluginMessages can avoid synchronization if it uses ConcurrentHashMap

replace synchronized block with ReentrantReadWrite lock to improve access granularity

@brosander, review it please.
Unfortunately, CHM.computeIfAbsent() was introduced only in Java 8, hence a direct replacement HM -> CHM is impossible now.

@akhayrutdinov this doesn't seem related to PDI-8935
Also, couldn't we use ConcurrentHashMap still?  The readwritelock can have multiple threads blocked waiting on the write lock at the same time, so you'd still get the chance of creating garbage PluginMessages but below you throw away the duplicate one right away (and no locking).  PluginMessages just have a String member variable and no other initialization and so should be really cheap to create in the race condition knowing we'll always return the first one put into the map:
PluginMessages result = MESSAGES_MAP.get( packageName );
if (result == null) {
  return MESSAGES_MAP.putIfAbsent( packageName, new PluginMessages( packageName ) );
}
return result;

Please let me know what you think.

@brosander, yes, that makes sense. I've updated the PR and changed its title.


git ui is not informative enough

tried to download latest version of lapce: 0.2.5. git shows changes kept in the project but once i commit them, it seems to do nothing. it is not clear whether commit is  saved and there is no indication where i can push to git repo

Duplicate of https://github.com/lapce/lapce/issues/793


Make RabbitMQ running SSL + allow irma-brain to use SSL with RabbitMQ (issue #22)

Make RabbitMQ running SSL + allow irma-brain to use SSL with RabbitMQ
This is related to issue:

Make RabbitMQ running SSL + allow irma-brain to use SSL with RabbitMQ: #22

Changelogs

New RabbitMQ configuration template added in "./extras/rabbitmq"
New folder SSL added in root repository, it will contain the certificates for irma-brain
Port configuration added in brain.ini for easy switching between NO SSL / SSL
SSL parameters added in brain.ini : activation switch, SSL certificate names in "./ssl"

Install
Making RabbitMQ running SSL

Copy the RabbitMQ configuration template provided in "./extras/rabbitmq" to "/etc/rabbitmq"
Restart RabbitMQ: sudo service rabbitmq-server restart

Allow irma-brain to use SSL with RabbitMQ

Update PORT in brain.ini from 5672 to 5671. Except if you change the default port defined in the the RabbitMQ configuration template provided in "irma-brain/extras/rabbitmq"
Update activate_ssl switch in brain.ini to value yes
Put the SSL certificates (CA_CERT, KEY_FILE, CERT_FILE) in "./ssl"
Update ca_certs, keyfile and certfile in brain.ini according to the filenames in "./ssl"

Note
If you are switching to ssl from an already running no_ssl version, please do the following on irma-brain RabbitMQ server:

sudo rabbitmqctl stop_app
sudo rabbitmqctl reset
sudo rabbitmqctl start_app
create again the RabbitMQ vhosts, usernames and passwords:
sudo ./extras/scripts/rabbitmq/rmq_adduser.sh probe probe mqprobe
sudo ./extras/scripts/rabbitmq/rmq_adduser.sh brain brain mqbrain
sudo ./extras/scripts/rabbitmq/rmq_adduser.sh frontend frontend mqfrontend

merged


Domains / Sort

Hi,
As per the other closed issued, if its possible I think a switch for domains and if domain selected only output to file.  (email, breach(s)) - emails on a domain change frequently so adding all to a file is really time consuming.
Also the ability to display results in order of breach date.

Read this
https://haveibeenpwned.com/DomainSearch
Domain Search needs authorisation


Missing CONTRIBUTING.md File in Repository

Describe the bug ✍
The repository currently lacks a CONTRIBUTING.md file, which is an important resource for guiding potential contributors on how to engage with the project effectively. A CONTRIBUTING.md file outlines the contribution guidelines, coding standards, and procedures for submitting pull requests, thus streamlining the contribution process and ensuring consistency across contributions.
Provide step by step information reproduce the bug 📄
I need to create a CONTRIBUTING.md file tailored to our project's specific needs and guidelines. This file should include information on how to contribute, coding conventions, testing procedures, and any other relevant guidelines for potential contributors. By providing this information upfront, we can lower the barrier to entry for new contributors and ensure that contributions align with project standards.
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

But i think so we link of code of conduct and CONTRIBUTING GUIDELINE is not working

@suhanipaliwal Please raise any other issue, then i'll assign you
If you have any doubt you can directly as on our discord (https://discord.gg/rZb46cCMmK), we have created seprate chennals for all projects...
Thank you :)


RedHat review - Update cache replication config

Describe the issue
Update cache replication config:

owners should not be default 1
implement cache replication monitoring

Notes:

with an owner set to 1, cache is not replicated to other nodes. Only work cache is sending the update about cache entry being invalidated
need to use mode anymore with current version as it by default SYNC

Definition of done

tested cache are replicated corerctly with the config change (depends on monitoring ready)
updated cache config for SSO instances

###Update:

cache replication will require more memory on each pod, changes need to be tested
currently having issue on k6 test to produce production like cache/session load, will need to work on it first

Update:

currently verified that cache is not replicated between restarted pods (rolling deployment will lose all user sessions)
tested redhat's suggestion with multiple owner config but failed with following message:

\|  247: <replicated-cache  name="sessions" owners="2"/>
--
  | \|                                          ^^^^ 'owners' isn't an allowed attribute for the 'replicated-cache' element
  | \|
  | \|                                               The 'replicated-cache' element doesn't allow any attributes

Update:
Tested cache (for user sessions) behaviour in difference cases:

rolling deployment: sessions are well maintained and no cache error logs
randomly killing pods: sessions will be gone and during restarting, will receive issues with work cache, which is trying to evict non-existing cache. (see https://access.redhat.com/solutions/4321411)

Thus, we can verify that replicated-cache is working as expected during normal operational tasks. Also in the situation where pods are deleted from the platform, cache issue will be able to resolve itself after some tries.
In addition, we were using distrubuted-cache with owners="3", as RedHat suggested, but had eventually updated to replicated-cache (see PR https://github.com/bcgov/ocp-sso/pull/48).
Will close this ticket without changes.


Send only streams data if channel is not closed

What type of PR is this?
/kind bug
What this PR does / why we need it:
The stream data is only read on read_all_with_timeout, which is called via exec sync or on on container creation error. This means we can end-up having a closed sender, but we should not fail in this case because it will lead into unexpected container termination.
Which issue(s) this PR fixes:
Fixes https://github.com/containers/conmon-rs/issues/720
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

Fixed bug where a closed stream sender causes container termination.

An alternative would be to read after successful container creation:
diff --git a/conmon-rs/server/src/container_io.rs b/conmon-rs/server/src/container_io.rs
index b1ebc52..c1e4a61 100644
--- a/conmon-rs/server/src/container_io.rs
+++ b/conmon-rs/server/src/container_io.rs
@@ -267,11 +267,9 @@ impl ContainerIO {
                                 .await
                                 .context("write to attach endpoints")?;

-                            if !message_tx.is_closed() {
-                                message_tx
-                                    .send(Message::Data(data.into(), pipe))
-                                    .context("send data message")?;
-                            }
+                            message_tx
+                                .send(Message::Data(data.into(), pipe))
+                                .context("send data message")?;
                         }
                         Err(e) => match Errno::from_i32(e.raw_os_error().context("get OS error")?) {
                             Errno::EIO => {
diff --git a/conmon-rs/server/src/rpc.rs b/conmon-rs/server/src/rpc.rs
index daea94e..5462ced 100644
--- a/conmon-rs/server/src/rpc.rs
+++ b/conmon-rs/server/src/rpc.rs
@@ -14,7 +14,7 @@ use std::{
     process, str,
     time::Duration,
 };
-use tokio::time::Instant;
+use tokio::{task, time::Instant};
 use tracing::{debug, debug_span, error, Instrument};
 use uuid::Uuid;

@@ -150,18 +150,28 @@ impl conmon::Server for Server {

                 // register grandchild with server
                 let io = SharedContainerIO::new(container_io);
+                let io_clone = io.clone();
                 let child = Child::new(
                     id,
                     grandchild_pid,
                     exit_paths,
                     oom_exit_paths,
                     None,
-                    io,
+                    io_clone,
                     cleanup_cmd,
                     token,
                 );
                 capnp_err!(child_reaper.watch_grandchild(child))?;

+                task::spawn(
+                    async move {
+                        if let Err(e) = io.read_all_with_timeout(None).await {
+                            error!("IO read failure: {:#}", e);
+                        }
+                    }
+                    .instrument(debug_span!("read_all_with_timeout")),
+                );
+
                 results
                     .get()
                     .init_response()

So the e2e tests are now looking way better with this change: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/cri-o_cri-o/6220/pull-ci-cri-o-cri-o-main-ci-e2e-conmonrs/1572188830188965888
The 2 failing tests are also failing on main right now.

/lgtm
good find!


Kept getting "nvidia container" issues and got error for using image downloader sh file

For windows, I have WSL2 and a 3080ti, and docker desktop.
Super interested in a quick lora training tool like this but kept getting an error that "nvidia" was not found when running the training (both in WSL2 and cmd).
Plus the .sh download file gave an error too.
Were you able to run this on Windows?

Hello.

"nvidia" was not found when running the training

I was able to reproduce the problem on WSL 2 and have fixed it on PR #2.
Could you please try running it again on the latest main branch?
To ensure that you can access the GPU within the Docker under the WSL 2 environment, please run the following command:
docker run -it --rm --gpus all ubuntu nvidia-smi

Plus the .sh download file gave an error too.

Well, I suspect that the script should be run via Docker.
Please run the following command on WSL 2. If some errors occur, could you provide me with more details about the error?
docker compose run download_images

Lemme try the latest version, thanks!


Version 0.5.0 deprecated

There seems to be an issue with the published version 0.5.0, it will be deprecated.
For now is safe and tested to use version 0.4.0, which has the same exact features.

Source of the issue is the sbt-scoverage plugin, removed for now.
New updated version v0.5.1 released.


Fsoc 144 - Add subscribe and Unsubscribe flags to solution list

Description
subscribed and unsubscribed flags added to solutions list; additionally changed how collection urls are parsed to ensure that they are query escaped
Type of Change

[ ] Bug Fix
[x] New Feature
[ ] Breaking Change
[x] Refactor
[ ] Documentation
[ ] Other (please describe)

Checklist

[x] I have read the contributing guidelines
[ ] Existing issues have been referenced (where applicable)
[x] I have verified this change is not present in other open pull requests
[ ] Functionality is documented
[x] All code style checks pass
[ ] New code contribution is covered by automated tests
[x] All new and existing tests pass

Codecov Report

Merging #107 (9845e4a) into main (d8a0b8b) will decrease coverage by 0.17%.
The diff coverage is 0.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #107      +/-   ##
==========================================
- Coverage   32.42%   32.26%   -0.17%
==========================================
  Files          27       27
  Lines        3559     3577      +18
==========================================
  Hits         1154     1154
- Misses       2315     2332      +17
- Partials       90       91       +1

Impacted Files
Coverage Δ

platform/api/collection.go
0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes


Update prometheus

What
Selectively bump prometheus export to 2.0.0 (as opposed to 2.0.1)
Ticket
VCD - Release of dependency and security patches (Sprint Wyvern)
Why
Breaking change Version 2.0.1 fails to start with NameError on PeriodicStats
How
Set Gemfile so use version 2.0.0

Confirmed metrics in dev again:

Moving up


Add help menu tests

Adds a class to test the help menu options "Documentation" and "Issues."

Hint: Reproduce the build on your machine so you can read the log it talks about. Google is your friend.

Good job :+1:


Strange issue with strange characters on linux

with this code:
from arrow_odbc import read_arrow_batches_from_odbc
         reader = read_arrow_batches_from_odbc(
            query=sql,
            connection_string=self.connection_string,
            max_binary_size=20000,
            max_text_size=20000,
        )
        print(sql)
        print(reader.schema)

I get this output:

SELECT [User - iD] AS [User_-_iD], [FirstName] AS [FirstName], [LastName] AS [LastName], [Age] AS [Age], [companyid] AS [companyid], CAST([time stämp] AS BIGINT) AS [time_stämp], CAST(GETUTCDATE() AS datetime2(6)) AS __timestamp, CAST(0 AS BIT) AS __is_deleted, CAST(1 AS BIT) AS __is_full_load FROM [dbo].[user]

User_-_iD: int64 not null
FirstName: string
LastName: string
Age: decimal128(15, 3)
companyid: string not null
time_stäm: int64
__timestamp: timestamp[us]
__is_deleted: bool
__is_full_load: bool

Please not the name of the [time_stämp] column in the schema

This does not happen on Windows because on windows arrow-odbc-py is compiled to use UTF-16 encoding to talk with the Database. On Linux and OS-X however the system local is usually UTF-8 and therefore arrow-odbc-py assumes the narrow encoding returned from the database is UTF-8. However, if your System local is configured not to be UTF-8 this assumption is wrong.
So my first piece of advice would be to check whether your System local is UTF-8. Other narrow ASCII encodings are currently not supported by arrow-odbc-py.

Ok, I'll try. For completness, here's the link to my failing gh action: https://github.com/bmsuisse/odbc2deltalake/actions/runs/8538422686

Looking at this it is more likely that somehow binary and character size is confused. I just do not know there. Yet, likely the system local is innocent. Would you kindly enable debug logging and execute again. In the beginning it should log the database column names and their type, even before conversion to arrow. I would be interested in seeing that.

You can log to standard error using: https://arrow-odbc.readthedocs.io/en/latest/arrow_odbc.html#arrow_odbc.log_to_stderr

Here we go:
DEBUG SQLAllocHandle allocated connection (Dbc) handle '0x55c451f30050'
WARN State: 01S00, Native error: 0, Message: [Microsoft][ODBC Driver 18 for SQL Server]Invalid connection string attribute
WARN State: 01000, Native error: 5701, Message: [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Changed database context to 'db_to_delta_test'.
WARN State: 01000, Native error: 5703, Message: [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Changed language setting to us_english.
DEBUG Database managment system name as reported by ODBC: Microsoft SQL Server
DEBUG ODBC driver reported for column 0. Relational type: BigInt; Nullability: NoNulls; Name: 'User_-_iD';
DEBUG ODBC driver reported for column 1. Relational type: Varchar { length: Some(100) }; Nullability: Nullable; Name: 'FirstName';
DEBUG ODBC driver reported for column 2. Relational type: WVarchar { length: None }; Nullability: Nullable; Name: 'LastName';
DEBUG ODBC driver reported for column 3. Relational type: Decimal { precision: 15, scale: 3 }; Nullability: Nullable; Name: 'Age';
DEBUG ODBC driver reported for column 4. Relational type: Varchar { length: Some(10) }; Nullability: NoNulls; Name: 'companyid';
DEBUG ODBC driver reported for column 5. Relational type: BigInt; Nullability: Nullable; Name: 'time_stäm';
DEBUG ODBC driver reported for column 6. Relational type: Timestamp { precision: 6 }; Nullability: Nullable; Name: '__timestamp';
DEBUG ODBC driver reported for column 7. Relational type: Bit; Nullability: Nullable; Name: '__is_deleted';
DEBUG ODBC driver reported for column 8. Relational type: Bit; Nullability: Nullable; Name: '__is_full_load';
DEBUG SQLColAttribute called with attribute 'ConciseType' for column '2' reported 12.
DEBUG SQLColAttribute called with attribute 'DisplaySize' for column '2' reported 100.
DEBUG Relational type of column 1: Varchar { length: Some(100) }
DEBUG SQLColAttribute called with attribute 'ConciseType' for column '3' reported -9.
DEBUG SQLColAttribute called with attribute 'DisplaySize' for column '3' reported 0.
DEBUG Relational type of column 2: WVarchar { length: None }
DEBUG SQLColAttribute called with attribute 'DisplaySize' for column '3' reported 0.
DEBUG SQLColAttribute called with attribute 'ConciseType' for column '5' reported 12.
DEBUG SQLColAttribute called with attribute 'DisplaySize' for column '5' reported 10.
DEBUG Relational type of column 4: Varchar { length: Some(10) }
INFO Column 'User_-_iD'
Bytes used per row: 8
INFO Column 'FirstName'
Bytes used per row: 409
INFO Column 'LastName'
Bytes used per row: 20009
INFO Column 'Age'
Bytes used per row: 26
INFO Column 'companyid'
Bytes used per row: 49
INFO Column 'time_stäm'
Bytes used per row: 16
INFO Column '__timestamp'
Bytes used per row: 24
INFO Column '__is_deleted'
Bytes used per row: 9
INFO Column '__is_full_load'
Bytes used per row: 9
INFO Total memory usage per row for single transit buffer: 20559
SELECT [User - iD] AS [User_-_iD], [FirstName] AS [FirstName], [LastName] AS [LastName], [Age] AS [Age], [companyid] AS [companyid], CAST([time stämp] AS BIGINT) AS [time_stämp], CAST(GETUTCDATE() AS datetime2(6)) AS __timestamp, CAST(0 AS BIT) AS __is_deleted, CAST(1 AS BIT) AS __is_full_load FROM [dbo].[user]
User_-_iD: int64 not null
FirstName: string
LastName: string
Age: decimal128(15, 3)
companyid: string not null
time_stäm: int64
__timestamp: timestamp[us]
__is_deleted: bool
__is_full_load: bool
DEBUG SQLFreeHandle dropped 0x55c451ef1b20 of type Stmt.
DEBUG SQLFreeHandle dropped 0x55c451f30050 of type Dbc.

As a workaround I'll give it a try to supply larger buffers than requested by the drivers ...

To be fair to Microsoft, the function in question SQLDescribeCol is document to return the length in characters not bytes. https://learn.microsoft.com/en-us/sql/odbc/reference/syntax/sqldescribecol-function?view=sql-server-ver16. Yet it is clearly intended to determine buffer lengths (usually expressed in bytes). Driver may be "fine" and the ODBC standard is broken here. Well, anyhow, seems in the narrow case I'll start multiplying its values by 3 and for wide code by 2, this should put it all on the safe side even in worst case scenarios. I need to fix this upstream in odbc-api though.

Hm, interesting. Would you think I can reproduce this behavior if I write it in C#/.Net Core using MSFT-only libs?

Maybe not, depends if ADO relies on ODBC. Also relies on how wasteful authors of these libs are then using memory. I could avoid a lot of trouble if I would just allocate 4kb of buffer for each column name you wanna now. In that case you the name would need to be very long and contain special characters.
A minimal reproducing example in plain C, maybe based on an example for SQLDescribeCol would be feasable though. Just watch the string length output argument for a column name whose name differ in byte and character length.

NameLengthPtr
[Output] Pointer to a buffer in which to return the total number of characters (excluding the null termination) available to return in *ColumnName. If the number of characters available to return is greater than or equal to BufferLength, the column name in *ColumnName is truncated to BufferLength minus the length of a null-termination character.

This formulation alone in the documents is problematic

Definitly this. If you want to see it fail under windows too, you can use a character in the column name which is 4 Bytes in UTF-16 instead of two. E.g. 𐐏. Also MariaDB has the same bug; PostgreSQL is fine though.

Sadly allocating larger buffers is not a suitable workaround. The drivers only copy the bytes in the length of character data in the application provided buffers. The "good" news is that this makes the bug more relevant to fix.
Sorry, I cannot help here. I really tried.

Thank you for all your efforts! I'll try creating a repo in C# and getting this to the correct guys at Redmond - it's an interesting bug, after all 🙂

Probably not even needed, just stating that SQLDescribeCol returns truncates column names those binary length exceeds the character length, is a precise description, which should enable them to reproduce.

Remark: Using SQLColAttribute instead of SQLDescribeCol might work.


Annotations file arrangement

Added a new create_and_link_node function in node.py to replace the compile method in task, launch plan, and workflow.
Fix the file handling behavior
Removed the native_kwargs_to_literal_map in launch plan cuz I realized we can use the translate_inputs_to_literals function instead.
Added a ReferenceTask, no workflows or launch plans yet.
Started moving things around.  Removed all the flytekit imports from setup.py and replaced with literals - we can work out a better solution in the future, maybe ask around for suggestions.
Moved logger out of flytekit init into a separate file, but imported it back into init, along with most things users will need.
Made typing a folder and moved flytefile into a file.  Renamed FlyteFilePath to FlyteFile

lgtm


Fix mismatched assignment of URL/token

In the future, we need to be adding tests for this new functionality

ty!


5.2.23 Arr::get backwards compat - array_key_exists() expects parameter 2 to be array

The changes in v5.2.23 changed the behavior of Arr::get(), thus breaking the fairly popular Laravel Former package I maintain.
In Laravel 5.2.23, Arr::get() no longer supports null in its $array argument:
$ composer require laravel/framework 5.2.22 # [ output snipped ]
$ echo 'array_get(null, 4);' | php artisan tinker
Psy Shell v0.7.2 (PHP 5.6.15 — cli) by Justin Hileman
=> null
$ composer require laravel/framework 5.2.23 # [ output snipped ]
$ echo 'array_get(null, 4);' | php artisan tinker
Psy Shell v0.7.2 (PHP 5.6.15 — cli) by Justin Hileman
PHP warning:  array_key_exists() expects parameter 2 to be array, null given in
  [path snipped]/vendor/laravel/framework/src/Illuminate/Support/Arr.php on line 145

While we can debate the merits of passing in null arrays to Arr::get(), this backwards compatibility breakage may affect a decent number of folks.

Reported to Former at https://github.com/formers/former/issues/514

@vlakoff would this be related to your various Arr changes?

I think this is because of https://github.com/laravel/framework/commit/41aec8abcc86e1bee4d72c3bb86173368458df11.
Before:

Test isset. False, so go on.
Enter the foreach loop which does static::accessible && static::exists. False, so return default.

After:

Test static::exists. Doesn't work because expects array or ArrayAccess.

 This is not expected! Note that previously it would have broken the same if Arr::has() had been used.
I should have a look at code of Former. But I repeat, you shouldn't have passed null to Arr::get()!

Thank you for unexpected breaking change on a patch release.

@vlakoff well obviously, but the previous behaviour did allow for it (and people will do it). All of this should just be changed in 5.3.

PR #12726.

The PR is merged. Waiting for the new release.


Keyboard shortcut

Is it possible to set a keyboard shortcut to toggle between the states?

Yes, from the about:addons page you click on the gear icon and in that menu “Manage extension shortcuts”, You should see Toggle Dark Mode in that list, and you can set the shortcut there.

Thank you, have learned something, had no idea about that shortcuts section.


[Event request] Table 36 - Applies-to Doc. No. - OnLookup

To port a NAV add-on to BC, we need an event added to the OnLookup trigger for the Applies-to Doc. No. field in table 36.  We need an event for OnAfterSetApplyToFilters right after the line:
CustLedgEntry.SetApplyToFilters(...
We need the event to pass in the CustLedgEntry variable so we can apply an additional filter to it.

Some of my issues newer than this one have been looked at/addressed, so i just want to check if this one is still going to be looked at.  We're hoping to be able to release our app for the May release.


changed submenu and content

Added topics to python-basics documentation also added content similar to w3 schools to intro-to-python React component and styled it using CSS! Made it responsive!

Some of the contents are not added can you please fill up those
Thank You
Instructions

Yes you can start working on it by taking latest pull from main branch
Don't Forget to join onboarding group for knowledge flow and direct conversation and support from other developers link  Curious Developers Community - Hacktoberfest
Please attach the screenshots and video of the development work once it is done
Contributions more than 3 will be awarded a certificate of appreciation from curious ecosystem please mail to<EMAIL_ADDRESS>for claiming certificate and please attach your issue number
Respect Co Developers And Please Reachout to admins if any issues

Thanks & Regards
The Curious Team

will fix the sidebar issue as well as attach screenshots!


Websocket Disconnected Issue

I found this issue while testing application on slow internet connections (well, it happens on all connections, it's just easier to reproduce when connection is slow enough). Here're steps to reproduce:

Open an app that establishes connection to meteor. Since it's a first launch, I handle 'wait until meteor connected' in app, so that no code can make meteor requests until app is connected, logged in, blablabla.
Minimize the app (without killing it) and wait few minutes - 5 minutes was enough in my case.
Open an app and initiate Meteor request (method call in my case). Since there are no events that say when Meteor client connected/disconnected/errored out, there is no way to monitor that status in my code. And in this case, logs specify that there was an error and the state is actually disconnected.
Callback is never called. My app is waiting for response indefinitely.

I see two issues with SwiftDDP related to this topic:

Function client.sendMessage. It calls directly self.socket.send(m) and doesn't check if connection is opened. Inside SwiftSocket code only sends a message when connection is opened. Otherwise, message is not sent and callback is not called, obviously.
Events. There's no way to track websocket events. The only events there are: User logged in, user logged out. I wish that client.events could be public and clients had an ability to hook to them. I saw that you are using DDPEvents in DDPClient, which prevents exposing it to the world. But those NSNotifications would be very helpful.

Yury

I see, that you are only using 'onConnected' event. If it doesn't break the rest of the code, I could easily fork and add those Notification Center events to those callbacks. Let me know.

Yea, I see what you're saying. I think the problem is minimizing the app. SwiftDDP has never really handled that well.
I agree, it might be good to have notifications for other websocket events.

You are right, but it's totally ok. If app knows what's going on with connection, it can control Meteor interaction.


add identifier naming rules

Fixes #3569
This contains a two related changes:

Create the new identifier rules article.
Add links from articles that create identifiers

While working on those changes, made the following changes:

cleanup links and whitespace
Reorder the Programming Guide TOC (it was alphabetical)

Summary
Describe your changes here.
Fixes #Issue_Number (if available)

Thanks @rpetrusha @mairaw @pkulikov I addressed all the feedback.


Use Groovy compiler options from Gradle Api

This patch uses Gradle Api for Groovy Compiler(since Gradle 6) to setup compilation with parameters.

Thanks for the contribution!


single-line store_listeners

Testing

Trade-offs

Dependencies

Screenshots

:tada: This PR is included in version 2.154.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update ocis reva + revert selenium

Combination of https://github.com/owncloud/ocis/pull/259 and https://github.com/owncloud/ocis/pull/254 to fix CI

still failing:
Error: Could not create the file "test_sort/a space 文件" for user "user1". Status:500 Internal Server ```

CC @dpakach

@individual-it any update on the test issues ?

it seems to be an issue with the updates, that started 4 days ago @dpakach is investigating

closing as the Selenium thing did not help, going back to https://github.com/owncloud/ocis/pull/259


Setup the deployment server conveniently and without requirements of internet connections

Before using kolla-ansible to deploy OpenStack services, the user needs to first setup the docker image registry and package repositories on the deployment server.
User needs a convenient and clear path to finish these steps, and make the deployment process does not rely on internet connections.

You may refer to how ICp made this, seems they have some docker image and put all stuffs in that image and do some scripts against the image... need dig more
another way would be ship the registry server anyway and build the registry before start deploy?

You may refer to how ICp made this, seems they have some docker image and put all stuffs in that image and do some scripts against the image... need dig more
another way would be ship the registry server anyway and build the registry before start deploy?

Thanks for the suggestion. yes, ICP is installed with a docker image. For CE, I didn't use this method since a script is just enough and convenient for user to setup all the needed stuff, building a docker image introduce extra efforts.
The registry images are all built before using this tool to setup the deployment server.

Did POC and Finish a script tool to automatically setup all the required stuffs on the deployment server.
Tool committed in the following patch:
https://github.com/mfcloud/kolla-ansible/commit/7de5fb2800fcdc05a70832790d785dd10e780fa7
Setups included in this tool:

Install docker-ce and run docker service.
Load required docker images used to run local repo server.
Run local repo server, PYPI server, docker registry
Update deployment server to use local repos
Install kolla-ansible and its dependencies.

So this issue can be closed.

Main process recorded in the following boxnote:
https://ibm.ent.box.com/notes/318354553234


Add configuration values for the latest features

What does this PR do?
Adds configuration values for the latest features:

cc.api_batch_size
nozzle.org_data_collection_interval

Motivation
What inspired you to submit this pull request?
Testing Guidelines
An overview on testing
is available in our contribution guidelines.
Additional Notes
Anything else we should know when reviewing?

Thanks for the review, merging.


Dart Style Linter

This issue was originally filed by @seaneagan

For semantic code issues (hints, warnings, errors) we have dartanalyzer.
For code style issues, we have dartfmt.
What's missing that I'm looking for is something which has the same kind of return code and output as dartanalyzer:
* type of rule broken
* id of rule broken
* location of rule break: file, line/character range
* message
... but instead checks the same rules as dartfmt, i.e. the dart style guide rules.
It could be a separate program, or maybe just an additional output option to dartfmt.
Such a program could then be integrated into many different editors, such as Sublime, Atom, Chrome Dev Editor (assuming it's written in dart), and the Dart Editor, to give a visualization of style issues, and allow selective/iterative fixing of them.
It could also be used as a build task to ensure code style before continuing with the build, both during development and in a continuous integration environment.
It would be important to have some amount of configurability of specific rules to include/exclude, or groups of rules to include/exclude, such as excluding the PREFER/AVOID rules or equivalently only including DO/DONT rules.  This same configuration could be used by dartfmt to do selective formatting.  It would probably be something like a .dartfmtrc in the project root i.e. something that is shared with a team via source control.  Might be nice to integrate with .editorconfig (http://editorconfig.org) as well.
Some prior art:
JavaScript: https://github.com/mdevils/node-jscs
Java: http://checkstyle.sourceforge.net/

The linter repo has moved to https://github.com/dart-lang/linter, is going strong and integrated into the command-line analyzer and server, with configuration support via .analysis_options.  Closing out this meta-bug in favor of more targeted ones moving forward.


#trivial add location of sous config

add location of config.yaml to sousConfigHelp

  Generated by :no_entry_sign: Danger

Codecov Report

Merging #534 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #534   +/-   ##
=======================================
  Coverage   68.78%   68.78%
=======================================
  Files         202      202
  Lines       11097    11097
=======================================
  Hits         7633     7633
  Misses       2911     2911
  Partials      553      553

Impacted Files
Coverage Δ

cli/sous_config.go
35% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc25b49...a645aa4. Read the comment docs.

Strictly speaking, we compute a path for the config file based on environment variables and conventions (https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html) Usually that means the config file is at ~/.config/sous/config.yaml but if it's somewhere different, it'd be good if sous config could accurately report that.
I'd be okay merging this with an eye towards refining it in a near-future UX concentrated session.

This will be true 99% of the time, so merging for now, but we should make it dynamic, or tell the user how to work it out themselves...


[Bug]: Error: Text strings must be rendered within a  component.

Provide environment information
"react-text-to-speech": "^0.14.6",
Link to reproduction - Issues with a link to complete (but minimal) reproduction code help us address them faster
npm install react-text-to-speech --save
To reproduce
After implementing this in typescript code getting below issue
I was trying to run the basic code which is mentioned in github
Code:
import React from "react";
import { useSpeech } from "react-text-to-speech";
export default function App() {
const {
Text, // Component that returns the modified text property
speechStatus, // String that stores current speech status
isInQueue, // Boolean that stores whether a speech utterance is either being spoken or present in queue
start, // Function to start the speech or put it in queue
pause, // Function to pause the speech
stop, // Function to stop the speech or remove it from queue
} = useSpeech({ text: "This library is awesome!" });
return (

{speechStatus !== "started" ? Start : Pause}
Stop

);
}
Error
Error: Text strings must be rendered within a  component.
This error is located at:
Describe the bug
Error
Error: Text strings must be rendered within a  component.
This error is located at:
Expected behavior
It should work as you mentioned in the doc.
Screenshots (if relevant)
No response
Additional information (if relevant)
No response

Hmm looks weird... It's working absolutely fine on my end.
Are you using plain React.js or any framework like Next.js or Remix.js?
Also, can you provide any other important information that I should be knowing regarding this issue?

I am using react native. Please check the below version
"react": "18.2.0",
"react-dom": "18.2.0",
"react-native": "0.72.4",

Ohh, now I got it.
Actually, react-text-to-speech uses the WebSpeech API available in modern browsers as its backbone. Since this API is not available on Android and iOS, I never targetted and tested this library for react-native. And this is probably the reason behind your issue.
Fortunately, there are some other libraries that provide text-to-speech functionality in react-native like react-native-tts or react-native-voice. Although it seems like both of them are no longer maintained, but I guess you should give them a try.


Nokogiri < 1.8.1

After checking in to mater, I got the following GitHub error:

We found a potential security vulnerability in one of your dependencies.
The nokogiri dependency defined in Gemfile.lock has a known critical severity security vulnerability in version range < 1.8.1 and should be updated.

I need to run bundle update and then checkin the resulting Gemfile.lock file.

The error disappeared after I checked in the updated Gemfile.lock, so I'm closing this for now.


Applying NBRDF with Mitsuba3

Dear Dr. Sztrajman
I'm Now trying to use this NBRDF with Mitsuba3, because old version of Mitsuba is not supported to be installed on recent environments.
There are two problems for applying NBRDF to Mitsuba3.

xml format is different
Since Mitsuba3 has different dependencies and components, xml format is different with that of NBRDF

cannot directly import npy
Since nbrdf_npy is only compatible with Mitsuba3, there is no known way to directly import npy for Mitsuba3

So I'm having these problems for applying NBRDF to Mitsuba3, and applying NBRDF for further research.
I hope you can help me with this :)
Thank You
Sincerely,
Chanho

Hi Chanho,
Thanks for the interest in the project. I have plans to update the code to be able to run in Mitsuba 3, but I don't know when I'll have the time for this unfortunately.

xml format is different
Since Mitsuba3 has different dependencies and components, xml format is different with that of NBRDF

My guess is that this should not be hard to adapt for the new Mitsuba version. But let me know if you find specific issues and I'll try to help.

cannot directly import npy
Since nbrdf_npy is only compatible with Mitsuba3, there is no known way to directly import npy for Mitsuba3

This is harder since it will likely require a rewrite of the plugin. As I said: I will try to do it when I have the time, but can't anticipate when. If you make any progress with the translation to Mitsuba3 I'll be happy to add it to the repo.
Best,
Alejandro

Thank you for your reply. I currently succeeded rendering with mitsuba3 and have some problems with autoencoder training and backpropagation through NBRDF. I think it will be a great help if I can refer autoencoder of NBRDF, but I think I cannot find any in this repository. Is there any reference or code that I can refer for this problem?

Thank you for your reply. I currently succeeded rendering with mitsuba3 and have some problems with autoencoder training and backpropagation through NBRDF. I think it will be a great help if I can refer autoencoder of NBRDF, but I think I cannot find any in this repository. Is there any reference or code that I can refer for this problem?

I successfully installed mitsuba0.6 on Ubuntu20.04 following https://github.com/dat-boi-lux/mitsuba-0.6.0-modern-linux.


Starter Kit link in readme

Issue type
Document link incorrect?
Issue description
Current behavior:

Empty starter kit
Don't need all the pages and modules and just looking for an empty starter kit for your next project? Check out our starter-kit branch.

Expected behavior:
The starter kit link should be https://github.com/ngx-rocket/starter-kit

Hi @kitex, why do you think we should link a third party project instead of the ngx-admin starter kit?

@nnixaa No. Just wondering both
https://github.com/akveo/ngx-admin
https://github.com/akveo/ngx-admin/tree/starter-kit
have same content right.

The content is different, starter-kit branch doesn't have all of the pages and components.