SslHandler does not send SSL close notify after receiving an invalid client certificate

Our server application implements a simple SSLHandler by extending SslHandler, and implements another handler (ServerHandler) for application message/exception handling. We are trying to get Netty SSLHandler to send an SSL close notify message before closing the channel.
However, whenever our client application sends an invalid SSL certificate, it never receives ssl close notify message before TCP reset. In "exceptionCaught" method of SSLHandler we invoke "close" method of SslHandler and in "exceptionCaught" method of ServerHandler we invoke "ExceptionEvent.getChannel().close()" method.
According to https://netty.io/4.0/api/io/netty/handler/ssl/SslHandler.html#close(), "...the close() method should be called to send the close_notify message to the remote peer. One exception is when you close the Channel - SslHandler intercepts the close request and send the close_notify message before the channel closure automatically...". However, it does not seem to work as expected.
Below are few log snippets:
TLS: Unable to verify Client Certificate for<EMAIL_ADDRESS><EMAIL_ADDRESS>OU=Enterprise, O=somewhere, L=Sunrise, ST=FL, C=US; with Algorithm: SHA256withRSA; AuthType: RSA; error: sun.security.validator.ValidatorException: No trusted certificate found
New I/O worker #1, fatal error: 46: General SSLEngine problem
sun.security.validator.ValidatorException: No trusted certificate found
%% Invalidated:  [Session-1, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
New I/O worker #1, SEND TLSv1 ALERT:  fatal, description = certificate_unknown
New I/O worker #1, WRITE: TLSv1 Alert, length = 2
New I/O worker #1, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLHandshakeException: General SSLEngine problem
New I/O worker #1, called closeOutbound()
New I/O worker #1, closeOutboundInternal()
New I/O worker #1, called closeInbound()
New I/O worker #1, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
ServerSSLHandler: exceptionCaught: Exception=javax.net.ssl.SSLHandshakeException: General SSLEngine problem
ServerSSLHandler: exceptionCaught: closing SSL handler
New I/O worker #1, called closeOutbound()
New I/O worker #1, closeOutboundInternal()
[Raw write]: length = 7
0000: 15 03 01 00 02 02 2E                               .......
ServerSSLHandler: exceptionCaught: closing SSL handler
ServerHandler: exceptionCaught. Exception: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
ServerHandler: exceptionCaught: closing channel
ServerHandler: channelDisconnected
New I/O worker #1, called closeOutbound()
New I/O worker #1, closeOutboundInternal()
ServerHandler: channelClosed
I can provide the code snippets of my sample application if required.

@SutarSuhas code snippets would help. Also do you use the OpenSSL bindings or not ?

@SutarSuhas ping

@SutarSuhas any news here ?

I am using Netty (version 4.0.27-final) with OpenSSL implementation provided by Netty. What i have observed that shutdown behavior in bad case is not clean. For example netty acting as Server have to verify the peer certificate and verification is failed e.g. unknown_certificate. OpenSSL sets a handshake error with an alert message "unknown_certificate". But this alert is not sent to the peer. I traced with wireshark. The following code in OpenSSLEngine.unwrap method throws the exception:
int lastPrimingReadResult = SSL.readFromSSL(ssl, EMPTY_ADDR, 0); // priming read
    // check if SSL_read returned <= 0. In this case we need to check the error and see if it was something
    // fatal.
    if (lastPrimingReadResult <= 0) {
        // Check for OpenSSL errors caused by the priming read
        long error = SSL.getLastErrorNumber();
        if (OpenSsl.isError(error)) {
            String err = SSL.getErrorString(error);
            if (logger.isDebugEnabled()) {
                logger.debug(
                        "SSL_read failed: primingReadResult: " + lastPrimingReadResult +
                                "; OpenSSL error: '" + err + '\'');
            }

            // There was an internal error -- shutdown
            shutdown();
            throw new SSLException(err);
        }
    }

The exception is thrown because OpenSSL has already set an error for verification failed. But there is still data (alert data with first byte 21, can see from eclipse debug) in BIO that has not been transferred.
How can i resolve the issue? For my application a clean shutdown is a critical requirement.

I will have a look once back from vacation

In case if error is there and Network bio has pending data(in this case alert data), i tried just return with NEED_WRAP and in next wrap() the pending data was consumed and written to wire. I can see the Alert data in Wireshark trace. This was just a quick test. We have to consume the pending bio data and afterwards exception must be thrown to inform SSLHandler about the problem.
 if (lastPrimingReadResult <= 0) {
        // Check for OpenSSL errors caused by the priming read
        long error = SSL.getLastErrorNumber();
        if (OpenSsl.isError(error)) {
            String err = SSL.getErrorString(error);
            if (logger.isDebugEnabled()) {
                logger.debug(
                        "SSL_read failed: primingReadResult: " + lastPrimingReadResult +
                                "; OpenSSL error: '" + err + '\'');
            }
           if (pendingWrittenBytesInBIO(networkBIO) > 0) {
	         return NEED_WRAP_OK;
           } else {
	         // There was an internal error -- shutdown
                 shutdown();
                 throw new SSLException(err);
           }
}

If you can provide a fix as pr I'm more then happy to review

Am 04.12.2015 um 12:09 schrieb ramahmoo<EMAIL_ADDRESS>In case if error is there and Network bio has pending data(in this case alert data), i tried just return with NEED_WRAP and in next wrap() the pending data was consumed and written to wire. I can see the Alert data in Wireshark trace. This was just a quick test. We have to consume the pending bio data and afterwards exception must be thrown to inform SSLHandler about the problem.
if (lastPrimingReadResult <= 0) {
// Check for OpenSSL errors caused by the priming read
long error = SSL.getLastErrorNumber();
if (OpenSsl.isError(error)) {
String err = SSL.getErrorString(error);
if (logger.isDebugEnabled()) {
logger.debug(
"SSL_read failed: primingReadResult: " + lastPrimingReadResult +
"; OpenSSL error: '" + err + ''');
}
if (pendingWrittenBytesInBIO(networkBIO) > 0) {
return NEED_WRAP_OK;
} else {
// There was an internal error -- shutdown
shutdown();
throw new SSLException(err);
}
}
—
Reply to this email directly or view it on GitHub.

@wolftobias what version of netty this is ?

This is 4.0.27-final  but we have tested with 4.0.33-fonal and same behavior.

@normanmaurer Any update on this issue?

@ramahmoo I think I will open a PR today.

I saw a similar thing in last week commit at
https://github.com/apache/tomcat/blob/trunk/java/org/apache/tomcat/util/net/openssl/OpenSSLEngine.java
They have used the sendHandshakeError flag to mark that there is some pending BIO data when handshake error occurred and explicitly return NEED_WRAP from getHandshakeStatus() if sendHandshakeError was true and set this flag to false. i.e. they try to invoke wrap one time after the ssl handshake failure. I adopted the above in Netty OpenSSLEngine, i can see wrap() is called one time by Netty SSLHandler after handshake error but

could not see the Alert in wireshark trace even there were alert bytes produced by wrap. This wrap call return back Finish state
netty assumes that everything went smooth with handshake

@ramahmoo just one thing... how did you check if the alert data was send back ? Wireshark and if so how ? Just looked at the bytes ?

Just looked the Bytes inside the Wrap, as per rfc5246 first byte found was 21 which indicates that it is alert data.

@ramahmoo I was more interested to understand how you detected you not received the alert data.

traced with wireshark, see the following wireshark trace screenshot

here is the result with Java socket based impl

@ramahmoo thanks!

@ramahmoo one last question... you are using OpenSslEngine on the server or client side when you see the issue ?

As client, but i think is also as server.

@ramahmoo when you said "using the JDK one" you are refer to use SslProvider.JDK and still netty right ?

No, just a simple java ssl socket based handshake test without netty.

can you share the code ?

On 15 Dec 2015, at 12:28, ramahmoo<EMAIL_ADDRESS>wrote:
No, just a simple java ssl socket based handshake test without netty.
—
Reply to this email directly or view it on GitHub https://github.com/netty/netty/issues/3900#issuecomment-164735612.

Test Client based on socket
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.URL;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.UnrecoverableKeyException;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.KeyManager;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509ExtendedKeyManager;

import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;

public class TestHttpsConnection {

	public static final String TEST_URL = "https://test-server:8443/test.html";
	public static final String TEST_REQUEST = "Test Request";
	public static KeyManager[] kms;
	public static TrustManager[] tms;

	@BeforeClass
	public static void init() throws Exception {

		// load client private key
		KeyStore clientKeys = KeyStore.getInstance("PKCS12");
		clientKeys
				.load(new FileInputStream(
						"test-keystore.p12"),
						"password".toCharArray());

		kms = getKeyManagers(clientKeys, "password".toCharArray(),
				"test-key-alias");

		// load server public key
		KeyStore serverPub = KeyStore.getInstance("JKS");
		serverPub
				.load(new FileInputStream(
						"test-truststore.jks"),
						"password".toCharArray());
		TrustManagerFactory trustManager = TrustManagerFactory
				.getInstance("SunX509");
		trustManager.init(serverPub);
		tms = trustManager.getTrustManagers();
	}

	private static KeyManager[] getKeyManagers(KeyStore keystore,
			char[] keyPassword, String keyAlias)
			throws NoSuchAlgorithmException, UnrecoverableKeyException,
			KeyStoreException {

		KeyManagerFactory keyManagerFactory;
		KeyManager[] keyManagers;
		KeyManager keyManager;

		keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory
				.getDefaultAlgorithm());
		keyManagerFactory.init(keystore, keyPassword);
		keyManagers = keyManagerFactory.getKeyManagers();

		if (keyAlias == null || "".equals(keyAlias))
			throw new KeyStoreException("There was no alias found!");

		if (keyManagers != null) {
			for (int i = 0; i < keyManagers.length; i++) {
				keyManager = keyManagers[i];
				if (keyManager instanceof X509ExtendedKeyManager) {
					keyManagers[i] = new AliasSelectorKeyManager(
							(X509ExtendedKeyManager) keyManager, keyAlias);
				}
			}
		}

		return keyManagers;
	}

	@Test
	public void testSSLClientWithSocket() {
		HttpsURLConnection httpsURLConnection = null;
		try {
			URL url = new URL(TEST_URL);
			SSLContext sc = getSSLContext();
			SSLSocketFactory socketFactory = sc
					.getSocketFactory();

			HttpsURLConnection.setDefaultSSLSocketFactory(socketFactory);
			HttpsURLConnection
					.setDefaultHostnameVerifier(new HostnameVerifier() {
						@Override
						public boolean verify(String arg0, SSLSession arg1) {
							return true;
						}
					});
			httpsURLConnection = (HttpsURLConnection) url.openConnection();
			httpsURLConnection.setDoOutput(true);
			httpsURLConnection.setRequestMethod("POST");
			httpsURLConnection.setRequestProperty("Content-type",
					"text/xml; charset=utf-8");
			httpsURLConnection.setRequestProperty("Connection", "close");
			OutputStream reqStream = httpsURLConnection.getOutputStream();
			reqStream.write(TEST_REQUEST.getBytes());

			StringBuffer sb = this.getContent(httpsURLConnection
					.getInputStream());

			if (sb != null) {
				Assert.assertTrue(sb.length() > 0);
			} else
				Assert.fail();
		} catch (Exception e) {
			e.printStackTrace();
			Assert.fail();
		} finally {
			if (httpsURLConnection != null)
				httpsURLConnection.disconnect();
		}
	}

	private StringBuffer getContent(InputStream inputStream) {
		if (inputStream != null) {
			try {
				StringBuffer sb = new StringBuffer();
				String s;
				BufferedReader br = new BufferedReader(new InputStreamReader(
						inputStream));

				while ((s = br.readLine()) != null) {
					sb.append(s);
				}
				br.close();
				return sb;
			} catch (IOException e) {
				e.printStackTrace();
			}
		}

		return null;
	}

	private SSLContext getSSLContext() throws Exception {
		SSLContext sc = SSLContext.getInstance("TLSv1.2");
		sc.init(kms, tms, new SecureRandom());
		return sc;
	}
}

Alias Selector Key Manager
import java.net.Socket;
import java.security.Principal;
import java.security.PrivateKey;
import java.security.cert.X509Certificate;

import javax.net.ssl.SSLEngine;
import javax.net.ssl.X509ExtendedKeyManager;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class AliasSelectorKeyManager extends X509ExtendedKeyManager {

	private static Logger log = LogManager
			.getLogger(AliasSelectorKeyManager.class);

	private final X509ExtendedKeyManager sourceKeyManager;
	private final String expectedAlias;

	public AliasSelectorKeyManager(X509ExtendedKeyManager keyManager,
			String expectedAlias) {
		if (keyManager == null)
			throw new IllegalArgumentException("Source Keymanger can not be null!");
		this.sourceKeyManager = keyManager;
		this.expectedAlias = expectedAlias;
	}

	@Override
	public String chooseEngineServerAlias(String keyType, Principal[] issuers,
			SSLEngine engine) {
		log.entry(keyType, issuers, engine);
		return log.exit(expectedAlias == null ? this.sourceKeyManager
				.chooseEngineServerAlias(keyType, issuers, engine)
				: expectedAlias);
	}

	@Override
	public String chooseEngineClientAlias(String[] keyTypes,
			Principal[] issuers, SSLEngine engine) {
		log.entry(keyTypes, issuers, engine);
		return log.exit(expectedAlias == null ? this.sourceKeyManager
				.chooseEngineClientAlias(keyTypes, issuers, engine)
				: expectedAlias);
	}

	@Override
	public String chooseServerAlias(String keyType, Principal[] issuers,
			Socket socket) {
		log.entry(keyType, issuers, socket);
		return log.exit(expectedAlias == null ? this.sourceKeyManager
				.chooseServerAlias(keyType, issuers, socket) : expectedAlias);
	}

	@Override
	public String chooseClientAlias(final String[] keyTypes,
			Principal[] issuers, final Socket socket) {
		log.entry(keyTypes, issuers, socket);
		return log.exit(expectedAlias == null ? this.sourceKeyManager
				.chooseClientAlias(keyTypes, issuers, socket) : expectedAlias);
	}

	@Override
	public X509Certificate[] getCertificateChain(String alias) {
		log.entry(alias);
		return log.exit(sourceKeyManager.getCertificateChain(alias));
	}

	@Override
	public String[] getClientAliases(final String keyType,
			final Principal[] issuers) {
		log.entry();
		return log.exit(this.sourceKeyManager
				.getClientAliases(keyType, issuers));
	}

	@Override
	public PrivateKey getPrivateKey(String alias) {
		log.entry();
		return log.exit(sourceKeyManager.getPrivateKey(alias));
	}

	@Override
	public String[] getServerAliases(String keyType, Principal[] issuers) {
		log.entry();
		return log.exit(sourceKeyManager.getServerAliases(keyType,
				issuers));
	}

}

@ramahmoo thanks again for all your help. With your help I was able to reproduce it and fix it :+1: (at least it seems so). I want to do some more tests here but expect a PR today or tomorrow.

@ramahmoo can you please verify if the fix works for you as well:
https://github.com/netty/netty/pull/4578
After I apply it I can see everything in wireshark :)

@normanmaurer
Unfortunately, we can't take this fix from version 4.0.3x while there was a high CPU usage(maybe somewhere a Race condition) reported with OpenSSLEngine implementation from this version. We are still using 4.0.27-final. Is there any chance of porting the fix to 4.0.27 version?

@ramahmoo unfortunally no... we will not back port the fix as a lot of internal changed. Any chance to open an issue with the details of the race ? also are you sure it is related to the OpenSslEngine ?

@ramahmoo btw you could try to port the change by yourself but OpenSslEngine has other problems in 4.0.27.Final

@SutarSuhas maybe you could have a look as well.

Fixed via https://github.com/netty/netty/pull/4578

@ramahmoo if you find another issue in the latest version of OpenSslEngine please open another issue.

@normanmaurer tested today with the revision where fixed. But could not see the alert(s) in wireshark. In handshake() NEED_WRAP is returned if something is pending in BIO. In next handshake() call the last stored exception is thrown if there is not more pending data. Can you attach your wireshark trace, test case and any java log file?

I found that Netty was flushing the TLS alert data filled by last wrap() call if subsequent wrap() call throws an SSLException. In catch case of the following method of SSLHandler, i just added ctx.flush() and i can see the now TLS alert data in wireshark trace as expected. @normanmaurer any comment?
private void wrapNonAppData(ChannelHandlerContext ctx, boolean inUnwrap)
			throws SSLException {
		ByteBuf out = null;
		ByteBufAllocator alloc = ctx.alloc();
		try {
			for (;;) {
				if (out == null) {
					out = allocateOutNetBuf(ctx, 0);
				}
				SSLEngineResult result = wrap(alloc, engine,
						Unpooled.EMPTY_BUFFER, out);

				if (result.bytesProduced() > 0) {
					ctx.write(out);
					if (inUnwrap) {
						needsFlush = true;
					}
					out = null;
				}

				switch (result.getHandshakeStatus()) {
				case FINISHED:
					setHandshakeSuccess();
					break;
				case NEED_TASK:
					runDelegatedTasks();
					break;
				case NEED_UNWRAP:
					if (!inUnwrap) {
						unwrapNonAppData(ctx);
					}
					break;
				case NEED_WRAP:
					break;
				case NOT_HANDSHAKING:
					setHandshakeSuccessIfStillHandshaking();
					// Workaround for TLS False Start problem reported at:
					// https://github.com/netty/netty/issues/1108#issuecomment-14266970
					if (!inUnwrap) {
						unwrapNonAppData(ctx);
					}
					break;
				default:
					throw new IllegalStateException(
							"Unknown handshake status: "
									+ result.getHandshakeStatus());
				}

				if (result.bytesProduced() == 0) {
					break;
				}

				// It should not consume empty buffers when it is not
				// handshaking
				// Fix for Android, where it was encrypting empty buffers even
				// when not handshaking
				if (result.bytesConsumed() == 0
						&& result.getHandshakeStatus() == HandshakeStatus.NOT_HANDSHAKING) {
					break;
				}
			}
		} catch (SSLException e) {
			/** TLS ALERT FIX START */
			ctx.flush();
			/** TLS ALERT FIX END */
			setHandshakeFailure(ctx, e);
			throw e;
		} finally {
			if (out != null) {
				out.release();
			}
		}
	}

@ramahmoo ah makes sense! I think it always worked here as I called ctx.flush()in channelReadComplete(...). Could you do a quick PR for this after sign our ICLA (if you not done this already):
http://netty.io/s/icla
Also please ensure you respect our commit message template:
netty.io/wiki/writing-a-commit-m…

@ramahmoo actually let me take a stab. There are a few other places which needs some attention.


Remove crcmod from direct dependencies

It is no longer in use.

Eliminates a binary dependency with no Linux wheels available on PyPI
The binary extension build process introduces non-reproducibility into the build
Developers should use crc32c instead where needed

Codecov Report

Merging #323 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #323   +/-   ##
=======================================
  Coverage   85.05%   85.05%
=======================================
  Files          37       37
  Lines        1733     1733
=======================================
  Hits         1474     1474
  Misses        259      259

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc70817...56853ef. Read the comment docs.

Added @maniarathi as a  reviewer.  This module should have been removed when its use was replaced by crc32c.


Composite operations on chunked array types not dispatched through chunk manager

What happened?
The arkouda-xarray chunk manager (found here) is designed to allow Arkouda's distributed arrays to operate as chunked arrays in XArray. (Note Arkouda's array's also implement the array API, which was being used for XArray interoperability before the chunk manager was created).
When executing various operations on XArray DataArrays or DataSets that are backed by the Arkouda chunk manager, I noticed that they are not being routed through the chunk manager itself. For example, calling myArkoudaBackedArray.mean(axis="some-axis") does not get routed through ArkoudaManager.reduction. However, the operation is successful and produces the correct result, indicating that the array-api implementation of mean is being used instead (i.e., the arkouda-backed array is being treated as a duck array).
https://github.com/jeremiah-corrado/arkouda-xarray-seasonal-avgs-example/blob/main/README.md contains a full example where I observed the above behavior. It includes instructions to get Arkouda set up for XArray interoperability.
pinging @TomNicholas, since we discussed this the other day.
MVCE confirmation

[ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[ ] Complete example — the example is self-contained, including all data and the text of any traceback.
[ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[ ] New issue — a search of GitHub Issues suggests this is not a duplicate.
[ ] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output
No response
Anything else we need to know?
No response
Environment

INSTALLED VERSIONS
commit: None
python: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:34:54) [Clang 16.0.6 ]
python-bits: 64
OS: Darwin
OS-release: 22.6.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.2
libnetcdf: None
xarray: 2024.5.0
pandas: 2.2.2
numpy: 1.26.4
scipy: 1.13.1
netCDF4: None
pydap: None
h5netcdf: None
h5py: 3.8.0
zarr: None
cftime: 1.6.3
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.8.4
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 70.0.0
pip: 24.0
conda: None
pytest: None
mypy: None
IPython: 8.24.0
sphinx: None

looking at the docstring of the chunk manager base class, it appears this only works for a select number of reductions: https://github.com/pydata/xarray/blob/d26144db962d9d620c7f7c4e7398c2f6b3bf6d47/xarray/namedarray/parallelcompat.py#L386-L390
In other words, this seems to be expected behavior? I wonder if var.reduce(np.mean) would go through the reduction method, though.

In other words, this seems to be expected behavior?

Reminding myself of how this works, I think yes, this is expected. The array type knows how to do a chunked mean on a chunked array without any further guidance from xarray, so it's fine to just use the array API to call the correct implementation of mean.

I wonder if var.reduce(np.mean) would go through the reduction method, though.

It won't (see NamedArray.reduce)
https://github.com/pydata/xarray/blob/cde720f21788a1dbff2bdb0775a56b3ee6da6b1a/xarray/namedarray/core.py#L914
but I also don't think it needs to either. reduce is really a special case of the more general reduction primitive. reduction accepts the func kwarg that reduce accepts, but also accepts combine_func and aggregate_func. Basically these allow tree-reductions made of alternating steps, rather than just the same function applied over and over. This comes from dask.array.reduction, and is needed in the chunkmanager because it is used in xarray (only once though apparently, in the implementation .first and .last methods).
https://github.com/pydata/xarray/blob/cde720f21788a1dbff2bdb0775a56b3ee6da6b1a/xarray/core/duck_array_ops.py#L795
If arkouda chose not to implement ArkoudaChunkManager.reduction I think the only thing that would break would be the .first and .last methods. .reduce would in general work, but only if the function you passed into .reduce knew how to handle the Arkouda Array type.
So actually other than documenting / testing this a bit better I believe there is not actually a bug to be fixed here?

I think the only thing that would break would be the .first and .last methods. .reduce would in general work, but only if the function you passed into .reduce knew how to handle the Arkouda Array type.

Okay that makes sense. Thank you both for digging into the implementation/docs to figure that out. I'll work on implementing  reduction properly in Arkouda's chunk manager so that those methods work.

I believe there is not actually a bug to be fixed here?

I think that's correct


Remove @ts-ignore usage

Remove @ts-ignore usage that do not require big changes. I would love for @wemeetagain or experienced devs in this codebase to check if there's a reasoning for some of the @ts-ignore occurrences that I might be missing

@mpetrunic Can you take a look at these type errors? I'm not familiar with pouch DB

https://github.com/ChainSafe/lodestar/pull/1066/checks#step:5:381
https://github.com/ChainSafe/lodestar/pull/1066/checks#step:5:384

@mpetrunic Can you take a look at these type errors? I'm not familiar with pouch DB

https://github.com/ChainSafe/lodestar/pull/1066/checks#step:5:381
https://github.com/ChainSafe/lodestar/pull/1066/checks#step:5:384

I can check on monday. Maybe pouchdb types aren't match for pouchdb version we are using

I can check on monday. Maybe pouchdb types aren't match for pouchdb version we are using

We are using
"pouchdb-adapter-memory": "^7.0.0",
"pouchdb-core": "^7.0.0",

And types
"@types/pouchdb-adapter-memory": "^6.1.3",
"@types/pouchdb-core": "^7.0.6",

@types/pouchdb-adapter-memory is at its latest version so maybe it has not been updated for the latest major version. However the conflicting types come from @types/pouchdb-core

I think ts-ignores are mostly in tests or in places where we had bad types but were too lazy to fix them (especially libp2p parts)

Right, that's my understanding. Vast majority of ts-ignores should/can be removed with a bit of work.
I can only think of one reason for ts-ignore, which you've already encountered, dynamic events which can't be typed.
RE: pouchdb, seems like the declared types are just wrong. I'd be interested in revisiting if we even need the extra layer of DB abstraction, perhaps purging pouchdb and using level exclusively and concretely.

@wemeetagain  So for this PR would you recommend reverting the pouch DB commits and leave that part untouched?

Yeah, I think we can visit that separately. Will make an issue now

Besides for dynamic events, the other area where @ts-ignore is used and may be concerning is here
https://github.com/ChainSafe/lodestar/blob/7f69a144dcdc3b4c00f25d6c87ff8ae3c5aee24a/packages/lodestar-types/src/ssz/generators/index.ts#L30-L39
Do you think it's worth it to try to fix it?

Ah, not opposed to trying, that section of code is likely a little confusing to outside observers. The ts-ignore doesn't help.
Here, we're creating a big object of SSZ type objects, IBeaconSSZTypes, but there is a tree of heirarchy, where some type objects rely on other objects existing.  The reason we're ts-ignoring here is that we're using a list of object names, typeNames: string[], which is sorted in this 'dependency order'. And IBeaconSSZTypes is not able to be dynamically indexed by strings. Perhaps if we say types[type as keyof IBeaconSSZTypes]or declaretypeNamesaskeyof IBeaconSSZTypes`. Or there might be some other way you know of.

@wemeetagain Everything seems okay to merge, it's the labeler that failed because the source branch comes from a fork


How to set the instrument to the OKEX Contracts?

Also OKEX Contracts position calling has different entry, not same with coin-coin calling.

Hi @helldragonx
To be honest, I have not tested at all on OKEX. Each exchange does have its own quirks. It might require looking into the store source code to see if these parameters can be passed.
Does OKEX have a testnet / paper trading account?

Closing as no response in a long time


plugin/http/propagation: Should Propagator.FromRequest return (trace.SpanContext, error) instead of (trace.SpanContext, bool)?

Hi everyone!
I'm looking at Cloud Tracer propagator and wondering if method
func (f *HTTPFormat) FromRequest(req *http.Request) (sc trace.SpanContext, ok bool)

should be defined in a different way and return error instead of a bool result. Like this:
func (f *HTTPFormat) FromRequest(req *http.Request) (sc trace.SpanContext, err error)

The reason is that bool result removes the opportunity to tell why request parsing actually failed. For  Cloud Tracer propagator possible cases could be:

missing / symbol
the string representing trace ID is not a hex-string
span ID is not an integer

...and others.
As a developer trying to integrate OpenCensus into an application, I can benefit from having more details about the failure (especially during the first steps of integration).
Also, looks like this pattern was reused in a definition of general HTTPFormat interface, which affects #393 .
Should we consider a change of FromRequest method definition?

Hello @elgris thank you for the request and welcome to OpenCensus!
That's actually a great point, we are not relaying back a bunch of reasons why a propagation failed
as in
https://github.com/census-instrumentation/opencensus-go/blob/aa2b39d1618ef56ba156f27cfcdae9042f68f0bc/plugin/http/propagation/google/google.go#L48-L84
Seems like the signature was predicated by the usage such as
https://github.com/census-instrumentation/opencensus-go/blob/65d32661a6ad0a76920f38d1d6dcc21aec56cfce/plugin/http/httptrace/httptrace.go#L174-L177
/cc @rakyll @ramonza

I think the tension is that it's not really an error if a request just doesn't have tracing information. That's kind of expected in the presence of sampling. I'm curious how you would use the error? If you're just instrumenting a library with OpenCensus, you shouldn't need to create an HTTPFormat yourself.

@ramonza at some point in a big company (I currently work for) service A may start sending requests to service B using incorrect HTTPFormat. There is no standard format defined (yet), so everyone is free to use one of many available formats: B3, Google Cloud, something completely custom as long as it implements HTTPFormat interface, etc.
In case of custom implementation the bugs are inevitable :) Especially when it comes to communication between services written in different languages (we have at least Java, Python, Go, JS, Scala, there is more).
Having this, it would be nice to distinguish between "request has no tracing data at all" (which we either can drop or talk to corresponding team so add implement tracing to their app) and "request contains malformed trace data" which is clearly a bug.
What do you think about this?
Also, the code snipped can be modified like this:
 sc, err = f.FromRequest(r)
 if err !=nil {
 	break
 }

Standard HTTPTrace plugin may drop it, but nothing stops of implementing your own plugin that does error processing on top of other customisations (e.g. changing naming scheme or adding mandatory attributes to the span).

Provided propagation impl is pluggable, This could be done internally
right? Particulary with setups that do composite inject and extract a
single error doesnt seem to work in practice.
Composite will be routine for many sites considering the trace context
format will require this for transition.
On 7 Feb 2018 6:57 am, "Ivan Kirichenko"<EMAIL_ADDRESS>wrote:

@ramonza https://github.com/ramonza at some point in a big company (I
currently work for) service A may start sending requests to service B
using incorrect HTTPFormat. There is no standard format defined (yet), so
everyone is free to use one of many available formats: B3, Google Cloud,
something completely custom as long as it implements HTTPFormat
interface, etc.
In case of custom implementation the bugs are inevitable :) Especially
when it comes to communication between services written in different
languages (we have at least Java, Python, Go, JS, Scala, there is more).
Having this, it would be nice to distinguish between "request has no
tracing data at all" (which we either can drop or talk to corresponding
team so add implement tracing to their app) and "request contains malformed
trace data" which is clearly a bug.
What do you think about this?
Also, the code snipped can be modified like this:
sc, err = f.FromRequest(r)
if err !=nil {
break
}
Standard HTTPTrace plugin may drop it, but nothing stops of implementing
your own plugin that does error processing on top of other customisations
(e.g. changing naming scheme or adding mandatory attributes to the span).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/census-instrumentation/opencensus-go/issues/408#issuecomment-363665940,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD61zV3oNdyqrpyB3_C0hTnITOOhc_Mks5tSTtLgaJpZM4R7i1X
.

That makes sense. I think we'd be open to making a change along those lines if you want to make a PR.

Posted before seeing @adriancole's reply. The case of composite format (accepting a wide range of possible inputs) does seem to be in conflict with returning an error. I think it might be best to wait until we have a concrete need for returning an error and then we can review a PR in the context of that concrete need and maybe look at different ways of handling the situation.

got it, thanks @ramonza and @adriancole.


ClickHouse exception:Unknown function timezone

When connect to clickhouse-server by jdbc, there is an exception:
Unexpected driver error occurred while connecting to database
ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 46, host: <IP_ADDRESS>, port: 8123; Code: 46, e.displayText() = DB::Exception: Unknown function timezone, e.what() = DB::Exception
ClickHouse exception, code: 46, host: <IP_ADDRESS>, port: 8123; Code: 46, e.displayText() = DB::Exception: Unknown function timezone, e.what() = DB::Exception
Code: 46, e.displayText() = DB::Exception: Unknown function timezone, e.what() = DB::Exception
Code: 46, e.displayText() = DB::Exception: Unknown function timezone, e.what() = DB::Exception
Other message: I can connect the clickhouse-server by clickhouse-client on another computer:
ClickHouse client version 1.1.54180.
Connecting to <IP_ADDRESS>:9000 as user default.
Connected to ClickHouse server version 1.1.54023.
Warning: could not determine server time zone. Proceeding with local time zone.

@dsfishchen  Which ClickHouse version do you use?

ClickHouse server version 1.1.54023.

I contrast this clickhouse to another normal clickhouse,the following functions was not found.
how can i repair it.
│ UUIDNumToString                │            0 │
│ UUIDStringToNum                │            0 │
│ arrayReverseSort               │            0 │
│ arraySort                      │            0 │
│ assumeNotNull                  │            0 │
│ caseWithExpr                   │            0 │
│ caseWithoutExpr                │            0 │
│ coalesce                       │            0 │
│ decodeURLComponent             │            0 │
│ hasColumnInTable               │            0 │
│ ifNull                         │            0 │
│ isNotNull                      │            0 │
│ isNull                         │            0 │
│ nullIf                         │            0 │
│ pointInEllipses                │            0 │
│ timezone                       │            0 │
│ toStartOfDay                   │            0 │

You just need to consider updating ClickHouse server. The version you named is pretty old, not even from this year.
Or at least use older jdbc driver release, it won't rely on relatively recent features.

I use a older jdbc driver release,it works!!
Thank you very much!!

ru.yandex.clickhouse
clickhouse-jdbc
0.1.13

Actually you can use recent versions of clickhouse-jdbc as well with additional settings
either through ClickHouseProperties
properties.setUseServerTimeZone(false);

or through Properties
properties.setProperty("use_server_time_zone", "false")

and passing it to
ru.yandex.clickhouse.ClickHouseDriver#connect(java.lang.String, ru.yandex.clickhouse.settings.ClickHouseProperties) or
ru.yandex.clickhouse.ClickHouseDriver#connect(java.lang.String, java.util.Properties) respectively.
There are couple of other settings regarding timezones defined there
https://github.com/yandex/clickhouse-jdbc/blob/master/src/main/java/ru/yandex/clickhouse/settings/ClickHouseConnectionSettings.java

Ok，thank you.
becouse of need to connect to clickhouse by DBeaver, so i choose an older jdbc driver.


OCPBUGS-4757: Default to legacy psa settings (#273)

The catalogSource api was recently updated to support running the associated pod in a "restricted" workflow as defined by the Pod Security Admission controller. By default, the catalogSource pods have been configured to run in "restricted" mode, which is disruptive to customers managing and deploying their own catalogSources as they need to rebuild their catalogs to run in "restricted" mode if a namespace is marked as "restricted".
In an effort to provide users with a bit more time to rebuilt their catalogSources, this change configures catalogSources to run in "legacy" mode by default. A series of other changes will be made to update the namespaces associated with marketplace and olm  to support catalogSources running in "legacy" mode by default.
Signed-off-by: Alexander Greene<EMAIL_ADDRESS>

/hold until https://github.com/operator-framework/operator-marketplace/pull/491 merges.

/jira refresh

/jira refresh

Verify failed, details: https://issues.redhat.com/browse/OCPBUGS-4757
/hold

This PR will need the upstream changes introduced here: https://github.com/operator-framework/operator-lifecycle-manager/pull/2906

/retest

/retest

/test e2e-gcp-olm

/retest

/retest

/retest

@jianzhangbjz this should be good for a retest.

/lgtm

Retest it and It works well, details: https://issues.redhat.com/browse/OCPBUGS-4757
/unhold

/lgtm

/cherry-pick release-4.12


docs: Vitepress routing not using SPA routing

I don't recall seeing this before. I wonder if this is caused by us needing to upgrade to the color mode?
Here is a video for reference.
https://github.com/bootstrap-vue-next/bootstrap-vue-next/assets/25397758/a96dfcba-e395-4bba-895c-371bf972a6e1

The docs app is being rendered in a sudo-MPA mode as there is an active hydration mismatch in Layout.vue

It should be noted that Vitepress in "dev" renders as a traditional SPA, so developing and browsing through dev will typically not show these types of issues. Disappointingly. Tbh, it likely has something to do with "isLargeScreen"

Interesting. I use the docs all the time and I have only just started noticing this (not in dev mode). I am always in dark mode as well. I swear I would have noticed this before.
//Tbh, it likely has something to do with “isLargeScreen"
Makes sense.

For any future people that found this because they use Vitepress with BVN
When any link has target="_self" it will cause Vitepress to ignore SPA routing, thus causing a full page reload. This conflicts with the libraries internal default which places _self as the default target. To fix this use :target="null"

To fix this use :target="null"

Was a lie, this is no longer an issue. "_self" is no longer the default -- the default of target is undefined. (_self is implicit by the browser)


CI: add markdown link checking

Changes:

adds ci/markdown_link_check.sh & ci/markdown_link_check.sh
adds markdown-link-check step to CI

Currently failing:
FILE: ./execution_engine/src/shared/logging/README.md | 217s

189 | ERROR: 4 dead links found! | 219s
190 | [✖] https://docs.rs/casper-engine-shared/latest/casper_engine_shared/logging/fn.initialize.html → Status: 404 | 219s
191 | [✖] https://docs.rs/casper-engine-shared/latest/casper_engine_shared/logging/fn.log_metric.html → Status: 404 | 219s
192 | [✖] https://docs.rs/casper-engine-shared/latest/casper_engine_shared/logging/fn.log_duration.html → Status: 404 | 219s
193 | [✖] https://github.com/CasperLabs/CasperLabs/tree/master/execution-engine/engine-metrics-scraper → Status: 404

spoke with Fraser, both casper_engine_shared and engine-metrics-scraper are no longer maintained or published. I'll rip these out.

bors r+

bors r+

bors r+

bors r+

bors r+


pass along gdpr consent string and gdpr applies through get

note will most likely have the same issue as schain
still pending the result of this discussion: https://yieldmo.atlassian.net/browse/FS-7695

I think this is good to go since we don't think it will be super long.

LGTM


For anyone out there trying to make this work in 2021

Hi all so I'm kinda new to backend/frontend server/node.js and found this amazing piece and thought I could learn from it. After some hours I finally make it work on ubuntu 20.04LTS. I'm sure there's many out there like me who's just starting out so here are some problems I encountered along with their solutions:

there's a typo in chatfun.ts lead to an error in the release script, you need to change one of the import packages from "Jquery" to "jquery". This one took me the longest to figure out lol I've tried installing a dozen versions of jquery before realizing the typo.

Microsoft has stopped supporting Dotnet version 2.2 in 2019, so you need to install it manually by downloading the binary file from the Microsoft site (it's listed under "unsurpoted section" I think) and run the following command:

mkdir -p $HOME/dotnet && tar xf dotnet-sdk-2.2.100-linux-x64.tar.gz -C $HOME/dotnet
export DOTNET_ROOT=$HOME/dotnet
export PATH=$PATH:$HOME/dotnet

It can be downgraded to version 2.1 fairy easily (I assume by reading official docs) but I'm certainly not skilled enough to do so haha.

for the same reason, you also need to install libssl1.0 since the default version is way newer. You can do it following this page: https://www.garron.me/en/linux/install-ruby-2-3-3-ubuntu.html. It's for old ruby but works for us too.

Hope that helps!

Hi @skysky2333!
Thanks for your comment. That's so helpful! If you know how to contribute, you can do these changes and else, I'll fix them.

Hello! Do you fix them?

Hi @ysakovs!
Unfortunately, I didn't fix these problems so far, but if you know how to contribute, you can fix them and send me your PR.
Good luck

Hi! Update for fix?

@skysky2333, @ysakovs After a long time, I fixed the problems 😅


Change Light and Dark mode Toggle icon

Hi,
Based on user behaviour analytics, I observed that most of the readers aren't able to figure out the light and dark button toggle icon in the blog.
Maybe changing it to something more obvious and widely used like Sun+Moon, can help? For ref.
Thanks.

That's a good idea, I will try to implement as soon as I have time for it.

This is already implemented in the latest version ;)


Get error when I access Store()

I installed pyoxigraph using pip install pyoxigraph
When I reference store like:
from pyoxigraph import *
from pyoxigraph import Store
store = Store()

I get the following exception:
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_4604/3945786218.py in <module>
      1 from pyoxigraph import *
----> 2 from pyoxigraph import Store
      3 store = Store()

ImportError: cannot import name 'Store' from 'pyoxigraph' (C:\Users\Admin\miniconda3\envs\oxi\lib\site-packages\pyoxigraph\__init__.py)

I dont know if I need to install or activate something else to be able to access Store()

Hi! Thank you for trying Oxigraph and sorry for that.
I made a mistake while pushing the first beta for the upcoming 0.3 version: the release script updated the website documentation even if the default package on pypy is still from the 0.2 branch. I have reverted this change.
In v0.2, there is no Store class but a MemoryStore if you want the data to stay in memory and a SledStore if you want data to be stored on disk. v0.3 merges the two classes into a single Store class, hence the documentation discrepency.
Sorry again.

@Tpt Thanks for your reply. So, finally, the following option doesnt exist in python's wrapper:
RocksDbStore: a file system implementation based on the RocksDB key-value store. It requires the "rocksdb" feature to be activated. The clang compiler needs to be installed to compile RocksDB.
Can you please confirm? If yes, do you have any estimation for when it will be added?

@ali3assi yes, RocksDbStore is not available in pyoxigraph 0.2. However it is going be available under the name Store in the upcoming 0.3 version. I already pushed a first beta on Pypy and the documentation is here. Feel free to try it and give me feedbacks. I am dropping the other "stores" ("rust in-memory" and Sled) in v0.3 to ease the development and maintenance.

@Tpt Thanks for you quick reply. Yes it is working now. I wanna tested and give you later my feedbacks


fix(dynamic): handle selector properly when invoking action

fixes #31

Codecov Report

Merging #33 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #33   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           3      3
  Lines          90    108   +18
=====================================
+ Hits           90    108   +18

Impacted Files
Coverage Δ

src/index.js
100% <100%> (ø)
:arrow_up:

src/fetchers.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba98cc2...9bfa010. Read the comment docs.


Loading RN App from Url, Auto-Update JS.

My idea , is to load my RN app from url ,
to achieve auto-update capabilities to my app.
My quesiton in StackOverflow here
In the beginning i thought there two ways to solve it.

New Require2 fn , inside the v8 engine, the function, will have Require2(jsUrl, staticFallbackJs) this way i could load js files remotely, in case of no conectivty it will load a static resource.
Native module , that updates  the js bundle on every load.

After i played with the environment, my conclusion was different,
changing jsCodeLocation variable in AppDelegate.m , to my own compiled js.
//Point RN to my complied js file
jsCodeLocation = [NSURL URLWithString:@"http://s3-bucket.com/index.ios.bundle.js"];

index.ios.bundle.js content is the minified dev version of RN webpack server ,
http://localhost:8081/index.ios.bundle?dev=false&minify=true
Pointing  jsCodeLocation variable to my bundle.js url  gives the following:

Auto update app , js can be updated to end users , without releasing a new app
Dynamic npm modules as long as they are pure js.
I still have to handle a static version , and to check connectivity , on load time .

I be happy to hear thoughts ,and ideas about this solution.

Hi there, this is discussion is better suited for http://discuss.reactjs.org/, StackOverflow, IRC or Slack.
Also check out:

https://github.com/reployapp/react-native-versions
https://apphub.io/


⚠ ClamAV has degraded performance

In a7e0c44, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 195 ms

Resolved: ClamAV performance has improved in 20bb988.


Passing configuration values to the operator

Type of question
Are you asking about community best practices, how to implement a specific feature, or about general context and help around the operator-sdk?
I'm asking about community best practices
Question
What did you do?
What I'd like to understand is how to pass to the operator variables that are used to set up it.
For instance the operator needs an API KEY. Which is the best way to do it? It is not clear to me.
Environment

operator-sdk version:
v0.1.0

I think there are probably two ways to do this.

Environment variables, for something like an API KEY that you will probably want to keep as a secret and then you can just add the secret to the operator as environment variables.

Command line args. Most folks use https://godoc.org/github.com/spf13/pflag. I think this should be for options i.e logging verbosity.

@hasbro17 thoughts?

I'm interested to the 1.
I'm not sure in which of the steps I would have to pass it from this guide my https://github.com/operator-framework/operator-sdk/blob/master/doc/user-guide.md#1-run-as-a-deployment-inside-the-cluster

You would need to edit the deployment in deploy/operator.yaml file.
This doc might help: https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables
I think what you will need to do is in the pod spec to add the:
...
spec:
  template:
    spec:
      containers:
        - name: ....
          ...
          env:
            - name: API_KEY
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: apiKey

Does that help?

Yeah this helps a lot and then I could retrieve it from the Reconcile function by querying it via the namespace and the name of the deployment. Am I right?

Perfect. This solve all my questions. Thanks a lot @shawn-hurley for this quick reaction and explanation


Remove extra whitespace from merge.

@dotnet-bot skip ci please

@alexperovich,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


Inconsistent path transformation

Hello again,
I'm currently exploring your solution and try to integrate it with TypeScripts external module concept.
I have noticed, that you try to maintain the MVC conventions by using capitalized folder and file names and then tranform it to uncapitalized paths to match the require convention.
So the issue I am currently experiencing with this is a bit complicated, by I try to explain it to you.
TypeScript is checking dependencies for modules by the actual file paths, so requiring a module like this works
import index = require(“Controllers/Root/Home/Index”);

but writing it as the follows results in a compiler-error
import index = require(“controllers/root/home/Index”);

because TypeScripts dependency-check is case sensitive, just like it is in require.js.
Now when bundling the files via autoBundles and using overrides I would generate for example a bundle with the following files:
•	 Controllers/Root/Home/Index
•	Controllers/Root/Home/Extended
And the corresponding generated path overrides would look like this:
{
 "controllers/root/home/Index": "bundles/baseBundle",
 "controllers/root/home/Extended": "bundles/baseBundle"
}

Now my baseBundle file looks looks like follows with code compiled by TypeScript
   define(“controllers/root/home/Index”, [“module”,”exports”], function(){..});
   define(“controllers/root/home/Extended”,
              [“module”,”exports”, “ Controllers/Root/Home/Index”], function(){..});

And there things go down. The module “controllers/root/home/Index” is properly defined, but require.js sees the dependency “ Controllers/Root/Home/Index” as a different file and loads it again,
ending in having loaded the file-contents twice.
In my opinion this is a rather inconsistent behavior,
because I would expect the compressor either to de-capitalize all of the dependencies or none of them.
Maybe there should be the possibility to disable de-capitalization of file paths as an option, for I do not see the need of it,
except matching  the conventions?
And by the way I am very impressed by this projects.
I see it as a great solution to bring AMD concepts into multipage apps.
Cheers!

While I agree that such inconsistencies are more trouble than they're worth in the long run, I'm reluctant to change this right now since the implications would be far reaching for existing projects. That being said, it's something we should fix for 3.0.
As a band-aid for your case I've made the autobundler apply the same transformation to require() dependencies that it applies to file paths when generating a module name, so in your case baseBundle should look like this:
 define(“controllers/root/home/Index”, [“module”,”exports”], function(){..});
   define(“controllers/root/home/Extended”,
              [“module”,”exports”, “ controllers/root/home/Index”], function(){..});

Please update both the require and require compressor packages to 2.1 and tell me if that works for you.

I tried and it works fine now for me!
Thank you very much.


An error when starting the docker, please can you help ? I

I ve installed the docker, and when starting it from portainer throws an error:
Error starting
userland proxy: listen udp
<IP_ADDRESS>:51820: bind: address
already in use
Anyone can help with these ? 51820 is the wireguard port, and yes, its in use. But how can i solve this ?
Thanks in advance

Do you have another wireguard instance running in your server??

I ve installed the docker, and when starting it from portainer throws an error:
Error starting
userland proxy: listen udp
<IP_ADDRESS>:51820: bind: address
already in use
Anyone can help with these ? 51820 is the wireguard port, and yes, its in use. But how can i solve this ?
Thanks in advance

Do you have another wireguard instance running in your server??

@magefesa I ran into the same thing.
My thinking was that I needed to run wireguard on the docker host.
But you shouldn't do that, Running (starting/stopping) is handled on the container.

That port is already being used on your host. Try typing "netstat -tulpn | grep 50802” to see what is using it. Probably another instance of Wireguard as @hsosa88 said

Same problem for me:
Creating wg-manager ...
Creating wg-manager ... error

ERROR: for wg-manager  Cannot start service wireguard: driver failed programming external connectivity on endpoint wg-manager (0e95dccebeca53721102b8117aa4ea11262984c856d4840b6822ac51ee931ee5): Error starting userland proxy: listen udp4 <IP_ADDRESS>:51820: bind: address already in use

ERROR: for wireguard  Cannot start service wireguard: driver failed programming external connectivity on endpoint wg-manager (0e95dccebeca53721102b8117aa4ea11262984c856d4840b6822ac51ee931ee5): Error starting userland proxy: listen udp4 <IP_ADDRESS>:51820: bind: address already in use
ERROR: Encountered errors while bringing up the project.

@mewforest did you check if there is something already running on that port on the host?


typo in the package description

description : Client-side library acquisition in ASP.NET Core with libmam.
its nothing important, just a typo 📄
libmam? XD

Yes, you are right.


[SPARK-35020][SQL] Group exception messages in catalyst/util

What changes were proposed in this pull request?
This PR group exception messages in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util.
Why are the changes needed?
It will largely help with standardization of error messages and its maintenance.
Does this PR introduce any user-facing change?
No. Error messages remain unchanged.
How was this patch tested?
No new tests - pass all original tests to make sure it doesn't break any existing behavior.

Test build #137998 has started for PR 32367 at commit 2c07f51.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42518/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42518/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42518/

Test build #137998 has finished for PR 32367 at commit 2c07f51.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137998/

Test build #138022 has started for PR 32367 at commit 94b9465.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42541/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42541/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42541/

Test build #138022 has finished for PR 32367 at commit 94b9465.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138022/

Test build #138032 has started for PR 32367 at commit 5fe5377.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42551/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42551/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42551/

Test build #138032 has finished for PR 32367 at commit 5fe5377.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138032/

ping @allisonwang-db

Test build #138094 has started for PR 32367 at commit 088ae92.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42614/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42614/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42614/

Test build #138094 has finished for PR 32367 at commit 088ae92.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138094/

ping @allisonwang-db

cc @cloud-fan

thanks, merging to master!

@allisonwang-db Thank you for review. @cloud-fan Thank you too.


Add ChatFiles.

Hi, I am the developer of ChatFiles. This project was completed using LangChain and its main function is to provide chat based on documents.

thanks for the add


BUG: tpch sf10 get chunk shape error

Describe the bug
I am running the tpch sf10 benchmark. I get the following error. sf1 can work in my environment.
Traceback (most recent call last):
  File "~/hpc/test_xorbits/df-bench/tpch/xorbits/xorbits_query.py", line 1250, in run_queries
    result = result.execute()
  File "~/hpc/xorbits/python/xorbits/core/adapter.py", line 310, in _wrapped
    return member_func(mars_entity, *args, **kwargs)
  File "~/hpc/xorbits/python/xorbits/core/adapter.py", line 444, in wrapped
    return from_mars(c(*to_mars(args), **to_mars(kwargs)))
  File "~/hpc/xorbits/python/xorbits/_mars/core/entity/executable.py", line 152, in execute
    return execute(self, session=session, **kw)
  File "~/hpc/xorbits/python/xorbits/_mars/deploy/oscar/session.py", line 1709, in execute
    return session.execute(
  File "~/hpc/xorbits/python/xorbits/_mars/deploy/oscar/session.py", line 1526, in execute
    execution_info: ExecutionInfo = fut.result(
  File "/fs/fast/u20200002/envs/ucx/lib/python3.9/concurrent/futures/_base.py", line 446, in result
    return self.__get_result()
  File "/fs/fast/u20200002/envs/ucx/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "~/hpc/xorbits/python/xorbits/_mars/deploy/oscar/session.py", line 1689, in _execute
    await execution_info
  File "~/hpc/xorbits/python/xorbits/_mars/deploy/oscar/session.py", line 102, in wait
    return await self._aio_task
  File "~/hpc/xorbits/python/xorbits/_mars/deploy/oscar/session.py", line 848, in _run_in_background
    raise task_result.error.with_traceback(task_result.traceback)
  File "~/hpc/xorbits/python/xorbits/_mars/services/task/supervisor/processor.py", line 387, in run
    async for stage_args in self._iter_stage_chunk_graph():
  File "~/hpc/xorbits/python/xorbits/_mars/services/task/supervisor/processor.py", line 171, in _iter_stage_chunk_graph
    chunk_graph = await self._get_next_chunk_graph(chunk_graph_iter)
  File "~/hpc/xorbits/python/xorbits/_mars/services/task/supervisor/processor.py", line 162, in _get_next_chunk_graph
    chunk_graph = await fut
  File "/fs/fast/u20200002/envs/ucx/lib/python3.9/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/fs/fast/u20200002/envs/ucx/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "~/hpc/xorbits/python/xorbits/_mars/services/task/supervisor/processor.py", line 157, in next_chunk_graph
    return next(chunk_graph_iter)
  File "~/hpc/xorbits/python/xorbits/_mars/services/task/supervisor/preprocessor.py", line 201, in tile
    for chunk_graph in chunk_graph_builder.build():
  File "~/hpc/xorbits/python/xorbits/_mars/core/graph/builder/chunk.py", line 431, in build
    yield from self._build()
  File "~/hpc/xorbits/python/xorbits/_mars/core/graph/builder/chunk.py", line 425, in _build
    graph = next(tile_iterator)
  File "~/hpc/xorbits/python/xorbits/_mars/services/task/supervisor/preprocessor.py", line 75, in _iter_without_check
    to_update_tileables = self._iter()
  File "~/hpc/xorbits/python/xorbits/_mars/core/graph/builder/chunk.py", line 308, in _iter
    self._tile(
  File "~/hpc/xorbits/python/xorbits/_mars/core/graph/builder/chunk.py", line 201, in _tile
    need_process = next(tile_handler)
  File "~/hpc/xorbits/python/xorbits/_mars/core/graph/builder/chunk.py", line 173, in _tile_handler
    tiled_tileables = yield from handler.tile(tiled_tileables)
  File "~/hpc/xorbits/python/xorbits/_mars/core/entity/tileables.py", line 80, in tile
    tiled_result = yield from tile_handler(op)
  File "~/hpc/xorbits/python/xorbits/_mars/dataframe/indexing/getitem.py", line 358, in tile
    return (yield from cls.tile_with_mask(op))
  File "~/hpc/xorbits/python/xorbits/_mars/dataframe/indexing/getitem.py", line 377, in tile_with_mask
    mask.rechunk(in_df.nsplits[: mask.ndim])
  File "~/hpc/xorbits/python/xorbits/_mars/dataframe/base/rechunk.py", line 199, in rechunk
    chunk_size = _get_chunk_size(a, chunk_size)
  File "~/hpc/xorbits/python/xorbits/_mars/dataframe/base/rechunk.py", line 191, in _get_chunk_size
    return get_nsplits(a, chunk_size, itemsize)
  File "~/hpc/xorbits/python/xorbits/_mars/tensor/rechunk/core.py", line 38, in get_nsplits
    return decide_chunk_sizes(tileable.shape, chunk_size, itemsize)
  File "~/hpc/xorbits/python/xorbits/_mars/tensor/utils.py", line 618, in decide_chunk_sizes
    return normalize_chunk_sizes(
  File "~/hpc/xorbits/python/xorbits/_mars/tensor/utils.py", line 73, in normalize_chunk_sizes
    raise ValueError(
ValueError: chunks shape should be of the same length, got shape: 2805000, chunks: (195000, 195000, 195000, 195000, 180000, 180000, 180000, 180000)

To Reproduce
To help us to reproduce this bug, please provide information below:

Python 3.9
main branch on github

Could you provide which query failed?


recursive error fix

Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?

What does this PR do?
Fixes #461.
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

The issue is that, HF's ModelOutput inherits dict therefore, when calling is_collection(x), it is always True resulting in an infinite loop within to_printable and prettyprint. Currently, I have added such condition to filter out those xs to avoid recursive error.
        if type(x) not in (
            dict,
            list,
            str,
            int,
            bool,
            tuple,
            torch.dtype,
            float,
            dtypes.floating,
            devices.Device,
            torch.memory_format,
        ):
            return x

I am uncertain regarding two points:

if this is the best way to filter xs that cause this recursive error
if x is a type that cause this recursive error, what should we do with it?

I have updated so that we check the type in tree_flatten. Not sure if there is a way to handle the if condition with only isinstance. (cc. @t-vi , @riccardofelluga)

Supporting dataclasses is high prio to us, so we would not want to block https://github.com/Lightning-AI/lightning-thunder/pull/632 if we decide that staying with the solution there is the way we want to go now. @kshitij12345 WDYT about not flattening the world here?

I think this fix is good and should not block #632. Thanks @k223kim @t-vi


python3Packages.imbalanced-learn: 0.4.3 -> 0.5.0

Motivation for this change
Noticed it was out of date while fixing the build.
Had to freeze it for python2 because scikit-learn>=0.21 is no longer compatible.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

@GrahamcOfBorg build python2Packages.imbalanced-learn python3Packages.imbalanced-learn


Comments disrupt the formatting then End of line is selected to CRLF

Example, this:
    //// comment

    method public void something(ttSomething):

        define buffer bSomething for Something.

        for each ttSomething:

            create bSomething.
        end.

    end method.

Becomes this:
    //// comment

    method public void something(ttSomething):

        define buffer bSomething for Something.

      for each ttSomething:

              create bSomething.
          end.

    end method.

This formatting issue occurs when the line ending is not set to LF (Line Feed). When set to LF, the formatting works correctly.
Also then comment is in this format /**/ formatting works correctly:
Example:
class Class final:

    /*comment*/
    method public void something(ttSomething):

        define buffer bSomething for Something.

        if a < b or a > b then do:
            message "not equals".
        end.
        else do:
            do while a < 100:
                if a = b then message "equals".
                else message "error".
            end.
        end.
    end method.

end class.

This:
class MyUtility:
  // used when two or more of same processes are not supposed to be executing at the same time
  method public log alreadyRequestedProcess(pcProgramName as char):
    return false.
  end method.
end class.

Becomes this:
class MyUtility:
  // used when two or more of same processes are not supposed to be executing at the same time
  method public log alreadyRequestedPr:
cess(pcProgramName as char):
     return false.
   end method.
end class.

If the comment is removed, the issue is gone


Migrate from warp to Axum to handle HTTPS

Goal: FDO supports HTTPS for its http-aware services
Acceptance criteria:

e2e tests remains green, no regressions
support secure API endpoints
the warp dependency is effectively removed from the project (cargo/cargo.locks)
support native-tls/openssl
services have configuration to setup certificates
documentation is updated to document how to set up HTTPS aware services
default to HTTPS (?) (question mark around self-signed certificates and how to get going with FDO easily w/o adding yet another pre-configuration step)

Testing (tentative):

Cargo.lock should not have references to rust-tls
add an e2e test with support for HTTPS and make sure it passes with self-signed certificates

This is a pretty good idea I think, though it would be good I think to make sure you still use the metadata in the Message trait to check the flows.
Look at https://github.com/fedora-iot/fido-device-onboard-rs/blob/main/http-wrapper/src/server.rs#L183 and line 202: this enforces that the flow of messages doesn't get broken (i.e. a later message sent without the prior setup) and that messages that need to be encrypted are that way, etc.
I'd strongly suggest to keep using this same data, since making someone add this to each route will fail eventually (someone will forget to add the check at some point).

This is a pretty good idea I think, though it would be good I think to make sure you still use the metadata in the Message trait to check the flows.
Look at https://github.com/fedora-iot/fido-device-onboard-rs/blob/main/http-wrapper/src/server.rs#L183 and line 202: this enforces that the flow of messages doesn't get broken (i.e. a later message sent without the prior setup) and that messages that need to be encrypted are that way, etc.
I'd strongly suggest to keep using this same data, since making someone add this to each route will fail eventually (someone will forget to add the check at some point).

awesome thanks for the input @puiterwijk !! cc @7flying


Your license should be the same as Det3D, that is, Apache 2.0

https://github.com/poodarchu/Det3D/blob/master/LICENSE

@poodarchu  I am not an expert about the difference between apache 2.0 or MIT. But I think it is allowed to use a different license for a repo with Apache 2.0 URL given that we have included the Det3D's license in the NOTICE files. Please correct me if I am wrong and I will change the license

If you want to use another license, you need to add Apache LICENSE to every unmodified file, and state the modifications in your own file. Or you can use the same license as Det3D.
https://en.wikipedia.org/wiki/Apache_License

Thank you for your interest in our repo. I think there is a misunderstanding here. We did not build this repo on your apache licensed version of the repo. The code is forked from the MIT licensed part: https://github.com/poodarchu/Det3D/tree/56402d4761a5b73acd23080f537599b0888cce07
As a curtesy we indicated your change of license in the NOTICE file. We understand that this was a mistake and will change the notice file accordingly to indicate that this repo is solely build on MIT licensed code.

I use the wrong license at the beginning, so it needs to change to Apache now.

We understand that you changed the license in your repo after we cloned it. However, this does not oblige anybody to do the same. The MIT license clearly grants that "rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software". However, if there were any files that were copied from already licensed repos under a different license, MIT might not apply to those. If this is the case, please let us know which files and from where, and we'll take appropriate action. However, all files that you were legally allowed to license under MIT will be kept under MIT in this repo.


Snapshots do not work with Yarn PnP

Describe the bug
expect(...).toMatchInlineSnapshot('...');

> TypeError: Cannot read properties of undefined (reading 'match')

Reproduction
Enable PnP via yarn config set nodeLinker pnp, run tests yarn vitest, see the tests fail. Disable PnP via yarn config set nodeLinker node-modules, run tests yarn vitest, see the tests working fine.
System Info
System:
    OS: macOS 11.6
    CPU: (8) arm64 Apple M1
    Memory: 141.69 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 18.16.0 - /opt/homebrew/bin/node
    Yarn: 4.0.0-rc.44 - /opt/homebrew/bin/yarn
    npm: 9.5.1 - /opt/homebrew/bin/npm
    Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman

Used Package Manager
yarn
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

https://github.com/koistya/vitest-workspaces
git clone https://github.com/koistya/vitest-workspaces.git
cd ./vitest-workspaces
yarn install
yarn test

vitest is being loaded from two different places:
/.yarn/__virtual__/vitest-virtual-66a00e0566/4/.yarn/berry/cache/vitest-npm-0.31.1-d67f37acf0-9.zip/node_modules/vitest/dist/vendor-vi.458e47b1.js
/node_modules/vitest/dist/vendor-vi.458e47b1.js

And the snapshotClient cannot be re-used when the assertion is being executed.

$ yarn info --virtuals vitest

─ vitest@npm:0.31.1
│  ├─ Instances: 1
│  ├─ Version: 0.31.1
│  │
│  ├─ Exported Binaries
│  │  └─ vitest
│  │
│  └─ Dependencies
│     ├─ @types/chai-subset@npm:^1.3.3 → npm:1.3.3
│     ├─ @types/chai@npm:^4.3.5 → npm:4.3.5
│     ├─ @types/node@npm:* → npm:20.2.4
│     ├─ @vitest/expect@npm:0.31.1 → npm:0.31.1
│     ├─ @vitest/runner@npm:0.31.1 → npm:0.31.1
│     ├─ @vitest/snapshot@npm:0.31.1 → npm:0.31.1
│     ├─ @vitest/spy@npm:0.31.1 → npm:0.31.1
│     ├─ @vitest/utils@npm:0.31.1 → npm:0.31.1
│     ├─ acorn-walk@npm:^8.2.0 → npm:8.2.0
│     ├─ acorn@npm:^8.8.2 → npm:8.8.2
│     ├─ cac@npm:^6.7.14 → npm:6.7.14
│     ├─ chai@npm:^4.3.7 → npm:4.3.7
│     ├─ concordance@npm:^5.0.4 → npm:5.0.4
│     ├─ debug@npm:^4.3.4 → npm:4.3.4
│     ├─ local-pkg@npm:^0.4.3 → npm:0.4.3
│     ├─ magic-string@npm:^0.30.0 → npm:0.30.0
│     ├─ pathe@npm:^1.1.0 → npm:1.1.0
│     ├─ picocolors@npm:^1.0.0 → npm:1.0.0
│     ├─ std-env@npm:^3.3.2 → npm:3.3.3
│     ├─ strip-literal@npm:^1.0.1 → npm:1.0.1
│     ├─ tinybench@npm:^2.5.0 → npm:2.5.0
│     ├─ tinypool@npm:^0.5.0 → npm:0.5.0
│     ├─ vite-node@npm:0.31.1 → npm:0.31.1
│     ├─ vite@npm:^3.0.0 || ^4.0.0 → npm:4.3.9
│     └─ why-is-node-running@npm:^2.2.2 → npm:2.2.2
│
└─ vitest@npm:0.31.1 [2b68c]
   ├─ Version: 0.31.1
   │
   └─ Peer dependencies
      ├─ @edge-runtime/vm@* → ✘
      ├─ @types/edge-runtime__vm@* → ✘
      ├─ @types/happy-dom@* → ✘
      ├─ @types/jsdom@* → ✘
      ├─ @types/playwright@* → ✘
      ├─ @types/safaridriver@* → ✘
      ├─ @types/vitest__browser@* → ✘
      ├─ @types/vitest__ui@* → ✘
      ├─ @types/webdriverio@* → ✘
      ├─ @vitest/browser@* → ✘
      ├─ @vitest/ui@* → npm:0.31.1 [2b68c]
      ├─ happy-dom@* → ✘
      ├─ jsdom@* → npm:22.0.0 [2b68c]
      ├─ playwright@* → ✘
      ├─ safaridriver@* → ✘
      └─ webdriverio@* → ✘

As long as all the "vitest" peer dependencies in all Yarn workspaces match exactly, there is a single instance of "vitest" (virtual) package and Vitest works as expected.

Not sure what can be done here? Why does yarn create separate Vite instances?

@sheremet-va PnP is more strict in regard to dependency resolution, if the same module has different peer dependencies it cannot be safely used from under different workspaces with their own set of peer dependencies for such module. E.g. one workspace has "happy-dom" dependency, while the other doesn't.
Root Workspace (vitest)
├── Workspace A (vitest)
└── Workspace B (vitest + happy-dom)

In a multi-workspace scenario (via vitest.workspace.ts), Vitest correctly loads "vitest" module from the target workspace, e.g. when running tests location in Workspace B, it uses "vitest" from Workspace B. This is good. But it doesn't work for pre-test hooks. So that when snapshots are loaded in one of the pre-test hooks, Vitest uses a vitest instance loaded from a different workspace (e.g. from the Root Workspace instead of Workspace B). I assume this can be easily fixed.

@sheremet-va Interestingly, when tests are being executed Vitest uses the correct instances of "vitest", but this particular hook is being run from under the (root?) instance of vitest:
9466872/packages/vitest/src/runtime/runners/test.ts#L54C9-L69

Vitest always uses the root worker to run tests (we can't change that), so distDir is always the root Vitest dir.
https://github.com/vitest-dev/vitest/blob/94668725e8d7bdfd5ed0f09b3c167d738bb114a0/packages/vitest/src/node/pools/threads.ts#L16
But when it runs tests, Vitest is resolved relative to the file it is imported from.
If we resolve entry to correct Vitest package, this should fix the error. You can already try this with something like this:
const rootHtml = join(config.root, 'index.html')
const { id: vitestEntry } = await rpc().resolveId('vitest/dist/entry.js', rootHtml, 'node')
const { run } = await import(vitestEntry)

@sheremet-va I'm testing with Vitest 0.32.0 and this issue still remains:
https://github.com/koistya/vitest-workspaces
$ yarn install
$ yarn test

 FAIL  |api| index.test.ts > foo
TypeError: Cannot read properties of undefined (reading 'match')
 ❯ api/index.test.ts:5:17
      3|
      4| test("foo", () => {
      5|   expect(foo()).toMatchInlineSnapshot('"OK"');
       |                 ^
      6| });
      7|

@sheremet-va We're on Vitest 0.34.4 and this issue still remains.
We're using NX to orchestrate our monorepo.
As a workaround I found that I can remove vitest from root package.json - and it looks like it's working.  But then of course I get:
(node:15252) [MODULE_NOT_FOUND] Error: @nx/vite tried to access vitest (a peer dependency) but it isn't provided by your application; this makes the require call ambiguous and unsound.
(Use `node --trace-warnings ...` to show where the warning was created)

I just updated to v0.34.5 with pnpm and seeing something very similar. I can try to set up a repro if that would help...

Update: ah, I just got a mismatch between vitest core and @vitest/coverage-v8. Upgrading them together helped. However, it was very difficult to track down the problem and some better error reporting would be very nice!


Don't validate memory values

Remove validation of values in CairoPie Memory.

@juanbono, I want to add a pathological CairoPie end to end test that will cover this and the previous issue against python VM output (see slack), I can do that once #1781 is merged.

@juanbono I amended an existing test to cover this, will add a test CairoPie with all the edge cases once I'm sure all our test CairoPies run.


Import error for sklearn

From<EMAIL_ADDRESS>on 2013-12-09T18:10:17Z
Spyder Version:  2.2.4
Python Version:  2.7.5
Qt Version    :  4.8.4, PyQt4 (API v2) 4.9.6 on Windows
pyflakes >=0.5.0:  0.7.3 (OK)
pep8 >=0.6      :  1.4.6 (OK)
IPython >=0.13  :  1.1.0 (OK)
rope >=0.9.2    :  0.9.4 (OK)
sphinx >=0.6.6  :  1.2b2 (OK)
matplotlib >=1.0:  1.3.0 (OK)
sympy >=0.7.0   :  None (NOK)
pylint >=0.25   :  1.0.0 (OK)
What steps will reproduce the problem?

type import scipy in console What is the expected output? What do you see instead? Expected output - inported module
Instead:
Traceback (most recent call last):
File "", line 1, in <module>
File "C:\Python27\lib\site-packages\sklearn_init_.py", line 31, in <module>
from . import __check_build
ImportError: cannot import name __check_build

Please provide any additional information below
. scikit-learn module is installed and works fine in IDLE
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=1674

From ccordoba12 on 2013-12-11T17:23:46Z
I have no idea what could be happening but maybe it could help to deactivate our scientific startup script in
Preferences > Console > Advanced Settings
Also: does this also happens when under our IPython console?
Summary: Import error for sklearn  <span class="oldvalue"> (was: Import error sklearn) </span>
Labels: Cat-Console


Exclude TestFunctionDescriptor on all supported platforms

The change is to exclude this FFI related test suite
to avoid unexpected failure due to incomplete code.
Signed-off-by: Cheng Jin<EMAIL_ADDRESS>

We need to temporarily disable the test suite as we have not merged all code listed (java & native) at https://github.com/eclipse-openj9/openj9/issues/15068.
Reviewer: @llxia
FYI: @ShelleyLambert, @Mesbah-Alam


Added name for EventId in logs

Added name for EventId in Hosting, DataProtection, EntityFrameworkCore. #6381

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: mikeo52 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Build failures look unrelated. I'll restart.

@aspnet-hello triage https://dev.azure.com/dnceng/public/_build/results?buildId=230970

This comment was made automatically. If there is a problem contact<EMAIL_ADDRESS>I've triaged the above build. I've created/commented on the following issue(s)
https://github.com/aspnet/AspNetCore-Internal/issues/2220
https://github.com/aspnet/AspNetCore-Internal/issues/2410
https://github.com/aspnet/AspNetCore-Internal/issues/2623
https://github.com/aspnet/AspNetCore-Internal/issues/2489
https://github.com/aspnet/AspNetCore-Internal/issues/2415
https://github.com/aspnet/AspNetCore-Internal/issues/2674

@anurse is there any chance that this could merge into asp.net core 3?

/azp run

@mikeo52 Can you resolve the conflict?
I'm hoping this would reach the  3.0 version.

FYI this is targeting the wrong branch for v3.x. It would need to be rebased and retargeted to release/3.0.
@Pilchie bar check for 3.x? Incremental diagnostic improvements.

Oof, this got lost in some of the push to get 3.0 out the door. I'm going to respin the builds and see where we land. There is also a conflict to resolve.

@mikeo52 are you up to try rebasing this on master and resolving the conflict so we can try to take this for 5.0 at least?

@anurse yeah I updated it locally since there are just two spots where it's conflicting, but I'm having a lovely fun time merging because of the same error someone else reported below.
https://github.com/aspnet/AspNetCore/issues/15125

Okay that makes absolutely no sense to me, but whatever.  On this PR's merge conflicts screen I can edit the merge to my repo via the browser and resolve the conflicts, but if I use the git command line I constantly run into the other issue.

Thanks

@Tratcher Is it possible to also merge this to 3.1?

No, 3.1 is only taking critical patches now.

I see, thanks.


Bug: MySQL invalid function data_part

Describe the bug
When running TPCH query 7 with MySQL, the following rewritten query is generated. It contains date_part function which is not supported by MySQL
explain select
    supp_nation,
    cust_nation,
    l_year,
    sum(volume) as revenue
from
    (
        select
            n1.n_name as supp_nation,
            n2.n_name as cust_nation,
            extract(year from l_shipdate) as l_year,
            l_extendedprice * (1 - l_discount) as volume
        from
            supplier,
            lineitem,
            orders,
            customer,
            nation n1,
            nation n2
        where
                s_suppkey = l_suppkey
          and o_orderkey = l_orderkey
          and c_custkey = o_custkey
          and s_nationkey = n1.n_nationkey
          and c_nationkey = n2.n_nationkey
          and (
                (n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
                or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
            )
          and l_shipdate between date '1995-01-01' and date '1996-12-31'
    ) as shipping
group by
    supp_nation,
    cust_nation,
    l_year
order by
    supp_nation,
    cust_nation,
    l_year;

 rewritten_sql=SELECT `shipping`.`supp_nation`, `shipping`.`cust_nation`, `shipping`.`l_year`, SUM(`shipping`.`volume`) AS `revenue` FROM (SELECT `n1`.`n_name` AS `supp_nation`, `n2`.`n_name` AS `cust_nation`, date_part('YEAR', `lineitem`.`l_shipdate`) AS `l_year`, (`lineitem`.`l_extendedprice` * (CAST(1 AS DECIMAL(20,0)) - `lineitem`.`l_discount`)) AS `volume` FROM `supplier` JOIN `lineitem` ON true JOIN `orders` ON true JOIN `customer` ON true JOIN `nation` AS `n1` ON true JOIN `nation` AS `n2` ON true WHERE (((((((`supplier`.`s_suppkey` = `lineitem`.`l_suppkey`) AND (`orders`.`o_orderkey` = `lineitem`.`l_orderkey`)) AND (`customer`.`c_custkey` = `orders`.`o_custkey`)) AND (`supplier`.`s_nationkey` = `n1`.`n_nationkey`)) AND (`customer`.`c_nationkey` = `n2`.`n_nationkey`)) AND (((`n1`.`n_name` = CAST('FRANCE' AS TEXT)) AND (`n2`.`n_name` = CAST('GERMANY' AS TEXT))) OR ((`n1`.`n_name` = CAST('GERMANY' AS TEXT)) AND (`n2`.`n_name` = CAST('FRANCE' AS TEXT))))) AND (`lineitem`.`l_shipdate` BETWEEN CAST('1995-01-01' AS DATE) AND CAST('1996-12-31' AS DATE)))) AS `shipping` GROUP BY `shipping`.`supp_nation`, `shipping`.`cust_nation`, `shipping`.`l_year` ORDER BY `shipping`.`supp_nation` ASC, `shipping`.`cust_nation` ASC, `shipping`.`l_year` ASC

Athena over ODBC has the same error

@peasee - could you please verify which dialect is working for Athena over ODBC. Also, can you use custom dialect just for Athena?
1- SELECT EXTRACT(YEAR FROM NOW()) AS l_year;
2- SELECT YEAR(NOW()) AS l_year;
3- SELECT date_trunc(YEAR, NOW()) as l_year;

only #1 above works with Athena ODBC


Sync with mid

This is an attempt to apply patches from the midstream spec sync to bring this repo into sync with midstream.
https://github.com/open-cluster-management/backlog/issues/15937
Started with midstream commit 6b8aea5ae948e951342f649902dd07bc96d9b3bd and went through commit 3b4d02adb337e05106b18751c643cbcc0feb4bdd on September 22, 2021.  Updates such as README changes specific to midstream or only to initiate a rebuild were not applied.

Remaining differences of note:

Makefile TAG variable set to edge vs latest
deploy/operator.yaml image tag set to edge vs latest-dev


Remove any dependency from the project

Both the lodash.merge and object-path package should be removed as dependency.

Yep. Definitely.

Any update on this one? :)
Thank you for the great plugin anyhow!!!

The object-path is already removed as dep. Still looking for a lean deep merge alternative.


Disable maximize button in avilonia

I want to disable maximize button or hide it.

You can set CanResize to false

Not working in windows.

See https://github.com/AvaloniaUI/Avalonia/issues/437#issuecomment-493003671. Duplicate?

You can set CanResize to false

I can confirm this works fine in Windows 10, maximize button gets disabled. This issue can probably be closed

Well... on my side, the maximize button doesn't disappear with these settings, so this is still an issue.
Using Avalonia 0.10.10 and Windows 11 (22000.318)

Minimize button is still enabled. WIth MessageBox.Avalonia, it's particularly an issue because the minimize/maximize buttons leave very little space for the MessageBox title! And you can minimize the MessageBox!?

Duplicate of https://github.com/AvaloniaUI/Avalonia/issues/3422


Switching between Insiders settings will display multiple statuses in the status bar

Environment data

VS Code version: 1.44.2
Extension version (available under the Extensions sidebar): 2020.6.77799-dev

Expected behaviour
Only one "Downloading Insiders..." should be shown
Actual behaviour

Steps to reproduce:

Swap the python.insidersChannel between weekly and daily

Notes:

May have been happening before https://github.com/microsoft/vscode-python/issues/10495
Probably caused by the fact that we don't stop previous downloading tasks before we start downloading again.

No longer relevant


Fix broken layout of error block on plan-preview comment

What this PR does / why we need it:
We need some line breaks to render the section heading correctly.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

No wonder 🙌
/approve


build dataset myself

Hello~, I want to load my built dataset，could you teach me how to chang the code. Thanks a lot.

Sure, Are you ready to customize your dataset? You need to copy config/config_SPEECHCOMMANDS.yml and set your parameters like the path of the dataset, model depth, max waveform duration, etc.


Add a prototype terraform provider and vsphere configs

What this PR does / why we need it: Adds support for terraform to kube-deploy.
Special notes for your reviewer: This is only prototype code that will be contributed to by others. There are a few TODOs that are created here and I'll migrate those to k8s/kube-deploy.

@kubernetes/kube-deploy-reviewers

/assign @krousey

You may also need to regenerate the provider config code.

All done seems like I need lgtm again.

Migrated to https://github.com/kubernetes-sigs/cluster-api/pull/103


EagerFunc tensor conversion error - dtype mismatch

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.13.1
Python version: 3.7
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: 10.0 / 7.4.2
GPU model and memory: 1080Ti

Describe the current behavior
GIST Link 1
GIST Link 2
I am trying to use tfe.pyfunc to combine eager block in the operation graph (See Link 1). There are four input arguments to the eager mode function of dtype tf.float32 and tf.int32. When I try to calculate the gradient of a tensor w.r.t some variables that the calculations involving them are happen in the eager block, the error in the Link 2 is raised (TypeError: Cannot convert provided value to EagerTensor. Provided value: 0.0 Requested dtype: int32).
Describe the expected behavior
The gradient is calculated and returned after calling tf.Session.run,
Code to reproduce the issue
See Link 1.
Other info / logs
See Link 2.
I posted a Pull Request for the fix of this issue. https://github.com/tensorflow/tensorflow/pull/29728

@kami93 Let us know is this still an issue. Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!


[Datagrids]: Add checkbox as a supported inlineEditType

The problem
Design calls for checkbox

The solution
Need checkbox to be supported, right now I only see text, number, selection, and date as supported options
Design link
No response
Links to other materials
No response
Owner/main maintainer(s)
IBM
Second/backup maintainer(s)
No response
Product/offering
No response
Business priority
High Priority = pressing release
Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I checked the current issues for duplicate issues

@AustinGitHub Hey! Thanks for opening this one.
Could you (or your designers) provide us some more information about the use case? Does the dropdown not work here? You can also have your designers reach out for questions on #ibmproducts-pal.

Sure

Hi @elycheea! The dropdown doesn't work well here because the check box is more like a toggle or boolean. In the below example, that table row can be designated as a key, or nullable. We technically could change it to a dropdown, but that would be a poor user experience: User would have to click on the cell to open the dropdown with true or false options in both of these. It's an extra click each time someone has to change it, and when you have a large table the true/false text is a lot less scannable than the simple checkbox is.
Also, according to Carbon and best UX practices in general for dropdowns, we should not use a dropdown when there are only 2 choices.


chore(client/macos): align the signed-in-and-connected tray icon

Closes #6052
The new icon is here https://www.figma.com/design/THvQQ1QxKlsk47H9DZ2bhN/Core-Library?node-id=1305-799&t=kASomGCOu5fsLmCs-0
The swipe diff makes it easy to see

Also, is the 95% opacity also intentional? I saw that some icons have 100% opacity black, and the connecting icons are mixed 45% and 95%, and the signed-in icon was 95% opacity, so I stuck with that.

Does the dark mode icon need adjusting too? Not sure about the opacity thing but macOS has design guidelines that may have been attempted to be followed for it (roop did those)

The dark mode icon looks okay in Figma, it's 2 pixels from the top of its bounding box whereas the mis-aligned light mode icon was 3 pixels from the top.
I could install Inkscape and measure precisely but all the stuff in Firefox and Github shows SVGs with a light or white background so it's hard to see those icons at all


Fix #5626: Discover and set virtual reality SDK(s) for the target platform

This change asks Unity for the set of supported virtual reality SDKs for the currently selected target platform and then explicitly sets those that are supported by MRTK v2.
Unity 2018 set default SDKs, 2019 appers to no longer do so.
Fixes: #5626

/azp run

/azp run


[docs]: Document Catalog Publishing Workflow in Meshery

This PR fixes #10797
Signed commits

[ ] Yes, I signed my commits.

🚀 Preview for commit de159e10835dec0be9450f90d83ee470eb56fd7f at: https://66bf6d284e548948848ced6c--meshery-docs-preview.netlify.app

@Kevin222004, let's discuss this PR during the Meshery dev meeting call, if you would. Please add it as an agenda item to the meeting minutes.

@Ashparshp sure

Adding into meeting minutes.. // @Ashparshp @Kevin222004


strip stop_words to avoid blanks

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
In some models, after decoding eos_token_id, the word starts with a blank. When it is encode back, the token_id is not the eos_token_id.
Modification
Please briefly describe what modification is made in this PR.
BC-breaking (Optional)
Does the modification introduce changes that break the backward compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here and update the documentation.
Checklist
Before PR:

[ ] Pre-commit or other linting tools are used to fix the potential lint issues.
[ ] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
[ ] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
[ ] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

[ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
[ ] CLA has been signed and all committers have signed the CLA in this PR.

when transformers==4.45.2
tokenizer.encode(eos_token_id) maybe starts with a blank in some models. which make stopwords useless.
test on  transformers==4.45.2

the evaluation score of turbomind_internlm2_5-7b-chat-4bits is correct. https://github.com/InternLM/lmdeploy/actions/runs/11493038307

Does this PR affect other datasets or models? @zhulinJulia24


Proxy should honor CONDUIT_PROMETHEUS_LABELS environment variable

The proxy should add the value of the CONDUIT_PROMETHEUS_LABELS environment variable as a label on all exported metrics.

Since the PR for this issue (#624) was closed because we decided not to add this feature, I'm closing the issue as well.


Update OpenOCD install instruction for #313

Hi, I faced same issue as #313. I thought it may help to add the resolution to the docs.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jamesmunns (or someone else) soon.
Please see the contribution instructions for more information.