.NET 8: S3.PutObjectAsync request throws exception "Request headers must contain only ASCII characters."

Describe the bug
In migrating to .NET 8, we've run into an issue when calling IAmazonS3.PutObjectAsync().  The call works when running on Windows (debug/dev environment), but when deployed to Linux servers (ECS/Fargate), any attempts to put an object results in the exception "Request headers must contain only ASCII characters".
The PutRequest object passed to the call contains no user-added headers, so it would appear that the offending header entry is being added within the call.  The call stack from the exception thrown is as follows:
at System.Net.Http.HttpConnection.g__ThrowForInvalidCharEncoding|56_0()
at System.Net.Http.HttpConnection.WriteString(String s, Encoding encoding)
at System.Net.Http.HttpConnection.WriteHeaderCollection(HttpHeaders headers, String cookiesFromContainer)
at System.Net.Http.HttpConnection.WriteHeaders(HttpRequestMessage request, HttpMethod normalizedMethod)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.S3.Internal.S3Express.S3ExpressPreSigner.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
Installed version of AWSSDK.S3 is 3.7.410.1, but I have tried the earlier version that worked with .NET 6 and still had the exception thrown.
Regression Issue

[X] Select this option if this issue appears to be a regression.

Expected Behavior
Call to PutObjectAsync() should succeed
Current Behavior
Exception thrown "Request headers must contain only ASCII characters."
Reproduction Steps
Deploy .NET 8 based code to ECS/Fargate Linux instance; call fails
Possible Solution
No response
Additional Information/Context
No response
AWS .NET SDK and/or Package version used
AWSSDK.S3 3.7.410.1
Targeted .NET Platform
.NET 8
Operating System and version
ECS/Fargate Linux instance

@indimini Good morning. Please share the following to troubleshoot the issue:

Sample reproduction code solution and deployment steps.
Locale of your execution environment (we have similar issue https://github.com/aws/aws-tools-for-powershell/issues/188 for PowerShell for some locale)
Are you logging or throwing an exception with text containing non-ASCII characters? (refer similar issue https://github.com/aws/aws-lambda-dotnet/issues/1231 which was fixed for Lambda environment)
Does the key or bucket name used while invoking PutObjectAsync call contain any non-ASCII characters?
Are you using S3Express?
Are you able to turn on verbose logging using below statements and share the detailed logs:Amazon.AWSConfigs.LoggingConfig.LogResponses = Amazon.ResponseLoggingOption.Always;
Amazon.AWSConfigs.LoggingConfig.LogTo = Amazon.LoggingOptions.Console;
Amazon.AWSConfigs.AddTraceListener("Amazon", new System.Diagnostics.ConsoleTraceListener());

Thanks,
Ashish

@ashishdhingra  - thanks for the quick response

Sample Code (Simplified):
  PutObjectRequest putRequest = new PutObjectRequest
  {
    BucketName = s3bucket,
    Key = s3key, // Use an Object Id so we don't have to modify any models.
    ContentBody = Convert.ToBase64String(bytes),
  };
  putRequest.Metadata.Add("contenttype", contentType);
  putRequest.Metadata.Add("uploadtime", DateTime.Now.ToString());
  putRequest.Metadata.Add("filename", HttpUtility.UrlEncode(fileName, Encoding.ASCII));

  PutObjectResponse response = await client.PutObjectAsync(putRequest);

This code works w/o issue on Windows, fails when deployed.  I've verified that the putRequest.Headers object we pass to the put method has no entries.

Locale of Environment:
Service ARN = arn:aws:ecs:us-east-1:834521116217:service-deployment/dev-cluster/jan2025-service/s-30HNRQCDVgxmBZwZlZr
Operating System/Architecture = Linux/X86_64
App Environment = Fargate
Network mode = awsvpc

Logging/Throwing: We log any exceptions to our log collection in our DB, then throw the exception.  Contents of call stack provided in original post; note, there is no Inner Exception for the caught exception.

Key or bucket names do not contain Non-ASCII characters. (Note, the code works without issue on Windows server, but fails only when deployed to Linux service in AWS/Fargate)

We are not using S3Express

I tried added the verbose logging but received no additional details.  Again, since this doesn't fail in Development on Windows, there is nothing to capture.  I did subclass the TraceListener and added code to capture any messages passed and store them in our log, but no messages were captured

You'll need to run this on your ECS cluster as well, but can you add AWSConfigs.LoggingConfig.LogMetrics = true; to the commands @ashishdhingra mentioned earlier? That'll include the request the SDK sent to S3 (including the headers):
AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console;
AWSConfigs.LoggingConfig.LogMetricsFormat = LogMetricsFormatOption.JSON;
AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
AWSConfigs.LoggingConfig.LogMetrics = true;

using var client= new AmazonS3Client();
await client.ListBucketsAsync(...);

Example logs:
AmazonS3Client 31|2024-12-10T15:06:31.132Z|INFO|Request metrics: {"properties":{"AsyncCall":"True","CanonicalRequest":"GET\n/\n\nhost:s3.us-west-2.amazonaws.com\nuser-agent:aws-sdk-dotnet-coreclr/3.7.410.2 ua/2.0 os/windows#10.0.22631.0 md/ARCH#X64 lang/.NET_Core#8.0.11 md/aws-sdk-dotnet-core#3.7.400.60 api/S3#3.7.410.2 cfg/retry-mode#legacy md/ClientAsync cfg/init-coll#1\nx-amz-content-sha256:...\nx-amz-date:20241210T200630Z\n\nhost;user-agent;x-amz-content-sha256;x-amz-date","StringToSign":"AWS4-HMAC-SHA256\n20241210T200630Z\n20241210/us-west-2/s3/aws4_request","ServiceName":"AmazonS3","ServiceEndpoint":"https://s3.us-west-2.amazonaws.com/","MethodName":"ListBucketsRequest","AmzId2":"...","StatusCode":"OK","BytesProcessed":"0","AWSRequestID":"..."},"timings":{"CredentialsRequestTime":0.8868,"RequestSigningTime":22.3808,"HttpRequestTime":582.7608,"ResponseUnmarshallTime":13.1556,"ResponseProcessingTime":29.586,"ClientExecuteTime":731.1859},"counters":{}}

GET
/

host:s3.us-west-2.amazonaws.com
user-agent:aws-sdk-dotnet-coreclr/3.7.410.2 ua/2.0 os/windows#10.0.22631.0 md/ARCH#X64 lang/.NET_Core#8.0.11 md/aws-sdk-dotnet-core#3.7.400.60 api/S3#3.7.410.2 cfg/retry-mode#legacy md/ClientAsync cfg/init-coll#1
x-amz-content-sha256:...
x-amz-date:20241210T200630Z

The SDK tries to fetch some information from the environment to build the user-agent, so I'm wondering if something wasn't parsed correctly.

when I capture the logs in my dev environment, calling ListBucketsAsync(), I see the following:
Amazon Verbose: 0 : Received response (truncated to 1024 bytes): [
c761c404745ae5badbe4174a608b6f0aa44dedf93f48fd3dbc85233c6609eb1badminrootagent-installation-bucket-skillsmart2020-10-26T13:50:10.000Zaws-cloudtrail-logs-834521116217-1509af7d2021-03-24T12:38:44.000Zcdktoolkit-stagingbucket-1u82o2oum2n6l2021-12-06T19:23:46.000Zcodepipeline-us-east-1-9859142755782023-11-04T06:48:14.000Zconnect-e6e1ed35f3b82020-04-03T17:03:52.000Zelasticbeanstalk-us-east-1-8345211162172021-12-06T17:44:49.000Zinsight-amazon-root2022-07-16T14:48:46]
Amazon Information: 1 : Request metrics: {"properties":{"AsyncCall":"True","CanonicalRequest":"GET\n/\n\nhost:s3.amazonaws.com\nuser-agent:aws-sdk-dotnet-coreclr/3.7.410.1 ua/2.0 os/windows#10.0.26100.0 md/ARCH#X64 lang/.NET_Core#8.0.8 md/aws-sdk-dotnet-core#3.7.400.59 api/S3#3.7.410.1 cfg/retry-mode#legacy md/ClientAsync cfg/init-coll#1\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20241210T204720Z\n\nhost;user-agent;x-amz-content-sha256;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","StringToSign":"AWS4-HMAC-SHA256\n20241210T204720Z\n20241210/us-east-1/s3/aws4_request\n63e7d88314de300d448a0e3a83d8a3049e9b31ccc24e85477f0160e203b0b97d","ServiceName":"AmazonS3","ServiceEndpoint":"https://s3.amazonaws.com/","MethodName":"ListBucketsRequest","AmzId2":"ldgHVuZaIrdLG5uvKrVYaK09vnI0pN3b/L2gcDhdcwpdGOn9RPmdXcQtjk8+xSWR1u5pHmIrrcg=","StatusCode":"OK","BytesProcessed":"0","AWSRequestID":"FTRHRX67QS7ZMH3R"},"timings":{"CredentialsRequestTime":0.625,"RequestSigningTime":11.3467,"HttpRequestTime":241.7178,"ResponseUnmarshallTime":7.1743,"ResponseProcessingTime":14.2298,"ClientExecuteTime":320.0976},"counters":{}}

@indimini Thanks for your reply. As @dscpinheiro mentioned, please collect logs from your ECS cluster as well. May be there is some extra header which is parsed by SDK which contains some characters not well handled.
Also share the .NET 8 version used on both your dev environment and ECS cluster. There was a known issue in some .NET version, so just curious if it popped back up in .NET 8 in certain version.
Thanks,
Ashish

@ashishdhingra - not sure how to get the logs from an ECS cluster.  Any guidance is appreciated.
The file version of System.Net.Http (where the exception is originating) is V 8.0.824.36612.

@ashishdhingra  - not sure if this helps, but I replaced the client.putObjectAsync() call with using the FileTransferUtility and still end up with the same Non-ASCII header exception.
This is the code I tested with.  This works when running on a Windows server (same behavior as the putObjectAsync())
    using MemoryStream ms = new MemoryStream(bytes);
    ms.Position = 0;
    Amazon.S3.Transfer.TransferUtility utility = new TransferUtility(client);
    TransferUtilityUploadRequest request = new TransferUtilityUploadRequest();
    request.BucketName = s3bucket;
    request.Key = s3key;
    request.ContentType = contentType;
    request.InputStream = ms;
    request.Metadata.Add("contenttype", contentType);
    request.Metadata.Add("uploadtime", DateTime.Now.ToString());
    request.Metadata.Add("filename", HttpUtility.UrlEncode(fileName, Encoding.UTF8));
    await utility.UploadAsync(request);

@ashishdhingra - not sure if this helps, but I replaced the client.putObjectAsync() call with using the FileTransferUtility and still end up with the same Non-ASCII header exception.
This is the code I tested with. This works when running on a Windows server (same behavior as the putObjectAsync())
    using MemoryStream ms = new MemoryStream(bytes);
    ms.Position = 0;
    Amazon.S3.Transfer.TransferUtility utility = new TransferUtility(client);
    TransferUtilityUploadRequest request = new TransferUtilityUploadRequest();
    request.BucketName = s3bucket;
    request.Key = s3key;
    request.ContentType = contentType;
    request.InputStream = ms;
    request.Metadata.Add("contenttype", contentType);
    request.Metadata.Add("uploadtime", DateTime.Now.ToString());
    request.Metadata.Add("filename", HttpUtility.UrlEncode(fileName, Encoding.UTF8));
    await utility.UploadAsync(request);

@indimini TransferUtility uses underlying S3 client behind the scenes and uses PutObjectAsync() if file size is within the limit. Hence, the same exception. For capturing ECS logs, enabling verbose logging writes to console. Please refer Logging and Monitoring in Amazon Elastic Container Service, for sending logs to CloudWatch, you might need to configure awslogs log driver in your task definitions.
Thanks,
Ashish

@indimini DateTime.Now.ToString() is culture-specific. Specify CultureInfo.InvariantCulture as a ToString() parameter.

Here are the Amazon logs from our ECS deployment:
Amazon Information: 0 : Region found using environment variable.                                                                                Amazon Information: 0 : The environment variable AWS_ENABLE_ENDPOINT_DISCOVERY was not set with a value.                                                                                Amazon Information: 1 : The environment variable AWS_MAX_ATTEMPTS was not set with a value.                                                                                Amazon Information: 2 : The environment variable AWS_RETRY_MODE was not set with a value.                                                                                Amazon Information: 3 : The environment variable AWS_EC2_METADATA_SERVICE_ENDPOINT was not set with a value.                                                                                Amazon Information: 4 : The environment variable AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE was not set with a value.                                                                                Amazon Information: 5 : The environment variable AWS_EC2_METADATA_V1_DISABLED was not set with a value.                                                                                Amazon Information: 6 : The environment variable AWS_USE_DUALSTACK_ENDPOINT was not set with a value.                                                                                Amazon Information: 7 : The environment variable AWS_USE_FIPS_ENDPOINT was not set with a value.                                                                                Amazon Information: 8 : The environment variable AWS_IGNORE_CONFIGURED_ENDPOINT_URLS was not set with a value.                                                                                Amazon Information: 9 : The environment variable AWS_DISABLE_REQUEST_COMPRESSION was not set with a value.                                                                                Amazon Information: 10 : The environment variable AWS_REQUEST_MIN_COMPRESSION_SIZE_BYTES was not set with a value.                                                                                Amazon Information: 11 : The environment variable AWS_ACCOUNT_ID_ENDPOINT_MODE was not set with a value.                                                                                Amazon Information: 12 : The environment variable AWS_SDK_UA_APP_ID was not set with a value.                                                                                Amazon Information: 0 : Unable to find a profile named 'default' in store Amazon.Runtime.CredentialManagement.CredentialProfileStoreChain                                                                                Amazon Information: 0 : Resolved DefaultConfigurationMode for RegionEndpoint [us-east-1] to [Legacy].                                                                                Amazon Verbose: 0 : Received response (truncated to 1024 bytes): [ c761c404745ae5badbe4174a608b6f0aa44dedf93f48fd3dbc85233c6609eb1badminrootagent-installation-bucket-skillsmart2020-10-26T13:50:10.000Zaws-cloudtrail-logs-834521116217-1509af7d2021-03-24T12:38:44.000Zcdktoolkit-stagingbucket-1u82o2oum2n6l2021-12-06T19:23:46.000Zcodepipeline-us-east-1-9859142755782023-11-04T06:48:14.000Zconnect-e6e1ed35f3b82020-04-03T17:03:52.000Zelasticbeanstalk-us-east-1-8345211162172021-12-06T17:44:49.000Zinsight-amazon-root2022-07-16T14:48:46]                                                                                Amazon Information: 1 : Request metrics: {"properties":{"AsyncCall":"True","CanonicalRequest":"GET\n/\n\nhost:s3.amazonaws.com\nuser-agent:aws-sdk-dotnet-coreclr/3.7.410.5 ua/2.0 os/linux#<IP_ADDRESS> md/ARCH#X64 lang/.NET_Core#8.0.11 exec-env/AWS_ECS_FARGATE md/aws-sdk-dotnet-core#3.7.400.63 api/S3#3.7.410.5 cfg/retry-mode#legacy md/ClientAsync cfg/init-coll#1\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20241216T153949Z\nx-amz-security-token:IQoJb3JpZ2luX2VjEGgaCXVzLWVhc3QtMSJGMEQCIHsE0BcRcG83qkyjNS4C3Y2AwRJLd7F0uoSNS3tYatcfAiBzkLRXSacOzQ/f0Mb1ww0JBYDP7u2Ny0LZHdtVS3qZHirtAwgxEAAaDDgzNDUyMTExNjIxNyIMaAsnF0JTtaqTlw+gKsoDKf/UKO25Bv27IFrXHrA+ULy1E/SNp5H5D/aNuRDA6XEmSrwuSLfGNE9szcMgsbKAsRVY7CzTDKOVvVSH1+nx9Hzn0D4ragSSXXr6RHIX7/xKZsKEE4xu+R7jzgFiBzTt3TCl+jtKs1oOarnkMi8lL5LYX+/QwbEmx9XyRddfxZqoA0Ak+h9UPB4S/Ox1DvrCJrjn0e+rZauwLIcnt+nmLXlNpQSbKPfF9KcYhsevOjEZiYbSqrkkD3qNyH9tF5zX8QXe4jJOy6py5oY3wd9homRX9mJkehlc+0kfy7O/DPW/XglvYkTDjlTA9NhFsQ3O5ARnnP+2af82ObehT0vTjmRNHH61YABif+z82ryTjoQk3xtlAb8NDNE4OzgpQGA/lnzWlnsVHduTfEt2KT+icZsKtmT3X0XGSkeUTCgw9YqczyehyV9F+dosoZ7ypi8HsYytAotLmVrpVUSTgwlq/2WMfO8DtrL1Td3mikJDZHLyru5fPrB0Ve1ogbQwQuByh3+y//fO4b+UySufzmG0em7/js/Dba8E65pd6igKYtMTROwWZBVBkgrHraLrUca3UAMTgGnsoFY3FPevMzgq6WR7Dy/OgP0T/x0wwJGBuwY6pgFR6YcHH0qDXkv/8vTYTXkYbu2t63k8LjtDP7S4+5zSoA3PbPKkg4xHHoL+POetON7SofUskElcVjJ/ZvuuZA6q67qXaIuFnCTgqUW2oJHCnsBOs2YMWJdPMF4S6iKbyg1ZqAOPXrJbOZdHaQHgw4EwDq8/J+jEimvrTmsIFrHqollBn66EzIVOdXez+4PiQ2DPT9WCCd2E0P8Zmuafa2515/DyKm2K\n\nhost;user-agent;x-amz-content-sha256;x-amz-date;x-amz-security-token\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","StringToSign":"AWS4-HMAC-SHA256\n20241216T153949Z\n20241216/us-east-1/s3/aws4_request\nedb7588a98a4d2cc5c181acb83cd79305fcc0d0cf1d4749b86ffde1de80753c7","ServiceName":"AmazonS3","ServiceEndpoint":"https://s3.amazonaws.com/","MethodName":"ListBucketsRequest","AmzId2":"dzHDKOxY2TOpwNevEs7dVPHeC9oG1tikmUYlDnTnqMQUXCYbsTwoKPI/GccDzOF6YVVe2QaSX4kPJRKoaPF+oA==","StatusCode":"OK","BytesProcessed":"0","AWSRequestID":"3YB8ER2WGAQRRKF6"},"timings":{"CredentialsRequestTime":58.2079,"RequestSigningTime":18.1309,"HttpRequestTime":53.9272,"ResponseUnmarshallTime":14.189,"ResponseProcessingTime":18.5681,"ClientExecuteTime":234.2948},"counters":{}}

If I try to capture results when making the put call, this is the only entry in the logs:
Amazon Verbose: 0 : Single encoded /WH-347/676054e67a39f1a8ff62fcfc with endpoint https://insight-next-dev-root.s3.amazonaws.com/ for canonicalization: /WH-347/676054e67a39f1a8ff62fcfc

It appears that the CultureInfo to the date to staring method does resolve this issue.


Materials should use git+https, not GitHub+https

The spec recommends using SPDX Download Location, which is of the form:
git+https://...

Currently the code outputs GitHub+https://... because it uses the Build.Repository.Provider predefined variable verbatim.
Instead, the code should convert to either git, svn, or tfvc.

Will get this done tonight. Should be a straightforward fix.


UISegmentedControl in UINavigationBar not hiding

When adding a UISegmentedControl to the navigation bar, the segmented control is not hiding when scrolling.

Nice catch, I'll fix it ASAP.

Hi @pangelovski
I'm testing this in the sample project and seems to be working fine: http://d.pr/v/t0eu
Which version are you using?

Hi @andreamazz, Im running an objective c project with the Swift version 2.0.0.
I should note that I added this line in the SwiftNavigationController.swift file
        /**
        if view.classForCoder.description() == "UINavigationButton" || view.classForCoder.description() == "UINavigationItemView" {
            view.alpha = alpha
        }
        **/

        if view.classForCoder.description() == "UINavigationButton" || view.classForCoder.description() == "UINavigationItemView" || view.classForCoder.description() == "UISegmentedControl" {
            view.alpha = alpha
        }

Seems reasonable. I've integrated the fix in version 2.0.1.


Fix flake8 issues

flake8 reports no issues anymore. test.py has unchanged performance

@Clickedbigfoot excellent! Thank you for this. :smiley:


Add vertically integrated tracer diagnostics

This will be computed every timestep and printed to the console with the other
diagnostics.

I'm not sure if it makes more sense to put this at the end or beginning of the timestep.


Add detection threshold adjustment to object detection python interface

Here is a small set of changes that adds adjust_threshold as an optional parameter to the object_detector python interface.  It also updates the face_detector.py example to exercise it.
There are some irrelevant commits that I reverted in here - let me know if you want me to squash these into one, but I think github merge has an option for that now.

Cool, thanks for the PR :)


[C++][Parquet] Support read BloomFilter in reader

Describe the enhancement requested
Currently, our C++ parquet support a bloom_filter class, but it breaks some standard. And, parquet C++ doesn't support reading bloom filter. I'd like to support reading bloom filter. I may split the task into these steps:

[ ] Fixing some issues that current BlockSplitBloomFilter is not aligned with standard
[ ] Support read BloomFilters

[ ] First, we need to have the interface here
[ ] Then, I will implement the logic here

Component(s)
C++, Parquet

@mapleFU
Is BloomFilter supported in c++ writer? seems like no

@mapleFU Is BloomFilter supported in c++ writer? seems like no

@rip-nsk Quick answer is no.
Write support is already in the plan, once the read support is complete.

Is BloomFilter supported in c++ writer? seems like no

No, I would like to support it during China's Spring festival.

Issue resolved by pull request 33776
https://github.com/apache/arrow/pull/33776


NameForm is Useless class type in complete flasky

NameForm is defined in Chapter 4.2 .  And it is useless class type in complete project flasky . Except for the definition here, it's not used anywhere else.

The NameForm class is used in the early part of the book, more specifically in chapter 4. You are correct that eventually this form isn't used anymore and can be removed.


Segfault in async-std/execu

from /var/log/messages
Jan  6 22:14:40 c2-7-sbg3 kernel: [64693.609394] async-std/execu[28429]: segfault at 0 ip 00007faa146058b0 sp 00007faa0fbfb3d8 error 4 in libc-2.28.so[7faa144cf000+148000]
Jan  6 22:14:40 c2-7-sbg3 kernel: [64693.615905] Code: c2 83 e8 ff 75 04 c5 f8 77 c3 f3 0f bc c8 0f b6 04 0f 0f b6 14 0e 29 d0 c5 f8 77 c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 <0f> 38 f0 07 0f 38 f0 0e 48 c1 e0 20 48 c1 e1 20 0f 38 f0 7c 17 fc
Jan  8 00:11:35 c2-7-sbg3 kernel: [158106.772220] traps: async-std/execu[25206] general protection ip:7f31d5cf724a sp:7f31d54eaf10 error:0 in libc-2.28.so[7f31d5c98000+148000]

I recieve just this in console with no backtrace
Segmentation fault
Any ideas?

Can you do cargo update and try running again? The async-crate dependency got some fixes recently that might be related.

if you mean this:
   Compiling async-task v1.2.0
   Compiling async-std v1.4.0 (https://github.com/async-rs/async-std.git#383057b8)

updated just now...
but where is this error originated from? is it possible that it comes from my projects code?

How about run on another machine or build with target musl? I just meet the same problem but because the machine's libc is broken...

Here is my gdb log: https://gist.github.com/serzhiio/a284da515c3a0620a8225b10655b5e0b

Update from conversations in chat: we haven't confirmed yet whether this code originates in async-std or @serzhiio's application since they also have unsafe atomics code as part of their application.

Lets close this issue since i rewrote part of my code and problem seems gone.


restart jenkins to ensure service is running

I have noticed on RHEL 7.2 and 7.3 that without a restart, the ensuing curl command times out after 60 retries, and the role fails.
This commit fixes the issue.
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (9 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (8 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (7 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (6 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (5 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (4 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (3 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (2 retries left).
FAILED - RETRYING: Wait for Jenkins to start up before proceeding. (1 retries left).
 [WARNING]: Consider using get_url or uri module rather than running curl

fatal: [XX.XX.XX.XX]: FAILED! => {"attempts": 60, "changed": false, "cmd": "curl -D - --silent --max-time 5 http://localhost:8080/cli/", "delta": "0:00:00.009602", "end": "2017-06-12 16:31:34.820858", "failed": true, "rc": 7, "start": "2017-06-12 16:31:34.811256", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
	to retry, use: --lim######

This shouldn't be necessary—it could be that there's something in the configuration that's causing that first startup to timeout or fail (usually related to something in the systemd configuration, or with permissions, something like that).
Adding this task causes all the automated builds to fail: https://travis-ci.org/geerlingguy/ansible-role-jenkins/builds/242112982
Please check out how the test is run in tests/test.yml and the other test playbooks—those are running continuously (once a day) and are currently passing and working in Travis CI. I also can confirm that building a new CentOS 7 instance locally seems to work okay.
Please reopen if you can provide a reproducible set of steps/playbook that will lead to that task failing.

Understood. I'll look into when I get some time. Thanks for taking the time to comment on this. Cheers!


Make new BYU Bomberman Level

One more thought on this. It might be nice to eliminate the other levels. One of the controllers is hooked up to the arrow keys, which means that if a student is just mashing buttons while the demo leader is trying to navigate menus, the student might mess with the navigation. It might be nice just to have a single BYU level, and the demo leader just has to hit enter and won't have to worry about the students messing with it.

Added the new level, and removed the other levels from the levels folder.

Sweet awesome.


cannot find init file for powerlevel10k

After the recent update, zsh_unplugged cannot automatically find the init file for romkatv/powerlevel10k
Plugin has no init file 'romkatv/powerlevel10k'.

The init file for powerlevel10k should be powerlevel10k.zsh-theme, maybe adding .zsh-theme could fix this.

Thank you! This is resolved. I'll add unit tests today to make sure future changes do not break things like this again so that you and others can safely rely on this project to work for you going forward.

ZTAP tests have been added. If there are other plugins that have issues, let me know and I'll add a unit test for them.


step directory: is ignored

after updating woodpecker to the latest image
  build-web:
    image: *node_image
    directory: web/
    commands:
      - corepack enable
      - pnpm install --frozen-lockfile
      - pnpm build

the directory is ignored
I suspect it is because of #4151  ... future investigation needed :/
cc @pat-s

https://ci.woodpecker-ci.org/repos/3780/pipeline/22341/9

rollback our instance to last image that is known to work: https://github.com/woodpecker-ci/infrastructure/commit/ca51fb2e79695e2ef1131c418543a3c2042644df

drafted an pull (non ready) #4319


Warm-up

Allow to do some number of read operations before gathering statistics, to warm-up the database. The length of warm-up can be limited by the number of operations or by the time period.

please don't include unrelated formatting changes.


Cannot get it to work with NextJs version >=10.0.6. Works fine with NextJs 10.0.5.

Describe the bug
warn  next-i18next/node_modules/i18next-fs-backend/cjs/readFile.js
Critical dependency: the request of a dependency is an expression
next-i18next/dist/commonjs/serverSideTranslations.js
Critical dependency: the request of a dependency is an expression

Error: Cannot find module '/Users/username/Development/test/next-i18next/examples/simple/next-i18next.config.js'
    at webpackEmptyContext (eval at ../../dist/commonjs sync recursive (/Users/username/Development/test/next-i18next/examples/simple/.next/server/pages/index.js:103:1), <anonymous>:2:10)
    at eval (webpack-internal:///../../dist/commonjs/serverSideTranslations.js:102:120)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  code: 'MODULE_NOT_FOUND'
}

Occurs in next-i18next version
8.1.2
Steps to reproduce

Clone this repo
upgrade next version in examples/simple/package.json to 10.0.6 or 10.0.7 or 10.0.8 or 10.0.9
run yarn run-example or yarn test

Expected behaviour
The example app to successfully demo next-i18next.
Screenshots
If applicable, add screenshots or a GIF to help explain your problem.
OS (please complete the following information)

Device: [e.g. MBP 2018 13"]
Browser: [e.g. Chrome 70.0.3538.77]

Additional context
Add any other context about the problem here.

Hi @thiraphong-c – can you please open an issue on the NextJs repo? If v10.0.5 works, but >=v10.0.6 does not, I would argue that they have broken semver and need to release a bug fix.

Hi, is this issue resolved? I also encountered this issue, but in my case, the same error arouses even with next.js 10.0.5

@Pomevak I haven't heard from @thiraphong-c. Can you please open an issue on the NextJs repository with steps to reproduce the error?

@Pomevak I haven't heard from @thiraphong-c. Can you please open an issue on the NextJs repository with steps to reproduce the error?

OK, I will try and see if I can make a minimal reproduction.

In next.config.js, I wrote
{
    target: "serverless"
}

commenting this line out have solved the issue for me .
So next-i18next does not support severless mode?

@Pomevak The next-i18next package definitely supports serverless deployments, and the example itself is hosted on Vercel.
You can see an explanation about the target: "serverless" config option here. Basically – don't use it.


[ADDON] - Open Telemetry

Add support for Open Telemetry add-on.
https://github.com/open-telemetry/opentelemetry-helm-charts

Duplicate of #278 . Closing.


ios webGl Game navigation with accessibility mode enabled

Hi, i am new you unity and webGL and English is not my native language so pardon my mistakes. I have created a Webgl Game on unity 2017.3.1 for visually impaired users. for that purpose. i used a box which navigates through all the game object through swiping on screen. its working fine on android(while turning accessibility on) but on iPhone. when i run the build on safari or chrome(while turned on voiceOver). neither the box is navigating on swiping nor any of the game UI is working . The swiping of screen as well as button clicking and other option are not reacting at all. kindly need help
Thank you.

This is seemingly unrelated to Botkit.


Processing a trace with a TraceClockType of SystemTime throws an exception

Processing a trace with a TraceClockType of SystemTime throws an exception:
System.ArgumentOutOfRangeException: 'The TraceTimestampValue does not represent a valid value in the range of the trace's clock. Parameter name: traceTimestampValue'
This exception was originally thrown at this call stack:
Microsoft.Windows.EventTracing.Metadata.SystemTimeTraceTimestampContext.GetRelativeTimestamp(Microsoft.Windows.EventTracing.TraceTimestampValue)
Microsoft.Windows.EventTracing.Metadata.SystemTimeTraceTimestampContext.GetDateTimeOffset(Microsoft.Windows.EventTracing.TraceTimestampValue)

With version 0.3.3 it works now without an exception.


Adding Navigation Design Patterns

Issue: https://github.com/grommet/hpe-design/issues/19

waiting on small style fix for blockquote tags (to add margins to both top and bottom)

This will need to live under the "Learn" area of the new grommet.io
When  releasing the new design, we'll add this content in.

Let's remove the Dashboard and Actions patterns on the new website, and replace the Navigation page with the content from this PR.

This PR is live here: http://grommet.github.io/docs/navigation


python3Packages.angr: 9.0.9297 -> 9.0.9355

Motivation for this change
Update the angr stack to 9.0.9355
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 132579 run on x86_64-linux 1

14 packages built:

python38Packages.ailment
python38Packages.angr
python38Packages.angrop
python38Packages.archinfo
python38Packages.claripy
python38Packages.cle
python38Packages.pyvex
python39Packages.ailment
python39Packages.angr
python39Packages.angrop
python39Packages.archinfo
python39Packages.claripy
python39Packages.cle
python39Packages.pyvex

Result of nixpkgs-review pr 132579 at de42ea90 run on x86_64-linux 1

14 packages built successfully:

python38Packages.ailment
python38Packages.angr
python38Packages.angrop
python38Packages.archinfo
python38Packages.claripy
python38Packages.cle
python38Packages.pyvex
python39Packages.ailment
python39Packages.angr
python39Packages.angrop
python39Packages.archinfo
python39Packages.claripy
python39Packages.cle
python39Packages.pyvex

1 suggestion:

warning: python-include-tests
Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.
Near pkgs/development/python-modules/ailment/default.nix:27:0:
   |
27 |     description = "The angr Intermediate Language";
   | ^

Result of nixpkgs-review pr 132579 run on x86_64-linux 1

14 packages built:

python38Packages.ailment
python38Packages.angr
python38Packages.angrop
python38Packages.archinfo
python38Packages.claripy
python38Packages.cle
python38Packages.pyvex
python39Packages.ailment
python39Packages.angr
python39Packages.angrop
python39Packages.archinfo
python39Packages.claripy
python39Packages.cle
python39Packages.pyvex

It's interesting that the bot also tries to build broken packages.

Looks like only failing for aarch64?

Looks like only failing for aarch64?

One of the dependencies doesn't support aarch64. Thus, down the line the builds for aarch64 will fail.

Yes the bot builds on aarch64 also it should not because it is marked as broken.


Questions from CfJJ for MA Prosecution Dataset

Notes from meeting with Sana on 6/26/20 that generated these questions
For all questions - only concerned with those under 21:

[ ] 1) Who is eligible today? People with only one charge that is not part of the list of (ineligible) offenses.
[ ] 2a) If we only change the number of offenses, how many are eligible based on only having one incident (that does not include an ineligible offense)?
[ ] 2b) How many would be eligible if we only limited to only exclude sex-based offenses or murder?
[ ] 3a) How many would be eligible based on who has not been found guilty (given current offenses that are eligible for expungement)?
[ ] 3b) How many would be eligible based on who has not been found guilty, except murder or sex offenses?

Clarifications from CfJJ:

for 2b and 3a, we wouldn't have the one incident limitation, but how hard is it to estimate if the one incident limitation was taken into consideration?  it might be helpful to have a comparison of both variables unless it's too time consuming, and you'd prefer i stick with one

Began writing scripts for this in How many expungable notebook. Not clear that we need a second issue #154  for this -- seems liek that's duplicate tracking, but I'll leave that to you @jeremylang

Sent question and status update emails to CfJJ. Started conversation about call or virtual meeting to go over the report.

Updates on the new policing bill from Sana, December 2020:
Just a quick update...the policing bill that is currently being negotiated has an expansion of the expungement law, that if the bill passes these changes would apply:

eligibility will be based on number of cases, not number of offenses
increases the number of eligible cases to a maximum of two cases that end in a conviction/adjudication and two cases that end in any other disposition that is not a conviction/adjudication
-we were unable to cut down the categories of offenses that are ineligible.  While our original bill would have only kept sex offenses and murder, our most recent negotiation was to not cut the offenses, but only limit ineligibility to those offenses that ended in a "felony conviction", meaning any of the offenses that are misdemeanors would then be eligible, cases that end with a non-conviction would be eligible, and any juvenile case (whether it ended in an adjudication or not would be eligible).  We are debating right now what we want to file next session whether we want to go for the "felony conviction" ineligibility and/or removing a few specific offenses altogether.

Updates on the new policing bill from Sana, December 2020:
Just a quick update...the policing bill that is currently being negotiated has an expansion of the expungement law, that if the bill passes these changes would apply:

eligibility will be based on number of cases, not number of offenses
increases the number of eligible cases to a maximum of two cases that end in a conviction/adjudication and two cases that end in any other disposition that is not a conviction/adjudication
-we were unable to cut down the categories of offenses that are ineligible.  While our original bill would have only kept sex offenses and murder, our most recent negotiation was to not cut the offenses, but only limit ineligibility to those offenses that ended in a "felony conviction", meaning any of the offenses that are misdemeanors would then be eligible, cases that end with a non-conviction would be eligible, and any juvenile case (whether it ended in an adjudication or not would be eligible).  We are debating right now what we want to file next session whether we want to go for the "felony conviction" ineligibility and/or removing a few specific offenses altogether.

@JoelTennyson - are these questions fully answered with your data visualizer? Or are there questions that we still need to answer from this question set? If they are answered, we need to add the link to the answers.


Allow pinch-zoom for mobile

If we look at the calendar on mobile, some of the information for a course is cut off (particularly the location). We should add some way to pinch-zoom and allow the user to see more information.
Example:

Too tough since we have no control over the calendar. We would need to change our calendar library to something that supports this.


Enable cache access

Hey,
I really want to be able to edit the cache. So I started refactoring a bit. Two things:

As you can see, I started to replace the Arc<> with just clones of String and PathBuf. Usage was inconsistent over the files and it seems unnecessary. I assume you wanted to gain speed/save RAM? Copying FFmpeg Param struct even over like 20 samples and 10 CRF search runs should be negligible RAM usage compared to what the video encoder takes, and speed should also not nearly be a bottleneck to any use case. If anything, the whole thread-safe accessing over tokyo threads and mutex locking could make things slower, compared to cheap memcpys (atleast thats my understanding). Would you be ok with me simplifying those structs?

Can I ask why you use sled? Again, performance should not be an issue (a few ms lookup vs minutes to hours of encoding), and with JSON we have the possibility to

edit the cache with text editors (deleting entries primarily),
version it in GIT (if experimenting or discovering bugs), and best of all
saving cleartext params instead of only hashes (so we can precisely look up past runs).

Would you allow me to change the backend to JSON (or similar)? It would probably make things easier instead of maintaining two formats (and should also make development alot easier since you can just look at the runs directly).

I don't want to come of as criticizing, I'm sure you noticed I enjoy working with and on your program and design decisions in the start are hard if not impossible to get right :) so thanks again!

I do intend to replace sled, I think with sqlite. Having multiple small json files is very inefficient on most filesystems, having one/few big json files is kinda just rolling our own embedded db.

I really want to be able to edit the cache

This isn't something I would generally envision users doing. What's your motivation for doing so?
For example, perhaps we could include time with cache and offer a cache pruning command + cache clearing command in the binary.

saving cleartext params instead of only hashes (so we can precisely look up past runs).

It would be interesting to more eagerly lookup info for crf-searches, but I do like the simplicity of the current approach  and determinism of the search itself. I also will probably keep hashing filenames as it makes the cache less sensitive privacy wise.


New method to define monitor point in Icepak for an specific object

This method assigns a monitor point in the centroid of the specific object and returns the name of the monitor created

LGTM.


chore: update readme

update readme

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I don't think this documentation is valid. Closing because stale.


Move connection into geoms

See https://github.com/camptocamp/c2cgeoportal/issues/2648#issuecomment-261486676

Thanks :-)


Fix #49 and some other bugfixes

I kept the changes confined per-commit so you can cherry-pick if you choose.

Thanks so much for this. I will merge all the commits over.


Add python round-trip testing to "pxt checkdocs"

Decompile doc snippets to python and compile them back to sts. Also there's an option in the code to do a syntax comparison with the before and after .ts. This succeeds for 528/656 snippets in minecraft docs, but it's brittle and not fully reliable so we don't enable it by default, but when combined with manual scrubbing it is useful for finding py2ts & ts2py issues.
Fixes a few minor py2ts/ts2py issues.

in minecraft 528/656 snippets round-trip and match automatically, the rest look correct with quick manual inspection
in arcade 411/650 snippets round-trip and match automatically, the rest look correct with quick manual inspection
in microbit 513/910 snippets round-trip and match automatically, the rest look maybe correct with quick manual inspection

TODO:

[x] one compile error in microbit docs
[x] move flag to command line

Strange, I'm getting linting errors:
ERROR: /home/travis/build/Microsoft/pxt/cli/cli.ts:5079:29 - misplaced opening brace
ERROR: /home/travis/build/Microsoft/pxt/cli/cli.ts:5094:29 - misplaced opening brace
Yet I don't get these errors when I run "npm test" locally, and furthermore these lines haven't been touched in >1 year.. what's going on?

[ ] fix


[nrf fromtree] fixes for failing tests

Fixes for watchdog and timer instability failures

@PerMac Can you open manifest PR?

The check for twister tests in the CI is red but my PR doesn't introduce any regression. The failing tests https://jenkins-ncs.nordicsemi.no/job/latest/job/sdk-zephyr/job/PR-548/7/#showFailuresLink
are the same as the ones in the sdk-zephyr/master: https://jenkins-ncs.nordicsemi.no/job/latest/job/sdk-zephyr/job/master/336/#showFailuresLink
It appears no twister tests were executed in this plan since the upmerge, when sanitycheck script changed its name to twister. The upmerge itself was red on sdk-zephyr due to this name change.
The fix for this change was introduced only recently in the CI pipeline.
I've tested in my HW test plan the corresponding manifest PR to sdk-nrf and there is no regression introduced by the commits I add.

Closed, included in #559


LGT328P/Timer 3

Timer 3  is added to the LGT8F328 comparing with the Atmega328. But the according application about the timer 3 is a rare example. Please add the timer 3 example.thank you very much

timer3 is not had in office atmega328p, so need add isr vector in avr-gcc toolchain


⚠ Bofrost.fr has degraded performance

In 1f4d2d2, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1062 ms

Resolved: Bofrost.fr performance has improved in 6e80d55.


Query regarding linear kernel

Hi,
I am using a linear kernel with '-t 0' option. In order to compute the weight vector, I am reading the model file line by line and adding coefficient*support_vector to my global weight vector. Once I go through all the support vectors, I will have my final weight vector. This was the equivalent of the approach described in libsvm FAQ.
As I am reading my model file line by line, it is taking some extra time for this computation. Is there a simpler way of getting the weight vector for thundersvm.
Thanks for your time.
Regards,
Aditya

ThunderSVM doesn't have that functionality yet, but we will consider implementing it in the future version. Thanks for the feedback!

Thank you.


build-windows.bat fixes

Adding the option to change the target platform(x86/x64)
Generating the .pdb file on debug builds
Correcting comments that called the script 'windows-build.bat' instead of 'build-windows.bat'

@henriquel1997 thanks for the improvement!


saver: added saving into a buffer

This patch adds saving into a buffer functionality into a Saver.
Main purpose of this api now is adding export functionality into ThorVG Viewer
@API Additions:
Result save(std::unique_ptr paint, const std::string& mimeType, uint8_t** buffer, uint32_t* size);

Closing as feature no more needed now


Upgrade Kiali addon to v1.63.1

Please provide a description of this PR:
Upgrades the kiali addon to the latest 1.63.1
Part of https://github.com/kiali/kiali/issues/5773

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

added cherry pick 1.17 label since this is for inclusion with 1.17 addons.

Kiali 1.63.1 is not yet released, but we want this in istio 1.17 so I'm taking the do-not-merge label off so we can merge this. Kiali 1.63.1 will be released before 1.17... and this PR will work now anyway so it is testable (it will just pull in 1.63.0 image, and the only thing different will be the warning saying "1.63.0 is not compatible with Istio 1.17" - that's what we are adding to 1.63.1 - the compatibility matrix update to show 1.63 is compatible with 1.17.

@jmazzitelli: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

gencheck_istio
6add797e0e19893eab5bf9e47c68543a55eace3c
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Bah! never mind - yeah, the kiali 1.63.1 helm charts aren't published and the CI test tries to get it (I guess it runs gen.sh to test). So we'll have to wait for Kiali 1.63.1 to be released. I'll try to see if that can be done this week.

kiali 1.63.1 released - helm charts now available. re-running CI

everything is green and ready - this just needs the appropriate approvals to merge.

In response to a cherrypick label: new pull request created: #43075


CC: Update support location directions

Product Version(s):

Issue:

Link to docs preview:

SME review:

[ ] An SME has approved this change.

DOCS review:

[ ] A member of the docs team has approved this change.

Additional information:

Merge checklist:

[ ] Deploy preview inspected wherever changes were made
[ ] Build completed successfully
[ ] Test have passed

After chatting with Chris a bit I'm going to close this PR.


Property 'json' doesn't exist on type 'd3'

I'm trying to implement d3 example http://bl.ocks.org/mbostock/4062045 using d3-ng2-service. I got d3 instance by this.d3 = d3Service.getD3(); and then I try to call d3.json() function
this.d3.json("miserables.json", function(error, graph){});
In this case compiler throws error "Property 'json' doesn't exist on type 'd3'"
"d3-ng2-service": "1.5.3",
What is the proper way to resolve this?

json is a method of the d3-request module. By design, I have chosen not to include this module with d3-ng2-service.
The rationale is simply to use Angular methods for handling http, rather than using D3. So the recommended approach would be to use Angular's http service instead.
Hope this clarifies.

Yes, that makes sense. Thank you for clarification.


[STORM-1496] Nimbus periodically throws blobstore-related exception

I would like to have this merged in the master as well

+1 Confirmed that blobstore-related error log from nimbus.log are gone. Also it fixes misfunction of 'Topology Visualization'.

@redsanket
Could you create a pull request which targets to master? It is easier to merge & backport.
Thanks for quick fix!

@HeartSaVioR yes I agree, that would be better, I am closing it now and will create a PR for master


Find alternative to NumPy structured arrays

Unlike NumPy, some array libraries do not support structured arrays (e.g., CuPy). Dask seems to have very little dependency on structured arrays, with the two cases I could find being da.unique and (nan|)arg(min|max).
My first idea to resolve the cases above was to check whether _meta supports structured arrays or not, and in the negative case use a dict to store as a replacement. The first difficulty is found in da.unique that uses blockwise to compute chunks, since blockwise doesn't seem to be capable of handling multiple output values today. Since results are independent from each other, one alternative would be to implement it as multiple blockwise operations, but I imagine that could lead to some performance degradation, although it may not be the case anymore after https://github.com/dask/distributed/pull/4967. My most immediate problem only requires the values from np.unique, meaning I could begin by writing an implementation that doesn't use structured arrays, dict and at the same time only calls blockwise once, and extend it over time as there's more demand for data resulting from return_index, return_inverse and return_counts.
For the (nan|)arg(min|max) case I thought using a dict would also be a solution, but there are some challenges there too. For instance, _tree_reduce uses _concatenate2 which obviously can't concatenate a dict. I believe one option would be to write a specialized implementation of _concatenate2 that would be capable of handling dicts, but that's less than ideal since it would be a function with probably a single use in the Dask codebase.
There seems to be some demand for this work, particularly for extending np.unique support to GPUs in https://github.com/dask/dask/issues/7482 and regarding its memory footprint in https://github.com/dask/dask/issues/6828 . For np.unique there's also been some discussion in https://github.com/dask/dask/issues/2851 to implement it as a reduction, which is certainly valuable but would still require structured arrays, so even that wouldn't resolve any of the issues here.
With all the above said I'm wondering what do people feel about the ideas I've come up to allow computing Dask arrays backed by non-NumPy ones (such as CuPy), are those acceptable? If not, are there any other ideas on how we could tackle this problem?
It's also worth noting that there was a request for CuPy to support structured arrays in https://github.com/cupy/cupy/issues/2031, but given the limited amount of Dask code that currently requires it and the complexity of writing GPU code to handle that I'm certain that the effort on the CuPy side would be significantly higher as it would require adding support for that within the entire codebase. Also, having a way to rely on a different approach than structured arrays (even if we keep both for optimal performance in the NumPy case) is certainly beneficial for other array types implementing __array_function__ that don't implement structured arrays.

@mrocklin since you were interested on the sgkit work that was mentioned during the monthly meeting, this is something I have to resolve for that, in case you have ideas to tackle the problem here.

I should have done this before, but I'm tagging @dask/array in case people have ideas too. To handle the second case ((nan|)arg(min|max)), I've produced https://github.com/dask/dask/pull/8014 as a possible solution. This resolved the issue with CuPy, and although IMO it doesn't pollute the code too much, it stillrequires the _concatenate2 change mentioned in the description.

With #8014 & #8021 in, I believe this can be closed.


Update netty-all, netty-codec-http to 4.1.54.Final

Updates

io.netty:netty-all
io.netty:netty-codec-http

from 4.1.53.Final to 4.1.54.Final.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.netty" } ]

labels: library-update

Coverage remained the same at 0.0% when pulling c160fb4c8cad85f5c8963a555a0c90b00cdd0f85 on scala-steward:update/netty-all-4.1.54.Final into 7044e3c443bf16e50d3e7a8872fbb9bd7bf0eef1 on andyglow:develop.

Coverage remained the same at 0.0% when pulling c160fb4c8cad85f5c8963a555a0c90b00cdd0f85 on scala-steward:update/netty-all-4.1.54.Final into 7044e3c443bf16e50d3e7a8872fbb9bd7bf0eef1 on andyglow:develop.


samtools: 1.13 -> 1.17

Description of changes
Our biologists need this upgrade.
htslib, which is a library part of samtools, must also be upgraded to 1.17 for samtools 1.17 to be compiled
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 225865 run on x86_64-linux 1

1 package marked as broken and skipped:

platypus

10 packages failed to build:

python310Packages.cnvkit
python310Packages.cnvkit.dist
python311Packages.cnvkit
python311Packages.cnvkit.dist
python311Packages.htseq
python311Packages.htseq.dist
python311Packages.pysam
python311Packages.pysam.dist
truvari
truvari.dist

16 packages built:

angsd
bcftools
deeptools
deeptools.dist
delly
htslib
mosdepth
nimPackages.hts-nim
octopus-caller
python310Packages.htseq
python310Packages.htseq.dist
python310Packages.pysam
python310Packages.pysam.dist
samtools
tebreak
tebreak.dist

Hi @bzizou and @natsukium
There is already one pull request for htslib 1.17 that has to be merged first
https://github.com/NixOS/nixpkgs/pull/219663

The derivation of truvari is too old and already broken.
The failure of python311Packages.pysam and python311Packages.htseq is fixed in #222906.
The python3Packages.cnvkit that depends on python3Packages.pomegranate already failed in the test regardless of this change.
I think this PR change is good.

Thank you!


mkdir /tmp/blobfuse on node required for working installation

What happened:
I followed the README instructions to install blobfuse on kubernetes (AKS).
My pod would not start due to errors
$ kubectl describe pod nginx-flex-blobfuse

...
Events:
  Type     Reason       Age                  From                                         Message
  ----     ------       ----                 ----                                         -------
  Warning  FailedMount  42m (x13 over 132m)  kubelet, aks-devuksouth-26257669-vmss000002  Unable to attach or mount volumes: unmounted volumes=[test], unattached volumes=[default-token-c5xpw test]: timed out waiting for the condition
  Warning  FailedMount  17m (x38 over 135m)  kubelet, aks-devuksouth-26257669-vmss000002  Unable to attach or mount volumes: unmounted volumes=[test], unattached volumes=[test default-token-c5xpw]: timed out waiting for the condition
  Warning  FailedMount  10m (x70 over 137m)  kubelet, aks-devuksouth-26257669-vmss000002  MountVolume.SetUp failed for volume "test" : invalid character 'E' looking for beginning of value

I followed the troubleshooting guide by inspecting the node logs with tail -f /var/log/blobfuse-driver.log
Wed Jun 10 14:06:29 UTC 2020 EXEC: mkdir -p /var/lib/kubelet/pods/143d746f-a5e6-4723-9b35-fe9a692df06d/volumes/azure~blobfuse/test
Wed Jun 10 14:06:29 UTC 2020 INF: AZURE_STORAGE_ACCESS_KEY is set
Wed Jun 10 14:06:29 UTC 2020 INF: export storage account - export AZURE_STORAGE_ACCOUNT=mmonteithblobtest
Wed Jun 10 14:06:29 UTC 2020 EXEC: blobfuse /var/lib/kubelet/pods/143d746f-a5e6-4723-9b35-fe9a692df06d/volumes/azure~blobfuse/test --container-name=test --tmp-path=/tmp/blobfuse -o allow_other  --file-cache-timeout-in-seconds=120
Wed Jun 10 14:06:29 UTC 2020 ERROR: { "status": "Failure", "message": "Failed to mount device /dev/ at /var/lib/kubelet/pods/143d746f-a5e6-4723-9b35-fe9a692df06d/volumes/azure~blobfuse/test, accountname:mmonteithblobtest, error log:Wed Jun 10 14:06:29 UTC 2020 EXEC: blobfuse /var/lib/kubelet/pods/143d746f-a5e6-4723-9b35-fe9a692df06d/volumes/azure~blobfuse/test --container-name=test --tmp-path=/tmp/blobfuse -o allow_other --file-cache-timeout-in-seconds=120" }

After some investigation, I found that the /tmp/blobfuse directory didn't exist and it needed to be created.
After running mkdir /tmp/blobfuse on the node, my pod started working
Wed Jun 10 14:08:31 UTC 2020 EXEC: mkdir -p /var/lib/kubelet/pods/143d746f-a5e6-4723-9b35-fe9a692df06d/volumes/azure~blobfuse/test
Wed Jun 10 14:08:31 UTC 2020 INF: AZURE_STORAGE_ACCESS_KEY is set
Wed Jun 10 14:08:31 UTC 2020 INF: export storage account - export AZURE_STORAGE_ACCOUNT=mmonteithblobtest
Wed Jun 10 14:08:31 UTC 2020 EXEC: blobfuse /var/lib/kubelet/pods/143d746f-a5e6-4723-9b35-fe9a692df06d/volumes/azure~blobfuse/test --container-name=test --tmp-path=/tmp/blobfuse -o allow_other  --file-cache-timeout-in-seconds=120
Wed Jun 10 14:08:31 UTC 2020 INFO: {"status": "Success"}

What you expected to happen:
/tmp/blobfuse should not need to be manually created
Environment:

Kubernetes AKS cluster (v1.16.9)
Linux

Looks like this bug has already been fixed in blobfuse master, but not released yet.
https://github.com/Azure/azure-storage-fuse/pull/413/files

correct, blobfuse 1.2.3 is breaking, to mitigate this issue, I think we could mkdir /tmp/blobfuse in our flexvolume blobfuse driver side

this issue is fixed by blobfuse-flexvolume:1.0.13, try with
kubectl apply -f https://raw.githubusercontent.com/Azure/kubernetes-volume-drivers/master/flexvolume/blobfuse/deployment/blobfuse-flexvol-installer-1.9.yaml


Include btc fees

As a miner, I am interested in seeing my total spent that includes the fees. Is that possible?

not without requiring a dependency to bitcoind, or a transaction indexer, so I'd like to keep stacks-dump simple to run and only depend on a node

not without requiring a dependency to bitcoind, or a transaction indexer, so I'd like to keep stacks-dump simple to run and only depend on a node


Simplified constructors, copy constructors & assignment operators

what
Several constructors, copy constructors & assignment operators in the codebase can be simplified or even removed because they're not used.
why
Simplify the codebase making it easier to maintain.
NOTE: In the case of RuleMessage, it addresses one of the FIXME in the codebase, see here
misc
This is part of a series of PRs to improve performance of the library (5/n). Previous: #3231

Failed conditions  8.8% Duplication on New Code (required ≤ 3%)

This is reporting duplicate code on changes where there are only changes to function signatures and where data is retrieved from:

examples/reading_logs_via_rule_message/reading_logs_via_rule_message.h
examples/using_bodies_in_chunks/simple_request.cc
src/transaction.cc

NOTE: I think Sonarcloud would like the two examples mentioned above to share the code that logs rules, which is clearly identical. However, in the case of examples, it makes sense that they're independent (even though they have "duplicate" code) to make it easier to use them as a simple reference and starting point for library users.

PS: I'll take a look at the Transaction constructors and see if they can be simplified as well. They actually look identical, other than the id parameter, so they could be refactor to use a delegating constructor.

Done
PS: Notice that Sonarcloud once again run on stale code after the push, and is still showing the old version of transaction.cc and thus still flags duplicate code (in code that no longer exists). ¯_(ツ)_/¯

Issues  17 New issues  0 Accepted issues

Edited commits to address the following issues reported by Sonarcloud:

headers/modsecurity/rule_marker.h

Merge this "if" statement with the enclosing one

Existing code, not introduced in this PR.

headers/modsecurity/rule_message.h

Do not use the constructor's initializer list for data member "m_severity". Use the in-class initializer instead (8)

Existing code, not introduced in this PR.

headers/modsecurity/rule_unconditional.h

Add a using-declaration to this derived class to inherit the constructors of "RuleWithActions", and remove the ones you manually duplicated

headers/modsecurity/rule_with_actions.h

Use "override" or "final" instead of "virtual"

src/rule_with_actions.cc

This function should be declared "const"

Existing code, not introduced in this PR.

src/rule_with_operator.cc

"std::move" should not be called on a const object.

Nice catch!

src/transaction.cc

Make the type of this parameter a pointer-to-const. The current type of "id" is "char *"

Existing code, not introduced in this PR.

Did not address:

src/transaction.cc

Replace this use of "void *" with a more meaningful type (2)

logCbData is a C-style opaque pointer required by the C callback interface.

Replace this const reference to "std::string" by a "std::string_view"

I wanted to address this, but using a std::string_view (which are passed by value) introduces a cppcheck warning, which would require a suppression.

This is part of a series of PRs to improve performance of the library (5/n). Previous: #3231

Even though the goal of the PR is to simplify the codebase, commit 2ad87f640f4445e2b6a62dc7bf68a94d0c382d98 (Reference RuleWithActions & Transaction object instead of copying values in RuleMessage) provides some performance improvements because it removes a number of variables copied from the associated rule & transaction.
This change increases performance of the benchmark test by about 2,4%-3,5%.
These results were obtained running the benchmark test with 100'000 iterations with the OWASP CRS v4 and are presented as a reference:

Mac mini (1st generation)

v3/master: 83,44 secs
This PR: 81,44 secs (2.4% improvement)

Debian (bullseye) container running on WSL on Windows Dev Kit 2023

v3/master: 121,83 secs
This PR: 117,45 secs (3.59% improvement)

Hi @eduar-hte
thanks again for this great PR!
Just one reminder to myself:

PS: I'll take a look at the Transaction constructors and see if they can be simplified as well. They actually look identical, other than the id parameter, so they could be refactor to use a delegating constructor.

Seems like the Transaction class public constructors are (not) a bit different that it documented in README.md:
Transaction *modsecTransaction = new Transaction(modsec, rules);

or in source documentation, so we need to align them.


Updating sample values to always return results

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Presence - add notification option

I use presence and have telegram bot activated, works all fine.
But I wish I could have an option within presence to activate/deactivate telegram notifications per address/person - I don't need telegram to notify me I arrived at home 😉

Added in v5.0.13
https://github.com/SeydX/homebridge-fritz-platform/blob/c4d669f1504594586da8f925e5b82177b2cfe927/example/example-config.json#L92


User with VIEWER role should have access to view everything unless he is a team member of a particular work

Overview

A user with VIEWER  roles should be able to view the workplan tab, but not make any changes
But if they are a TEAM MEMBER, then they should be able to edit the work plan

Actions included:

Add Task
Add Milestone
Edit Task
Edit Milestone
Row edits

@dinesh-aot,
I tested it with Taya, and though she wasn't able to add tasks or Milestones, she could:

edit status
add/edit issues
add team member

She should not be able to do anything like that if she is not a member of a team

This ticket is about only the first tab the workplan tab, the below ticket is for the status and issues
https://app.zenhub.com/workspaces/epictrack-63891ea941d309001fa292cf/issues/gh/bcgov/epic.track/1586

@Connor1892, should we create a separate ticket to disable stuff on the Issues/Status/Team/Nations tabs?
cause Viewer now can do all those things even if they are not a member of a team

@maria-kuzmenko linked both cards as dependant on this one


Question Page designing

Related Issue
Design: Question page
Closes: #76
Describe the changes you've made
I have designed the question pages from scratch in Figma using the mentioned parameters in the issue
Checklist:

[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.

Screenshots

Original
Updated

**

**

@Chadha93 Great work😊✨


i2c crashes without giving any info (IDFGH-1128)

Environment

Development Kit:     Goouuu-ESP32
Module or chip used:  ESP32-WROOM-32
IDF version (run git describe --tags to find it):     v3.3-beta1-210-gb173cf281
Build System:         [Make]
Compiler version (run xtensa-esp32-elf-gcc --version to find it):
1.22.0-80-g6c4433a5
Operating System:     [Windows]
Power Supply:         [USB]

Problem Description
cpu reset without giving out any info after operating on i2c after couple thousand reads
Expected Behavior
no crash/or crash giving some info for debug
Actual Behavior
cpu reset
Steps to repropduce
connect a i2c device, run the following code
// It helps if you attach a picture of your setup/wiring here.
Code to reproduce this issue

`#include <stdio.h>
#include "driver/i2c.h"
#include "driver/uart.h"
#include "sdkconfig.h"

uint8_t _i2cAddr = 0x4B;
#define ACK_EN 0x01
#define DEFAULT_UART_TIMEOUT_MS 30

void app_main()
{

    uart_config_t uart_config = {
        .baud_rate = 115200,
        .data_bits = UART_DATA_8_BITS,
        .parity = UART_PARITY_DISABLE,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE};
    uart_param_config(UART_NUM_1, &uart_config);
    uart_set_pin(UART_NUM_1, GPIO_NUM_1, GPIO_NUM_3, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);
    uart_driver_install(UART_NUM_1, 2048 * 2, 0, 0, NULL, 0);

    i2c_config_t conf;
    conf.mode = I2C_MODE_MASTER;
    conf.sda_io_num = 22;
    conf.scl_io_num = 21;
    conf.sda_pullup_en = GPIO_PULLUP_ENABLE;
    conf.scl_pullup_en = GPIO_PULLUP_ENABLE;
    conf.master.clk_speed = 100000;
    i2c_param_config(I2C_NUM_0, &conf);
    i2c_driver_install(I2C_NUM_0, I2C_MODE_MASTER, 0, 0, 0);

    uint8_t buffer[4];
    esp_err_t errx;
    while (1)
    {
        uart_write_bytes(UART_NUM_1, (const char *)"1,", 2);
        i2c_cmd_handle_t cmd = i2c_cmd_link_create();
        uart_write_bytes(UART_NUM_1, (const char *)"2,", 2);
        cmd = i2c_cmd_link_create();
        uart_write_bytes(UART_NUM_1, (const char *)"3,", 2);
        i2c_master_start(cmd);
        uart_write_bytes(UART_NUM_1, (const char *)"4,", 2);
        i2c_master_write_byte(cmd, (_i2cAddr << 1) | I2C_MASTER_READ, ACK_EN);
        uart_write_bytes(UART_NUM_1, (const char *)"5,", 2);
        i2c_master_read(cmd, buffer, 4, (i2c_ack_type_t)0x02);
        uart_write_bytes(UART_NUM_1, (const char *)"6,", 2);
        i2c_master_stop(cmd);
        uart_write_bytes(UART_NUM_1, (const char *)"7,", 2);
        for (int round = 1; round <= 64; round++)
        {
            errx = i2c_master_cmd_begin(I2C_NUM_0, cmd, DEFAULT_UART_TIMEOUT_MS / portTICK_PERIOD_MS);
            if (errx == ESP_OK)
            {
                break;
            }
            else if ((errx != ESP_OK) && (round < 64))
            {
                continue;
            }
            else
            {
                i2c_cmd_link_delete(cmd);
                uart_write_bytes(UART_NUM_1, (const char *)"receive ERROR\n", 20);
            }
        }
        i2c_cmd_link_delete(cmd);
        uart_write_bytes(UART_NUM_1, (const char *)"8\n", 2);
    }
    vTaskDelete(NULL);
}
`

// If your code is longer than 30 lines, GIST is preferred.
Debug Logs

we have a program monitor the uart output when it crashes, here is what we got:

1,2,3,4,5,6,7,8
1,2,3,4,5,6,7,8
1,2,3,4,5ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
count of rcv: 9106 , millis:37539603
1,2,3,4,5,6,7,8
1,2,3,4,5,6,7,8
1ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
count of rcv: 6914 , millis:25899271
1,2,3,4,5,6,7,8
1,2,3,4,5,6,7,8
1,2,3,4,5,6,ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
count of rcv: 6911 , millis:25906526
1,2,3,4,5,6,7,8
1,2,3,4,5,6,7,8
1ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
count of rcv: 6913 , millis:25868028

Other items if possible

[ ] sdkconfig file (attach the sdkconfig file from your project folder)
[ ] elf file in the build folder (note this may contain all the code details and symbols of your project.)
[ ] coredump (This provides stacks of tasks.)

taking out uart part, I got back trace:
`abort() was called at PC 0x4008271f on core 0
Backtrace: 0x40084dec:0x3ffb47d0 0x40085015:0x3ffb47f0 0x4008271f:0x3ffb4810 0x40082740:0x3ffb4830 0x4008288d:0x3ffb4860 0x400d8b12:0x3ffb4880 0x400dbd6d:0x3ffb4b90 0x40082695:0x3ffb4bc0 0x400e18f7:0x3ffb4c10 0x400e2637:0x3ffb4c30 0x400d2339:0x3ffb4c70 0x400d0b22:0x3ffb4cb0 0x4008776d:0x3ffb4cd0
Rebooting...
ets Jun  8 2016 00:22:57
rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
`

Hi, @trustmiao
If you tried 64 times and all failed, "i2c_cmd_link_delete" will be executed twice. I think this caused a crash.

We will rewrite it in better way, however, that wasn't where the esp32 reset, we could take out the whole part of code in  for {} , and it still resets

@trustmiao
Do you have WiFi task or  some of the tasks that will erase the flash?

i don't think so, is there a way to check just for sure?

address decoded with idf monitor
``
abort() was called at PC 0x4008271f on core 0
0x4008271f: lock_init_generic at G:/msys32/home/deyeen/esp/esp-idf/components/newlib/locks.c:81
Backtrace: 0x40084dec:0x3ffb47d0 0x40085015:0x3ffb47f0 0x4008271f:0x3ffb4810 0x40082740:0x3ffb4830 0x4008288d:0x3ffb4860 0x400d8b12:0x3ffb4880 0x400dbd6d:0x3ffb4b90 0x40082695:0x3ffb4bc0 0x400e18f7:0x3ffb4c10 0x400e2637:0x
3ffb4c30 0x400d2339:0x3ffb4c70 0x400d0b22:0x3ffb4cb0 0x4008776d:0x3ffb4cd0
0x40084dec: invoke_abort at G:/msys32/home/deyeen/esp/esp-idf/components/esp32/panic.c:707
0x40085015: abort at G:/msys32/home/deyeen/esp/esp-idf/components/esp32/panic.c:707
0x4008271f: lock_init_generic at G:/msys32/home/deyeen/esp/esp-idf/components/newlib/locks.c:81
0x40082740: lock_acquire_generic at G:/msys32/home/deyeen/esp/esp-idf/components/newlib/locks.c:134
0x4008288d: _lock_acquire_recursive at G:/msys32/home/deyeen/esp/esp-idf/components/newlib/locks.c:171
0x400d8b12: _vfprintf_r at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdio/../../../.././newlib/libc/stdio/vfprintf.c:860 (discriminator 2)
0x400dbd6d: vprintf at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdio/../../../.././newlib/libc/stdio/vprintf.c:39
0x40082695: esp_log_write at G:/msys32/home/deyeen/esp/esp-idf/components/log/log.c:215
0x400e18f7: i2c_cmd_link_append at G:/msys32/home/deyeen/esp/esp-idf/components/driver/i2c.c:1036
0x400e2637: i2c_master_stop at G:/msys32/home/deyeen/esp/esp-idf/components/driver/i2c.c:1036
0x400d2339: app_main at G:/esp32_apps/i2c_bno_debug/main/i2c_example_main.c:50
0x400d0b22: main_task at G:/msys32/home/deyeen/esp/esp-idf/components/esp32/cpu_start.c:518
0x4008776d: vPortTaskWrapper at G:/msys32/home/deyeen/esp/esp-idf/components/freertos/port.c:403
``

Please check the return value of i2c_cmd_link_create()

thanks alot, problem solved, we should not i2c_cmd_link_create twice.


DeepSpeed FusedLamb vs Apex FusedLamb

Nvidia's apex also has a good implementation of FusedLAMB. What's the advantage/disadvantage of using apex's FusedLAMB compared with DeepSpeed's lamb implementation?

Hi Liwei, There should be no disadvantage of using DeepSpeed's lamb implementation compared to Apex. On the other hand, there are certain cases where we have noticed an advantage with DeepSpeed's Lamb compared to Nvidia's apex. The DeepSpeed's lamb implementation allows the user the control the min and the max lamb coefficient value. We have found that tuning these parameters can lead to a better convergence time than using the lamb coefficients without any upper or lower bounds for very large batch sizes.

Thanks. That's really good to know.


[test] Add integration testing using Docker

We can execute integration testing on GitHub CI action.

It's hard to run the oceanbase cluster with the github/azure virtual environment, consider not to use docker for testing.


Question about pickr

How I can modify the postion and style of pickr ?
and how I can use pickr that is show directly in the element without use "useAsButton"?

How I can modify the postion and style of pickr?

What exactly do you want to modify? For styles just use plain CSS - all classes are prefixed with pcr- though.

and how I can use pickr that is show directly in the element without use "useAsButton"?
Is the inline option what you're looking for?

I use this library but the color pickr not showing in the view only when click on a button, if i don't use "useAsButton" the color pickr not display at all, i want to use this color pickr and show it with the display of the view directly not after a click

How I can change the left and top of color pickr ?

Closing because of inactivity.


[RFR] Allow optional project_id on projection class

First pass at making project_id optional in projection class.
When fetching anything that requires project_id, check to see if undefined, if so, fetch it from atlas_cloud endpoint.
Needs to be timed with prod deploy of atlas_cloud that introduces this route
https://github.com/nomic-ai/atlas_cloud/pull/1812

[!IMPORTANT]
Make project_id optional in AtlasProjection, add datasetId() to fetch it if undefined, update methods, add tests, and coordinate with prod deploy.

Make project_id optional in AtlasProjection class.
Add datasetId() method to fetch project_id if undefined.
Remove error for missing project_id in constructor.
Update getTags(), getTagStatus(), updateTagMask(), and project() to use datasetId().
Add test in project.test.js to verify datasetId() fetches project_id correctly.
Bump version in package.json from 0.11.0 to 0.11.1.
Coordinate with prod deploy of atlas_cloud for new route.

This description was created by  for 91c9a2c7dbf4cfb0fbbabe8847e8d5eafa3307a3. It will automatically update as commits are pushed.

When you merge this in, let's prefix the squashed commit with refactor: or feat:, as we're trying to adhere to semantic versioning


Fix Makefile & logging level in stackdriver sink

This change is

/lgtm

/lgtm for last commit.

/retest


chore: update vendor

What this PR does

update clientset vendor
update version to v0.3.1

why we need it
Special notes for your reviewer:

Jenkins has halted.  I'll merge the pr.


Add ulimit -n 2048 workaround in scripts for mac build

@MichalStrehovsky @schellap PTAL. Turns out this is what was causing mac build issues during the dotnet restore step inside build.proj.

We maintain a very similar build.sh and init-tools.sh to what's in the CoreFX repo. The only mention of ulimit in the CoreFX repo is in the init-tools.sh script and we have that line here too. I'm wondering why it's not kicking in.

@MichalStrehovsky It's not kicking in because init-tools.sh is a child-process and ulimit -n 2048 is only effective within that child process. Changing it in the parent-process build.sh is effective in that process and all children. It's not clear to me why CoreFx doesn't need it though - I'll give it a try when I get a chance.

LGTM. The init script would have to be sourced to get that for the other builds.

Thanks!


Support arguments whose values are None

The docs say: Note that any dictionary key whose value is None will not be added to the URL’s query string.
Is there a chance that this could be controlled by the user to include None values as well? Perhaps with the help of a flag.

@rohithpr What would a None value be turned into?

@Lukasa - null perhaps?
>>> json.dumps({'val':None})
'{"val": null}'

@rohithpr The only place we turn dictionaries into JSON is the json= keyword argument, and that is not put into the URL's query string. Query strings in URLs are not JSON, they're urlencoded key-value pairs.

I should ask: what is your actual problem? What are you trying to achieve that you believe you cannot with requests?

No, I didn't mean to say that it should follow JSON. I just said that None could be converted to null along the lines of JSON.

So again, I'm inclined to ask about what exactly you're trying to do. My reading of this issue is that you want the None value turned into something, but you don't care what it is. That's an unusual request: ordinarily I'd expect someone to have a specific service they're talking to that makes it natural to represent the data in a certain way (namely, to have None in there somewhere).
What's the reasoning here?

I'd like to pass a null value for some arguments in a PUT (or any other) request.
data = { 'p1': 'hello', 'p2': None}
requests.put(url, data)

On doing this, the server gets only 'p1' and not 'p2'.
In the project that I'm working on key:null is used to delete a key from a record while updating the database. I don't have control over the server side code and I'm unable to pass null values using requests.

If you want null that's something you should control by looping over the parameters you are passing and swapping None for 'null'. This is not something requests is going to change.
def convert_None_to_null(params):
    new_params = {]
    for (key, value) in params.items():
        if value is None:
            value = 'null'
        new_params[key] = value
    return new_params

I see.
So, the use of null in this fashion is specific to your server: it is not generally true about the URL query string. Requests tries to restrict itself to the subset of behaviour that is universally understood in the query string. Implementation-specific variances have to handle that themselves: otherwise, requests will grow a million different feature flags that subtly change its behaviour in all sorts of weird ways. This kind of combinatorial expansion of the API and the possible configurations of requests would make it a nightmare to use, maintain, and debug.
@sigmavirus24 has provided you a code snippet that should do what you're looking for (with some small edits). I hope you find that helpful!

@Lukasa @sigmavirus24 - Thanks! :)


Greek and Hebrew drops when exporting to a PDF

I'm using the pandoc plugin in Obsidian that converts the Obsidian markdown to Latex and then to PDF. If I export to docx, it works fine. But when I try to do the Latex --> PDF route, I can only make Everson Mono or Lucida Sans Unicode work (more or less). Any other font does not format the Hebrew. But in a docx export, it looks fine.
My YAML looks like this:

title: Notes on Tabernacle
author: Stuart Thiessen
lang: en-US
documentclass: extarticle
mainfont: Everson Mono
monofont: Everson Mono
toc: true
header-includes:

\usepackage{tcolorbox}
toc-title: "Contents"
fontsize: 9pt
number-sections: true
name: Tabernacle

Your problem is more related to LaTeX usage than Pandoc usage, I guess.
I have a workflow which supports exporting of hebrew and greek, involving obsidian-pandoc at the .md to .tex stage.
For greek and hebrew treatment, I use LuaLaTeX and an automatic detection of greek or hebrew language, which are using their own fonts. If you are interested I may be able to publish a demo somewhere

According to what I've read, you have to use a font that includes Hebrew or Greek characters. The default LaTeX fonts do not include those character sets.
It looks like Limezy found a way to detect Hebrew or Greek and to use an appropriate font only for sections in those languages.

@thiessenstuart on an unrelated and funny note, I attended your presentation at Topeka Bible Church a few weeks ago!

@ParkerRobb  Thank you! You are right. The font does need to include Hebrew and Greek characters. I was using the Greek and Hebrew fonts from the Society for Biblical Literature (https://www.sbl-site.org/educational/biblicalfonts.aspx) and from SIL International but somehow they were not coming quite right. But you are right (and I forgot to return here to mention) that it did turn out to be a LaTeX issue. I ended up having to set up YAML in my Obsidian Markdown file to include the necessary parameters that get passed to LaTeX to get it to render the Greek and Hebrew better. :)
Re; TBC ... Wow! Small world. :)

But still Everson Mono is the only font I have found yet that can render it right. So still something to figure out ... sometime.

I ended up having to set up YAML in my Obsidian Markdown file to include the necessary parameters that get passed to LaTeX to get it to render the Greek and Hebrew better. :)

What did you end up putting in the YAML to get it to somewhat work?

For greek and hebrew treatment, I use LuaLaTeX and an automatic detection of greek or hebrew language, which are using their own fonts. If you are interested I may be able to publish a demo somewhere

@Limezy would you mind sharing your workflow?

@ParkerRobb I'm still fine tuning this rather complex workflow. I plan to publish it as a demo vault or long form article for the (hopefully) coming Obsidian October contest !

@thiessenstuart I just found a solution that allows me to write RTL Hebrew text in a LTR line: take a look at the bottom reply at this forum thread. I got it to work by combining a couple things:

Wrapping the Hebrew text in a   HTML span (which is hidden in Obsidian Live Preview).
Using Linux Libertine font for the whole document.

(I did not add any YAML metadata; the only Pandoc options I specified on the command line were --pdf-engine=xelatex -V papersize:letter -V mainfont="Linux Libertine O".)
Hope you can glean something helpful for your own situation.
Denis' answer on the above forum page talks about using different fonts for different languages, and I found more information in the Overleaf documentation, but I have yet to experiment with it. You might find both of those to be helpful, @Limezy.

About writing RTL Hebrew text in a LTR line I'm not sure to understand what's your problem ? This is allowed by default within Obsidian, without fancy invisible HTML tags ?

@Limezy It works fine if I only have a single RTL word. But if I have a sequence of RTL words inside a predominantly LTR line, both Obsidian and Pandoc incorrectly reverse the word order (and sometimes put the niqqud under the wrong letters).

Can you please copy here a minimal non-working example ? I'm interested as this seems to be quite unfortunate... For obsidian I guess it should be a filed as a prosemirror bug ? For pandoc, do you mean that rtl sections are reversed in the .tex output ?

With long enough RTL parts inside LTR lines, I indeed get a strange line return, that is treated as being LTR. But I guess this is normal ? How would the line return be treated differently in that case ? Is that the problem you are talking about @ParkerRobb ?
Obsidian

LuaLaTeX

@Limezy I've attached a few files to demonstrate what I'm talking about: the original Markdown file, and the Pandoc-generated PDF.
RTL in LTR line test.md
RTL in LTR line test.pdf
Here's the output of an intermediate LaTeX file, which you'll notice changes the HTML span to a \foreignlanguage span:
Notice the inline Hebrew words are reversed. The word \emph{b'Ivrit}
should be second when reading from right to left.

\hypertarget{without-html-span}{%
\section{Without HTML span}\label{without-html-span}}

This is some text with טקסט בעברית in the middle.

\hypertarget{with-html-span}{%
\section{With HTML span}\label{with-html-span}}

This is some text with \foreignlanguage{hebrew}{טקסט בעברית} in the
middle.

The word reversal is hard to reproduce in Obsidian. Sometimes Preview Mode switches the inline RTL words, and sometimes it doesn't. 🤷♂

@ParkerRobb thanks !
As far as I understand, in fact your problem is at LaTeX compilation stage, not exactly Pandoc ?
Indeed, generating a pdf with Pandoc is possible but it's only a shortcut where Pandoc will launch a XeLaTeX compilation of a .TeX intermediate file it has been making. Your .TeX file seems good above, and I get exactly the same on my side.

Your problem should be fixable by using a slightly modified LaTeX template for Pandoc and LuaLaTeX as your engine.
On my side I prefer to restrict Pandoc to doing his own job, that is making the .TeX using my LaTeX template and using Latexmk for the .TeX to pdf stage, which makes possible multiple compilation runs to use Biber / BibTeX.
Here is the output of your example using my standard compilation workflow :

As you can see, you could get rid of these annoying html spans ! I never encountered word reversing so far.
I'll soon publish my templates and scripts that you should be able to adapt, either to still use only Pandoc as your pdf compilation engine going through LuaLaTeX, either using a script to start with Pandoc and continue with Latexmk as I'm doing.

As far as I understand, in fact your problem is at LaTeX compilation stage, not exactly Pandoc ?

Your problem should be fixable by using a slightly modified LaTeX template for Pandoc and LuaLaTeX as your engine.

Interesting. I look forward to seeing the details of how you handle it!
Have you ever encountered word reversing in Obsidian, @Limezy?

@ParkerRobb sure I'll send you a link to my latex template and compilation script even though they are not yet polished enough to be sent to a wider audience within the Obsidian community.
No, I have never experienced any Hebrew or RTL word reversing in obsidian, except for the line jump case I have described above. To be very honest though, I'm not good at Hebrew and most of the work I do is to help other members of our Anthropologie biblique group who are the real ones needing to work with and compile Hebrew inside Obsidian. They have never complained about any word reversal so far.

@Limezy have you been able to organize your solution enough to share?

@ParkerRobb here is a first link where you should have all, but still a bit disorganized and probably requiring a bit of work on your side. Theoretically if you use a Mac you can just download the files, install the fonts, setup your Obsidian and it'll work right away

Nice, thanks! I'll dig in and see what I can figure out.

Hi @thiessenstuart and @Limezy,
I wanted to share with you guys some things I've figured out and conclusions I've reached regarding our multilingual typesetting.
After doing quite a bit of research (and reporting a bug in Pandoc) in the months since I entered this discussion, I've pieced together an Obsidian and Pandoc configuration and workflow that works great for me. In short, I've decided to continue using the HTML lang tags for a couple reasons:

My Obsidian only displays Hebrew correctly consistently, i.e. in the right word order and right-aligned (for blocks), if it's tagged with the language. The HTML tags are hidden in both Reading and Live Preview modes, so their presence isn't a problem for me.
LaTeX and the Babel package can autodetect the language of content, as you mentioned @Limezy, but not if the languages use the same script. If this is the case, the different languages must be explicitly distinguished via tags. Distinguishing languages that use the same script is important for loading the correct hyphenation patterns.

Your method does work, @Limezy, if every language uses a different script. But especially due to the second point, I decided to pursue a more generalized approach that works for any combination of languages, and is not limited to distinguishing languages by script.
If you want to know more, I have uploaded detailed documentation of my multilingual configuration with plenty of references and footnotes.

Looks like an impressive work @ParkerRobb many thanks, I'll have a look whenever possible

Hi @ParkerRobb, impressive work. I now understand much better what kind of problems you were trying to avoid with your solution, that looks indeed very handy. Does the div thing still work in your Obsidian ? When I tried to copy paste your minimal example in my Obsidian it didn't work as I had a kind of code block shown instead of invisible tags.
How would you handle things like
Au commencement, au principe בְּרֵאשִׁית (ראשׁ) patati patata…
In Obsidian, the parenthesis will always go to the left, whereas I want it to remain on the right ! (It's just Hebrew words taken independently from their language into a French sentence). Would be interested to have your advice !


Node-6.x compatability

Expected behavior
It installs when "npm install"ing with node v6
Actual behavior
Does not since current version published in NPM (0.13.22) does not have 6 in engines
Enviroment Details
Node v6

Karma version (output of karma --version):
Relevant part of your karma.config.js file

Steps to reproduce the behaviour

Using node v6 install karma

1.0 has full 6.0 compatability


Targeting more types and fields automatically using reflect

Is it useful to target property names, types, and tags with new methods in this library?
E.g.

A tag targeted by AddTagProvider which is an alias for AddProvider
a type by AddTypeProvider (reflect-based)
a field by AddFieldProvider matching field name?
leaves room for other types of strategies later

It would probably require providing different "targeting" strategies for determining faking values. I understand there'd have to be precedence for applicability too. I think each of these could be valuable, though I don't know if its considered applicable here.
I can provide some implementation/PR if this is within scope. I'm going to try things out tomorrow and come back with some more details.

+1
(especially if you would like to fake values of a dependency where tags can't be used)