fix(add): Don't panic with --offline

For some reason, I defined my own --offline flag and it didn't get
updated with the global --offline flag, so it started failing.
The new test previously paniced and now it doesn't.
Fixes #10814

r? @ehuss
(rust-highfive has picked a reviewer for you, use r? to override)

@bors r+

:pushpin: Commit 18f7dfe0f584d779506991d93483b40815d24296 has been approved by weihanglo

:hourglass: Testing commit 18f7dfe0f584d779506991d93483b40815d24296 with merge f2c52767ad049f190d32b7de46e4d159a5462085...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing f2c52767ad049f190d32b7de46e4d159a5462085 to master...


Keras on top of Pyspark

Hi,
Our product is based on Spark and we also have solutions what was written in Keras.
Is there an working solution that I can run Keras code on top of Apache Spark ( Pyspark ) ?
Thank you very much ,
Boris

Checkout Elephas by @maxpumperla.


Cloud Storage: timeouts not being respected

Environment details

OS type and version: Any
Java version: Any
Storage version: 1.104.0

Steps to reproduce
So, there's this application doing reads and writes in several objects of a Google Cloud Storage bucket. While the vast majority of these operations complete under 500ms, there are some that take from 10 to 40s. The disparity of values seemed to suggest the higher latencies were caused by retries. As such, stricter retry settings were configured.
But while setting a total timeout of 10 seconds, there were still some calls taking 20 seconds.
Looking at the source code, it seems the total timeout is only considered when a retry occurs. The first attempt doesn't seem to be limited in the time it takes. Specifically, here: the retryingFuture.getAttemptSettings() object holds a value for rpcTimeout that doesn't seem to be considered anywhere.
Code example
The Storage object is created like this:
val retrySettings = RetrySettings.newBuilder()
  .setMaxAttempts(3)
  .setInitialRetryDelay(Duration.ofSeconds(1))
  .setMaxRetryDelay(Duration.ofSeconds(2))
  .setRetryDelayMultiplier(2.0)
  .setTotalTimeout(Duration.ofSeconds(10))
  .setInitialRpcTimeout(Duration.ofSeconds(10))
  .setRpcTimeoutMultiplier(1.0)
  .setMaxRpcTimeout(Duration.ofSeconds(10))
  .build()
return StorageOptions.newBuilder().setRetrySettings(retrySettings).build().service

Stack trace
com.google.cloud.storage.StorageException: Read timed out
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:229)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.write(HttpStorageRpc.java:776)
	at com.google.cloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:60)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.storage.BlobWriteChannel.flushBuffer(BlobWriteChannel.java:53)
	at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151)
	...

External references such as API reference guides
n/a
Any additional information below
Apologies if there's any improper configuration on my side, or if this is reported in the wrong project. Thank you 🙏

Hi @pedro-carneiro,
Thank you for your patience on this issue. There might be a missing context to the underlying request to respect the provided timeouts. I'm investigating further, but just chiming in after a while.

Here's additional info that I've been using to understand the timeout config:

Name
Definition
Default Value

Max Attempts
MaxAttempts defines the maximum number of attempts to perform.
6

Initial Retry Delay
InitialRetryDelay controls the delay before the first retry. Subsequent retries will use this value adjusted according to the RetryDelayMultiplie
1 second

Max Retry Delay
MaxRetryDelay puts a limit on the value of the retry delay, so that the RetryDelayMultiplier can't increase the retry delay higher than this amount
32 seconds

Retry Delay Multiplier
RetryDelayMultiplier controls the change in retry delay. The retry delay of the previous call is multiplied by the RetryDelayMultiplier to calculate the retry delay for the next call.
2

Total Timeout
TotalTimeout has ultimate control over how long the logic should keep trying the remote call until it gives up completely. The higher the total timeout, the more retries can be attempted.
50 seconds

Initial RPC Timeout
InitialRpcTimeout controls the timeout for the initial RPC. Subsequent calls will use this value adjusted according to the RpcTimeoutMultiplier.
50 seconds

RPC Timeout Multiplier
RpcTimeoutMultiplier controls the change in RPC timeout. The timeout of the previous call is multiplied by the RpcTimeoutMultiplier to calculate the timeout for the next call.
1

Max RPC Timeout
MaxRpcTimeout puts a limit on the value of the RPC timeout, so that the RpcTimeoutMultiplier can't increase the RPC timeout higher than this amount.
50 seconds

rpcTimeout is considered in the ExponentialRetryAlgorithm.
However, as you pointed out a request isn't interrupted until longer than 10 seconds. Repro provided at the end.
Hi @benwhitehead and @noahdietz, the Java gax library retry strategy is very complex one of you might have more context.
Do you know why configuring totalTimeout does not interrupt a request at the totalTimeout set and instead fails with Read timed out at a longer time?
I attempted reproing this with the Storage Python emulator sleeping for 20000 seconds on data transfer.
The request eventually timeout but takes at least 20+ seconds when totalTimeout is set to 10 seconds.
To repro, I did the following:

Started the emulator in branch on port 9000

Add the following integration test:

@Test
  public void testEmulator() throws IOException {
    String blobName = "test-empty-blob";
    RetrySettings retrySettings = RetrySettings.newBuilder()
            .setMaxAttempts(3)
            .setInitialRetryDelay(Duration.ofSeconds(1))
            .setMaxRetryDelay(Duration.ofSeconds(2))
            .setRetryDelayMultiplier(2.0)
            .setTotalTimeout(Duration.ofSeconds(10))
            .setInitialRpcTimeout(Duration.ofSeconds(1))
            .setRpcTimeoutMultiplier(1.0)
            .setMaxRpcTimeout(Duration.ofSeconds(10))
            .setJittered(false)
            .build();
    Storage storage = StorageOptions.newBuilder()
            .setHost("http://localhost:9000")
            .setRetrySettings(retrySettings)
            .build().getService();
    BlobInfo blobInfo = BlobInfo.newBuilder("bucket", blobName).build();
    int numberOfBytes = 0;
    try (WriteChannel writer = storage.writer(blobInfo)) {
      writer.setChunkSize(256*1024);
      byte[] content = BLOB_STRING_CONTENT.getBytes(UTF_8);
      for (int i = 0; i < 1024*1024; i++) {
        numberOfBytes += writer.write(ByteBuffer.wrap(content, 0, content.length));
      }
    }
  }

Watch it take longer than 10 seconds to fail and can't find why this is happening.

I'm only really familiar with the gRPC implementation of the gax-java mechanics but I am familiar with the retry/timeout logic as a whole. I know for gRPC, that the underlying implementation uses the gRPC RPC deadline mechanism. Essentially gax-java does some math to determine the RPC attempt timeout, then gives that to gRPC to calculate a deadline. So gax-java doesn't actually manage the timing and cancel calls, gRPC does.
I don't know what the equivalent mechanism is for HTTP. @vam-google might be able to help us determine how it works.
I will follow up tomorrow 6/2 with more details/links, I'm traveling today and was just catching up.

Also, taking a step up in the stack, gax-java retry machinery will stop making attempts once the totalTimeout has been consumed/reached. Furthermore, RPC attempt timeouts will be truncated to the remainder in the totalTimeout if the "next" timeout is larger than the time remaining. But again, the in flight request cancellation/management is handled at the transport level and for gRPC it has its own mechanics for canceling the attempt in flight.
I have a feeling it is not implemented for HTTP.

Thanks @noahdietz! That helps point out that interruption of a request may not occur in the gax-java library - pending input from @vam-google and additional links when you're back tomorrow.
@pedro-carneiro I dug into into request timeouts given TotalTimeout does not interrupt requests based on the configured TotalTimeout and instead uses default timeout of 20 seconds which can be configured by doing the following snippet. @BenWhitehead we will need to reconcile the complexity for timeouts when updating retry strategy.
    TransportOptions transportOptions = HttpTransportOptions.newBuilder().setReadTimeout(1000).build();
    Storage storage = StorageOptions.newBuilder()
            .setRetrySettings(retrySettings)
            .setTransportOptions(transportOptions)
            .build().getService();

There 3 settings:

Timeout Name
Description
Default Value

Connection Timeout
timeout in milliseconds to establish a connection or 0 for an infinite timeout
20000 ms

Write Timeout
timeout in milliseconds to send POST/PUT data or 0 for an infinite timeout.
0 - infinite timeout

Read Timeout
timeout in milliseconds to read data from an established connection or 0 for an infinite timeout
20000 ms

Sharing this from @vam-google:
Hi guys,
This is a sophisticated topic, and in general, the term "timeout" is heavily overloaded, to properly answer the question we need very rigorously define which of the many timeouts we are talking about.
Without going into too much detail, it is worth mentioning, that Storage is a very different client from most of the other ones, it is a hand-written and HTTP1-based one, and even though it is integrated with gax-java, the integration is not as deep and fundamental as for the autogenerated gRPC-based ones. The rpcTimeout configs in RetrySettings are appreciated by the GAPIC gRPC clients, but for the manual http1-based clients, only as subset of properties from RetrySettings is actually used, and rpcTimeout is not one of them (i.e. it is basically ignored). There are many reasons why that happened, but basically everything comes to and attempt to make two very incompatible library sets (manual http1-based and generated gapic grpc-based) as similar to each other on the surface as possible. If you dive deep enough, the differences between the client types become more and more visible (like this one).
For GAPCI gRPC-based clients RetrySettigns determine both types of timing parameters - retries (i.e. when to an attempt), and each individual rpc timeouts (i.e. protocol-level thing). For manual clients like Storage, RetrySettigns determine only the retries part.
As @frankyn pointed out, the proper way to configure individual rpc timeout is HttpTransportOptions.setReadTimeout() and HttpTransportOptions.setConnectTimeout(), those are not coming from gax-java and instead of configuring gRPC channel properties (this is what RetrySettings rpcTimeouts are about) would affect directly the timeout properties for HttpRequest from the google-http-client, which is the generic http client used by the manual client libraries.
I hope that helps.

Thanks @noahdietz! I hope this helps address the gotcha for @pedro-carneiro. @BenWhitehead I think you'll want to scope this gotcha into the redesign work for retries.
Closing issue as this question was addressed, and we are working on making this experience better for java-storage. We will include this feedback in redesign work we are doing.


[BI-1704] When downloading phenotype data, now returns the most recent

BI-1704 - If there are repeat measures for a specific observation unit/trait combo, then the phenotype download will return the oldest observation as determined by the collection date of the observation stored in the database. This is inconsistent with a user's expectation that the latest observation be returned in the file.
see Issue 4419
Checklist

[ ] Refactoring only
[ ] Documentation only
[ ] Fixture update only
[X] Bug fix

[ ] The relevant issue has been closed.
[ ] Further work is required.

[ ] New feature

[ ] Relevant tests have been created and run.
[ ] Data was added to the fixture

[ ] Data was added via a patch in /t/data/fixture/patches/.

[ ] User-Facing Change

[ ] The user manual in /docs has been updated.

[ ] Any new Perl has been documented using perldoc.
[ ] Any new JavaScript has been documented using JSDoc.
[ ] Any new legacy JavaScript has been moved from /js to /js/source/legacy.

Has this change been discussed with the Breedbase team? If so, maybe we can create another PR to merge upstream.

Yes, https://github.com/solgenomics/sgn/issues/4419 was opened and discussed at their lab meeting a couple of weeks ago.  I think we'll need to apply this fix in our develop branch to then be able to push upstream


Brewing stand shows nothing in recipes

if you search for something and then delete it you won't see any recipe
also if you click show all and then click again show brewings it also won't show any recipe

can you provide your Minecraft log and I'll get on this

latest.log

This is going to be real annoying to fix


Add Import.io connector as an extension

Import.io has shortened the path to quickly getting web data extracted and provides an API key.
Someone can work on an Import.io connector/extraction extension for OpenRefine.
Some Features to include:

Import.io User ID
Import.io API key
Import.io GUID
Import.io timeout in seconds
Retries (int)
Min time between queries (ms)
Start Page
Max Page
Webpage/URL to extract

A dialog (k,v) to also allow Custom mapping the Import.io field names to the existing or new OpenRefine Column names.

My interest has waned from this data source, as it's considered too opinionated now for general use or research by peers.
Closing.


DeprecationWarning: Buffer() is deprecated due to security and usability issues

I got the following error message on console of Developer Tools in VSCode.
[Extension Host] (node:5344) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
I checked this issue on microsoft/vscode,  and I found out that this error is caused by an extension.
So I disabled and enabled each extension to find out which extension was causing the error, and finally found out that the Intellicode was the cause of the error.
Product and Version
Version: 1.52.1 (user setup)
Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523
Date: 2020-12-16T16:34:46.910Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19042

Steps to Reproduce / Scenario:

Disable all installed extensions and enable Visual Studio Intellicode.
Reload VScode.
Open developer tools, see error on console.

@leti367  please can you take a look?

@leti367  please can you take a look?

Sure. I will look into it today.

From: Mark Wilson-Thomas<EMAIL_ADDRESS>Sent: Monday, January 25, 2021 11:43:15 AM
To: MicrosoftDocs/intellicode<EMAIL_ADDRESS>Cc: Lei Tian<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/intellicode] DeprecationWarning: Buffer() is deprecated due to security and usability issues (#268)
@leti367https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fleti367&data=04|01|leti%40microsoft.com|10e9bfdf0a0b46d104a608d8c169758b|72f988bf86f141af91ab2d7cd011db47|1|0|637472005978842527|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=SJBLuQ3fnE5NH26m3tvPykOP0cXkPJdykowDmZyeoio%3D&reserved=0 please can you take a look?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fintellicode%2Fissues%2F268%23issuecomment-767067186&data=04|01|leti%40microsoft.com|10e9bfdf0a0b46d104a608d8c169758b|72f988bf86f141af91ab2d7cd011db47|1|0|637472005978852524|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=cYISQ3OafrIu7zLI8aFKrMVYn%2FIjAVMDGaD9bQHGLVw%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGNKBFDO3NKXBVYSJLGHOLLS3XCVHANCNFSM4WCTGPNQ&data=04|01|leti%40microsoft.com|10e9bfdf0a0b46d104a608d8c169758b|72f988bf86f141af91ab2d7cd011db47|1|0|637472005978852524|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=mzCe9fF0S76sKWDiyWCxtanoUt2%2FNKdd6T9VD%2BV5UAA%3D&reserved=0.

Sure. I will look into it today.

From: Mark Wilson-Thomas<EMAIL_ADDRESS>Sent: Monday, January 25, 2021 11:43:15 AM
To: MicrosoftDocs/intellicode<EMAIL_ADDRESS>Cc: Lei Tian<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/intellicode] DeprecationWarning: Buffer() is deprecated due to security and usability issues (#268)
@leti367https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fleti367&data=04|01|leti%40microsoft.com|10e9bfdf0a0b46d104a608d8c169758b|72f988bf86f141af91ab2d7cd011db47|1|0|637472005978842527|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=SJBLuQ3fnE5NH26m3tvPykOP0cXkPJdykowDmZyeoio%3D&reserved=0 please can you take a look?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fintellicode%2Fissues%2F268%23issuecomment-767067186&data=04|01|leti%40microsoft.com|10e9bfdf0a0b46d104a608d8c169758b|72f988bf86f141af91ab2d7cd011db47|1|0|637472005978852524|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=cYISQ3OafrIu7zLI8aFKrMVYn%2FIjAVMDGaD9bQHGLVw%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGNKBFDO3NKXBVYSJLGHOLLS3XCVHANCNFSM4WCTGPNQ&data=04|01|leti%40microsoft.com|10e9bfdf0a0b46d104a608d8c169758b|72f988bf86f141af91ab2d7cd011db47|1|0|637472005978852524|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=mzCe9fF0S76sKWDiyWCxtanoUt2%2FNKdd6T9VD%2BV5UAA%3D&reserved=0.

@leti367 do you remember the outcome here?

Did a quick look at it, it comes from two packages: "@vscode/extension-telemetry" and "request", we are using the latest version for both packages. The only thing we can do is to file new github issues for these two packages, but since this is a warning, I don't think they will fix it very soon.


Getting two transaction against a insertion

Hi Team ,
I am implementing contract events for fabcar using node sdk api.so in fabcar go chaincode createCar function ,i have set event in following way.
=============
    ` `   ctx.GetStub().SetEvent("CreateCar", carAsBytes)``
=============
and for contract events at node js, i m using listener in following way.
===============
try {
       listener = async (event) => {
        const car = JSON.parse(event.payload.toString());
        const eventTransaction = event.getTransactionEvent();
        const eventBlock = eventTransaction.getBlockEvent();
                };
        await contract.addContractListener(listener);
        } catch (eventError) {
                        console.log('<-- Failed: Setup contract events - ${eventError}');
        }

    await contract.submitTransaction('createCar' ,'CAR1112', 'Honda', 'Accord', 'Black', 'Tom' );

===============
But when i invoke createcar function,getting two blocks and two transaction id from contract listener object.
Response is like below.

============================
--> Start contract event stream to peer in Org1
event.eventName========= CreateCar
car =================== {"make":"Honda","model":"Accord","colour":"Black","owner":"Tom"}
transaction========= 9d9111ff665d4fd870bb6c6042a8da3ca6910660bab9915290063504337b4bff
transaction status========= VALID
EventBlock.blocknuber========= 27
event.eventName========= CreateCar
car =================== {"make":"Honda","model":"Accord","colour":"Black","owner":"Tom"}
transaction========= a32eb2a5366862f17708802aa7bb86e2b19208c7d9f6d913c5bc5ac0e0632d01
transaction status========= VALID
EventBlock.blocknuber========= 28

============================
Why two blocks are generated for single transaction.let me correct if anything wrong.

Regards,

I had another look at this today and I think the mystery is solved. I started with a newly created blockchain and performed two runs of my simple sample application (above).
Output of the first application run:
Received event: CreateAsset
Payload: {
  ID: 'item-989',
  Color: 'blue',
  Size: '10',
  Owner: 'Sam',
  AppraisedValue: '100'
}
Transaction ID: 6d861709e19fc287b3888e1090416bb999502662b5a9ac1604aa86f172e45b14
Block number: 6

Output of the second application run:
Received event: CreateAsset
Payload: {
  ID: 'item-989',
  Color: 'blue',
  Size: '10',
  Owner: 'Sam',
  AppraisedValue: '100'
}
Transaction ID: 6d861709e19fc287b3888e1090416bb999502662b5a9ac1604aa86f172e45b14
Block number: 6
Received event: CreateAsset
Payload: {
  ID: 'item-12',
  Color: 'blue',
  Size: '10',
  Owner: 'Sam',
  AppraisedValue: '100'
}
Transaction ID: d55c593e2087d0369f1af64a4d722070b6abaa38140e7d5e1a1435715a940d9f
Block number: 7

Notice that the first run outputs a single chaincode event (from block 6) while the second outputs two chaincode events (from blocks 6 and 7). The reason is that, when you don't specify an eventing start position, eventing actually begins from the last committed block. I would personally prefer it to start from the next committed block (as it does in the new @hyperledger/fabric-gateway API for Fabric v2.4+), but there were reasons for this behaviour based on the eventing implementation in fabric-common.
If you want to avoid duplicating or missing events on subsequent application runs, you can use a checkpointer to persist your current eventing position between application runs.
If I change my client application to attach the event listener like this:
await contract.addContractListener(listener, {
    checkpointer: await DefaultCheckpointers.file('checkpoint.json'),
});

I get the following output for the first application run:
Received event: CreateAsset
Payload: {
  ID: 'item-254',
  Color: 'blue',
  Size: '10',
  Owner: 'Sam',
  AppraisedValue: '100'
}
Transaction ID: 7b50171806b1313dfca87b45ac628a83242c295175da0ff2f4b02e3f14e3c9dd
Block number: 6

And only this output for the second application run:
Received event: CreateAsset
Payload: {
  ID: 'item-919',
  Color: 'blue',
  Size: '10',
  Owner: 'Sam',
  AppraisedValue: '100'
}
Transaction ID: 95f40f62dfb698701d48377c66abe6bf30cb2ea7b18a879664215cefc88e1064
Block number: 7

Closing under the assumption that this has answered your query. If there is any remaining issue, add a comment and we can reopen it.


nginx controller fails startup and 100% cpu usage with 100000 hosts

Hi, I am testing a configuration with 100000 hosts split into 10 ingresses. Each host looks like this:
- host: "test99999.testcluster.test.com"
    http:
      paths:
      -
        path: /
        backend:
          serviceName: web
          servicePort: 80
      -
        path: /testecho
        backend:
          serviceName: echoheaders-x
          servicePort: 80

nginx controller 8.6.3 appears to work fine with this configuration but the latest 0.9.0-beta.13 fails to start. It gets stuck on startup with one controller process using 100% cpu. The monitoring port 10254 is inaccessible (connection times out). nginx.conf is never generated.

@Gregy please post the ingress logs

Sure.
Nginx config was still not updated and the nginx was not listening on 80 at the time the logs end.
default-verbosity.txt
verbose-logs.zip

@Gregy do you have a script that creates this ingress rules?
What are you trying to achieve with this test?

Script is here:
#!/bin/bash
cat <<EOF

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: testingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  backend:
    serviceName: web
    servicePort: 80
  rules:
EOF

for i in `seq $1 $2`; do
cat <<EOF
  - host: "test$i.testcluster.test.com"
    http:
      paths:
      -
        path: /
        backend:
          serviceName: web
          servicePort: 80
      -
        path: /testecho
        backend:
          serviceName: echoheaders-x
          servicePort: 80

EOF
done

I ran it 10 times to get 100000 hosts split across 10 ingresses.
I am trying to see if the ingress controller can scale to this number of hosts. I have a site with tens of thousands of subdomains that I need to manage (distribute the subdomain's traffic to multiple pods) and want to know if nginx-lb can do it. It works with the 0.8.3 but the beta gets stuck somewhere.

@Gregy please update the image to  quay.io/aledbf/nginx-ingress-controller:0.225 and add the flag --update-status=false
This version contains several improvements in the utilization of cpu and ram resources.
I tested this changes with 12000 ingress rules.
That being said, I'm not sure using an ingress controller beyond 10000 rules makes sense. One of the reasons for this is that we use several informers in the controller (it's a mirror of the information contained in the api server, like ingress, secrets, services, endpoints, nodes) that consume resources (ram) and after any change in those resources we need to compare the running configuration with the new state of the cluster (his is a cpu intensive task) and reload nginx.

@Gregy another detail: please don't use just one ingress rule, use something like
COUNT=1
while [ $COUNT -le 12000 ]
do
echo "
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-$COUNT
spec:
  rules:
  - host: test$COUNT.testcluster.test.com
    http:
      paths:
      -
        path: /
        backend:
          serviceName: http-svc
          servicePort: 80
      -
        path: /testecho
        backend:
          serviceName: http-svc
          servicePort: 80
" | kubectl create -f -
COUNT=$(( $COUNT + 1 ))
done

Thank you very much.
With your image nginx config is generated and the loadbalancer is working as expected even with 100000 hosts. The cpu usage problem remains though. I am not talking about usage when regenerating the config, that is expected. One of the controller threads is spinning all the time even when no changes are being made. I took a cpu profile:
(pprof) top
29.82s of 30.05s total (99.23%)
Dropped 31 nodes (cum <= 0.15s)
Showing top 10 nodes out of 12 (cum >= 29.91s)
      flat  flat%   sum%        cum   cum%
    22.38s 74.48% 74.48%     26.89s 89.48%  k8s.io/ingress/core/pkg/ingress.(*Server).Equal
     3.02s 10.05% 84.53%     29.91s 99.53%  k8s.io/ingress/core/pkg/ingress.(*Configuration).Equal
     2.97s  9.88% 94.41%      2.97s  9.88%  runtime.memeqbody
     1.45s  4.83% 99.23%      1.45s  4.83%  runtime.eqstring
         0     0% 99.23%     29.91s 99.53%  k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).(k8s.io/ingress/core/pkg/ingress/controller.syncIngress)-fm
         0     0% 99.23%     29.91s 99.53%  k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).syncIngress
         0     0% 99.23%     29.91s 99.53%  k8s.io/ingress/core/pkg/task.(*Queue).(k8s.io/ingress/core/pkg/task.worker)-fm
         0     0% 99.23%     29.91s 99.53%  k8s.io/ingress/core/pkg/task.(*Queue).Run
         0     0% 99.23%     29.91s 99.53%  k8s.io/ingress/core/pkg/task.(*Queue).worker
         0     0% 99.23%     29.91s 99.53%  k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil

My guess would be it is this gorutine?
goroutine 103 [runnable]:
k8s.io/ingress/core/pkg/ingress.(*Server).Equal(0xc4282b2000, 0xc47c80f790, 0x18600)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/types_equals.go:276 +0x54a
k8s.io/ingress/core/pkg/ingress.(*Configuration).Equal(0xc420104880, 0xc42020b700, 0x0)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/types_equals.go:76 +0x205
k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).syncIngress(0xc4202e7590, 0x133e3e0, 0xc4203422d0, 0xc4203422d0, 0xc420342200)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:284 +0xa35
k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).(k8s.io/ingress/core/pkg/ingress/controller.syncIngress)-fm(0x133e3e0, 0xc4203422d0, 0xa, 0xc4247dfdf8)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:169 +0x3e
k8s.io/ingress/core/pkg/task.(*Queue).worker(0xc4202de570)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/task/queue.go:89 +0x11e
k8s.io/ingress/core/pkg/task.(*Queue).(k8s.io/ingress/core/pkg/task.worker)-fm()
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/task/queue.go:49 +0x2a
k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc4205f7fa8)
	/home/aledbf/go/src/k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x5e
k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc47ea9bfa8, 0x3b9aca00, 0x0, 0x1210e01, 0xc42022a480)
	/home/aledbf/go/src/k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134 +0xbd
k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc4205f7fa8, 0x3b9aca00, 0xc42022a480)
	/home/aledbf/go/src/k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x4d
k8s.io/ingress/core/pkg/task.(*Queue).Run(0xc4202de570, 0x3b9aca00, 0xc42022a480)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/task/queue.go:49 +0x55
created by k8s.io/ingress/core/pkg/ingress/controller.GenericController.Start
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:1216 +0x4cb

Also one error shows up in the logs:
E0919 08:32:17.843102       6 queue.go:62] could not get key for object force-sync: object has no meta: object does not implement the Object interfaces

Regarding nginx-lb's suitability for the task:
I think any system capable of handling this kind of use case (lots and lots of subdomains, each with its own config) would need to regenerate and reload the config on each change. I think the resources taken would be comparable.

I thought the problem could be caused by too low sync-period so I tried to increase it by setting --sync-period=10h. The behavior remained the same.

@Gregy please use quay.io/aledbf/nginx-ingress-controller:0.226

Closing. The refactoring is already present in master

@Gregy please reopen if you still have issues after using the image quay.io/aledbf/nginx-ingress-controller:0.226

Thanks. I investigated the cpu usage again and this is the timeline:
T+0 - init
T+5 - log message "backend reload required"
T+95 - ingress backend successfully reloaded and nginx is correctly serving requests but ingress controller go code is still spinning its wheels in the gorutine I posted earlier
T+2130 - go code stops spinning
I guess I am missing something. What is the code doing for that half an hour if the nginx config is finished and working in the first 2 minutes?
I tried with both 0.225 and 0.226. The only difference seems to be that the error is gone from the logs in 0.226. Unfortunatelly adding new ingress doesn't work in 0.226. I get the event message in the logs but no  "backend reload required" message and nginx config doesn't reload.
Unfortunatelly I cannot reopen the issue :(

@Gregy some context about how we process the ingress rules: when the ingress controller start it receives   the list of ingress from api server and an event handler is called (per ingress rules). Each call adds the ingress in a worker queue and in a different goroutine we check if the running configuration is equals to the new one or not. In case the configuration are not equals we trigger a nginx reload.
In 0.226 I changed the initial sync to avoid queuing all the rules avoiding the cpu usage you saw earlier.
Even when this change the ram usage in the ingress controller is too high and the cpu usage you see now is just the golang gc, not the equals comparison.
Please keep in mind that you need to disable the update of the status field using --update-status=false

Thanks!
I understand the ram usage is high. It is to be expected. My nginx-lb uses 20G for 100000 hosts and that is fine with me.
The log message "backend reload required" is printed after it compares the config and decides it needs to reload, right? If that is the case it is very quick and is not the cause of my problem.
I cannot see how the high cpu usage for 30 minutes from start is garbage collector's fault. It would collect something right? The ram usage during those 30 minutes is constant. Also it only seems to happen after start. When I edit the config nginx reloads but the high cpu usage doesn't come back. Can I somehow disable gc to be sure?
Can you replicate my problem where the ingress doesn't trigger reload after adding new ingress on 0.226? It works for me on 0.225 but not on 0.226.

I0919 19:20:05.845399       7 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"testingress1", UID:"8a60d824-9c77-11e7-808b-42010a800091", APIVersion:"extensions", ResourceVersion:"588646", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/testingress1
I0919 19:20:10.213901       7 controller.go:291] backend reload required
I0919 19:21:45.384423       7 controller.go:300] ingress backend successfully reloaded...
I0919 19:28:42.574588       7 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"testingress201000", UID:"bf149b1c-9d70-11e7-ac86-42010a800091", APIVersion:"extensions", ResourceVersion:"589224", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/testingress201000
<nothing happens here>

As you can see from the log nginx reloads fine on update event but not on create event.

@Gregy please use quay.io/aledbf/nginx-ingress-controller:0.227

@Gregy are you connected to the kubernetes slack?

Tested 225 and 226. I was eager to test as we also experienced high CPU usage, although on a smaller scale. Also we saw and still do see reload taking ~12 seconds.
Our setup:

2 ingress controllers (limits are 1.5 cores and 10GB)
--sort-backends=true  and --update-status=false
~3400 ingresses
frequent changes in ingresses

The changes you introduced here are indeed making the CPU usage lighter. Although reloads are still taking a lot of time. I also noticed that it took around 4 minutes for 225 to actually be able to serve requests, while with 226 it was 12 minutes.
Stats from 225

Stats from 226

Stats from 0.9.0-beta.13

memory usage above could be ignored, we have a faulty socket.io app ;)

@cu12 please update to 0.227

I tested with 227, it took ~18 minutes to serve the first requests, so I missed some of them as readinessProbe.initialDelaySeconds was set to 900.
Reload still takes ~12 seconds:
I0919 22:34:37.135251       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:35:01.751290       7 controller.go:292] backend reload required
I0919 22:35:13.914218       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:35:21.918488       7 controller.go:292] backend reload required
I0919 22:35:34.714913       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:35:41.564061       7 controller.go:292] backend reload required
I0919 22:35:55.310296       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:35:58.941131       7 controller.go:292] backend reload required
I0919 22:36:12.816486       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:36:17.984489       7 controller.go:292] backend reload required
I0919 22:36:30.978228       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:37:03.339946       7 controller.go:292] backend reload required
I0919 22:37:18.724050       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:37:22.534802       7 controller.go:292] backend reload required
I0919 22:37:36.557198       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:37:40.178868       7 controller.go:292] backend reload required
I0919 22:37:54.467715       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:37:58.044858       7 controller.go:292] backend reload required
I0919 22:38:11.107285       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:38:15.235995       7 controller.go:292] backend reload required
I0919 22:38:28.638520       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:38:32.136863       7 controller.go:292] backend reload required
I0919 22:38:45.727476       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:38:49.537017       7 controller.go:292] backend reload required
I0919 22:39:02.925242       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:39:06.572189       7 controller.go:292] backend reload required
I0919 22:39:20.810804       7 controller.go:301] ingress backend successfully reloaded...
I0919 22:39:24.469368       7 controller.go:292] backend reload required
I0919 22:39:37.479885       7 controller.go:301] ingress backend successfully reloaded...

@Gregy  @cu12 please use quay.io/aledbf/nginx-ingress-controller:0.228

I was running 229 in the past 24 hours, here how does it looks like now. I haven't experienced any significant improvements from 227, reloads are still taking ~12 seconds.

@aledbf also about v0.229 was theresyncPeriod in listers.go changed inside the image from default 600s to something else? Because when running this particular version it took about 2h for a new certificate to get added to local store after secret creation.

haven't experienced any significant improvements from 227, reloads are still taking ~12 seconds.

There's nothing with can do about the reload time. This is the time it take nginx to reload with your running workload.

With 0.229 I get no significant cpu usage from pod start until the first ingress change. After I change ingress configuration and nginx reloads I get high cpu usage again.
Profile top: ```
2390ms  6.94%  6.94%     7510ms 21.80%  runtime.mallocgc
2260ms  6.56% 13.50%     4850ms 14.08%  runtime.scanobject
1440ms  4.18% 17.68%     5980ms 17.36%  runtime.mapassign_fast64
1230ms  3.57% 21.25%     1270ms  3.69%  runtime.heapBitsForObject
1220ms  3.54% 24.79%     1220ms  3.54%  runtime.memclrNoHeapPointers
1090ms  3.16% 27.95%    11360ms 32.98%  k8s.io/ingress/vendor/github.com/imdario/mergo.deepMerge
1080ms  3.13% 31.09%    27180ms 78.90%  k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).getBackendServers
1060ms  3.08% 34.17%     1060ms  3.08%  runtime.nextFreeFast
980ms  2.84% 37.01%     1110ms  3.22%  runtime.heapBitsSetType
870ms  2.53% 39.54%      870ms  2.53%  runtime.duffcopy
My guess would be k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).getBackendServers ?

Probably this gorutine? ```
goroutine 100578 [runnable]:
k8s.io/ingress/core/pkg/ingress/annotations/parser.GetIntAnnotation(0x155fd2f, 0x28, 0xc42fa72870, 0x1e401e0, 0x0, 0x0)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/annotations/parser/main.go:96 +0x105
k8s.io/ingress/core/pkg/ingress/annotations/healthcheck.healthCheck.Parse(0x7f558b195988, 0xc42044a2d0, 0xc42fa72870, 0xc4202b0a38, 0xc420227b60, 0x1e117f0, 0xc4202b0a10)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/annotations/healthcheck/main.go:55 +0xa8
k8s.io/ingress/core/pkg/ingress/annotations/healthcheck.(*healthCheck).Parse(0xc42022a590, 0xc42fa72870, 0xc4202b41c0, 0xc4235e3201, 0x4, 0x4)
	<autogenerated>:1 +0x4f
k8s.io/ingress/core/pkg/ingress/controller.(*annotationExtractor).Extract(0xc42044a250, 0xc42fa72870, 0x5)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/annotations.go:93 +0x112
k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).getBackendServers(0xc42044a1e0, 0xc42bc4dc20, 0x18, 0x0, 0x0, 0x0, 0x0)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:486 +0x30d
k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).syncIngress(0xc42044a1e0, 0x1340ba0, 0xc452f32f90, 0xc452f32f90, 0xc452f32e00)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:256 +0xb5
k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).(k8s.io/ingress/core/pkg/ingress/controller.syncIngress)-fm(0x1340ba0, 0xc452f32f90, 0xa, 0xc4235e3df8)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:172 +0x3e
k8s.io/ingress/core/pkg/task.(*Queue).worker(0xc4201e8360)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/task/queue.go:89 +0x11e
k8s.io/ingress/core/pkg/task.(*Queue).(k8s.io/ingress/core/pkg/task.worker)-fm()
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/task/queue.go:49 +0x2a
k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc42058ffa8)
	/home/aledbf/go/src/k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x5e
k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc4235e3fa8, 0x3b9aca00, 0x0, 0x1210e01, 0xc4204c0360)
	/home/aledbf/go/src/k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134 +0xbd
k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc42058ffa8, 0x3b9aca00, 0xc4204c0360)
	/home/aledbf/go/src/k8s.io/ingress/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x4d
k8s.io/ingress/core/pkg/task.(*Queue).Run(0xc4201e8360, 0x3b9aca00, 0xc4204c0360)
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/task/queue.go:49 +0x55
created by k8s.io/ingress/core/pkg/ingress/controller.(*GenericController).Start
	/home/aledbf/go/src/k8s.io/ingress/core/pkg/ingress/controller/controller.go:1230 +0x6b6

@aledbf you mentioned earlier that each change event is added to a queue and then processed? What happens when multiple ingresses are changed at the same time? Configuration is reloaded for all changed ingresses at once and the queue is cleared? Or is the nginx supposed to be reloaded for each ingress change one by one?

getBackendServers also bites us.

@cu12 please use quay.io/aledbf/nginx-ingress-controller:0.232


Tabs Visibility

I have a component that is using the tabs functionality.  The component is not visible by default, allowing the user to click a button to show it.  The tab is not respecting its parent component visibility however and is still visible.
It seems like this is due to the active tab classes of 'active fade in'.  Is there any way to have all the tabs inactive/non-visible? I see the tabs are 0 index based which doesn't appear to allow for this functionality. Currently I am trying to use a hack that finds the active tab and removes the classes on load of component but this is not ideal and requires me to repeat the hack anytime the user turns the component visibility to false.

try to use a tab index unexistent, like -1 or a number bigger that the tab length (if you have 5 tabs, use 5 o 6, the index is zero based).

I tried -1 and null as a test and was unsuccessful.  I'll try a number bigger than the tab length as well and report back if it works.


Natives

Is your feature request related to a problem? Please describe.
We shouldn't rely on third-parties like Fluffy Manager, support can be dropped any time and people don't like to have 2 to 3 different tools to MOD one game.
As for being related to a problem, we been unable to MOD MHR ever since Update 16, due to lack of updates on Fluffy Manager.
This wouldn't be a problem if we were relying solely on REFramework.
Describe the solution you'd like
Native support to load loose files, it will simplify modding.
Describe alternatives you've considered
For MHR there's already a library that gives us this feature, however is outdated and buggy.
Additional context
This should be one of the basic REFramework features, since is the go to tool to mod CAPCOM games.

While this would certainly be a nice feature, are you sure these mods you're referring to are broken? The mods that they're loading can cause infinite loading and other issues due to CRC mismatches in the new game data and whatnot.
I'll consider it though, games other than MHRise are missing something like FirstNatives to facilitate fast mod development.

Added a month or so ago with the "LooseFileLoader" option.


bug: Cannot get final name for export 'SEMATTRS_HTTP_STATUS_CODE'

Provide environment information
System:
OS: macOS 14.4
CPU: (12) arm64 Apple M2 Max
Memory: 1.51 GB / 32.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 21.4.0 - ~/.nvm/versions/node/v21.4.0/bin/node
Yarn: 1.22.21 - /opt/homebrew/bin/yarn
npm: 10.2.4 - ~/.nvm/versions/node/v21.4.0/bin/npm
pnpm: 9.14.2 - ~/Library/pnpm/pnpm
bun: 1.0.6 - ~/.bun/bin/bun
Describe the bug
After upgrading to<EMAIL_ADDRESS>I started getting the following error when building a NextJS 14 project:
./node_modules/.pnpm/@trigger.dev+sdk@3.3.6_zod<EMAIL_ADDRESS>+ 6 modules
Cannot get final name for export 'SEMATTRS_HTTP_STATUS_CODE' of ./node_modules/.pnpm/@opentelemetry+semantic-conventions<EMAIL_ADDRESS>./node_modules/.pnpm/@trigger.dev+sdk@3.3.6_zod<EMAIL_ADDRESS>+ 6 modules
Cannot get final name for export 'SEMATTRS_HTTP_STATUS_CODE' of ./node_modules/.pnpm/@opentelemetry+semantic-conventions<EMAIL_ADDRESS>
Downgrading back to<EMAIL_ADDRESS>allows the build to complete successfully.
Reproduction repo
n/a
To reproduce

Install<EMAIL_ADDRESS>Create a task triggered in a Server Action
Run pnpm build on<EMAIL_ADDRESS>project

Additional information
I tried following suggestions in similar issues related to NextJS:

https://github.com/vercel/next.js/issues/60807
https://github.com/aws/aws-sdk-js-v3/issues/5488

Adding the following to next.config.js:
serverComponentsExternalPackages: [
<EMAIL_ADDRESS><EMAIL_ADDRESS>      '@opentelemetry',
      '@opentelemetry/semantic-conventions'
 ]

Unfortunately to no avail.

@rafalzawadzki Is this happening in a monorepo, or just a regular repo? Do you possibly have code that reproduces this? I'm unable to myself so far.

Same here - any resolutions?

@ericallam it's a standard Next repo (not monorepo, no turbo etc)
unfortunately can't provide the code
@techjason downgrading trigger was the workaround for now

@rafalzawadzki I'm guessing in the "To reproduce" steps above you meant to say "Install @trigger.dev/sdk@latest"?
@techjason any chance you have a repo that can reproduce this? Could you provide any more details on your stack?

+1 same issue here on the following version on a nextjs application
<EMAIL_ADDRESS>"3.3.7",
<EMAIL_ADDRESS>"3.3.7",
    "trigger.dev": "^3.0.12",

@ming1in any chance you could share a minimal repo that reproduces this?

Okay I've managed to reproduce this bug and here's what I found:
Between 14.0.0 and 14.2.7, there is a Next.js build issue that can occur in certain circumstances:

Call a server action from a client component
The server action imports a named function (for example, mockTriggerHelloWorldTask) from another file (let's call it triggerWrapper.ts)
triggerWrapper.ts imports a Trigger.dev task defined in another file.
The mockTriggerHelloWorldTask function does NOT make use of the Trigger.dev that was imported.

This will (for some reason) cause builds to fail with the above error (or one very similar). There are two fixes I've found so far for this:

Upgrade to Next.js 14.2.8 or later
OR, add this to your next config:

/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    // Add this line to fix the "Cannot get final name for export" error
    // Alternatively, you can upgrade to Next.js 14.2.8 or later
    serverComponentsExternalPackages<EMAIL_ADDRESS>  },
};

export default nextConfig;

@rafalzawadzki @techjason @rafalzawadzki


feat(kit): use 1 as min segment value in Date-related masks

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows Conventional Commits
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Refactoring
[ ] Code style update
[ ] Build or CI related changes
[ ] Documentation content changes

What is the current behavior?
Closes #148
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[x] No

cypress tests fail

removing default values from getSegmentValueLength revealed a mistake here :peka:


Add explicit_elapse option to AnimationNodeTimeSeek to handle some processes

Closes https://github.com/godotengine/godot/issues/97836

Add explicit_elapse option to handle is_external_seeking flag with AnimationNodeTimeSeek.

Thanks!


.xp error

Everyday this happens. Sometimes in few hours after restart, sometime in few days after restart. .imagesreload doesnt help

Per message in discord, I get the same error on various commands, which require a bot restart to correct.
http://prntscr.com/kmupf3
Windows Server 2016, Virtual Environment, using the prebuilt installer/updater, usually at least 25-15% RAM available at all times on it

Never saw this happen on local or public bot. This only happens when the image you've specified is not an actual image, or website where image is hosted unavailable i guess. Also could happen if you flush your redis cache. while bot is running. (Maybe you're running out of ram?)

Nope. With ram all ok. imgur works good. (https://i.imgur.com/idn8QTq.png). I have never flushed redis cache while bot running


feat: Implement standalone mocha runner and loader

This PR adapts the https://github.com/microsoft/vscode-extension-test-runner/ fork codebase to a standalone mocha test runner extension.

[x] Rebrand
[x] Implement standalone testloading with JS (commonjs, ESM) and TypeScript support
[x] Ensure proper sourcemap support
[x] Ensure running and debugging of tests are working
[x] Update and extend tests
[x] Setup GitHub Actions for building and testing the extension
[x] Implement proper logging to output window.
[ ] Reviewing all copyright and license details
[ ] Contribute code to Mocha Project / Rebrand to personal project
[ ] Publish extension.

This is great! Really need this 👍 looking forward to the release

@marcus-pousette Highly depends on the feedback at https://github.com/mochajs/mocha/issues/5039 if the Mocha project will adopt this extension or if I publish it under my name 😁

@Danielku15 makes sense!
I started trying out this branch. I am not sure if you are getting this but with this simple test repo. The line numbers are wrong. I am using Node v20.12.0
https://github.com/marcus-pousette/mocha-vscode-test

@marcus-pousette Yeah, that's something I'm still tackling. Internally I use ESBuild to transpile TypeScript to JavaScript (including Source Maps) and then I need to translate back the positions correctly.
For plain JS the positoning is already fine, for TS (and JavaScript with source maps) the bits are still missing.

Testing monorepos. I have two issues, see image

Sidebar have duplicates of the same tests. For some reason node_modules show up here.
Codelens locations are wrong (same for my previous comment/test)

Node v20.12.0
Here is the repo
https://github.com/marcus-pousette/mocha-vscode-monorepo-test

@marcus-pousette Yeah, that's something I'm still tackling. Internally I use ESBuild to transpile TypeScript to JavaScript (including Source Maps) and then I need to translate back the positions correctly.
For plain JS the positoning is already fine, for TS (and JavaScript with source maps) the bits are still missing.

Ah I see! I kind of spent some time testing different Node versions and noticed how later ones handle whitespace differently. Not sure if that is helpful here but

Not sure if helpful but the jest-vscode extension uses https://github.com/jridgewell/trace-mapping behind the scenes through the istanbul-lib-source-maps package

@marcus-pousette Thanks for the helpful test repos. I quickly fixed the TypeScript source map stuff, line numbering seem fine in my the projects I use for testing.
Regarding the mono-repo: Can you update the repository to a state where mocha itself (e.g. running npx mocha) is functional and delivers the correct test results when executed in respective directories?
Currently the repo seems not yet in a functional state to work with mocha standalone. I currently strive for consistency between the CLI and the VS Code extension. If you can fix that I can check easier where the inconsistency in the globbing occurs.

Great. I will try the code out again if you have updated.
I updated
https://github.com/marcus-pousette/mocha-vscode-monorepo-test.git
now so that
npx mocha works in root and all packages

The line number issues still seem to persist sadly in both my test repos (after I pulled your changes from 9c25be698920e1d4420a0cede24eb1edada477d6 )

@marcus-pousette Be sure to rebuild and reload your vs code. I tested with your repos, everything is fine now. 😉

My bad, I forgot to rebuild the extension 🙈 Yes, the line locations seem to work now. This is awesome!
When you have the monorepo open, do you see a "node_modules" folder in the sidebar? I kind of see duplicated tests here and the icons for running the tests only turn green if I run the node_modules one.
Will try out this extension now on a big repo I am working on

Yes I see the node_modules and it seem to be also picked up like that in Mocha itself. Here I ran npx mocha --reporter=json with additional tests added to node_modules (+ TS_NODE_SKIP_IGNORE=true to make ts-node happy) and we can see how Mocha behaves:

Mocha filters out duplicates based on the suite and test names but the Plugin currently structures the tests in folders and files. This is where the difference becomes visible.
They have default ignore patterns when using watch but not when normally executing:

https://mochajs.org/#-watch-ignore-filedirectoryglob
https://mochajs.org/#-ignore-filedirectoryglob-exclude-filedirectoryglob

I just added support for the ignore option. If you change your mocharc like this, the tests are as you'd expect:
{
    "require": "ts-node/register",
    "loader": "ts-node/esm",
    "extensions": ["ts", "tsx"],
    "spec": [
      "**/test/**/*.spec.ts"
    ],
+    "ignore": [
+      "**/node_modules/**"
+    ],
    "watch-files": [
      "src"
    ]
}

Not perfect from an end user perspective, I agree. But in-line with the Mocha behavior. Might be worth discussing over at https://github.com/mochajs/mocha that it should ship a default ignore for the node_modules.

Ah, thanks for the explanation, that makes sense! Yes this update solves that problem. This is great!
I now tried it on a larger repository and these are the issues I am observing right now

If the names of the tests are the same in two test files then it seems like the test runner is running both of the tests even if I only want to run one (by either click in the sidebar or the codelens run button). I update the test mono-repo to showcase this problem. Try for running one of the tests and watch the output log

I need a root tsconfig.json. I am not sure whether this is expected or not but normally I think you can have a monorepo without this and when you build the code the tsconfig in each sub folder will be used when building and testing. Without a root tsconfig.json you can't run npx mocha in the root, but I am not sure if this is needed/wanted?

It seems like everytime I run a test it needs to build the entire project and go through over all tests. This might be related to (1) or (2). But it takes something like 10 seconds for me to run a simple test in a larger repo I am working on.

If the names of the tests are the same in two test files then it seems like the test runner is running both of the tests...

Will look into that. Mocha does not have directly some "unique ids" we could use to reference tests so it rather works with passing in suite and test names to the npx mocha command. But it should be possible to generate some better arguments in such cases.

I need a root tsconfig.json even if I can run npx mocha successfully in subfolders...
It seems like everytime I run a test it needs to build the entire project and do all type-checking etc. ...

I think this is not related to this extension but Mocha in general. If you have concerns regarding this you might reach out to the Mocha devs to discuss potential improvements.
It's just a gut feeling but it could be caused by 'ts-node' you are using. I personally prefer tsx as package to run TypeScript directly in node. It uses ESBuild underneath which is very efficient in translation and rather skips most of the TSC stuff itself. This extension here also uses ESBuild to transpile TS to JS when discovering the tests.
Here an example mocharc from one of my projects: https://github.com/CoderLine/alphaTab/blob/develop/.mocharc.json

Top level await inside of a test file does not seem to work, or experimental decorators even if they are allowed by the tsconfig

What specific problems are you facing? Is it the test discovery (not seeing tests, getting errors? or the the test execution. Remember: the plugin just calls node <node args> <path to mocha> <mocha args>. If test execution fails it might be a general Mocha problem, running npx mocha is a good test to see if its in the extension here or Mocha itself.
If you have discovery problems it might be simply the fact that I do not load any tsconfig yet. Even though ultimately I want to add this bit, it is currently within my expectations.
https://github.com/Danielku15/mocha-vscode/blob/feature/mocha-standalone/src/extract/syntax.ts#L29C29-L29C45
https://github.com/Danielku15/mocha-vscode/blob/feature/mocha-standalone/src/extract/evaluate.ts#L150
I'm grateful for any examples you can provide for testingwith. I still have to dive in how ESBuild might auto-discover configs and then pass on the right tsconfig.
This PR is just the "starting point" and certainly quite some follow up work items are still needed to get it out of alpha/beta stage. The developments here effectively just a few days old 😁

Will look into that. Mocha does not have directly some "unique ids" we could use to reference tests so it rather works with passing in suite and test names to the npx mocha command. But it should be possible to generate some better arguments in such cases.

Thanks!

I think this is not related to this extension but Mocha in general. If you have concerns regarding this you might reach out to the Mocha devs to discuss potential improvements.

I see. Well, in my head I was just imagining that it should not matter if I open the project at the root or at specific package in vscode. Running the tests should go equally fast. Now if I put the .mocharc.json in a package and open that folder as the root I can run the same test almost instantly (that previously took seconds to launch).  But might be a mocha thing as you said... I also need perhaps read up on whats possible with vscode extension but it feeeels like it should be possible to scan for package.json in all subfolders and use the  mocha config file that is closest, and only force esbuild to interact with that specific folder when running the tests.

It's just a gut feeling but it could be caused by 'ts-node' you are using. I personally prefer tsx as package to run TypeScript directly in node. It uses ESBuild underneath which is very efficient in translation and rather skips most of the TSC stuff itself. This extension here also uses ESBuild to transpile TS to JS when discovering the tests.

Using tsx made things a bit faster  ! but still  about 10 seconds to launch some tests 🤔

What specific problems are you facing? Is it the test discovery (not seeing tests, getting errors? or the the test execution. Remember: the plugin just calls node   . If test execution fails it might be a general Mocha problem, running npx mocha is a good test to see if its in the extension here or Mocha itself.

I see, well I tested that to make sure this was not the case. Also updated the monorepo now to showcase that this only occurs for the test explorer which is interesting.

If you have discovery problems it might be simply the fact that I do not load any tsconfig yet. Even though ultimately I want to add this bit, it is currently within my expectations.

Aaah that would explain this behaviour I think 🙂
Thanks for the links to the code, I willl play around with the esbuild transform function a bit to see if I get something simple working but the  format: 'cjs' // vm.runInNewContext only supports CJS comment makes me think there will be issues down the road. Just setting that to esm  and/or providing a tsconfigRaw with proper configuration does not seem to work as I expect..

This PR is just the "starting point" and certainly quite some follow up work items are still needed to get it out of alpha/beta stage. The developments here effectively just a few days old 😁

Totally understand. I am amazed how fast you accomplish things! 👍👍

Might have to use https://nodejs.org/api/vm.html#class-vmmodule instead of vm.runInNewContext

I am just writing this here for memory but it also seems like when using tsx instead of ts-node the line numbers get wrong  so the debugger breakpoints does not work and error message does not show up on the right place when test fails. This is not related to this extension because I also observe this behaviour when running npx mocha

Might have to use https://nodejs.org/api/vm.html#class-vmmodule instead of vm.runInNewContext to get the esm parts to work. Looks like top level await does not work with cjs output format (https://github.com/evanw/esbuild/issues/253)

I tapped into using ESM instead of CSJ in the beginning but it currently seems not possible to use it. Beside the fact that it is an experimental feature, you need to know in advance the named exports of a module which are imported by the test file. Returning a dynamic proxy as default export still leads to errors.
https://nodejs.org/api/vm.html#class-vmsyntheticmodule
https://nodejs.org/api/vm.html#modulelinklinker

I am just writing this here for memory but it also seems like when using tsx instead of ts-node the line numbers get wrong so the debugger breakpoints does not work and error message does not show up on the right place when test fails. This is not related to this extension because I also observe this behaviour when running npx mocha

The problem seems to be known: https://github.com/privatenumber/tsx/issues/506

Seems like it is impossible to override the spec arg with specific files to run (?). '--run' options does not seem to have any effect.

An interesting finding. Its true that Mocha combines the --config (or autodetected config) and additional args parsed and this might result in problems filtering the tests.
https://github.com/mochajs/mocha/blob/e263c7a722b8c2fcbe83596836653896a9e0258b/lib/cli/options.js#L239-L243
Writing a config file for each test run seems an overkill and I would have also performance and synchronization concerns. According to this logic it might be best to --ignore everything and then re-add individual files via --file.

The other vscode extension (https://github.com/hbenl/vscode-mocha-test-adapter) seem to spawn a child process and interact with the 'mocha' lib directly to add all tests.

I wanted to avoid this path. I somehow have security and performance concerns evaluating all test files in full extend as part of the test discovery. Its more a gut feeling than something proven.
Using a custom UI combined with --dry-run should technically work but it will also require to either spawn and maintain a long running process with mocha for executing this part, or we would spawn processes for every test file.
The test discovery is already configurable. It could become an opt-in for devs to choose this mode in their project.

I merged this PR now as a foundation work.
I organized a bit the work in https://github.com/orgs/CoderLine/projects/15/views/1
Further discussions and bugreports we can handle over at https://github.com/CoderLine/mocha-vscode/discussions and https://github.com/CoderLine/mocha-vscode/issues

The problem seems to be known: https://github.com/privatenumber/tsx/issues/506

Yeap. But with Node 18.12 I can get the locations to work with ts-node but never succeeded with tsx no matter Node version.

According to this logic it might be best to --ignore everything and then re-add individual files via --file

I tried thefile arg on its own and it did not work. but never used theignore, so it sounds possible. Hopefully this will get the same performance improvement as rewriting the spec everytime (although with this change the test runs instantly now in my big repo)

Further discussions and bugreports we can handle over at https://github.com/CoderLine/mocha-vscode/discussions and https://github.com/CoderLine/mocha-vscode/issues

Sounds good!


"Go to bottom" button

With some threads on /pt/ reaching decent lengths it can get quite tedious to scroll to the bottom to see new comments so I suggest some kind of button that navigates you to the bottom of a page.

Good idea, implemented.


Python  - Basic Planner Error

Describe the bug
I define Native function
`from semantic_kernel.functions import kernel_function
class CustomAPIPlugin:
"""
Description: api plugin for planner
"""
@kernel_function(
description = "Get news from the web",
name = "NewsPlugin",
)
def get_news_api(self,location:str) -> str:
return """Get news from the """ + location + """."""
@kernel_function(
        description="Search Weather in a city",
        name="WeatherFunction",
    )
def ask_weather_function(self,city: str) -> str:
    return city + "’s weather is 30 celsius degree , and very hot."

@kernel_function(
        description="Search Docs",
        name="DocsFunction",
    )
def ask_docs_function(self,docs: str) -> str:
    return "ask docs :" + docs`

when I use Basic Planner in python show errors

maybe we need to set
available_functions_string += "description: " + str(description) + "\n"

Thanks for filing the issue. This is the same as #5309. A fix is already in main and will be included in the next release.


Selecting specific foreach tasks by ID/input

Lets say I'm running analyze_person step using foreach and passing in personID as input.
I want to be able to access the data artifacts produced by the step for person x, person y, person z.
I know joining the foreach outcomes eventually when it is done running that step for all people is an option, but there are a lot of people to process, and I don't want to have to wait until it completes for all people to utilize the results. or lose the processed outcomes if it the flow fails halfway for some reason.
The documentation for Step.task states that :
Convenience method to return the unique Task associated with this Step. If a Step has more than one Task, this will return any of them (no order guaranteed).
Is there a way I can access a specific task through some ID or input param.

@Viveckh You can access a specific task by using its identifier - Task('MyFlow/1/a/42') where MyFlow is the flow name, 1 is the run ID, a is the step name and 42 is the task ID. You can also do -
step = Step('MyFlow/1/a')
task = step['42']

Thanks @savingoyal ! Is the task ID something I can assign?
If not, is it something I can access from within the task while its' running, so I can store it elsewhere and tie person x to the their specific task ID

@Viveckh please let me know how to access this current object ?

this seems to work -
step = Step('MyFlow/1/a')
task_list=list(step.tasks())


Better support for editting structures with unknown bytes

Currently, when we edit a packed structure, all the padding bytes will vanish.
Also, it would be nice if we can directly toggle whether a structure is packed or not.

Not really related to packed or not since the issue will exist in regular structures as well. Need to extend our type parser to support an offset keyword or some sort of padding type that we can represent in the C representation that will then round-trip.

This was fixed back in 3.1 with the addition of the Clang type parser.


Dangling preauth_tx signers due to CAP 28

To reproduce:
Given an account A

Add a pre-auth signer (S) to A, associated to a transaction T, with an unachievable sequence number. (e.g. T has A as a source and a sequence number lower than Sequence(A))
Submit T
Core doesn't accept T but removes S
Horizon still displays S in GET accounts/A
It's impossible to remove S from Horizon's state. Trying to remove S manually with SetOps(Account=A, Signer=S,  Weigth=0) doesn't help, since Core has already removed S in (3).

I think this may be a blocker for 1.2 since, otherwise, this will result in Horizon's DB being polluted and I don't see an easy way to clean it up later on.

It turns out that, in my setup, Core is not deleting T in (3) and it is indeed manually removable.
I need to figure out why my local environment is not following CAP 28, I am using the stellar-core:latest container which I thought came with CAP 28. I need to investigate this.

I was misinterpreting CAP 28. Core will only clear the signer when there is a problem in signature verification. If the sequence number is invalid, the signer won't be cleared.

Do you think that's the right call in CAP 28? While slightly painful, it's probably not too late to tweak the CAP.

Also, if this is the right approach, can the CAP language be tweaked to be clearer?

Core will only clear the signer when there is a problem in signature verification. If the sequence number is invalid, the signer won't be cleared.

This is not totally correct either. CAP28 will take effect with incorrect sequences as well, but they need to be included in the ledger, which will only happen if they are correct with respect to the previous ledger (i.e. there needs to be another, concurrent transaction in the same Ledger which makes them invalid due to application ordering). More on https://github.com/stellar/go/issues/2377#issuecomment-618431408

Do you think that's the right call in CAP 28? While slightly painful, it's probably not too late to tweak the CAP.

I think the problem is on my interpretation of CAP 28, which turns out to be more subtle than I thought (thanks a lot to @jonjove  for clarifying it).

Also, if this is the right approach, can the CAP language be tweaked to be clearer?

@sisuresh mentioned he will be clarifying it (but not changing the implications of the CAP).


Deleted and uninstalled it self on update

Prerequisites

[ ] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
x Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
x Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
x Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
x Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
When installing the latestest version (here)[https://github.com/atom/atom/releases/tag/v1.27.0] Atom showed the normal squirel however when it went to reboot it never opened backup and all shortcuts stoped working in the atom folder in "C:\Users\username\AppData\Local\atom" it had removed atom.exe and Updated.exe i went into the version number and launch atom from there to which it didnt open anything in now that folder has just emptied it self
Downloading the installer to try reinstall atom now fatals saying "cant find Update.exe" (even if remove the atom folder from local it will then on install make a folder called atom and then fatal saying cant find Update.exe)
Steps to Reproduce

Get the latest update (via the auto updater)

Expected behavior:
To update normally
Actual behavior:
It deletes it self and refuses to re install
Reproduces how often:
First time ever had an issue updating and cant try to re run as now fails to install
Versions
You can get this information from copy and pasting the output of atom --version and apm --version from the command line. Also, please include the OS and what version of the OS you're running.
I dont know its now an unregistered task but its this version it updated to https://github.com/atom/atom/releases/tag/v1.27.0
Running Windows 10 version 1709
Note: At the time gitkraken did also have an update pending and i shut kraken just before atom and both share electron/squriel updater so maybe something broke in the temp directory

Note found a solution but gonna leave open cause seems relevant for others (and should probably get fixed)
Doing a shutdown doesn't fix it, however, doing a reboot does


Slow working between steps "Adding files" and "Generating new stub".

Hi.
I noticed a slow working between steps "Adding files" and "Generating new stub".
But CPU and SSD isn't loaded on 100%.
The CPU utilisation is ~24% on the php7.2 build process.
I have also done the tmpfs disk for source project and for the /tmp directory.
This did not speed up the build process.
? Adding files
    // I WAIT THIS ~HALF-TIME from total build time (~48 sec)
    > 5485 file(s)
? Generating new stub

Why does this phase/step work for so long? Thanks.

Hi, thanks for the report!
Did you enable the PhpScoper compactor? If that's the case then it is somewhat expected: scoping requires to parse the file into an AST (with nikic/PHP-Parser) and then traverse it to alter it before printing the modified content again. Box speeds things up by using parallelisation thanks to amphp, but I doubt it's gonna change much before https://github.com/humbug/box/pull/429.
If it's not I'm interesting in more details such as the output with the --debug flag or the dumped configuration

Closing due to lack of feedback. Feel free to come back to it whenever you wish and I'll re-open the issue


Can I get different events between mouseup and mousedown?

Colorpicker only has one callback for selecting colors. I need to distinguish mouseup and mousedown events.
While dragging the picker – this means “I'm choosing a color, not finished.”
When I picked a color from the palette or released the picker – this means “I'm finished to choose.”
I'm building a web-based content editor and I would like to push an undo history only when a user has finished choosing a color. Currently, tons of history had been pushed while choosing a color :(

hi @lqez
ok, I am understand.

When  onLastUpdate is fire

[ ] change color saturation
[ ] change hue bar
[ ] change opacity
[ ] change Input color text
[ ] click predefined color

@lqez
i released  codemirror-colorpicker v1.9.53  and @easylogic/colorpicker v1.9.65.
Please check it.

Awesome!

refer to https://colorpicker.easylogic.studio/colorpicker/api.html#callback-for-change-and-hide


add repo: data-umbrella/event-board-web

Repo:
https://github.com/data-umbrella/event-board-web
Issues:
https://github.com/data-umbrella/event-board-web/issues
good first issues:
https://github.com/data-umbrella/event-board-web/issues?q=is%3Aopen+is%3Aissue+label%3A"good+first+issue"
ℹ Repository information
The repository has:

[x] At least three issues with the good first issue label.
[x] At least 10 contributors.
[x] Detailed setup instructions for the project.
[x] CONTRIBUTING.md
[x] Actively maintained.

cc: @SangamSwadiK


reloader ignores file open events

Ignore file opened events in the Watchdog Reloader.

fixes #2603

Checklist:

[x] Add tests that demonstrate the correct behavior of the change. Tests should fail without the change.
[x] Add or update relevant docs, in the docs folder and in code.
[x] Add an entry in CHANGES.rst summarizing the change and linking to the issue.
[x] Add .. versionchanged:: entries in any relevant code docs.
[x] Run pre-commit hooks and fix any issues.
[x] Run pytest and tox, no tests failed.

@davidism thanks for the feedback! I ended up updating the requirements files in order to update to/past the changed version of watchdog. Please let me know if you have any other feedback


Squashed commit of the following:

commit edc6fab75e4c834b069e2064248b04a552b87d7a
Author: Pramod Immaneni<EMAIL_ADDRESS>Date:   Sun Jan 25 18:26:50 2015 -0800
Changed name renew to relogin, added docs

commit 8defce6cf9337a6df09ced5e00dbe867db458750
Author: Pramod Immaneni<EMAIL_ADDRESS>Date:   Sun Jan 25 17:52:55 2015 -0800
Added debug statements

commit d40a886e5613d9f53e27cd523bd1b64c60431cca
Author: Pramod Immaneni<EMAIL_ADDRESS>Date:   Sun Jan 25 17:47:06 2015 -0800
Added renewer and also changed name of keytab property

commit 0d7216c69542ec20a96c22e1315619f3e5666102
Author: Pramod Immaneni<EMAIL_ADDRESS>Date:   Sun Jan 25 12:22:44 2015 -0800
Added ability to specify user name in principal and keytab that would be dynamically substituted with unix user the application is running as at runtime

"Squashed commit of the following" is not a good message. Also, why do we need all the individual commits in the message?

This was autogenerated. Edited the pull request message.

It wasn't about the title of the pull request, but the commit message (which is still the same).


Stale web sessions still not begin cleaned

Apparently the fix didn't actually fix it. Web sessions are still not being cleaned up properly. Individually they're not much but when they pile up they take a good chunk of space.

The issue was because the registration page "/register" was rendering a view which automatically saved a session. The fix simply doesn't save a session unless there's a username attached to the session.


Could not build module foundation?

Thanks for this repository! 👍
I am having some problem to build my app after adding this library. Not sure if I missed anything but the error "Could not build module foundation" comes out while the other libraries could import foundation. Tried clean and build again but no luck.
Xcode version: 10.2.1
RN version: 0.59.9

Thank you.

I see, interesting.. I'll investigate :)

I haven't been able to find a solution yet, did you find anything yourself yet?

Thanks for your follow-up. I import the native SDK and used the Native Modules to call the native API at the end.


How to debug issues in URL?

I'm trying to debug an encoding issue in the URL. I want to quickly test an HTTP request language, and I want to see the results of:
request.get("http://httpbin.org/get/test");
request.get("http://httpbin.org/get/café");
request.get("http://httpbin.org/get/🐶");

I want to see if the library encodes the URLs using percent-escaping correctly or not. It would be very useful if httpbin.org would let me use custom URLs like that.

I'd be open to a /anything/:anything endpoint.

I'd be subject to Flask rules, of course.

This pull request #360 adds a /anything/:anything endpoint, but unfortunately, it isn't perfect for debugging issues with Unicode in the URL, because Flask automatically decodes and unescapes the URL. Consider:
$ curl 'http://<IP_ADDRESS>:5000/anything/café' | jq .url
"http://<IP_ADDRESS>:5000/anything/cafÃ©"
$ curl 'http://<IP_ADDRESS>:5000/anything/caf%C3%A9'
"http://<IP_ADDRESS>:5000/anything/café"

It would be great if the second command printed out the unescaped URL.


'Unhandled JS Exception: undefined is not an object

Environment
Environment:
OS: macOS High Sierra 10.13.4
Node: 8.11.3
Yarn: Not Found
npm: 5.6.0
Watchman: Not Found
Xcode: Xcode 9.4 Build version 9F1027a
Android Studio: 3.1 AI-173.4720617
Packages: (wanted => installed)
react: 16.3.1 => 16.3.1
react-native: 0.55.4 => 0.55.4
Description
RCTFatalException: Unhandled JS Exception: undefined is not an object (evaluating 'n.View.propTypes.style')', reason: 'Unhandled JS Exception: undefined is not an object (evaluating 'n.View.prop..., stack:
@821:2441
d@2:755
@820:81
d@2:755
@474:361
d@2:755
@359:135
d@2:755
@309:218
d@2:755
@308:141
d@2:755
@12:42
d@2:755
n@2:326
global code@880:8
'
*** First throw call stack:
(0x1846c3164 0x18390c528 0x100c15954 0x100c13414 0x1846caad0 0x1845a936c 0x1845ade1c 0x100c293a0 0x100c6d76c 0x100c6d4cc 0x184042a54 0x184042a14 0x18404c96c 0x18404d2fc 0x18404dd20 0x18405603c 0x1842eaf1c 0x1842eab6c)
libc++abi.dylib: terminating with uncaught exception of type NSException

it is in the react-native-elemets not in react native
that library won't be update with react native upgrade
issue solved.....


EFCore Set null one property of entity on preview 9

I need to save a entity that implements baseEntity
If i modify IdStatus, efcore Update it correctly, but if idStatus value does not change on other props updating => for some reason efcore set it to null
idStatus = 20 => try to update to 12 => success
idStatus = 12 => try to update description => idStatus = null, desc ok
i'm on

.net core 3.0 preview 9

efcore 3.0 preview 9

database first model creating

i use baseManager with iBaseEntity as common crud manager (IBaseEntity has id, DateMod ,UserMod

TTableEntity implements IBaseEntity interface

error seems occur only on IdStatus (nullable smallint, short?), but it works fine with other "nullable smallint, short?" prop

NOTES:

updated obj does not have tracking so i need to take an istance of same obj then copy props to it => then save
before 'SaveChanges()' idstatus is still 20

Some CODE for You
- Update Method (base manager where TTableEntity:class)
    public void Update(TTableEntity updated, int userid)
		{
			toSave = GetById(updated.Idnrr);

			PropertyCopierExtention<TTableEntity, TTableEntity>.Copy(updated, toSave);
                        //tosave.idstatus == 20 !
			toSave.DateMod = DateTime.Now;
			toSave.UserMod = (short)userid;

			_dbContext.Update(toSave);
			_dbcontext.SaveChanges();//after save => idstatus == null
		}
	public interface ISimpleEntity
	{
		int Idnrr { get; set; }
	}
	*public interface IBaseEntity : ISimpleEntity
	{
		short? UserIns { get; set; }
		DateTime? DateIns { get; set; }
		short? UserMod { get; set; }
		DateTime? DateMod { get; set; }
	}
    public partial class Tbl : IBaseEntity { }//i have to do this in other file
    public partial class Tbl
    {

        *public int Idnrr { get; set; }
        public string Description { get; set; }
        public DateTime? startDate { get; set; }
        public DateTime? endDate { get; set; }
        public short? IdStatus { get; set; }
        public byte? id2{ get; set; }
        public short? IdRequester { get; set; }
        *public short? UserIns { get; set; }
        *public DateTime? DateIns { get; set; }
        *public short? UserMod { get; set; }
        *public DateTime? DateMod { get; set; }
        //other 140 public props
    }

**- PropertyCopier code **
   public static class PropertyCopierExtention<TSource, TTarget> where TSource : class where TTarget : class
   {
       public static void Copy(TSource source, TTarget target)
   	{
   		var sourceProperties = source.GetType().GetProperties();
   		var targetProperties = target.GetType().GetProperties();

   		foreach (var sourceProperty in sourceProperties)
   		{
   			foreach (var targetProperty in targetProperties)
   			{
   				if (sourceProperty.Name.ToLower() == targetProperty.Name.ToLower() && sourceProperty.PropertyType == targetProperty.PropertyType)
   				{
   					targetProperty.SetValue(target, sourceProperty.GetValue(source));
   					break;
   				}
   			}
   		}
   	}
   }

Examples:

i tried to set toSave.IdStatus manually (no propCopier) with the same value saved in db (that means i don't want to change the value db.idstatus = 20 / tosave.idstatus = 20).After savechanges i have db.idstatus = null (A)
If i try to update again (null to 20 ) => success , db.idstatus = 20 (B)
i tried to change status value from 20 to 12 => updated.status = 12, after propscopy tosave.status = 12 (before copy was 20) => SaveChanges() db.status = 12 OK, it works (C)
BUT
-if i need to update some other value, or execute save again, after (C) (with updated.status=12 and db.status=12 ) => after propscopy toSave.status=12(event before) => saveChanges() db/tosave status prop becomes null (D)

@Maci94 Can you provide a small runnable project that shows this behavior?

@AndrewBoklashko i've added more examples , i hope you can help me
i've just updated .net core 3.0 to preview 9 and ef Core too
i didn't know what to do

@AndriySvyryd

@Maci94 I cannot reproduce this. Please create a GitHub repo with a runnable project.

i did more test on it and i found the snippets causes problem
View edit Post

@AndriySvyryd

uhm , sorry

This issue is lacking enough information for us to effectively reproduce. Please post a runnable project/solution or complete code listing that demonstrates the behavior you are seeing.

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.
BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.


Return a CompletableFuture from API methods that checks whether block…

…ing methods are called from an event loop.
We have heard many stories of bad experiences because of blocking the event loop in an armeria server request. While we wouldn't be able to check all cases of this (e.g., calling JDBC from an event loop), it's fairly simple to check this for futures returned by Armeria methods and I think this covers a lot of cases.  This changes to return a special implementation of CompletableFuture that logs from blocking methods with a stack trace.
Let me know any general feedback and then I'll apply to other methods (though I guess aggregate() is the most common way to get a future from Armeria).

Maybe a user can just use BlockHound? https://github.com/reactor/BlockHound
Netty integration is coming soon: https://github.com/netty/netty/pull/9687

BlockHound seems cool and could be a best practice when running an Armeria app to catch other things like JDBC. But I think registering an agent is tedious enough that it could still be worth having something out of the box. The cost / performance of manually instrumenting these three common methods seems pretty good to me. What do you think?

@trustin Any more thoughts on this? I notice there could be JDK version issues in the future with BlockHound - https://github.com/reactor/BlockHound/issues/33. It's a very cool tool, but agents are so complex I think this is could be a small but effective improvement.

Sorry. Let me check soon. Didn't have a chance to check the code yet.

I have switched to EventLoopCheckingCompletableFuture where we return a future to users. Though it might be simpler to just use it everywhere, let me know.

Hmm test doesn't seem to work well on Windows. Happen to have any ideas why that could be?

Maybe REPORTED_THREADS.add() returned false? How about using a different event loop for each test method?

Doh duh... thanks for finding it!

maybeLogIfOnEventLoop could be static by the way :-)


Feedback & discussion

Feel free to comment here

Origin:
- hosts: undercloud
  gather_facts: true
  environment:
    PLAYBZ: new_playbook
  become: true
  become_user: huhu
  tasks:

    - become: true
      environment:
        BLOCK_VAR: somevalue
        OVER_VAR: blockvalue

      block:

        - name: Install dnsmasq and bind-utils
          package:
            name: "{{ packages }}"
            state: present
          vars:
            packages:
              - dnsmasq
              - bind-utils

        - name: Configure - change cache size
          lineinfile:
            dest: /etc/dnsmasq.conf
            regexp: '^#cache-size=.[0-9]*$'
            line: 'cache-size=1500'
            backrefs: yes

        - name: Start and enable dnsmasq service
          systemd:
            name: dnsmasq
            state: started
            enabled: yes

    - name: Run shell command to grep something
      shell: echo "ddkfdkdf" | grep "dfdsfds" || true
      register: yaya

    - name: And this is with command
      command: dnf makecache

    - command: run-this command --option asksk
      environment:
        MYENV: otherfile
        OVER_VAR: task_value

    - shell: echo "Ddd" | grep "Dd" > /tmp/dddd
      args:
        chdir: /tmp/
      environment:
        MYENV: somefile

    - name: Gather variables for each operating system
      include_vars: "{{ item }}"
      with_first_found:
        - skip: true
          files:
            - "{{ ansible_facts['distribution'] | lower }}-{{ ansible_facts['distribution_version'] | lower }}.yml"
            - "{{ ansible_facts['distribution'] | lower }}-{{ ansible_facts['distribution_major_version'] | lower }}.yml"
            - "{{ ansible_facts['os_family'] | lower }}-{{ ansible_facts['distribution_major_version'] | lower }}.yml"
            - "{{ ansible_facts['distribution'] | lower }}.yml"
            - "{{ ansible_facts['os_family'] | lower }}-{{ ansible_facts['distribution_version'].split('.')[0] }}.yml"
            - "{{ ansible_facts['os_family'] | lower }}.yml"
      tags:
        - always

    - name: "Manage containers from {{ tripleo_container_manage_config }}"
      when:
        - (all_containers_hash|length) > 0
        - tripleo_container_manage_cli == 'podman'
      become: true
      block:
        - name: "Update container configs with new config hashes"
          include_tasks: /home/sshnaidm/sources/various/puppet_config.yml
          when:
            - tripleo_container_manage_check_puppet_config|bool
        # - name: "Delete orphan containers from {{ tripleo_container_manage_config }}"
        #   include_tasks: delete_orphan.yml
        #   when:
        #     - tripleo_container_manage_clean_orphans|bool
        # - name: "Create containers from {{ tripleo_container_manage_config }}"
        #   include_tasks: create.yml

    - name: Check if /etc/sysconfig/podman_drop_in exists
      stat:
        path: /etc/sysconfig/podman_drop_in
      register: podman_drop_in_file

    - name: Set podman_drop_in fact
      set_fact:
        podman_drop_in: true
      when:
        - podman_drop_in_file.stat.exists

    - copy:
        src: "{{ role_path }}/files/{{ item }}"
        dest: "/usr/libexec/{{ item }}"
        mode: '0700'
        owner: root
        group: root
      loop:
        - 'tripleo-container-shutdown'
        - 'tripleo-start-podman-container'

    - name: "Create /usr/lib/systemd/system/tripleo-container-shutdown.service"
      copy:
        src: "{{ role_path }}/files/tripleo-container-shutdown-service"
        dest: "/usr/lib/systemd/system/tripleo-container-shutdown.service"
        mode: '0644'
        owner: root
        group: root

    - name: Enable and start tripleo-container-shutdown
      systemd:
        name: tripleo-container-shutdown
        state: started
        enabled: true
        daemon_reload: true

    - name: "Create /usr/lib/systemd/system/netns-placeholder.service"
      copy:
        src: "{{ role_path }}/files/netns-placeholder-service"
        dest: "/usr/lib/systemd/system/netns-placeholder.service"
        mode: '0644'
        owner: root
        group: root

    - name: "Create /usr/lib/systemd/system-preset/91-netns-placeholder.preset"
      copy:
        src: "{{ role_path }}/files/91-netns-placeholder-preset"
        dest: "/usr/lib/systemd/system-preset/91-netns-placeholder.preset"
        mode: '0644'
        owner: root
        group: root

    - name: Enable and start netns-placeholder
      systemd:
        name: netns-placeholder
        state: started
        enabled: true
        daemon_reload: true

Ouput:
- targets: undercloud
  jobs:
  - NAME: Set playbook env value for PLAYBZ
    ENV: PLAYBZ new_playbook

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
## BLOCK-CONTEXT:
# become: True
# TASK-CONTEXT:
# name: Install dnsmasq and bind-utils
# package:
#   name: '{{ packages }}'
#   state: present
# vars:
#   packages:
#   - dnsmasq
#   - bind-utils

  - NAME: Install dnsmasq and bind-utils
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
## BLOCK-CONTEXT:
# become: True
# TASK-CONTEXT:
# name: Configure - change cache size
# lineinfile:
#   dest: /etc/dnsmasq.conf
#   regexp: ^#cache-size=.[0-9]*$
#   line: cache-size=1500
#   backrefs: yes
  - NAME: Configure - change cache size
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
## BLOCK-CONTEXT:
# become: True
# TASK-CONTEXT:
# name: Start and enable dnsmasq service
# systemd:
#   name: dnsmasq
#   state: started
#   enabled: yes
  - NAME: Start and enable dnsmasq service
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# register: yaya
  - NAME: Run shell command to grep something
    RUN: |-
      export PLAYBZ="new_playbook";
      echo "ddkfdkdf" | grep "dfdsfds" || true

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
  - NAME: And this is with command
    RUN: |-
      export PLAYBZ="new_playbook";
      dnf makecache

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
  - NAME: Unnamed task
    RUN: |-
      export PLAYBZ="new_playbook";
      export MYENV="otherfile";
      export OVER_VAR="task_value";
      run-this command --option asksk

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
  - NAME: Unnamed task
    RUN: |-
      cd /tmp/;
      export PLAYBZ="new_playbook";
      export MYENV="somefile";
      echo "Ddd" | grep "Dd" > /tmp/dddd

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Gather variables for each operating system
# include_vars: '{{ item }}'
# with_first_found:
# - skip: true
#   files:
#   - "{{ ansible_facts['distribution'] | lower }}-{{ ansible_facts['distribution_version']\
#     \ | lower }}.yml"
#   - "{{ ansible_facts['distribution'] | lower }}-{{ ansible_facts['distribution_major_version']\
#     \ | lower }}.yml"
#   - "{{ ansible_facts['os_family'] | lower }}-{{ ansible_facts['distribution_major_version']\
#     \ | lower }}.yml"
#   - "{{ ansible_facts['distribution'] | lower }}.yml"
#   - "{{ ansible_facts['os_family'] | lower }}-{{ ansible_facts['distribution_version'].split('.')[0]\
#     \ }}.yml"
#   - "{{ ansible_facts['os_family'] | lower }}.yml"
# tags:
# - always

  - NAME: Gather variables for each operating system
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
## INCLUDE-CONTEXT:
# name: Update container configs with new config hashes
# when: ['tripleo_container_manage_check_puppet_config|bool']
## BLOCK-CONTEXT:
# name: Manage containers from {{ tripleo_container_manage_config }}
# when: ['(all_containers_hash|length) > 0', "tripleo_container_manage_cli == 'podman'"]
# become: True
  - NAME: This is first task from include
    RUN: |-
      export PLAYBZ="new_playbook";
      grep "dddd" /etc/yum.conf.d/*

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
## INCLUDE-CONTEXT:
# name: Update container configs with new config hashes
# when: ['tripleo_container_manage_check_puppet_config|bool']
## BLOCK-CONTEXT:
# name: Manage containers from {{ tripleo_container_manage_config }}
# when: ['(all_containers_hash|length) > 0', "tripleo_container_manage_cli == 'podman'"]
# become: True
# TASK-CONTEXT:
# name: This is second task from include
# copy:
#   dst: /tmp/fdfew
#   src: /tmp/frgfd
  - NAME: This is second task from include
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Check if /etc/sysconfig/podman_drop_in exists
# stat:
#   path: /etc/sysconfig/podman_drop_in
# register: podman_drop_in_file
  - NAME: Check if /etc/sysconfig/podman_drop_in exists
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Set podman_drop_in fact
# set_fact:
#   podman_drop_in: true
# when:
# - podman_drop_in_file.stat.exists

  - NAME: Set podman_drop_in fact
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# copy:
#   src: '{{ role_path }}/files/{{ item }}'
#   dest: /usr/libexec/{{ item }}
#   mode: '0700'
#   owner: root
#   group: root
# loop:
# - tripleo-container-shutdown
# - tripleo-start-podman-container

  - NAME: Unnamed task
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Create /usr/lib/systemd/system/tripleo-container-shutdown.service
# copy:
#   src: '{{ role_path }}/files/tripleo-container-shutdown-service'
#   dest: /usr/lib/systemd/system/tripleo-container-shutdown.service
#   mode: '0644'
#   owner: root
#   group: root
  - NAME: Create /usr/lib/systemd/system/tripleo-container-shutdown.service
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Enable and start tripleo-container-shutdown
# systemd:
#   name: tripleo-container-shutdown
#   state: started
#   enabled: true
#   daemon_reload: true
  - NAME: Enable and start tripleo-container-shutdown
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Create /usr/lib/systemd/system/netns-placeholder.service
# copy:
#   src: '{{ role_path }}/files/netns-placeholder-service'
#   dest: /usr/lib/systemd/system/netns-placeholder.service
#   mode: '0644'
#   owner: root
#   group: root
  - NAME: Create /usr/lib/systemd/system/netns-placeholder.service
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Create /usr/lib/systemd/system-preset/91-netns-placeholder.preset
# copy:
#   src: '{{ role_path }}/files/91-netns-placeholder-preset'
#   dest: /usr/lib/systemd/system-preset/91-netns-placeholder.preset
#   mode: '0644'
#   owner: root
#   group: root
  - NAME: Create /usr/lib/systemd/system-preset/91-netns-placeholder.preset
    ECHO: Conversion of task module {task_module} is not implemented yet!

## PLAYBOOK-CONTEXT:
# become: True
# become_user: huhu
# TASK-CONTEXT:
# name: Enable and start netns-placeholder
# systemd:
#   name: netns-placeholder
#   state: started
#   enabled: true
#   daemon_reload: true
  - NAME: Enable and start netns-placeholder
    ECHO: Conversion of task module {task_module} is not implemented yet!

this looks great!