Debugger.paused  hangs (unable to pause the debugger from node.js)

Hi,
We are facing an issue for a while.
There is a node.js server running, and we have a client connected to it. The client has a simple TextBox, where you can write javascript code. Once the desired code is written, you can send it to the node.js server.
In Node.js the received javascript code will be written in a .js file, and after that it will be started in a childProcess (execFile function).
We want to use chrome remote interface to be able to pause the execution of the js file, edit it and after resume the execution. This is important to do it from code (not with chrome://inspect) because when pausing the execution with the debugger we also have to pause some other things.
Our problem is that await Debugger.paused() is hangs, the debugger never gets paused, therefore calling await Debugger.resume() we are getting the error: Can only perform operation while paused.
Here is a code snippet about how we are connecting the debugger:
const CDP = require('chrome-remote-interface');

exports.RemoteDebuggerInterface = class {
    constructor(port) {
        this.port = port;
        this.connected = false;
    }

    async connect(isScriptPausedByDefault = true) {
        let debuggerTargets = await this.getDebuggerTargets();

        if (debuggerTargets && debuggerTargets.length > 0) {

            let debuggerTarget = debuggerTargets[0];

            this.debuggerClient = await new CDP({
                port: this.port,
                target: debuggerTarget
            });

            if (this.debuggerClient) {
                console.log('Connected to remote debugger on: ' + debuggerTarget['webSocketDebuggerUrl']);
                console.log('DevTools can be reached on: ' + debuggerTarget['devtoolsFrontendUrl']);
            }
            this.connected = true;

            this.subscribeToClientEvent(this.debuggerClient, 'Runtime.executionContextDestroyed').then((message) => {
                console.log('Execution context destroyed :' + message);
            });

            if (isScriptPausedByDefault) {
                await this.debuggerClient.Runtime.runIfWaitingForDebugger();
            }
            this.subscribeToClientEvent(this.debuggerClient, 'Debugger.scriptParsed').then((message) => {
                console.log(message);
            });

            let id = await this.debuggerClient.Debugger.enable();
            console.log('Debugger enabled with unique id: ' + id);

          //to not copy a lot of code, we tried here to pause the debugger
          await this.debuggerClient.Debugger.paused();
          //by calling resume, we are getting the error: Can only perform operation while paused.
          await this.debuggerClient.Debugger.resume();

            return this.debuggerClient;
        }
        else {
            throw Error('error at connecting to the remote debugger');
        }
    }

Also, we are not able to open devtools by the given link (devtoolsFrontendUrl). The only way to open is using NIM extension for Chrome. BUT if the devtools is opened in chrome, then we get an error in node.js:
TypeError: Parameter "url" must be a string, not undefined
message:"Parameter "url" must be a string, not undefined"
stack:"TypeError: Parameter "url" must be a string, not undefined
at Url.parse (url.js:103:11)
at urlParse (url.js:97:13)
at Chrome._start (...\node_modules\chrome-remote-interface\lib\chrome.js:130:31)
I know I have a lot of questions, but I would like to start a discussion about this topic, because this is very important for us.

Component
Version

Operating system
Win8 64 bit

Node.js
v8.11.3

Chrome/Chromium/...
69.0.3497.100

chrome-remote-interface
0.26.1

Is Chrome running in a container? NO

Would something like this help?
const CDP = require('chrome-remote-interface');
const child_process = require('child_process');

const child = child_process.spawn('node', ['--inspect-brk=9222', 'script.js']);
child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);

setTimeout(() => {
    CDP(async(client) => {
        const {Runtime} = client;
        try {

            console.log('the execution is paused here, resuming...');

            await Runtime.runIfWaitingForDebugger();
        } catch (err) {
            console.error(err);
        } finally {
            client.close();
        }
    }).on('error', (err) => {
        console.error(err);
    });
}, 1000); // TODO better poll for success

To elaborate on that, please note that Debugger.paused is an event, conversely to Debugger.pause which is a method. So according to your code if isScriptPausedByDefault is true (default), Runtime.runIfWaitingForDebugger is called and the Debugger.paused event never fired, thus the hanging.

Also, we are not able to open devtools by the given link (devtoolsFrontendUrl). The only way to open is using NIM extension for Chrome. BUT if the devtools is opened in chrome, then we get an error in node.js

I guess that's a limitation of Chrome, it recently became possible to debug the same target simultaneously using more than one CDP/DevTools instance, but if I recall correctly not all the methods of all the domains allow that, and yours might just be one example, that is, if certain methods are used, then the webSocketDebuggerUrl field disappears from the target object so you get that error. But please take the above with a grain of salt since I've not tested it.

Thank you for the answer!
Yes, sorry for the mistake!
So, let's assume that the code is the same like in my very first comment, with one difference.
          //to not copy a lot of code, we tried here to pause the debugger
          await this.debuggerClient.Debugger.pause();
          await this.debuggerClient.Debugger.paused();
          //by calling resume, we are getting the error: Can only perform operation while paused.
          await this.debuggerClient.Debugger.resume();

In this case we call the pause method, and after that we should wait for pausing with the paused event. Right?
But paused will be never fired as currently we can see.
We are calling the pause and the resume method from these two methods:
Resume:
async resumeApplication() {
        this.checkIfConnectedToClient();
       //to check that the event is really fired, and resumed is always occurs
        this.debuggerClient.on('Debugger.resumed', (message) => {
            if (message.method === 'Debugger.resumed') {
                console.log(message.params);
            }
        });

        await this.debuggerClient.Debugger.resume();
        console.log('Executable file was resumed');
    }

Pause:
    async pauseApplication() {
        this.checkIfConnectedToClient();
         //to check that the event is really fired, and paused is never occurs
        this.debuggerClient.on('Debugger.paused', (message) => {
            if (message.method === 'Debugger.paused') {
                console.log(message.params);
            }
        });

        await this.debuggerClient.Debugger.pause();
        console.log('Application was paused');
    }

It is strange, that at the very first run the resume works, but after a pause when we attempt to call the resume, we are getting the error message mentioned before.
At the first time resume works probably because of the -brk swtich? But after a pause manually, paused event never occurs :/
Sorry if I'm not too clear, I'm really new to this.
But anyway, a big thank you for helping us!

In this case we call the pause method, and after that we should wait for pausing with the paused event. Right?

The full chain would be something like this:
// start ALWAYS with --inspect-brk

// XXX the order matters here!
await Runtime.runIfWaitingForDebugger();
await Debugger.enable();

console.log('resumed first time');

while (true) {
    await seconds(1); // XXX to simulate the interaction
    await Debugger.pause();   console.log('pausing...');
    await Debugger.paused();  console.log('paused');
    await Debugger.resume();  console.log('resuming...');
    await Debugger.resumed(); console.log('resumed');
}

Unfortunately the above never works basically. The reason, I guess, is that there's a race between the method and the event, e.g., the Debugger.paused event arrives before the actual await, so the correct version would be (full source code):
const CDP = require('chrome-remote-interface');

const child_process = require('child_process');

const child = child_process.spawn('node', ['--inspect-brk=9222', 'script.js']);
child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);

async function seconds(n) {
    return new Promise((fulfill) => setTimeout(fulfill, n * 1000));
}

setTimeout(() => {
    CDP(async(client) => {
        const {Debugger, Runtime} = client;
        try {

            Debugger.paused(async () => {
                console.log('paused');
                await Debugger.resume();
                console.log('resuming...');
            });

            Debugger.resumed(async () => {
                console.log('resumed');
                await seconds(1); // XXX to simulate the interaction
                await Debugger.pause();
                console.log('pausing...');
            });

            // XXX the order matters here!
            await Runtime.runIfWaitingForDebugger();
            await Debugger.enable();

        } catch (err) {
            console.error(err);
        }
    }).on('error', (err) => {
        console.error(err);
    });
}, 1000); // TODO better poll for success

So in short you cannot rely on await for these kind of events so you must make sure to enforce a consistent paused/resumed state using callbacks.

Does the above make sense?

Ok, then I'm closing this for now, but please don't hesitate to comment further if you need.


always show "Unable to install Xcode"

Describe the bug
always show "Unable to install Xcode"
To Reproduce
just downloaded xcodes.app, sign in my apple id, selected a version, click INSTALL
Expected behavior
install the target version of xcode
Screenshots

Version

OS: macOS Monterey 12.0.1
Xcodes: 1.4.0(12)

Hi @zrx89757 Can you make sure you're logged in properly via the Open Login option:

Usually download issues are a result of that.

Check my comment here:
https://github.com/RobotsAndPencils/XcodesApp/issues/215#issuecomment-1117091473

As of v1.8 we no longer require apple accounts to download Xcode so these errors won't show up anymore 🥳


Add new features to the sample.

Implementation of below features.
1.Sentiment Analysis.
2.Text To Speech.
3.Knowledge Connectors.

Thank you!


Raise go-resty version once again

With the upgrade to v2.5.0 of go-resty i accidentally introduced an issue for some users that had an older release of go-resty already in their gosum db.
This is due to the re-release of go-resty which should never happen in the go world.
As soon as there is a newer version out, we should update to that.
relates to: https://github.com/go-resty/resty/issues/415

There is now a new version, see: https://github.com/go-resty/resty/releases


Pass initialization options from the client to the server.

@sdankel noticed in #2 that the init_options are being ignored when passed through to the server, as below:
if not configs.sway_lsp then
   configs.sway_lsp = {
     default_config = {
       cmd = {'forc-lsp'},
       filetypes = {'sway'},
       -- This was missing!
       on_attach = on_attach,
       --  These are ignored for some reason :(
       --  init_options = {
       --  logging = { level = 'trace' },
       --  inlayHints = { maxLength = 25, renderColons = true, typeHints = true },
       --  trace = { trace = true, server = 'messages' }
       },
       root_dir = function(fname)
         return lspconfig.util.find_git_ancestor(fname)
       end;
       settings = {};
     };
   }
 end

We should dig a bit harder to identify what part of the pipeline is failing to either pass or accept these arguments.

This configuration worked:
       init_options = {
         logging = { level = 'trace' },
         inlayHints = { maxLength = 25, renderColons = true, typeHints = true },
         trace = {},
         debug = { showCollectedTokensAsWarnings = 'off' }
       },

It looks like all of the fields in the initialization options must be present or else the config is ignored.
Additionally, I noticed that TraceConfig is currently ignored. We might want to remove it if it's replaced by the LoggingConfig

We may also want to make the configs optional, if we want to allow passing partial init config.


When a protected branch exists, git-stack takes much longer

I noticed in a large repository that git-stack invocations seem to take a while:
$ time git-stack
master XXXXXXXXXXXX
git-stack  24.36s user 0.69s system 99% cpu 25.064 total

But if I delete master, then it completes immediately:
$ git checkout --detach && git branch -D master
HEAD is now at 548ba95f3c936 XXXXXXXXXXXX
branchless: processing checkout
Deleted branch master (was 548ba95f3c936).
$ time git-stack
548ba95 XXXXXXXXXXXX
git-stack  0.04s user 0.01s system 94% cpu 0.051 total

Is there any way to debug what's going on here?
This is on a large repository (500k+ commits) so it's understandable if you don't want to support this use-case.

 I took a screenshot of the .svg and uploaded that instead. Let me know if you want the original .svg and I can try to upload it somewhere else.)

Thanks for digging into that!

We are using revwalk for

Building our commit graph
Calculating distance between commits
Finding parent commits
Getting commits in a range

It seems starting a revwalk is the most expensive step and most of that time comes from forcing it to be a topological sort.
So it seems like if we'd need to avoid a topological sort to make things faster though we'll need these walks to be topological.  I assume doing a lot of this ourselves would speed it up.

What commit graph needs to be built/what distance needs to be calculated/etc. if you have only one branch? I would have figured that would only be necessary if you have multiple branches, or if the local branch tracks a remote branch which is far off (and even then, I wouldn't expect it to take 25s — that's around the time it takes to traverse every commit in the repository, in my memory).

if you have only one branch?

We could bypass some of the checks, for example, speeding up distance between HEAD and each protected branch (to find the closest) can easily be optimized by protected_branches.contains(head).  However, I decided to leave the code general for now so its easier to reproduce the performance issues so we can optimize it generally and then worry about it in the particular cases afterwards.

What commit graph needs to be built/what distance needs to be calculated/etc. if you have only one branch?

I was also listing every place we use revwalk.  The "distance between HEAD and master" was the one taking the most time in my tests.  In theory, that should be a trivial revwalk since it should terminate on the first item returned but all of the time is taken up before that first item is returned.

Also, each algorithm needed to be run in its own process, not just its own git2::Repository, because of caching.

#203 resolves most cases.
atm we don't support merge-commits and just topologically sort to resolve them.  That is the main thing left to see the performance issue go away.

@epage Thanks for the quick 😉  fix! It works a lot faster now in my repository.

Thanks for the feedback as I don't normally get to work with large repos lately.
btw is this just for comparison purposes or is there a way it you are using it in your workflow along with git-branchless?  Be curious to know how you might be using them together.

@epage I figured I'd try it out to manage my interactions with Github. Specifically, I don't have a way of pushing all branches in my current stack, which git stack --push seems to do for me (well, it pushes all of the ready ones, which is fine for me).

For more discussion on --push, see

https://github.com/gitext-rs/git-stack/discussions/189#discussioncomment-2276150
https://github.com/gitext-rs/git-stack/issues/191


Default branch was renamed to main

For more info: https://github.com/github/renaming
I know this is controversial, and nobody asked for it (yet). Still, I went ahead and unilaterally made the change, to avoid more pain in the future.
This issue is just to notify everyone, please adjust your local environments accordingly:
git branch -m master main
git fetch origin
git branch -u origin/main main

Regardless of whether I like this or not, (1) it's a debate that is not going away, and it will hit every project sooner or later, and (2) I will keep doing things I consider are important and impactful to make the tech industry and the open source universe more welcoming to others, as I've been doing for almost a decade now, and encourage others to do the same.
The open source universe has a diversity, inclusion and representation problem, and we have the moral obligation to fix it. This is just a baby step in that direction. The battle doesn't end here.

This was also made official by GitHub ... all newer repositories will be by default be named as Main rather than Master as it used to be the case 😄 [Shared just in case anyone wants some info]

A small correction @astrojuanlu ... the command should be git branch -u origin/master main than git branch -u origin/main main ... Checked it right now as it throws some unexpected errors ... Please check & confirm :)


bug: wrong zone selected when apex listener host in gateway

issue here is that the domain is an apex domain but the controller ignores this and finds a sub zone if present
The controller will fail with an error around the zone id and records . The controller selects the hcapapps.net zone rather than the specific.hcpapps.net zone
[Id: /hostedzone/Z06040283U9V7KNOSLFVI] when submitting change batch","controller":"dnsrecord","controllerGroup":"kuadrant.io","controllerKind":"DNSRecord","DNSRecord":{"name":"external-private","namespace":"demo"},"namespace":"demo","name":"external-private","reconcileID":"ff9b9e6d-2c2e-4991-bd7b-80d7f1442924","rootHost":"private.hcpapps.net","ownerID":"18mfjtpq","zoneID":"/hostedzone/Z06040283U9V7KNOSLFVI","zoneDomainName":"hcpapps.net","region":null,"error":"InvalidChangeBatch: [RRSet of type CNAME with DNS name private.hcpapps.net. is not permitted as it conflicts with other records with the same DNS name in zone hcpapps.net.]

Gateway:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: external
  namespace: demo
spec:
  gatewayClassName: istio
  listeners:
  - allowedRoutes:
      namespaces:
        from: Same
    hostname: '*.demos.kuadrant.io'
    name: api
    port: 443
    protocol: HTTPS
    tls:
      certificateRefs:
      - group: ""
        kind: Secret
        name: api-external-tls
      mode: Terminate
  - allowedRoutes:
      namespaces:
        from: Same
    hostname: specific.hcpapps.net
    name: private
    port: 80
    protocol: HTTP

HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: test
  namespace: demo
spec:
  hostnames:
  - specific.hcpapps.net
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: external
    namespace: demo
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: toystore
      port: 80
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /

The example given here is a bit confusing since it will look for a hosted zone to create a "specific" record in, the first one it will try is "hcpapps.net" as that is the only option in this case. I wouldn't really consider that a bug, it's how it always worked with ManagedZones, and the exact same thing would have happened if a ManagedZone for hcpapps.net existed in the NS. If there is a matching hosted zone for  "hcpapps.net" that your credentials give you access to in the provider why shouldn't it try and use that? If there are existing records in the zone that conflict then i would say the original error was correct.


PublishDataPacket with Reliable stops sending packets after being invoked a few times. No error returned.

Video Demoing Issue
https://github.com/user-attachments/assets/7a13940a-d7c6-4933-84ad-44a61c8bdba3
Logs
PublishDataPacket Demo Logs.csv
See Column Q for most relevant logs.
Code Snippet:
// Handle speech to text results.
client.OnResult(func(result *domain.SpeechToTextResult) {
	result.AttendeePeerID = attendeePeer.ID

	if result.IsFinal {
		if err := d.transcriptionRepository.Create(
			d.ctx,
			&result.Transcription,
		); err != nil {
			d.log.Log(logging.Entry{
				Severity: logging.Error,
				Payload: map[string]interface{}{
					"Message":  "Transcription result failed transmission to the database.",
					"Error":    err,
					"Room":     d.roomId,
					"Track":    publication.SID(),
					"Identity": rp.Identity(),
				},
			})
			return
		}

		d.log.Log(logging.Entry{
			Severity: logging.Info,
			Payload: map[string]interface{}{
				"Message":  "Transcription result stored in the database. No errors returned.",
				"Room":     d.roomId,
				"Track":    publication.SID(),
				"Identity": rp.Identity(),
			},
		})
	}

	// Map the model to match a typical API response. Include an
	// extra
	// field to indicate if the transcription is final.
	packet := struct {
		api.TranscriptionResponse
		IsFinal bool `json:"isFinal"`
	}{
		mapping.ModelTranscriptionToApiTranscriptionResponse(
			&result.Transcription,
		),
		result.IsFinal,
	}

	data, err := json.Marshal(packet)
	if err != nil {
		d.log.Log(logging.Entry{
			Severity: logging.Error,
			Payload: map[string]interface{}{
				"Message":  "Transcription result json Marshal failure.",
				"Error":    err,
				"Room":     d.roomId,
				"Track":    publication.SID(),
				"Identity": rp.Identity(),
			},
		})
		return
	}

	d.log.Log(logging.Entry{
		Severity: logging.Info,
		Payload: map[string]interface{}{
			"Message":  "Transcription packet marshaled. No errors returned.",
			"Room":     d.roomId,
			"Track":    publication.SID(),
			"Identity": rp.Identity(),
		},
	})

	// Publish transcription results to the room as a data
	// packet.
	if err := d.room.LocalParticipant.PublishDataPacket(lksdk.UserData(data),
		lksdk.WithDataPublishReliable(true),
		lksdk.WithDataPublishTopic(string(domain.Topic_Transcription)),
	); err != nil {
		d.log.Log(logging.Entry{
			Severity: logging.Error,
			Payload: map[string]interface{}{
				"Message":  "Transcription result failed transmission to the WebRTC room.",
				"Error":    err,
				"Room":     d.roomId,
				"Track":    publication.SID(),
				"Identity": rp.Identity(),
			},
		})
		return
	}

	d.log.Log(logging.Entry{
		Severity: logging.Info,
		Payload: map[string]interface{}{
			"Message":  "Transcription packet sent with PublishDataPacket. No errors returned.",
			"Room":     d.roomId,
			"Track":    publication.SID(),
			"Identity": rp.Identity(),
		},
	})
})

My temporary workaround is to send the data message using the LiveKit API. I substitute PublishDataPacket for:
liveKitClient.SendData(
	ctx,
	&livekit.SendDataRequest{
		Room:                  room,
		Topic:                 &topic,
		Data:                  data,
		Kind:                  livekit.DataPacket_RELIABLE,
	},
)

And this works and restores live transcription messages to my Flutter front-end. They don't suddenly stop anymore. But then I have to deal with rate-limits. So its not a scalable solution.

I put together a miniature, minimum-code example that triggers this bug. Looks like payload sizes on reliable 1126 bytes and above trigger this bug.
Interestingly, I could not trigger this bug on my local machine. Only upon deploying the same executable to Google Cloud Run/Jobs did I start seeing this issue.

Issue has been tracked down to a GCP Cloud Run/Jobs issue where the MTU size set by default (and unchangeable) is 1280 bytes. But Pion/LiveKit need at least 1400 bytes. The temporary solution is packet fragmentation. The permanent solution... don't use GCP Cloud Run/Jobs. Use GCE VM.


Can't use 'calculate-checksum' command for changesets where ID or AUTHOR or FILEPATH value contains '::'

Environment
Liquibase Version: 4.19.0
Liquibase Integration & Version: CLI
Liquibase Extension(s) & Version: NONE
Database Vendor & Version: ANY
Operating System Type & Version: ANY
Infrastructure Type/Provider: NONE
Description
calculate-checksum command is requiring to provide --changeset-identifier parameter in the format filepath::id::author
This format doesn't allow to use the command when ID or AUTHOR or FILEPATH have :: in the value.
liquibase calculate-checksum --help
Calculates and prints a checksum for the changeset
[deprecated: calculateChecksum]

Usage: liquibase calculate-checksum [OPTIONS]
Calculates and prints a checksum for the changeset with the given id in the
format filepath::id::author

      --changeset-identifier=PARAM
                         [REQUIRED] Changeset ID identifier of form filepath::
                           id::author
                         (liquibase.command.changesetIdentifier OR liquibase.
                           command.calculateChecksum.changesetIdentifier)
                         (LIQUIBASE_COMMAND_CHANGESET_IDENTIFIER OR
                           LIQUIBASE_COMMAND_CALCULATE_CHECKSUM_CHANGESET_IDENTI
                           FIER)
                         [deprecated: --changesetIdentifier]

Steps To Reproduce
Add next changelog.xml changelog to the Liquibase project:
<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd">

  <changeSet author="Me" id="1::createTable">
    <createTable tableName="PRIMARY_TABLE">
      <column name="ID" type="INT"/>
    </createTable>
  </changeSet>

</databaseChangeLog>

Execute: liquibase calculate-checksum --changeset-identifier="changelog.xml::1::createTable::Me"
Actual Behavior
Liquibase fails to parse the identifier:
liquibase calculate-checksum --changeset-identifier="changelog.xml::1::createTable::Me"
Starting Liquibase at 13:05:58 (version 4.19.0 #6648 built at 2023-01-17 15:02+0000)
Liquibase Version: 4.19.0
Liquibase Open Source 4.19.0 by Liquibase

Unexpected error running Liquibase: No such changeSet: changelog.xml::1::createTable

For more information, please use the --log-level flag

Expected/Desired Behavior
calculate-checksum command should not have limitations based on the ID or AUTHOR or FILEPATH values.
I would propose to replace --changeset-identifier parameter with a similar functionality like rollback-one-changeset PRO command has.
It is using three separate parameters to identity a changeset.
Example:
liquibase rollback-one-changeset --changesetPath="changelog.xml"  --changesetId="1::createTable" --changesetAuthor="Me" --force
Starting Liquibase at 13:20:54 (version 4.19.0 #6648 built at 2023-01-17 15:02+0000)
Liquibase Version: 4.19.0
Liquibase Pro 4.19.0 by Liquibase licensed to Liquibase Pro Customer
Rolling Back Changeset: changelog.xml::1::createTable::Me
rollback-one-changeset executed for user@jdbc:postgresql://localhost:5432/db
Liquibase command 'rollback-one-changeset' was executed successfully.

Workaround

Don't use :: in the ID or AUTHOR or FILEPATH values.

Not applicable for existing changesets

Hello! This is indeed a bug. We wil add it to our queue. If you want to help speed up the process, you can always submit a PR with a fix and we will help you through the process. Thanks!

Hi @nvoxland and @FBurguer,
I would try to fix this bug and submit a PR. Is the proposed behaviour in the issue description a valid solution?
If yes I would try to spread out the paramters for calculate-checksum into the proposed parameters.
Thx for your time :)
Best regards
Julien


model_json_schema() not creating JSON schema when subclassed

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
In the code below if BaseModel is used rather than the MyBaseModel subclass, then the results are as expected, however with the code below, the JSON schema is not produced and no error is raised.
from __future__ import annotations
from pydantic import BaseModel

class MyBaseModel(BaseModel):
    pass

class B(MyBaseModel):
    b: A

class A(MyBaseModel):
    a: str

print(B.model_json_schema())

expected result:
{'$defs': {'A': {'properties': {'a': {'title': 'A', 'type': 'string'}}, 'required': ['a'], 'title': 'A', 'type': 'object'}}, 'properties': {'b': {'$ref': '#/$defs/A'}}, 'required': ['b'], 'title': 'B', 'type': 'object'}
(or an error advising the model hasn't been initialiased)
actual result:
 {'properties': {}, 'title': 'MyBaseModel', 'type': 'object'}
If MyBaseModel has any fields set then it also works ok, e.g. this works as expected:
class MyBaseModel(BaseModel):
    x: None

Appreciate this is an edge case but thought is worth noting if there's something that can be done.
Example Code
No response
Python, Pydantic & OS Version
pydantic version: 2.1.1
        pydantic-core version: 2.4.0
          pydantic-core build: profile=release pgo=false mimalloc=true
                 install path: ...s\Python\venv\Lib\site-packages\pydantic
               python version: 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
                     platform: Windows-10-10.0.19044-SP0
     optional deps. installed: ['email-validator', 'typing-extensions']

Selected Assignee: @dmontagu

Thanks for reporting!
I believe I have found the source of the issue, and have opened a PR to resolve it #7020.


Documentation: Installation guide, required specs and libraries

Have rough installation page, does not include any information for troubleshooting if there are installation issues or have any information about required specs beyond Python 2.x (J 7/5/19)

Assigning this to myself. I'm going to make a release soon v1.x, and will write installation instructions based on that pypi release.


add update-logrotate.sh, implement feature to ensure /usr/bin/logrotate.d/logrotate.conf's configuration is kept update to date with log directories' log files

New Feature - Keep logrotate.conf up to date
Overview on changes
added a new bash /usr/bin/logrotate.d/update-logrotate.sh
modified /usr/bin/logrotate.d/docker-entrypont.sh's cron job to always run this update-logrotate.sh script, before running the logrotate command
if [ "$1" = 'cron' ]; then
  /usr/bin/go-cron "${logrotate_croninterval}" /bin/bash -c "/usr/bin/logrotate.d/update-logrotate.sh; ${logrotate_cron_timetable}"
fi

End Result
The /usr/bin/logrotate.d/logrotate.conf's config will be up to date with the log_dirs's log files (newly added log files' relevant logrotate configs will be added, removed logs files' relevant logrotate configs will be deleted)
Test

Copy the below cotent to a run.sh file

dir_name=$(cd `dirname $0` && pwd)

docker container run -it --rm --name logrotate-docker \
    -v ${dir_name}/containers:/var/lib/docker/containers \
    -v ${dir_name}/docker:/var/log/docker \
    -e "LOGS_DIRECTORIES=/var/lib/docker/containers /var/log/docker" \
    -e "LOGROTATE_COMPRESSION=compress" \
    -e "LOGROTATE_SIZE=50M" \
    -e "LOGROTATE_DATEFORMAT=-%Y%m%d" \
    -e "LOG_FILE=/var/log/docker/test" \
    -e "LOGROTATE_CRONSCHEDULE=* * * * * *" \
    shuliyey/logrotate

Note: shuliyey/logrotate is my forked version of the repo, can build the new image first and use the built image here

Run the run.sh file

./run.sh

On a new terminal session docker exec -it logrotate-docker bash

Add/Remove some logs files in the relevant LOGS_DIRECTORIES.

Check content in /usr/bin/logrotate.d/logrotate.conf will be update correspondingly

closing this pull-request, moving this pull-request to another pull-request on feature branch


Codecraft Legacy

Hi!
First of all, I'm here posting this because I find Codecraft a wonderful educational tool.
From my perspective, I love codecraft from 3 different reasons:

Via the Codecraft Assistant, a student could program Wio Terminal without installing everything:
Just as a Code Generator, is helping students to sketch the backbone of a program, (which could be later pasted in the IDE and edited). This made my students save TON of time, and it would be great to find a way to add module that produce code based on custom or more popular libraries, as well as more (if not all) Grove modules.
The TinyML stuff is amazing, but maybe it makes the wohle thing more difficult to mantain, as the whole thing is an API to Edge impulse (possibly? I don’t know what is working behind the sink).

Q: How is the building process managed by Codecraft? Are you using the Arduino CLI?
Q: One of the first things I would try to do if I succeded in building the app (which I'm not right now) would be create custom blocks, simply generating code (point 2). Creating a simple wifi-connection block, together with a jsonizer block and mqtt, would allow a dramatic speeding up of my teaching classes (hopefully many others)
Q: If the direction is to dismiss it from Seeed (ide.tinkergen.com), it would be interesting to distribute it as a docker compose file in order to be easily buildable and hostable around.

Dear vongomben,
Thank you for your valuable feedback on the CodeCraft project. As an educational tool, we highly appreciate your support and suggestions.
I would like to address your queries one by one:

How is the building process managed by Codecraft? Are you using the Arduino CLI?

CodeCraft utilizes both the Arduino and gcc compilers to build the code, which offers greater flexibility in terms of code optimization and customization.

Creating custom blocks for CodeCraft

We welcome contributions from the community in customizing CodeCraft's blocks to generate code for popular libraries and modules. You can modify the blocks' code by locating the corresponding file that matches the block ID, which can be identified in the XML configuration of the block. Following the structure of existing blocks, you can create your own custom blocks that generate code for your desired modules and libraries.

Distributing CodeCraft as a Docker Compose file

We welcome community contributions towards CodeCraft and one interesting option could be distributing it as a Docker Compose file. This would enable easy building and hosting for users who wish to use CodeCraft independently.
In summary, we highly value and appreciate your suggestions for improving CodeCraft. We encourage community involvement in customizing the blocks and adding support for popular libraries and modules, making it a more robust tool for educators and students. Thank you for your contributions to the open-source community.

Thanks for the answer.
Super interesting road map, when do you think this will happen?
Regarding point 2: I'm now failing in building Codecraft locally (I've opened some issues about it). Whenever I'll be able to I will be feedbacking you about the custom block build 💪
ciao for now


Confusing instructions at "Call the DLL from Python"

At section
"Call the DLL from Python"
The python code refers to two different superfastcode.fast_tanh and superfastcode2.fast_tanh2 functions
but the instructions above only show how to create
superfastcode.fast_tanh
Maybe user can work out how to write the tan2h function but it is quite confusing. Actually it is not clear what the difference between tanh and tanh2 is, but seems that it takes different time to run.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 40e3b6b1-a3d2-f4f2-b1ad-dd654a062343
Version Independent ID: b2418b84-04f4-60d2-ec1d-291a5d4d0c51
Content: Write C++ extensions for Python - Visual Studio
Content Source: docs/python/working-with-c-cpp-python-in-visual-studio.md
Product: visual-studio-windows
Technology: vs-python
GitHub Login: @JoshuaPartlow
Microsoft Alias: joshuapa

@perdigao1 -- Luis, thank you for your feedback.  Please consider these resources:

Visual Studio on Q&A
Visual Studio MSDN Forum
Stack Overflow
Reddit

@JoshuaPartlow -- Joshua, please look into this issue.

Hi @perdigao1 - Perhaps @zooba can comment/confirm as he just recently updated the page. Offhand though, I think the difference might just be in naming as it is meant to demonstrate the difference between using the CPython extensions and PyBind11. If you look at the 4th step of the PyBind11 section, you'll notice that it references the same tanh_impl function as the preceding section.

Closing this issue given time since last reply. If this is still an issue, particularly after reviewing the recently updated version of the article, please feel free to reopen.


On Mac build, multiplayer button has no effect

When pressing the Multiplier game button from the MBHaxe Ultra main menu:
Expected behaviour: it should show an 'Enter your name' dialog, like it does on the web build.
Actual behaviour: it doesn't do anything and the main menu is still the active canvas content. No errors are shown in the logs.
It used to work in v1.1.2 but stopped working in v1.1.3. I play on macOS 14.

This is related to the process where the game tries to connect to the master server. Check your internet connectiom?

Oops, I found what was the problem, it looks like my settings.json file was corrupted. The highscoreName field was null instead of a string. Changed it back to a string and now the button works.
I didn't make that string -> null change by myself, so I guess this field must have been set to null somewhere in the game.

Good catch, game should ensure that it should not be null


Server side rendering

The problem with server side rendering is that window doesnt exist. Neiter can it look what px the body has. Would be nice if the prop element also could be a numeric value in px.
units.convert('rem', '16px', 16);

Could you be able to do a pull-request for that? I would find it useful too :-)


Loading CSS

Bug Report
I am getting the error
Module parse failed: Unexpected token (1:0)
You may need an appropriate loader to handle this file type.

To Reproduce
In the Button.js component file I am importing CSS: import "./button.css";.
I am using Docz in a create-react-app where the CSS is working fine.
Expected behavior
The CSS should style the button when it is displayed in the documentation.
Environment

OSMojave
Node/npm version: [e.g. Node 10.15.2/npm 6.9.0]

Also, when I apply global styles using index.css, they do not show up on the components when displayed by Docz. How do I get them to work? Shouldn't that be explained in the documentation?! I can't see it anywhere.

https://github.com/pedronauck/docz-plugin-css


[core] Don't suppress output for methods with many nop instructions

Jadx outputs fallback instructions when decompilation fails. In https://github.com/skylot/jadx/blob/master/jadx-core/src/main/java/jadx/core/codegen/MethodGen.java#L293-L296 , this output is suppressed for methods that contain more than 100 instructions. While this sounds like a good idea, it also hits some methods that are mostly comprised of nop instructions, which are skipped in the output anyway.
The easiest solution is replacing the check with Stream.of(insnArr).filter(insn -> insn != null && insn.getType() != InsnType.NOP).count() > 100, though there are also solutions that are more efficient. I'm not sure if this code path is hot enough to justify using a more efficient approach.
If this change sounds like a good idea I can build a PR for it.

@yawkat sure, this looks like a nice improvement. Performance is not a high priority here, because fallback not used very often, so your solution is good enough.
By the way, for small changes, you can directly open PR. I add issue requirements mostly for big change coordination (to prevent merge conflicts) and also to help choose a good start point and architecture hints.

Yea, just didn't want to build a test case before knowing a fix would make sense. Submitted.


Requirements bump for develop

pip-compile -U --resolver=backtracking requirements.in > requirements.txt

Out of date dependency bump no longer relevant


Bundled React library throws Invalid hook call error

Bug report

What is the current behavior?
I get the following React error when trying to use my library built with Webpack
Error: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
1. You might have mismatching versions of React and the renderer (such as React DOM)
2. You might be breaking the Rules of Hooks
3. You might have more than one copy of React in the same app
See https://reactjs.org/link/invalid-hook-call for tips about how to debug and fix this problem.

If the current behavior is a bug, please provide the steps to reproduce.
I've created a simple library https://github.com/vasilev-alex/webpack-example which reproduces the above error. As soon, as I import a React hook in my lib I get the above error.
What is the expected behavior?
I can use Webpack to bundle React libraries.
Other relevant information:
webpack version: 5.46
Node.js version: v12.22.1
Operating System: macOS Big Sur
Additional tools: -
I also posted this comment https://github.com/facebook/react/issues/13991#issuecomment-886656170 in the respecting issue https://github.com/facebook/react/issues/13991, none of the suggested solutions helped me.
P.S. using Rollup for the same library works fine.

Try to disable minimication and update deps - terser plugin and terser itself

@alexander-akait I'm not using Terser for the library, and Terser that comes with react-scripts is used only in production mode

You have mixed version of webpack, please wait when CRA will be update own code base for webpack v5 (run yarn list webpack to check it), so some plugins used from webpack v4 and some from webpack v5 (unfortunately they are not intended for this use and have breaking changes between v4 and v5), so you have broken output, unfortunately we can't fix it on our side, sorry, feel free to feedback

@alexander-akait that makes sense. Thanks!

@alexander-akait just tried to downgrade webpack to 4.44.2 and it doesn't seem to work either

Something wrong on CRA side, can you try to setup it again from get started?

@alexander-akait same thing again : (

I just wonder if anybody else faces this issue

Yes, me too, because a lot of developers use CRA
Why you set libraryTarget?

commonjs2 but I also tried
{
  // ...
  libraryTarget: 'module',
},
 experiments: {
    outputModule: true
},

Just remove libraryTarget

Then I get the following error:

Do you want to create library or application? CRA for applications

I'm building a library that I use in a CRA app

hm, looks like different react versions, give me time I will look at this deeply

Remove LoaderOptionsPlugin and add externals: { react: 'react', 'react-dom': 'react-dom',}, set devtool: "source-map" for production, you have two versions of react, it is invalid, also you can use libraryTarget: 'module', note - commonjs2 and module generates different exports so import/require can be different in your application, previously, developers generated both commonjs and ecma modules versions of library, now you can generate only ecma modules and use import { Counter } from "your-library";

environment: {
  module: true,
},

You need this due https://github.com/vasilev-alex/webpack-example/blob/main/package.json#L21 and your browsers doesn't support ECMA modules, when you bundle library you don't need this, most of developers uses bundlers so imports will be bundled, for direct usage ECMA modules should be supports in browser, so I recommend to remove it, if you have app and library in one repo - you can use monorepo or solution from above


Conductor is sending POST instead of GET

I've built a task in my workflow that should call GET but doing POST call
    "name": "task_1",
    "taskReferenceName": "task_1",
    "type": "HTTP",
    "inputParameters": {
        "http_request": {
            "uri": "http://localhost/database",
            "method": "GET",
            "body": {
                "param": "paramData"
            }
        }
}

Could be because you are including "body" for the GET request.
GET request would not have a body part.
There is a "header" parameter.
headers | A map of additional http headers to be sent along with the request.

My recommendation is to avoid sending GET requests with payload data:
It is discouraged on the HTTP RFC 2616 specification and most of the server implementations use this as a base guideline.
For a more detailed explanation, see accepted answer on: https://stackoverflow.com/questions/978061/http-get-with-request-body
Having said that, if you think the GET request is the way to go for your use case, you could put that data as one/several parameters in the url:
 "method": "GET",
 "uri": "http://myhostname/api?myparam=myparamValue",

Hope it helps :)

@josedab, that helps a lot. Much thanks!


Why is there a security warning?

https://learn.microsoft.com/en-us/aspnet/core/fundamentals/routing?view=aspnetcore-5.0#complex-segments-1

When using System.Text.RegularExpressions to process untrusted input, pass a timeout. A malicious user can provide input to RegularExpressions causing a Denial-of-Service attack.

The two links only lead to generic information. But it remains unclear how in this world an unknown input might cause a DoS when processing it with a known regular expression. Never heard about that. Please explain, I want to understand the problem.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 32374077-e02e-d6a6-1cea-05e58447b7cd
Version Independent ID: 86aeab13-1c25-9fc1-4e56-0390cb40c242
Content: Routing in ASP.NET Core
Content Source: aspnetcore/fundamentals/routing.md
Product: aspnet-core
Technology: aspnetcore-fundamentals
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Associated WorkItem - 64611

Hello! Thanks for contacting us. This sounds like a general question about using ASP.NET Core. While we try to look at and respond to all issues, for questions like this we recommend posting to a community support group like Stack Overflow with the asp.net-core tag.

Stack Overflow
ASP.NET Core Slack
ASP.NET Gitter

Potential bugs in the ASP.NET Core framework and product feedback can be reported at dotnet/aspnetcore issues (include full repro steps and cross-link to a repro project in GH when appropriate). For Visual Studio, use the Report a Problem gesture or Suggest a Feature gesture within VS, which open internal issues for the VS team. For Visual Studio Code, you can open an issue at microsoft/vscode issues.

This sounds like a general question

This must be a misunderstanding. My question is not about using the product but about a missing part in the documentation. That's why I posted the issue here. The issue is that there is a security warning in the text, but it is completely unhelpful because it doesn't explain what the problem is that I should avoid and how to do that (other than using a timeout which seems like a last-resort but not preventing the root cause). In this state, the entire note might as well be removed.


Make releases as tags

I would like to be able to reproduce the builds of the extension. Can you please mark all releases as tags before uploading them to the extension site?
This way people would also be able to compile the extension by themselves if they want to.

Good idea. I've started to do this for new releases now. https://github.com/duckduckgo/duckduckgo-privacy-extension/releases


Use more obscure symbols when mapping rebus answers to rebus symbols.

@orta for review

I agree, but I switched the order so that ❶ et al comes first. I think that feels more like a "sigil" than lesser used (in English) characters like À - because they are more primary letters in other langs


Ilike condition

Hello,
I'm using the like function of the selectBuilder to filter my results
go 	builder.Where(builder.Like("names", "%"+*queryParams.Names+"%"))
I want the condition to be case insensitive, so I found 'ilike' function on postgres. I see the condition is not implemented, ¿is there a way to implement my own filter conditions?

The Where() method accepts any valid SQL string as input, so you can simply write your SQL like following.
builder.Where(
    "names ilike " + builder.Var("%"+queryParams.Names+"%")
)

Thanks @huandu!


⚠ Node - 54360d94.yalaso.top has degraded performance

In 5702df4, Node - 54360d94.yalaso.top (http://54360d94.yalaso.top/api/v1/ping) experienced degraded performance:

HTTP code: 200
Response time: 3329 ms

Resolved: Node - 54360d94.yalaso.top performance has improved in 8effd5b.


Unrecognized target framework netcoreapp3.1

The VS Code Azure Functions (Extension v1.6.0 - 2021-11-04) "add API to static web app instructions"  say to use VSCode F1: "Azure Static Web Apps: Create HTTP Function" instruction.
The error message on vscode at the end of the wizard is:
Unrecognized target framework "netcoreapp3.1". Available frameworks: "net6.0", "net6.0".
Source: Azure Functions (Extension)

My dotnet --list-sdks shows a 3.1.415 and dotnet --lsit-runteims shows  3.1.21.
Also, if relevant:
>npm list --global --depth=0
+--<EMAIL_ADDRESS>
How do I specify the runtime?

Hi @OzBob
Could you run func --version for me and tell me what version of the Function Core Tools that you have installed?  I think you may have a version that is no longer compatible with .NET 3.1.

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Can't add more item if destination array is pre-defined

What is happening is that it doesn't work if destination array is already predefined.
If I set destination array as predefined and then try to add more items I'm getting the error:
custom-listbox.component.html:22 ERROR TypeError: Cannot add property 2, object is not extensible
    at Array.push (<anonymous>)
    at _loop_1 (angular-dual-listbox.js:416)
    at CustomListboxComponent.push../node_modules/angular-dual-listbox/fesm5/angular-dual-listbox.js.DualListComponent.trueUp (angular-dual-listbox.js:424)
    at CustomListboxComponent.push../node_modules/angular-dual-listbox/fesm5/angular-dual-listbox.js.DualListComponent.moveItem (angular-dual-listbox.js:565)
    at Object.handleEvent (custom-listbox.component.html:22)
    at handleEvent (core.js:29239)
    at callWithDebugContext (core.js:30309)
    at Object.debugHandleEvent [as handleEvent] (core.js:30036)
    at dispatchEvent (core.js:19859)
    at core.js:28448

The problem occurs only if source and destination are array of objects. With array of strings it works as expected.

Without seeing the source for your custom-listbox and the arrays of objects, it is impossible for me to guess the cause. Can you create a small reproducer in stackblitz of the problem you are having?
I am not seeing it in the demo, which uses arrays of objects.

Same problem here. I'm using Angular 8.

@renanvm - could you provide a small code sample that reproduces the error?

           <dual-list [source]="regioes" [height]="'300px'" [format]="format" [(destination)]="regioesSelecionadas"></dual-list>

`export class DistribuidorRegiaoComponent implements OnInit {
regioes: any[] = [];
regioesSelecionadas: any[] = [];
ngOnInit() {
this.regiaoService.query().subscribe(res => {
this.regioes = res.body;
});
}
`
It loads just the first item of the array

@renanvm - in your example, what is an example of res.body?

@renanvm - in your example, what is an example of res.body?

(2) [{…}, {…}] 0: {id: 1051, nome: "Norte", distribuidor: null} 1: {id: 1052, nome: "Sul", distribuidor: null} length: 2 __proto__: Array(0)

@renanvm: Please change your template to be:
<dual-list key="id" display="nome"
  [source]="regioes" [height]="'300px'" [format]="format" [(destination)]="regioesSelecionadas"></dual-list>

Here is the code I used for the component:
import { Component, OnInit } from '@angular/core';
import { DualListComponent } from 'angular-dual-listbox';
import { RegiaoService } from './regiao.service';

@Component({
    selector: 'app-root',
    templateUrl: './app.component.html',
    styleUrls: ['./app.component.scss']
})
export class AppComponent implements OnInit{
    format: any = DualListComponent.DEFAULT_FORMAT;
    regioes: any[] = [];
    regioesSelecionadas: any[] = [];

    constructor(private regiaoService: RegiaoService) {
    }

    ngOnInit() {
        this.regiaoService.query().subscribe(res => {
            this.regioes = res;
        });
    }
}

Here's what I used for the service:
import { Injectable } from '@angular/core';

import { BehaviorSubject, Observable, of } from 'rxjs';

@Injectable({
    providedIn: 'root'
})
export class RegiaoService {
    private regiao: any[] = [
        {id: 1051, nome: "Norte", distribuidor: null},
        {id: 1052, nome: "Sul", distribuidor: null}
    ];
    private regiaoSubj: BehaviorSubject<any>;

    constructor() {
        this.regiaoSubj = new BehaviorSubject<any>(this.regiao);
    }

    query(): Observable<any> {
        return this.regiaoSubj.asObservable();
    }
}

I'm not seeing any errors and I see Norte and Sul available to select. Either this is not the same issue @harishajdarevic saw or the reproducer is not complete enough for your use case to replicate the error or it was a simple misconfiguration not providing the key and display in your dual-list.

Thanks @czeckd . Now everything is working. Really was a misconfiguration.


Show Back-to-Top Button Only After Scrolling some distance down

Currently, the back-to-top button appears immediately when the page loads, which can be improved further. To improve the user experience, I would like to update the button's behavior so that it only becomes visible after the user scrolls down a certain distance (e.g., 200px).
Since every page in the website has a back to top button, I would like to add this feature to all the pages
@vivekvardhan2810  Kindly assign me this issue under the gssoc and hactoberfest labels

for now we are not looking for this.
so i am closing this issue.
please understand.
And one more thing i am going to be inactive from today evening, including project also.


No final.hic file generated

Hi,
Here is my command and it took almost 28 days to finish.
run-asm-pipeline.sh -m diploid -i 5000 --splitter-coarse-resolution 100000 --splitter-fine-resolution 1000 Perrie_HiC.asm.hic.p_ctg.fa merged_nodups.txt
From *.rawchrom.fasta to get FINAL.fasta, it took 26 days (super SLOW). The majority time had been used to generate alignments.txt file.
rawchrom.fasta file size is 2.1G. I am wondering if the file size is too big for the pipeline to handle.
I got *.final.asm, final.cprops, *final.assembly, *FINAL.fasta and *FINAL.assembly, however, there is no final.hic file generated.
I improved the pipeline performance by doing the following:
@debian:~/tools/improved-3d-dna$ diff run-asm-pipeline.sh ../3d-dna/run-asm-pipeline.sh
194,195c194
< #default_merger_lastz_options="--gfextend\ --gapped\ --chain=200,200"
< default_merger_lastz_options="--gfextend\ --gapped\ --chain=200,200\ --allocate:traceback=1.99G"

default_merger_lastz_options="--gfextend\ --gapped\ --chain=200,200"
831,834c830
<     # improve the performance
<     # wrapped.fasta is much faster than orig_fasta
<     awk -f ${pipeline}/utils/wrap-fasta-sequence.awk ${orig_fasta} > ${genomeid}.wrapped.fasta
< 	awk -f ${pipeline}/edit/edit-fasta-according-to-new-cprops.awk ${genomeid}.rawchrom.cprops ${genomeid}.wrapped.fasta > ${genomeid}.rawchrom.fasta

awk -f ${pipeline}/edit/edit-fasta-according-to-new-cprops.awk ${genomeid}.rawchrom.cprops ${orig_fasta} > ${genomeid}.rawchrom.fasta

Any suggestions why the final.hic file was missing, and why the alignment step was SO slow, how can we speed it up?
Thanks,
jing

Hello Jin,
Yes, your action to address the speed issue is correct regarding wrapping the fasta. This is discussed on the forum (aidenlab.org/forum.html http://aidenlab.org/forum.html). I believe also this is done automatically in the latest 3d-dna release (associated with Hoencamp et al., 2021 aka phasing branch https://github.com/aidenlab/3d-dna/releases/tag/201008).
We unfortunately do not support the diploid mode any longer, but thank you for your comments regarding traceback. I think it can probably be helpful if the contig N50 is very large. There is no final.hic with the diploid mode as liftover of read alignment positions is not a trivial task after alt haplotype merge. This is expected behavior. To build the final.hic you will actually have to rerun Juicer and visualize the contact map as your new draft (see p.5 of the Genome Assembly Cookbook dnazoo.org/methods http://dnazoo.org/methods).
Best,
Olga

On Jun 22, 2022, at 8:22 AM, ithacajing @.***> wrote:
Hi,
Here is my command and it took almost 28 days to finish.
run-asm-pipeline.sh -m diploid -i 5000 --splitter-coarse-resolution 100000 --splitter-fine-resolution 1000 Perrie_HiC.asm.hic.p_ctg.fa merged_nodups.txt
From *.rawchrom.fasta to get FINAL.fasta, it took 26 days (super SLOW). The majority time had been used to generate alignments.txt file.
rawchrom.fasta file size is 2.1G. I am wondering if the file size is too big for the pipeline to handle.
I got *.final.asm, final.cprops, *final.assembly, *FINAL.fasta and *FINAL.assembly, however, there is no final.hic file generated.
I improved the pipeline performance by doing the following:
@Debian https://github.com/Debian:~/tools/improved-3d-dna$ diff run-asm-pipeline.sh ../3d-dna/run-asm-pipeline.sh
194,195c194
< #default_merger_lastz_options="--gfextend\ --gapped\ --chain=200,200"
< default_merger_lastz_options="--gfextend\ --gapped\ --chain=200,200\ --allocate:traceback=1.99G"
default_merger_lastz_options="--gfextend\ --gapped\ --chain=200,200"
831,834c830
< # improve the performance
< # wrapped.fasta is much faster than orig_fasta
< awk -f ${pipeline}/utils/wrap-fasta-sequence.awk ${orig_fasta} > ${genomeid}.wrapped.fasta
< awk -f ${pipeline}/edit/edit-fasta-according-to-new-cprops.awk ${genomeid}.rawchrom.cprops ${genomeid}.wrapped.fasta > ${genomeid}.rawchrom.fasta
awk -f ${pipeline}/edit/edit-fasta-according-to-new-cprops.awk ${genomeid}.rawchrom.cprops ${orig_fasta} > ${genomeid}.rawchrom.fasta
Any suggestions why the final.hic file was missing, and why the alignment step was SO slow, how can we speed it up?
Thanks,
jing
—
Reply to this email directly, view it on GitHub https://github.com/aidenlab/3d-dna/issues/151, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLAMG2ODDCTRBFZBNDO6ULVQMHQZANCNFSM5ZQGU4UA.
You are receiving this because you are subscribed to this thread.

Hi Olga,
Thank you so much for your explanation and advice.
Best,
Jing

Hi,
I run it with haploid mode for 8 rounds of correction and didn't get the final.hic file either? Intreastingly, I soft linked the merged_nodups.txt file from the juicer output folder, and the 3d-dna somehow modified it to an empty file. Do you know what could be the reason?
best,
Cui

I am sorry, this is too vague of a description to diagnose. Please note that 8 rounds of correction are almost never necessary and you get the most bang for your buck within the first 2-3 rounds. Thanks, -Olga

On Aug 3, 2022, at 3:28 AM, smallfishcui @.***> wrote:

Hi,
I run it with haploid mode for 8 rounds of correction and didn't get the final.hic file either? Intreastingly, I soft linked the merged_nodups.txt file from the juicer output folder, and the 3d-dna somehow modified it to an empty file. Do you know what could be the reason?
best,
Cui
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.

thanks Olga. I somehow managed to finish the pipeline in another run, and the result looks quite satisfying!
best,
Cui


Interfaces can still be accessed after the runtime has been uninitialized

Continuing conversation that started in #97, it is currently possible to call interface methods after CoUninitialize has been called. It is uncertain whether this is actually unsafe or whether the program will be properly aborted once an interface method is actually called.
One possible solution is to tie the lifetime of the runtime to the lifetime of each interface but this makes the crate much less ergonomic. One other solution is to get rid of the explicit runtime struct and simply expose an unsafe CoUninitialize function that requires the user verifies that no active interfaces are still alive when it is called.
On top of this other constructs could be built that make things easier to handle but might require some sort of runtime cost (e.g., a reference counter construct that only call CoUninitialize when all interface references have been dropped).

It is very difficult to track the relationship of interfaces to apartments. Some interfaces are agile and thus do not require an apartment at all. Others are proxies are thus imply potentially multiple apartments. Most projects should not be calling the various functions for creating and tearing down apartments as the hosting environment often does that for you. That is especially so when you're inside a DLL. Either way, calling CoUninitialize is even less valuable as David Cutler does a better job of cleaning up than we could ever do by hand. 😉

In that case, maybe its better to remove the ApartmentRuntime type entirely and make deinit_apartment unsafe? That would allow users to call init_apartment once at the beginning of their program and optionally control if/when to call CoUninitialize. That also offloads the burden of proving CoUninitialize's safety to the user, which is probably better than putting CoUninitalize in a Drop impl.


Remove inexistent threads#list API

The Threads API doesn't expose a method to get the list of threads. It isn't listed on the official documentation:

Trying that endpoint returns the following error:
"OpenAI HTTP Error (spotted in ruby-openai 6.3.1): {"error"=>{"message"=>"Your request to GET /v1/threads must be made with a session key (that is, it can only be made from the browser). You made it with the following key type: secret.", "type"=>"invalid_request_error", "param"=>nil, "code"=>"missing_scope"}}"

All Submissions:

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?

Thanks, @ignacio-chiazzo - looks like it was undocumented and removed in November. I'm not call this a breaking change as it was not documented and was gone within a week of this method being added.


OpenSSL 3.x support needed for Fedora, CentOS, RHEL

UAMQP Package Version: 1.4.2
Operating System: Fedora 36 (Rawhide)
Python Version: 3.10.0rc2

Describe the bug
Fedora 36 contains [OpenSSL 3|https://fedoraproject.org/wiki/Changes/OpenSSL3.0] and uamqp fails to compile with it. (Full error pasted below.)
To Reproduce
Steps to reproduce the behavior:

podman run --rm -it registry.fedoraproject.org/fedora:rawhide
In the container, install packages: dnf -y install cmake make gcc-c++ openssl-devel cython
In the container, install uamqp: pip -vvv install --no-binary :all: uamqp

Expected behavior
I expected the module to be built.
Additional context
    [ 51%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/tlsio_openssl.c.o
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘engine_destroy’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:752:9: warning: ‘ENGINE_free’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      752 |         ENGINE_free(tls->engine); // Release structural reference.
          |         ^~~~~~~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:9:
    /usr/include/openssl/engine.h:493:27: note: declared here
      493 | OSSL_DEPRECATEDIN_3_0 int ENGINE_free(ENGINE *e);
          |                           ^~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘engine_load’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:762:5: warning: ‘ENGINE_by_id’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      762 |     tls->engine = ENGINE_by_id(tls->engine_id);
          |     ^~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:9:
    /usr/include/openssl/engine.h:336:31: note: declared here
      336 | OSSL_DEPRECATEDIN_3_0 ENGINE *ENGINE_by_id(const char *id);
          |                               ^~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘add_certificate_to_store’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:951:24: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
      951 |             bio_method = BIO_s_mem();
          |                        ^
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘create_openssl_instance’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:1047:9: warning: ‘TLSv1_2_method’ is deprecated: Since OpenSSL 1.1.0 [-Wdeprecated-declarations]
     1047 |         method = TLSv1_2_method();
          |         ^~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:5:
    /usr/include/openssl/ssl.h:2024:50: note: declared here
     2024 | OSSL_DEPRECATEDIN_1_1_0 __owur const SSL_METHOD *TLSv1_2_method(void); /* TLSv1.2 */
          |                                                  ^~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:1051:9: warning: ‘TLSv1_1_method’ is deprecated: Since OpenSSL 1.1.0 [-Wdeprecated-declarations]
     1051 |         method = TLSv1_1_method();
          |         ^~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:5:
    /usr/include/openssl/ssl.h:2016:50: note: declared here
     2016 | OSSL_DEPRECATEDIN_1_1_0 __owur const SSL_METHOD *TLSv1_1_method(void); /* TLSv1.1 */
          |                                                  ^~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:1055:9: warning: ‘TLSv1_method’ is deprecated: Since OpenSSL 1.1.0 [-Wdeprecated-declarations]
     1055 |         method = TLSv1_method();
          |         ^~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:5:
    /usr/include/openssl/ssl.h:2008:50: note: declared here
     2008 | OSSL_DEPRECATEDIN_1_1_0 __owur const SSL_METHOD *TLSv1_method(void); /* TLSv1.0 */
          |                                                  ^~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘tlsio_openssl_init’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:1213:5: warning: ‘ERR_load_BIO_strings’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
     1213 |     ERR_load_BIO_strings();
          |     ^~~~~~~~~~~~~~~~~~~~
    In file included from /usr/include/openssl/cryptoerr.h:17,
                     from /usr/include/openssl/crypto.h:38,
                     from /usr/include/openssl/comp.h:22,
                     from /usr/include/openssl/ssl.h:28,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:5:
    /usr/include/openssl/cryptoerr_legacy.h:31:27: note: declared here
       31 | OSSL_DEPRECATEDIN_3_0 int ERR_load_BIO_strings(void);
          |                           ^~~~~~~~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘tlsio_openssl_deinit’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:1241:5: warning: ‘ERR_remove_thread_state’ is deprecated: Since OpenSSL 1.1.0 [-Wdeprecated-declarations]
     1241 |     ERR_remove_thread_state(NULL);
          |     ^~~~~~~~~~~~~~~~~~~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:6:
    /usr/include/openssl/err.h:473:30: note: declared here
      473 | OSSL_DEPRECATEDIN_1_1_0 void ERR_remove_thread_state(void *);
          |                              ^~~~~~~~~~~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c: In function ‘tlsio_openssl_setoption’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:1713:13: warning: ‘ENGINE_load_builtin_engines’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
     1713 |             ENGINE_load_builtin_engines();
          |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c:9:
    /usr/include/openssl/engine.h:358:28: note: declared here
      358 | OSSL_DEPRECATEDIN_3_0 void ENGINE_load_builtin_engines(void);
          |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~
    [ 52%] Building C object deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/x509_openssl.c.o
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c: In function ‘load_certificate_chain’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:81:28: error: invalid use of incomplete typedef ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
       81 |                 if (ssl_ctx->extra_certs != NULL)
          |                            ^~
    In file included from /usr/include/openssl/ssl.h:31,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/inc/azure_c_shared_utility/x509_openssl.h:7,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:4:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:83:45: error: invalid use of incomplete typedef ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
       83 |                     sk_X509_pop_free(ssl_ctx->extra_certs, X509_free);
          |                                             ^~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:84:28: error: invalid use of incomplete typedef ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
       84 |                     ssl_ctx->extra_certs = NULL;
          |                            ^~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c: In function ‘load_key_RSA’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:140:5: warning: ‘EVP_PKEY_get1_RSA’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      140 |     RSA* privatekey = EVP_PKEY_get1_RSA(evp_key);
          |     ^~~
    In file included from /usr/include/openssl/x509.h:29,
                     from /usr/include/openssl/ssl.h:31,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/inc/azure_c_shared_utility/x509_openssl.h:7,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:4:
    /usr/include/openssl/evp.h:1348:16: note: declared here
     1348 | struct rsa_st *EVP_PKEY_get1_RSA(EVP_PKEY *pkey);
          |                ^~~~~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:150:9: warning: ‘SSL_CTX_use_RSAPrivateKey’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      150 |         if (SSL_CTX_use_RSAPrivateKey(ssl_ctx, privatekey) != 1)
          |         ^~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/inc/azure_c_shared_utility/x509_openssl.h:7,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:4:
    /usr/include/openssl/ssl.h:1803:12: note: declared here
     1803 | __owur int SSL_CTX_use_RSAPrivateKey(SSL_CTX *ctx, RSA *rsa);
          |            ^~~~~~~~~~~~~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:162:9: warning: ‘RSA_free’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      162 |         RSA_free(privatekey);
          |         ^~~~~~~~
    In file included from /usr/include/openssl/x509.h:36,
                     from /usr/include/openssl/ssl.h:31,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/inc/azure_c_shared_utility/x509_openssl.h:7,
                     from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:4:
    /usr/include/openssl/rsa.h:293:28: note: declared here
      293 | OSSL_DEPRECATEDIN_3_0 void RSA_free(RSA *r);
          |                            ^~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c: In function ‘x509_openssl_add_engine_key’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:227:5: warning: ‘ENGINE_init’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      227 |     if (!ENGINE_init(engine))
          |     ^~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:14:
    /usr/include/openssl/engine.h:620:27: note: declared here
      620 | OSSL_DEPRECATEDIN_3_0 int ENGINE_init(ENGINE *e);
          |                           ^~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:234:9: warning: ‘ENGINE_set_default’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      234 |         if (!ENGINE_set_default(engine, ENGINE_METHOD_ALL))
          |         ^~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:14:
    /usr/include/openssl/engine.h:708:27: note: declared here
      708 | OSSL_DEPRECATEDIN_3_0 int ENGINE_set_default(ENGINE *e, unsigned int flags);
          |                           ^~~~~~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:241:13: warning: ‘ENGINE_load_private_key’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      241 |             EVP_PKEY* evp_key = ENGINE_load_private_key(engine, x509privatekey_id, NULL, NULL);
          |             ^~~~~~~~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:14:
    /usr/include/openssl/engine.h:638:11: note: declared here
      638 | EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id,
          |           ^~~~~~~~~~~~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:264:9: warning: ‘ENGINE_finish’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
      264 |         if (!ENGINE_finish(engine))  // Release functional reference.
          |         ^~
    In file included from /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:14:
    /usr/include/openssl/engine.h:628:27: note: declared here
      628 | OSSL_DEPRECATEDIN_3_0 int ENGINE_finish(ENGINE *e);
          |                           ^~~~~~~~~~~~~
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c: In function ‘x509_openssl_add_certificates’:
    /tmp/pip-req-build-n02l9k5x/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/x509_openssl.c:353:24: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
      353 |             bio_method = BIO_s_mem();
          |                        ^
    gmake[2]: *** [deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/build.make:804: deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/adapters/x509_openssl.c.o] Error 1
    gmake[1]: *** [CMakeFiles/Makefile2:943: deps/azure-c-shared-utility/CMakeFiles/aziotsharedutil.dir/all] Error 2
    gmake: *** [Makefile:146: all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-n02l9k5x/setup.py", line 343, in <module>
        setup(
      File "/usr/local/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib64/python3.10/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib64/python3.10/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python3.10/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.10/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib64/python3.10/distutils/command/install.py", line 579, in run
        self.run_command('build')
      File "/usr/lib64/python3.10/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib64/python3.10/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/lib64/python3.10/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib64/python3.10/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib64/python3.10/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/pip-req-build-n02l9k5x/setup.py", line 214, in run
        self.build_cmake(ext)
      File "/tmp/pip-req-build-n02l9k5x/setup.py", line 271, in build_cmake
        subprocess.check_call(joined_cmd, shell=True, universal_newlines=True, env=build_env)
      File "/usr/lib64/python3.10/subprocess.py", line 369, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command 'cmake --build . --config Release' returned non-zero exit status 2.
    Running setup.py install for uamqp ... error

If you're in a hurry and you want to reproduce the issue quickly from Fedora 36, just build this container using docker or buildah:
FROM registry.fedoraproject.org/fedora:rawhide
RUN dnf -y install cmake cython gcc-c++ findutils git make openssl-devel patch python3-devel python3-pip
RUN git clone https://github.com/Azure/azure-uamqp-python
# Apply a small patch to compile with GCC 11.x.
RUN curl -sL https://src.fedoraproject.org/rpms/python-uamqp/raw/rawhide/f/python-uamqp-1.2.14-gcc11.patch | patch -p0 -d azure-uamqp-python
# Don't treat warnings as errors.
RUN grep -rl "\-Werror" azure-uamqp-python/ | xargs sed -i 's/-Werror//'
RUN pip install --no-binary :all: azure-uamqp-python/

thanks for reaching out. openssl 3.0 is fairly new, we will take a look into it.

Thanks, @yunhaoling! 🎉

hey @major , apologize for the late update.
As you might know, uamqp is depending on a C library: https://github.com/Azure/azure-c-shared-utility and from what I learned it doesn't yet support OpenSSL3.0, discussion could be found here: https://github.com/Azure/azure-c-shared-utility/discussions/566.
I feel sorry that currently there's nothing I could with the underlying C code except waiting for the C team to support it -- What I can do is that I'll leave a comment in the discussion adding a data point for the openssl 3.0 support, hope that could help to some extent :).
May I ask whether you're using uamqp library alone to build application or you're using azure-eventhub, azure-servicebus instead?

@yunhaoling Thanks for the reply! I'm packaging azure-cli on Fedora and this is one of the requirements within azure-cli or one of the Python SDKs. In the meantime, I can link to OpenSSL 1.1 and that works.

Looking forward to openssl3 support

Red Hat Enterprise Linux (RHEL) 9 made the move to OpenSSL 3.0 as well and I'm unable to package uamqp there, either. That affects CentOS Stream 9, Rocky, Alma, etc.

azure-c-shared-utility merged support https://github.com/Azure/azure-c-shared-utility/pull/604
as did
azure-uamqp-python, https://github.com/Azure/azure-uamqp-c/pull/423

Hey,
uamqp-1.6.0 from pip still affected.

Hey, any update on this? still a problem for 1.6.3

A version bump would be useful so uamqp could be updated in the affected distributions.
openSUSE is now affected as well as it recently switched to OpenSSL 3.0.

Hi All,
sorry for the delay in response. I'm working on a few changes to bring about openSSL 3.0 support and am planning to release in the next cycle based on internal testing etc.
I was able to build the module successfully on Fedora rawhide using the following:
FROM fedora:rawhide
RUN dnf -y install cmake cython gcc-c++ findutils git make openssl-devel patch python3-devel python3-pip
RUN git clone --single-branch --branch fix_openssl3 https://github.com/kashifkhan/azure-uamqp-python.git
RUN pip install --no-binary :all: azure-uamqp-python/

Hey, any update on this? still a problem for uampq 1.6.3 on Linux Alpine

The same on Alpine 3.17 with python 3.10.10

Any news on if this is going to be merged in?

Still running into the issue on Fedora + OpenSSL 3.x + uamqp 1.6.4

@kashifkhan any thoughts on when this code will be published? I've been building from main successfully since the end of March and would appreciate not having to take the extra stop of building manually each time I have to deploy :-)

@ kashifkhan - just checking in to see what the plan is here, please

@toddbu sorry for the delay. We plan on doing a release at the end of this month with the changes that are currently in main.

@toddbu I have now released uamqp 1.6.5 which includes all the latest changes.

Thanks @kashifkhan. I went to go test this today but it appears that one of my teammates removed our dependency on this module. Sorry :-(

@toddbu no worries, just out of curiosity are you using uamqp directly or azure event hubs or service bus python libraries ?
same question for @hgyn59


Add a guidance of how to deploy Log Hub using other region's Cognito User Pool

Describe the issue
There are a couple regions where Cognito User Pool is missing. However, the Cognito User Pool is a standard OpenID Connect Provider. I need a guidance, how to use the Cognito User Pool in other regions to launch the Log Hub solution.
Links
https://awslabs.github.io/log-hub/en/implementation-guide/deployment/

Here is the guide. https://awslabs.github.io/log-hub/en/implementation-guide/deployment/with-oidc/#option-1-using-cognito-user-pool-from-another-region

这是指南。https://awslabs.github.io/log-hub/en/implementation-guide/deployment/with-oidc/#option-1-using-cognito-user-pool-from-another-region

this url access 404

We recently move the repo from awslabs to aws-solutions organization. Here is the new url https://aws-solutions.github.io/log-hub/en/implementation-guide/deployment/with-oidc/#option-1-using-cognito-user-pool-from-another-region


Consider local time formats when reading runtimes

I had a crash where my time format in the recon-surf was 'Di 16. Mai 17:36:34 CEST 2023', which is localized to german.
I'm not sure why that's the case, but it wouldn't hurt to have a fallback to try to read the local timeformat. The function dateutil.parser.parse does not consider local timeformats, but datetime.datetime.strptime does.
Edit: This in not a problem in docker, but when running the recon-all pipeline locally. With this fix in the stable branch the pipeline runs locally without error.

Looks fine with me. Before merging, I would suggest that @AhmedFaisal95 makes sure that this does not break anything inside the time extraction scripts.

Hi @ClePol. Thanks for the suggestion.
The locale check makes sense, but we can't use datetime.datetime.strptime because it fails in some cases where the recon-surf date-time string contains an extra empty space in a single-digit day sub-string, e.g. "#@# Intensity Normalization2 Tue Jan  3 06:30:17 UTC 2023" (instead of "#@# Intensity Normalization2 Tue Jan 17 HH:MM:SS UTC 2023").
This is why we switched from datetime.datetime.strptime to dateutil.parser.parse, which is more robust can handle this case, a while ago (see https://github.com/Deep-MI/FastSurfer/commit/073d01955640ec2f2c0a67e9aea21970fb28e28a).
I suggest that we find a different solution that still considers the local locale settings.
Let me know if you have any ideas.

Hi,
thanks for taking a look.
I implemented this as a fallback to use when the current method fails. So
to me it seems like an absolute improvement, to try a second function. A
straightforward fix for your failure case would be to detect the single
digit day substring and prepend a zero. Alternatively there could be a
second fallback. Unfortunately I don't have time to test this right now,
but I'll update if I run into this again during local testing.
On Tue, Jun 13, 2023, 10:09 Ahmed Faisal Abdelrahman <
@.***> wrote:

Hi @ClePol https://github.com/ClePol. Thanks for the suggestion.
The locale check makes sense, but we can't use datetime.datetime.strptime
because it fails in some cases where the recon-surf date-time string
contains an extra empty space in a single-digit day sub-string, e.g. "#@#
Intensity Normalization2 Tue Jan 3 06:30:17 UTC 2023" (instead of "#@#
Intensity Normalization2 Tue Jan 17 HH:MM:SS UTC 2023").
This is why we switched from datetime.datetime.strptime to
dateutil.parser.parse, which is more robust can handle this case, a while
ago (see 073d019
https://github.com/Deep-MI/FastSurfer/commit/073d01955640ec2f2c0a67e9aea21970fb28e28a
).
I suggest that we find a different solution that still considers the local
locale settings.
Let me know if you have any ideas.
—
Reply to this email directly, view it on GitHub
https://github.com/Deep-MI/FastSurfer/pull/302#issuecomment-1588772782,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNFF6ASFNF5ZDWCUENFCWDXLAN4FANCNFSM6AAAAAAYFES7KM
.
You are receiving this because you were mentioned.Message ID:
@.***>

will merge this for now as it is only a fallback. If any of you has an idea how to do this better, please do another PR.


Bug: In ACL : It is no longer possible to check that a reference from another table matches the current record

Hi,
Until now, to check that a reference from another table matches the current record we use the expression rec.id in permissions. It worked very well.
But recently it considered it as an error and the permission can't be changed or saved with that expression.
In this exemple we create a rule that say the user can only see the regions to which it is connected. The rule has been validated few weeks ago. But we recently found that this rule was considered invalid, even though it is taken into account in the data display for users.

Thanks for help

Fixed in this commit.


pkg/trace/api: add discovery endpoint

TODO:

[x] Add a test to ensure JSON keys never change regardless of field changes in the code.

Example response from http://localhost:8126/hello:
{
	"Version": "0.99.0",
	"GitCommit": "fab047e10",
	"GitBranch": "master",
	"BuildDate": "2020-12-04 15:57:06.74187 +0200 EET m=+0.029001792",
	"GoVersion": "1.15.6",
	"Endpoints": [
		"/v0.3/traces",
		"/v0.3/services",
		"/v0.4/traces",
		"/v0.4/services",
		"/v0.5/traces",
		"/v0.5/stats",
		"/profiling/v1/input"
	],
	"Features": [
		"nostats"
	],
	"Config": {
		"Enabled": true,
		"Hostname": "datadog-gbbr.local",
		"DefaultEnv": "gabriel.aszalos",
		"ConfigPath": "/opt/datadog-agent/etc/datadog.yaml",
		"Endpoints": [
			{
				"Host": "http://localhost:8888",
				"NoProxy": false
			}
		],
		"BucketInterval":<PHONE_NUMBER>0,
		"ExtraAggregators": null,
		"ExtraSampleRate": 1,
		"TargetTPS": 10,
		"MaxEPS": 200,
		"ReceiverHost": "localhost",
		"ReceiverPort": 8126,
		"ReceiverSocket": "",
		"ConnectionLimit": 0,
		"ReceiverTimeout": 20,
		"MaxRequestBytes": 52428800,
		"StatsWriter": {
			"ConnectionLimit": 0,
			"QueueSize": 0,
			"FlushPeriodSeconds": 0
		},
		"TraceWriter": {
			"ConnectionLimit": 0,
			"QueueSize": 0,
			"FlushPeriodSeconds": 0
		},
		"ConnectionResetInterval": 0,
		"StatsdHost": "localhost",
		"StatsdPort": 8125,
		"LogLevel": "debug",
		"LogFilePath": "/var/log/datadog/trace-agent.log",
		"LogThrottling": false,
		"MaxMemory": 0,
		"MaxCPU": 0,
		"WatchdogInterval":<PHONE_NUMBER>0,
		"ProxyURL": null,
		"SkipSSLValidation": false,
		"Ignore": {},
		"ReplaceTags": null,
		"AnalyzedRateByServiceLegacy": {},
		"AnalyzedSpansByService": {},
		"DDAgentBin": "/opt/datadog-agent/bin/agent/agent",
		"Obfuscation": null
	}
}

Hi @gbbr
What do values like those below mean? It looks to me like a feature flag hasn't been set. The client wouldn't expect these to be null but booleans. These would parse as false in some languages but I believe they are actually set by default in the trace agent.
"Obfuscation": null,
"ReplaceTags": null

This one looks like an actually nullable value, rather than the consequence of an unset feature flag, and I'd argue complete absence is a better policy in this case.
"ProxyURL": null

@richardstartin first off, nothing that is in the Config is a feature flag. A "feature flag" is simply an experimental option that may be enabled or not (and will eventually be removed in subsequent versions) currently showing up in FeatureFlags section.
Obfuscation and ReplaceTags are both objects, not booleans. I have made a new commit to leave this out from the response when they are null. They specify the configuration for the obfuscator, basically the contents of that section in the datadog.yaml file (see this link. As for the tag replacer, see this link. Most of what's in the Config key is from that file...

If it helps, here's a fully populated object from the tests: https://github.com/DataDog/datadog-agent/blob/c16442c8acbd858bd47bc40c3aef451e6aa6866b/pkg/trace/api/api_test.go#L687-L843

Removed ProxyURL, probably not interesting at all to a user...

@dougqh @richardstartin I took all of your feedback, and made the Go code uglier in order to get prettier JSON. I also manually selected which config options should show up, and removed any which seemed like they might contain sensitive information or be irrelevant to the user. Furthermore, I made the obfuscation configurations as booleans as requested.
Please check the updated PR description and let me know your thoughts.


first pass on merging eslint rules

Closes: RHOAIENG-2534
Description

Added frontend eslint  and tsconfig rules to the backend, as well as package.json packages.
You can see the added rules here
How Has This Been Tested?

cd backend && npm run test
npm run build
Test Impact

I have not implemented functionality, just refactored code.
Request review criteria:

Self checklist (all need to be checked):

[ ] The developer has manually tested the changes and verified that the changes work (ran npm run test)
[x] Commits have been squashed into descriptive, self-contained units of work (e.g. 'WIP' and 'Implements feedback' style messages have been removed)
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

[ ] Included any necessary screenshots or gifs if it was a UI change.
[ ] Included tags to the UX team if it was a UI/UX change (find relevant UX in the SMEs section).

After the PR is posted & before it merges:

[ ] The developer has tested their solution on a cluster by using the image produced by the PR to main

/retest

/lgtm
/approve
did a bunch of ad hoc testing of various parts of the UI


[Completion Regression] text_edit_text returns None constantly

Not sure when the regression happened but this column check:
            if edit_range.start.col <= word_col:
                # if edit starts at current word, we can use it.
                # if edit starts before current word, use the whole thing and we'll fix it up later.
                return edit_text

here returns always False now. Which leads the completion to fallback to label instead of using textEdit. As you can see bellow:

Logs with some debug variables:
item: {'textEdit': {'newText': 'apply($0)', 'range': {'end': {'line': 3, 'character': 7}, 'start': {'line': 3, 'character': 7}}}, 'label': 'apply[A](xs: A*): List[A]', 'sortText': '00000', 'preselect': True, 'insertTextFormat': 2, 'filterText': 'apply', 'data': {'symbol': 'scala/collection/immutable/List.apply().', 'target': 'file:/home/ayoub/workspace/testproject/?id=root'}, 'kind': 2}

LSP start: {'line': 3, 'character': 7}
_last_row: 3, last_col: 6
LSP: textEdit from col 7, 7 applied at col 6
edit_range.start.col: 7, word_col: 6

For me the condition here is incomplete and doesn't handle the case when the completion happens after last word beginning. In this case a .
I think we are safe to remove the condition.
I can make a PR but wanted first to confirm the issue with @tomv564

OS and language server: Ubuntu with Metals
How you installed LSP: Master
prefer_label_over_filter_text is set to True

Thanks for reporting. I think word_col is incorrectly provided to format_completion by plugin/completion.py
The logic there looks up the start of the current word being completed. In your example sublime tells us that the . is a word - we should probably only use the start-of-word lookup if the cursor is at a word, see is_at_word.
Can have a look at a fix for this soon, or feel free to open a PR.

we should probably only use the start-of-word lookup if the cursor is at a word

You mean in the text_edit_text function ?

No, the needed check was in CompletionHandler's handle_response method - see the merged PR above.


SyntaxError: Unexpected token 'export'

When I run next-js project with simple setup, I get this error.
error - /home/chinmay/Projects/next-useauth-starter/node_modules/react-use-auth/auth0.js:4
export { Auth0 } from "./dist/auth0";
^^^^^^

SyntaxError: Unexpected token 'export'

My code:
__app.js
return (
    <>
      <AuthConfig
        navigate={(url) => router.push(url)}
        authProvider={Auth0}
        params={{
          domain: "useauth.auth0.com",
          clientID: "GjWNFNOHqlino7lQNjBwEywalaYtbIzh",
        }}
      />

      <Component {...pageProps} />
    </>
  );

@jasonLaster can you confirm if you are facing this too?

Can you share your repo?
What version of Node are you on?

node v12.16.3. Looks like it also repros with v14.15.3

also experiencing the same error when upgrading from pre-1.0.0 here is a link to a reproduction repo running node 14.2.0

Same issue running node v12.19.0. Any updates here?

This seems to be a workaround: import { Auth0 } from 'react-use-auth/dist/auth0'

Thanks to @jasonLaster for the fix ❤
Just published v2.0.3, you should be good now @catrawalkar @d-sanderson @benheb. Closing for now

I also ran into this issue with Netlify Identity. I used @benheb's solution to get it to work
import { NetlifyIdentity } from "react-use-auth/netlify-identity";
// Change to:
import { NetlifyIdentity } from "react-use-auth/dist/NetlifyIdentity";


feat: Add support to configure Plugin Agent Retry Backoff Policy.

Signed-off-by: sid8489<EMAIL_ADDRESS>
Motivation
Plugin agent Pod main container makes http call to plugin containers. That Http call is auto retried for transient errors using a Retry Backoff Policy. Currently the Retry Backoff Policy is hardcoded to default values. These default values might not be fit for certain use cases (eg: My Plugin Container is a python http server, but I am downloading the source code at run time, so this can take some seconds and default retry policy is too aggressive for this). So proposing to make it configurable .
Modifications

Modified Retry Backoff Policy used by Plugin agent to lookup for environment variables and configure correspondingly.
Added support to supply Env Variable to Extended Agent Pod Init and Main  containers  through Executor Config of Workflow Controller

Verification
TODO: If these changes are acceptable, I will add the verification details.

Initially

Can we reuse existing retry environment variables?

Initially I thought of using existing variables only. But then it couples the retry policy of executor init container and plugin container. One might want to just configure  retry policy  of the plugin executor container or just of the executor init container. Let me know your thoughts, IMO plugin retry policy should be allowed to configured independently.

Let's reuse existing env vars for now.

Let's reuse existing env vars for now.

Have made the change.

Could you fix the checks?

Closing since this has been inactive for several months. Feel free to re-open if you pick it up again.


MAINT: uninitialised var if basinhopping niter=0. closes #5915

See #5915 for more information.

@@            master   #5919   diff @@
======================================
  Files          238     238
  Stmts        43653   43653
  Branches      8194    8194
  Methods          0       0
======================================
- Hit          34077   34075     -2
- Partial       2604    2605     +1
- Missed        6972    6973     +1

Review entire Coverage Diff as of a5ff445

Powered by Codecov. Updated on successful CI builds.