Product, Brand and Description missing

probably related to the changes here:
https://github.com/stampzilla/gozwave/blob/master/nodes/node.go#L192
My device looks like this. The name should default to brand - product
{
  "type": "lamp",
  "node": "9e774f59-d47d-4d40-9553-4b6248cdd768",
  "id": "1009",
  "name": " -  (Address: 1009)",
  "online": true,
  "state": {
    "on": null
  },
  "tags": null
}

Might be fixed by #28. Device probably isn't in the old database. Look for your device here and if it's listed it will probably show up when the PR is merged.

The devices are in the old database. But when we implemented the command
class discovery for devices not in the database something broke and I
haven't had time to look into it. This issue was kinda reminder to myself
to look into it.
On Wed, May 24, 2017, 01:18 Luke Scott<EMAIL_ADDRESS>wrote:

Might be fixed by #28 https://github.com/stampzilla/gozwave/pull/28.
Device probably isn't in the old database. Look for your device here
http://www.cd-jackson.com/index.php/zwave/zwave-device-database/zwave-device-list
and if it's listed it will probably show up when the PR is merged.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/stampzilla/gozwave/issues/26#issuecomment-303564548,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABF-FeBkxvb3hom_OFgTQmLVaTraqLXPks5r82lZgaJpZM4MwMka
.


Reduce restrictions on which metrics may be named utilization

Related to https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/24905
Changes
We have 2 new sets of metrics we'd like to introduce in the kubeletstats receiver in the collector around usage of memory and memory with respect to configured resource requests and limits. During our discussion we found that resource limits fit nicely into the semantic convention's definition for limit and therefore a metric with respect to resources.limits fit nicely into the naming convention for utilization.  But when considering a metric with respect to resources.requests, things broke down. The "limit" defined by resources.requests in Kubernetes is not a fixed limit but a soft limit, and a metric with respect to resources.requests needs to go above 1. This metric represents a "soft" utilization, one where the most interesting and important results are when the usage is exceeding the limit.
This PR reduces the restrictions on what types of metrics should use utilization in the name. It loosens the restriction of utilization from a fixed limit, restricted between 0 and 1, to a concept of a limit, with the only restriction being that values be represented as a ratio.
This change allows us to introduce metrics like

k8s.pod.cpu.limits.utilization
k8s.pod.cpu.requests.utilization
k8s.pod.memory.limits.utilization
k8s.pod.memory.requests.utilization
k8s.container.cpu.limits.utilization
k8s.container.cpu.requests.utilization
k8s.container.memory.limits.utilization
k8s.container.memory.requests.utilization

which represent the cpu/memory usage of pods and containers with respect to configures requests and limits (see the linked issue for details on how we calculate these values for pods)
Merge requirement checklist

[ ] CONTRIBUTING.md guidelines followed.
[ ] CHANGELOG.md updated for non-trivial changes.
[ ] schema-next.yaml updated with changes to existing conventions.

/cc @dmitryax @jinja2

I think this one is worth talking about broadly, but looking into the details and understanding the need, I agree with the decision here, and think this does belong as a utilization metric.  I need to look more at the implications of lifting this restriction.
Would love more input from the general @open-telemetry/specs-semconv-approvers

I also read the thread in the collector and it does seem to align well with our utilization term. I wonder though if lifting the constraint will be a cause of problems for existing alerts/dashboards. I think it should be fine, as of today these were "hard" limits and they won't go above 1 anyway. Only the metrics for the k8s scenarios as described would go over, but since they are new all is good.

Are there more discussions to be had around this change?  Should I bring it to the semantic convention SIG meeting?

Should I bring it to the semantic convention SIG meeting

@TylerHelmuth I think that would be good, to raise awareness!


[Feature request]: Hero teaser with paid content chip (interim solution)

Preflight Checklist

[X] I have read the Contributing Guidelines for this project.
[X] I agree to follow the Code of Conduct that this project adheres to.
[X] I have searched the issue tracker for an issue that matches the one I want to file, without success.

Request type
Request for enhancement of a component
It affects the following packages
Components
Functionality
Hero Teaser needs a new variant.
The interim solution is the hero teaser as it is today, plus paid content chip. The final solution includes 1 or 2 partner logos floating over the image.
Link to design proposal file
https://www.figma.com/file/FwQzAhDa5jjtb3boC16GYi/Marketing%2FMonetarisierung?type=design&node-id=9749%3A16590&mode=design&t=33JuW6rnPuQdKhO0-1

figma component: https://www.figma.com/design/9r6xSfNmEfCFxl1yFYedrj/Lyne-Components?node-id=30086-43164&t=CJIkbe4AMsoNsaWp-11

Closed by #2949


Blazepose Javascript solution takes 100% CPU utilization.

System information

I am using the example code provided on mediapipe's github
Running the JS solution for pose estimation
MediaPipe version: all versions have this issue
Bazel version: all versions have this issue
Solution: Pose
Programming Language and version : JavaScript - latest

Describe the expected behavior:
When the model runs on a browser it is using 100% CPU. Due to this other services are getting choked. Is there a way to reduce the amount of CPU it is utilizing and making it lighter? I have even used modelComplexity in different modes as well but it hasn't been very useful.
These are the options I am using:
pose.setOptions({
staticimagemode: false,
modelComplexity: 0,
selfieMode: true,
smoothLandmarks: true,
enableSegmentation: false,
smoothSegmentation: false,
minDetectionConfidence: 0.5,
minTrackingConfidence: 0.5
});

These are the pose options I am using.
pose.setOptions({
staticimagemode: false,
modelComplexity: 0,
selfieMode: true,
smoothLandmarks: true,
enableSegmentation: false,
smoothSegmentation: false,
minDetectionConfidence: 0.5,
minTrackingConfidence: 0.5
});

Hello @prajwal-v-athreya,
We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.
You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you


How to use sass-resources-loader with compass and susy ?

I was using since now sass-loader to compile scss powered by compass and susy.
However my setup has changed now and i have to use sass-resources-loader. It's works perfectly but i am running into some difficulties to get compass and susy to be working. I get this error :
(Emitted value instead of an instance of Error) postcss-import:
.../styles/scss/nav.scss:3:1: Cannot find module '~susy/sass/susy'
 from '.../styles/scss'
Here my former situation
    {test: /\.scss$/,
 loader: "style-loader!raw-loader!sass-loader?includePaths[]=" + path.resolve(__dirname, "./node_modules/compass-mixins/lib") + "&includePaths[]=" + path.resolve(__dirname, "./mixins/app_mixins")},

And my current situation
{
        test: /\.s(a|c)ss$/,
        use: ['babel-loader', 'raw-loader', 'postcss-loader',
          { loader: 'sass-resources-loader',
            options: {
              resources: ['./styles/scss/general.scss','./styles/scss/nav.scss']

            }
          }
        ]
      }

Any suggestion ?

@jackmatrix We'll need you to debug this as we don't use this setup. Once you resolve this, please open a doc PR.

Closing due to inactivity. Please request to reopen if you like.


Out of Memory during Cal Calibration

I was attaching an external Compass and did several Mag Calibrations. Three Side Calibration did not finish (all Icons Green, text ok, but Progress bar did not reach end and OK Button was not shown). I then changed to 6 Side Calibration and tried again. This time I got the following Error:
Start the individual calibration steps by clicking one of the buttons above.
[cal] calibration started: 2 mag
[cal] Skipped scale calibration, sensor 1
[cal] ERROR: out of memory
[cal] calibration failed: mag

QGC 2.9.7b - 60 / Fimware shown in QGC is 1.6.4dev (Master from today).
Despite this disconnecting a Mag and redoing Calibration did not reset the MagID. Without manualle erasing all Mag Infos it was not possible to identify the newly externally attached mag. Missing Mag was also not shown in the Sensors in System Summary Screen, Mags shown with green Indication despite that I had disconnected the external Mag.

This PR should help with that:
https://github.com/PX4/Firmware/pull/4687

Fixes are on master. Can you retry?

I'll try. Not sure I can redo it....

Hi @LorenzMeier, I am having the exact same issue on QGC. Seems to have occurred when I upgraded Pixracer firmware to 1.4.1 as it worked ok before this. Am trying to replicate the Caipriniha VTOL. Any suggestions as to why this is happening?
[cal] calibration started: 2 mag
[cal] Skipped scale calibration, sensor 1
[cal] ERROR: out of memory
[cal] calibration failed: mag

@silkstone Are you sure your board is getting enough power? Otherwise please re-flash v1.4.2 via QGC. Release notes are here: https://github.com/PX4/Firmware/releases/tag/v1.4.2


Can't edit Theme

Operating system or device - Godot version:
Arch Linux - Godot version May 12, 2016 (48dc4d92a8f3c9eb42456eda2433c6fcaff0c723)
Issue description (what happened, and what was expected):
When choose edit Theme Godot will freeze.
Steps to reproduce:

Add node that inherit from control (from me is Button)
Inspector->Theme->Edit
Godot freeze.

Link to minimal example project (optional but very welcome):
None

I get a crash when I try this. But this is actually a problem with recent GCC as reported in #4623. I don't have the problem with Clang. So I'll archive this as duplicate of #4623, as it's the same problem.
For the time being, I recommend building Godot with Clang (add CC="clang" CXX="clang++" to the SCons command line).


Support for a logging framework other than Log4Net

At the moment, based on the README logging on .NET Core only supports log4net.
Based on the github page for log4net:

Apache Log4net was voted to dormant status on April 1, 2020. Dormant status
indicates that the project is not being actively maintained, that no future
releases are planned and, most likely, there are few or no developers left
in the Apache Logging project familiar with this code base.

Given this is no longer being maintained, can this be moved to a more generic logging framework?
Can I suggest finding some way to support a generic ILogger from Microsoft.Extensions.Logging? Even if this means implementing a logger rather than having a logger injected into it, just feels like a nicer long-term option.

Hi @doug-ferris,
Thanks for the feedback.
This request has already been put in our backlog, which will be investigated soon. Please stay tuned and come back later to check.

I was going to create an issue regarding the use of a custom internal ILogger in the XRay middleware:

I was really surprised when I found out you were not using the native interface from Microsoft.Extensions.Logging in there.
Log4net is a very legacy logging library in the .Net ecosystem at this point and would be a poor choice if picking one library was even needed. The fact that it is a static instance is also really bad for any integrations. You could just inject ILogger<T> in the middleware constructor instead.
Do you have any updates on this @lupengamzn ?


chore(devops): Update signer release version

Motivation
We introduced breaking changes and new endpoints in CFS.
In this PR, I updated the release version of the signer canister which in turn updates the declaration files.
I also fixed the breaking changes with minimal edits. Proper error management will be added in subsequent PRs.
Changes
Manual changes

Change signer release to 0.2.1.
Fix breaking changes in the declarations in canisters/signer.canister.ts.

Automatic changes
All changes within src/declarations/signer/* were created by executing npm run generate after running ./scripts/build.signer.sh with the new release.
Tests
I deployed the signer to the local replica with ./scrtips/deploy.signer.sh using --mode reinstall because I already had one signer installed and I had to force the reinstallation.
After reinstallation I was able to run Oisy locally.

@AntonioVentilii-DFINITY @peterpeterparker please review
@DenysKarmazynDFINITY this PR includes the changes after the breaking changes to the signer canister. You'd need to redeploy the signer canister to have the latest version.

LGTM!
Do you think we should already release a version with this, since it contains adaptation to breaking changes?

The breaking changes of the CFS have not been deployed yet to mainnet. They are only used when btc flat is enable, so it doesn't matter.
This doesn't stop us nor puts pressure on a next release. But we shouldn't wait too long before we deploy the new CFS to prod and also Oisy. I'll coordinate with Max.


Leaflet.js - Draw polyline to nearest point from a location

I need to draw a polyline between two points using the shortest distance.
Where I am - see fiddle here: http://jsfiddle.net/Vsq4D/1/
The problem is that when I draw the line it does use the shortest distance. In the above example it draws from the US to China with the line going to the right, the longest way round the Earth, not the left, which is the shortest way.
Any ideas how to draw the line according to the shortest distance?
Looks like this is a issue in leaflet ?
Thanks

Hi there @phpmind, the issues page is usually for bug reports or discussions of core features, your question would be better answered in Stack Overflow or similar.
To give you a push in the right direction: The type of route you are looking for is usually called a Great Circle Route, and there are some add-ons that can help you with that, like Leaflet.Geodesic

As @luiscarloscb , this tracker is only for bugs, so I'll close this.
Also see https://stackoverflow.com/questions/41680276/leaflet-api-issue-plotting-route-between-long-distances and http://stackoverflow.com/questions/40532496/wrapping-lines-polygons-across-the-antimeridian-in-leaflet-js .


Account for renamed UM binary

Update manager has been renamed to kanto-update-manager. Set the log file path accordingly  to account for the new name.

Closing until rename is finalized


Uncaught SyntaxError:  Unexpected token '<' (http://localhost:8080/[object Module]:1:1)

Receiving error when opening pdfviewer.  This only happens on the first call.  Subsequent calls does not receive this error.  Below is the sample code.
`pdfReport.generate(); // Generate report
var is = new ByteArrayInputStream(pdfReport.getOutputStream().toByteArray());
resource = new StreamResource(filing.getTaxReport().getCode(), () -> is);
resource.setContentType("application/pdf");
resource.setCacheTime(0);`

Hello @allenvpn312, what version of the component are you using?

Hi, I am using version 16.4.1.
On Fri, Jan 21, 2022 at 5:29 AM Paola De Bartolo @.***>
wrote:

Hello @allenvpn312 https://github.com/allenvpn312, what version of the
component are you using?
—
Reply to this email directly, view it on GitHub
https://github.com/vaadin-component-factory/vcf-pdf-viewer-flow/issues/6#issuecomment-1018506561,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACIXSWJOMLFVT356WAHL2NTUXFNVLANCNFSM5KZNYBRQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***
com>

--
Regards,
Allen

My question was about which version of vcf-pdf-viewer-flow are you using, is it 2.0.0 (for Vaadin 21) or 1.0.1 (for Vaadin 14)? Thnaks.

Oh, sorry.  I’m using Vaadin 14 so it’s version 1.0.1.
On Fri, Jan 21, 2022 at 11:17 AM Paola De Bartolo @.***>
wrote:

My question was about which version of vcf-pdf-viewer-flow are you using,
is it 2.0.0 (for Vaadin 21) or 1.0.1 (for Vaadin 14)? Thanks.
—
Reply to this email directly, view it on GitHub
https://github.com/vaadin-component-factory/vcf-pdf-viewer-flow/issues/6#issuecomment-1018789752,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACIXSWJ3O6EQYGDCD6OMJZDUXGWK3ANCNFSM5KZNYBRQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***
com>

--
Regards,
Allen

Hi again, I wasn't able to reproduce your issue. There's a new version out, see https://vaadin.com/directory/component/pdf-viewer/1.0.2. Can you tell me if the issue remains with this new version? If so, can you share a small example so I can reproduce the issue?
Regards

Ok, I apply the updated version and let you know. Thanks
On Fri, Jan 28, 2022 at 7:17 AM Paola De Bartolo @.***>
wrote:

Hi again, I wasn't able to reproduce your issue. There's a new version
out, see https://vaadin.com/directory/component/pdf-viewer/1.0.2. Can you
tell me if the issue remains with this new version? If so, can you share a
small example so I can reproduce the issue?
Regards
—
Reply to this email directly, view it on GitHub
https://github.com/vaadin-component-factory/vcf-pdf-viewer-flow/issues/6#issuecomment-1024318215,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACIXSWPAP5MYTBRCJJ62VBLUYKXPHANCNFSM5KZNYBRQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***
com>

--
Regards,
Allen


locate dataset

https://www.kaggle.com/datasets/sripaadsrinivasan/kickstarter-campaigns-dataset

Organize the data. Remove data that we dont need.

Seed the data into the database.


Enable passing more compile options to ctest.

See https://github.com/ecmwf-ifs/ectrans/issues/79.

Is this okay to merge?

Hi @DJDavies2, sorry I had not noticed this PR before. Thanks for the reminder.
This looks perfectly fine to me. Thanks!


feat: support user-defined type guards as return type

This PR adds support for user-defined type guards as return type. Fixes #207
It depends on deno_doc v0.6.0+, so it doesn't work with the current latest version of Deno (1.11.2).
To see it working locally, you have to specify the path to the canary or a binary you built on your own from the latest source. In my case, I changed api/docs.ts L27-32 to:
  const proc = Deno.run({
    cmd: ["/Users/yusuktan/Repos/github.com/magurotuna/deno/target/release/deno", "doc", sourceFile, "--json", "--reload"],
    stdout: "piped",
    stderr: "piped",
    env: { "DENO_DIR": "/tmp/denodir" },
  });

then I ran vercel dev and attempted to generate doc for the following snippet.
export function f1(val1: A | B): val1 is A {}
export function f2(val2: any): asserts val2 is string {}
export function f3(val3: any): asserts val3 {}
export function assertIsDefined<T>(val4: T): asserts val4 is NonNullable<T> {
  if (val === undefined || val === null) {
    throw new AssertionError(
      `Expected 'val' to be defined, but received ${val}`
    );
  }
}
export class C {
  isSomething(): this is Something {
    return this instanceof Something;
  }
}

export type A = string | number;

Finally, the generated doc looks like:

Ref: https://github.com/denoland/deno_doc/pull/105

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Probably this is ready because of #204


How to leverage this network to solve video stream?

Excellent work! But I just wonder how can I use this network to process video stream so that I can get a similar result, just as shown in your demo.

Thanks, you may refer to https://github.com/antabangun/coex, In this repo, you can find demo.py.2023-11-06 10:03:32>"小艾同学" 写道：
Excellent work! But I just wonder how can I use this network to process video stream so that I can get a similar result, just as shown in your demo.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

Thank you!


Quality control for hifi and hic data

For hifi data, I only run CSS software with reads QV value great 0.99 parameter，but HiC data is not filtered. I don't know if this processing of raw data will have an impact on the assembly result ? or what do you recommend for HIC and hifi data quality control ？ thanks~

@shilpagarg I am so sorry to disturb you,could you help me ? thanks~


Update doc for how to add another API and Controller in one operator.

Description of the change:
Motivation for the change:

@gyliu513 Thanks for suggesting this change, however in the interest of keeping the quick start concise I think it's better if we don't add another section just to point this out.
The getting-started guide is a better place for this but even there, to me it's fairly obvious that you can define multiple APIs and controllers.
Going to close this for now, although feel free to suggest improvements to the getting-started guide.

Hi @gyliu513,
Yes. It would make more sense in the Getting Started. Could you please do a PR there with? Also, please use the example context to illustrate the possibility.


feat: output full content for rss feed

I have tested on examples/theme-leerob of this repo and anything works well.
The post at examples/theme-leerob/docs/posts/hello-world.md:
---
title: "Hello World"
titleImage: "https://img.alicdn.com/imgextra/i2/O1CN01Q93IFG1zkvJLH40c6_!!6000000006753-2-tps-2560-1440.png"
titleImageCaption: "介绍"
publishedAt: "2023/09/24"
---

Happy to see you here! This is a demo post.

RSS feed:
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <id>https://your-site.com/</id>
    <title>Docaid</title>
    <updated>2023-02-13T06:53:29.591Z</updated>
    <generator>https://github.com/jpmonette/feed</generator>
    <author>
        <name>Your Name</name>
<EMAIL_ADDRESS>        <uri>https://your-site.com/</uri>
    </author>
    <link rel="alternate" href="https://your-site.com/"/>
    <link rel="self" href="https://your-site.com/rss.xml"/>
    <subtitle>Docaid</subtitle>
    <logo>https://img.alicdn.com/imgextra/i1/O1CN01FwDVoc1YGhgE28l5B_!!6000000003032-2-tps-200-200.png</logo>
    <icon>https://img.alicdn.com/imgextra/i1/O1CN01FwDVoc1YGhgE28l5B_!!6000000003032-2-tps-200-200.png</icon>
    <rights>Docaid</rights>
    <entry>
        <title type="html"><![CDATA[Hello World]]></title>
        <id>https://your-site.com/posts/hello-world</id>
        <link href="https://your-site.com/posts/hello-world"/>
        <updated>2023-09-23T16:00:00.000Z</updated>
        <![CDATA[Hello World]]>
        <content type="html"><![CDATA[
Happy to see you here! This is a demo post.

]]></content>
    </entry>
</feed>

Reeder 5 mac app:

Just found a bit problem with it, it doesn't respect the content field in transform options. I will fix it later.

It looks some misleading in your transform options:
The content field is the source content of Markdown here:
https://github.com/sorrycc/docaid/blob/78bf26e1f5f8ad9d0bf9b5e8a21600b7fb108f21/docaid/src/utils/parseDocs.ts#L93-L104
But you use it to transform to HTML content here:
https://github.com/sorrycc/weekly/blob/54494dc297e22503301318b8ceb8a8d2f763cebe/.umirc.ts#L74-L84
We should make it consistent, can the transform option modify the source content of the Markdown file? Or modify output only? (In this case, we should use the html field instead of the content field.)


MacOS: Using vkCmdCopyImageToBuffer for capturing the backbuffer on the first frame returns an empty, red texture

Hi,
(problem potentially related to: #156)?
If using vkCmdCopyImageToBuffer to capture the backbuffer on the first frame on MacOS an empty red texture is returned.
I attached a modified version of the cube demo in the other issue I posted alongside this one (added as related) to demonstrate the problem:
The method for taking the screenshot works similarly to Sascha Willems' screenshot example, except instead of setting up to call vkCmdCopyImage/vkCmdBlitImage you do the following:

Make a VkBuffer using  the VK_BUFFER_USAGE_TRANSFER_DST_BIT usage flag.
Record and submit a command buffer that calls vkCmdCopyImageToBuffer.
Get the written-to-buffer's mapped memory after the submission of the command buffer.

Again, the file attached in the other issue has defines starting at line 59 to hopefully make testing/reproducing easier. =)
After that, I seem to capture the previous frame. But on Windows I always capture the current frame.

@floorman
Please see my comments on issue #156.


Riemann sink fixes

Some fixes for riemann sink:

Workarounds issue with int64 reflection in goryman
Reuses code for metrics appending
Avoid setting client to nil while sending events as it causes fatal errors

I signed it!

LGTM, just a nit.
Thank you for the contribution!

@piosz time type implements String(), so I think it's fine with %s, this is how it appears in logs:
Exported 4001 events to riemann in 162.355751ms

Thanks! :)


Create and test script to convert from Bitcoin Core to Bitcoin XT

Since BitcoinXT and Bitcoin Core are so similar in operation until a big block fork, in theory the database schema should be compatible.
I do not have a Bitcoin Core instance to test with, but it would be a good idea to test the following procedure
Stop bitcoind
Replace bitcoin binaries with bitcoinxt binaries
Restart daemon
Let BitcoinXT pick up where Bitcoin Core left off
If this does work, it might be good to write a good script to accomplish this in an automated way that can be executed like
wget https://raw.githubusercontent.com/bitcoinxt/bitcoinxt/master/upgrade_from_core.sh | sh
or
wget https://bitcoinxt.software/upgrade | sh
If we make it so a single line can convert Bitcoin Core to Bitcoin XT in a few moments, we might see an increase of people switching implementations.
If there is support for this concept, I can put together the upgrade script and test it.

Is there a reason we don't / can't allow users the option to just install bitcoinxt onto an existing bitcoin installation?

I just tried this, and was able to. But it needed to reindex the whole blockchain everytime.

Outdated in specifics.  With the UTXO database update soon we'll be database compatible with the other cash implementations, aside from not supporting obfuscation.  Compatibility with the BTC chain will only be maintained while it's trivial to do so.


[Fix] Miscellaneous bug fixes

changes:

removed "Forget Password" button

feature not planned to be implemented

disabled resize option for collab page textarea components

bottom right corner no longer has the resize icon/option

handle Piston's rate limiter Requests limited to 1 per 200ms error

display proper error message instead of "undefined 'run' error"
this is tested and confirmed by triggering the rate limit by sending multiple requests in a parallel payload

  const requestPayload = {
  language: language,
  version: langVer[language],
  files: [{ content: code }],
};

const [response1, response2] = await Promise.all([
  API.post("/execute", requestPayload),
  API.post("/execute", requestPayload),
]);

LGTM 👍 Thanks for implementing the bug fixes and keeping our coding standards high! 💯
ps: would be great if its possible to center the login button

centered & stretched the button width to fit the other fields' width !


Quick open is much slower

Issue Type: Bug
When I use the quick open/command palette to open a file (ie -- by typing the file name directly into the command pallette), the results render much slower than in previous versions.  It used to be the case that I would see filenames as I typed, with results refreshing immediately.  Following the latest monthly update, this is not the case.  Often after I finish typing I can count several seconds before seeing the file name results!
I have the following setting turned off:
"search.quickOpen.includeSymbols": false,

Not sure whether or not this issue is a duplicate or not; see also the issue below.  It seems that it might refer to something else:
https://github.com/Microsoft/vscode/issues/60419
VS Code version: Code 1.29.1 (bc24f98b5f70467bc689abf41cc5550ca637088e, 2018-11-15T19:06:21.742Z)
OS version: Darwin x64 17.7.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (8 x 2800)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (0.18GB free)

Process Argv
-psn_0_495737

Screen Reader
no

VM
0%

Extensions (19)

Extension
Author (truncated)
Version

language-x86-64-assembly
13x
2.2.11

ActiveFileInStatusBar
Ros
1.0.3

better-toml
bun
0.3.2

solargraph
cas
0.17.6

gitignore
cod
0.6.0

gitlens
eam
8.5.6

remotehub
eam
0.2.0

nasc-touchbar
fel
1.11.0

vscode-pull-request-github
Git
0.2.3

jsonnet
hep
0.1.0

git-tree-compare
let
1.4.0

python
ms-
2018.10.1

cpptools
ms-
0.20.1

csharp
ms-
1.17.1

Go
ms-
0.7.0

vscode-docker
Pet
0.4.0

ruby
reb
0.20.0

vscode-open-in-github
sys
1.10.0

vscode-proto3
zxh
0.2.2

(2 theme extensions excluded)

looks to be related:
https://github.com/Microsoft/vscode/issues/63857
https://github.com/Microsoft/vscode/issues/63840

Can you try downloading the Insiders' build and tell me whether it's any better there now? https://code.visualstudio.com/insiders

Nope -- it even took so long to show file results that I was able to grab a screenshot (not unusual on the new build):

However, one interesting observation -- after waiting for the results to finally load, I can edit the text in the pallet and the file results update nearly instantly.  I can even delete all the text and type a new string (results, still updating quickly as I type).  I just can't leave the command pallet and go back.  Once I do that multi-second waiting happens again!

Is this a very large workspace? Do you see a process named rg running when the progress bar is running?
Is it possible that you have lots of symlinks in your workspace, or search.useIgnoreFiles disabled, or something like that?

It is a large workspace, but I do have search.userIgnoreFiles enabled.
I would tell you about rg except that I can't seem to be able to see the process table while this is happening!  ps hangs and both top and the builtin vscode process explorer stops updating!  As soon as the results finish populating, ps returns accurately and top/vscode process explorer return to normal.  I have literally never seen this before and I have no idea whats going on.  I don't know enough about how on OS X these process read the process table in order to even speculate about this.

Same problem here, before 1.29 the results showed up pretty much instantly without a noticeable delay. Since 1.29 it takes 20-30 seconds for any results to show up when starting Code for the first time after a system reboot (Windows 10). After that it is usually quite fast again but sometimes still with a longer delay that I can't recall noticing before 1.29.
The workspace contains ~1.500 C# files and is stored on an M.2 SSD.
No settings have been changed by me between 1.29 and the previous version.
I can provide further info if needed but am not familiar with the internal workings and debug tools of VS Code, so detailed instructions would be required.

I see the exact same behavior when running rg directly -- the process table stops updating, and it takes about 10s to run, and the process does not respond to a sigstop -- implying that it's in uninterruptible sleep.  The very interesting thing is:
This behavior presents itself whether or not I invoke rg correctly with the --files argument, or without any arguments causing it to print the usage error message.  In both cases the process runs for > 10s!

A "real" run -- arguments --file, outputs the list of files, and returns with status code 0:real	0m10.688s
user	0m0.000s
sys	0m0.005s

Providing no arguments resulting in a usage error message, and returning status code 1:real	0m10.686s
user	0m0.008s
sys	0m0.012s

I'm going to try to trace the system calls to see what's happening!

That is very strange! I wonder whether you see anything similar with a grep -R? I'm very curious to see what is blocking it. FYI @BurntSushi

@roblourens Hmm, yeah I don't think I have any ideas either, although it's a bit hard to follow this issue. Perhaps someone could describe a reproduction in more detail. Otherwise, it's difficult to know what command is actually being run, on which inputs and what the output is.
In my limited experience, I recall that the Windows anti-malware scanner (I think?) slowing things down considerably. I disabled it long ago in my test setup. Maybe something has changed with respect to that? Just taking a shot in the dark.

This is on Mac, so antimalware shouldn't be an issue here. All I know here is that it seems that the rg process takes a very long time to complete, even when it does no real work "Providing no arguments resulting in a usage error message" and has some weird side effects on the system: "process table stops updating"

For me quick open is as fast as it used to be again since the 1.30.0 update.

Oh, then yeah, I have no idea. I don't think I could even venture a guess. At this point, I'd probably start looking at syscalls (on Linux, I'd use strace, I don't know what the equivalent is on macOS). Is there a syscall that's being blocked? If so, that might be insightful. But otherwise, ripgrep isn't doing anything weird here AFAIK.

Ok -- I got to the bottom of this:  TL;DR -- In the end it was anti-virus software, but figuring this out involved a bit of digging:

I used dtruss (a dtrace based MacOS utility) to trace the syscalls.  And shockingly, even though the process took over 10s to run, it seemed that the program was only actually executing for a few ms.
I wrote "hello world" in c -- and same behavior -- it took 10.6s to run, but the program was only executing for 1.5ms
I suspected that this might have something to do with the way processes are forked, so I wrote another c program to fork and run my hello world, and traced that -- and that's where I saw something very strange.  The syscall execve (which loads a program) ran for about 10.6s!
At first I was stumped: what has access to the kernel and can interfere with something as basic as the loading of a process?  Then my coworker pointed out that this is exactly what our antivirus was doing to scan binaries (via some integration with the kernel).  I turned it off -- everything is better.

@josh-tepper Interesting find! Do you happen to know why this problem started happening recently but didn't occur previously?

Thanks for troubleshooting and following up! Mac antivirus, that's interesting. What's it called?

No idea.  But I can say that vscode wasn't the only affected program.  I had a huge headache with the chrome 71 update, docker, postman, etc.  It seems that my system was increasingly affected over the last three weeks. It wasn't clear that it was all related until I turned off the antivirus tool and it solved everything.  We're still investigating the root cause.

Can't get viruses if you can't use your computer 👍


conversion between json and yaml - provide template func and cmd

provide template func and cmd to support json and yaml mutual conversion

Add test case and doc for it: https://github.com/upcmd/up/blob/master/tests/functests/c0165.yml
Check the doc site for details


Storefront: Wcag and accessibility review

Quick review done

[x] wave complains about a empty button on the menu. Screen reader do not pick up on it.
[x] give designsystem logo a better alt text. For example "Forside Designsystemet"
[x] give decorative image on "God praksis" a better alt text or have none as it could be considered decorative.
[x] add lang attribute to pages
[ ] some HTML validator issues. See example below (retest to see if its not a plugin or something on my computer doing it)
[x] we should have a jump to main content button. It is most apparent on the "grunnleggende" page where you have to tab trough the whole left side menu to get to the content
[x] missing H1 on "god praksis" and jumping over an heading level on "roadmap"
[x] remove "f" as alt text on buttons with icon in the footer. (Slack, github and Figma)
[x] something weird with the e-mail link in the footer when using screen reader. Se example below.
[x] remove alt text for the slack logo in the "Bli med i Slack-kanalen" button.

Some extra mobile stuff nitpicky stuff

[x] the menu on "grunnleggende" takes up a lot of space on small screens
[x] when using the menu in the header, it stays open after you click a link. You cannot click outside it to close it, you have to use the "x" button.

Also:

Jeg legger dette ut i en egen issue, slik at vi kan avslutte denne og merge inn https://github.com/digdir/designsystem/pull/504

Also:

Jeg tror Øyvind la til språk. Hvilken side var det på?


chore: fix repository

Pull Request Test Coverage Report for Build 600660360

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 88.038%

Totals

Change from base Build 587753409:
0.0%

Covered Lines:
368

Relevant Lines:
418

💛  - Coveralls


Inference test enhance

Allow specifying the number of quantization groups in the inference test script using a quantize_groups argument.
This PR is complimentary to PR #3519 on the main repo, and should be merged with (or after) merging PR #3519.
Tagging @RezaYazdaniAminabadi as a reviewer on PR #3519.

need to rewrite according to recent code changes in this repo, will reopen shortly


dogstatsd: add a way to shard time samplers by origin

What does this PR do?
This brings more fairness to the aggregators (although it is probably defeated by the current flush strategy) but also improves compressability
Motivation
By grouping metrics from the same containers together we hope to reduce entropy and improve the compression ratio.
Additional Notes
We may want to adjust the key strategy a bit more
Possible Drawbacks / Trade-offs
If two containers send different origins that end up having the exact same tags this might break aggregation.
Describe how to test/QA your changes
none

/merge -c --cancel

/merge


nextcloud: 21.0.0 -> 21.0.1

Motivation for this change
Bump the version,
I also added a comment to quickly get the sha from nextcloud
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 118882 run on x86_64-linux 1

1 package built:

nextcloud21

Currently runs on my server

Result of nixpkgs-review pr 118882 at c3367cc4 run on x86_64-linux 1

1 package built successfully:

nextcloud21

1 suggestion:

warning: missing-phase-hooks
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/servers/nextcloud/default.nix:18:5:
   |
18 |     installPhase = ''
   |     ^

Ported to stable as a6847cb5460b47241eb35d1a6588652411642066.


[Draft] kanicov: A tool to visualize coverage in Kani

This PR adds a new tool to visualize coverage results stored in a .kanicov file generated with the new coverage implementation from #3119. At the moment, the tool reports only regions which are known to be covered with the green color.
For example, if we take the Rust code:
1 fn _other_function() {
2    println!("Hello, world!");
3 }
4
5 fn test_cov(val: u32) -> bool {
6     if val < 3 || val == 42 {
7         true
8     } else {
9         false
10    }
11 }
12
13 #[cfg_attr(kani, kani::proof)]
14 fn main() {
15    let test1 = test_cov(1);
16    let test2 = test_cov(2);
17    assert!(test1);
18    assert!(test2);
19 }

The tool prints this to terminal:

I'm expecting the tool to be used for testing purposes as well, with "colored regions" being replaced with markers ``` and '''.
The tool still contains a lot of duplicated code from kani-driver. It also needs a proper set of options to filter certain results. This is marked as a draft because it depends on #3119.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

Are we planning to release this tool or is it only for developers?

The plan is to release the tool. More details in the upcoming RFC.

Note to self: Consider highlighting uncovered spans in red as the default behavior. At the very least, discuss in the RFC.

Just marked this as ready for review!

Why the issue #3543 is classified as a soundness issue now? As far as I understood, the coverage report feature will crash in the cases you mentioned in the issue. Will it report false coverage reports in any case? This is important to fix before stabilization, but it seems to me unnecessary to classify this as a soundness issue.

The reason #3543 was classified as a soundness issue is because we were filtering out the results which could lead to a crash in kani-cov. Filtering out these results can be considered a soundness issue because, even if they're spurious failures coming from the Rust coverage instrumentation, we could be hiding other issues. It all depends on how we define soundness for coverage and the origin of these coverage artifacts.
Regardless, I've changed how we handle the checks with these out-of-bounds regions to replicate the spurious failure instead. Since we only need to extend the line by one space to represent the check in such cases, we just do that now. In other words, we are now reporting the same spurious failures, which may be more confusing for users but avoids soundness concerns. For example, for the test in tests/coverage/assert we get this report:
/home/ubuntu/kani/tests/coverage/assert/test.rs
   1|     | // Copyright Kani Contributors
   2|     | // SPDX-License-Identifier: Apache-2.0 OR MIT
   3|     |
   4|     | #[kani::proof]
   5|    1| fn foo() {
   6|    1|     let x: i32 = kani::any();
   7|    1|     if x > 5 {
   8|     |         // fails
   9|    1|         assert!(x < 4);
  10|    1|         if x < 3 ```{'''
  11|    0| ```            // unreachable'''
  12|    0| ```            assert!(x == 2);'''
  13|    0| ```        }'''``` '''
  14|    1|     } else {
  15|    1|         // passes
  16|    1|         assert!(x <= 5);
  17|    1|     }
  18|     | }

Note the single region represented at the end of line 13? In the original source code, that space doesn't exist. But we can create that space in this representation and report the same result that the instrumentation gives us. It's less noticeable in the terminal output, but we also "draw it" there:

In addition to this change, I've added some more documentation to the code handling this special case and updated #3543 (labeled as "spurious failure" now).  Please don't hesitate to ask for more clarifications if you need them. @carolynzech I've also implemented your suggestions to use the type aliases in other modules of kani-cov. Thanks!


Add RandomWordSampler for infinite groups

PRASampler is a bad default for infinite groups as it generates exponential long words leading to OOM situations on simple rand(M, 2) calls

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.70%. Comparing base (2f95567) to head (dde4eb5).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #54      +/-   ##
==========================================
+ Coverage   97.14%   98.70%   +1.55%
==========================================
  Files           5        5
  Lines         140      154      +14
==========================================
+ Hits          136      152      +16
+ Misses          4        2       -2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests


Add functional tests

The main barrier to this from the onset is that router requires a k8s apiserver to talk to...
I'm thinking of using a containerized, single node k8s to fulfill that requirement.  See http://kubernetes.github.io/docs/getting-started-guides/docker/
Router config is driven off of annotations on the router rc, application services, and secrets.  It's probably possible to see the containerized k8s with some of these things and then assert that router-generated nginx config appears as expected.
The obvious sticky point here might be that the containerized k8s needs to run privileged.

Just a quick update on this.  This turns out to not be at all as easy as it looks at first glance.
So far, I have encountered two major issues:

From the k8s docs referenced in the OP:

Many of these containers run under the management of the kubelet binary, which attempts to keep containers running, even if they fail. So, in order to turn down the cluster, you need to first kill the kubelet container, and then any other containers.
You may use docker kill $(docker ps -aq), note this removes all containers running under Docker, so use with caution.

This means there isn't a reliable way to shut down the containerized test-cluster without possible adverse affects on the other containers running on the same host.

Running a k8s cluster via docker as documented in the link in the OP will have the apiserver binding to the host machine's localhost:8080.  This is somewhat problematic because folks using docker-machine won't be able to connect to the test cluster using kubectl from their Mac.  Fixing this would require making the apiserver bind to <IP_ADDRESS>:8080 on the host machine (docker-machine), and that in turn would require customizations to the hyperkube image.  Alternatively, kubectl could be included in the deis/go-dev image used for Deis component development, and then containerized processes could be used to manipulate the test cluster from within the docker host-- e.g. to submit a test manifest for the router.  I'll open a separate issue against deis/docker-go-dev to propose this.

Ultimately, I think this needs to be deferred for a bit until we have improved capabilities with regard to bringing up (and bringing down) a fully containerized k8s cluster.  Perhaps there is room for a new tool that makes short work of this and can be re-used across projects to enable k8s clusters for functional testing.  (I'm imagining something in the same vein as micro-kube... perhaps "nano-kube".)

https://github.com/mikedanese/k8s-on-k8s or https://github.com/polvi/monokube might be helpful for this.

@jgmize thanks for this.  I'll definitely be giving those a look.


Introduce retry logic into "get ClusterID" function

Description
This PR make the GetClusterID more reliable in the situation when the ConfigMap it tries to read from has not been created yet.
I reused the retry primitive included in the k8s client-go library, in order to be more consistent with the rest of the project. The function wait 30 seconds for 10 times. We could hard-code an opinionated configuration (current implementation) or make configurable by the caller (if useful, maybe not).
Fixes #521
Plan

[x] Implementation
[x] Local build
[x] Preliminary tests on development k8s cluster
[x] Write test cases

I added three test cases to cover every possible situation. Only the last one is significant actually, the first twos are reported only for completeness (and be aware that the first one will make the testing process 3 minutes longer): we could keep only the last one.

/rebase

Hi @lomluca, can you please squash your commits?

Ok, I followed your feedback :) Hope it's ok now, let me know if there are any other change you would do. If not, I'm going to squash everything.

/rebase

/merge

/merge


django.db.utils.OperationalError: duplicate column name: department_id

\Online Examination\New folder\django-lms-main\env\lib\site-packages\django\db\backends\sqlite3\base.py", line 357, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: duplicate column name: department_id

Hi,
I am very to Django; I am trying to find the setting.py file but can't find it.  After cloning, the LMS gets the following error.  Please tell me how I can resolve these issues.  I appreciate any help you can provide.

Hi there,
Could you elaborate more on that please, is this error occurring when
running python manage.py migrate? Also, have you cloned the latest
version, because I pushed changes like 4 days ago?
Basically, the error is reporting that the column department_id is
already been created and you're trying to re-create it, which is not
possible. You could solve this by dropping/deleting your database and
re-run python manage.py makemigrations and python manage.py migrate
respectively.
Let me know for further help
Best regards
Adil Mohammed
On Wed, May 17, 2023 at 2:20 PM Tayy3bkh4n @.***> wrote:

Hi,
I am very to Django; I am trying to find the setting.py file but can't
find it. After cloning, the LMS gets the following error. Please tell me
how I can resolve these issues. I appreciate any help you can provide.
—
Reply to this email directly, view it on GitHub
https://github.com/adilmohak/django-lms/issues/1#issuecomment-1551208205,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOPB3IQUG2DGZDYXEDOHRM3XGSYAJANCNFSM6AAAAAAW4HG5U4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Trying to add 2 nodes simultaneously to closure_tree table results in a deadlock

When two nodes are added simultaneously to the closure tree table, this results in a deadlock and the query to acquire lock runs infinitely:
 (0.7ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t46e4b4acfd42770bad34d3e2a13a458a
  (0.4ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t9eaf21f51a8c5a1f2e39f6e8c4971b27
  (0.4ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS tae827dc66e71e92f4c9f6edecae1cb54
  (0.3ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t7f7d47e378ffde7f311e56fde6613ab5
  (0.3ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS ta7af1b261b90cba9e4d7399c96f17e26
  (0.3ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t3ed658e2e0d3008375e7356cc8e1c4a2
  (0.4ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t080acfd6e5536c50c4917ce271671be6
  (0.3ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t850056c6b58f75d8171bdcb69def558a
  (0.3ms)  SELECT GET_LOCK('b218f48596dc37d16bbcd16d1e18a116c', 0) AS t1ece95b52215d7c5e420ea9c0057aa72

You can reproduce the problem easily by using the code below (where List is the closure_tree model):

Thread.new do
  parent = List.find(86)
  new_node = List.new(:name => "test deadlock 1")
  parent.add_child(new_node)
end
Thread.new do
  parent = List.find(82)
  new_node = List.new(:name => "test deadlock 1")
  parent.add_child(new_node)
end

We faced this problem on production, and the infinite locking loop results in mysql being unresponsive, which in turn results in unresponsive app.
closure_tree version: 6.2.0
ruby: 2.3.0
rails: <IP_ADDRESS>

I have just discovered a similar problem in production on Postgresql. Have not been able to track down what is causing it, but I discovered an infinite loop of
SELECT pg_try_advisory_lock(1597746110,0) AS t57b1486ecc3691a98e06a13406a11286
in my log.
Database is unresponsive until I restart the app.
closure_tree version: 6.0.0
ruby 2.2.0
rails 4.2.5

Yikes. Thanks for reporting. Can you see if reverting #234 fixes the issue? That's the most recent update to locks.

Reverting that commit doesn't seem to fix this.

Same here.
I had to revert back to version 6.0 in production, but the problem still appears though much less frequent with version 6.0 then 6.2.
I'm trying to find time to write tests to try and reproduce this problem locally.

Can confirm, this issue has been around for a while. 6.2 seems to have increased the frequency from  a monthly to hourly occurrence.
Ruby 2.3.1
Rails <IP_ADDRESS>
Closure tree 6.2
PostgreSQL.

yikes. Are all you using postgres? What version?

FWIW, I've been looking through the diffs between 6.0.0 and 6.0.2. Nothing immediately nefarious jumps out. https://github.com/mceachen/closure_tree/compare/v6.0.0...master

We're using: PostgreSQL 9.4.5 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 4.9.1-16ubuntu6) 4.9.1, 64-bit

I am killing the lock by hand.
SELECT * FROM pg_locks;
SELECT pg_terminate_backend('')
the lock does not respond to a pg_kill_backend when in this state.
I am downgraded to 6.1. In ~10,000 inserts I had 1 lockup. In 6.2 it was occurring almost immediately. We are also on PostgreSQL 9.4.5 AWS hosted.

Thanks for the additional info.
@DuaneCompton did you have 1 lockup/10k inserts with v6.1?

@mceachen There are 6 threads running that are simultaneously under load. under v6.1 I am averaging 1 lockup/10k inserts. In v6.2 it locks basically immediately.
Might be two separate issues at play. I know under v6.0.0 we experienced intermittent lock-ups as well.

We use MySql, therefore I don't think this is a postgres specific issue.

@mceachen Anything I can do from a bug gathering standpoint to help troubleshoot this issue? We still run into a lock up on a weekly basis on 6.0. I am probably going to write a little script to free the lock if the table locks for longer than N minutes. But, before I do so is there anyway I can help?

Can you capture the current running queries? Without a test to replicate
the issue on demand, any change we make will be a random shot in the dark.

@mceachen will do.

Has anyone had any more luck in tracking down this issue ?
I've tried to provoke this issue by creating multiple records in parallell in separate threads as was mentioned initially by @amitsaxena. But no luck so far. I've forked off 6 threads each creating a 1000 records. Tried on both 6.2 and 6.0.

@tomaadland are you unable to reproduce the issue? I was consistently getting the bug as soon as I execute the code above.

@amitsaxena as soon as I have some time on my hands i'll create a public project with an example model similar to one I have in production with the tests. I'll post it here to let you know.

@mceachen So, I've noticed when we hit this state there is really just one lock that is hanging around. Is there any reason you could see that we couldn't use the timeout in your with_advisory_lock gem to at least an infinite lock?
I was also curious about this line: https://github.com/mceachen/closure_tree/blob/master/lib/closure_tree/hierarchy_maintenance.rb#L82
Any reason to not expire the previous lock before taking a new one?

If you auto-expire a lock when it's contended, the first lock is made
irrelevant.
The deadlock is either due to a bug in the RDBMS, or, much more likely, a
circular dependency due to transaction boundaries.
For example, if one of your data mutation code paths gets lock A then lock
B, and another gets B then A, you'll regularly see deadlocks.

So, its not a dead lock per se. If I move around the locking obtain/release in just the critical sections of the write/delete/children call then it will be detected as a deadlock. What is actually happening is:
writer 1 obtains a lock in the context of a rebuild!.
writer 2 calls add_child (which I believe is going to trigger a rebuild). writer 2 spins for a while attempting to obtain the lock in which case writer 1 is paused.
writer 2 fails to obtain the lock and writer 1 continues writing.
repeat with writer 3...N and writer 1 basically never makes it through a rebuild.
What I am not knowledgable enough to know is why writer 1 does not continue writing when it has already obtained the lock.

Can you maybe point to the code, or talk more about what's going on in writer 1 and 2?
Deadlocks I've seen in the past occur when there are multiple code paths that have transaction blocks that edit models in different order.

 Sorry, its not a deadlock in the traditional sense, its actually worse since a deadlock will be detected and aborted. In this case, the first process has obtained the lock and is proceeding on its way when the second process comes in and attempts to obtain a lock. It fails since writer one already has the a lock  (which is the correct behavior IMO). What I am having trouble understanding is why writer 1 hangs until writer 2 gives up attempting to obtain its lock.

If I run something like:
parent = Tag.create(name: 'lock 1')
(1..100).each do |i|
p "adding #{i}"
new_node = Tag.create(:name => "test deadlock #{i}")
ActiveRecord::Base.transaction do
parent = parent.add_child(new_node)
end
end
on two separate machines upon starting the second both will shortly arrive in a semi-deadlocked situation. The script that I started the call on will then eventually expire and the first writer will continue. I've tried moving around the transaction/without it. Below is one potential outcome.
Writer 1 :
D, [2017-02-16T07:31:34.537506 #21545] DEBUG -- :   Tag Exists (0.4ms)  SELECT  1 AS one FROM "tags" INNER JOIN "tag_hierarchies" ON "tags"."id" = "tag_hierarchies"."ancestor_id" WHERE "tag_hierarchies"."descendant_id" = $1 AND "tags"."id" = $2 LIMIT 1  [["descendant_id", 7903], ["id", 7905]] W, [2017-02-16T07:31:34.537919 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/hierarchy_maintenance.rb:27:in_ct_validate'
D, [2017-02-16T07:31:34.541672 #21545] DEBUG -- :   SQL (1.6ms)  UPDATE "tags" SET "parent_id" = $1, "updated_at" = $2 WHERE "tags"."id" = $3  [["parent_id", 7903], ["updated_at", "2017-02-16 07:31:34.538876"], ["id", 7905]]
W, [2017-02-16T07:31:34.542309 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/model.rb:138:in `add_child'
D, [2017-02-16T07:31:34.545266 #21545] DEBUG -- :    (0.5ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t2ea0208ee2526226ecc8d65da8f4c493
W, [2017-02-16T07:31:34.546018 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in execute_successful?' D, [2017-02-16T07:31:34.646474 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS ta2ef67039c80d539454f14205900baac W, [2017-02-16T07:31:34.647598 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in execute_successful?'
D, [2017-02-16T07:31:34.785180 #21545] DEBUG -- :    (0.8ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tafb0ba91177bd447d53f97cae6f97ae0
W, [2017-02-16T07:31:34.786015 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:34.910538 #21545] DEBUG -- :    (0.8ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t878276a11847c5bb14ee22e966d51222
W, [2017-02-16T07:31:34.911517 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.014669 #21545] DEBUG -- :    (1.2ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t3996341ab230c95a3222cf08044a7593
W, [2017-02-16T07:31:35.015517 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.098349 #21545] DEBUG -- :    (0.8ms)  SELECT pg_try_advisory_lock(1814493058,0) AS td67fe931e1e97081594bf1d3b7b577ac
W, [2017-02-16T07:31:35.098631 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.210146 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS td2b2a4adf0f2fb2e61565b1a8d58da8b
W, [2017-02-16T07:31:35.210534 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.351044 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t6ba2e7b92cdaf2d1905f1a6214e7efa2
W, [2017-02-16T07:31:35.351374 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.429454 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t8388cb7c01e7341bd1f63e3404fe8e9b
W, [2017-02-16T07:31:35.429904 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.520427 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tb6c70aa0fe9699434e66857ba96e1c7d
W, [2017-02-16T07:31:35.520766 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.616061 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS taef48c46023f84d4e4c6454fce8dfed7
W, [2017-02-16T07:31:35.616458 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.756555 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t14f7ff2ec5dae5c9961188d75d2f0ea7
W, [2017-02-16T07:31:35.756914 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.829528 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t363a59f8e61d4ce5c0fcfc5abf6e8404
W, [2017-02-16T07:31:35.829871 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:35.978112 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t57ace053678ffafac925ebec1a163abe
W, [2017-02-16T07:31:35.978513 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.053677 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t809dd7878f94941becb585a2ee937e3e
W, [2017-02-16T07:31:36.053976 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.144780 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t95ae05753fbbc871f7f4b9e217edaf48
W, [2017-02-16T07:31:36.145119 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.214093 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS te4c9a243c5292b64fc2dbcd6718ad1e6
W, [2017-02-16T07:31:36.214426 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.352478 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t943f3fe082a67f4b994402d07af31816
W, [2017-02-16T07:31:36.352826 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.443962 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t2fc1b466decc5f2ff052e3a2f0bfe292
W, [2017-02-16T07:31:36.444380 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.565932 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t090f3c20fdf182117573fca00c766c8e
W, [2017-02-16T07:31:36.566271 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.703930 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS te1e6cab767dc9a3e48e376bd8dcd367a
W, [2017-02-16T07:31:36.704215 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.785398 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t11b9f20f59a201e831f1dee0ddf05983
W, [2017-02-16T07:31:36.785673 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:36.929739 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tf407d5d01b89ef84e483c0eb6a49be00
W, [2017-02-16T07:31:36.930024 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.023233 #21545] DEBUG -- :    (1.0ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t0a9aa93724eb8e7e930239be4ad66a39
W, [2017-02-16T07:31:37.023588 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.114824 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t70b425ed146db34d41a2450d4b2f73da
W, [2017-02-16T07:31:37.115114 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.264179 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tb40138d10bfda7bc5426b4b9012361bf
W, [2017-02-16T07:31:37.264531 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.399035 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t7291bb3658aa9df4f58122b7e963bd28
W, [2017-02-16T07:31:37.399411 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.461447 #21545] DEBUG -- :    (0.6ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tcd90c527a9780a5dc20ce90a4056ba0e
W, [2017-02-16T07:31:37.461748 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.572679 #21545] DEBUG -- :    (1.0ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t9aa44bdd7df70e779b53c5e915c81f04
W, [2017-02-16T07:31:37.573035 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.626635 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t49bb359801afe46b0773bc9fc1045007
W, [2017-02-16T07:31:37.626946 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.780170 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tb9d21a3f79cb01841fa72beb3023f957
W, [2017-02-16T07:31:37.780519 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:37.887276 #21545] DEBUG -- :    (1.0ms)  SELECT pg_try_advisory_lock(1814493058,0) AS tfa30eaba8a3621750333aa7822966ed0
W, [2017-02-16T07:31:37.887972 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:38.025141 #21545] DEBUG -- :    (0.9ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t9c2239516040c026b37f60c1d42f0322
W, [2017-02-16T07:31:38.025471 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:38.157474 #21545] DEBUG -- :    (0.8ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t5d761d7df4e465655ed302d7d256740a
W, [2017-02-16T07:31:38.157775 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:38.253226 #21545] DEBUG -- :    (0.7ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t23ee83242b724f8bee81abc5b04f0a50
W, [2017-02-16T07:31:38.253502 #21545]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in execute_successful?'
Writer 2
D, [2017-02-16T07:31:34.457059 #21548] DEBUG -- : DEBUG: Chewy strategies stack: [2] <- bypass @ deadlock.rb:1 D, [2017-02-16T07:31:34.465650 #21548] DEBUG -- :    (0.5ms)  BEGIN W, [2017-02-16T07:31:34.465916 #21548]  WARN -- :   ↳ deadlock.rb:3:in block in <top (required)>'
D, [2017-02-16T07:31:34.488118 #21548] DEBUG -- :   SQL (0.8ms)  INSERT INTO "tags" ("name", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id"  [["name", "lock 1"], ["created_at", "2017-02-16 07:31:34.485771"], ["updated_at", "2017-02-16 07:31:34.485771"]]
W, [2017-02-16T07:31:34.488464 #21548]  WARN -- :   ↳ deadlock.rb:3:in `block in <top (required)>'
D, [2017-02-16T07:31:34.539682 #21548] DEBUG -- :    (0.5ms)  SELECT pg_try_advisory_lock(1814493058,0) AS t9f2a3d3d5ce591e0cbcad87a06e46c26
W, [2017-02-16T07:31:34.539915 #21548]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:31:34.601380 #21548] DEBUG -- :   SQL (0.7ms)  INSERT INTO "tag_hierarchies" ("ancestor_id", "descendant_id", "generations") VALUES ($1, $2, $3)  [["ancestor_id", 7904], ["descendant_id", 7904], ["generations", 0]]
W, [2017-02-16T07:31:34.601783 #21548]  WARN -- :   ↳ /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/hierarchy_maintenance.rb:66:in `block in rebuild!'
D, [2017-02-16T07:32:34.608471 #21548] DEBUG -- :    (60004.6ms)  UPDATE "tags"
SET "position" = t.seq + -1
FROM (
SELECT "id" AS id, row_number() OVER(ORDER BY position) AS seq
FROM "tags"
WHERE "parent_id" IS NULL
) AS t
WHERE "tags"."id" = t.id
W, [2017-02-16T07:32:34.609206 #21548]  WARN -- :   ↳ /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/numeric_order_support.rb:39:in `reorder_with_parent_id'
D, [2017-02-16T07:32:34.613072 #21548] DEBUG -- :    (1.4ms)  SELECT pg_advisory_unlock(1814493058,0) AS t5a0d17d7a9f5b664b7d1f87c2252d30c
W, [2017-02-16T07:32:34.613436 #21548]  WARN -- :   ↳ /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in `execute_successful?'
D, [2017-02-16T07:32:34.614966 #21548] DEBUG -- :    (0.4ms)  ROLLBACK
W, [2017-02-16T07:32:34.615220 #21548]  WARN -- :   ↳ deadlock.rb:3:in `block in <top (required)>'
D, [2017-02-16T07:32:34.616004 #21548] DEBUG -- : DEBUG: Chewy strategies stack: [2] -> bypass @ deadlock.rb:1
/usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in async_exec': PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block (ActiveRecord::StatementInvalid) : SELECT pg_advisory_unlock(1814493058,0) AS t5a0d17d7a9f5b664b7d1f87c2252d30c 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in block in exec_no_cache'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract_adapter.rb:484:in block in log' 	from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/notifications/instrumenter.rb:20:in instrument'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract_adapter.rb:478:in log' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in exec_no_cache'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/postgresql_adapter.rb:584:in execute_and_clear' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/postgresql/database_statements.rb:50:in select_value'
from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:14:in execute_successful?' 	from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/postgresql.rb:9:in release_lock'
from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/base.rb:81:in ensure in yield_with_lock' 	from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/base.rb:81:in yield_with_lock'
from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/base.rb:65:in yield_with_lock_and_timeout' 	from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/base.rb:48:in with_advisory_lock_if_needed'
from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/concern.rb:16:in with_advisory_lock_result' 	from /usr/local/bundle/gems/with_advisory_lock-3.0.0/lib/with_advisory_lock/concern.rb:10:in with_advisory_lock'
from /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/support.rb:103:in with_advisory_lock' 	from /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/hierarchy_maintenance.rb:63:in rebuild!'
from /usr/local/bundle/gems/closure_tree-6.0.0/lib/closure_tree/hierarchy_maintenance.rb:39:in _ct_after_save' 	from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:432:in block in make_lambda'
from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:228:in block in halting_and_conditional' 	from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:506:in block in call'
from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:506:in each' 	from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:506:in call'
from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:92:in __run_callbacks__' 	from /usr/local/bundle/gems/activesupport-<IP_ADDRESS>/lib/active_support/callbacks.rb:778:in _run_save_callbacks'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/callbacks.rb:302:in create_or_update' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/persistence.rb:120:in save'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/validations.rb:37:in save' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/attribute_methods/dirty.rb:21:in save'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:286:in block (2 levels) in save' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:351:in block in with_transaction_returning_status'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in block in transaction' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/transaction.rb:184:in within_new_transaction'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in transaction' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:220:in transaction'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:348:in with_transaction_returning_status' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:286:in block in save'
from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:301:in rollback_active_record_state!' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/transactions.rb:285:in save'
from /usr/local/bundle/gems/newrelic_rpm-<IP_ADDRESS>7/lib/new_relic/agent/instrumentation/active_record_helper.rb:26:in block in save' 	from /usr/local/bundle/gems/newrelic_rpm-<IP_ADDRESS>7/lib/new_relic/agent.rb:586:in with_database_metric_name'
from /usr/local/bundle/gems/newrelic_rpm-<IP_ADDRESS>7/lib/new_relic/agent/instrumentation/active_record_helper.rb:25:in save' 	from /usr/local/bundle/gems/activerecord-<IP_ADDRESS>/lib/active_record/persistence.rb:34:in create'
from deadlock.rb:3:in block in <top (required)>' 	from /usr/local/bundle/gems/chewy-0.8.4/lib/chewy/strategy.rb:60:in wrap'
from /usr/local/bundle/gems/chewy-0.8.4/lib/chewy.rb:179:in strategy' 	from deadlock.rb:1:in <top (required)>'
from /usr/local/bundle/gems/zeus-0.15.3/lib/zeus/load_tracking.rb:50:in load' 	from /usr/local/bundle/gems/zeus-0.15.3/lib/zeus/load_tracking.rb:50:in load'
from /usr/local/bundle/gems/railties-<IP_ADDRESS>/lib/rails/commands/runner.rb:60:in <top (required)>' 	from /usr/local/bundle/gems/railties-<IP_ADDRESS>/lib/rails/commands/commands_tasks.rb:123:in require'
from /usr/local/bundle/gems/railties-<IP_ADDRESS>/lib/rails/commands/commands_tasks.rb:123:in require_command!' 	from /usr/local/bundle/gems/railties-<IP_ADDRESS>/lib/rails/commands/commands_tasks.rb:90:in runner'
from /usr/local/bundle/gems/railties-<IP_ADDRESS>/lib/rails/commands/commands_tasks.rb:39:in run_command!' 	from /usr/local/bundle/gems/railties-<IP_ADDRESS>/lib/rails/commands.rb:17:in <top (required)>'
from bin/rails:9:in require' 	from bin/rails:9:in '
`

And by 👍 I mean 😭. I'll see if I can wire up a test to do this.

Thank you. I will see what I can do as well. I do not believe at this time that its an a/b b/a situation, because I think the lock acquisition time is much higher than the deadlock check timer on the db. I'm wondering if maybe calling a complete table lock might be better in the rebuild! function rather than calling an additional lock for each delete/child insert (maybe I am misunderstanding how those locks work though).

I think a table lock (which I had dismissed as being too coarse a mutex) is an interesting alternative. I'll code up that as an option.

Have almost the same problem(after upgrading to Rails 5), with Rails 4.2.5 it worked well
I have accepts_nested_attributes_for my hierarchy model,
and in that model I also have accepts_nested_attributes_for for its children
And after saving the parent item I got deadlock, and child item is silently ignored
Tried versions 6.2 and 6.4

With version 6.0 it produces no deadlock, but still can't add a child

I thinks it's more about Rails update than closure_tree update

@mceachen It appears the issue I am running into is specifically (and I am guessing amitsaxena as well) around the numeric reordering (https://github.com/mceachen/closure_tree/blob/master/lib/closure_tree/hierarchy_maintenance.rb#L76)
when calling add_child. Writer 1 will hang on grabbing
https://github.com/mceachen/closure_tree/blob/master/lib/closure_tree/hierarchy_maintenance.rb#L63
while writer 2 will attempt to write into:  https://github.com/mceachen/closure_tree/blob/master/lib/closure_tree/numeric_order_support.rb#L39
If you call directly children.create() it will avoid this issue, though I would guess any concurrent movement of leafs with numeric ordering enabled could result in this.
If I replace the with_advisory_lock from support.rb with just a model_class.lock(true) the deadlock appears to go away.
I did also note that if you attempt a destroy_all on a large tag set > 10000 (with our without numeric ordering)
the destroy will fail with fatal: exception reentered.
Hope this is helpful.

@DuaneCompton that is super helpful, and shouldn't be hard to write a breaking test. Do you want to have a go?

I will give it a shot. Thanks.

@DuaneCompton any progress here? I just ran into this issue.

Thanks to @serenaf, she highlighted the MySQL docs that reveal v5.7.5 fundamentally changed how get_lock() works. Fixing that issue should resolve this issue for MySQL users, at least.

@uhlenbrock this issue (which seems to be with_advisory_lock not playing nicely with newer versions of postgres, mysql, and rails) is currently the most pressing for closure_tree. More eyes on it are welcome.

@mceachen I have started to take a look. I have two PRs open on with_advisory_lock to get the testing infrastructure ready to replicate this issue.

@uhlenbrock Any ideas on how to replicate this in a test ?

@tomaadland I have started work over here: https://github.com/ClosureTree/with_advisory_lock/issues/17

I am working to build a failing test based on the suggestion above:
describe Tag do
  context 'in a race condition' do
    it 'should not break the database' do
      parent1 = Tag.create!(name: 'parent tag 1')
      parent2 = Tag.create!(name: 'parent tag 2')

      Thread.new do
        new_node = Tag.new(name: 'test deadlock 1')
        parent1.add_child(new_node)
      end
      Thread.new do
        new_node = Tag.new(name: 'test deadlock 1')
        parent2.add_child(new_node)
      end
    end
  end
end

Via DB=postgresql appraisal activerecord-5.1 rake spec, the following error is raised:
SystemStackError:
       stack level too deep

I believe this is because of Thread.abort_on_exception = true. The test does not raise an error with this line commented out, but also takes between 10 and 20 seconds.  I also found that it is true that calling create instead of new avoids any issue.
Does this appear to adequately reproduce the issue?

I tried running this test in the gem and I get the SystemStackError
When I created a similar unit test in the app that is using this gem and when calling create! instead of new for creating new children instances to be added I get postgresql to err on deadlock.
ERROR:  deadlock detected at character 8
So in my case the
new_node = Tag.create!(name: "test deadlock 1")
causes a deadlock.

I may have found a workaround that works for me -- though at present I'm not sure of what other consequences may arise from my solution.
I'd long since upgraded the database to MariaDB 10.x from some older MySQL without apparent incident.  Until one day when I needed to reparent something, and then...
Mysql2::Error: Lock wait timeout exceeded; try restarting transaction

At the time, i wrote an awful script that monkey-patched the part of closure_tree that dealt with locking (IIRC), and managed to get my reparenting done.
Now, however, I'm going to need to reparent more often, so I took some time today to do a little more research with a simple test script...
#!/usr/bin/ruby

$LOAD_PATH.unshift '/idw/lib'
Dir.chdir('/idw') do
  require 'rubygems'
  require 'bundler'
  Bundler.require(:default)   # TODO: split this out to just what's needed
  require '/idw/config/environment'
end

require 'pp'

# connect to the right database for the curent environment
IDW::Base.establish_connection(YAML.load(File.read('/idw/config/database.yml'))[ENV['IDW_ENV']])
IDW::Base.connection_pool.with_connection do

  hc = IDW::Hostclass.find_by(name: 'NS_OPMS')
  pp hc.self_and_ancestor_ids

  phc = IDW::Hostclass.find_by(name: 'NS_ORCAWAVE')
  pp phc.self_and_ancestor_ids

  phc.children << hc

end

Which, when run...  Failed:
$ ./reparent
D, [2018-03-09T21:57:01.685550 #148] DEBUG -- :    (0.3ms)  SET NAMES utf8,  @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'),  @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
D, [2018-03-09T21:57:01.739543 #148] DEBUG -- :    (0.2ms)  SET NAMES utf8,  @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'),  @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
D, [2018-03-09T21:57:01.750891 #148] DEBUG -- :   IDW::Hostclass Load (0.4ms)  SELECT  `hostclasses`.* FROM `hostclasses` WHERE `hostclasses`.`name` = 'NS_OPMS' LIMIT 1
D, [2018-03-09T21:57:01.773695 #148] DEBUG -- :    (0.7ms)  SELECT `hostclasses`.`id` FROM `hostclasses` INNER JOIN `hostclass_hierarchies` ON `hostclasses`.`id` = `hostclass_hierarchies`.`ancestor_id` WHERE `hostclass_hierarchies`.`descendant_id` = 214 ORDER BY `hostclass_hierarchies`.generations asc
[214, 199, 2]
D, [2018-03-09T21:57:01.774403 #148] DEBUG -- :   IDW::Hostclass Load (0.4ms)  SELECT  `hostclasses`.* FROM `hostclasses` WHERE `hostclasses`.`name` = 'NS_ORCAWAVE' LIMIT 1
D, [2018-03-09T21:57:01.776208 #148] DEBUG -- :    (0.6ms)  SELECT `hostclasses`.`id` FROM `hostclasses` INNER JOIN `hostclass_hierarchies` ON `hostclasses`.`id` = `hostclass_hierarchies`.`ancestor_id` WHERE `hostclass_hierarchies`.`descendant_id` = 215 ORDER BY `hostclass_hierarchies`.generations asc
[215, 199, 2]
D, [2018-03-09T21:57:01.776909 #148] DEBUG -- :    (0.3ms)  BEGIN
D, [2018-03-09T21:57:01.781243 #148] DEBUG -- :   IDW::Hostclass Exists (0.5ms)  SELECT  1 AS one FROM `hostclasses` INNER JOIN `hostclass_hierarchies` ON `hostclasses`.`id` = `hostclass_hierarchies`.`ancestor_id` WHERE `hostclass_hierarchies`.`descendant_id` = 215 AND `hostclasses`.`id` = 214 LIMIT 1
D, [2018-03-09T21:57:01.782238 #148] DEBUG -- :   IDW::Hostclass Exists (0.3ms)  SELECT  1 AS one FROM `hostclasses` WHERE `hostclasses`.`name` = 'NS_OPMS' AND (`hostclasses`.`id` != 214) LIMIT 1
D, [2018-03-09T21:57:01.783606 #148] DEBUG -- :   SQL (0.4ms)  UPDATE `hostclasses` SET `parent_id` = 215, `updated_at` = '2018-03-09 21:57:01' WHERE `hostclasses`.`id` = 214
D, [2018-03-09T21:57:01.786316 #148] DEBUG -- :    (0.3ms)  SELECT GET_LOCK('3bd772cb25a662070242f7efd5ca79ef2', 0) AS t5e93ac4ad69f0ed6f2c5c9584d8b188b
D, [2018-03-09T21:57:01.788037 #148] DEBUG -- :    (1.4ms)  DELETE FROM `hostclass_hierarchies`
WHERE descendant_id IN (
  SELECT DISTINCT descendant_id
  FROM (SELECT descendant_id
    FROM `hostclass_hierarchies`
    WHERE ancestor_id = 214
       OR descendant_id = 214
  ) AS x )

D, [2018-03-09T21:57:01.793990 #148] DEBUG -- :    (0.2ms)  BEGIN
D, [2018-03-09T21:57:52.175943 #148] DEBUG -- :   SQL (50381.2ms)  INSERT INTO `hostclass_hierarchies` (`ancestor_id`, `descendant_id`, `generations`) VALUES (214, 214, 0)
D, [2018-03-09T21:57:52.176687 #148] DEBUG -- :    (0.5ms)  ROLLBACK
D, [2018-03-09T21:57:52.177324 #148] DEBUG -- :    (0.4ms)  SELECT RELEASE_LOCK('3bd772cb25a662070242f7efd5ca79ef2') AS t72718680058c1a95838500647177a561
D, [2018-03-09T21:57:52.177853 #148] DEBUG -- :    (0.4ms)  ROLLBACK
/idw/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in `_query': Mysql2::Error: Lock wait timeout exceeded; try restarting transaction: INSERT INTO `hostclass_hierarchies` (`ancestor_id`, `descendant_id`, `generations`) VALUES (214, 214, 0) (ActiveRecord::StatementInvalid)

...but provided some clues as to what was going on.

BEGIN transaction 1

UPDATE the node with its new parent
SELECT GET_LOCK stuff...
DELETE the old hierarchy information
BEGIN transaction 12

try to INSERT the new hierarchy information, but times out

ROLLBACK
SELECT RELEASE_LOCK

ROLLBACK

The (currently) third solution to https://stackoverflow.com/questions/6000336/how-to-debug-lock-wait-timeout-exceeded-on-mysql mentions that InnoDB's default isolation level is REPEATABLE_READ, whereas other DBs use READ_COMMITTED.
Setting my database's default isolation level to READ_COMMITTED...
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;

...did the trick and allowed the reparenting script (and another more complicated script) to work as anticipated.  Not sure I'd want to leave this on globally, might be best to set it as tightly around the code as possible (before that first transaction).  Maybe something like...
FooModel.transaction(isolation: :read_committed) do
  # the reparenting stuff
end

I'm not confident this solves the other two-thread deadlock issue, as that appears to be happening with non-MySQL/MariaDB databases, also.

I hadn't thought about this before @tomalok 's post, but I believe with_advisory_lock could no-op if it could reliably determine that the current database is configured as READ COMMITTED or supports MVCC. It'd be good to get a simple breaking test in place to verify this hypothesis.

we are facing the same issue. Any update on this?

I can reliably reproduce this deadlock with the following test case, which concurrently removes children from a nested set with numeric sort order. The ordering seems to be the culprit, because it tries to update siblings' sort indices which leads to an implicit row lock, which, in conjunction with the implicit row locks taken out when rails updates the parent_id on records (which happens before the advisory_lock is attempted). So a possible scenario would be:

A updates child1.parent_id (and now holds an implicit lock on child1)
B updates child2.parent_id (implicit lock on child2)
A wins the race and gets the advisory lock
B now waits for the advisory lock
A tries to update siblings' sort column, among them child2.
A waits to get the lock on child2 for the update.

This kind of deadlock where one writer waits for the advisory lock while the other waits for a row level lock apparently isn't detected by either MySQL or Postgres, leading to an endless wait.
The only way I found to fix this is to get the advisory lock before even updating the parent_id, i.e. by calling record._ct.with_advisory_lock { record.save } in your application code.
test case:
  it "fails to deadlock from removing siblings with sort_order" do
    skip("unsupported") unless run_parallel_tests?

    @root = Label.create!(name: 'root')
    60.times{ |i| Label.create! name: i, mother_id: @root.id }
    @root.reload

    # remove 40 random children in parallel threads
    descendant_ids = @root.descendant_ids
    ids_to_remove = descendant_ids.sample(40).shuffle
    ids_to_keep = descendant_ids - ids_to_remove
    Parallel.map(ids_to_remove, in_threads: max_threads) do |id|
      ActiveRecord::Base.connection_pool.with_connection do
        Label.find(id).update(mother_id: nil)
      end
    end

    Label.where(id: ids_to_remove).each do |l|
      assert_nil l.mother_id
      assert l.root?
      assert l.leaf?
    end
    @root.reload
    assert_equal 20, @root.children.count
    assert order = @root.children.pluck(:column_whereby_ordering_is_inferred).uniq
    assert_equal (0..19).to_a, order
  end

The test will get stuck in an endless loop, unless the call to ct_reorder_prior_siblings_if_parent_changed in the rebuild! method is commented out. It might worth thinking about the necessity of that reordering - maybe having non-consecutive sort indices isn't even an issue?


Improve integration testing

Ideally, we should be able to run tests against sample applications running test code in a webserver. This will allow us to test things like framework integrations, response codes, and other edge cases that are hard to test with only unit tests.

I used this library for integration tests in zipkin: https://github.com/jcchavezs/httptest-php do you think it could work here?

Any opinion on ^ @chingor13 @tmatsuo?

Your library looks super useful for something like testing our Guzzle integrations.
As for testing framework integrations, we'd probably want some sample applications where we install the library and possibly run a local webserver and hit the app with requests - checking that stuff like trace context propagation works.
This issue is probably too general and we can break it into smaller issues.


Am I using Fractal Correctly?

I'm using Fractal for a Laravel API I'm developing.  I'm finding that my endpoints are very slim (which is fine!) but that a lot of the heavy lifting is happening in my transformers.  For example, lets say your endpoint is User/Dashboard.  So you'll need to retrieve:

the User (User Model)
Profile info (Profile Model)
Friends (Friend Model)
Favorite Books (Book Model) for the sake of example.

So my endpoint might look something like:
// DashboardController
public function index()
{
$user = $this->userRepository->find(Authorizer::getResourceOwnerId());
return $this->respondWithItem($user, $this->userTransformer);
}

So that's pretty simple.  But my URL might look something like:
user/dashboard?include=profile,friends,books
and my Transformers will be responsible for interacting with the Models / Repositories and querying for the correct data sets (Friends, Books, Profile, and any constraints that are tied to those).
In fact, because so many of my endpoints are 'User Oriented' , most of them end up fetching the User (just like the above Controller method) and passing it off to the transformer to figure out what happens next, based on the include parameters it receives.
Is this an appropriate use of the Transformers or am I tackling this incorrectly?

I would suggest that you're not tackling this correctly, the transformers should only be concerned with what data gets sent to the API consumer. I would suggest creating some form of service class to handle your flow and only use the transformers to transform data to a standardised form of output.


Adds backoffTime option to throttle on getRecords

By getting records immediately after a successful GetRecords call there is the
potentially starving other Kinesis consumers on the stream. This change adds an
optional backoffTime (ms) option that sleeps before getting records again.

Any plan to merge this?

Pretty please :+1:


Why can't I import pykima?

I've run python setup.py install and pykima seems to have been put in my site-packages correctly. However, I get a ModuleNotFoundError when I try to import it. Any idea why this would happen?

Python 2 or 3?
Assuming no typos, this can be an issue with sys.path not including the right path or worst case scenario a circular import.
I'll try to debug it in the morning.

Python 3.6.4 through Anaconda 5.1.0 (the latest version). I can import pykima if I'm in the same directory as setup.py, but not if I'm in another directory?
Would the sys.path thing affect other packages as well? Because DNest4's Python package (installed the same way) works fine.
Sorry I'm not very knowledgeable about Python environments and packages.

The setup.py didn't actually specify the package it wanted to install, but Travis didn't catch this because I tested it in the same directory. It's fixed now, I think (you can do setup.py install again). Thank you for catching all these bugs!