IRC disconnects but never re-connects (freenode)

Node version: v6.11.4
Discord IRC Version: 2.4.2 (from npm)
Discord IRC seems to randomly disconnect from IRC, and fail to re-connect at any point. I have to keep going in and restarting it. It stays connected to Discord just fine, but not IRC.
17 Nov 21:23:39 - SEND: PRIVMSG #litecoin :<07redacted> some example message
17 Nov 21:23:39 - Network error: Error: This socket is closed

:+1:

What version are you running? (discord-irc -V) Also are you running it in debug mode ("debug": true inside ircOptions, NODE_ENV=development as an environment variable), and if so does it show any output that might explain the issue?
@PWKad: I'm not sure what you're trying to indicate with a thumbs up, or why you did it as a separate message – GitHub has a reactions thing you can use to put a thumbs up on the message itself, or if you're also encountering the issue you could explain that (and hopefully add some more details to help track down the issue)?

Looks like I'm actually on 2.4.2, not sure if there was a fix for this issue in 2.5.0 but I can update and test.

I'm on my phone and reactions don't show up here.  Sorry for the double notification

There probably was not, but the upstream IRC library had a lot of updates, especially around things testing its core functionality, so it's possible it did. (I don't recall all the changes I made.)

Seems that 2.5.0 actually broke the bot completely for me...
$ discord-irc --config config.json
info: Connected to Discord
info: Connected to IRC
18 Nov 23:41:27 - ERROR: { prefix: 'verne.freenode.net',
  server: 'verne.freenode.net',
  command: 'err_cannotsendtochan',
  rawCommand: '404',
  commandType: 'error',
  args: [ 'SGDiscordBot', '#litecoin-dev', 'Cannot send to channel' ] }
error: Received error event from IRC prefix=verne.freenode.net, server=verne.freenode.net, command=err_cannotsendtochan, rawCommand=404, commandType=error, args=[SGDiscordBot, #litecoin-dev, Cannot send to channel]
18 Nov 23:41:27 - ERROR: Unhandled message: { prefix: 'verne.freenode.net',
  server: 'verne.freenode.net',
  command: '671',
  rawCommand: '671',
  commandType: 'normal',
  args:
   [ 'SGDiscordBot',
     'SGDiscordBot',
     'is using a secure connection' ] }
18 Nov 23:41:27 - ERROR: Unhandled message: { prefix: 'services.',
  server: 'services.',
  command: '328',
  rawCommand: '328',
  commandType: 'normal',
  args: [ 'SGDiscordBot', '#litecoin', 'https://litecoin.org' ] }

Update: false alarm. it's just not printing as verbosely as it did in 2.4

Right – 2.4 had a bug (again with the upstream IRC library) that accidentally put it into permanent debug mode, and it's probably no longer doing that. If you put it in debug mode ("debug": true inside ircOptions, NODE_ENV=development as an environment variable), instead of it just being there through a glitch, then this time you might get more out of it if/when it fails. (I'm not sure.)

I experienced the same issue with version 2.5.0 installed from npm (with Node version 8.9.2). I've set the recommended debug options and will report back if it happens to me again.

Well, this issue has finally happened again, and this time with the recommended debug options enabled 🎉 . Here is the console output (the configuration errors were from earlier runs), configuration and the output log (sensitive information removed where necessary). Based on the log, it seems that the IRC stopped getting pinged around January 29th, and reconnected multiple times the day before. I hope this helps!

@scribblemaniac It ended up reconnecting 10 times, then stopped, which is part of the standard configuration. If you want to stop that, you could change it in the ircOptions – it seems kinda unintuitive to set it to that, to me, but it's currently that in the code. (Adding "retryCount": null inside "ircOptions": { … } in your config should work.)
The logs seem to be outputting pings twice, and it looks like it's connecting twice. Are you running the bot with two separate configurations, or running it twice, or is this erroneous? There were issues in the past with it connecting twice even from just one bot, is why I ask.

It ended up reconnecting 10 times, then stopped, which is part of the standard configuration.

I will add the workaround you suggested, but issue still remains: why it's disconnecting/reconnecting in the first place? It's not a network issue on my side because the connection to the Discord server is fine even while there are issues with the IRC connection. Freenode wasn't down either or both connections would've failed at the same time.

Separately, the logs seem to be outputting pings twice, and it looks like it's connecting twice. Are you running the bot with two separate configurations, or running it twice, or is this erroneous?

In the configuration you'll see that I have it listening to two different IRC channels, so this is expected behavior I imagine.

You're right, I suppose there is a window of time where the IRC could fail but a Discord heartbeat would not be sent. The high latency (5352ms) in the Discord heartbeat after the IRC fail also points to a network issue.
However, looking at the logs, it looks different when I purposefully disconnect from the network than when this issue occurs. Consider the last IRC messages:
29 Jan 00:08:35 - CyclingPingTimer 21: server silent for too long, let's send a PING
29 Jan 00:08:35 - SEND: PING 9194
29 Jan 00:08:35 - Received: :barjavel.freenode.net PONG barjavel.freenode.net :9194
29 Jan 00:08:50 - CyclingPingTimer 21: server silent for too long, let's send a PING
29 Jan 00:08:50 - SEND: PING 9195
29 Jan 00:08:58 - CyclingPingTimer 21: ping timer stopped
29 Jan 00:08:58 - CyclingPingTimer 21: ping timeout!
29 Jan 00:08:58 - Connection got "close" event
29 Jan 00:08:58 - Disconnected: reconnecting
29 Jan 00:08:58 - Maximum retry count (10) reached. Aborting

The note that there is a successful IRC ping sent at 00:08:35, then one timeout fail at 00:08:58, and then the connection is aborted. When I disconnect manually, it looks more like this:
9 Feb 13:13:59 - CyclingPingTimer 1: server silent for too long, let's send a PING
9 Feb 13:13:59 - SEND: PING 8287
9 Feb 13:14:04 - CyclingPingTimer 0: ping timer stopped
9 Feb 13:14:04 - CyclingPingTimer 0: ping timeout!
9 Feb 13:14:04 - Connection got "close" event
9 Feb 13:14:04 - Disconnected: reconnecting
9 Feb 13:14:04 - Waiting 2000ms before retrying
9 Feb 13:14:06 - Attempting socket connection to IRC server
9 Feb 13:14:06 - Network error: Error: getaddrinfo ENOTFOUND chat.freenode.net chat.freenode.net:6667
9 Feb 13:14:06 - Connection got "close" event
9 Feb 13:14:06 - Disconnected: reconnecting
9 Feb 13:14:06 - Waiting 2000ms before retrying
.
. (a bunch more attempts to reconnect)
.
9 Feb 13:14:27 - Attempting socket connection to IRC server
9 Feb 13:14:27 - Network error: Error: getaddrinfo ENOTFOUND chat.freenode.net chat.freenode.net:6667
9 Feb 13:14:27 - Connection got "close" event
9 Feb 13:14:27 - Disconnected: reconnecting
9 Feb 13:14:27 - Maximum retry count (10) reached. Aborting

Is there a counter for IRC ping fails that is perhaps not being reset? Because it doesn't look like it has actually retried 10 times. It's also a little bit suspicious that the two IRC connections do not fail at the same time: the first one fails at 28 Jan 23:46:23, and the second one 26 minutes later at 29 Jan 00:08:58. If my network or freenode was down, wouldn't they both fail at the same time?

This complicates things, but is still explainable without it being a bug, I think.

Is there a counter for IRC ping fails that is perhaps not being reset? Because it doesn't look like it has actually retried 10 times.

It does seem to have retried approximately 10 times, actually, for each. The number after "CyclingPingTimer" in the logs is a number that counts up for each connection. As it's at 21, it seems both instances of the bot have tried reconnecting about 10 times (these numbers start at 0 and 1, because there are two connections, and then if the bots both reconnect 10 times sequentially, these increase to 20 and 21). As far as I know, that only increases when it reconnects, so it seems the connection has reconnected ten times, per instance, over the entire course of the bot's running. This might not be expected behavior, but I think it's intended behavior. (I'm not sure, I didn't write the upstream IRC library originally.)

However, looking at the logs, it looks different when I purposefully disconnect from the network than when this issue occurs.

The error in the logs is due to an ENOTFOUND error, which I believe only occurs if it can't look up the DNS of the server when it tries to connect. This therefore doesn't occur if freenode is down, or if your DNS can resolve when the bot tries to reconnect; it does occur if your network is down. I'm not completely sure, but I think it doesn't suggest anything wrong with the behavior.

It's also a little bit suspicious that the two IRC connections do not fail at the same time: the first one fails at 28 Jan 23:46:23, and the second one 26 minutes later at 29 Jan 00:08:58. If my network or freenode was down, wouldn't they both fail at the same time?

I don't think so. The bots reset their ping timer upon receiving messages from the server, so if they connect at different times, or take differing lengths to get ping responses or keep track of different messages in different channels, the timers could get out of sync and the network could go down when one is looking for a ping and return before the other does. That is, the following could happen:
00:00: Bot 1 gets a message from the server
00:13: Bot 2 gets a message from the server
00:14: -- Network connection breaks --
00:15: Bot 1 requests a ping, as it's been 15 seconds since its last message
00:23: Bot 1 times out, as it's been 8 seconds since it pinged
00:24: -- Network connection returns --
00:28: Bot 2 requests a ping, as it's been 15 seconds since its last message
00:28: Bot 2 receives a pong, and so doesn't time out

Due to them being out of sync, and the ping cycle and ping timeout times being different, the network could have gone down just long enough to cause a timeout on one while the other bot doesn't even notice a problem. Increasing these values should help stop this problem.

Okay, everything does seem to be working as intended. I don't necessarily agree with the design of retryCount, but at any rate I can see now that that would be an upstream issue. Thank you very much @Throne3d for taking the time to address all of my concerns, I hope I was not too much of a bother 😞 . For now I've set retryCount to null and I'll see how that goes. 👍

That's fine! I agree, the behavior of retryCount is confusing, and I might look into changing it – I was under the impression that was specifically not how it worked. We might want to remove the config default from discord-irc, at least, since it's weird for a bot that should have a good uptime and stay online for very long periods of time, so people are likely to encounter this issue with normal use.

this issue persists 3 years later
on a irc relay bot, it somehow disconnected from the irc and i had to manually restart it.
i am not going to login to a server everyday just to restart some random bot that's going to fail 12 hours later.

this issue persists 3 years later
on a irc relay bot, it somehow disconnected from the irc and i had to manually restart it.
i am not going to login to a server everyday just to restart some random bot that's going to fail 12 hours later.

@chaziz94 The comments above explain how you might work around this – you can set "ircOptions": {"retryCount": null} inside your configuration :^)
If that doesn't work, you could always come back with some logs of the issue (or if possible a repro), so I could look into solving it, rather than complaining about it remaining unsolved :P

@chaziz94 The comments above explain how you might work around this – you can set "ircOptions": {"retryCount": null} inside your configuration :^)
If that doesn't work, you could always come back with some logs of the issue (or if possible a repro), so I could look into solving it, rather than complaining about it remaining unsolved :P

Had same issue (disconnects every 10+ minutes and reconnects only 10 times).
Suggested settings seem to solve the problem, I used these in [code]ircOptions[/code] section:
[code]"millisecondsOfSilenceBeforePingSent": 20000,
"millisecondsBeforePingTimeout": 30000,
"retryCount": null,[/code]

Had same issue (disconnects every 10+ minutes and reconnects only 10 times).
Suggested settings seem to solve the problem, I used these in [code]ircOptions[/code] section:
[code]"millisecondsOfSilenceBeforePingSent": 20000,
"millisecondsBeforePingTimeout": 30000,
"retryCount": null,[/code]

github doesn't support bbcode

I was confused because the default retryCount is already null according to irc-upd, the underlying IRC library used. However, discord-irc overrides this default to 10, causing an irrecoverable disconnect as it gives up reconnecting if it loses the network for half a minute or so. Adding "retryCount": null indeed fixes this. The parameter retryDelay also defines how long to wait between retries, defaulting to 2000ms.


[KW-Search] Remove backwards compatibility for new documents/tables API

See here
Depends on #8494 #8493
When breaking compatibility - i.e. making title and mime_type mandatory - we should also :

enforce the check on parent_id , which can be null iff parents.length is 1  ( https://github.com/dust-tt/dust/pull/9261/files#r1880230852 ), so enforcing valid value for parent_id
rename parents to parent_ids
enforce the check on document_id === parent_ids[0] and remove the automated fill when parents === []

This one will span over a month, and is not necessary for the ship.
Will move to Olympus and assign to me

First todo is to send an email to users of v1
Our own connectors is already good
The fixing PR should do all mentioned above
Note: logging is a first step to detect all potential holes and fix them; it was done by https://github.com/dust-tt/dust/issues/9380

final v1 design question: should we ask people to send with document_id = parent in API v1?


UILabel Preview Example not working

UILabel Preview Example in project SwiftUIKitExample.xcodeproj does not work.
        UILabel() // <- This is a `UIKit` view.
            .swiftUIView(layout: .intrinsic) // <- This is a SwiftUI `View`.
            .set(\.text, to: "Hello, UIKit!") // <- Use key paths for updates.
            .fixedSize() // <- Make sure the size is set
            .previewLayout(.sizeThatFits)
            .previewDisplayName("UILabel Preview Example")
    }

Expected to see the text "Hello, UIKit!" but that text is not visible in the preview :(
https://user-images.githubusercontent.com/4176826/193900800-b6b01da1-5898-43b8-a3a8-2606e380f04c.mov
Tested with Xcode 14.0.1 and iPhone 13 (iOS 16) simulator

I think it's working on Xcode 14.


Detected as MATPLOTLIB._TRI.PYD, W32.Worm.Butterfly by Webroot

I've not determined if this is false positive or real detection - simply reporting it.

Must be something with your local matplotlib. The code here uses matplotlib properly and is not malicious.

I just used your Windows binary. I don't have matplotlib installed locally, apart from the one in this project binary. Thanks


Fix broken by upgrade UI

Some components have broken styling due to upgrade of styled-components package.
Example:

It turns out that due to the upgrade of styled-components we have got a conflict between media styles and normal styles for some reusable components like Hero, Divider, CardContainer, etc.
It seems like it's better to roll back the version of styled-components to 2.4.1.


release 2.1.0

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: tlmquintino:x: cladinapoliYou have signed the CLA already but the status is still pending? Let us recheck it.


[BUG] ndkports: Pull Request: Create parent directory before trying to extract file from dependency aar

Hello,
Where could I send patches for ndkports?
Current HEAD doesn't build:
# git clone https://android.googlesource.com/platform/tools/ndkports
Cloning into 'ndkports'...
remote: Total 1790 (delta 127), reused 1790 (delta 127)
Receiving objects: 100% (1790/1790), 103.46 MiB | 10.85 MiB/s, done.
Resolving deltas: 100% (127/127), done.

# cd ndkports
# ./gradlew build -PndkPath=~/android-sdk/ndk/25.2.9519653/
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :curl:prefab FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':curl:prefab'.
> java.io.FileNotFoundException: /home/vilius/ndkports/curl/build/port/dependencies/openssl-1.1.1s-SNAPSHOT/AndroidManifest.xml (No such file or directory)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2m 33s
17 actionable tasks: 17 executed

Build breaks because dependency extraction logic tries to extract files into subdirectories that aren't created yet:
ls -l /home/vilius/ndkports/curl/build/port/dependencies/
total 4
drwxr-xr-x 2 user user 4096 Sep 24 01:29 generated

Current code
buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt has logic to create subdirectories, but it doesn't work, because the first entry is in the aar file is AndroidManifest.xml which doesn't trigger the if (entry.isDirectory) { codepath.
    private fun extract(aar: File, extractDir: File) {
        ZipFile(aar).use { zip ->
            zip.entries().asSequence().forEach { entry ->
                zip.getInputStream(entry).use { input ->
                    val outFile = extractDir.resolve(entry.name)
                    if (entry.isDirectory) {
                        outFile.mkdirs()
                    } else {
                        outFile.outputStream().use { output ->
                            input.copyTo(output)
                        }
                    }
                }
            }
        }
    }

The patch
I wasn't sure if java.util.zip.ZipFile entries are 100% guaranteed to come in hierarchical order, parent first, so I removed ZipFile directory processing entirely and created those same directories when processing ZipFile not-directory entries.
From 1c5f84b2370c319bcd1ec0b8a2312a3959b70815 Mon Sep 17 00:00:00 2001
From: Vilius Sutkus '89<EMAIL_ADDRESS>Date: Sun, 24 Sep 2023 02:24:58 +0300
Subject: Create parent directories before extracting file from dependency aar

diff --git a/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt b/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt
index d58f1fa..c02c46e 100644
--- a/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt
+++ b/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt
@@ -64,18 +64,17 @@ abstract class PrefabTask : DefaultTask() {

     private fun extract(aar: File, extractDir: File) {
         ZipFile(aar).use { zip ->
-            zip.entries().asSequence().forEach { entry ->
-                zip.getInputStream(entry).use { input ->
-                    val outFile = extractDir.resolve(entry.name)
-                    if (entry.isDirectory) {
-                        outFile.mkdirs()
-                    } else {
+            zip.entries().asSequence()
+                .filter { ! it.isDirectory }
+                .forEach { entry ->
+                    zip.getInputStream(entry).use { input ->
+                        val outFile = extractDir.resolve(entry.name)
+                        outFile.parentFile.mkdirs()
                         outFile.outputStream().use { output ->
                             input.copyTo(output)
                         }
                     }
                 }
-            }
         }
     }

The patch can be simpler and do less unnecessary IO with mkdirs if the ZipFile is guaranteed to supply entries in hierarchical order, parent first:
From 44ddef576be7e3af186f72948da93de6593a6c72 Mon Sep 17 00:00:00 2001
From: Vilius Sutkus '89<EMAIL_ADDRESS>Date: Sun, 24 Sep 2023 03:02:20 +0300
Subject: Create parent directories before extracting file from dependency aar

diff --git a/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt b/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt
index d58f1fa..fe11316 100644
--- a/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt
+++ b/buildSrc/src/main/kotlin/com/android/ndkports/PrefabTask.kt
@@ -64,11 +64,12 @@ abstract class PrefabTask : DefaultTask() {

     private fun extract(aar: File, extractDir: File) {
         ZipFile(aar).use { zip ->
+            extractDir.mkdirs()
             zip.entries().asSequence().forEach { entry ->
                 zip.getInputStream(entry).use { input ->
                     val outFile = extractDir.resolve(entry.name)
                     if (entry.isDirectory) {
-                        outFile.mkdirs()
+                        outFile.mkdir()
                     } else {
                         outFile.outputStream().use { output ->
                             input.copyTo(output)

Regards,
Vilius

Mine is still building but I think I've gotten past curl already and I haven't seen any errors. It might be that the iteration order of the zip is based on the insertion order when it was created, which would probably be based on file system order. Whatever the case, upload the patch to gerrit and we'll get it merged.
https://gerrit-review.googlesource.com/Documentation/intro-gerrit-walkthrough-github.html is probably much more useful than the AOSP docs I linked earlier. You do not need a full AOSP tree. git push $WHATEVER_YOUR_REMOTE_IS HEAD:refs/for/master is most likely what you need.


'Refer header required'

{ [bad_request: Something wrong with the request]
status: 400,
name: 'bad_request',
message: 'Something wrong with the request',
error: true,
reason: 'Referer header required.' }
I get that error after installing this plugin.
Tried updating node, puchdb, everything, etc. and still this error. Normal requests work fine.

can anyone help?
im using express as a restful API. and a fresh install ubuntu 14.04 and couchdb 1.6.1
========================================
{ email<EMAIL_ADDRESS>password: 'cumonnow',
_id: 'lol' }
pouchdb:api http://admin:admin@localhost:5984/report_users +1ms request { method: 'POST',
url: '_find',
body:
{ selector: { email<EMAIL_ADDRESS>},
fields: [ '_id', 'email' ] } }
PUT /register 200 17.804 ms - 4
pouchdb:api http://admin:admin@localhost:5984/report_users +8ms request error { [bad_request: Something wrong with the request]
status: 400,
name: 'bad_request',
message: 'Something wrong with the request',
error: true,
reason: 'Referer header required.' }
{ [bad_request: Something wrong with the request]
status: 400,
name: 'bad_request',
message: 'Something wrong with the request',
error: true,
reason: 'Referer header required.' }

It seems as though I can't save/create a design doc with node-cradle either.
Is there a way to inject a dummy referer header into every request somehow? Someone please help :(

This plugin does not work with CouchDB, only with Cloudant. If you try it against CouchDB you will see a weird error about "referer header required."


[HOPS-1541] provenance

Make sure there is no duplicate PR for this issue

Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added and passed (for bug fixes / features)
[*] HOPS JIRA issue has been opened for this PR
[*] All commits have been squashed down to a single commit
[*] The commit message has the following format: [HOPS-XXX] message

Post a link to the associated JIRA issue
https://hopshadoop.atlassian.net/browse/HOPS-1541

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
feature - provenance

What is the new behavior (if this is a feature change)?
Log filesystem operations into a log tailed by epipe

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
This JIRA has changes to the database (add tables)

Other information:

Can one of the admins verify this patch?

Test tests are passing.

From what I understand the changes look good to me. I let you see with @maismail for his remark.
I'll merge when @maismail has aproved and the tests have passed.


Ability to get stack region & account ID from backend resources

Environment information
System:
  OS: macOS 15.1.1
  CPU: (12) arm64 Apple M2 Pro
  Memory: 137.39 MB / 32.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 20.12.2 - ~/.local/state/fnm_multishells/13709_1732902102431/bin/node
  Yarn: 1.22.22 - ~/.local/state/fnm_multishells/13709_1732902102431/bin/yarn
  npm: 10.5.0 - ~/.local/state/fnm_multishells/13709_1732902102431/bin/npm
  pnpm: 9.6.0 - ~/.local/state/fnm_multishells/13709_1732902102431/bin/pnpm
NPM Packages:
  @aws-amplify/auth-construct: 1.5.0
  @aws-amplify/backend: 1.8.0
  @aws-amplify/backend-auth: 1.4.1
  @aws-amplify/backend-cli: 1.4.2
  @aws-amplify/backend-data: 1.2.1
  @aws-amplify/backend-deployer: 1.1.10
  @aws-amplify/backend-function: 1.8.0
  @aws-amplify/backend-output-schemas: 1.4.0
  @aws-amplify/backend-output-storage: 1.1.3
  @aws-amplify/backend-secret: 1.1.5
  @aws-amplify/backend-storage: 1.2.3
  @aws-amplify/cli-core: 1.2.0
  @aws-amplify/client-config: 1.5.2
  @aws-amplify/deployed-backend-client: 1.4.2
  @aws-amplify/form-generator: 1.0.3
  @aws-amplify/model-generator: 1.0.9
  @aws-amplify/platform-core: 1.2.2
  @aws-amplify/plugin-types: 1.5.0
  @aws-amplify/sandbox: 1.2.6
  @aws-amplify/schema-generator: 1.2.5
  aws-amplify: 6.10.0
  aws-cdk: 2.171.1
  aws-cdk-lib: 2.171.1
  typescript: 5.7.2
CDK environment variables:
  CDK_DOCKER = finch

Describe the feature
When working with many CDK resources, it's useful to have access to the region and account ID of the stack where I'm deploying my Amplify app.
Hardcoding these values can be problematic at times, so ideally I'd like to be able to interpolate these values, similar to what I can do already in CDK.
Ideally, I'd like to be able to do something like this or similar to this:
import { defineFunction, stack } from "@aws-amplify/backend";

export const myFunction = defineFunction({
  name: "my-function",
  layers: {
   "@aws-lambda-powertools/*":
      `arn:aws:lambda:${stack.region}:${stack.account}:layer:myLayer:12`,
  },
});

alternatively, I'd like to be able to use a regular CloudFormation variable and have Amplify leave it as-is in the final stack:
import { defineFunction, stack } from "@aws-amplify/backend";

export const myFunction = defineFunction({
  name: "my-function",
  layers: {
   "@aws-lambda-powertools/*":
      "arn:aws:lambda:\${AWS::Region}:123456789012:layer:myLayer:12"
  },
});

Use case
For example, when working with Lambda layers, customers need to specify the ARN of the layer they want to attach to a function. These ARNs need to be in the same region where the function is being deployed.
Hardcoding the region makes the stack not portable.

Thank you for filing this, marking this as feature request for further improvements.


Add support for rollup-plugin-node-resolve's browser option

Hi and thanks for a great and easy to use module! A perfect preconfigured rollup for me :)
This PR adds support for rollup-plugin-node-resolve's browser option, to be able to respect a package.json's browser field.

looks good 🍺

um, maybe we can make this default to true? since webpack does 😗

Oh, forgot that this option already defaults to true in rollup-plugin-node-resolve

I would be 👍 for true as default. I'm afraid rollup-plugin-node-resolve does not default to true, as stated in the readme:
browser: true,  // Default: false


Using url() to build links for the home page campaign tiles

What does this PR do?

The url() function permits language and prefix to be passed in as
options to accomodate our url structure.

Where should the reviewer start?
Look at the URLs on the home page for the campaign tiles to make sure they have the appropriate prefix

The tile pattern is also used on the "campaign group" node (e.g. /volunteer/animals) so it's worth checking that as well! (Not that it looks like it'll break anything, just as a heads up!)

@DFurnes Thanks for the heads up. There's also a similar construct in the campaign-block.tpl.php, but I'm not sure where that's used. Would you happen to know?

Ack, I think that's just old code. :grimacing: Should be totally safe to delete.

@DFurnes Glad you brought up /volunteer/animals, b/c it's not working properly now on Thor with the 'br' or 'mx' preview. I think it's a separate ticket though b/c the URLs work, but the images are what's broken.
![alt text][screenshot]
[screenshot]: https://s3.amazonaws.com/f.cl.ly/items/452R1K3R131y1i0R2l2v/Image 2015-10-01 at 1.18.25 PM.png "Screenshot"

kewl :+1:


Initialized spark session with  “configure_spark_with_delta_pip” stuck

Bug
I am working on testing new features for deltalake 2.0 and facing below issue while trying to initialized spark session with  “configure_spark_with_delta_pip”. Any advice on what might be missing would be greatly help and appreciated.
Spark : 3.2
slack:  https://delta-users.slack.com/archives/CJ70UCSHM/p1658850509055239.
Steps to reproduce
import pyspark
from delta import *
builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
.config("spark.jars","gs://xxxxx/g1/jars/delta-core_2.12-2.0.0.jar")
)
spark = configure_spark_with_delta_pip(builder).getOrCreate()

Observed results

Expected results

Further details

Environment information

Delta Lake version: 2.0
Spark version: 3.2
Scala version: 2.12
Python Version : 3.8

Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

[ ] Yes. I can contribute a fix for this bug independently.
[ ] Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
[ ] No. I cannot contribute a bug fix at this time.

Thanks @BShraman - will try to repro this locally.  Quick question - for .config("spark.jars","gs://xxxxx/g1/jars/delta-core_2.12-2.0.0.jar") by any chance have you tested this locally and still get the same error?  Per the Slack thread, wanted to re-iterate that this is happening to delta-core_2.12-2.0.0.jar but not delta-core_2.12-1.2.0.jar.  Thanks!

Thanks @BShraman - will try to repro this locally. Quick question - for .config("spark.jars","gs://xxxxx/g1/jars/delta-core_2.12-2.0.0.jar") by any chance have you tested this locally and still get the same error? Per the Slack thread, wanted to re-iterate that this is happening to delta-core_2.12-2.0.0.jar but not delta-core_2.12-1.2.0.jar. Thanks!

Hi @dennyglee  , if i run this locally with ".config("spark.jars.packages","io.delta:delta-core_2.12:2.0.0")" it is pulling dependency from maven repository and everything runs without any issue. Fetching dependency directly from maven is blocked in our organization.
Alternate option i am trying with passing delta jar explicitly after downloading locally and this is where i am facing an issue.

Based on some quick testing, I believe you may have delta-spark==1.2.0 or lower installed in your environment.  Could you try upgrading it to delta-spark==2.0.0 and see if that helps?   Note, I'm still investigating another issue related to this.
Details
Running the following commands directly from the local python shell.
Local environment
Local installs using sdkman Spark, Java, and Scala version management  and virtual environment for Python version management.

Spark 3.2.0
Python 3.9.10
Scala 2.12.8

With delta-spark==1.2.0 installed
i.e. with pip install delta-spark==1.2.0
Running with Delta 1.2
import pyspark
from delta import *

my_packages = ["io.delta:delta-core_2.12:2.0.0"]

builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
)

spark = configure_spark_with_delta_pip(builder, extra_packages=my_packages).getOrCreate()

Result: Works as expected
Running with Delta 2.0
import pyspark
from delta import *

my_packages = ["io.delta:delta-core_2.12:2.0.0"]

builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
)

spark = configure_spark_with_delta_pip(builder, extra_packages=my_packages).getOrCreate()

Result: Fails with the following error (which is similar to your error)
io.delta#delta-core_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-34819e71-fc19-4aac-8ebd-fab64b77ba27;1.0
	confs: [default]
	found io.delta#delta-core_2.12;2.0.0 in central
	found io.delta#delta-storage;2.0.0 in central
	found org.antlr#antlr4-runtime;4.8 in central
	found org.codehaus.jackson#jackson-core-asl;1.9.13 in spark-list
:: resolution report :: resolve 118ms :: artifacts dl 4ms
	:: modules in use:
	io.delta#delta-core_2.12;2.0.0 from central in [default]
	io.delta#delta-storage;2.0.0 from central in [default]
	org.antlr#antlr4-runtime;4.8 from central in [default]
	org.codehaus.jackson#jackson-core-asl;1.9.13 from spark-list in [default]
	:: evicted modules:
	io.delta#delta-core_2.12;1.2.0 by [io.delta#delta-core_2.12;2.0.0] in [default]
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   5   |   0   |   0   |   1   ||   3   |   0   |
	---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-34819e71-fc19-4aac-8ebd-fab64b77ba27
	confs: [default]
	0 artifacts copied, 3 already retrieved (0kB/3ms)
22/08/07 18:06:39 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/08/07 18:06:40 WARN SparkSession: Cannot use io.delta.sql.DeltaSparkSessionExtension to configure session extensions.

With delta-spark==2.0.0 installed
i.e. with pip install delta-spark==2.0.0
import pyspark
from delta import *

my_packages = ["io.delta:delta-core_2.12:2.0.0"]

builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
)

spark = configure_spark_with_delta_pip(builder, extra_packages=my_packages).getOrCreate()

Result:: Works as expected

@BShraman - I'm happy to help you figure this one out.

Alternate option : i am trying with passing delta jar explicitly after downloading locally and this is where i am facing an issue.

Yea, I don't think you need to pass the Delta JAR in if you've already downloaded locally.
Here's your code snippet with some comments:
import pyspark
# this next line shows that you already have Delta installed.  If you didn't have Delta installed, this line would error out.
from delta import *

builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
.config("spark.jars","gs://xxxxx/g1/jars/delta-core_2.12-2.0.0.jar") # I don't think you need this because you already have Delta installed
)

spark = configure_spark_with_delta_pip(builder).getOrCreate()

Does this work?
import pyspark
from delta import *

builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
)

spark = configure_spark_with_delta_pip(builder).getOrCreate()

Let me know if this sorts out the issue for you.  I am happy to keep helping till you figure this out.

This is the draft PR I created to experiment with this BTW.  I was getting the same warning errors Denny got when I mixed & matched my locally installed Delta version with a JAR file that had a different version (e.g. when I had Delta 1.2 installed locally and attached the Delta 2.0 JAR file).

Could we also ship delta-spark in a self contained fashion? Having extra dependency on maven is not as seamless in many pyspark dependent orgs.

@santosh-d3vpl3x - It's a great idea and that's what's been done.  delta-spark is in PyPi and doesn't have a Maven dependency.  You can just run pip install delta-spark.

@MrPowers Thats good to know. https://docs.delta.io/latest/quick-start.html#python along with https://github.com/delta-io/delta/blob/v2.1.0rc1/python/delta/pip_utils.py#L26-L28 gave me an impression that delta-spark is just a sugar coating over existing flow.

@santosh-d3vpl3x - Yea, we should make those installation instructions more clear.  Here's a blog post I wrote on installing PySpark & Delta Lake with conda.  I will try to make more content that'll make it easier to get up-and-running with Delta Lake.  Thanks for the comments / feedback.

@MrPowers I Spent some time going through the code for delta-spark to understand what it is doing at the moment.

It is indeed a thin wrapper and tool for convenience but not a self complete package like pyspark. configure_spark_with_delta_pip adds maven coordinate to the spark session builder. If you have clean ivy dir/maven dir and disconnected from the internet then the instruction here don't work.
2 . Also to check if we need the configure_spark_with_delta_pip call, excluding configure_spark_with_delta_pip indicates:

import pyspark
from delta import *

builder = pyspark.sql.SparkSession.builder.appName("MyApp") \
    .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension") \
    .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")
    .getOrCreate()

Results in java.lang.ClassNotFoundException: io.delta.sql.DeltaSparkSessionExtension

@MrPowers
Here are my pip list

Spark Configuration :
#####################
import pyspark
from delta import *
builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
.config("spark.jars","gs://xxxxx/g1/jars/delta-core_2.12-2.0.0.jar")
)
spark = configure_spark_with_delta_pip(builder).getOrCreate()
############
import pyspark
from delta import *
my_packages = ["io.delta:delta-core_2.12:2.0.0"]
builder = (
pyspark.sql.SparkSession.builder.appName("MyApp")
.config('spark.master','local[*]')
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog","org.apache.spark.sql.delta.catalog.DeltaCatalog")
)
spark = configure_spark_with_delta_pip(builder, extra_packages=my_packages).getOrCreate()
Both are giving me same error as below :
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.2.2.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
:: loading settings :: url = jar:file:/opt/spark/jars/ivy-2.5.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
Ivy Default Cache set to: /home/xxxxx/.ivy2/cache
The jars for the packages stored in: /home/xxxxx/.ivy2/jars
io.delta#delta-core_2.12 added as a dependency
io.delta#delta-core_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-9f5c2d62-314f-4857-a764-d72116906ec7;1.0
confs: [default]

Some users can't access JAR files via Maven.  It would be cool if we could upload the JAR files to PyPi and give an option to fetch the relevant JAR from PiPy for users that don't have access to Maven.

I use the following conf, solved the problem, and works perfectly
#Create a spark Context class, with custom config
conf = SparkConf()
#conf.set('spark.sql.debug.maxToStringFields', 100)
conf.set('spark.default.parallelism', 700)
conf.set('spark.sql.shuffle.partitions', 700)
conf.set('spark.driver.memory', '30g')
conf.set('spark.driver.cores', 8)
conf.set('spark.executor.cores', 8)
conf.set('spark.executor.memory', '30g')
conf.set("spark.jars.packages", "io.delta:delta-core_2.12:2.0.0")
conf.set('spark.sql.extensions', 'io.delta.sql.DeltaSparkSessionExtension')
conf.set('spark.sql.catalog.spark_catalog', 'org.apache.spark.sql.delta.catalog.DeltaCatalog')
sc = SparkContext.getOrCreate(conf)
#import library
from delta import *
#we need to config sparksession
my_packages = ["io.delta:delta-core_2.12:2.0.0"]
builder = (pyspark.sql.SparkSession.builder.appName("Myapp")
.config('spark.master','local[*]'))
spark = configure_spark_with_delta_pip(builder, extra_packages=my_packages).getOrCreate()
my spark env:
delta-spark                   2.0.0
findspark                     2.0.1
pyspark                       3.2.3
sparksql-magic                0.0.3

my spark env:
delta-spark 2.0.0
pyspark 3.2.2
for anyone have problems with pyspark structure streaming read kafka and write to delta, this is my config:
my_packages = ["io.delta:delta-core_2.12:2.0.0", "org.apache.spark:spark-sql-kafka-0-10_2.12:3.2.2"]
builder = (
SparkSession.builder.master("local[*]")
.config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.config(
"spark.sql.catalog.spark_catalog",
"org.apache.spark.sql.delta.catalog.DeltaCatalog",
)
)
spark = configure_spark_with_delta_pip(
builder, extra_packages=my_packages
).getOrCreate()


draft: [dir 458] webkit workaround

Please do not merge, we should discuss this first.
See the comments in the file.
This will "force" clicks only in the context where this is needed (when tests run locally on webkit).
However, I am not really comfortable with the logical complexity this adds to the tests. I think I'd rather keep them simple and live with tests failing locally in webkit until we know more.

closed - misnamed branch, will make a new one.


Upgraded scalacheck-ops to 1.3.0 and build plugins

@egprentice @vodopis @chrissalij-r @audax-shreyaspurohit
I'm creating this version to fix the scalacheck-ops dependency in rally.

Coverage increased (+0.2%) to 67.303% when pulling 74a71e06579d9400815624e70c75f8bd0282f692 on upgrade-scalacheck-ops into 081552f1f14d1daed2db5e34f7603d2e8f725dd3 on master.

Coverage increased (+0.2%) to 67.303% when pulling 74a71e06579d9400815624e70c75f8bd0282f692 on upgrade-scalacheck-ops into 081552f1f14d1daed2db5e34f7603d2e8f725dd3 on master.


Not able to use as Could not find a declaration file for module 'use-react-router-breadcrumbs'.

Could not find a declaration file for module 'use-react-router-breadcrumbs'. 'c:/git/node_modules/use-react-router-breadcrumbs/dist/cjs/index.js' implicitly has an 'any' type.

Thanks @jocgaurav this will be fixed when the PR above is merged/deployed to v 1.0.2


Multi bus topology test

Construct a virtual network that has 2 network buses.
Add parameters to topology test that puts cluster traffic on a separate bus to ingress.
Add CI build target for topology tests.

LGTM.

LGTM.


abc.go.com

The past day imasdk.googleapis.com seems to be blocking video streaming. Ublock filter had white-listed this for abc.go.com but is seems the all-in-one list is overriding it. Unblocking imasdk.googleapis.com allows video streaming but now the stream / search bar is not visible.
Some relation to #54

I will try to solve this issue too in version 2.8.


Make TOS and Privacy Policy pages

Make it simple

This was done awhile ago, but it is in the 10/12/2022 release and beyond.


run_all_metrics: detect script path

Add SCRIPT_PATH to run_all_metrics.sh script so it can execute
tests successfully from any location.
Fixes #168
Signed-off-by: Brett T. Warden<EMAIL_ADDRESS>

lgtm
just FYI I added #172 and assigned to you as well as I ran into it whilst testing this :-)
And it looks like you ran into the annoying Travis 'cannot find SHA' issue. Try a minor mod and force re-push and see if that clears it...


Make use of newer API for darklaf installation.

Description

Newer versions of Darklaf offer to get a list of all available themes.
This PR replaces the manual setup of each Theme with iteration of said list.

Instead of checking the name of a Theme to determine whether it is dark make use of the Theme#isDark method.

Darklaf version 2.4.4 fixes an issue, where viewing a JList could freeze the application. It also fixes misplaced tree hint popups (the popups that reveal the complete content of a tree cell if obstructed) on Windows.

Checklist:

[x] My code follows the [code style][style-guide] of this project.
[ ] I have updated the documentation accordingly.

Thanks for the nice patch.


Support orientation on custom via points

Hello,
The ability to send custom via points is great to force a robot to pass through predefined set of points.
Via points are currently defined only by their x and y. It would be great to able to constraints the robot to pass through these points with a specific orientation.
How hard that would be to implement this behavior in teb_local_planner? Any pointers. I may try.
Best

Hi,
this might be a good idea and I don't think that it is hard to implement.
Just create a copy of the via point cost function (edge)  (in edge_via_point.h).
Change the edge to a 2 dimensional error and to accept a PoseSE2 as via-point:
class EdgeViaPointSE2 : public BaseTebUnaryEdge<2, const PoseSE2*, VertexPose> {...}

The error might then be computed using the following distance metric:
 _error[0] = (bandpt->position() - _measurement->position()).norm(); // as in EdgeViaPoint
 _error[1] = g2o::normalize_theta(bandpt->theta() - _measurement->theta());

Afterward, you must add this particular edge creation case to optimal_planner.cpp.
We could think of changing the API, in particular ViaPointContainer to a container of PoseSE2.
If you need further hints, let me know.

Thanks for the pointers. I will try to have a look at it in the following weeks.

Before modifying the present implementation I am trying to understand how it works. What is suppose to be the workflow for sending a robot to custom waypoints?
Currently, this is what I do:

With the robot stoped, generate a nav_msgs/Path
Publish the generated path on /move_base/TebLocalPlannerROS/via_points
Nothing happens, so I tried:
Send a goal to move base:
The via points sent through the topics appear in the Teb markers, the robot starts to follow a path going to the goal and passing by the generated via points, it reached the goal, but then instead of stopping, it goes back on forth indefinitly between the center of the waypoints (it seems) and the goal:

Maybe I miss something?

according to your animation, you probably have the use case to provide via-points using an external script.
To be honest, this is a use case I have not yet played a lot with, so there might be some things to improve.
The most common via-point use case is to obtain them from the global plan in every planning cycle (which is indeed implemented).
But of course, it should be possible to also add them via a topic (I added this due to some user request #63 in merge request #74).
However, I just had a quick look at void TebLocalPlannerROS::customViaPointsCB(const nav_msgs::Path::ConstPtr& via_points_msg) and I am obviously not transforming the via points the planning frame (usually map). Hence, we require to provide the via points already in the correct coordinate systems a-priori.
In your animation, we can see, that the planner does not take the via-points into account correctly.
This is because it is quite difficult to reach via-points which are not within the line of sight towards the goal. The trajectory needs to be extended which is done in the following manner:
Each via-point is associated with the closest pose of the trajectory (in void TebOptimalPlanner::AddEdgesViaPoints()). The weight of the via-point cost function implies an attraction force resulting in extending the trajectory slightly. The autoResize algorithm then adds new poses. For complex via-point scenarios, it might be really difficult to converge properly.
Currently, I ignore via-points behind the start or beyond the goal to avoid such situations.
This is implicitly done in void TebOptimalPlanner::AddEdgesViaPoints(). If the closest pose to a via-point is the start or the goal pose, it does not have any effect on the optimziation result, since the start and goal are fixed during optimization.
You can see some magic, that I force the algorithm to attach via-points with at least the second vertex in case parameter via_points_ordered=True. You could try to repeat your scenario from the animation with this option turned on. I am really interested in how it looks like.
Otherwise, please have a look at void TebOptimalPlanner::AddEdgesViaPoints().
It can definitely be improved. However, the best would be to consider the via-points also in the TEB initialization. If we would initialize the TEB along the via-points, even the current implementation of void TebOptimalPlanner::AddEdgesViaPoints() should work fine to my mind.

Thanks a lot for your detailed answer (as always). I was indeed adding waypoints through the viapoint topic. But I realized that it wasn't the cleanest way to do it, which you confirmed later in your answer.
I managed to achieve the desired behavior (make the robot navigate through a series of waypoint) by using move base flex and the mbf_msgs::ExePath action (which has a path field of type nav_msgs::Path)
Note: In my implementation, I actually use the mbf_msgs::MoveBase action to go to the first waypoint and then use the ExePath for the rest of my waypoints. I can conveniently use two different controllers (ie local planner) with two different sets of parameters thanks to move base flex.
I noticed that if the global_plan_viapoint_sep is inferior (or negative) to the distance between my waypoints, I get the expected behavior, see below with  global_plan_viapoint_sep: 0.25, weight_viapoint: 1000 and a distance between two waypoint of 0.5.

Otherwise, if global_plan_viapoint_sep is superior to the distance between my waypoints, it seems that teb is generating them on the fly. Here with global_plan_viapoint_sep: 0.25, weight_viapoint: 1000 and a distance between two waypoint of 0.1.

Anyway, I reached the desired behavior and don't need to implement immediately the orientation constraints to the viapoints. But it is definitely something I will consider in the next few months

Hello,
Has there been any work on this enhancement?
If not, we have a use-case that requires this, and I will therefore begin implementing based on @croesmann 's suggestions above
Best regards

#109

hi,this behaviour is really useful. Can teb_local planner avoid obstacle when the obstacle occure on the path that given by myself ?  Actually, I do the similar job. I replace the "global_planner" with a "waypoint_planner"(i send a fixed global path to local_planner). I would like to use "teb_local_planner" to avoid obstacle when obstacle occure in local cost map (the global waypoints may through obstacle,so teb_local_planner has to generated a new local path to avoid collision without following global path),after pass the obstacle succefully, the local path should back to global watpoint path.
However, when i try this in gazebo, the teb_local_planner's global plan(not waypoint path given by myself,but teb_local_planner generated to let local plan follow) will never change and always following my waypoints global path.
is there any suggestion for my problem??
Thanks
with reagrds

@guochence I cant immediately see the issue with your settings, other than the difference that I do not use a costmap converter.
I can tell you that in my case, even with global paths through obstacles, it is possible for my TEB local path to avoid the obstacles (if they are sufficiently small)

@zerodamage
thanks for your reply. May i ask few questions about yamls?
1.about "global_plan_viapoint_sep", did u set this parameter? or just =-1 to disable it?
2.About costmap, which plugins do u use in your global_costmap and local_costmap respectively？ Do i need use "inflation_layer" and "static_layer" in local_costmap while teb_planner has "min_obstacle_dist" and "inflation_dist"?
3.About "min_obstacle_dist" in teb_planner. I do not understand why the planner need this params while local_cost_map can apply "inflation_layer" .
thanks for you patient.
with regards

@guochence

is set to a distance smaller than my global plan spacing (0.1m)
while I still use inflation layer in the local costmap, I dont believe it is actually necessary, because only lethal obstacles are used by teb_local_planner
same answer as above, there is no need to apply the inflation layer, because the non-lethal results of it are not used for anything


Regarding License

As specified the Leshan are made available under the terms of the Eclipse Public License v1.0  and Eclipse Distribution License v1.0 which accompany this distribution.
Can we consider this is under Dual License or as as Single License as EPL and EDL separately

I'm not sure to understand the difference between "Dual License" and " Single License as EPL and EDL separately".
You can find the official answer from eclipse here : https://www.eclipse.org/legal/eplfaq.php#DUALLIC

Dual license means we have to follow the obligations either EPL 1.0 or EDL 1.0.
Single License means the Leshan will have to follow the obligations for both the license.
In this way which license we should consider.

It depends on your needs :
Here are synthetic views of well-known licenses :

https://choosealicense.com/appendix/
https://en.wikipedia.org/wiki/Comparison_of_free_and_open-source_software_licenses

This could help you to compare (EDL is just a BSD 3-Clause "New" or "Revised" License)
As I understand :

EPL should bring you a little more protection (mainly about patent) in exchange for some distribution constraints.
EDL/BSD is about more distribution flexibility with a little less "protection".

I'm not an expert about that so I advice you to read the https://www.eclipse.org/legal/eplfaq.php .
If all of this information is not enough, you could directly contact<EMAIL_ADDRESS>or/and<EMAIL_ADDRESS>If you find more information, do no hesitate to share it with us.

Can we go ahead with EDL v1.0 for any product release

Hi @ricky001 - like @sbernard31 suggested, please direct your question to<EMAIL_ADDRESS>


Add GKeyFile to the API (fixup)

A fixup of #136. Closes #135.

Huge :+1: ! Waiting for appveyor and I merge.


Delete popover doesnt close when clicked outside of

Click X to delete block
Click outside of delete popover
Popover should close (same functionality as clicking "no")

Yeah common issue which i can look at.


Allow to customize the defmt log format

This PR adds the --log-format flag to probe-rs run. It allows to specify a custom log format for the defmt logs.
Defaults
If no log format is specified a default format is chosen depending on if there is a timestamp available and if the location should be shown.
// Format options:
// 1. Custom format
// 2. Default with timestamp with location
// 3. Default with timestamp without location
// 4. Default without timestamp with location
// 5. Default without timestamp without location
let format = log_format.unwrap_or_else(|| match (show_location, has_timestamp) {
    (true, true) => "{t} {L} {s}\n└─ {m} @ {F}:{l}",
    (true, false) => "{L} {s}\n└─ {m} @ {F}:{l}",
    (false, true) => "{t} {L} {s}",
    (false, false) => "{L} {s}",
});

Log format format
The log format is simple, a few examples can be seen in the default section above. A specification of it can be found here and will soon be published in the defmt book where it will be rendered nicely.
It is also possible to style the log format. Please refer to the specification above.
For example the log format "{t} {L} {s}\n└─ {m} @ {F}:{l}" does the following: It shows the timestamp, the log level, the actual log line, followed by a newline, then the module path, a literal @, the filename and lastly the line number.
Drawbacks
Because of how cargo parses its .cargo/config.toml the runner needs to switch to an array of strings. Otherwise cargo does not accept " or ' as part of runner. But this isn't really a drawback of this PR, more of how cargo parses cli parameters in the runner field.
Before:
[target.'cfg(all(target_arch = "arm", target_os = "none"))']
runner = "probe-run --chip nRF52840_xxAA"

After:
[target.'cfg(all(target_arch = "arm", target_os = "none"))']
runner = [
  "probe-rs", "run",
  "--chip", "nRF52840_xxAA",
  "--log-format", "{t:010} [{L:bold}] {f}:{l} {s}",
]

(At least we could not figure out how to use the --log-format with a string. We only got it to work with the array of strings. Please tell us if you make it work!).
TODO

[ ] Switch to crates-io version, as soon as merged in defmt-decoder https://github.com/knurling-rs/defmt/pull/781

Looking very good :) happy to merge as soon as upstream is released :)

Just released defmt-decoder v0.3.9. The PR should be ready to go 🎉


Set images based on direction when moving

Be able to specify images per direction and "default" for when not moving of images are not specified for direction

Done


Update WeexGcanvas.podspec

替换sd封装库

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

SDWebImage通用性更好


[ctdb/ss] Add support for AD DNS Registration

The non-clustered SmbShare instances are able to host additional containers that watch for changes with a Services (public) IP addresses and register those to the AD DNS. This should be supported and tested in clustered (ctdb) modes too.

Done in #126


[Bug!] Cannot install Eufy integration

Please check existing (open or closed) issues before creating a new one. (https://github.com/fuatakgun/eufy_security/issues?q=is%3Aissue+)
If this is not an issue, you can use Discussions (https://github.com/fuatakgun/eufy_security/discussions) section.
Describe the bug
A clear and concise description of what the bug is.
I had been able to use the Eufy security integration and view my streams (albeit slowly since I do not use WebRTC). for my outdoor camera. However, the integration entities and camera feed stopped working one day. The HA integration add-on states that it "failed to set up"
Steps to reproduce the behavior:

Go to Settings ---> Devises & Services -->
The Eufy Security add-on fails to set up

Expected behavior
The integration should install and the entities + camera feed should be working.
Additional information
Go to Settings -> System -> Repairs -> Click on 3 dots (...) -> System Information and get Version and Installation Type to below fields;

Home Assistant Installation Type (OS - Supvervise- d - Core):   Home Assistant OS 11.2, 	supervisor-2023.12.0
Home Assistant Core Version: core-2023.12.3
Eufy Security Add-on Version:  1.3.0
Eufy Security Integration Version: It cannot install

Hardware Information;

Camera Model:
Eufy Security Solocam S40

Live Streaming Protocol (RTSP/P2P):
RTSP although (apparently) I am unable to extract the RTSP stream URL for this particular model.

Debug Logs from Home Assistant (https://github.com/fuatakgun/eufy_security/discussions/624) - Without this, I will automatically resolve this issue

Logs from Eufy Security Add-on (either from Add-ons page of Home Assistant or Docker output):

Logger: homeassistant.config_entries
Source: config_entries.py:402
First occurred: 9:42:30 AM (1 occurrences)
Last logged: 9:42:30 AM
Error setting up entry localhost for eufy_security
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
result = await component.async_setup_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/eufy_security/init.py", line 50, in async_setup_entry
await coordinator.initialize()
File "/config/custom_components/eufy_security/coordinator.py", line 39, in initialize
await self._api.connect()
File "/config/custom_components/eufy_security/eufy_security_api/api_client.py", line 71, in connect
await self._set_schema(SCHEMA_VERSION)
File "/config/custom_components/eufy_security/eufy_security_api/api_client.py", line 147, in _set_schema
await self._send_message_get_response(OutgoingMessage(OutgoingMessageType.set_api_schema, schema_version=schema_version))
File "/config/custom_components/eufy_security/eufy_security_api/api_client.py", line 324, in _send_message_get_response
return await future
^^^^^^^^^^^^
custom_components.eufy_security.eufy_security_api.exceptions.FailedCommandException: {'type': 'result', 'success': False, 'messageId': 'set_api_schema.b5c06399464343b7a367a4cd69b0f3fb', 'errorCode': 'schema_incompatible'}

Would you share your device with me in case further debugging required? (Yes/No):
Only as a last resort.

Additional context
I think that's everything. I greatly appreciate your help with this issue. Thank you!

Update to add-on latest version

This was indeed the issue. Thank you very much fuatakgun!


[BUG] docs.getzep.com is being blocked on office premise wifis

When I try to access getzep.com it is being blocked by office wifi due to the suspicion of domain parking.
FortiGuard Intrusion Prevention - Access Blocked
Category: Domain Parking

Hey, we use Webflow for our website hosting and Cloudflare for DNS. I've no idea why your security service is identifying our domain as suspicious. Perhaps contact your IT team?

Thank you for smooth reply. That happened in multiple different occasions around Hong Kong. Seems like url was flagged a threat in a software used by many companies. I was worried that same thing might be happening for other people too. Thanks I will try to resolve it on my side.


Max open invoices?

A new invoice is created when the user specifies an amount and clicks on the Tip button. This gives the end-user (tipper) the ability to initiate new invoices on the host's lightning node. What are the limitations when it comes to max open invoices on a node? Should expiry time be less than the default 3600 in this case?

Right, it does give users the ability to freely create new invoices on the c-lightning node. That's not much different from an e-commerce website allowing users to store shopping cart information, invoices, etc. on the server. The main difference is that here its a one-click operation, while the checkout process on e-commerce is usually more complex. But that won't matter much to bots that wants to DDoS the service...
As for your question: the limitation is really up to server hosting it. Each invoice is a row in c-lightning's sqlite database and Lightning Charge's sqlite database. How many of these you can process and store depends on the storage, memory and CPU available on the server.
Note that Lightning Charge deletes unpaid invoices from both itself and c-lightning a few days after they expire, so the invoices will eventually be removed if they're not paid.

Should expiry time be less than the default 3600 in this case?

Yes, you can make it smaller by adding the following in nanotip.js:
@@ -39,7 +39,6 @@ app.post('/', (req, res, next) =>
     amount: req.body.amount
   , currency: req.body.currency
   , description: app.settings.title
+  , expiry: 120 // expire in 2 minutes
   , webhook: app.settings.url + '/webhook/' + webhookToken
   , metadata: { source: 'nanotip', checkout: { redirect_url: thankyouUrl }, info: req.body.info }
   })

I could also add this as a configuration option you can set, created #4 to keep track of that.

Each invoice is a row in c-lightning's sqlite database and Lightning Charge's sqlite database. How many of these you can process and store depends on the storage, memory and CPU available on the server.

Thank you, that is what I was wondering. I wasn't sure how many resources an invoice took from the c-lightning node, for example if each invoice had to be actively "listened for" by the node for new payments and if a flood of new invoices could cause stress on the node. If each invoice is only as intensive as a row in sqlite then, as you've stated, there's really no reason to be concerned over the number of open invoices.

IIf each invoice is only as intensive as a row in sqlite then, as you've stated, there's really no reason to be concerned over the number of open invoices.

I verified this with @rustyrussell who is more qualified on this than me, and yes - its just a row in the db.


Question: should db/migrate be part of the default paths?

When using engines we had to manually add db/migrate to the list of paths.
Stimpack.config.paths.concat(%w(db/migrate))

Should that path be part of the default paths? Alternatively, would it be possible to add that path for engines?

This on is tricky because the way that Rails looks for "other migration paths" in only in config/database.yml under the migrations_paths key (https://edgeguides.rubyonrails.org/active_record_multiple_databases.html).
Afaict adding extra db/migrate paths doesn't do anything. You can experiment and let me know! Thanks.

@ngan With the change I posted all our migrations are run. We even have a test that checks that the resulting schema has not changed.

Adding db/migrate to Stimpack.config.paths allows developers to put migrations into packs rather than storing them globally in db/migrate.
So unless you also use migrations_paths, the downside of this is that rails generate migration does not take these paths into account when creating new migrations, so you'd have to move the file by hand.

Currently our application uses multiple database, and certain packs have their databases' migrations located at packs/pack_name/db/migrate. So without having stimpack know which packs use their own database, this could produce unexpected results when using this feature of Rails.
For now, our recommendation if you don't use multiple databases would be to manually add these paths:
    # In `config/application.rb`
    Stimpack::Packs.all.map do |pack|
      config.paths['db/migrate'] << pack.path.join('db/migrate')
    end

Later if we can make stimpack aware of packs that use separate databases, we could bake this right into stimpack.


Add Config#for_badge

This optimization saves about 5MB of unneeded intermediate hashes...

The changes look good apart from the failing build.


Plotting ica properties returns ValueError (dtype 64)

I'm trying to follow the Artifact Correction with ICA example with Python 2.7.14 under Spyder in 64-bit mode (conda version 4.3.29). Everything works as expected until:
ica.plot_properties(raw, picks=0)
which leads to an ValueError.

File "", line 1, in
ica.plot_properties(raw,picks=picks)
File "...\Anaconda3\envs\py27\lib\site-packages\mne\preprocessing\ica.py", line 1435, in plot_properties
figsize=figsize, show=show)
File "...\Anaconda3\envs\py27\lib\site-packages\mne\viz\ica.py", line 241, in plot_ica_properties
psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args)
File "", line 2, in psd_multitaper
File "...\Anaconda3\envs\py27\lib\site-packages\mne\utils.py", line 728, in verbose
return function(*args, **kwargs)
File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\psd.py", line 292, in psd_multitaper
n_jobs=n_jobs, verbose=verbose)
File "", line 2, in psd_array_multitaper
File "...\Anaconda3\envs\py27\lib\site-packages\mne\utils.py", line 728, in verbose
return function(*args, **kwargs)
File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 521, in psd_array_multitaper
low_bias=low_bias)
File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 201, in dpss_windows
select_range=(N - Kmax, N - 1))
File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 847, in eigvals_banded
return eig_banded(a_band, lower=lower, eigvals_only=1,
File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 587, in eig_banded
raise ValueError('expected two-dimensional array')
File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 478, in _check_select
if sr.dtype.char.lower() not in 'lih':
ValueError: when using select="i", select_range must contain integers, got dtype int64

I'm quite new to Python, but I guess it has something to do with using the 64-bit version? I tried it with 32-bit mode, but then I ended up with a MemoryError. Does anyone have a suggestion on how to resolve this problem?

This one is a bit weird. Are you on Windows?
Can you change line 509 of mne/time_frequency/multitaper.py from:
n_times = x.shape[-1]

to
n_times = int(x.shape[-1])

To see if it fixes the problem?

Are you on Windows?

Yes I am using Windows 7 (64-Bit).

To see if it fixes the problem?

I changed the line accordingly, but the error still remains the same.

If I skip this step and move on to the artifact detection using EOG information multitaper.py produces another error:

File "...\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 509, in psd_array_multitaper
dshape = int(x.shape[:-1])
TypeError: int() argument must be a string or a number, not 'tuple'

windows does not use standard int64. One fix is certain to do a np.astype
somewhere
but for this you need to debug after putting a breakpoint at the right
place.
what is the value of sr.dtype ?

what is the value of sr.dtype ?

The value of sr.dtype is int64
Somehow the tuple select_range from multitaper.py (line 201) ends up as a dstype = int64 via np.asarray(select_range)?

no idea

I checked for sr.dtype.char.lower() and it is set to 'q' which is not in 'lih'. When adding q to the string the correct figure would be produced.
If I just create the numpyarray manually and check for dtype.char I get the following:

sr.astype(np.int64)
Out[21]: array([1], dtype=int64)

sr.dtype.char.lower()
Out[22]: 'l'

I have no idea why the type is 'q' in one case, but 'l' in the other.

Me neither, but let's go ahead and add it to the list (it's c long long equiv)
https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.scalars.html

... oh wait it's in a SciPy function. You need to cast it in MNE to make it happy, maybe pass np.int64(N) or something?

BTW doing sr.astype(np.int64) returns something, it does not change sr in place

After adding np.int64(N) to line 201 in multitaper.py the value error is solved:

w = linalg.eigvals_banded(ab, select='i',
select_range=(np.int64(N) - Kmax, np.int64(N) - 1))

How can I fix this issue? I am struggling with the same problem!

how do we replicate? can you replicate on sample data?

Yes, both sample data using this script https://martinos.org/mne/stable/auto_examples/preprocessing/plot_run_ica.html#sphx-glr-auto-examples-preprocessing-plot-run-ica-py and on my data give me the same issue mentioned before. I changed those lines suggested by nrosjat in the multitaper.py file, but I keep getting the same value error.

`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
 in ()
56 ########################################################################
#######
57 # Plot properties of ECG components:
---> 58 ica.plot_properties(epochs, picks=ecg_inds)
D:\Users\Davide\Anaconda2\lib\site-packages\mne\preprocessing\ica.pyc in plot_pr
operties(self, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_ar
gs, figsize, show)
1433                                    topomap_args=topomap_args,
1434                                    image_args=image_args, psd_args=psd_a
rgs,
-> 1435                                    figsize=figsize, show=show)
1436
1437     @copy_function_doc_to_method_doc(plot_ica_sources)
D:\Users\Davide\Anaconda2\lib\site-packages\mne\viz\ica.pyc in plot_ica_properti
es(ica, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_args, fig
size, show)
239     plot_lowpass_edge = inst.info['lowpass'] < Nyquist and (
240         psd_args['fmax'] > inst.info['lowpass'])
--> 241     psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args)
242
243     def set_title_and_labels(ax, title, xlab, ylab):
D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu
ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat
ion, picks, proj, n_jobs, verbose)
D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, *
args, **kwargs)
726         with use_log_level(verbose_level):
727             return function(*args, **kwargs)
--> 728     return function(*args, **kwargs)
729
730
D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu
ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat
ion, picks, proj, n_jobs, verbose)
290                                 bandwidth=bandwidth, adaptive=adaptive,
291                                 low_bias=low_bias, normalization=normali
zation,
--> 292                                 n_jobs=n_jobs, verbose=verbose)
D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in
psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, norma
lization, n_jobs, verbose)
D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, *
args, **kwargs)
726         with use_log_level(verbose_level):
727             return function(*args, **kwargs)
--> 728     return function(*args, **kwargs)
729
730
D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in
psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, norma
lization, n_jobs, verbose)
519     n_tapers_max = int(2 * half_nbw)
520     dpss, eigvals = dpss_windows(n_times, half_nbw, n_tapers_max,
--> 521                                  low_bias=low_bias)
522
523     # decide which frequencies to keep
D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in
dpss_windows(N, half_nbw, Kmax, low_bias, interp_from, interp_kind)
199         # only calculate the highest Kmax eigenvalues
200         w = linalg.eigvals_banded(ab, select='i',
--> 201                                   select_range=(np.int64(N) - Kmax, np.i
nt64(N) - 1))
202         w = w[::-1]
203
D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eigvals_b
anded(a_band, lower, overwrite_a_band, select, select_range, check_finite)
845     return eig_banded(a_band, lower=lower, eigvals_only=1,
846                       overwrite_a_band=overwrite_a_band, select=select,
--> 847                       select_range=select_range, check_finite=check_fini
te)
848
849
D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eig_bande
d(a_band, lower, eigvals_only, overwrite_a_band, select, select_range, max_ev, c
heck_finite)
585         raise ValueError('expected two-dimensional array')
586     select, vl, vu, il, iu, max_ev = _check_select(
--> 587         select, select_range, max_ev, a1.shape[1])
588     del select_range
589     if select == 0:
D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in _check_se
lect(select, select_range, max_ev, max_len)
476             if sr.dtype.char.lower() not in 'lih':
477                 raise ValueError('when using select="i", select_range mu
st '
--> 478                                  'contain integers, got dtype %s' % sr.d
type)
479             # translate Python (0 ... N-1) into Fortran (1 ... N) with +
1
480             il, iu = sr + 1
ValueError: when using select="i", select_range must contain integers, got dtype
int64`

I get the same error when trying to run https://www.martinos.org/mne/stable/auto_examples/connectivity/plot_sensor_connectivity.html#sphx-glr-auto-examples-connectivity-plot-sensor-connectivity-py

what does:
mne.sys_info()
return on your system? what OS are you using? win32 or win64?

In [2]: mne.sys_info()
D:\Users\Davide\Anaconda2\lib\site-packages\h5py_init_.py:36: FutureWarning:
Conversion of the second argument of issubdtype from float to np.floating is
deprecated. In future, it will be treated as np.float64 == np.dtype(float).typ e.
Platform:      Windows-7-6.1.7601-SP1
Python:        2.7.14 |Anaconda, Inc.| (default, Nov  8 2017, 13:40:45) [MSC v.1
500 64 bit (AMD64)]
Executable:    D:\Users\Davide\Anaconda2\python.exe
CPU:           Intel64 Family 6 Model 42 Stepping 7, GenuineIntel: 4 cores
Memory:        7.9 GB
mne:           0.15.2
numpy:         1.14.0 {lapack=mkl_rt, blas=mkl_rt}
scipy:         1.0.0
matplotlib:    2.1.2
sklearn:       0.19.1
nibabel:       2.2.1
mayavi:        4.5.0
pycuda:        Not found
skcuda:        Not found
pandas:        0.22.0

@nrosjat @Davi93 can you check to see if #5039 fixes your problem?

I applied your changes but I still get this error:

Same for me. I get the same error again after replacing my changes by the fix.

@larsoner I think it worked for me but I know get this error: RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyqt' or 'pyside'.
I guess is not related with the previous issue though.

Can you try our latest and greatest updated conda environment (merged today or yesterday):
https://github.com/mne-tools/mne-python/blob/master/environment.yml

I have managed to build that environment in miniconda. I do not get any error now, but the mayavi windows that should show the connectivity results just stucks as soon as it appears and crashes.
I am also struggling in creating the environment with the yml file, anaconda seems to stuck somewhere as some packages are not installed (as mne or all the packages listed after it.
ps. I uninstalled and reinstalled anaconda and miniconda before doing it.

Now using miniconda and ipython, even simple functions as raw.plot are not working, the windows are not responding and crashing.
The raw.plot function is instead working if I use a python terminal, but I get an error when trying to do:
"from mayavi import mlab" (from the sensor connectivity script). In Ipython I don't get this error, but the output windows are not working.

Grrrr
Can you try pip install --upgrade --no-deps pyqt to see if this fixes it?

"Could not find a version that satisfies the requirement pyqt (from versions: )
No matching distribution found for pyqt
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' comm
and." :/

Sorry, I meant pyqt5. Are you on gitter? We could iterate there more easily.

https://gitter.im/mne-tools/mne-python

After installing the pre-mentioned environment (https://github.com/mne-tools/mne-python/blob/master/environment.yml) I was still having problems with 3d visualization. The Mayavi window was not showing anything/freezing. I also get an error message from another window stating: "GL version 2.1 with the gpu_shader4 extension is not supported by your graphics driver but is required for the new OpenGL rendering backend. Please update your OpenGL driver. If you are using Mesa please make sure you have version 10.6.5 or later and make sure your driver in Mesa supports OpenGL 3.2.".
The issue was solved as suggested by Larsoner, using the %gui qt tweak and by making python using the NVIDIA card in Intel/NVIDIA switchable graphics feature as following: in the NVIDIA control panel IIRC you should  set which programs use the GPU and add the python.exe from your env (e.g., C:\Users\whoever\miniconda3\envs\mne\bin\python.exe). You should add Ipython.exe and Spider.exe as well.
That worked for me.

Thanks for the feedback. This should be added to our install doc

Actually I think it would fit better in faq.rst, maybe with a clearer link from the main install_mne_python.rst. We don't want that install page to become the landing page for every possible system config / install problem.

fine with me


plotting module is not provided

The img_regression script imports a "plotting" module on line 12 viz.
https://github.com/crisbodnar/ndp/blob/5a463308a3461819d7e62473f056eca8a2aa5f01/main/img_regression.py#L12
But this custom module is not provided in the codebase.

Will close after it's fixed. Thanks!

Any update on the "plotting" module?

Any updates ? @crisbodnar

Sorry, this took so long! This has been added by https://github.com/crisbodnar/ndp/pull/6.


Add restriction to gist:TimeInterval

All time intervals will have a duration--should we add this as a restriction?

The definition says:

A span of time with a known start time, end time, and duration. As long as two of the three are known, the third can be inferred.

So maybe the restriction should be:
(start & end) U (start & duration) U (end & duration)
?
That said, I agree there should be restrictions.

@ct-martin Given OWA, we don't need to make it that complicated. They all have a start, end, and duration, although not all three need to be asserted. So I think adding restrictions requiring each of the three is valid.

Here's a first pass definition for us to consider:
gist:TimeInterval
	a owl:Class ;
	rdfs:subClassOf
		[
			a owl:Restriction ;
			owl:onProperty gist:actualStartDateTime ;
			owl:cardinality "1"^^xsd:nonNegativeInteger ;
		] ,
		[
			a owl:Restriction ;
			owl:onProperty gist:actualStartDateTime ;
			owl:cardinality "1"^^xsd:nonNegativeInteger ;
		] ,
		[
			a owl:Restriction ;
			owl:onProperty gist:hasMagnitude ;
			owl:onClass gist:Duration ;
			owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
		]
		;
	.

To Michael's question, I am thinking we could use hasMagnitude to link TimeInterval and Duration. Each time interval would have exactly one start time, one end time, and exactly one magnitude that is a duration. I've suggested qualified cardinality on the magnitude, thinking that time intervals could have other kinds of magnitudes associated with them (e.g., the number of events that unfolded within them).

@dylan-sa
Dylan's proposal looks good to me. Normally I don't like using hasMagnitude, but in this case the meaning is obvious.

I doubt if one would ever assign number of events in a time interval as a magnitude of the time interval. Nevertheless, the definition seems OK.

Dylan's suggestion used a min 1 cardinality. Specifically:
		[
			a owl:Restriction ;
			owl:onProperty gist:hasMagnitude ;
			owl:onClass gist:Duration ;
			owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
		]

It should be someValuesFrom

I like using hasMagnitude here. Defining a hasDuration predicate contradicts our best practice of not embedding the object type in the predicate.

This is speculative, but I wonder if we could also put 0 cardinality on plannedStartDate and plannedEndDate: this is one of the features distinguishing events from time intervals. Events may be planned, time intervals just exist.

I like using hasMagnitude here. Defining a hasDuration predicate contradicts our best practice of not embedding the object type in the predicate.

Agreed.

Maybe, but one of the arguments against TimeInterval is that it was hard to distinguish from Event. This would be one option - though of course not all Events have planned dates. It may be conceptually helpful. Let's see what other people think.

DECISION: Implement according to Dylan's proposal above.


useGroqBeta breaks preview

Hi
We really love live preview and view it as one of the "wow" moments when showing off Sanity to clients.
Unfortunately for a lot of our projects the query size causes the browser to basically break so we've had to stop using this approach. Therefore we are really excited about his beta!
Using  "next-sanity": "^0.4.1-beta.0" the preview works great as intended. But when adding:
export const usePreviewSubscription = createPreviewSubscriptionHook({
  ...config,
  useGroqBeta : true
})

It just stops working without any errors.

@heggemsnes Do you have the hook configured to return error?
const { data, loading, error } = usePreviewSubscription(query, { ...

When adding useGroqBeta : true and logging out the error, I get the following error:
react_devtools_backend.js:4061 ReferenceError: marks is not defined
    at parseString (groq-js.esm.js?4826:1497:1)
    at parseObject (groq-js.esm.js?4826:1442:1)
    at parseExpr (groq-js.esm.js?4826:862:1)
    at parse$1 (groq-js.esm.js?4826:784:1)
    at parse (groq-js.esm.js?4826:2796:1)
    at eval (groq-store.js?181b:1:5227)

When I remove useGroqBeta : true it works most of the time, but it's slow.

Closing this as the marks is not defined were resolved a few versions back when we upgraded from the beta. :)
The slowness issue is still a known issue. And if you have a repro case feel free to share it, much appreciated 🙇


Update README.md

Related issue

What changes does this PR do?
Screenshots (If any)
Questions

Add imrosun


support hideNavbar query parameter

Added the logic to hide the navbar if hideNavbar=true is present as a URL query parmaeter. In this case, the navbar won't be shown, but the logic will continue to run.
Opening this PR for review so we can discuss whether this is the best way to go about this or not. While investigating this issue, I wanted to make sure it would work for the webapps, upon testing it for boolean-search, I came across this issue.

This PR can be reviewed again. I merged canonical-tag into this branch because I thought I was going to need the getHash function I added in the other one, but realized I didn't need it after we discussed the nuance about using ? instead of #.
Some apps (navbar and boolean-search) don't have a hash so it would replace the query string with the hash and not parse the query params properly. It doesn't make sense to hide the navbar in the navbar app, but I did want to make sure the functionality was there if someone wanted to embed a static page in an iframe.

The changes to hide the navbar after checking if it's contained within an iframe have been made. By default, the navbar will be hidden inside of an iframe unless the inverse of the property is defined (hideNavbar=false). If no query parameter is defined, it is treated as null and the default behavior will take place.
Outside of an iframe, the navbar will show by default, and only be hidden if hideNavbar=true.
This can be tested by navigating to a place where we have iframe content, here in record app and here in boolean-search. By changing the src or data attribute, you can force the iframe to load new content and see how our pages behave when injected into these frames.
I also made sure to test the hideNavbar param for all chaise apps when it's part of the current browser's URL. Currently, it's not easy to test this for boolean-search itself because of the way the content is loaded, detailed here in issue #67.


Update documentation

Pull Request Test Coverage Report for Build 416

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 90.839%

Totals

Change from base Build 414:
0.0%

Covered Lines:
3193

Relevant Lines:
3515

💛  - Coveralls


Crash with strange characters in scripts (ä,ö,ü...)

Hacketyhack crashes when I save a script just containing some letters, some of them being ä, ö or ü, and then restart hacketyhack.
Error is invalid byte sequence in US-ASCII
I believe it shouldn't crash (then maybe a shoes issue) or at least hacketyhack should catch this and load anyway but without the compromised script.
And I thought that my code messed things up :-/

Hrm. Can you get a trace on this? I thought we were coercing everything to UTF8...

Hey I can't copy from the shoes error console so I made a screenshot. Encoding seems to be US-ASCII although my system encoding and everything is imo correctly set to UTF-8.
I got the error by creating a script simply containing
ä

and saving it with the name ae - then I restarted hacketyhack, that's the error you see. hacketyhack is current Master version and shoes is current development version (both freshly pulled from upstream):

No worries. The reproducibility is the big thing. Thanks.


Not getting any socks proxies

All proxies I ever get either have no protocols property, or protocols is ["http"].
Example: https://gist.githubusercontent.com/ledlamp/b494b4f1ce13575eeb3ffbbc07fbacf6/raw/5c0ea0dd1375bb94132bc23e05b636828f0eff80/proxies.txt

Some of the recently fixed sources only include basic information right now. The master branch includes some changes that will scrape more info like protocol and anonymity-level, but that hasn't been included in a release yet.

Released a new version, so you should be getting a decent number of socks proxies now with the following command:
proxy-lists --protocols="socks4,socks5"


Update test app go version to 1.15

Go 1.13 is now deprecated by the Go buildpack 1.9.20. I have tested this change and confirmed that the smoke tests pass with the updated test app.

Thanks @peterellisjones !


Starting trxd without configuration file results in a Lua PANIC

There is no error message or any other hint provided if there is no configuration file, instead Lua PANICs and trxd is aborted.
./trxd -dl
PANIC: unprotected error in call to Lua API (attempt to index a nil value)
Aborted

Thansk for the report, I just fixed it.


Add Home Media Center

https://github.com/danbovey/awesome-home-media-center
This adds a list of tools and software that are useful for people who own a home media center such as Plex or Kodi. Tools to automate watching TV and movies.
By submitting this pull request I confirm I've read and complied with the below requirements.
Failure to properly do so will just result in the pull request being closed and everyone's time wasted. Please read it twice. Most people miss many things.

I have read and understood the contribution guidelines and the instructions for creating a list.
This pull request has a descriptive title. For example, Add Name of List, not Update readme.md or Add awesome list.
The entry in the Awesome list should include a short description about the list project/theme (not the list itself), and should be added at the bottom of the appropriate category.
The list I added

has been around for at least 30 days,
is the result of hard work and is the best I could possibly produce,
is a non-generated Markdown file in a GitHub repo,
is not a duplicate,
includes a succinct description of the project/theme at the top (example),
includes a project logo/illustration (whenever possible) at the top right of the readme (example). (The logo should link to the project website or any relevant website. The logo should be high-DPI, so set it to maximum half the width of the original image.),
only has awesome items with descriptions (Awesome lists are curations of the best, not of everything),
contains the awesome badge on the right side of the list heading,
has a Table of Contents section named Contents as the first section,
has an appropriate license (Meaning something like CC0, NOT a code licence like MIT, BSD, Apache, etc) (If you use a license badge, it should be SVG, not PNG),
has contribution guidelines,
doesn't include a Travis badge,
has been checked for inconsistent formatting, spelling, and grammar:

Each link description starts with an uppercase character and ends with a period.
Drop all the A / An prefixes in the descriptions.
Consistent naming. For example, Node.js, not NodeJS or node.js.

Also, I'd love to see some more resources such as tutorials, setups, community etc. on Home Media Center.

Looks like a good start, but needs more content.
Include some blog posts, tutorials, links to community resources (Stack Overflow tag, IRC, chat, forums, communities, Reddit, etc. Example: https://github.com/sindresorhus/awesome-electron#community), videos of talks, book, learning material, etc.
Closing for now. Try again in a couple of months ;)


[dv] Integrity error generation in memory busses

First commit changes the old memory_error_seq so that it uses the new sequence library.
Second commit improves memory interface to have an integrity error response on demand, also changes the memory_error_seq to inject integrity errors as well as bus errors.
Not sure but we are seeing an load integrity related internal interrupt which then triggers alert_major_bus_o. So, I turned off the assertion that checks no alert is triggering in the testbench.
Note that this is part one of the two part task of having internal interrupts generated and modelled in cosim environment.
Resolves #1731
Resolves #1732

This does not take into account the type of memory request. That means we won't differentiate between seeing an integrity error on the memory interface while doing a store or a load (either way if rdata integrity bits are faulty, we're seeing an internal NMI -is this okay? WDYT @GregAC ). I believe this might not be ideal and we should only send an integrity error while we are doing a read. Should I change it?

That means we won't differentiate between seeing an integrity error on the memory interface while doing a store or a load (either way if rdata integrity bits are faulty, we're seeing an internal NMI -is this okay? WDYT @GregAC ). I believe this might not be ideal and we should only send an integrity error while we are doing a read. Should I change it?

With the current Ibex RTL integrity errors on rdata for writes are ignored. However we're planning on going back to checking it (see https://github.com/lowRISC/opentitan/issues/14611) so the current behaviour is fine.

Oh and we do need a check that an integrity error triggers an alert. This can be done as an assert for the data side. Instruction side requires a little more thought (a fetch may not actually get executed so the integrity error doesn't get consumed and doesn't trigger an alert). Though strictly that's a V2S task. So for this PR at least we don't need to worry about it, but we must not forget about it (best to create an issue to track).

While simulating with this version I found a minor mismatch regarding two back to back integrity errors (with load instructions). RTL side does not take the faulty data from memory into the registers, Spike does.
I might need help about how to tell Spike to not go into a fault but also not process the retired instruction.

Test pass rate for this memory test is now around 60%. We hit internal NMI coverpoints though. I suspect almost all failures are related with the issue that I mentioned in the previous message.

As discussed in the Ibex DV Sync I think we should avoid generating integrity errors in the riscv_mem_error_test and introduce a new riscv_mem_error_intg_test that does have integrity error. Just add a config option for core_ibex_mem_error_test that enables generating integrity errors and a new entry to the testlist that turns that on.
With that done this can pass CI and we can get it merged. If the new test pass rate is 60% that should do for our V2 targets and we can fix up remaining issues in V3.

I think it might be cleaner to split out the cosim/NMI changes into a separate PR? Those changes are quite significant on their own. And could you add some more detail to the PR comment, the messages for 7222d4a012ba1b28b09cb5bd4d0f39cd5f70e9ac and 66f3e2dfc4416ba8d831f4200d525955f6fd518b are a good start but I think some more context would be really useful in the future.

Thanks @hcallahan-lowrisc for having a look, I updated the commit messages for almost all of them and dropped that one unnecessary commit about catching external RVFI signals -that one should belong to another PR I agree. Latest changes should make the mem_test pass in CI (as can be seen in Ibex CI + Private CI).
Might worth just another quick look @GregAC since now it includes cosim integration too.

I think this PR increases our test time for mem_error_test (after changing it with the new sequence we probably are injecting way more memory errors) and tips it over 30 mins. What should we do about it?

I think this PR increases our test time for mem_error_test (after changing it with the new sequence we probably are injecting way more memory errors) and tips it over 30 mins. What should we do about it?

Let's try dropping the frequency of memory errors and see what that does to the run time.

@GregAC can you have a look at https://github.com/lowRISC/ibex/pull/1811/commits/66d8d0fc0b087fa48f74cf8e0d343b582e2ad8e9 to see if this makes sense?

@GregAC can you have a look at 66d8d0f to see if this makes sense?

It does make sense, though having reviewed our discussion I think we want to leave the RTL behaviour as is. This means we need to instead alter cosim to not do the register writes on ECC failures. I will take a look at this.

@GregAC can you have a look at 66d8d0f to see if this makes sense?

It does make sense, though having reviewed our discussion I think we want to leave the RTL behaviour as is. This means we need to instead alter cosim to not do the register writes on ECC failures. I will take a look at this.

I've worked out how I'm going to do this, just need to implement it. I'm going to use a method that doesn't require spike changes (as otherwise these would be quite invasive to suppress a register write on a load).

I've added some stuff and reworked commits here to give https://github.com/lowRISC/ibex/pull/1811 which supersedes this PR and should get the memory integrity test running and passing.


fix anchor link open in FF

If I don't preventDefault on drop in FF and the draggable item has an anchor link, the anchor link will be opened.
Reference: https://developer.mozilla.org/en-US/docs/Web/Events/drop

@gaearon ?

This was fixed in #219 in capture phase (committed as 2299ece).

Thanks for the PR though!

The fix for this issue was released in<EMAIL_ADDRESS>Please verify that it works.