Improve control over number of threads used in an mne call

Description
I propose to use threadpoolctl to improve control over the number of threads used throughout mne and in calls to external libraries like numpy. This is apparently the direction that numpy has moved to as discussed here: https://github.com/numpy/numpy/issues/11826
Reasoning
I have had trouble completely controlling the number of threads used by various mne functions. Many mne functions have n_jobs arguments that control the number of threads used in that function, but there are cases where code within that function can escape this limit due to externally defined reference values. And then there are functions like mne.chpi.filter_chpi that do not have the n_jobs argument, but can still parallelize. It is possible to control this with environment variables as dicussed here, but that only works before you import the respective library, e.g. numpy. The easiest way to control thread limits after an import has happened appears to be threadpoolctl.
Proposed Implementation
I have successfully use the syntax
from threadpoolctl import threadpool_limits
with threadpool_limits(limits=n_jobs, user_api="blas"):
    mne.do_something()

to control threads used in an mne call. The same could be used internally to make better use of the existing n_jobs argument without forcing the user to do it themselves. If this proves successful, it might make sense to add the n_jobs argument in even more places.

I like this idea but it's tricky to make work out of the box. I suspect the optimal behavior depends on the length of the files, the number of channels, epochs...

thanks for the reply @agramfort . In contrast, I think it would be straight forward. The current n_jobs argument already allows the user to define the number of threads to use. The only change would be, that this would actually apply to all libraries that are called by mne. I don't see why you would need to implement an "optimal" behaviour. In most cases (e.g. numpy) the default is a maximum of 128 threads per job. I do not think that this would need to be overwritten, unless the user explicitly wants to.
And there are good reasons why you would want to consistently define a max thread number, e.g. when you work on shared resources or want to run multiple jobs on the same compute server. As it is now, a user needs to go out of their way to make sure that all restrictions via environment variables are set before anything else is imported - or do some additional research to find, what I referenced above.

how would you do this? if n_jobs=1 you make sure one thread is used?
So suddenly all computations in MNE are monothread?
it means we would need to add n_jobs in many places? note that we use
usually processes and not threads for parallel
do I get what you want to do correctly?

Message ID: @.***>

how would you do this? if n_jobs=1 you make sure one thread is used?
So suddenly all computations in MNE are monothread?

Well I guess, the question would then be, whether n_jobs=1 should be the default. I have seen other libraries use a default of 0 or -1 to signify, that the user does not want to change the default behaviour. Then you could use a globally set default or decide not to change anything. In my opinion, being able to set n_jobs implies that the code being run would use at maximum that number of available CPU threads. However, in mne that only applies to some of the code, while other code (e.g. some numpy calls) rely on externally defined variables that don't change when you set n_jobs. This does not matter, when you set n_jobs to be the total number of CPU threads, but it does when you explicitly want it to be less. In particular, if you do not modify the default of n_jobs=1 as a user, the code run might still actually parallelize, which may be undesired. Apart from functions, that have the argument n_jobs, what should be the default behaviour for functions that don't use it? As a user, I would expect that these functions always run monothreaded. However, mne.chpi.filter_chpi by default fills up to 128 CPU threads and there is no apparent way to control it. I think the same can happen for reading functions, am I correct? The problem here is, that you can't efficiently run multiple jobs in parallel on a large compute server, if some of them once in while try to take all available resources. Once your jobs start competing for CPU threads, everything runs a lot less efficient. At the same time, other code may not ne able to make good use of 128 threads, which is why it makes sense to run multiple jobs in parallel with a defined number of maximum threads per job.
As a user, I assumed that n_jobs would do exactly that. If you do not want that to be the case, an alternative could be to explicitly document ways to do that as a user on mne.tools and to document that in the usage of n_jobs as well.
If you decide to empower n_jobs as suggested, a new default of 0 or -1 or "auto" could mean to automatically infer the maximum number of available CPU threads and use that number, possibly up to a maximum of 128. That appears to be the current default for e.g. OPENBLAS_NUM_THREADS.

note that we use usually processes and not threads for parallel

I actually meant available CPU resources, not threads or processes as they are used in the context of python. I have used CPU threads now to clarify.

it means we would need to add n_jobs in many places?

Essentially yes. But that could be done gradually.

I guess it comes down to, what you would like n_jobs to mean. As long as it is consistent, a valid choice is to define it as "max number of threads/processes used in places that we control". But in that case some guidance in form of an example would be helpful to users who would like more/full control.
In my current code, the use of threadpoolctl as described above seems to do the job. However, if used on an mne function that also accepts n_jobs, it needs to be specified at twice this way. Once for threadpool_limits and once in the mne function call. Reducing that to one place would be more elegant and less ambiguous, but I can understand if you do not wish to change the meaning of n_jobs.

I would try to avoid a behavior that deviates to big libraries like
scikit-learn.
what is unclear to me is how big of the change is the change you suggest.
in the lab we ask users to set OMP_NUM_THREADS to 1 in their .bashrc
on the shared machines and to nice their jobs with "nice -5 python ..."
Message ID: @.***>

I would try to avoid a behavior that deviates to big libraries like scikit-learn.

According to this resource the default for sklearn is n_jobs=-1  in which case all available resources are used.

what is unclear to me is how big of the change is the change you suggest.

The function mne.parallel.check_n_jobs already contains the necessary code: If a negative n_jobs is passed, the number of CPU cores is used to calculate the actual n_jobs. So the first step would be to set all defaults to n_jobs=-1. That can be done very easily with most IDEs. Of course, we should ensure that check_n_jobs is actually called wherever necessary.
If we want to achieve full control over thread numbers, it would make sense to add the threadpool_limits context manager immediately after the call to check_n_jobs to set the limit for everything that comes after. To begin with, we could do that everywhere, where the n_jobs argument is already present. There might also be alternatives to explicitly using threadpool_ctl but that might require some research.
Whatever the solution is, it could be gradually rolled out to every other function in mne that implicitly parallelizes.

All these changes are non-breaking in the sense that actively setting n_jobs still produces the same result, but more consistently (which would appear like a bug fix). On the other hand the default behaviour for "not setting n_jobs" changes to "use all cores". Whether that is a bad thing or not depends on expectation, but it is coherent with scikit.

in the lab we ask users to set OMP_NUM_THREADS to 1 in their .bashrc
on the shared machines and to nice their jobs with "nice -5 python ..."

That would be enforcing monothreading, assuming it catches all cases (might need to add more enviroment variables, see this stackoverflow answer. Even so, wouldn't it make more sense to use a Python API instead of having users manipulate their .bashrc?

let me think about this. We will discuss this in the next MNE core dev meeting. You're welcome to join. It will be on Friday 22nd 5PM CET on MNE discord channel.
2 remarks:

in scikit-learn we default to 1 which is for me much less dangerous than defaulting to -1
using threadpool_limits context would mean indenting huge code blocks and I am not a fan to adding this everywhere. As I don't see then where we would not do this. We have numerical code everywhere and I've seen this done elsewhere but maybe you can point to other packages in the pydata ecosystem that have done what you suggest?

I have not thought about threadpoolctl much, but I've seen it used in SciPy (with modifications from sklearn):
https://github.com/scipy/scipy/pull/14441
And they mention there that it's also what's being used by NumPy. Given that scikit-learn uses Joblib to spawn new processes, as well, we can probably learn from their experience and try to do the same things.

using threadpool_limits context would mean indenting huge code blocks and I am not a fan to adding this everywhere.

Two ideas (and I think the second is better):

We've gotten around this before by changing things like fid = open(...) to with open(...) via:def my_fun(...):
    do_something_slow
    on_many_lines

todef my_fun(...):
    with context():
        _my_fun(...)

def _my_fun(...):
    do_something_slow
    on_many_lines

But really we should just do it by adding a @threadpool_controlled decorator or so that uses a context manager. Then we "just" need to add this to functions that need it. In practice we'd get it almost for free if we add it to verbose, which already decorates most slow functions anyway...

@dafrose we usually try to follow what sklearn does, under the assumption that they have thought about this stuff a lot. It sounds like they, in turn, mostly deligate to joblib. With that in mind, I propose we follow their model by:

Setting n_jobs=None which means "use 1 if nothing configured". By using joblib contexts, this can be effectively changed to other values by users. So you could do:
with joblib.parallel_backend('threading', n_jobs=4):
    raw.filter(..., n_jobs=None)

And by using the (now default) value of n_jobs=None, you'll end up using 4 threads. This uses threadpoolctl under the hood, so should play nicely with linalg libraries according to their docs.

Update filter_chpi to have n_jobs argument.

This seems like it would allow MNE functions that use n_jobs to take as many threads as available according to the joblib.parallel_backend param.

+1 to now default to None in n_jobs and use this semantic from sklearn

Message ID: @.***>

@agramfort thanks for the invite. I will see whether I can make it on Friday.
@larsoner +1 for adding the context manager as a decorator or to an existent decorator.
Regarding n_jobs=None: I like the idea, because it does not change the current default behaviour, but allows to hand over control to a lower-tier context. However, it should always be clear, what takes preference. Unless otherwise explained, I would expect that the call raw.filter(..., n_jobs=<a_number>) should overrule whatever an external context defines, unless the value is None as you defined above. Do I understand correctly, that setting n_jobs=-1 would still mean that all available cores are used?

Regarding n_jobs=None: I like the idea, because it does not change the
current default behaviour, but allows to hand over control to a lower-tier
context. However, it should always be clear, what takes preference. Unless
otherwise explained, I would expect that the call raw.filter(...,
n_jobs=<a_number>) should overrule whatever an external context defines,
unless the value is None as you defined above. Do I understand correctly,
that setting n_jobs=-1 would still mean that all available cores are used?Message
ID: @.***>

yes it's the sklearn behavior.

@larsoner thanks for the offer. I would love to, but I am afraid it would take some time. I already have a few PRs on my todo list, one of them already for mne and I haven't gotten to do any of them yet... So if it can wait for a few weeks TM, maybe. But I won't mind, if someone else did it until then.


chore: update MSW generated javascript

This commit adds the changes produced after updating the MSW version.

@indraraj @uidoyen fyi, I'll just merge this once it passes


Should Instances be able to communicate with an outside world?

@ludaesch pointed out that whenever you launch a Tale based on OpenRefine, there's a yellow banner saying: "There's a newer version of OpenRefine, please update". It turns out that OR performs http call during boot-up to check if newer version is available.
This particular example is benign. However, in general software could be doing something that's malicious (tracking users without their knowledge) or would make reproducible impossible (lazy downloading dependencies, which may go away). It can be easily prevented by disabling outgoing data transfer. It has some drawbacks:

software that really needs outgoing connections to work, won't work anymore in WT
installing software (pip, conda, apt etc) won't be possible, yet it's something that we plan to allow user to do as a part of "WT as a dev environment" scenario.

In Labs Workbench, we ran into the usual problem of mining and other undesirable activities.  This is still an open issue for us, but we've discussed 1) restricting outbound traffic to 80/443 (which doesn't solve the problem, if tunneling is used) or 2) restricting to a subset of known sites for OS packages, pypi, etc. I recall that Cyverse shuts off all egress. I personally like the idea of selective egress filtering.  With the move to Kubernetes, we could use Network Policies to have relatively configurable egress filtering using labels.


Can't install via JitPack

I was a fan of dart-petitparser and wanted to give the Kotlin version a try.
It seems like the build is broken on JitPack right now, so I can't add it as a dependency to my project until that's fixed.
https://jitpack.io/#petitparser/kotlin-petitparser

My project is running on GraalVM (so JVM, yeah?) and I'm seeing this during man install:
[ERROR] Failed to execute goal on project kiwi:
Could not resolve dependencies for project sh.scheme.kiwi:kiwi:jar:0.0.1-main-SNAPSHOT:
Failed to collect dependencies at com.github.petitparser:kotlin-petitparser-jvm:jar:1.0.0:
Failed to read artifact descriptor for com.github.petitparser:kotlin-petitparser-jvm:jar:1.0.0:
The following artifacts could not be resolved: com.github.petitparser:kotlin-petitparser-jvm:pom:1.0.0 (absent):
Could not transfer artifact com.github.petitparser:kotlin-petitparser-jvm:pom:1.0.0 from/to jitpack.io (https://jitpack.io):
status code: 401, reason phrase: Unauthorized (401) -> [Help 1]

This is what I added to my pom.xml:
    <dependency>
      <groupId>com.github.petitparser</groupId>
      <artifactId>kotlin-petitparser-jvm</artifactId>
      <version>1.0.0</version>
    </dependency>

I also tried adding just kotlin-petitparser but I got errors saying that I need some DOM-related packages:
[ERROR] Failed to execute goal on project kiwi:
Could not resolve dependencies for project sh.scheme.kiwi:kiwi:jar:0.0.1-main-SNAPSHOT:
The following artifacts could not be resolved: org.jetbrains.kotlin:kotlin-dom-api-compat:jar:1.9.0 (absent):
org.jetbrains.kotlin:kotlin-dom-api-compat:jar:1.9.0 was not found in https://jitpack.io during a previous attempt.
This failure was cached in the local repository and resolution is not reattempted until the update interval of jitpack.io has elapsed or updates are forced -> [Help 1]

I am not an expert in Java/Kotlin dependency management, but using com.github.petitparser.kotlin-petitparser:kotlin-petitparser:1.0.0 in Gradle worked for me. I guess the equivalent in Maven would be:
<dependency>
  <groupId>com.github.petitparser.kotlin-petitparser</groupId>
  <artifactId>kotlin-petitparser</artifactId>
  <version>1.0.0</version>
</dependency>

See the string templates at https://jitpack.io/#petitparser/kotlin-petitparser/1.0.0.

Thanks for correcting my groupId. I got it to install, thanks!


[#2945] Improve Yarn Compatibility

Closes #2945

Moves webpack and webpack-cli to list of peer dependencies
Updates commands to call webpack directly

I feel like this could break things. peer dependencies are not installed by default. They're meant to be provided by the root installed package.

@thecrypticace That's true, but postcss is already a peer dependency, so I figured users would already be fine with adding a couple more.
That said, I've set this back as a draft while I work on another solution that I've referenced in the linked issue. Thanks for taking a look!

Postcss is a peer dependency and we handle installation for you when you first run Mix if it's not installed. We could do the same with webpack but that seems to defeat the purpose since mix is a wrapper around webpack. If we wanted to such a change couldn't happen until Mix v7 at the earliest.

Oh, it does! Yarn 2+ issues a warning on first yarn install that the postcss is missing.
➤ YN0002: │ example@workspace:. doesn't provide postcss (pf3a0b), requested by laravel-mix

So, of course, I went ahead and installed it manually before calling yarn mix. But I just tried it right now without installing postcss before running yarn mix and sure enough mix installed it for me. And, yeah, that does complicate things. I'll keep poking around to see what potential solutions there might be. And thanks for the input! You've been very helpful.


color tag categories for Data & Products

For instance, we could:

[ ] Add legend with colors associated by category (region, org, tool, product type)
[ ] Add Javascript to page, eg now https://marinebon2.netlify.app/product_viewer/

$("[data-filter^='.js-id-region--']").
  css('background-color', 'blue');
$("[data-filter^='.js-id-org--']").
  css('background-color', 'green');
$("[data-filter^='.js-id-tool--']").
  css('background-color', 'red');
$("[data-filter^='.js-id-product_type--']").
  css('background-color', 'yellow');

Took some trial and error but it is done!

So cool! The text is hard to read though with current color palette:

See this ticket too https://github.com/marinebon/www_marinebon2/issues/39#issuecomment-1036629379


BUG: to_datetime when called with a unit and coerce is buggy

closes #11758

@jorisvandenbossche straightforward but lots of cases.
Stuff like mixed float/int/string (that are integer/float-like) with some NaT's thrown in (these should pass).


Store state of chat enlarge toggle

Store state of chat enlarge toggle in local storage, so it can be restored after reload or switching back from queue or file browsing.

I find it very useful to see users currently logged in when i open the interface. But i agree that it's annoying when navigating trough the tabs. I implemented a compromise...it's state is saved for the session. When u hit F5 it's restored.


[BUG] Custom TW class not recognized

Describe the bug
Custom TW classes are being underlined.
To Reproduce
Steps to reproduce the behavior:

On 1.14.3
extend Tailwind colors
Error still shown

Expected behavior
It should recognize the classes from the tailwind.config.js
Screenshots

Environment (please complete the following information):

OS: Windows 10
Softwares + version used:

VSCode (latest)
npm 6.14.13, node v14.17

Additional context
I've also added a config property to the eslint rule. That didn't help

Hi @asharonbaltazar, it looks like the plugin does not get the config.tailwind.js file...
I can help if you are willing to set up a dummy project demonstrating the issue, as simple as possible as long as it demonstrate the problem you are experiencing.

Okay, while I try to recreate the env, here's some code. Perhaps the error is obvious:
tailwind.config.js
const defaultTheme = require('tailwindcss/defaultTheme');
const colors = require('tailwindcss/colors');

module.exports = {
    ...
    theme: {
        extend: {
            ...
            textColor: {
                primary: '#48466D',
                ...
                alt: '#707070',
            },
        },
        plugins: [],
        colors: {
            ...defaultTheme.colors,
            transparent: 'transparent',
            ...
            lime: colors.lime,
        },
    },
};

.eslintrc.js
module.exports = {
    root: true,

    settings: {
        ...
        react: {
            version: 'detect',
        },
        tailwindcss: {
            callees: ['classNames'],
        },
    },
    plugins: ['@typescript-eslint', 'prettier', 'tailwindcss'],
    extends: ['airbnb', 'prettier', 'plugin:tailwindcss/recommended'],
    parser: '@typescript-eslint/parser',
    ...
    rules: {
              'tailwindcss/no-custom-classname': [
                  'error',
                  {
                      config: tailwind.config.js,
                      whitelist: ['abcRioButtonSvg'],
                  },
              ],
              ...
          },
};

Maybe it's a workaround, but I did require the file and used that (const tailwind = require('./tailwind.config')) instead of a path for the config property. It worked!

Glad you find a solution for your issue.
As indicated in the docs, the config can take a string (which is considered to be a path to the config to be loaded, or a plain object):

It is also possible to directly inject a configuration as plain object like { prefix: "tw-", theme: { ... } }.

In your case, requiring the file passes a plain object containing the tailwind config 🎉


Feature/validation messages

This looks good - we'll want to keep the locale per session, so that different users can have different locales. So I don't think it'll be an application property but rather some kind of 'lang' param. We'll want to document this as an example in the showcase as well.


Change PAC serviceaccount access based on feature enable/disable

ref https://github.com/openshift-pipelines/pipelines-as-code/pull/890#discussion_r999784219
auto-configure-new-github-repo is disabled by default so PAC service account shouldn't have access to create repo
we can update pac controller role based on whether user has enabled the feature or not

@chmouel whats the priority on this?


TURD: Research bonuses apply incorrectly to non-pyAL packs

Mod source
PyAE Beta
Which mod are you having an issue with?

[X] pyalienlife
[ ] pyalternativeenergy
[ ] pycoalprocessing
[ ] pyfusionenergy
[ ] pyhightech
[ ] pyindustry
[ ] pypetroleumhandling
[ ] pypostprocessing
[ ] pyrawores

Operating system

=Windows 10

What kind of issue is this?

[ ] Compatibility
[ ] Locale (names, descriptions, unknown keys)
[ ] Graphical
[ ] Crash
[ ] Progression
[ ] Balance
[ ] Pypostprocessing failure
[X] Other

What is the problem?
This mod purports to fix an issue with the TURD research bonuses for non-pyAL packs.  Double check and/or port as needed
https://mods.factorio.com/mod/Py_Fix_Research_TURD
Steps to reproduce
No response
Additional context
No response
Log file
No response

TURD still on development by kingarthur.

Fixed. Thanks for the report.


[APPM-83] 'Clear' button doesn't work in 'Web Application Resources' section in app create and edit screens.

Reference: https://wso2.org/jira/browse/APPM-83

Not related to IoT server, hence closing the issue.


Update git:// github.com submodule URL to use https://

Pull Request (PR) description
github.com permanently disabled unencrypted git on 15/3/22: https://github.blog/2021-09-01-improving-git-protocol-security-github/, which means that updating this repository's submodules currently fails.
This PR changes the submodule URL in question to an equivalent HTTPS URL that works.
cf. https://github.com/voxpupuli/puppet-augeasproviders_core/pull/38
This Pull Request (PR) fixes the following issues
n/a

I can remove it if you like. Just not sure if it is ok to do. I don't know this module well enough to make that call.

I would say that if CI passes, then it should be safe to do so. It's only used for testing and doesn't end up in the resulting module on the forge.

The submodule was removed.


Added page object model page for workitem detail page - intentional i…

…n the same module as the work item page as it acts like a child page. Building out the test infrastructure to enable the tests to remain small, easy to read/maintain.
Also - corrected the formatting - using default eclipse indentation.

Can one of the admins verify this patch?

@ldimaggi I'm getting the following Failure:
A Jasmine spec timed out. Resetting the WebDriver Control Flow.

@joshuawilson can you whitelist Len so that the Jenkins job starts?

@ldimaggi please rebase

@ldimaggi It still fails for me. I got the following.

Work item board should contain 14 items.
Message:
Failed: {"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept-Encoding":"gzip,deflate","Connection":"Keep-Alive","Content-Length":"27","Content-Type":"application/json; charset=utf-8","Host":"localhost:29592","User-Agent":"Apache-HttpClient/4.5.1 (Java/1.8.0_45)"},"httpVersion":"1.1","method":"POST","post":"{"id":":wdc:1475810227204"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/7ce074f0-8c3c-11e6-92ea-dda06741ac3d/element/:wdc:1475810227204/click"}}
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'josh-w541', ip: '<IP_ADDRESS>', os.name: 'Linux', os.arch: 'amd64', os.version: '4.7.2-101.fc23.x86_64', java.version: '1.8.0_45'
Driver info: driver.version: unknown

Works fine.


1 turn and 4 output

I just tried your library with the complete example (with button), but I got a weird output:
Simple Counter
RIGHT
1
RIGHT
2
RIGHT
3
RIGHT
4

Turned the encoder one time and got 4 output value.
I'm using WeMos D1 mini with KY-040, Arduino IDE v1.8.2

Hi,
I think this is due to the type of encoder.
You can now define in the constructor the number of clicks supposed to happen on a single turn.


Updates home readme for getting started experience

I received some feedback that the getting started experience was poor for AR so I've been updating toolkit doc across the board; this PR updates the readme slightly to match the pattern from the other repos (e.g. https://github.com/esri/arcgis-toolkit-dotnet)

@nCastle1 once this is merged, please create a follow up PR for the v.next branch.


[AIRFLOW-1495] Add TaskInstance index on job_id

Dear Airflow maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-1495

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

Column job_id is unindexed in TaskInstance, it was used as
default sort column in TaskInstanceView.
This commit adds the required migration to add the index on
task_instance.job_id on future db upgrades.
Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

Just the Alembic migration, I tested it manually with airflow upgradedb to verify the index creation.
Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #2520 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2520      +/-   ##
==========================================
- Coverage    70.1%   70.09%   -0.01%
==========================================
  Files         150      150
  Lines       11545    11545
==========================================
- Hits         8094     8093       -1
- Misses       3451     3452       +1

Impacted Files
Coverage Δ

airflow/models.py
87.29% <0%> (-0.05%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42cad60...a85dfd8. Read the comment docs.

Nice! LGTM.


Program Crash after vTaskDelete (IDFGH-2978)

Development Kit: [ESP32-Wrover-Kit|

Kit version (for WroverKit/): [v4]

Module or chip used: [|ESP32-WROVER-B|

IDF version :: v4.2-dev-701-g0ae960f2f

Build System: [|CMake & idf.py]

Compiler version xtensa-esp32-elf-gcc (crosstool-NG esp-2019r2) 8.2.0

Operating System: [|Linux|]

Using an IDE?: [|Yes eclipse & idf.py from command line

Power Supply: [USB|

Problem Description
Im trying to achieve the following:

connect to a wifi network
After the MQTT loop starts then, im starting another parallel task which shall publish MQTT topics.
IF the Wifi disconnects, then MQTT client is stopped and the task created should be deleted.
Upon reconnection the  MQTT client shall be restarted and along with the task.

Expected Behavior

Irrespective of disconnection the task shall be deleted and recreated without errors.

Actual Behavior

Even though the task handle in not NULL.

vTaskDelete(taskhandle);

crashed the program
53) HELLO :: sent publish successful, msg_id=0
task for loop 995 seconds...
I (644563) HELLO :: sent publish successful, msg_id=0
E (644563) MQTT_CLIENT: Client has not connected
I (644563) HELLO :: sent subscribe successful, msg_id=-1
I (644563) HELLO :: sent publish successful, msg_id=0
task for loop 994 seconds...
I (645073) HELLO :: sent publish successful, msg_id=0
E (645073) MQTT_CLIENT: Client has not connected
I (645073) HELLO :: sent subscribe successful, msg_id=-1
I (645073) HELLO :: sent publish successful, msg_id=0
task for loop 993 seconds...
I (645583) HELLO :: sent publish successful, msg_id=0
E (645583) MQTT_CLIENT: Client has not connected
I (645583) HELLO :: sent subscribe successful, msg_id=-1
I (645583) HELLO :: sent publish successful, msg_id=0
task for loop 992 seconds...
I (646093) HELLO :: sent publish successful, msg_id=0
E (646093) MQTT_CLIENT: Client has not connected
I (646093) HELLO :: sent subscribe successful, msg_id=-1
I (646093) HELLO :: sent publish successful, msg_id=0
I (646253) wifi:state: run -> init (3c0)
I (646253) wifi:pm stop, total sleep time: 3820150 us / 4282049 us

I (646253) wifi:new:<2,0>, old:<2,0>, ap:<255,255>, sta:<2,0>, prof:1
WIFI_EVENT EVENT WIFI_EVENT_STA_DISCONNECTED : the event id is 5
WIFI_EVENT EVENT eTaskGetState(hello_task) : the state is 1
WIFI_EVENT task !NULL ***<PHONE_NUMBER>
I (647003) wifi station: retry to connect to the AP
I (647003) wifi station: connect to the AP fail
WIFI_EVENT EVENT WIFI_EVENT_STA_DISCONNECTED : the event id is 5
WIFI_EVENT EVENT eTaskGetState(hello_task) : the state is 1
WIFI_EVENT task !NULL ***<PHONE_NUMBER>
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x400954a3  PS      : 0x00060433  A0      : 0x80094181  A1      : 0x3ffd56b0
A2      : 0x3ffdf39c  A3      : 0xffffffff  A4      : 0x00000001  A5      : 0x3ffd55a0
A6      : 0x00000026  A7      : 0x3ffd55a0  A8      : 0x00000000  A9      : 0x3ffb3b50
A10     : 0x3ffc1348  A11     : 0x00000000  A12     : 0x00060420  A13     : 0x00060423
A14     : 0x00000001  A15     : 0xff000000  SAR     : 0x00000004  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000004  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff

Backtrace:0x400954a0:0x3ffd56b0 0x4009417e:0x3ffd56d0 0x400d31e9:0x3ffd56f0 0x4015c15d:0x3ffd5720 0x4015bbf6:0x3ffd5750 0x4015bcd1:0x3ffd5790 0x40094b79:0x3ffd57b0

ELF file SHA256: cd59ddc1587f039b

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x1e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4
load:0x3fff0034,len:6892
load:0x40078000,len:14332
ho 0 tail 12 room 4
load:0x40080400,len:3964
entry 0x40080680
I (31) boot: ESP-IDF v4.2-dev-904-g7cbc593ef 2nd stage bootloader
I (31) boot: compile time 14:05:23
I (31) boot: chip revision: 1
I (35) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (42) boot.esp32: SPI Speed      : 40MHz
I (47) boot.esp32: SPI Mode       : DIO
I (51) boot.esp32: SPI Flash Size : 4MB
I (56) boot: Enabling RNG early entropy source...
I (61) boot: Partition Table:
I (65) boot: ## Label            Usage          Type ST Offset   Length
I (72) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (80) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (87) boot:  2 factory          factory app      00 00 00010000 00100000
I (95) boot: End of partition table
I (99) boot_comm: chip revision: 1, min. application chip revision: 0
I (106) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x1e454 (123988) map
I (162) esp_image: segment 1: paddr=0x0002e47c vaddr=0x3ffb0000 size=0x01b9c (  7068) load
I (165) esp_image: segment 2: paddr=0x00030020 vaddr=0x400d0020 size=0x8c348 (574280) map
I (385) esp_image: segment 3: paddr=0x000bc370 vaddr=0x3ffb1b9c size=0x01db0 (  7600) load
I (389) esp_image: segment 4: paddr=0x000be128 vaddr=0x40080000 size=0x00400 (  1024) load
I (392) esp_image: segment 5: paddr=0x000be530 vaddr=0x40080400 size=0x15124 ( 86308) load
I (450) boot: Loaded app from partition at offset 0x10000
I (450) boot: Disabling RNG early entropy source...
I (450) cpu_start: Pro cpu up.
I (454) cpu_start: Application information:
I (459) cpu_start: Project name:     wifi_station
I (464) cpu_start: App version:      1
I (468) cpu_start: Compile time:     Mar 27 2020 14:05:22
I (474) cpu_start: ELF file SHA256:  cd59ddc1587f039b...
I (480) cpu_start: ESP-IDF:          v4.2-dev-904-g7cbc593ef
I (487) cpu_start: Starting app cpu, entry point is 0x40081204
I (478) cpu_start: App cpu up.
I (497) heap_init: Initializing. RAM available for dynamic allocation:
I (504) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (510) heap_init: At 3FFB9540 len 00026AC0 (154 KiB): DRAM
I (517) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (523) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (529) heap_init: At 40095524 len 0000AADC (42 KiB): IRAM
I (536) cpu_start: Pro cpu start user code
I (554) spi_flash: detected chip: generic
I (554) spi_flash: flash io: dio
I (555) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (563) wifi station: [APP] Startup..
I (563) wifi station: [APP] Free memory: 261248 bytes
I (573) wifi station: [APP] IDF version: v4.2-dev-904-g7cbc593ef
I (613) wifi station: ESP_WIFI_MODE_STA
I (623) wifi:wifi driver task: 3ffd7360, prio:23, stack:3584, core=0
I (623) system_api: Base MAC address is not set
I (623) system_api: read default base MAC address from EFUSE
I (643) wifi:wifi firmware version: b88ceb8
I (643) wifi:wifi certification version: v7.0
I (643) wifi:config NVS flash: enabled
I (643) wifi:config nano formating: disabled
I (653) wifi:Init dynamic tx buffer num: 32
I (653) wifi:Init data frame dynamic rx buffer num: 32
I (663) wifi:Init management frame dynamic rx buffer num: 32
I (663) wifi:Init management short buffer num: 32
I (673) wifi:Init static rx buffer size: 1600
I (673) wifi:Init static rx buffer num: 10
I (683) wifi:Init dynamic
Steps to reproduce
see attached project zip file please.
station_issue.zip

Maybe you are trying to delete task that is already deleted and taskhandle is no longer valid, or it is not valid at all?

@chegewara if you look at this line
task !NULL ***<PHONE_NUMBER>

and my code in the .zip file .
Im checking if the taskhandle is not null. when the task is not created then the task handle points= NULL.
See the .zip file

Just look at this logs:
WIFI_EVENT task !NULL ***<PHONE_NUMBER>
I (647003) wifi station: retry to connect to the AP
I (647003) wifi station: connect to the AP fail
WIFI_EVENT EVENT WIFI_EVENT_STA_DISCONNECTED : the event id is 5
WIFI_EVENT EVENT eTaskGetState(hello_task) : the state is 1
WIFI_EVENT task !NULL ***<PHONE_NUMBER>

It looks like you are trying to delete the same task twice:
WIFI_EVENT task !NULL ***<PHONE_NUMBER>

The answer is very simple, I done it multiple times when i was more beginner:
You call vTaskDelete(loopHandle), and you assume, that it will be NULL-pointer.
Unfortunately it will stay on the address where the task was, and this is the cause it crashes.
Well, the task will be deleted, but that does nothing to the incoming parameter (the pointer).
After you delete the task, you have to set the loopHandle variable to NULL manually.

@bitmakk thanks for the tip. Yes the code works now.
Questoin : is there a cleaner way to do it. Monitoring to check if the task is deleted is not working either because, in getting a stack overflow in sys event. Increasing the stack size did not solve the problem

@bitmakk thanks for the tip. Yes the code works now.
Questoin : is there a cleaner way to do it. Monitoring to check if the task is deleted is not working either because, in getting a stack overflow in sys event. Increasing the stack size did not solve the problem

It is clean, what you have to do is:
if(taskhandle != NULL){
vTaskDelete(taskhandle);
taskhandle = NULL;
}

@chegewara Ok. Cool.


article about glassmorphism on android

This is an android explaining how to achieve glassmorphism on android using Blurview Library.

Hello @othimar!
Thank you for your contribution!
Please, can you delete the package-lock.json file, as it is not needed here?

Hey @othimar!
Please, rebase your branch to main. I fixed the GitHub actions, so it will work know.

@othimar there is an issue with the yarn.lock file. Please, can you delete it and run yarn install again?

I deleted the yarn.lock file. But I'm getting errors every time I run yarn install. I even cloned the original project but I keep getting those errors. I'm going to try again tomorrow.


XMPP PubSub

Greetings!
I am Schimon, and I would want to advise to support XMPP PubSub (more accurately XMPP PEP) as a mean to store bookmarks.
The Blasta project store bookmarks on XMPP profiles and also aggregates bookmarks from XMPP profiles to generate a public database of bookmarks.
Please refer to the project Blasta for realization.

https://git.xmpp-it.net/sch/Blasta (code)
https://video.xmpp-it.net/w/cfozoUeVLFbBFMCCSCJ1Dn (video)
https://blasta.woodpeckersnest.eu (instance)

Integration with linkding https://git.xmpp-it.net/sch/Blasta/issues/16

I have read some of the reviews for the linkding extension and I have a comment.
Chris Foobar

Each page view is transmitted to the linkding server. Regardless of whether the page is to be bookmarked or not. Presumably this is to check whether an entry already exists. If this is the case, it would be a very poor implementation. It would be enough if this check takes place when the user decides to bookmark.

Developer response

The behavior you describe is opt-in, it must be explicitly enabled in the extension's options ("Preload page information while browsing"). By default the extension will only check whether a URL is bookmarked when triggering the bookmark action. You might have an outdated version of the extension, or you have ran into a bug, in which case I would ask you to report it in the extensions's Github repo.

My response
The XMPP implementation of Blasta is only client side. The HTTP is of type server, but the communication with XMPP is of type client.
Hence, the check for an existing bookmark is made against each XMPP account, which means, that there is no need to connect to a Blasta server to conduct this check.
So, this is a benefit in hosting bookmarks on XMPP PubSub.

Once a big and established poject such as linkding would implement XMPP, the we would rapidly add a new XEP for utilizing PubSub to store annotations.
Please join to our XMPP group chat to discuss this matter xmpp:schapps@chat.woodpeckersnest.space?join

I have read some of the reviews for the linkding extension and I have a comment.
Chris Foobar

Each page view is transmitted to the linkding server. Regardless of whether the page is to be bookmarked or not. Presumably this is to check whether an entry already exists. If this is the case, it would be a very poor implementation. It would be enough if this check takes place when the user decides to bookmark.

Developer response

The behavior you describe is opt-in, it must be explicitly enabled in the extension's options ("Preload page information while browsing"). By default the extension will only check whether a URL is bookmarked when triggering the bookmark action. You might have an outdated version of the extension, or you have ran into a bug, in which case I would ask you to report it in the extensions's Github repo.

My response
The XMPP implementation of Blasta is only client side. The HTTP is of type server, but the communication with XMPP is of type client.
Hence, the check for an existing bookmark is made against each XMPP account, which means, that there is no need to connect to a Blasta server to conduct this check.
So, this is a benefit in hosting bookmarks on XMPP PubSub.

Once a big and established poject such as linkding would implement XMPP, the we would rapidly add a new XEP for utilizing PubSub to store annotations.
Please join to our XMPP group chat to discuss this matter xmpp:schapps@chat.woodpeckersnest.space?join


Preventing SDKs from automatically adding assemblies

Consider as an example here the VS SDK.  When installed on a machine it will cause new references to be available to programs by default.
This is certainly convenient for many simple applications.  It allows for references to be added with no alterations to the standard MSBuild project file.
This is decidedly inconvenient though for more complex applications.  In particular those where predictability of the build is more important than little conveniences like this.  For such projects it would be nice to opt-out of automatically inserting references to the build.  That helps ensure builds are consistent from machine to machine.
Take the following PR as an example of why this important.  There are two commits here.  The first of which builds clean on a machine with Dev14, but fails on a machine with Dev15.  This happens because the Dev15 SDK inserts extra references that causes conflicts with the change.  It took the second commit to cleanup the Dev15 SDK errors.
Ideally we could avoid this problem by having a clean / predictable build option for MSBuild that ignored global state like the VS SDK.

This would be fantastic - as a (hopefully short) aside, one of my long term goals for the VSSDK reference story is to get completely out of installing any reference assemblies at all but it doesn't seem reachable in the near future - it would break many existing projects. We (our team) need to do better to support deterministic building and the best tool for that right now is NuGet.
Having said that, I also know that usage of AssemblysFolderEx in the registry is pretty widespread - I would be in support of a build option that allows the assembly search path to be configurable. Also, I believe this is similarly proposed in #772.

I've made some progress here by customizing the AssemblySearchPaths property:

https://github.com/dotnet/roslyn/pull/16249

Been able to remove GAC, AssemblyFoldersEx, AssemblyFolders and a couple of other items from our search so far.
It's workable and gets me close to the proposal.  The problem though is I feel like this is still very undocumented.  The original value for AssemblySearchPaths is the following:
    <AssemblySearchPaths Condition=" '$(AssemblySearchPaths)' == ''">
      {CandidateAssemblyFiles};
      $(ReferencePath);
      {HintPathFromItem};
      {TargetFrameworkDirectory};
      {Registry:$(FrameworkRegistryBase),$(TargetFrameworkVersion),$(AssemblyFoldersSuffix)$(AssemblyFoldersExConditions)};
      {AssemblyFolders};
      {GAC};
      {RawFileName};
      $(OutDir)
    </AssemblySearchPaths>

There are some comments explaining the meaning of some of these values.  For others though the only way I could discern what they did was to simply delete them, rebuild and see what broke.  Would be nice to better understand what these values mean.

@jaredpar What if we introduce a boolean property for controlling this?  Let's say it's called RestrictAssemblySearchPaths and when you set it to true, then the default AssemblySearchPaths is your narrowed down list.  This would allow users to toggle it for code bases where they only want references to come from packages or the build itself.  Users could still override the search paths with a custom list but a toggle like this would probably satisfy the major scenarios.

@jeffkl that might work.  I'm still a bit uncertain though exactly what I've narrowed my list down to.  I have some ideas on what they are but that's mostly based on what broke when I deleted them.  Maybe it would help to understand a bit better what these mean and what type of references are expected in those locations.

There is documentation for the include paths in the SearchPaths section of https://docs.microsoft.com/en-us/visualstudio/msbuild/resolveassemblyreference-task, but it's a bit stale ({AssemblyFolders} isn't actually restricted to Visual Studio.NET 2003, for instance).

We've talked about it but I didn't see it written down here:
In addition to a big "don't look in the machine-dependent locations" flag, we should also have an off-switch property for each location. That way, even a project that uses one of the "bad" locations can shut off the others.

I have no idea how we didn't notice this until now but I'm going to mark this as a duplicate (we had almost the exact same discussion there).

Duplicate of #772


X-Formers install causing issues

Whenever I install xformers it gives me errors when I try to create an image in automatic1111. When I rerun the install block as suggested it gives me a ton of errors and won't even launch the webui. Is anyone else having issues? I miss being able to have xformers working. Only fix is to shutdown the machine and start over.

Ok well I'll need some logs.


Fix Bad Sad on Stackoverflow errors

Purpose
Fixes #18966
Stacktrace will be printed in console if doesn't trap.
error: {ballerina}StackOverflow
	at main:bar(main.bal:2)
	   main:bar2(main.bal:6)
	   main:bar(main.bal:2)
	   main:bar2(main.bal:6)
	   main:bar(main.bal:2)
	   main:bar2(main.bal:6)
	   main:bar(main.bal:2)
	   main:bar2(main.bal:6)
	   main:bar(main.bal:2)
	   main:bar2(main.bal:6)

Approach
Try catch block code gen has been improved to handle stackoverflow methods as well.
ex. sample codegen
      ErrorValue e1;
        try{

        } catch (ErrorValue e) {
            e1 =e;
        } catch (StackOverflowError e) {
 // Future we can add any exception types here
 // if we need to trap them as well. ex StackOverflowError | Ex1 | Ex2
            e1 = BallerinaErrors.trapError(e);
        }

Check List

[ ] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #19101 into ballerina-1.0.x will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff                @@
##           ballerina-1.0.x   #19101   +/-   ##
================================================
  Coverage            15.73%   15.73%
================================================
  Files                   47       47
  Lines                 1265     1265
  Branches               197      197
================================================
  Hits                   199      199
  Misses                1053     1053
  Partials                13       13

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 03b2b1e...c664864. Read the comment docs.


Fix cnn_dailymail (dm stories were ignored)

https://github.com/huggingface/datasets/pull/4188 introduced a bug in datasets 2.2.0: DailyMail stories are ignored when generating the dataset.
I fixed that, and removed the google drive link (it has annoying quota limitations issues)
We can do a patch release after this is merged

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Cannot open TUN/TAP dev /dev/net/tun: No such file or directory

Hello,
Did I do something wrong?
As far as I can tell this should work:
core@mirror ~ $ docker run --volume /etc/openvpn:/etc/openvpn -p 1194:1194/udp --cap-add=NET_ADMIN --rm kylemanna/openvpn openvpn --config "network7.conf"
[...]
Tue Apr  7 10:54:47 2015 ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)
Tue Apr  7 10:54:47 2015 Exiting due to fatal error

Maybe the fact this is Core OS is significant, can't see why though - Core OS itself has tun support:
core@mirror ~ $ ls -l /dev/net/tun
crw-rw-rw- 1 root root 10, 200 Apr  7 10:48 /dev/net/tun

Also:
core@mirror ~ $ docker --version
Docker version 1.5.0, build a8a31ef-dirty

Hmmm. ----privileged works fine, but the documentation says that is only required for Docker < 1.2 - wonder if some other --cap-add= option is required for 1.5???
Thanks

Not sure what the problem is.  Seems like an issue with your kernel or hosting provider.  Can you share these details?

VM is from Hetzner: https://www.hetzner.de/en/hosting/produkte_vserver/vx6
Kernel is standard CoreOS:
CoreOS stable (607.0.0)
core@mirror ~ $ uname -a
Linux mirror 3.18.6 #2 SMP Sat Feb 28 02:23:04 UTC 2015 x86_64 Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz GenuineIntel GNU/Linux
Actually might have to do a rethink on how I do this. Now I understand things better - I was hopping to be able to update the routes on the host server, something that may not actually be possible from within a docker container...

Same problem for me.
Host OS is an archlinux LTS up to date.
Linux 3.14.28-1-lts #1 SMP Thu Jan 8 21:04:11 CET 2015 x86_64 GNU/Linux

Everything went fine until the start of the container, i the have :
Sun Apr 12 21:38:42 2015 ROUTE_GATEWAY <IP_ADDRESS>/<IP_ADDRESS> IFACE=eth0 HWADDR=02:42:ac:11:00:50
Sun Apr 12 21:38:42 2015 ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such device (errno=19)
Sun Apr 12 21:38:42 2015 Exiting due to fatal error

A quick google seem to indicate that the problem is about the --privileged flag, but i've run my container as stated within the docs with --cap-add=NET_ADMIN
docker --version                                                                                [21:44:07]
Docker version 1.5.0, build a8a31ef

Seems strange.  Works fine for me on a fresh DO droplet running Ubuntu
14.04 and Docker 1.5-1 via docker.io.  Command:
docker run --volumes-from ovpn-data --rm -p 1194:1194/udp --cap-add=NET_ADMIN kylemanna/openvpn

Kernel version via uname -a:
Linux openvpn-test-20150413 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
On Sun, Apr 12, 2015 at 12:48:17PM -0700, Serge wrote:

Same problem for me.
Host OS is an archlinux LTS up to date.
Linux 3.14.28-1-lts #1 SMP Thu Jan 8 21:04:11 CET 2015 x86_64 GNU/Linux
Everything went fine until the start of the container, i the have :
Sun Apr 12 21:38:42 2015 ROUTE_GATEWAY <IP_ADDRESS>/<IP_ADDRESS> IFACE=eth0 HWADDR=02:42:ac:11:00:50
Sun Apr 12 21:38:42 2015 ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such device (errno=19)
Sun Apr 12 21:38:42 2015 Exiting due to fatal error
A quick google seem to indicate that the problem is about the --privileged
flag, but i've run my container as stated within the docs with --cap-add=
NET_ADMIN
docker --version                                                                                [21:44:07]
Docker version 1.5.0, build a8a31ef
—
Reply to this email directly or view it on GitHub.*

Seems strange.  Works fine for me on a fresh DO droplet running Ubuntu
14.04 and Docker 1.5-1 via docker.io Debian repo.  Command:
docker run --volumes-from ovpn-data --rm -p 1194:1194/udp --cap-add=NET_ADMIN kylemanna/openvpn

Kernel version via uname -a:
Linux openvpn-test-20150413 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

The ovpn_run script will create it for you if needed:

Thanks.
Think I must have missed this. Don't know how @SR-G was starting openvpn, however looks like I was doing it wrong... Unfortunately, I am not in a position to test this any longer.

Thanks for the update.  Closing unless someone can reproduce.

hi,
FYI, I just experienced the same problem:
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

on a Raspberry Pi 1 Host with Hypriot 0.6:
$ uname -a
Linux openvpn 4.1.12-hypriotos+ #1 PREEMPT Tue Nov 3 19:37:31 UTC 2015 armv6l GNU/Linux

$ docker --version
Docker version 1.9.0, build 76d6bc9

I figured that it works with the --privileged option, but not with --cap-add=NET_ADMIN.
however, as I did not want to give the docker container full privileged access, I tried a little around and finally figured: it works if you also add the /dev/net/tun device:
docker run --cap-add=NET_ADMIN --device=/dev/net/tun ...

so maybe this helps someone out there, for me it works fine now. :-)

@blacky14 Thanks for the hint wrt --device=/dev/net/tun. That's the tweak I needed with my docker run command in a manually configured Alpine container (not with docker-openvpn). As you said, it saves from having to run mknod and works just as well.

I am seeing this problem with CentOS Atomic.
Specifically, either of these:
docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN kylemanna/openvpn
docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN --device=/dev/net/tun kylemanna/openvpn
Results in this:
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory
The only thing that doesn't is this:
docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --privileged kylemanna/openvpn
Installation particulars:
[root@admin ~]# uname -a
Linux admin 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@admin ~]# cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)
[root@admin ~]# docker --version
Docker version 1.12.6, build 3a094bd/1.12.6
[root@admin ~]#
The ISO I started with came from here: http://cloud.centos.org/centos/7/atomic/images/CentOS-Atomic-Host-7-Installer.iso

In my case the issue was that CentOS Atomic uses SELinux by default. I had to do the following prior to following the README instructions:
checkmodule -M -m -o docker-openvpn.mod docker-openvpn.te
semodule_package -o docker-openvpn.pp -m docker-openvpn.mod
semodule -i docker-openvpn.pp
modprobe tun
This is clearly stated in the docs but I had only read the quick start. Might be worth mentioning in the main README at the top of Quick Start that some precursor steps need to be taken if you're using SELinux.

@vchav73 thanks for the update and glad to hear you got it working!

mkdir -p /dev/net
mknod /dev/net/tun c 10 200
chmod 600 /dev/net/tun


Request for defund Fortuna, Can email

Hi @ksanguinetti, thanks for your interest in adding Fortuna to the website. Emails on defund12.org are all written by community members. To add your city, you must provide:

Email addresses (at least one) of the government officials you’d like to receive the message
Detailed message concerning the budgeting decisions at stake in your community

Until you provide this content, we cannot post your city.
Follow our email style guidelines for quicker acceptance. Remember that messages must be city-specific and focused on defunding the police. If your message is copied from an external source, please provide a link to the original source.
Your request will be closed until you reply with the necessary content. Thank you!


[IZPACK-505] Installer is showing progress in English for Simplified Chinese

@hiteshbosamiya thanks, good catch!


SignTransaction get wrong result while chainId is not 61

In emerald-wallet, it shows that Emerald Vault returned signature from wrong Sender
the tx sent to emerald-js is right so I guess it might be emerald-rs (emerald-cli)'s fault, But I dont know how to deal with it.
New chainId is 111. Do I need to change the calculation of sigV?

@maoxs2, nope no changes to siqV required from user.
Will test and fix it.

Bug still exists.
My chainId is 111, so if do calc the v will be 257 which is more than u8::max_value(), so it will lead to an overflow.
solved with removing all u8 chainId/sig.v and replaced with u16.
Thanks for your help.


DOC-4197: add TCEs to the geospatial query page

Description

Tabbed code examples (TCEs) for the geospatial query page. Please merge as soon as possible.

Checklist

[ ] Does npm test pass with this change (including linting)?
[ ] Is the new or changed code fully tested?
[ ] Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

dont know why this test is failing, when I checkout the pr and run it manually it passes.

i think I see why, this version isn't up to date with GEO support maybe.

The test runs correctly on my system. I'm not really sure what the issue is.


Add downloadable fonts  (Google flav. only- OSS falls back to current system "monospace", fixes #458)

This actually adds some functionality, so please check the crap out of it.  I tried to make it as compatible with #565 as I could, so that if you wanted to ALSO bundle in your own fonts beyond these downloadable ones, the code from @DAlexIV will hopefully slip right in.
Also, this was definitely working before the slew of changes added in #608 was merged.  I built it again post-those changes and it still seemed to work, but wasn't tested as thoroughly.
Also-- some of the monospaced fonts (VT323 and Space Mono) kern a few letter combos like "ff" or "fi" which narrow the space used and it's not quite monospace any more and that screws up Connectbot's backspacing.  I looked briefly for ways to ignore kerning hints (there's a deprecated attribute/method) in that Paint object and it still looks possible, but I didn't explore too much.  For now those two fonts are commented out.
Anyway seems to work okay.  Please try it!
Here are those chromebook images again. Works great in Chrome OS.  If I think of anything else to add i'll do so.
Again, thanks to @DAlexIV who got this started before the move to minsdk 14 made the downloadable part possible.
ft

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

While you're using the Chromebook, could you fix the double back arrow
thing? I forgot what the flag was they presented at Google I/O, but it was
something simple.
Embarassingly I don't have a new enough Chromebook to develop on.

Sure...I think you're talking about this, no?
<meta-data android:name=
     "WindowManagerPreference:SuppressWindowControlNavigationButton"
     android:value="false" />

Do you want this backspace protection on all activities?  Override onBackButtonPressed() everywhere too?
ft

I'm not sure where onBackButtonPressed is called from (from onKey somehow), so we might already intercept it where it makes sense.

I'll take a look-- I don't think the current Samsung Pro (my CB) is honoring the manifest flag yet though so it makes it kind of hard to test/confirm it's working.  Let me play with it a bit.
Also, I was thinking-- having multiple windows to permit more than one connection at a time might be cool.  That's just a thought-- i haven't tried it yet.  Off the top of your head, do you see any situation in which having more than one active session active in a window simultaneoulsy might confuse things?

Just following up-- did a ChromeOS update yesterday and the double-back arrows persist even on Google apps like the Play Store..  When the manifest settings are recognized by my CB and I can confirm everything works, I'll submit the changes as a separate PR.

Hey @Kruton-- quick question-- this commit, #612, #611, #610, #609 etc have been sitting around for 7 months now and are going out of date.  I'm also looking forward to #627, #615, and others.  Is there anything further that needs to be done to get you to accept these commits?  I thought maybe the project was dead, but I see there is ongoing development in the last few weeks.
I'd like to keep the Connectbot up to date on my chromebook with upstream changes, but w/the font and other support that I see languishing in the PR section.
Please advise & happy holidays.

Sorry @fat-tire I'm rewriting everything involving the UI and I'm a bit behind schedule. I'll take a look at all these PRs and get them working with the new UI rewrite.

@kruton -- Gotcha.  That's exciting to hear.  Looking forward to the new UI and the updates!
Happy holidays (again)

Hey quick update-- since this was written almost a year ago, androidX/Jetpack was introduced by Google, so you'll probably want to replace:
import android.support.v4.provider.FontRequest;
import android.support.v4.provider.FontsContractCompat;

with
import androidx.core.provider.FontRequest;
import androidx.core.provider.FontsContractCompat;

And of course add the above to build.gradle.
Cheers.

Hey folks, could someone share what happened here, the PR was great forever with merge on hold, then it suddenly just died without any explanation? I really wished this for support of nerd fonts.

Nevermind... just moved to Termux.

Dunno what happened besides four+ years passing.  :man_shrugging:

@fat-tire, okay no problem, Termux is just much better and I was missing on it, so it was in the end good for me to have this missing which forced into alternatives. Sad your work seem to have been wasted.


prow/e2e-bookInfoTests-trustdomain.sh stuck forever

See: https://github.com/istio/istio/pull/12844
It has been in state:
prow/e2e-bookInfoTests-trustdomain.sh Expected — Waiting for status to be reported
For over 24 hours

I don't think that's correct, I just approve the change to enable the test https://github.com/istio/test-infra/pull/1193

I reran the test and it worked fine, so if we aren't seeing this anywhere else I guess its fixed


Update ur_control.launch.py

Removed typo that was creating an error when launching

ros2 launch ur_bringup ur_control.launch.py ur_type:=ur5 robot_ip:=yyy.yyy.yyy.yyy use_fake_hardware:=true launch_rviz:=true
[INFO] [launch]: All log files can be found below /home/thomas/.ros/log/2022-02-17-18-49-03-979371-ubuntu-fx505dv-450590
[INFO] [launch]: Default logging verbosity is set to INFO
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<LaunchService._process_one_event() done, defined at /opt/ros/galactic/lib/python3.8/site-packages/launch/launch_service.py:226> exception=SubstitutionFailure("executable 'spawner.py' not found on the libexec directory '/home/thomas/workspace/ros_ur_driver/install/controller_manager/lib/controller_manager' ")>
Traceback (most recent call last):
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/launch_service.py", line 228, in _process_one_event
await self.__process_event(next_event)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/launch_service.py", line 248, in __process_event
visit_all_entities_and_collect_futures(entity, self.__context))
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
[Previous line repeated 2 more times]
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 38, in visit_all_entities_and_collect_futures
sub_entities = entity.visit(context)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/action.py", line 108, in visit
return self.execute(context)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch_ros/actions/node.py", line 484, in execute
ret = super().execute(context)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/actions/execute_process.py", line 837, in execute
self.__expand_substitutions(context)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/actions/execute_process.py", line 681, in __expand_substitutions
cmd = [perform_substitutions(context, x) for x in self.__cmd]
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/actions/execute_process.py", line 681, in
cmd = [perform_substitutions(context, x) for x in self.__cmd]
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/utilities/perform_substitutions_impl.py", line 26, in perform_substitutions
return ''.join([context.perform_substitution(sub) for sub in subs])
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/utilities/perform_substitutions_impl.py", line 26, in
return ''.join([context.perform_substitution(sub) for sub in subs])
File "/opt/ros/galactic/lib/python3.8/site-packages/launch/launch_context.py", line 197, in perform_substitution
return substitution.perform(self)
File "/opt/ros/galactic/lib/python3.8/site-packages/launch_ros/substitutions/executable_in_package.py", line 84, in perform
raise SubstitutionFailure(
launch.substitutions.substitution_failure.SubstitutionFailure: executable 'spawner.py' not found on the libexec directory '/home/thomas/workspace/ros_ur_driver/install/controller_manager/lib/controller_manager'

The file spawner.py doesn't exist and is just "spawner".

This is indeed not a typo, but that was a migration over different ROS versions.
Nevertheless, this should not cause any issues currently, as for both, the galactic and noetic release still support using the spawner.py notation. Depending on the branch and version that you are building your controller_manager (you seem to be building from source) this could introduce "special" situations.
However, I don't really see anything against making that change, support for the .py notation will probably be dropped at some point. And since using spawner also works in galactic, I think we can make that change now, as well. @destogl would you agree?


Support for aliases

I just experienced that nix search netcat gave me the result netcat-openbsd while search.nixos.org didn't, which I guess is because netcat-openbsd is an alias for libressl.nc. I don't know if it's possible/feasible to support aliases here, but if it is then it would probably help a lot of people, since aliases are used to name things so they are easier to find for users.

@kristoff3r thank you for reporting. This I hope to get addressed with #39

This should be actually addressed when we add flakes support (#127), since there we will actually rewrite how we export the package set.
I'm closing this ticket, but creating a reference and a comment in flakes issue.


Illegal Date 2017-06-31 to 2017-06-30

Source news report confirmed actual retirement date of Friday June 30 2017. This was causing fun date parsing errors :)

Pull Request Test Coverage Report for Build 511

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 64.333%

Totals

Change from base Build 510:
0.0%

Covered Lines:
965

Relevant Lines:
1500

💛  - Coveralls

thanks!

Ideally scripts/retire.py would be updated to reject illegal dates. I'll open an issue and do the work if y'all don't mind.

That'd be great!
On Mon, Nov 11, 2019 at 10:54 AM Lee Henderson<EMAIL_ADDRESS>wrote:

Ideally scripts/retire.py would be updated to reject illegal dates. I'll
open an issue and do the work if y'all don't mind.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/openstates/people/pull/195?email_source=notifications&email_token=AAAB6YR5GCLW5SLC5PJ4Y2DQTF5Z5A5CNFSM4JLV3PP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDXH5AI#issuecomment-552500865,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAB6YVBUUE65HYWNZRBADDQTF5Z5ANCNFSM4JLV3PPQ
.


Page hangs during in Facebook window after clicking 'Post'

What is your Test Scenario?

I'm trying to create a petition on change.org, and share it to Facebook.
What is the Current behavior?

I'm able to log in to Facebook and initiate the Share, however the page hangs after clicking 'Post' in Facebook. The same thing happens if I also choose 'Cancel'.
I'm unable to manually test this in the TestCafe browser due to this issue.
What is the Expected behavior?

Expected behaviour would be for the page to re-direct when either clicking 'Cancel' or 'Post'
What is your web application and your TestCafe test code?

Your website URL (or attach your complete example):
www.change.org

Your complete test code (or attach your test files):

const firstNameInput = Selector('[name="firstName"], [name="first_name"]').filterVisible();
const lastNameInput = Selector('[name="lastName"], [name="last_name"]').filterVisible();
const emailInput = Selector('.js-email-sign, input[name="email"]').filterVisible();
const petitionAddress = '/p/united-nations-send-matt-damon-to-mars-to-retrieve-opportunity';

fixture('Share Petition to Facebook').page('https://www.change.org');

 test.('Guest user should see facebook when sharing a petition', async () => {
  await t.navigateTo(petitionAddress);
  await t
      .typeText(firstNameInput, "Some")
      .typeText(lastNameInput, "Name")
      .typeText(emailInput<EMAIL_ADDRESS>      .click(Selector('button[type="submit"]'))
      .click(Selector('.js-skip'));

// initiate share:
  await t
      .click(Selector('.js-facebook-share-dialog'))
      .typeText(Selector('[id="email"]'), "your.email")
      .typeText(Selector('[id="pass"]'), "your.password")
      .click(Selector('[name="login"]'));

// Write message and share
await t
      .typeText(Selector('.mentionsTextarea'), "Random Text")
      .click(Selector('[name="__CONFIRM__"]'));

// Check Page is returned to change.org
await t.expect(getCurrentUrl()).contains(`change.org/${petitionAddress}`);
});

As mentioned, when clicking either 'Cancel' or 'Post' (/'Confirm'), the page hangs in the Facebook post window.

Your complete configuration file (if any):

{
  "browsers": ["chrome", "firefox"],
  "src": "./tests/**/*.js",

  "assertionTimeout": 10000,
  "pageLoadTimeout": 30000,
  "selectorTimeout": 10000,

  "quarantineMode": false,
  "skipJsErrors": true,

  "disablePageCaching": true,

  "reporter": [
    {
      "name": "xunit",
      "output": "artifacts/test-results/results.xml"
    },
    {
      "name": "spec"
    }
  ],

  "screenshots": {
    "fullPage": true,
    "takeOnFails": true,
    "pathPattern": "${TEST_ID}/${RUN_ID}_${FILE_INDEX}.png",
    "path": "artifacts/screenshots"
  },

  "videoOptions": {
    "singleFile": false,
    "failedOnly": true,
    "pathPattern": "${TEST_INDEX}/${RUN_ID}_${FILE_INDEX}.mp4"
  },
  "videoPath": "artifacts/recordings"
}

Your complete test report:

 1) AssertionError: expected

   'https://www.facebook.com/v3.2/dialog/share?channel=https%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_arbiter.php%3Fversion%3D45%23cb%3Df25a0076a291eac%26domain%3Dwww.change.org%26origin%3Dhttps%253A%252F%252Fwww.change.org%2<BLAH>'
      to include 'change.org/p/united-nations-send-matt-damon-to-mars-to-retrieve-opportunity'

      Browser: Chrome 79.0.3945.117 / macOS 10.15.2
      Screenshot: <PATH>/screenshots/test-1/_errors/1.png

         37 |  await createPetitionAndShareToFacebook(user);
         38 |
         39 |  await facebookShare.loginToFacebook();
         40 |  await facebookShare.verifyFacebookSharePage();
         41 |  await facebookShare.cancelShare();
       > 42 |  await t.expect(getCurrentUrl()).contains(`change.org/p/${t.ctx.petition.slug}`);
         43 |});
         44 |

         at <anonymous> (<PATH>/petitions/share/guest_share.js:42:35)

 1/1 failed (57s)

Screenshots:

![Screenshot 2020-01-09 at 4 39 31 PM](https://user-images.githubusercontent.com/37812673/72118564-53626880-3306-11ea-8978-a239d2fa8bb0.png)

Steps to Reproduce:

Go to my website ...
Execute this command...
See the error...

Your Environment details:

testcafe version: 1.7.0
node.js version: 10.15.3
command-line arguments: testcafe chrome share.js
browser name and version: chrome 79.0.3945.117
platform and version: MacOs Catalina 10.15.2
other:

Hi @rob4629
At present, TestCafe doesn't support this scenario. The scenario will be supported after the capability mentioned in the https://github.com/DevExpress/testcafe/issues/912 thread is implemented.


large image zoom is faulty since 0.3.0

Im using material for mkdocs version 9.1.11 with your plugin, and a large image (for example, 3071 x 1896; 150dpi; 32bit; png) doesn't work well with zoom. On clicking the zoom icon,  the picture isn't enlarged. Instead it moves to the left. Sometimes it moves beyond the screen.
I tested with versions 0.2.1, 0.3.0 and the latest, and it only works as it should (zoom icon indeed blows up the image in the browser) in 0.2.1.
Is there an extra configuration option to be set which I am missing since 0.3.0 or is it a bug?
Tested in firefox and chrome.

Could you provide your mkdocs.yaml or reproducible repository?

Can confirm, it looks like when you zoom it doesn't zoom to the point of the image you click but instead shifts the image left off screen.
With version 0.2.1 it will zoom but force to the right edge, shifting the image to the left.
Version 0.1.0, 0.2.0, 0.3.0, and 0.3.1, all seem to send the image off the screen to the left.  If I maximize my browser on my ultrawide monitor I can't see the zoomed image but when i click on the screen I can see the un-zoom effect of the image becoming smaller and coming from off screen from the left.
My doc is using a large resolution png file, roughly 10000x5400

Thanks for reporting this bug and providing a reproducible repository. This bug has been fixed in v0.3.6 by changing the default width from 100% to auto, because 100% width will cause zooming offset incorrectly.


Shortcut to list down only methods/fucntions from JS or TSX file in the VS Code Quick Open (CTRL+P) window

Issue Type: Feature Request
For now, when I click CTRL + E and then @, The quick open windows shows me all the methods and variables declared in the currenly opened file.

It would be great if there is a way to list down only the Javascript functions/methods so that we can easily navigate between methods quickly.
VS Code version: Code 1.48.0 (db40434f562994116e5b21c24015a2e40b2504e6, 2020-08-13T07:50:42.600Z)
OS version: Windows_NT x64 10.0.19041

Type @: instead of @ to sort the list by type

Thanks for the comment, it works partially though.
Please find the below screenshot, the @: jumbling variable assignments with method name list. Expected vscode to identify these and group them properly.

I may be misunderstanding your screenshot but the part to the right of the symbol name is the symbol that contains it. @: groups based on the type of the symbol itself

Expecting behavior similar to Visual Studio 2019 code window. Please find the screenshot of VS2019 below. The methods and only the global variables are listed in the navigation dropdown menu. However, in VSCODE it consider private variables inside the methods (which is good). It would be great if the VSCODE quick open menu group methods and variables (can be made configurable to ON or OFF grouping from the VSCODE settings).

...and when I said "grouped", I meant this:


Add dividers between article sizes in homepage

[x] Frontpage [[date]]
[x] In other news
[x] Throwback news

Done https://github.com/fmaclen/promptspree/pull/88


Component Updates

@dmlloyd I think you forgot to deploy these releases :) I can't see the versions under http://repository.jboss.org/nexus/content/groups/public/org/wildfly/wildfly-naming-client/ and http://repository.jboss.org/nexus/content/groups/public/org/wildfly/transaction/wildfly-transaction-client/

They're there now, sorry about that.

Retest this please


Add httpath multiaddr component

Ref: https://github.com/ipni/go-libipni/blob/dfad9b30814eaede8c7e934f120b632bb180511c/maurl/convert.go#L35
Ref: https://github.com/ipni/go-libipni/issues/42

Let's do http-path https://github.com/multiformats/multiaddr/pull/164 instead


Supported seeking over log lines which contain multibyte characters.

If a log has multibyte, check-log.rb unexpectedly behaves because String#size doesn't tell byte size but the number of characters.
However, ruby 1.8.6 or lower doesn't seem to support bytesize method. I wonder whether I should support lower version.
■Before Fixing
※The string "ＴＥＳＴ" is multibyte alphabets
% echo 0 > /var/cache/check-log/default/tmp/test.log
% echo "ＴＥＳＴ" > /tmp/test.log
% ./check-log.rb -f /tmp/test.log -q 'test' -e 'UTF-8'
CheckLog OK: 0 warnings, 0 criticals for pattern test.
% cat /var/cache/check-log/default/tmp/test.log
5
% echo "test" >> /tmp/test.log
% ./check-log.rb -f /tmp/test.log -q 'test' -e 'UTF-8'
Check failed to run: invalid byte sequence in UTF-8, ["./check-log.rb:188:in `match'", "./check-log.rb:188:in `match'", "./check-log.rb:188:in `block in search_log'", "./check-log.rb:183:in `each_line'", "./check-log.rb:183:in `search_log'", "./check-log.rb:135:in `block in run'", "./check-log.rb:129:in `each'", "./check-log.rb:129:in `run'", "/usr/local/ruby-2.1.6/lib/ruby/gems/2.1.0/gems/sensu-plugin-1.1.0/lib/sensu-plugin/cli.rb:56:in `block in <class:CLI>'"]

■After Fixing
※The string "ＴＥＳＴ" is multibyte alphabets
% echo 0 > /var/cache/check-log/default/tmp/test.log
% echo "ＴＥＳＴ" > /tmp/test.log
% ./check-log.rb -f /tmp/test.log -q 'test' -e 'UTF-8'
CheckLog OK: 0 warnings, 0 criticals for pattern test.
% cat /var/cache/check-log/default/tmp/test.log
13
% echo "test" >> /tmp/test.log
% ./check-log.rb -f /tmp/test.log -q 'test' -e 'UTF-8'
CheckLog CRITICAL: 0 warnings, 1 criticals for pattern test.
% cat /var/cache/check-log/default/tmp/test.log
18

Fixed in https://github.com/sensu-plugins/sensu-plugins-logs.


Add a parameter to remove excess console messages

Is your feature request related to a problem? Please describe.
When I run more than 200 tests, I can't see my post scripts due to the output of video reporter : those 2 lines :
this.write('\nGenerated:' + JSON.stringify(this.videos, undefined, 2) + '\n\n');
this.write(`\n\nVideo reporter Done!\n`);

Describe the solution you'd like
add a parameter to the wdio conf video reporter, that disables the this.write...
Describe alternatives you've considered
I forked the repo and temp fixed it but it interferes with npm install
Additional context
Add any other context or screenshots about the feature request here.

Sounds like a reasonable feature request. Any contributions are welcome!

https://github.com/webdriverio-community/wdio-video-reporter/pull/77

Merged and released


Various systemd-related improvements

This fixes Debian/Ubuntu, fixes shutdowns with UNIX sockets, deals with multiple instances, and more.
It fixes #271 and #283. It supersedes #274.

Just added a further commit to create a systemd-tmpfiles entry after finding my server did not survive a reboot. At this point, I'm thinking the systemd cookbook would be a nice idea but I'll see what you think of all this first.

I usually like to avoid having dependencies on other cookbooks simply because it always seems to eventually cause some sort of headache.  However I currently am not steeped to deeply in the systemd world and haven't even looked at that cookbook.  It may be a good idea to use it.

What's the status of this one?
Are you planning to merge this Pull Request into the official repo soon? What blocks it?
cc @KieranHayes

@chewi can you rebase this to solve the conflicts?

Done.

Thanks!


New Get Started Section for Performance: Wrap up and demo doc

This is part of the Get Started section for performance, wrapping up and demoing the final results.

Talked with Dave about removing references to HTTP caching and requests, and then this doc is very close to done! I did a full read today, and it's gonna be a useful doc for developers. So with those small changes, I'll do one more sweep and we may be ready to push a first round of docs for this doc set.

:tada: This has been pushed live to https://developers.google.com/web/


Falling off map

Design some sort of reset and loss state with the Game Manager so that the scene is reset if the player goes too far off-screen.

Please see #26.
One can update the game state and scene by calling methods in the GameManager.


chore(main): release 0.1.10

:robot: I have created a release beep boop
0.1.10 (2023-05-08)
Features

dependencies: Install extra dependencies if provided in pipeline (#23) (426bae4)

Bug Fixes

CLI: Do not add a workspace that cannot be found online (#27) (8190a34)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/BLSQ/openhexa-sdk-python/releases/tag/0.1.10 :sunflower:


internal::file_reader fails on multi-lined Windows files

Hi, I appear to have found a slight bug in internal::file_reader.
If a file with multiple newlines is created on Windows, file_reader fails with "unable to fread() file  size  errno 0". Creating the same file on Linux, or converting with 'dos2unix', causes file_reader to successfully parse the file. Conversely, a single line Windows file will also parse successfully. The core of the problem seems to be due to Windows using "\r\n" to indicate newlines, or more specifically, the different ways fseek/ftell and fread count the "\r\n" sequence.
In file_reader::read(), fread is given the number of characters it is expected to read. That number is calculated in file_reader::size(), which uses fseek/ftell to count the number of characters in the file (counting "\r\n" as two characters). However, std::fread automatically converts all "\r\n" to '\n', causing it to actually read in a smaller memory chunk than told. Since it is unable to read in the specified number of characters, it returns 0, causing the if check to fail and throwing the error shown above. In all other respects, the read completed successfully.
Also, if you stop execution right before the throw and look at the constructed string, the last N characters of the string are '\0', with N being the exact number of newlines in the file.

Could you please check whether changing the second argument to std::fopen() in pegtl/internal/file_reader.hh line 64 from "r" to "rb" fixes the issue?
Of course then the line endings will be \r\n rather than \n, is that acceptable to you? Otherwise we could try using std::feof() instead of checking the number of read bytes.

Yeah, changing to "rb" fixes the issue.

Ok, I've changed it to "rb" in the PEGTL master branch, too.

We have now updated the eol rule to accept both Unix and DOS line endings.


Uncaught PHP Exception Shopware\Core\Checkout\Payment\Exception\UnknownPaymentMethodException

Describe the bug
When we have the Adyen plugin installed we can't use manual created payment methods anymore (Invoice, Pay at delivery, Pay at pickup etc.). The user gets redirect to /checkout/order with a 404 message.
The following error is visible in the logs:
request.ERROR: Uncaught PHP Exception Shopware\Core\Checkout\Payment\Exception\UnknownPaymentMethodException: "The payment method 1a116955b1294a45a64aa6e1507e9028 could not be found." at /var/domains/domain.com/application/vendor/shopware/core/Checkout/Payment/PreparedPaymentService.php line 56 {"exception":"[object] (Shopware\\Core\\Checkout\\Payment\\Exception\\UnknownPaymentMethodException(code: 0): The payment method 1a116955b1294a45a64aa6e1507e9028 could not be found. at /var/domainsdomain.com/application/vendor/shopware/core/Checkout/Payment/PreparedPaymentService.php:56)"} []
Versions
Shopware version: <IP_ADDRESS>
Plugin version: 3.4.2
To Reproduce
Steps to reproduce the behavior:

Pay with manually created payment method
Error above is thrown

Expected behavior
Payment is handled by merchant, customer sees the confirm/finish page instead of a 404 error

Hi @MelvinAchterhuis we reverted all references to Checkout/Payment/PreparedPaymentService.php for the 3.4.2 release, could you check to see if removing the plugin and reinstalling it from scratch in version 3.4.2 fixes this issue?

This still does not work in the latest release (3.6.0) with the same scenario.

Hi @MelvinAchterhuis can you share the logs from the Adyen plugin? You would find them in var/logs/adyen. So we can see where the plugin interacts with your custom payment method.

@peterojo if I may too:
api.log

[2022-09-12T08:15:06.747084+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:15:06.747297+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"Foodspring_DE","countryCode":"GB","amount":{"currency":"GBP","value":2519},"shopperReference":"738bf2056ecf41eb9e0c555be058a586","shopperLocale":"en-GB"} [] []
[2022-09-12T08:15:07.160035+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000203966","merchantName":"foodspring GmbH"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"Foodspring_DE"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:15:22.234679+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:15:22.234828+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"Foodspring_DE","countryCode":"GB","amount":{"currency":"GBP","value":2519},"shopperReference":"738bf2056ecf41eb9e0c555be058a586","shopperLocale":"en-GB"} [] []
[2022-09-12T08:15:22.681165+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000203966","merchantName":"foodspring GmbH"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"Foodspring_DE"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:28:10.629200+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:28:10.629377+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"Foodspring_DE","countryCode":"GB","amount":{"currency":"GBP","value":2389},"shopperReference":"c80d933c356240fea3156627f2a8110f","shopperLocale":"en-GB"} [] []
[2022-09-12T08:28:11.200893+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000203966","merchantName":"foodspring GmbH"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"Foodspring_DE"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:28:21.081703+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:28:21.081868+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"Foodspring_DE","countryCode":"GB","amount":{"currency":"GBP","value":2389},"shopperReference":"c80d933c356240fea3156627f2a8110f","shopperLocale":"en-GB"} [] []
[2022-09-12T08:28:21.400122+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000203966","merchantName":"foodspring GmbH"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"Foodspring_DE"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:44:52.572725+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:44:52.573149+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"Foodspring_DE","countryCode":"GB","amount":{"currency":"GBP","value":2389},"shopperReference":"c80d933c356240fea3156627f2a8110f","shopperLocale":"en-GB"} [] []
[2022-09-12T08:44:52.824206+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000203966","merchantName":"foodspring GmbH"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"Foodspring_DE"},"name":"Google Pay","type":"paywithgoogle"}]} [] []

adyen.log

[2022-09-05T14:41:39.484738+00:00] adyen_generic.ERROR: Invalid Merchant Account [] []
[2022-09-05T14:41:39.680105+00:00] adyen_generic.ERROR: Invalid Merchant Account [] []

There is nothing specific in the logs. I've traced it down to PaymentHandlerRegistry::handlers not containing the manual payment method's handler_identifier or not being equal to $expectedHandlerType. In both cases, it stems from PreparedPaymentService. In and of itself it's not bad that PreparedPaymentService is called but somewhere along the call stack something's being deleting data from handlers.

@peterojo if I may too:
api.log

[2022-09-12T08:15:06.747084+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:15:06.747297+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"XXX","countryCode":"GB","amount":{"currency":"GBP","value":2519},"shopperReference":"738bf2056ecf41eb9e0c555be058a586","shopperLocale":"en-GB"} [] []
[2022-09-12T08:15:07.160035+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000000000","merchantName":"XXX"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"XXX"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:15:22.234679+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:15:22.234828+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"XXX","countryCode":"GB","amount":{"currency":"GBP","value":2519},"shopperReference":"738bf2056ecf41eb9e0c555be058a586","shopperLocale":"en-GB"} [] []
[2022-09-12T08:15:22.681165+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000000000","merchantName":"XXX"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"XXX"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:28:10.629200+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:28:10.629377+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"XXX","countryCode":"GB","amount":{"currency":"GBP","value":2389},"shopperReference":"c80d933c356240fea3156627f2a8110f","shopperLocale":"en-GB"} [] []
[2022-09-12T08:28:11.200893+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000000000","merchantName":"XXX"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"XXX"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:28:21.081703+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:28:21.081868+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"XXX","countryCode":"GB","amount":{"currency":"GBP","value":2389},"shopperReference":"c80d933c356240fea3156627f2a8110f","shopperLocale":"en-GB"} [] []
[2022-09-12T08:28:21.400122+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000000000","merchantName":"XXX"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"XXX"},"name":"Google Pay","type":"paywithgoogle"}]} [] []
[2022-09-12T08:44:52.572725+00:00] adyen_api.INFO: Request url to Adyen: https://checkout-test.adyen.com/checkout/v68/paymentMethods [] []
[2022-09-12T08:44:52.573149+00:00] adyen_api.INFO: JSON Request to Adyen:{"channel":"Web","merchantAccount":"XXX","countryCode":"GB","amount":{"currency":"GBP","value":2389},"shopperReference":"c80d933c356240fea3156627f2a8110f","shopperLocale":"en-GB"} [] []
[2022-09-12T08:44:52.824206+00:00] adyen_api.INFO: JSON Response to Adyen:{"paymentMethods":[{"brands":["visa","mc"],"name":"Credit Card","type":"scheme"},{"brands":["visa","mc"],"configuration":{"merchantId":"000000000000000","merchantName":"XXX"},"name":"Apple Pay","type":"applepay"},{"configuration":{"merchantId":"50","gatewayMerchantId":"XXX"},"name":"Google Pay","type":"paywithgoogle"}]} [] []

adyen.log

[2022-09-05T14:41:39.484738+00:00] adyen_generic.ERROR: Invalid Merchant Account [] []
[2022-09-05T14:41:39.680105+00:00] adyen_generic.ERROR: Invalid Merchant Account [] []

There is nothing specific in the logs. I've traced it down to PaymentHandlerRegistry::handlers not containing the manual payment method's handler_identifier or not being equal to $expectedHandlerType. In both cases, it stems from PreparedPaymentService. In and of itself it's not bad that PreparedPaymentService is called but somewhere along the call stack something's being deleting data from handlers.

@peterojo Could you check out my comment? Thanks a lot in advance!

Hi @jakub-groncki, my apologies for the delay,
I see no issues from the api.log logs you sent, and the adyen.log you sent has a 7 days earlier timestamp (as you mentioned, this was before configuring the merchant account) so no issues there either that would cause any payment methods to fail.
Maybe I'm missing something, but could you please explain the steps to "manually" create a payment method? Could it be that your custom payment method is missing a handler_identifier?

I'm doing a very simple thing:
Settings -> Payment -> Add payment method
I fill in the name and save it. Then I assign it to the sales channel. After that, when selecting it at checkout (it's visible there) I'm getting the error mentioned by @MelvinAchterhuis. So there's not much to it besides that. It does have the handle identifier in the db.

Hi @jakub-groncki thanks for the update, I have been able to reproduce the issue with the plugin installed, I'll troubleshoot a bit and get back to you when I have found the cause and fixed it

@MelvinAchterhuis @jakub-groncki FYI, I found the issue and fixed it in the mentioned PR, we will release soon. You can also apply this fix anytime on your installation to get it working.


Autocomplete for logged in user.

Adds Autocomplete in modal forms of logged-in users.

We have introduced this PR as a theme template override on our store and can confirm it works like a charm. Would benefit from being included in the repo.


Fix panic in shim not logged

Carry #4298
Fix #4274

This broke arm64 I think, https://travis-ci.com/github/kind-ci/containerd-nightlies/jobs/407844501

vendor/github.com/containerd/fifo/fifo.go:51:12: undefined: syscall.Dup2

dup2 is not on arm64 https://github.com/golang/go/issues/11981

filed https://github.com/containerd/fifo/issues/27, I think it should be fairly straightforward to fix.


chore: python-dw to 0.0.6

chore: Promote python-dw to version 0.0.6

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment