cannot get mem stats for docker containers

I am running on Debian Jessie, and have done as described in the README:
$ cat /etc/default/grub  | grep GRUB_CMDLINE_LINUX_DEFAULT
GRUB_CMDLINE_LINUX_DEFAULT="quiet cgroup_enable=memory swapaccount=1"

but I do not get mem stats for the containers.

Can you please verify it with the latest docker image which has python installed. It might be handled by the collection plugins which didn't run until now. For me it works with exactly the same settings:
root@serenity:/src/netdata/bin# cat /etc/default/grub  | grep GRUB_CMDLINE_LINUX_DEFAULT
GRUB_CMDLINE_LINUX_DEFAULT="quiet cgroup_enable=memory swapaccount=1"

I would close this by intuition, but it might be that some basic switch is missing at the time when you run docker. You can use this as an utility script:
#!/bin/bash
NAME="netdata"
DOCKERFILE="titpetric/netdata"
if [ ! -z "$1" ]; then
        echo "Running custom netdata: $1"
        DOCKERFILE=$1
fi
docker rm -f $NAME
docker pull $DOCKERFILE
EXPOSE="-p 8081:8081"
docker run -e NETDATA_PORT=8081 --restart=always -v /tmp/docker-proxy-acl/docker.sock:/var/run/docker.sock -d --cap-add SYS_PTRACE -h $NAME --name $NAME -v /proc:/host/proc:ro -v /sys:/host/sys:ro $EXPOSE $DOCKERFILE

It should be what you need. The -v is to pass the socket created by docker-proxy-acl - it's used to resolve names of containers within a dockerized netdata.

I have docker-proxy-acl setup with the -a containers flag. Does it potentially need another flag?
kyle@manta:~$ cat /etc/default/grub  | grep GRUB_CMDLINE_LINUX_DEFAULT
GRUB_CMDLINE_LINUX_DEFAULT="quiet cgroup_enable=memory swapaccount=1"

I ran the script you provided, but I don't get mem statistics for the containers in the test container either.

No. I use the same. And memory statistics for containers should come from the filesystem and not the docker.sock api (if you don't get ab501ba710f and similar for container names - docker-proxy-acl works).
Find a container with docker ps -a and execute: "find /sys/ | grep [container-id] | grep mem", paste output here.

Just for posterity:
root@serenity:~# find /sys/ | grep 68b0861afcdf | grep mem
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.mem_hardwall
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.mems
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.memory_pressure
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.memory_spread_page
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.memory_spread_slab
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.mem_exclusive
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.effective_mems
/sys/fs/cgroup/cpuset/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cpuset.memory_migrate
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.pressure_level
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.use_hierarchy
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.swappiness
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.memsw.failcnt
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/tasks
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.limit_in_bytes
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.memsw.max_usage_in_bytes
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cgroup.procs
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.usage_in_bytes
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.memsw.limit_in_bytes
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.failcnt
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.force_empty
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.memsw.usage_in_bytes
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.max_usage_in_bytes
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.numa_stat
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cgroup.clone_children
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.oom_control
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/notify_on_release
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.stat
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/cgroup.event_control
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.move_charge_at_immigrate
/sys/fs/cgroup/memory/docker/68b0861afcdfbe18c5af6e5014482e79c7d8b0ac6cdeec6a745d40ae62d21f20/memory.soft_limit_in_bytes

And kernel: Linux serenity 4.3.0-1-amd64 #1 SMP Debian 4.3.5-1 (2016-02-06) x86_64 GNU/Linux

@ktsaou - I'm not wrong about this am I, the statistics still come from the filesystem? I do know that Docker has a statistics API also, I'm just not tracking if the use of it has been changed in netdata - I suppose it hasn't.

root@manta:~# find /sys/ | grep dbc5ce2e9258 | grep mem
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.mem_hardwall
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.mems
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.memory_pressure
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.memory_spread_page
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.memory_spread_slab
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.mem_exclusive
/sys/fs/cgroup/cpuset/docker/dbc5ce2e9258ab32ca485492dd50525695fc70ca1b784561938aa41a48b88613/cpuset.memory_migrate

Seems you're missing quite a few files. Please:

paste the kernel version,
run update-grub (or whatever the script is) to update kernel flags,
reboot the host,
rerun tests.

I'm sure you have done all these, but there's no way to verify. The whole memory subsystem, as you see, is missing from your /sys/fs/cgroup - so the fault is in an external system (driver, cgroups, kernel).

Updating the kernal flags did the trick. Thanks.

@ktsaou - I'm not wrong about this am I, the statistics still come from the filesystem?

Yes, statistics come from /sys only at the moment.

@kmlucy you're welcome :)


[RFC] standardize PyBytes <-> Vec or &[u8] or Cow<[u8]>

When generating PyBytes from Vec<u8> or &[u8] or Cow<[u8]> somewhat different behaviors are observed:
Python::with_gil(|py| {
    let bytes_py = PyBytes::new_bound(py, b"foobar");
    dbg!(&bytes_py);  // &bytes_py = b'foobar'

    let bytes_vec = bytes_py.extract::<Vec<u8>>().unwrap();
    let py_obj = bytes_vec.to_object(py);
    let maybe_py_bytes = py_obj.downcast_bound::<PyBytes>(py);
    dbg!(maybe_py_bytes);  // Err(DowncastError { from: [102, 111, 111, 98, 97, 114], to: "PyBytes" })

    let bytes_slice = bytes_py.extract::<&[u8]>().unwrap();
    let py_obj = bytes_slice.into_py(py);  // to_object(py) does not work!
    let maybe_py_bytes = py_obj.downcast_bound::<PyBytes>(py);
    dbg!(maybe_py_bytes);  // Ok(b'foobar')

    let bytes_cow = bytes_py.extract::<Cow<'_, [u8]>>().unwrap();
    let py_obj = bytes_cow.to_object(py);
    let maybe_py_bytes = py_obj.downcast_bound::<PyBytes>(py);
    dbg!(maybe_py_bytes);  // Ok(b'foobar')

    Ok(())
})

Ideally, I would like to be able to handle all 3 convesions the same way, with either into_py and/or to_object.  I'm not 100% what the best approach would be but I took a stab at ToPyObject to showcase how that might work. Would love some feedback on the approach as well as pointers on how to best proceed to address IntoPy and FromPyObject, assuming all this doable.

Thanks for the PR, and sorry it's taken a little while to get a review!
This is definitely a topic with a situation in PyO3 which is currently unsatisfying, see also #2888 with some past discussion in this area. (cc @adamreichold)
I think as much as it feels impure, it would be desirable for Vec<u8> to interoperate with Python bytes as this is such a common performance footgun. I think also if I can find a generic solution then #1200 can also support slices to-python better.
Regarding the conversion traits we currently also have #4060 for the to-python direction which would reduce the number of to-python traits down to just one. Maybe while we're making changes this is a good moment to explore a solution for this. (cc @Icxolu)
I definitely want this case to be better, and I still can't see any solution to implement the desired behaviour other than hanging the implementation off FromPyObject and IntoPyObject by an additional method.
I wonder, can we maybe solve this by adding additional methods to those traits which are effectively private? Basically we make one or more of the types in their signature PyO3-private and then we give a default implementation so that users can't touch it.
e.g.
trait FromPyObject<'py> {

    /* existing stuff ... */

    fn extract_bytes(&self, override: pyo3::impl_::Override) -> ... {
        /* default implementation returns None or fails or similar
    }
}

where the pyo3::impl_::Override type cannot be instantiated by PyO3 users, so they cannot call this method. I wonder if it's possible to go further and make the type unnameable, so they cannot implement it either. I'm sure I saw ideas related to this somewhere.

Sounds like Sealed trait methods, just put the Override type in a private module with no public reexports.

After rebasing #4060 and giving this a first look, I have some observations about combining these two ideas:

I think the general idea of using a sealed trait-method for this should work with it as well.
The default implementation would need some additional trait bounds around Error, but since it would be sealed and the relevant collection already have the same bound, this should not be a problem.
The Target type for Vec<T>, etc would have to be PyAny (instead of PyList), since it could now be PyBytes (or even something else in the future) as well. A bit sad, but not the end of the world.
The current implementation uses self (by value) for the conversion. This makes working with borrowed iterators extremely inconvenient, since it requires a lot of HRTBs all over the place (and have not checked yet whether it would work out in the end). So currently all collections are implemented by value only, so that they can use the owning iterators... This would mean that this might only work for Vec<T> and not &[T] or Cow<T>.  I'm still a bit thorn whether that's the right choice to make, or whether we should take &self instead. It seems like we're making trade-offs either way, but that's more of a discussion for #4060.

The default implementation would need some additional trait bounds around Error, but since it would be sealed and the relevant collection already have the same bound, this should not be a problem.

I don't entirely follow why this is necessary, though I guess it would become clearer to me in time.

The Target type for Vec<T>, etc would have to be PyAny (instead of PyList), since it could now be PyBytes (or even something else in the future) as well. A bit sad, but not the end of the world.

Ah, yes. That might actually be a good thing to help users realise that there's something a bit funny going on. I also sort of wonder if there's a way to exploit the type system to achieve this statically, though that'll be an experiment all on its own.

The current implementation uses self (by value) for the conversion. This makes working with borrowed iterators extremely inconvenient, since it requires a lot of HRTBs all over the place (and have not checked yet whether it would work out in the end). So currently all collections are implemented by value only, so that they can use the owning iterators... This would mean that this might only work for Vec<T> and not &[T] or Cow<T>.  I'm still a bit thorn whether that's the right choice to make, or whether we should take &self instead. It seems like we're making trade-offs either way, but that's more of a discussion for Experiment: Fallible conversion trait #4060.

👍 I'm looking forward to carving out some time to play around with #4060 in detail as soon as we've shipped 0.22 (which I think is mostly waiting on me to rebase stuff...)

Thanks for all the issue links and overview of the problems with PyBytes <-> Vec<u8>!

Regarding the conversion traits we currently also have https://github.com/PyO3/pyo3/pull/4060 for the to-python direction which would reduce the number of to-python traits down to just one.

From a user's perspective this would be a great improvement since it makes the code more readable and whenever there is "just one way of doing X", makes using the said code more ergonomic.

I think the general idea of using a sealed trait-method for this should work with it as well.

This would also be amazing for the Vec<u8> -> PyBytes specialization! I now also understand a bit more the design choices behind from_py_object_bound_sealed::Sealed. Happy to close this PR and continue the conversation on #4060.


bootstrap requires jQuery

I tried to include jquery in my webpack.config.js and therefore i created a new ProviderPlugin. My webpackconfig.js looks like this:
var webpack = require('webpack');
var path = require('path');

module.exports = {
    entry: './static/js/main.js',
    output: {
        filename: 'bundle.js',
        path: path.resolve(__dirname, './static/dist')
    },
    plugins: [
        new webpack.ProvidePlugin({
            "$": "jquery",
            "jquery": "jquery",
            "jQuery": "jquery",
            "window.jQuery": "jquery"
        })
    ]
};

And my index.html has the following ending:
</div>
        <script src="dist/bundle.js"></script>
        <script src="js/bootstrap.min.js"></script>
    </body>

As i can see in the resulting bundle.js, jquery is inserted in the bundle.js file.
But as soon as i go on my index.html it tells me this error msg from bootstrap: "bootstrap.min.js:6 Uncaught Error: Bootstrap's JavaScript requires jQuery
at bootstrap.min.js:6"
Why is this happening? Is this a bug, or am i doing something wrong?
I am using the latest chrome version and webpack 2.2.1

@krackmoe You doing something wrong. Webpack knew nothing about bootstrap, and bootstrap can't get jQuery from bundle because modules are scoped inside bundle. You need to include bs to your config as a second entry or include it to bundle.

It looks like you just deleted our lovely crafted issue template. It was there for good reasons. Please help us solving your issue by answering the questions asked in this template. I'm closing this. Please either update the issue with the template and reopen, or open a new issue.


#1 add punctuation

Resolves #1

Sorry, I can't get back to this until tomorrow!
On Tue, Apr 20, 2021 at 5:23 PM Jon Peck @.***> wrote:

@.**** requested changes on this pull request.
It looks like not all the haikus were changed?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/peckjon/haikus-for-june/pull/2#pullrequestreview-640533564,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ATYSYTTGSSET2CUGDBPEDQDTJYLHRANCNFSM43JIYF4Q
.


Bootstrap 4 Bar isn't showing on Get Started page

I have noticed the bootstrap 4 bar doesn't show on the Get Started Page.
CSS: http://i.imgur.com/xSvfLdk.png
Get Started: http://i.imgur.com/5TPwN5v.png

Fixed in 3.3.7 via #18446.


Update parameter serialization

This should correspond to a list of parameters and might be bigger than the previous file format. For that I think we need to support both native Julia serialization (binary small serialization) and a readable serialization like YAML.

[ ] YAML serialization
[ ] Native serialization

I'll take the opportunity to:

[ ] Provide a function to order a list of parameters in order to save compute
[ ] Maybe provide a default (G,Q) learners setting


Finalized basic controller layer.

Finalized basic controller layer + minor improvements.
Tested via unit-tests and local server deployment.

Consider abandoning the idea of creating interfaces for controllers as it heavily affects readability and first glance understanding of functionality.

Fixed!


Resolve Vertex Scheduler Character Limit Issue for Long Pipeline Names

Open Source Contributors Welcomed!
Please comment below if you would like to work on this issue!
Contact Details [Optional]
<EMAIL_ADDRESS>What happened?
A user encountered an issue where the Vertex scheduler in ZenML does not accept pipeline names longer than 64 characters. This limitation requires users to shorten their pipeline names to successfully schedule their pipelines on Vertex AI, which can be restrictive and inconvenient.
Task Description
Investigate and address the issue with the Vertex scheduler's character limit for pipeline names in ZenML. The goal is to either increase the character limit or implement a mechanism that automatically handles longer names without user intervention.
Expected Outcome
Users should be able to schedule pipelines on Vertex AI without the constraint of a 64-character limit on pipeline names.
ZenML should provide a seamless experience, either by accommodating longer names or by intelligently managing them within the existing Vertex scheduler limitations.
This fix will enhance user convenience and flexibility in naming pipelines.
Steps to Implement

Reproduce the issue to understand the exact limitations and behavior of the Vertex scheduler with respect to pipeline names.
Explore solutions such as increasing the character limit or automatically abbreviating/transforming long pipeline names while preserving their uniqueness and recognizability.
Implement the chosen solution in a way that is transparent and intuitive for the user.
Test the implementation with various pipeline name lengths to ensure compatibility and functionality.
Update documentation to inform users about the handling of pipeline names in Vertex scheduler and any best practices or limitations.

Additional Context
This issue is crucial for users who require descriptive and potentially lengthy names for their pipelines, especially in complex or large-scale MLOps environments.
Code of Conduct

[ ] I agree to follow this project's Code of Conduct

@christianversloot if I'm not completely mistaken, we had a conversation about this in Slack? Unfortunately the slack history hasn't been retained.

Correct, but that was for SageMaker orchestrator. There, I believe we limit name length to circumvent this error. It could perhaps be inspiration for a fix here :)

@christianversloot ah yes! https://github.com/zenml-io/zenml/pull/1505 was the PR.

@strickvl @christianversloot
I would like to work on this... Please assign!

has the bug been solved? if not i would like to solve it

The bug has not been solved yet @Merthoshan. @ashutosh887 are you working on it, if not, can let @Merthoshan take a shot.

Where can we discuss if we're facing some issues understanding the bug? @strickvl
I've worked it

Here is the place.

is there a simpler way to replicate this I could take this but It's going to be difficult for me to replicate this if this needs me to setup GCP which requires some cost. @strickvl


AttributeError: module 'large_image' has no attribute 'getTileSource'

I tried extracting the patches using your code. However, I received the following error message:
AttributeError: module 'large_image' has no attribute 'getTileSource'
Do you have any idea why this happens? Thank you!

How did you install it?  And what OS and Python version are you on?  It sound like an incorrect installation.

Hi @manthey, My operating system is Windows, and I have tried both Python 3.7 and Python 3.10. What should I use? Is there a requirement.txt file you could share?

It should work (with some sources) on Python3.6+ on Windows.  For instance, on a clean Windows 10 with Python 3.10, I just did "pip install large-image[pil]", which installs the base library and the pil tile source, and it works fine.  For many tile sources, you need the appropriate binary library installed (e.g., openslide or GDAL).  There is a conda forge installation which has some of these.

All of the tile sources work on linux; not all of them have Windows-compatible binaries (e.g., I don't know if the mapnik or vips sources can work on windows).

Hi @manthey , I reinstalled it again with "pip-install large-image[pil]" and the error was like this:
File "C:\self-learning-transformer-mil\src\preprocessing\extract_patches.py", line 10, in
from openslide import open_slide
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\openslide_init_.py", line 30, in
from openslide import lowlevel
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\openslide\lowlevel.py", line 54, in
lib = cdll.LoadLibrary('libopenslide-0.dll')
File "C:\Users\user\anaconda3\envs\hipt\lib\ctypes_init.py", line 452, in LoadLibrary
return self.dlltype(name)
File "C:\Users\user\anaconda3\envs\hipt\lib\ctypes_init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
Do you have any suggestions for what I should do?

I want to extract the patches of CAMELYON16 (.tif format). Is that available on windows?

Since you're on windows and using conda as your environment/package manager, you should try to install the dependencies like open slide through conda to get the binaries/dlls.

Uninstall openslide as you initially installed with pip, and this will likely conflict if trying to install it with conda: pip uninstall openslide-python
Install openslide through conda: conda install -c conda-forge openslide-python. This will install openslide too which will have the DLLs

Hope this helps!
FWIW, a few of the large-image source modules are published on conda-forge (mostly for geospatial applications). We  probably want to consider add more of the source modules for use cases like this so that you can fully install with conda and not have a mix of conda and pip installed packages

Hi @banesullivan, thanks for the suggestions. I have already uninstalled openslide using pip and reinstalled it through conda. It seems like the error now is "No module named openslide" and below is the output when installing openslide through conda (It says openslide-python isn't available from current channels).
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:

openslide-python

Current channels:

https://conda.anaconda.org/conda-forge/win-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org

and use the search bar at the top of the page.

You need to specify the conda-forge channel
conda install -c conda-forge openslide-python

Hi @banesullivan, I already followed your suggestion to install openslide-python through conda with the below command:
conda install -c conda-forge openslide-python
However, I still received an error below:
PackagesNotFoundError: The following packages are not available from current channels:

openslide-python

Do you have any ideas?

Ah my bad. It looks like the open slide package (open slide-pythons dependency) doesn't build for windows: https://anaconda.org/conda-forge/openslide/files
I'm not too sure where to go from here
Perhaps we can accomplish what you're after without open slide... let me read back in the issue on what you're trying to do

No worries. If you have any suggestions for how to resolve this issue, please let me know. Thanks

So what exactly are you trying to do? I'm confused by what you mean by "patches"
Can you share the image file you're working with?

Hi @banesullivan, I am trying to extract patches of the CAMELYON16 dataset (.tif format)
(https://drive.google.com/drive/folders/0BzsdkU4jWx9Bb19WNndQTlUwb2M?resourcekey=0-FREBAxB4QK4bt9Zch_g5Mg)

Would you please elaborate on what you mean by "patches"? Would you provide an example of what you are trying to do?

In the context of whole slide imaging, a patch is a small portion or subset of the entire slide image.  These images can be several gigabytes in size and contain millions of pixels.
To manage the large size of these images, they are often divided into smaller patches, which can be more easily processed and analyzed. Patch-based analysis is a common method used in digital pathology to extract features or perform tasks on the tissue sample.
FYI, I followed this code for extracting the patches:
https://github.com/gokberkgul/self-learning-transformer-mil/blob/main/src/preprocessing/extract_patches.py

I don't think I get DLL issues, but instead, the error looks like this:
File "C:\self-learning-transformer-mil\src\preprocessing\extract_patches.py", line 211, in
main(args)
File "C:\self-learning-transformer-mil\src\preprocessing\extract_patches.py", line 195, in main
tile_slide(slide_path, out_folder, args.size, overlap, args.magnification, args.tissue_threshold, thresholds)
File "C:\self-learning-transformer-mil\src\preprocessing\extract_patches.py", line 115, in tile_slide
ts = large_image.getTileSource(slide_path)
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\large_image\tilesource_init.py", line 79, in getTileSource
return getTileSourceFromDict(AvailableTileSources, *args, **kwargs)
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\large_image\tilesource_init.py", line 67, in getTileSourceFromDict
raise TileSourceException('No available tilesource for %s' % pathOrUri)
large_image.exceptions.TileSourceException: No available tilesource for E:/CAMELYON16/images_train\normal_001.tif

Can you try installing large-image-source-tiff from conda-forge and then running the script?:
conda install -c large-image-source-tiff

This is what I get:
CondaValueError: too few arguments, must supply command line package specs or --file

Sorry, late at night here. I meant to add the conda-forge channel:
conda install -c conda-forge large-image-source-tiff

My error now is this:
Traceback (most recent call last):
File "C:\self-learning-transformer-mil\src\preprocessing\extract_patches.py", line 10, in
from openslide import open_slide
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\openslide_init_.py", line 30, in
from openslide import lowlevel
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\openslide\lowlevel.py", line 55, in
lib = cdll.LoadLibrary('libopenslide-0.dll')
File "C:\Users\user\anaconda3\envs\hipt\lib\ctypes_init.py", line 452, in LoadLibrary
return self.dlltype(name)
File "C:\Users\user\anaconda3\envs\hipt\lib\ctypes_init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 127] The specified procedure could not be found

large_image.exceptions.TileSourceException: No available tilesource for E:/CAMELYON16/images_train\normal_001.tif

Are you sure that you have the normal_001.tif file in the specified path?

Yes, I'm pretty sure. You can check the below screenshot. Do I miss something?

Traceback (most recent call last):
File "C:\self-learning-transformer-mil\src\preprocessing\extract_patches.py", line 10, in
from openslide import open_slide
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\openslide_init_.py", line 30, in
from openslide import lowlevel
File "C:\Users\user\anaconda3\envs\hipt\lib\site-packages\openslide\lowlevel.py", line 55, in
lib = cdll.LoadLibrary('libopenslide-0.dll')
File "C:\Users\user\anaconda3\envs\hipt\lib\ctypes_init.py", line 452, in LoadLibrary
return self.dlltype(name)
File "C:\Users\user\anaconda3\envs\hipt\lib\ctypes_init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 127] The specified procedure could not be found

Maybe try adding the dll path to the sys in the beginning of the script?
import sys
sys.path.insert(0,'/path/to/dll')

Hi @gokberkgul , actually I already solved the open slide issue. The only remaining is large-image ("No available tilesource")

Traceback (most recent call last):
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\large_image\tilesource_init_.py", line 27, in loadTileSources
sourceClass = entryPoint.load()
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\pkg_resources_init_.py", line 2470, in load
self.require(*args, **kwargs)
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\pkg_resources_init_.py", line 2493, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\pkg_resources_init_.py", line 800, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (large-image 1.0.4 (c:\users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages), Requirement.parse('large-image>=1.18.0'))
Failed to loaded tile source tiff
Traceback (most recent call last):
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\libtiff\libtiff_ctypes.py", line 53, in
libtiff = ctypes.cdll.LoadLibrary("libtiff.dll")
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\ctypes_init_.py", line 442, in LoadLibrary
return self.dlltype(name)
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\ctypes_init.py", line 364, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\large_image\tilesource_init_.py", line 27, in loadTileSources
sourceClass = entryPoint.load()
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\pkg_resources_init_.py", line 2471, in load
return self.resolve()
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\pkg_resources_init_.py", line 2477, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\large_image_source_tiff_init_.py", line 35, in
from .tiff_reader import TiledTiffDirectory, TiffException,
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\large_image_source_tiff\tiff_reader.py", line 35, in
from libtiff import libtiff_ctypes
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\libtiff_init_.py", line 23, in
from .libtiff_ctypes import libtiff, TIFF, TIFF3D  # noqa: F401
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\libtiff\libtiff_ctypes.py", line 57, in
raise ImportError('Failed to find TIFF library. Make sure that'
ImportError: Failed to find TIFF library. Make sure that libtiff is installed and its location is listed in PATH|LD_LIBRARY_PATH|..
0%|

Hm. I'm able to read this with the large-image-source-gdal module but not the tiff module:
@manthey, any ideas what's going wrong here:
>>> import large_image_source_tiff
>>> large_image_source_tiff.open('~/Downloads/normal_152.tif')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bane.sullivan/Software/girder/large_image/sources/tiff/large_image_source_tiff/__init__.py", line 741, in open
    return TiffFileTileSource(*args, **kwargs)
  File "/Users/bane.sullivan/Software/girder/large_image/large_image/cache_util/cache.py", line 216, in __call__
    raise exc
  File "/Users/bane.sullivan/Software/girder/large_image/large_image/cache_util/cache.py", line 209, in __call__
    instance = super().__call__(*args, **kwargs)
  File "/Users/bane.sullivan/Software/girder/large_image/sources/tiff/large_image_source_tiff/__init__.py", line 106, in __init__
    alldir = self._scanDirectories()
  File "/Users/bane.sullivan/Software/girder/large_image/sources/tiff/large_image_source_tiff/__init__.py", line 190, in _scanDirectories
    dir = TiledTiffDirectory(self._largeImagePath, directoryNum, validate=False)
  File "/Users/bane.sullivan/Software/girder/large_image/sources/tiff/large_image_source_tiff/tiff_reader.py", line 146, in __init__
    self._loadMetadata()
  File "/Users/bane.sullivan/Software/girder/large_image/sources/tiff/large_image_source_tiff/tiff_reader.py", line 306, in _loadMetadata
    value = self._tiffFile.GetField(field)
  File "/opt/miniconda3/envs/geospatial/lib/python3.9/site-packages/libtiff/libtiff_ctypes.py", line 1417, in GetField
    return convert(data)
  File "/opt/miniconda3/envs/geospatial/lib/python3.9/site-packages/libtiff/libtiff_ctypes.py", line 384, in <lambda>
    ctypes.POINTER(ctypes.c_uint32), lambda _d: _d.contents),
ValueError: NULL pointer access

@bryanwong17, could you try install the GDAL module:
conda install -c conda-forge large-image-source-gdal

Then running your script? This isn't the most ideal, but it may work for you.

Hi @banesullivan, sorry for the late reply.
Here is my error now after installing the GDAL module:
Traceback (most recent call last):
File ".\extract_patches.py", line 10, in
from openslide import open_slide
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\openslide_init_.py", line 30, in
from openslide import lowlevel
File "C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\openslide\lowlevel.py", line 50, in
from . import _convert
ImportError: cannot import name 'convert' from partially initialized module 'openslide' (most likely due to a circular import) (C:\Users\user\anaconda3\envs\stylegan-pytorch\lib\site-packages\openslide_init.py)

I changed different env and there was an error when importing "large_image_source_tiff"

Hi @banesullivan @gokberkgul @mantheym, thanks for helping me. The code now runs perfectly on my computer after installing WSL: Ubuntu version


Provide an event that gets fired when the video has loaded

A lot of the properties only get loaded after a set amount of time when the video gets loaded. It would be nice to know when this actually is through a callback or event...

Hi,
This could be works:
viewer.addListener('recorded', 'meta', 'onMediaMetaReady');


navbar and preloader design as separate component

separate navbar and preloader component designed which is responsive ……and can be use in any html file as separate component

Great! This kinds of commit message is expected.


Vale and VSCode

How do I enable VSCode to mark suggestions, warnings, etc inline in *.md file?
I tried both released and pre-released version of 'Vale VSCode' extension.
My settings:

I do not see the problems inline in a file, nor do I see any problems listed below.

I don't see a value for the config or the path to vale cli. This is more of a support issue with VSCode. See https://github.com/chrischinchilla/vale-vscode

brewed

@ksuess see also https://github.com/plone/documentation/blob/6.0/.vscode/settings.json. I use PyCharm, so I can't verify it would work in Training.


Multiple top-level model instances

As pointed out by @blimlim, the Experiment in experiment.py stores two separate instances of the top-level model driver in Experiment.model and in the list Experiment.models. For model drivers such as access, setup and archive is not run for the access instance in Experiment.models list, only for the top-level instance Experiment.model.
There was already a TODO to rename models to submodels to make it more clear:
https://github.com/payu-org/payu/blob/bdf0c66f143d64b2ecf06289d5cb93e994822f68/payu/experiment.py#L150-L151
Could still have self.models = [self.model] + self.submodels for when wanting to iterate over all model drivers, but have only one instance of the top-level model.
What might need to be checked is accessom2 driver archive method. For the Experiment.models instance which archives before other submodel model drivers run archive(), it copies directories from work directories restart path to the output restart path. Then when the top level Experiment.model instance run after other submodel archives, it copies a file from a mom work path to a cice5 restart path.
So maybe there just needs to be a top-level post_setup/post_archive method or similar, that runs after other submodel setup/archives methods have run? This might make it more clear what runs before submodel methods and what runs after submodel methods.

I think payu needs a design document to explain this stuff 🥲

A refactor of some of this would be good I think. Would need to spend time thinking about it to have any useful input.


dockerfile deps

Background
lnd docker build fails for: https://github.com/lightningnetwork/lnd/blob/master/Dockerfile
Steps to reproduce
from the lnd root: docker build .
Expected behaviour
should build without failure
Actual behaviour
fails with the following error:
The command '/bin/sh -c apk add --no-cache     git     make &&  git clone https://github.com/lightningnetwork/lnd /go/src/github.com/lightningnetwork/lnd &&  cd /go/src/github.com/lightningnetwork/lnd &&  make &&  make install' returned a non-zero code: 2

fix is here: https://github.com/lightningnetwork/lnd/pull/2257

Hi, I had a different issue with the current Dockerfile. Seems like no dependencies got downloaded:

\033[0;32m Building debug lnd and lncli.\033[0m
GO111MODULE=on go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5.1-beta-106-g349551373d7fd571c2a917908e7155c6c6f9706a" github.com/lightningnetwork/lnd
nat/upnp.go:9:2: cannot find package "github.com/NebulousLabs/go-upnp" in any of:
/usr/local/go/src/github.com/NebulousLabs/go-upnp (from $GOROOT)
/go/src/github.com/NebulousLabs/go-upnp (from $GOPATH)
...

fixed it locally by adding
go get -d ./... before the make command
I guess dependencies should be handled in the make file, so it's more a hack than a solution. Hope it helps someone!

@adler99 did you run build from / or /docker?

I added the following to the Dockerfile:
&& go get -d ./...
RUN cd  /go/src/github.com/lightningnetwork/lnd && make ...
I needed to add cd again, I guess go get changes the directory...
And then those commands run smoothly:
build the image:
docker build -t lnd . --no-cache
create the container:
docker create --link bitcoin-server --name=lnd -v ${PWD}/data:/root/.lnd -p 9735:9735 -p 10009:10009 lnd:latest

Fixe by https://github.com/lightningnetwork/lnd/pull/2257


Substreams Sink to KV (gRPC/Connect-Web API)

Now that we have a Substreams sink to KV (https://github.com/streamingfast/substreams/issues/93), we will know try to have something easier for the developers to consume the KV store.
We are going to slap a gRPC + Connect Web.on top of the substream-sink-kv. At the infrastructure level, we expect a single binary to run that will do 2 things:

Continuously sink the KV with on the storage engine
Serve through a Connect-Web (HTTP) interface proxy to a gRPC server serving a low-level API on top of the sinked key/values.

rpc {
  // Proper documentation for all request, also describes what happen on error cases (for example, on XXX it returns `InvalidArgument` gRPC code!
  Get(GetRequest) GetResponse
  GetMany(GetManyRequest) GetManyResponse
  Scan(ScanRequest)…
  Prefix(PrefixRequest)…
}

message GetResponse {
  any.Any value = 1;
}

message ScanRequest{
  uint64 limit;
  ...

And the payload would simply be an any.Any, with the value of the key, and the client would do the decoding, because he knows what he’s querying.
While doing this don't forget:

[ ] Create a small example pure HTML/JavaScript that consumes the Connect-Web interface
[ ] Document the gRPC interface properly with details about what it does but especially the error paths.
[ ] Write down important running information about the project (you need to run this, and this, and that, blah blah)

What do we do with maximum results and pagination for requests such as "Scan" and "Prefix" ?
Without "time-traversal" properties on the kv store, we will end up with the same design problem as eth rpc calls...

Small analysis of concurrency problems:

on Badger (local DB only), we can implement WriteLock around the operations of a block
on TiKV, there are ways of wrapping our changes in a transaction (then make it "one block == one transaction"
on BigTable, there are no transactions or decent locking possibility, but we could use the timestamps of cells to filter by block number: https://cloud.google.com/bigtable/docs/using-filters#timestamp-range

on our own 'netkv', I don't see a good solution..

note: before closing this issue, we'll create another issue to write the examples (TokenBalance, NFT owners...) with different priority

Reopening because the task will be closed when instructions will have been sent to Pinax and our Discord

Also, here a few things that are missing:

[ ] Proper release to GitHub version 0.1.0 of the binary + README update for instructions (fix indentation of first instruction also)
[ ] Some minor update to README.md:

Note or Warning should be formatted with new GitHub Note/Warning syntax (see raw source of this comment for raw markup):

Note This is an indented note

Needs a paragraph in between

Warning And a warning

Run command authentication note should also show error message that is displayed when not authenticated, the note in itself could be clearer.

(these commands have been tested while running the 'devel/local/start.sh' in another terminal) should be in a note after the commands. Same thing for browser note (this has been tested while running the 'devel/local/start.sh' in a terminal)

Cargo crate heading is wrong, it's bigger than others, also, the section should be improve to better explain how to add it and use it, something quick

[ ] Tutorial in the documentation about all this, substreams-sink-files tutorial currently in PR can be used
[ ] Instructions is sent to Pinax about the tool ready to be tested
[ ] Discord quick announcement also if devs want to test it, it should be quick and point to tutorial in documentation and README notes.

@maoueh please review https://substreams.streamingfast.io/developers-guide/substreams-sinks/substreams-sink-kv#consume-the-key-value-data-from-a-web-page-using-connect-web
then we can close this issue :D

All good, just some alignment of code blocks with bullet list, closing anyway so we move on.


Numpad support not working with laptop

Hey, love the concept but my computer's numpad support isn't being picked up by OW. I don't know how to modify the code to make a personal patch, so if there's a way you could add rebinding to the mod settings menu for people whose numpad options are limited, it would help a lot!

Okay nevermind I come back an hour later and it works no issue. Outer wilds is finicky with it's inputs huh. Thanks for the awesome mod


⚠ Bofrost.nl has degraded performance

In 2d79e1e, Bofrost.nl (https://www.bofrost.nl) experienced degraded performance:

HTTP code: 200
Response time: 2682 ms

Resolved: Bofrost.nl performance has improved in 3a1ba38 after 6 minutes.


AEM Environment Indicator Not Sticky to Header | AEM Cloud

Issue is on the AEM cloud where user logged in via adobe id (SSO)
Expected Behavior
As per the implementation, Indicator should be stick with the top nav.
Actual Behavior
If user log into the cloud environment via adobe id. User see the indicator not stock with the top nav.
Steps to Reproduce
Logged with adobe id and authenticate with SSO.
User will see Landing page of AME Sites and see the env. indicator is not sticky on the top nav.
We observed the header is spectrum-Shell spectrum--darkest header which was earlier  coral-shell-header in AEM Cloud
Links
Links to related assets, e.g. content packages containing test components

@vivekdhiman what is your OSGi configuration for injecting the indicator? Is the problem the header is changing classes? Ultimately you should be able to use the css-override OSGi config to position your indicator where you want it. You can alos use that inner-html to include JS that can perform fancier/checks and manipulations.
Also, it looks like you are NOT using Unified Shell, is that right?

Hi @davidjgonzalez , We are able to fix this with custom JS and CSS. I believe all the cloud environments where user login with SSO this issue exists. Below is the config we used earlier
{
    "browser-title-prefix":"Author",
    "css-color":"#FF7F50",
    "css-override":"#acs-commons-env-indicator {                     background-color: #800080;                     color: #FFF;                     position: fixed;                     top: 0;                     left: 50%;                     width: 126px;                     margin-left: -63px;                     height: 30px;                     font: bold 18px/29px sans-serif;                     text-align: center;                     border: solid 1px black;                     border-top-width: 0;                     z-index:<PHONE_NUMBER>00000;     }",
    "inner-html": "STAGE<script>document.getElementById('acs-commons-env-indicator').onmouseover = function() { if(this.style.left === '50%') { this.style.left = '25%';} else {this.style.left = '50%'; }};</script>"
}

This should be fixed in 6.0.0.
IIUC the problem was caused by Unified Shell (the new white top nav bar in AEM CS). Unified shell brings in AEM CS as an iframe under it - so some of the JS to inject the indicator didnt work quite right.
It's worth noting that AEM CS now indicates prod/stage/dev/rde OOTB in a little pill in the top left, so not sure how valuable this feature is in AEM CS now.
If you can re-open is youre still seeing strange behavior, or if i misunderstood what you were reporting.


feat: Allow mapped lua functions to have arguments

With a lot of lua plugins it is quite common to have bindings that call a lua function which can take some additional options. It would be nice if these arguments could be stored together with the function. Then upon invoking the mapping, the arguments could be unpacked and forwarded to the function. I think having an option called args that holds the arguments as an array would be a nice improvement.
One could then do
["<Leader>ff"] = { require"telescope.builtin".find_files, args = { require"telescope.themes".get_ivy() }, "Find files, but with ivy-theme" }

instead of
["<Leader>ff"] = { function() require"telescope.builtin".find_files(require"telescope.themes".get_ivy()) end, "Find files, but with ivy-theme" }

Sorry, I'm not a fan of this. I'd rather keep it simple and similar to the vim.keymap api.
Thank you for the suggestion though!


When access management turned on, PATCH will return "Not Implemented" status

Describe the bug
The AccessPlugin currently has set PATCH as not-implemented. AccessManager checkPatchOp(PatchOp op) method is not implemented.
To Reproduce
Deploy server with Access control enabled. Operation will failed with not implemented. Turn access control off, patch works.
Expected behavior
Patch should function if ACLs permit.

Fix checked in to 0.6-dev


Still maintained?

It seem the maintenance of this library has stalled considering the number of outstanding open issues and PRs. It would be greatly appreciated to fix some of the critical bugs this lib has.

I have been busy with other aspects of life. Yes, I'll take up time to fix some of the critical bugs and merge PRs.


#400 CHANNEL_INVALID only using some Phone Numbers

Hi, i'm trying to scrape user info from a supergroup.
I've declared the inputChannel object like this in js:

const inputChannel = {
  '_': 'inputChannel',
  'channel_id':<PHONE_NUMBER>,
  'access_hash': 'xxxxxxxxxxxxx...'
};

This is the call to getParticipiants method:
const { users, count } = await client('channels.getParticipants', {
      channel: inputChannel,
      filter: { _: 'channelParticipantsRecent' },
      offset: counter * 200,
      limit: 200
    });

Now, the problem is that with some Phone Numbers (after the app gets authorized) i'm getting
[1.788] Error 400 CHANNEL_INVALID 2 1
(node:2850) UnhandledPromiseRejectionWarning: Error: 400 CHANNEL_INVALID CODE#400 CHANNEL_INVALID
    at NetworkerThread.processError (/Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:573:12)
    at /Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:728:34
    at Generator.next (<anonymous>)
    at step (/Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:76:202)
    at /Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:76:459
    at Promise._execute (/Users/mirco/node_modules/bluebird/js/release/debuggability.js:303:9)
    at Promise._resolveFromExecutor (/Users/mirco/node_modules/bluebird/js/release/promise.js:483:18)
    at new Promise (/Users/mirco/node_modules/bluebird/js/release/promise.js:79:10)
    at /Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:76:99
    at NetworkerThread.processMessage (/Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:770:7)
    at /Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:336:21
    at Generator.next (<anonymous>)
    at step (/Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:76:202)
    at /Users/mirco/node_modules/telegram-mtproto/lib/service/networker/index.js:76:383
    at tryCatcher (/Users/mirco/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/mirco/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/Users/mirco/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/Users/mirco/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/Users/mirco/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/Users/mirco/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/Users/mirco/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/Users/mirco/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)

The channel is valid (and it's working with my primary phone number). It's not a country code issue because i tried another Italian Phone Number (i'm from italy) and with other one i'm still getting the same error. Code is the same! I'm using nodejs<EMAIL_ADDRESS>Also, the phone number that is working is not even member of the Supergroup.
Any ideas?
Thanks!

Solved by getting Channel ID and Access Hash directly from channel name using contacts.resolveUsername(). I think the Access Hash is different for every user, and thats maybe why it was not working with other accounts than mine.

@ZombieTheBest can you explain more about that?
I got the same error and I don't know how to fix it.
Thanks


Stub out location for document producer schema in docs

#2030
#2029
#2033  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @nickgraz and the rest of your teammates on  Graphite


Spraying doesn't start due to UnicodeDecodeError

I have replaced URLs with REDACTED due to privacy concerns. Please tell me if this prevents my issue from being solved.
Command line input and output:
user@pop-os:~/Downloads/o365spray$ python3 o365spray.py --spray -U ../../users.txt -P ../rockyou.txt --count 2 --domain REDACTED

            *** O365 Spray ***

>----------------------------------------<

   > version        :  3.0.0
   > domain         :  REDACTED
   > spray          :  True
   > userfile       :  ../../users.txt
   > passfile       :  ../rockyou.txt
   > count          :  2 passwords/spray
   > lockout        :  15.0 minutes
   > validate_module:  getuserrealm
   > spray_module   :  oauth2
   > rate           :  10 threads
   > safe           :  10 locked accounts
   > timeout        :  25 seconds
   > start          :  2022-09-29 16:47:37

>----------------------------------------<

[2022-09-29 16:47:37,905] [info] Validating: REDACTED
[2022-09-29 16:47:38,414] [info] [WARNING] The following domain appears to be using O365, but is Federated: REDACTED
	[!] --> ADFS AuthURL: REDACTED
[2022-09-29 16:47:38,414] [info]

[ ? ]	Switch to the ADFS module for password spraying [Y/n] n
Traceback (most recent call last):
  File "/home/user/Downloads/o365spray/o365spray.py", line 6, in <module>
    main()
  File "/home/user/Downloads/o365spray/o365spray/__main__.py", line 272, in main
    spray(args, output_directory, enum)
  File "/home/user/Downloads/o365spray/o365spray/core/handlers/sprayer/spray.py", line 39, in spray
    passlist += Helper.get_list_from_file(args.passfile)
  File "/home/user/Downloads/o365spray/o365spray/core/utils/helper.py", line 135, in get_list_from_file
    list_ = [line.strip() for line in f if line.strip() not in [None, ""]]
  File "/home/user/Downloads/o365spray/o365spray/core/utils/helper.py", line 135, in <listcomp>
    list_ = [line.strip() for line in f if line.strip() not in [None, ""]]
  File "/usr/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 923: invalid continuation byte

Thank you in beforehand!

This error is due to the passwords stored within rockyou.txt - some passwords are not UTF-8 encoded.
I will look into options regarding varying encodings with password/username file handling, but for now if you want to resolve this issue you can do one of two things:

Use a modified version of rockyou.txt that does not include the problematic passwords.
Locally update the encoding type for the file read handler in o365spray located at: https://github.com/0xZDH/o365spray/blob/master/o365spray/core/utils/helper.py#L134. I believe latin-1 should work with rockyou.txt, but this could potentially cause further issues as I have not tested it myself.

Option 1 worked as a workaround, haven't tried no 2 yet. I will leave this issue open as the root of the problem in the application remains for now, though.


Add EntityListProvider with hooks/filters

Introduces a useEntityListProvider hook to request catalog entities from the backend with provided filters.
This is inspired by #2791 and @freben's experimental branch. The UI closely follows #2791, where the catalog tabs are instead shown as a component Type dropdown, which only shows types that exist in the catalog.

There are a few goals captured here:

Enable showing all catalog entities, not just components (enabled by, but not implemented in this PR).
Provide a generic "catalog entity list" functionality that can be used by CatalogPage, ScaffolderPage, ApiExplorerPage, etc (this only converts CatalogPage, for PR size considerations, but I did convert ScaffolderPage locally as a proof of concept).
Allow more backend filtering, as a step-wise performance improvement (does not consider pagination yet).
Make it easy to customize CatalogPage by minimizing the JSX footprint.
Make it easy to add custom filters.

Adds an EntityListProvider to wrap the Catalog home page. This provides a list of resolved entities based on provided filters, which can choose to implement either catalog-backend query filters, or frontend function filters.
I inverted control so the provider/hook doesn't know or care whether a particular filter is frontend or backend. This should make it easy to convert frontend filters to backend ones once more advanced queries are supported. Initial filters can be provided, and the filter context is strongly typed but extensible via a generic on the provider and hook.
The footprint of the CatalogPage with these changes becomes:
<CatalogLayout>
  <Content>
    <EntityListProvider initialFilters={initialFilters}>
      <ContentHeader title="Components">
        <CreateComponentButton />
        <SupportButton>All your software catalog entities</SupportButton>
      </ContentHeader>

          <EntityTypePicker />
          <UserListPicker />
          <EntityTagPicker />

        <CatalogTable columns={columns} />

    </EntityListProvider>
  </Content>
</CatalogLayout>

The goal is to make this so small and composable that customization is trivial. The div elements are only for layout and could probably be removed also by making the children aware of grid template areas (indirectly, e.g. useEntityFilterStyles()).
Replaces #5549 wherein I royally hosed DCO.
Co-authored-by: Mike Lewis<EMAIL_ADDRESS>:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Looks great and I believe this will solve a very real UX need.

Thanks for taking up reigns on this @timbonicus 😀. I've had no time to work on a large effort like #2791 recently. I've only quickly skimmed over the PR, but the direction seems good so far. Looking forward to this shipping 🎉.


Question about test

I used the guide to test but it didn't works:
pi@Pivo:~/hx711 $ g++ -Wall -o prog main.cpp -lhx711 -llgpio
pi@Pivo:~/hx711 $ ./prog 23 24 -23 156209
./prog: error while loading shared libraries: libhx711.so: cannot open shared object file: No such file or directory
pi@Pivo:~/hx711 $ sudo ./prog 23 24 -23 156209
./prog: error while loading shared libraries: libhx711.so: cannot open shared object file: No such file or directory
The callibration and simple & advanced tests works well.
What I do bad?
Thanks.

Did you run sudo make install?
You may also need to run ldconfig afterward.

Thanks for the kick up, the sudo ldconfig is the way. Thanks again.


[#111509456] - Lint checks for shell scripts

What
Use shellcheck to check all our shell scripts for common errors. This also fixes all the errors that were reported.
Details
This includes some refactorings:
The bosh-manitest/build_manifest.sh has been refactored to be more consistent
with the cloudfoundry one and take the paths to output files in environment
variables. This was done to avoid some issues with array variables in POSIX
scripts that were being flagged up by shellcheck (bash isn't available in
alpine based containers). See 9138c37
The travis build has been refactord to be a single job instead of a matrix
build as it's more efficient this way. This also included creating a
Makefile which will make it easier to run locally. (see 2538f91)
How to test
Run the tests localy - make test (these will require shellcheck > 0.4.0 to be
installed locally).
On Mac - brew install shellcheck
On Linux:
git clone git@github.com:koalaman/shellcheck.git
sudo apt-get install cabal-install ghc
cd shellcheck
cabal update
cabal install # installs into ~/.cabal/bin
Confirm that the deployment process still works after the various refactorings.
Who can review
Anyone but @alext  or myself.
##Notes : We can re-enable the SC2086 checks in the s3init.sh and s3get.sh scripts after Shellcheck 0.4.3 is available for everyone.

This looks great.
I'm in the process of testing the pipeline changes.
I got bored of waiting for brew to compile Haskell. What do you think about allowing people to override the shellcheck command so that you can use a Docker container? e.g.
diff --git a/Makefile b/Makefile
index c7f6094..f26845a 100644
--- a/Makefile
+++ b/Makefile
@@ -1,5 +1,7 @@
 .PHONY: test spec lint_terraform lint_shellcheck

+SHELLCHECK=shellcheck
+
 test: spec lint_terraform lint_shellcheck

 spec:
@@ -12,4 +14,4 @@ lint_terraform:
        find terraform -mindepth 1 -maxdepth 1 -type d -print0 | xargs -0 -n 1 -t terraform graph > /dev/null

 lint_shellcheck:
-       find . -name '*.sh' -print0 | xargs -0 shellcheck
+       find . -name '*.sh' -print0 | xargs -0 $(SHELLCHECK)

With this container:
FROM debian:sid

RUN apt-get -yq update && apt-get -yq upgrade
RUN apt-get -yq install shellcheck

ENTRYPOINT ["shellcheck"]
CMD []

And run like:
➜  paas-cf git:(shellcheck) ✗ make test SHELLCHECK="docker run -v $(pwd):/mnt -w /mnt shellcheck"
…
find . -name '*.sh' -print0 | xargs -0 docker run -v /Users/dcarley/projects/paas/paas-cf:/mnt -w /mnt shellcheck

I think that's a superb idea - Will get it implemented tomorrow.

When building a new container, we should probably build the latest version.  That apt package is not the most recent....

0.4.3 is already in brew
brew info shellcheck
shellcheck: stable 0.4.3 (bottled), HEAD
Static analysis and lint tool, for (ba)sh scripts
http://www.shellcheck.net
/usr/local/Cellar/shellcheck/0.4.1 (6 files, 17.2M)
  Poured from bottle
/usr/local/Cellar/shellcheck/0.4.2 (6 files, 17.6M)
  Poured from bottle
/usr/local/Cellar/shellcheck/0.4.3 (6 files, 17.6M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/shellcheck.rb

but using shellcheck from a container sounds like a better idea.

The container matches what's used in Travis. Using a later version would allow us to re-enable the check SC2086, but makes the installation in Travis much more ghetto. I think we should hold off upgrading for the moment.

:+1:

Note: we talked about not documenting the Dockerfile approach for now. I'm going to try getting shellcheck working again from a Homebrew bottle. If enough people find that they need to run the tests locally and having trouble installing specific versions of shellcheck then we can review it again.


esp_bt_mem_release issue with ESP_BT_MODE_BTDM as parameter

Calling one of those or both to release memory works:
esp_bt_mem_release(ESP_BT_MODE_CLASSIC_BT);
esp_bt_mem_release(ESP_BT_MODE_BLE);

but calling:
esp_bt_mem_release(ESP_BT_MODE_BTDM);
is causing crash in line:
https://github.com/espressif/esp-idf/blob/80c5547b971128fdecbbcf6d8178196d35a15b73/components/bt/bt.c#L800
It can be issue with my simple test code, because i dont have turned on classic bt in menuconfig.
PS what is the point of having two times the same code here(just asking):
https://github.com/espressif/esp-idf/blob/80c5547b971128fdecbbcf6d8178196d35a15b73/components/bt/bt.c#L796-L805
and here:
https://github.com/espressif/esp-idf/blob/80c5547b971128fdecbbcf6d8178196d35a15b73/components/bt/bt.c#L819-L828

@chegewara If you check, mem_start and mem_end point to different regions in all these calls:

BTDM BSS and Data
BT BSS and Data

Its not the point. When i try only this command then it crash:
esp_bt_mem_release(ESP_BT_MODE_BTDM);
But i can use separate both commands together to release all bt memory and it works:
esp_bt_mem_release(ESP_BT_MODE_CLASSIC_BT);
esp_bt_mem_release(ESP_BT_MODE_BLE);

There is no need to build special test app for this, i am using this:
app_main(){
esp_bt_mem_release(ESP_BT_MODE_CLASSIC_BT);
esp_bt_mem_release(ESP_BT_MODE_BLE);
}

I am assuming that
esp_bt_mem_release(ESP_BT_MODE_CLASSIC_BT); +esp_bt_mem_release(ESP_BT_MODE_BLE);
is the same as esp_bt_mem_release(ESP_BT_MODE_BTDM);
but again i may be wrong.
Only i know this command is causing crash esp_bt_mem_release(ESP_BT_MODE_BTDM);

@dhrishi I see what you mean, my bad sorry.

@chegewara No. Calling the below APIs in succession
esp_bt_mem_release(ESP_BT_MODE_CLASSIC_BT);
esp_bt_mem_release(ESP_BT_MODE_BLE);
is not equivalent to calling esp_bt_mem_release(ESP_BT_MODE_BTDM);
If ESP_BT_MODE_CLASSIC_BT or ESP_BT_MODE_BLE mode is passed to esp_bt_mem_release(), it just releases the controller memory. It does not maintain the state of your previous calls and hence doesn't call the conditions which check for ESP_BT_MODE_BTDM.
Although, if ESP_BT_MODE_BTDM is passed then apart from freeing the controller memory, it releases BTDM and BT library data and BSS to heap.
For more info and details about appropriate usage of this API, please refer its documentation.

Is this issue solved now?

I dont know, is it solved? Is it issue at all?

It may not be considered as an issue, as the API has been used incorrectly or under wrong conditions. Although, an internal MR is already submitted which should handle this error case.
I will update when it gets merged. @chegewara you can then give it a try and if it fixes your issue then we will close this.

@chegewara Can you try with latest ESP IDF and see if you don't see a crash anymore?

@dhrishi when i use this code then all works fine:
esp_bluedroid_deinit();
 esp_bt_controller_disable();
 esp_bt_controller_deinit();
 esp_bt_mem_release(ESP_BT_MODE_BTDM);

and no, i app still crash if i use only  esp_bt_mem_release(ESP_BT_MODE_BTDM);. Logs:
I (3076) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
I (3095) heap_init: At 3FFBDB5C len 00000004 (0 KiB): DRAM
D (3114) heap_init: New heap initialised at 0x3ffc1048
I (3130) heap_init: At 3FFC1048 len 0001EFB8 (123 KiB): DRAM
I (3150) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (3169) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
D (3190) heap_init: New heap initialised at 0x4008e010
I (3205) heap_init: At 4008E010 len 00011FF0 (71 KiB): IRAM
I (3225) cpu_start: Pro cpu start user code
D (3261) clk: RTC_SLOW_CLK calibration value: 3461850
D (1069) intr_alloc: Connected src 46 to int 2 (cpu 0)
D (1070) intr_alloc: Connected src 57 to int 3 (cpu 0)
D (1071) intr_alloc: Connected src 24 to int 9 (cpu 0)
I (1076) cpu_start: Starting scheduler on PRO CPU.
D (0) intr_alloc: Connected src 25 to int 2 (cpu 1)
I (4) cpu_start: Starting scheduler on APP CPU.
D (1100) heap_init: New heap initialised at 0x3ffe0440
D (1105) heap_init: New heap initialised at 0x3ffe4350
I (1111) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations
D (1119) spiram: Allocating block of size 32768 bytes
D (1124) intr_alloc: Connected src 16 to int 12 (cpu 0)
D (1130) BTDM_INIT: Release DRAM [0x3ffae6e0] - [0x3ffaff10]

D (1136) BTDM_INIT: Release DRAM [0x3ffb0000] - [0x3ffb6388]

D (1142) BTDM_INIT: Release DRAM [0x3ffb8000] - [0x3ffb9a20]

D (1147) BTDM_INIT: Release DRAM [0x3ffbdb28] - [0x3ffbdb5c]

D (1153) BTDM_INIT: Release BT BSS [0x3ffbfdc8] - [0x3ffbfdcc]

ESP_ERROR_CHECK failed: esp_err_t 0xffffffff (ESP_FAIL) at 0x400e673b
0x400e673b: esp_bt_mem_release at D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c:917 (discriminator 1)

file: "D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c" line 917
func: esp_bt_mem_release
expression: heap_caps_add_region(mem_start, mem_end)

Backtrace: 0x4008a0fc:0x3ffbafd0 0x4008a544:0x3ffbaff0 0x400e673b:0x3ffbb010 0x400d5159:0x3ffbb040 0x400d514f:0x3ffbb060 0x400d0b56:0x3ffbb080
0x4008a0fc: invoke_abort at D:/msys32/home/imper/esp/esp-idf/components/esp32/panic.c:660

0x4008a544: _esp_error_check_failed at D:/msys32/home/imper/esp/esp-idf/components/esp32/panic.c:676

0x400e673b: esp_bt_mem_release at D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c:917 (discriminator 1)

0x400d5159: SampleServer() at D:/msys32/home/imper/ble_test/main/SampleServer.cpp:102

0x400d514f: app_main at D:/msys32/home/imper/ble_test/main/main.cpp:41

I should read docs more often.

Here is log when i use full bt deinit and app crash:
D (1259) BTDM_INIT: Release DRAM [0x3ffae6e0] - [0x3ffaff10]

D (1265) BTDM_INIT: Release DRAM [0x3ffb0000] - [0x3ffb6388]

D (1271) BTDM_INIT: Release DRAM [0x3ffb8000] - [0x3ffb9a20]

D (1277) BTDM_INIT: Release DRAM [0x3ffbdb28] - [0x3ffbdb5c]

D (1283) BTDM_INIT: Release BTDM BSS [0x3ffc0b38] - [0x3ffc13d8]

D (1289) BTDM_INIT: Release BTDM Data [0x3ffbdbb4] - [0x3ffbdbd4]

ESP_ERROR_CHECK failed: esp_err_t 0xffffffff (ESP_FAIL) at 0x400e7423
0x400e7423: esp_bt_controller_mem_release at D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c:1130

file: "D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c" line 896
func: esp_bt_controller_mem_release
expression: heap_caps_add_region(mem_start, mem_end)

Backtrace: 0x400920b8:0x3ffbafa0 0x40092504:0x3ffbafc0 0x400e7423:0x3ffbafe0 0x400e7445:0x3ffbb010 0x400d5dee:0x3ffbb040 0x400d5ddb:0x3ffbb060 0x400d15b2:0x3ffbb080
0x400920b8: invoke_abort at D:/msys32/home/imper/esp/esp-idf/components/esp32/panic.c:660

0x40092504: _esp_error_check_failed at D:/msys32/home/imper/esp/esp-idf/components/esp32/panic.c:676

0x400e7423: esp_bt_controller_mem_release at D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c:1130

0x400e7445: esp_bt_mem_release at D:/msys32/home/imper/esp/esp-idf/components/bt/bt.c:1130

0x400d5dee: SampleServer() at D:/msys32/home/imper/ble_test/main/SampleServer.cpp:102

0x400d5ddb: app_main at D:/msys32/home/imper/ble_test/main/main.cpp:41

0x400d15b2: main_task at D:/msys32/home/imper/esp/esp-idf/components/esp32/cpu_start.c:473

@chegewara Thanks for your response. A couple of things here:

Initially (with no logs) I thought the issue would have been that start and end are same, the region being empty, and hence heap_caps_add_region() should not be called. This is fixed with: eb181220384d1e974fb4c8beb8a618bd4f2862c0
With the logs you have provided for both cases (I assume that you have still not enabled classic BT), the address range which cause the crash:
(0x3ffbfdc8, 0x3ffbfdcc) and (0x3ffbdbb4, 0x3ffbdbd4) seem to be invalid (Please refer file soc_memory_layout.h) and hence it has asserted from heap_caps_add_region () on purpose.
Let me know if the explanation helps.

@dhrishi One word of explanation. I did like you suggested before, ive read docs and tried this code (before updating esp-idf):
esp_bluedroid_disable();
 esp_bluedroid_deinit();
 esp_bt_controller_disable();
 esp_bt_controller_deinit();
 esp_bt_mem_release(ESP_BT_MODE_BTDM);

It worked, so like i said i have to read docs more often.
Now, you asked me to test with only one command esp_bt_mem_release(ESP_BT_MODE_BTDM);, so i updated to latest commit and here is the problem. Test app crash when i am using only esp_bt_mem_release(ESP_BT_MODE_BTDM); or when i am doing full bt deinit, and yes i have turned on both classic bt and ble.

(0x3ffbfdc8, 0x3ffbfdcc) and (0x3ffbdbb4, 0x3ffbdbd4) seem to be invalid
@dhrishi no they are just too small to create a new heap

@negativekelvin Yes. That issue has arisen due to latest optimization in BT.
Releasing BTDM data to heap gives this error as the size is too less (around 32 bytes) to create a new heap. I am already working on it so that esp_bt_mem_release handles it correctly.

@dhrishi What it should do is try to combine the bss,data sections and create 2 heap regions not 4

This issue should have been solved in the latest IDF, please have a try. Thanks.

@dhrishi @TianHao-Espressif can this fragmentation be improved?? i think yes.
D (262) BTDM_INIT: .data initialise [0x3ffae6e0] - [0x3ffaff10]         D (1122) BTDM_INIT: Release DRAM [0x3ffae6e0] - [0x3ffaff10]
I (750) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
D (262) BTDM_INIT: .bss initialise [0x3ffb0000] - [0x3ffb09a8]          D (1122) BTDM_INIT: Release DRAM [0x3ffb0000] -
D (272) BTDM_INIT: .bss initialise [0x3ffb09a8] - [0x3ffb1ddc]
D (272) BTDM_INIT: .bss initialise [0x3ffb1ddc] - [0x3ffb2730]
D (282) BTDM_INIT: .bss initialise [0x3ffb2730] - [0x3ffb6388]                                                          [0x3ffb6388]
I (761) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
D (292) BTDM_INIT: .bss initialise [0x3ffb8000] - [0x3ffb9a20]          D (1132) BTDM_INIT: Release DRAM [0x3ffb8000] - [0x3ffb9a20]
I (772) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
D (292) BTDM_INIT: .bss initialise [0x3ffbdb28] - [0x3ffbdb5c]          D (1132) BTDM_INIT: Release DRAM [0x3ffbdb28] - [0x3ffbdb5c]
I (778) heap_init: At 3FFBDB5C len 00000004 (0 KiB): DRAM
                                                                        D (1162) BTDM_INIT: Release BT Data [0x3ffbdb60] - [0x3ffbdcf8]
                                                                        D (1142) BTDM_INIT: Release BTDM Data [0x3ffbdcf8] - [0x3ffbdd18]

                                                                        D (1152) BTDM_INIT: Release BT BSS [0x3ffc0b40] - [0x3ffc9c30]
                                                                        D (1142) BTDM_INIT: Release BTDM BSS [0x3ffc9c30] - [0x3ffca4d0]

I (789) heap_init: At 3FFCC580 len 00013A80 (78 KiB): DRAM
I (796) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (802) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (814) heap_init: At 40091EF0 len 0000E110 (56 KiB): IRAM


Updated chat with retrieval

I've updated imports here: 4789c459449f42e737e3f01978d3256e1f7e6bcb

@CSalle I am really busy sometimes, but I've finally pushed most of these changes to main. In the end, I used your comments on #13 - they were immensely useful! I hope I didn't miss anything. It's both on the softupdate and the main branches. Thanks again, much appreciated!

Thank you, Carlos!


mini.ai - using textobject auto closes cmdline window

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md
[X] I have updated 'mini.nvim' to latest version

Module(s)
mini.ai
Description
Using either the i or a textobjects in the cmdline window (q:, q/, q?) will abruptly close said window. I believe the cause is in this function, in particular the <C-\><C-n> sequence. Perhaps <Esc> or <C-c> would suffice?
Neovim version
v0.9.0-dev-883+g1391385ba
Steps to reproduce

Open the command line window via q:.
Insert (hello) into an open line.
Position cursor inside the parenthesis. Use vi( in normal mode.

Note that this can be reproduced with any built in textobject. If performing a change (EG: ci(), nvim will enter insert mode, but will exit the cmdline window as soon as the user returns to normal mode.
Expected behavior
The cmdline window shouldn't close after using mini.ai textobject.
Actual behavior
The cmdline window abruptly closes.

Yes, you are right. Rarely use command-line window myself, so didn't notice this earlier.
I'll fix this soon (probably tomorrow).

Should be fixed now in main.
Thanks again for noticing!


Question: What is the basic auth for regarding the api section in the config file?

What is the credentials referring to regarding the api section? Is this with the origin or replica or the adguardhome-sync docker itself?
api:
  # Port, default 8080
  port: 8080
  # if username and password are defined, basic auth is applied to the sync API
  username: username
  password: password

Ah ok so re-reading it this is where you "define" user credentials I'm assuming the purpose is to add some basic security to the api endpoint?

@bakito different question does this docker sync metrics as well? I see a flag for statsConfig

No, only Statistics configuration is synced.


PC Lags a lot for some seconds when game is closing.

What happened?
When i close the game and i have the plugin enabled, the pc starts lagging a lot, and then after some seconds, the game closes.
Version
3.2
Godot Version
4.2.stable
Exact steps to reproduce this error
When i close the game and i have the plugin enabled, the pc starts lagging a lot, and then after some seconds, the game closes.
GDScript
Doesn't have to do with the code.

Godot output
Works normally.

Additional information
Nothing.
Checks

[X] I tried reinstalling the addon or tried to fix it myself with other methods.
[X] I tried restarting Discord and Godot completely.

is already fixed but i broke linux support. i got sick so i didn't have time to fix it for release.

Strange, it still happens to me, and i downloaded the 3.2 version of the plugin.

(I'm on windows btw).

i just told you i didn't have time to fix stuff for releasing it

Ohh yeah ok, my bad. I'll wait for the next release then, have a nice day!

no problem! i will remind you :)

got fixed but make sure to read the release nodes


Path card sign change

Part of https://github.com/quasar-analytics/quasar/issues/1716
It changes the signature of pathCard to close values over some M.
For zygoM matryoshka was upgraded to 0.18.3
Commit cf56d33 will be reverted once https://github.com/slamdata/matryoshka/pull/78 is merged to matryoshka

I won’t merge until we discuss slamdata/matryoshka#78 further, since it affects the commits here.


fix: change ComptimeStringMap to StaticStringMap

reflect changes made in https://github.com/ziglang/zig/commit/8af59d1f98266bd70b3afb44d196bbd151cedf22

Thanks! Since this breaks 0.12.0 compatibility we'll need to do the regular stuff.
Actually though, I want to see if it's possible to support multiple Zig versions using this comptime if on the Zig version. It seems like it would work and that would make things easier for users. It doesn't seem to hard to add, but then we'll need to set up some CI for that. We can merge this for now and create 0.12 compatibility after that, though.


Request for security contact

I have found a security issue in this code that I want to report privately. Please contact me via e-mail for details.

Is the issue in the spark code, or the fhir dot net client code?

It's in the src\Spark.Engine directory of this repository. Does that mean that it's in the Spark code? I'm sorry if I'm a bit unclear, but I'm not familiar with this project at all. I found the issue by searching for code that matched a certain pattern as part of some research I'm doing.

For anyone else looking into this. I've shared the details with @brianpos through e-mail.


update netCDF-Java libs

I updated my pom.xml to use this one, and now I don't get the dependency errors.  Woohoo!
I do get the following failed test, however.  Any ideas?
Results :

Failed tests:   htmlReadTest(gov.noaa.eds.threddsutilities.util.XMLUtilTest)

Tests in error:
  translateTest(gov.noaa.eds.threddsutilities.util.ThreddsTranslatorUtilTest): Configuration problem: http://www.ngdc.noaa.gov/metadata/published/xsl/UnidataDDCount-HTML.xsl TransformerConfigurationException. javax.xml.transform.TransformerException: org.xml.sax.SAXParseException; systemId: http://www.ngdc.noaa.gov/metadata/published/xsl/UnidataDDCount-HTML.xsl; lineNumber: 1; columnNumber: 50; White spaces are required between publicId and systemId.

Tests run: 5, Failures: 1, Errors: 1, Skipped: 0

@geoneubie, there must be another http: sneaking around somewhere, right?
I got the above error using the 2 changes to https: you suggested here: https://github.com/noaaroland/uafnciso/pull/1/files

These tests look pretty brittle...
But yea here:
https://github.com/noaaroland/uafnciso/blob/master/src/test/java/gov/noaa/eds/threddsutilities/util/ThreddsTranslatorUtilTest.java#L11
And then there's a hardcoded path below here that you might update to /tmp
https://github.com/noaaroland/uafnciso/blob/master/src/test/java/gov/noaa/eds/threddsutilities/util/ThreddsTranslatorUtilTest.java#L16
On Wed, Jan 11, 2017 at 3:47 PM, Rich Signell<EMAIL_ADDRESS>wrote:

@geoneubie https://github.com/geoneubie, there must be another http:
sneaking around somewhere, right?
I got the above error using the 2 changes to https: you suggested here:
https://github.com/noaaroland/uafnciso/pull/1/files
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/noaaroland/uafnciso/pull/4#issuecomment-271989124,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADIU3aKueKECARPInKcQ70E3Ugo8r1zBks5rRT_MgaJpZM4Lg-mg
.

@noaaroland , does this repo actually build successfully for you?

Yes, @rsignell-usgs of course it builds.
Once I commit my local changes that it. Anyhoo, I updated the pom and made a release of a jar that you should try.


can an alert route to two receivers?

I like to have two receivers for each alert.  One for slack/devops notification. another goes to logstach => elasticsearch for auditing purpose.  I wonder if we can achieve this today?

User questions are best asked at https://groups.google.com/forum/#!aboutgroup/prometheus-users
https://www.robustperception.io/sending-alert-notifications-to-multiple-destinations/


Starterkit route issue

When doing a durandal grunt task build of a project based on the starterkit I have come up against a route error.
Initially hitting the app with my defined routes #settings or #search works fine, however if the project is hit without a route an error results:
TypeError: instance is undefined
http://localhost:9001/app/main.js
Line 3820

Which is:
function hasChildRouter(instance, parentRouter) {
    return instance.router && instance.router.parent == parentRouter;
}

So routes are working, it's not a relative path issue, but the welcome route fails.
https://github.com/BlueSpire/Durandal/blob/master/src/starterkit/app/viewmodels/welcome.js
It appears to be this needs the app module loaded:
define(['durandal/app'], function (app) {

@mryellow Could you submit a PR with the above code changes and corresponding tests?  Should be simple enough to emulate the workflow you described and showing it doesn't break any more, if not just let me know.

Will be good practice for me, haven't done a public PR before, should be straight-forward enough.
Was day-dreaming about this one the other day and it may be a little quirky. My builds had been working, something changed, I imagine app was on a global scope somewhere or some sync issue with things being loaded in time.
Will see what I can do over the next few days.

How is this coming along?

Been a bit busy.... but thinking about it I'm not sure there is anything to test, or that I can easily duplicate the issue. Was something that only popped up when boiling down the config to a more production ready build and was changing a few things at once when it happened.


Witness vote page incl. related informations needed

Need a page to vote for witness like
https://steemit.com/~witnesses
https://steemd.com/witnesses
maybe add related infos on witness like: https://steemit.com/steemit/@thecryptofiend/witness-questionnaire-a-solution-for-voter-confusion

This is not a priority, it a feature we can add in longer term.


LL-7607 Add inline app installs analytics events

Tracking events whenever there is an inline app install (i.e: an action that requires an app to be installed on the device so it automatically gets installed).
track(
   "In-line app install",
   { appName, flow },
];

The value of flow will default to "unknown" and for the main flows that I could identify and test, it will take a specific value:
Flows tested and working:

[x] Add account -> inline app install -> event with prop flow: "add account"
[x] Buy -> has no account -> add account -> inline app install -> event with prop flow: "buy"
[x] Buy -> has account -> inline app install -> event with prop flow: "buy"
[x] Receive -> has account -> inline app install -> event with prop flow: "receive"
[x] Send -> has account -> inline app install -> event with prop flow: "send"
[x] Swap -> has no account -> add account -> inline app install -> event with prop flow: "swap"

Flows not tested (not enough funds to test):

[ ] Swap -> has account -> inline app install -> should send an event with prop flow: "swap"

"Sell" flows: no inline app install because it goes through the manager to install the "Exchange" app
Type
Analytics
Context
LL-7607
Parts of the app affected / Test plan

Set env variable ANALYTICS_CONSOLE=1
Rebuild the app
Test all the aforementioned flows

@gre could you redo a quick review please now that CI is passing? 🙏

@ofreyssinet-ledger
New swap form has been merged in develop, I resolved the conflict but you've to add analytics again on  src/screens/Swap/FormSelection/AccountSelect.js.

@nabil-brn it should be fine now


[Segment Replication] Move CLUSTER_REPLICATION_TYPE_SETTING out of experimental feature flag

Description
This PR will move CLUSTER_REPLICATION_TYPE_SETTING out of experimental feature flag and make it generally available for the users.
Related Issues
Resolves #6656 , Related to #7420

Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

I think this should have added a CHANGELOG entry since it's making a config GA. @mch2 / @Rishikesh1159

@kotwanikunal We already added a changelog entry in previous PR: https://github.com/opensearch-project/OpenSearch/pull/7420, that should be good I think

@kotwanikunal We already added a changelog entry in previous PR: #7420, that should be good I think. Do you think we need one more entry for this moving out of feature flag?

Nope. This should suffice. Please update on the documentation page that it is not under the feature flag anymore or add docs for the config if it does not exist.

Thanks. Sure I have opened a new issue in documentation repo to document this new setting: https://github.com/opensearch-project/documentation-website/issues/4197


Sanitiziing headers in AWS processing is broken after refactoring

See this for reference - https://github.com/spring-cloud/spring-cloud-function/commit/e7f4008b1e4313fb802dd11a18a312c4dc3361d4

Actually what is also related is this - https://github.com/spring-cloud/spring-cloud-function/commit/ac7a3fe873706d6ccd5b737d526c0016e3fdd3e2 commit. Since Kotlin initialization was moved to lazy it was not bootstrapping all the functions.


drivers/foc: support for BEMF sensing

Summary
drivers/foc: support for BEMF sensing
Impact
The next step towards BEMF sensing.
Lower-half logic is not yet implemented.
Releted issue: https://github.com/apache/nuttx/issues/8098
Testing
b-g431b-esc1

@shizacat is this what you are looking for? Amazing @raiden00pl !!!

@acassis  Yep! @raiden00pl it is great! Thank you very much!