error: 'class tensorflow::TensorShape' has no member named 'AddDimWithStatus'

System information

OS Platform and Distribution: Ubuntu 18.04.1 [64-bit]
TensorFlow installed from: source
TensorFlow version: 2.4
Python version: 3.7.5 [64-bit]
Installed using: virtualenv
Bazel version: 3.4.0
GCC/Compiler version (if compiling from source):  7.5.0 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
Hardware: Raspberry Pi 4 Model B Rev 1.4
Architecture: aarch64

Describe the problem
I could build TF 2.4 on a RPi 4 with Ubuntu 18.04 (64-bit) using bazel. However, when I tried doing this on a fresh one, it failed to build. What should I look into to make a comparison and see where the problem is?
Both have the same GCC version (7.5.0) and same bazel version (3.4.0).  Also in both caes, I'm using virtual enviornment (Python 3.7). I get this error:
tensorflow/core/kernels/sparse_split_op.cc:71:34: error: 'class tensorflow::TensorShape' has no member named 'AddDimWithStatus'                       dense_shape.AddDimWithStatus(input_shape_flat(i)));
Provide the exact sequence of commands / steps that you executed before running into the problem
ubuntu@ubuntu:~$ sudo apt-get install build-essential pkg-confi gi2c-tools avahi-utils joystick libopenjp2–7-dev libtiff5-dev gfortran libatlas-base-dev libopenblas-dev libhdf5-serial-dev git ntp

ubuntu@ubuntu:~$ sudo apt-get install python3.7 python3.7-dev python3.7-venv python3.7-distutils python3.7-lib2to3 python3.7-gdbm python3.7-tk python3-pip

ubuntu@ubuntu:~$ python3.7 -m venv py37env

ubuntu@ubuntu:~$ source py37env/bin/activate

(py37env) ubuntu@ubuntu:~$ python3.7 -m pip install — upgrade pip

(py37env) ubuntu@ubuntu:~$ pip install cython  wheel numpy h5py pybind11

(py37env) ubuntu@ubuntu:~$ pip install -U keras_preprocessing — no-deps

(py37env) ubuntu@ubuntu:~$ git clone https://github.com/tensorflow/tensorflow.git

(py37env) ubuntu@ubuntu:~$ cd tensorflow/

(py37env) ubuntu@ubuntu:~/tensorflow$ git checkout r2.4

ubuntu@ubuntu:~/tensorflow$ python3.7 configure.py
You have bazel 3.4.0 installed.
Please specify the location of python. [Default is /home/ubuntu/py37env/bin/python3.7]:
Found possible Python library paths:
 /home/ubuntu/py37env/lib/python3.7/site-packages
Please input the desired Python library path to use. Default is [/home/ubuntu/py37env/lib/python3.7/site-packages]
Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: n
No CUDA support will be enabled for TensorFlow.
Do you wish to download a fresh release of clang? (Experimental) [y/N]: n
Clang will not be downloaded.
Please specify optimization flags to use during compilation when bazel option “ — config=opt” is specified [Default is -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding “ — config=<>” to your build command. See .bazelrc for more details.
 — config=mkl # Build with MKL support.
 — config=mkl_aarch64 # Build with oneDNN support for Aarch64.
 — config=monolithic # Config for mostly static monolithic build.
 — config=ngraph # Build with Intel nGraph support.
 — config=numa # Build with NUMA support.
 — config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
 — config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
 — config=noaws # Disable AWS S3 filesystem support.
 — config=nogcp # Disable GCP support.
 — config=nohdfs # Disable HDFS support.
 — config=nonccl # Disable NVIDIA NCCL support.

(py37env) ubuntu@ubuntu:~/tensorflow$ bazel build //tensorflow/tools/pip_package:build_pip_package

(py37env) ubuntu@ubuntu:~/tensorflow$ python
Python 3.7.5
[GCC 8.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>>
Any other info / logs
(py37env) ubuntu@ubuntu:~/tensorflow$ bazel build //tensorflow/tools/pip_package:build_pip_package
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=142
INFO: Reading rc options for 'build' from /home/ubuntu/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/ubuntu/tensorflow/.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=0 --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --noincompatible_prohibit_aapt1 --enable_platform_specific_config --config=short_logs --config=v2
INFO: Reading rc options for 'build' from /home/ubuntu/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/home/ubuntu/py37env/bin/python3.7 --action_env PYTHON_LIB_PATH=/home/ubuntu/py37env/lib/python3.7/site-packages --python_path=/home/ubuntu/py37env/bin/python3.7 --config=xla --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:short_logs in file /home/ubuntu/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/ubuntu/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:xla in file /home/ubuntu/tensorflow/.bazelrc: --define=with_xla_support=true
INFO: Found applicable config definition build:linux in file /home/ubuntu/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels
INFO: Found applicable config definition build:dynamic_kernels in file /home/ubuntu/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
DEBUG: Rule 'io_bazel_rules_go' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1557349968 -0400"
DEBUG: Repository io_bazel_rules_go instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule git_repository defined at:
  /home/ubuntu/.cache/bazel/_bazel_ubuntu/ad1e09741bb4109fbc70ef8216b59ee2/external/bazel_tools/tools/build_defs/repo/git.bzl:195:33: in
DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1556410077 -0400"
DEBUG: Repository io_bazel_rules_docker instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule git_repository defined at:
  /home/ubuntu/.cache/bazel/_bazel_ubuntu/ad1e09741bb4109fbc70ef8216b59ee2/external/bazel_tools/tools/build_defs/repo/git.bzl:195:33: in
WARNING: Download from https://mirror.bazel.build/github.com/aws/aws-sdk-cpp/archive/1.7.336.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (171 packages loaded, 26394 targets configured).
INFO: Found 1 target...
ERROR: /home/ubuntu/tensorflow/tensorflow/core/kernels/BUILD:4689:18: C++ compilation of rule '//tensorflow/core/kernels:sparse_split_op' failed (Exit 1)
In file included from ./tensorflow/core/framework/op_kernel.h:35:0,
                 from tensorflow/core/kernels/sparse_split_op.cc:19:
tensorflow/core/kernels/sparse_split_op.cc: In member function 'void tensorflow::SparseSplitOp::Compute(tensorflow::OpKernelContext*)':
tensorflow/core/kernels/sparse_split_op.cc:71:34: error: 'class tensorflow::TensorShape' has no member named 'AddDimWithStatus'
                      dense_shape.AddDimWithStatus(input_shape_flat(i)));
                                  ^
./tensorflow/core/framework/op_requires.h:52:29: note: in definition of macro 'OP_REQUIRES_OK'
     ::tensorflow::Status _s(__VA_ARGS__);                    \
                             ^~~~~~~~~~~
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 46060.242s, Critical Path: 750.27s
INFO: 9369 processes: 9369 local.
FAILED: Build did NOT complete successfully

@hassan-shehawy
This issue is already reported and tracked at #49850.


Headless camera

Versions

Python: 3.7
OS: Fedora 29
Kivy: master and pipy (1.10.1)
Kivy installation method: pip

Description
I am trying to run the camera in a standalone mode/headless without GUI, so it can run as a Service. @tshirtman already helped out (on discord) by setting wakelock on my application, which works, but it drains the battery, also I think this is a bug.
Here is a question on StackOverFlow with my failed tries, so I though the problem was because of the GUI, so I created my own camera class based on kivy.uix.camera.Camera which is the code below.
For some reason capture nor _camera_loaded are never called
Code and Logs
# Based on https://github.com/kivy/kivy/blob/master/kivy/uix/camera.py#L36
from time import sleep
from kivy.core.camera import Camera as CoreCamera
from kivy.event import EventDispatcher
import time
import PIL

class Camera(EventDispatcher):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._camera = None
        self.index = 0
        on_index = self._on_index
        fbind = self.fbind
        fbind('index', on_index)
        fbind('resolution', on_index)
        self.resolution = (640, 480)
        self.play = True
        on_index()

    def capture(self, instance):
        camera = instance
        texture = camera.texture
        image_data = texture.pixels
        size = texture.size
        fmt = texture.colorfmt.upper()
        timestr = time.strftime("%Y%m%d_%H%M%S")
        pil_image = PIL.Image.frombytes(mode=fmt, size=size, data=image_data)
        pil_image.save("IMG_{}.png".format(timestr))
        print("Captured")

    def _on_index(self, *largs):
        if self.index < 0:
            return
        if self.resolution[0] < 0 or self.resolution[1] < 0:
            return
        self._camera = CoreCamera(index=self.index, resolution=self.resolution, stopped=False)
        self._camera.bind(on_load=self._camera_loaded)
        if self.play:
            self._camera.start()
            self._camera.bind(on_texture=self.capture)

    def _camera_loaded(self, *largs):
        print("Camera Loaded")
        self.texture = self._camera.texture
        self.texture_size = list(self.texture.size)

    def on_play(self, instance, value):  # I can't find how this is handled on kivy.uix.camera.Camera
        if not self._camera:
            return
        if value:
            self._camera.start()
        else:
            self._camera.stop()

c = Camera()
c.play = True
print("Taking 5 seconds of photos...")
sleep(5)
print("Turning off taking photos")
c.play = False

Results in
(anti_robbery_drawer) [roberto@roberto anti_robbery_drawer]$ python kivy_service_osc/src/service.py
[INFO   ] [Logger      ] Record log in /home/roberto/.kivy/logs/kivy_19-04-24_0.txt
[INFO   ] [Kivy        ] v1.11.0.dev0, git-eec3a30, 20190419
[INFO   ] [Python      ] v3.7.3 (default, Mar 27 2019, 13:41:07)
[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)]
[INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_pil, img_gif (img_ffpyplayer ignored)
[INFO   ] [Camera      ] Provider: opencv(['camera_picamera', 'camera_gi'] ignored)
Taking 5 seconds of photos...
Turning off taking photos

Which should be:
(anti_robbery_drawer) [roberto@roberto anti_robbery_drawer]$ python kivy_service_osc/src/service.py
[INFO   ] [Logger      ] Record log in /home/roberto/.kivy/logs/kivy_19-04-24_0.txt
[INFO   ] [Kivy        ] v1.11.0.dev0, git-eec3a30, 20190419
[INFO   ] [Python      ] v3.7.3 (default, Mar 27 2019, 13:41:07)
[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)]
[INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_pil, img_gif (img_ffpyplayer ignored)
[INFO   ] [Camera      ] Provider: opencv(['camera_picamera', 'camera_gi'] ignored)
Camera Loaded  <-------- This print is missing
Taking 5 seconds of photos...
Captured <--------------------- 5 seconds of this printing
Captured
Captured
Captured
Turning off taking photos

Any other information you require, please ask

I forgot to tell about this script should be able to run as a Service in the future, and here is the link where this is being used.
https://github.com/robertpro/anti_robbery_drawer

Your setting c.play = True doesn't do anything because play is not a kivy property as you did not inherit from the Camera class.

Thanks @matham, did you take a look to the stackoverflow question, I did use Camera class right there: https://stackoverflow.com/questions/55794588/how-to-run-a-camera-kivy-app-in-headless-mode

It seems you're blocking the main kivy thread. You cannot run kivy in a "headless" mode as the gui always needs to run and the main thread cannot be blocked,

@matham I see yeah, that makes sense, thanks!


[BUG] Context menu long load when open file is empty

Which Glance version and IDEA version?
CodeGlance Pro 1.3.8
PhpStorm 2022.1.1 Build #PS-221.5591.58
Do you enabled reader mode option?
Don't know what this is or if it applies.
Could you show Glance config picture and minimap picture?

What steps will reproduce the issue?

Create a blank file and open it in the editor.
Right-click on a folder in the project panel. A loader shows for a few seconds before the context menu opens.
Left-click inside editor, then right-click. A loader shows for a few seconds before the context menu opens.
Disable CodeGlance or open a non-empty file. Context menu opens instantly in both cases above.

thanks report,please try this
CodeGlance Pro-1.3.9.zip

Yup it looks like that fixes it. Thanks for the quick patch!


[import] Proxy host cookie can lead to hit the wrong host

Let's say we import https://www.a.com/page.html page. The requested page via the proxy is: http://localhost:3001/page.html?host=https://www.a.com.
This will set the hlx-proxyhost cookie value to https://www.a.com so that resources referenced on the https://www.a.com/page.html page (like images, js, css...) will not need the host query parameter (if their url do not contain the host name). For example, an image with src="/image.png" will be served by the proxy from https://www.a.com without requiring the host query param because the cookie contains the host.
Now if during the import process we need some resources from another host, something like a json file https://www.b.com/sheet.json. We can leverage the proxy and request http://localhost:3001/sheet.json?host=https://www.b.com which will prevent the CORS issues.
But... this request will re-set the hlx-proxyhost cookie value to https://www.b.com and then corrupt the import process: the subsequent requests without the host query param will try to fetch from b.com and not a.com like described above.
To solve that problem, we need to change how the cookie is set or consumed:

we want the cookie to be set to a.com so that we do not have to re-write all resources url / href / src to append the host query parameter
if we need to hit a different host, we can specify the host query param to get around the cookie value but NOT override the cookie value

As soon as we start dealing with different hosts, I think we need https://github.com/adobe/helix-cli/issues/2072.
cc @mhaack

Can we set the cookie based on the URL of the imported page itself at the time you click import? With that it should be safe to not run into any race conditions.


Modding API

all of the below are currently possible, using internal Flamework apis, but I'd like to expose a proper API for doing all of the above.
requesting feedback on potential use cases and what people would generally want this API to include

custom lifecycle events
custom decorators
custom singletons

maybe not fully relevant to this, but can we have a way to add custom lifecycle events to components too? This might actually already be a thing and I just couldn't figure out how to do it lol

yep, that's not really possible currently..
the component system is separate from Flamework and components are created/removed arbitrarily, so the modding API will have to be able to account for that.

decorator metadata querying
Custom decorators may not always need all the metadata that Flamework provides, so they should be able to specify what will be generated at compile time
/**
  @Reflect design:paramtypes design:returntype guard:paramtypes guard:returntype
*/
declare const MyDecorator: Decorator;

https://github.com/rbxts-flamework/core/tree/modding-v2
https://github.com/rbxts-flamework/transformer/tree/modding-v2
https://github.com/rbxts-flamework/components/tree/modding-v2


Add German Localization

Description
This PR adds a German translation of JsonSchema.Net to make it easier for German speakers to understand error messages.
Links
None.
Checks

[x] I have read and understand the Code of Conduct and Contribution Guidelines.

@dotnet-policy-service agree company="Adliance"

Thanks for submitting this!


Consider removing collapsable DIVs

For the following reasons:

I'm not sure if these are Accessible.
It may not be obvious to those with vision that these can be clicked on
Expanding them won't add complexity

Great points! I originally included collapsible divs as way to reduce the text overwhelm on initial view of readme and was something I see used in GitHub Skills. However, as we have continued to refine the text it has gotten simplified and removing any need to collapse sections. Thanks for the suggestion!


[Query] True explanation behind the Guidance project?

Hello,
I am a computational linguist working on an exciting project regarding NLP and machine translation.
Custom logits processors and, thereby, constrained decoding are of high interest to me, and I have found Guidance's strategies to be the most successful, after testing many of my own.
However, I cannot find a formal publication, such as a research paper, that introduces Guidance, at least not publicly. In fact, as far as I understand, the project is nearly unknown by the mainstream media.
I would like to get to know the project better, especially regarding how it was initially built. Please, if you are associated with the developers and able, provide me with some documentation on the methodology and from where it derives. I am not the best at reading source code and understanding it immediately.
Thank you! :)

Hi @oflakne26,
Thank you so much for the kind words and your interest in guidance :). Sorry to make you search -- we haven't written up a formal publication yet, but plan to do so in the near future!
I can share some high level details of the project's history and how it works. Definitely feel free to ask more questions if there's anything I missed or if you'd like more detail.
The project started at Microsoft Research in ~October of 2022. A few of us were given early access to GPT-4, and started collaborating across the company to build the very first version of Copilot (back then, it was called Bing Chat or by its internal codename Sydney). We were all floored by the new capabilities of these models -- keep in mind that ChatGPT hadn't even been released yet -- and wrote about our research experiences in the Sparks of AGI paper: https://arxiv.org/abs/2303.12712
However, as amazed as we were by the potential, building real world applications on top of these models was incredibly painful (even more so back then when the instruction tuning was less powerful). Integrating LLMs into traditional software stacks was painful for a number of reasons:

Fundamentally, we couldn't guarantee that the models would output text in a way that downstream software could understand. This meant writing a ton of validation, healing, and retry logic which is painful and a bad user experience too.

On the flip side, we often wanted to add programmatic control flow to the way the models executed, i.e. conditional logic and loops on the outputs (e.g. if the model indicates it wants to do a web search, hook into the Bing Index).

It was messy to manage prompt templates and changing interfaces to model APIs.

LLM model inference was prohibitively expensive and had lots of room for optimization.

Being unable to control models makes them more susceptible to jailbreaks or unintentional responsible AI harms.

To address these challenges, we started building Guidance as an internal tool just for our own development. As we began to rely on it more and more, and discover new optimization opportunities or new features, we decided to open source it and share it with the community in May of 2023.
The first version of guidance was a DSL (https://en.wikipedia.org/wiki/Domain-specific_language) which used a handlebars-like template language syntax that looked something like this:
program = guidance("""
{{~#system~}}
You are a helpful assistant.
{{~function_def get_current_weather}}
{{~/system~}}

{{~#user~}}
Get the current weather in New York City.
{{~/user~}}

{{~#while True~}}
    {{~#assistant~}}
    {{gen 'answer' temperature=1.0 max_tokens=50 functions="auto"}}
    {{~/assistant~}}

    {{#if answer_function_call}}
        {{~#function~}}
        {{answer_function_call()}}
        {{~/function~}}
    {{else}}
        {{break}}
    {{/if}}
{{/while}}
""")

which was great in many ways (works across languages, the programs can be serialized safely as simple strings, etc.). However, it also 1) had a steep learning curve for the python community, 2) was painful to debug for end users and 3) was a pain to keep developing and maintaining, since any time we wanted to add a new language feature (eg while loops), we had to update the entire language itself.
We made the decision over the summer to switch to a more Pythonic interface, which is the current syntax you see today. We'll probably write up a blog at some point about the design decisions that went into this interface change too!
Ok, all that said, how does guidance work?
Fundamentally, guidance does three things today:

Make it easy to develop programmatic workflows around interacting with LMs, including prompting

Enable users to control model outputs to e.g. follow a regular expression or select from a list of items.

Optimize interactions with language models at inference time, with features like guidance acceleration to speed up inference and [token healing] (https://towardsdatascience.com/the-art-of-prompt-design-prompt-boundaries-and-token-healing-3b2448b0be38) to get higher quality model outputs.

We'll write up a proper academic paper about our approach at some point, but on a high level, the python programs users write get turned into a formal grammar (https://en.wikipedia.org/wiki/Formal_grammar) under the hood which is a formal way of specifying what is and isn't allowed to be generated by the model. These are used in e.g. programming languages to determine what is and isn't a valid program. Similarly, while users of guidance feel like they're writing just plain python, we help them formally specify exactly what the model is and isn't allowed to generate.
I'm sure you know all of this already @oflakne26 but wrote it out for the benefit of future readers)
Once we have a grammar, we use a modified Earley parser (https://en.wikipedia.org/wiki/Earley_parser) under the hood to build token masks. To generate text, language models are doing next token prediction - just determining what token is most likely to follow the text that they have already seen. To do this, they calculate a probability distribution over all possible tokens before sampling one, and then autoregressively continue the process. By "token masking", I mean that our parser manipulates this probability distribution, suppressing any tokens that aren't allowed by the grammar so that the model can only sample tokens that are valid.
We chose to use an Earley parser because it's among the most flexible parsing algorithms out there (which means users can write much more expressive constraints). This differentiates us from most other constrained decoding libraries out there, which use simpler parsers that can only handle e.g. regular grammars. Earley parsers come with tradeoffs -- they can be quite slow if implemented naively -- so we've taken great care to write a very high performance implementation (for details check out https://github.com/microsoft/llguidance).
I talk about these concepts in more detail with a lot of heavy visuals in my //build talk from earlier this year :) https://www.youtube.com/watch?v=qXMNPVVlCMs
This post is getting quite long, but hope it gives you some insight! Happy to answer any further questions you have, and thanks again for your excitement and interest :)


The requireSnapshots fails only the first time (retries runMode >= 1)

Hello,
I'm trying to use the --env requireSnapshots=true parameter. However, if I have the Cypress options set to retries runMode:2, then only the first run fails, and subsequent attempts create the snapshots.
pnpm cypress run --browser electron --spec "cypress/tests/visual/recordView.visual.cy.ts" --env requireSnapshots=true

Cypress config:
retries: {
runMode: 2,
},

The same behavior is also for 1 retry, seems that this functionality works only for the first run

Will be fixed in #13

Released in 8.0.0 - https://github.com/simonsmith/cypress-image-snapshot/releases/tag/8.0.0


Scrapegraph returns relative path URLs instead of absolute path Possible Bug?

Describe the bug
When using gpt4o as the llm and scraping a webpage to return a list of links, sometimes the paths returned are :

relative paths (OR)
full path with an incorrect prefix/domain usually "http://example.com"

The behaviour was consistent until 3 days ago i.e. it always returned full paths on a large dataset as well. Since then, I had to uninstall Scrapegraph and reinstall the library and that's when this issue started popping up.
Expected behavior
For example : asking to scrape a website www.some-actual-website.com and return a list of webpages that contain information about the contact details of the company, used to consistently/always return a json like :
{"list_of_urls": "['www.some-actual-website.com/about','www.some-actual-website.com/contact-us']"}

However, now I get either :
{"list_of_urls": "['https://example.com/about', 'https://example.com/contact-us']"}

OR
{"list_of_urls": "['/about','/contact-us']"}

I'm curious , shouldn't the list of URLs being parsed/scraped be a straightforward output? Is the final output always produced by the LLM?
Desktop (please complete the following information):

Ubuntu 22.04
Chromium Browser with Playwright

I think for your case, returned HTML content from the fetch node doesn't contain any link and LLM is making up some random links that are similar to the prompted ones. Did you try disabling headless in graph config?
graph_config = {     ...     "verbose": True,     "headless": False, }

@ekinsenler Tried this with "headless": False and Xvfb and still yields the same results.
Is this anything to do with SmartScraperGraph which scrapes the source url only and instead i should use DeepScraperGraph which can go further? Or am i completely off? 🤔

SmartScraperGraph doesn't use URL search inside the HTML content as far as I know. For that purpose, I am using DeepScraperGraph. But you need to implement a filter on some level inside the search_link_node to prevent unrelated links from getting fetched.
I am also working on a similar project. DeepScraperGraph doesn't have a parameter to control the max_depth, therefore scraping doesn't terminate even with some filters such as filtering out URLs that are outside of the domain name or filtering out different language versions of the website.

@ekinsenler Hmm... I'm curious - Is this not a functional example yet? There seems to be a max_depth parameter. However, I am facing errors using this Scraper https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/2333b513aafae3c358225a8f82f6c01964c0514e/examples/openai/deep_scraper_openai.py

I don't think max_depth is functional yet. DeepScraperGraph is also buggy as it requires you to use an embedder_model that seems not functioning inside the code. I fixed the error on my local and run the graph but it gets lost inside the URL tree potentially looping between the same URLs.

You're right I had the same issue. I suppose there's not much I can do now other than wait for the developers to push DeepScraperGraph to release? I would be really interested if you get this working on your local/branch :)

I am waiting on a confirmation if devs are already working on the fix of this issue or else I am going to create a pull request.

@sandeepchittilla @ekinsenler  I can confirm that the DeepScraper is broken, and your issues are caused by a problem on our side - I'll give you a better explanation in a few days, and then see what can be done to fix the problem.

Let's get back to this.
Basically, a "deep scraper" with crawling capabilities has been on the roadmap for a while. We had a contributor make a system to implement it, but it was very heavy and slow. It didn't check for loops, it used a SmartScraperGraph instance on every page, and more importantly, it introduced a signal-based approach that was hard to parallelize and required significant modifications to the existing graph engine. Therefore, this design was rejected.
Around that time, part of the team started working on a proper deep scraper, with a more modular desing that fit better within the exsiting framework. See #260 for more information. The work was left undone, due to shifting priorities. The SearchLinkNode, for example, was intended to be a piece of the DeepScraperGraph pipeline.
The examples for the DeepScraperGraph are still based on deprecated code that relies on a RAG-based approach. This was removed around a month ago. We're currently focusing on cleaning up and refactoring our baseline, so most dead code will soon be removed, along with broken examples. Hopefully we'll get back to developing the deep scraper, as it would be a killer feature for the library. It's also hard to implement, though, and there's already much work on our plates.
Thanks for taking interest in this library, and in this topic in particular. We'll leave this issue open for now.

In my Pydantic model, I add a description which says to use the absolute URLs and it works.
class Content(BaseModel):
    url: str = Field(description="The absolute URL of the content")
    title: str

@f-aguzzi thank you for taking the time out to respond and explaining the context! Understood!
@datashaman thanks for the response. Just to be clear, you request a response in your pydantic class format (prob. JSON?) and sent this as part of the prompt; where you specify the description for the field, yes?

@sandeepchittilla correct, the field description hints to the LLM that it should use absolute URLs in the response.
this was using the smart graph (not deep) with gpt-4o-mini as the LLM.


Azure Pipelines build failed: NuGet restore failed

The nuget command failed with exit code(1) and error(Errors in packages.config projects
Unable to find version '2.1.190405001-prerelease' of package 'Microsoft.UI.Xaml'.
https://api.nuget.org/v3/index.json: Package 'Microsoft.UI.Xaml.2.1.190405001-prerelease' is not found on source 'https://api.nuget.org/v3/index.json'.
Unable to find version '10.30.180808002' of package 'Taef.Redist.Wlk'.
https://api.nuget.org/v3/index.json: Package 'Taef.Redist.Wlk.10.30.180808002' is not found on source 'https://api.nuget.org/v3/index.json'.)
Packages failed to restore

Package in "./dep/packages"

@wolking Thanks


Creating freeCodeCamp's CSS Style guide

I would like to propose freeCodeCamp's CSS Style guide to improve overall maintainability of the codebase. As the FreeCodeCamp has a lot of contributors, and everyone writes the CSS in a different way. And it is hard to keep of track of different ways of writing CSS.
Initial Recommendations:
Comments

Add comments, when you are starting a new LESS file, so the developers know what are they working with, also add comments when you start new section which is related the code block, it helps with the readability, see EXAMPLES below (I have encountered these style of comments across the code base, therefore I would like to continue writing it, instead of proposing a new style)

Example 1 - when starting a new LESS file:

//
// Navigation
// --------------------

Example 2 - when adding a section, related to navigation

//
// Navigation - Mobile Styles
// ------------------------------

Spacings

Add spaces after a property, it will be much easier to read such a codebase, and maintain. See EXAMPLES below.

Example 1 - Correct Way

p {
  color: #fff;
}

Example 2 - Incorrect Way
What is wrong?
No space after a colon

p {
  color:#fff;
}

Colors

Prefer lowercase, instead of uppercase. Now across the CSS, contributors are mixing both. Lower case is faster to write, easier to read. Again improves overall maintainability, if the CSS has consistent convention. Also if possible shorten the hex value. See EXAMPLES below.

Example 1 - Correct Way

p {
  color: #fff;
}

Example 2 - Incorrect Way
What is wrong?
upper-cased hex value, not shortened

p {
  color: #FFFFFF;
}

RGBA or RGB colors

Add spaces after values, improves readability. See EXAMPLES below:

EXAMPLE 1 - Correct Way

p {
  color: rgba(123, 123, 0, 0.1);
}

EXAMPLE 2 - Incorrect Way
What is wrong?
No spaces after values

p {
  color: rgba(123,123,0,0.1);
}

##No unit when declaring 0

In property such as box-shadow, if you set a 0 value, then you should avoid adding the unit. See EXAMPLE below.

EXAMPLE 1 - Correct Way

p {
  box-shadow: 1px 1px 0 rgb(0, 0, 0);
}

EXAMPLE 2 - Incorrect Way
What is wrong?
Units after 0

p {
  box-shadow: 1px 1px 0px rgb(0, 0, 0);
}

Avoid floating numbers

If possible you should avoid floating numbers in CSS for different sizing, as it could render a bit differently. See link attached below.

Browser Rounding

@tomasvn We just launched our new curriculum and learning platform. Are you still interested in helping with this?
All of our CSS is in need of a good refactor, and this would be a good opportunity to not only create a style guide but to turn around and immediately apply it to the project, and make sure all our repos are in accordance with it.

@QuincyLarson Sure, I can take some of my time and help you with refactoring the CSS, as the freecodecamp is quite a large codebase

@tomasvn That would be amazing!
freeCodeCamp doesn't have all that much CSS. freeCodeCamp's CSS is mainly located in "main.less". And we're using Bootstrap 3.0.
We may eventually move to Bootstrap 4.0 or just rewrite our CSS to use Flexbox. But for the time being, we're using React Bootstrap on several of our applications, and that is currently locked to Bootstrap 3.0. So I recommend just going in and cleaning up the existing custom CSS in main.less and making it less redundant and more logically arranged.

@QuincyLarson got it

@tomasvn Thanks for your patience - I just saw this. Yes - client/less/main.less should be the right file.

@tomasvn Need help with this? I'll be happy to contribute

@tomasvn have you had a chance to start working on this yet? If not, @borisyordanov Yes - we would welcome your contributions with this.

@borisyordanov sorry for delay, sure go ahead I have some problems running dev env on windows, so you can start, when I fix my stuff I can help you refactor it

If there is still need I can help with this

@Jgriebel1990 i couldn't get in touch with tomasvn so no progress has been done as far as i know. Feel free to take over

@jgriebel1990 feel free to take over, i wasnt able to get my copy working on windows machine, all i can do is a code review once you are done

@tomasvn it sounds like @Jgriebel1990 and @borisyordanov haven't gotten a chance to tackle this - are you still interested in helping with this?

Related: https://github.com/freeCodeCamp/freeCodeCamp/issues/35081

@QuincyLarson No luck, running it locally

@tomasvn what issues did you come across when setting up locally

@tomasvn Please read this - https://github.com/freeCodeCamp/freeCodeCamp/blob/master/docs/how-to-setup-freecodecamp-locally.md

@raisedadead With the updates to https://github.com/freeCodeCamp/design-style-guide, is this issue needed?

Its more on the roadmap. Eventually it will be the central resource for our UI components.

CSS Border Style
The border-style property specifies what kind of border to display.
The following values are allowed:
dotted - Defines a dotted border
dashed - Defines a dashed border
solid - Defines a solid border
double - Defines a double border
groove - Defines a 3D grooved border. The effect depends on the border-color value
ridge - Defines a 3D ridged border. The effect depends on the border-color value
inset - Defines a 3D inset border. The effect depends on the border-color value
outset - Defines a 3D outset border. The effect depends on the border-color value
none - Defines no border
hidden - Defines a hidden border
The border-style property can have from one to four values (for the top border, right border, bottom border, and the left border).
EXAMPLE-
p.dotted {border-style: dotted;} p.dashed {border-style: dashed;} p.solid {border-style: solid;} p.double {border-style: double;} p.groove {border-style: groove;} p.ridge {border-style: ridge;} p.inset {border-style: inset;} p.outset {border-style: outset;} p.none {border-style: none;} p.hidden {border-style: hidden;} p.mix {border-style: dotted dashed solid double;}

Hey! What's the progress on this? I am available to help!

@ManasMahanand, thank you for your interest.
It would be great if you could look into the available css linters, compare them and see if there is anything better than stylelint.
Then, add the linter to our repo and  change the CSS files to make sure  the repo is linting error free.

@ahmadabdolsaheb Will do!

Wait what do you mean add to repo?
Do you want me to install the linter as a dependency, like eslint is already?
Or is there a way of installing tests on github itself, so it won't allow merging unless the style guide is followed?

Ok, looks like stylelint is a good option itself. I am installing that.
For experimenting, I will set rules as suggested by OP. I will then remove all rules and open a pr. We can then discuss and set rules

https://stylelint.io/user-guide/rules/about
You can have a look at this link to see the types of rules that can be set

currently there is already a lint:css command that uses prettier. We can change it to use stylelint.
Any how, I have installed it, and it successfully detects errors based on rules set. Shall I open a PR?

No need to remove prettier, prettier has config for stylelint rules that you can use to extend prettier config file

Awesome. Will work on it tomorrow.

I didn't find prettier config for stylelint rules (or how to do that), but currently I have installed another package stylelint-config-prettier that will check for conflicts and will turn off any rules that does conflict.
What I am thinking is in lint:css command we can also run the stylelint command as well along with prettier

(Opinion) I feel that we can write all custom stylelint rules based on what makes sense for fcc, and even completely remove prettier. This is just an opinion, it is left to you guys to decide.

Thanks @ManasMahanand on your progress. I am tagging @ojeytonwilliams for a second opinion.

I think prettier does a good job with formatting, so I'd rather keep that.  If we use https://github.com/prettier/stylelint-prettier then prettier is used as a plugin of stylelint and we get the best of both worlds.

@ManasMahanand I was wondering about your progress with your solution. let me know if I could be of any help.

Since we are moving away from custom css and leaning towards our component libraries's tailwind for most of our styling, adding CSS style guide would not be necessary; However, I added a note to add a CSS linter to our component library which should address this issue.
I will go ahead and close this issue accordingly. Let me know if this issue has been closed by mistake or if it needs further discussion. Thank you.


Added throttle to turn_on/off methods

Description:
When calling the backend lib/device too fast I got an error (-6) that the device was unavailable.
By throttling the calls everything works as expected.

[x] Local tests with tox run successfully.
[ ] TravisCI does not fail. Your PR cannot be merged unless CI is green!
[x] Fork is up to date and was rebased on the dev branch before creating the PR.
[x] Commits have been squashed.

By applying throttle we're dropping calls to turn the lights on/off. Maybe it means that you inserted a too high value for signal_repetition? How come you're calling this multiple times to begin with?

Have the default value of 1 for signal repetition. The problem occurs when
I ran homeassistant.turn_off on group.all_switches and group.all_lights.
Maybe it's better to do some error handling when the exception is thrown?
Any suggestions on how to proceed with that?
Den lör 5 mars 2016 17:20Paulus Schoutsen<EMAIL_ADDRESS>skrev:

By applying throttle we're dropping calls to turn the lights on/off. Maybe
it means that you inserted a too high value for signal_repetition? How
come you're calling this multiple times to begin with?
—
Reply to this email directly or view it on GitHub
https://github.com/balloob/home-assistant/pull/1477#issuecomment-192680816
.

I am surprised that this would solve it when the problem occurs when operating on a group. It should call the method on each object only once.

My guess was that it was being executed in papralell somewhere, probably
the c lib was returning control before it was done. And its strange that I
started to work, but it could have been fluke because it's not always
occurring! Would adding a new try 0.5 sec later be an acceptable solution
(for some max tires times) when handling this specific exception?
Den lör 5 mars 2016 17:31Paulus Schoutsen<EMAIL_ADDRESS>skrev:

I am surprised that this would solve it when the problem occurs when
operating on a group. It should call the method on each object only once.
—
Reply to this email directly or view it on GitHub
https://github.com/balloob/home-assistant/pull/1477#issuecomment-192681436
.

Sleeps are never the solution. What about locks?

Well that would only work if the concurrency problem is in the python side!
But it might be, I'll start some testing!
Den lör 5 mars 2016 18:53Paulus Schoutsen<EMAIL_ADDRESS>skrev:

Sleeps are never the solution. What about locks?
—
Reply to this email directly or view it on GitHub
https://github.com/balloob/home-assistant/pull/1477#issuecomment-192698224
.

We can use a lock to avoid calling into C in parallel.

Will send a new PR with correct locking instead.


Integrate with Heptio's Ark

@kris-nova to come up with a plan and flesh out implementation details

Take a snapshot.

Create a cluster using Kubicorn.
Deploy Applications.
Take a snapshot using Kubicorn.
a. Take a snapshot of the cluster using GetActual.
b. Take a snapshot of the application using Ark.
c. Push to state store.

Edit a snapshot

kubicorn edit
a. Read the snapshot from the state store.
b. Write the snapshot to the local fs.
c. Open the file in user's text editor.
d. Save the temp file.
e. Write to state store.

Restore a snapshot

kubicorn apply --snapshot <name>
a. Apply cluster.
b. Ark restore.


write_fixed_all

Define write_all functions that take fixed buffers.
Adds write_fixed_all for TCP and Unix streams.
Adds write_fixed_all_at for File.
Also adds an example: tcp_listener_fixed_buffers that uses read_fixed and write_fixed_all.

@FrankReh This should also be applicable to udp and (I think) also the zero copy variants

@ollie-etl Turns out the UdpSocket can take a write, as it can be connected separately first, so even write_fixed should be defined for the UdpSocket. Just realized that today.

@ollie-etl Actually, could you clarify what you meant? An _all option doesn't seem necessary for UDP sockets as the writes have to be all or nothing anyway.

@FrankReh I meant write_fixed, rather than write_fixed_all.
An explosion in number of methods is occurring. I think the case for a builder API overhaul is growing stronger

And realised my initial comment was irrelevant. Please disregard


[design] either readthedocs style or bootstrap style

Bootstrap style is maybe nicer, need to figure out floats.
Otherwise fixed left menu is probably fine, appearing right toc is also probably fine
use em for geometry

https://example.readthedocs.io/en/latest/
https://getbootstrap.com/docs/5.0/getting-started/introduction/

if not too much of a hassle, probably centered is nice


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v4.3 to before v5.4.155",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=69aba7948cbe53f2f1827e84e9dd0ae470a5072e",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=0594f1d048d8dc338eb9a240021b1d00ae1eb082"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "69aba7948cbe53f2f1827e84e9dd0ae470a5072e",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "0594f1d048d8dc338eb9a240021b1d00ae1eb082",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "nvmem: Fix shift-out-of-bound (UBSAN) with byte size cells\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.4.155 by commit 0594f1d048d8dc338eb9a240021b1d00ae1eb082, it was introduced in version v4.3 by commit 69aba7948cbe53f2f1827e84e9dd0ae470a5072e. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001998


In Tobira, videos with a resolution of 0p exists

Cf. https://tobira.opencast.org/speakers/cortona-week/v/AIFgWQbMG9y.

https://github.com/elan-ev/tobira/issues/1242


Better text descriptions for test scenarios

The generated test contains some narative information like this

However this information is missing in the generate summary https://github.com/heathfrankel/inferno-au-core/blob/main/lib/au_core_test_kit/generated/v0.3.0/summary.md
Text narrative should be added to the summary.

Hello @ir4y
What do you think about the updated summary?
https://github.com/heathfrankel/inferno-au-core/blob/main/lib/au_core_test_kit/generated/v0.3.0/summary.md


obs: Add ubuntu 19.04 testing

Now that we have obs packages for ubuntu 19.04, we should add it in the
testing script.
Fixes #884
Signed-off-by: Gabriela Cervantes<EMAIL_ADDRESS>

/AzurePipelines run

/test


Add ellipse primitive

Similar to Circle, defined by top left as per #274, but with a width/height given as a Size. E.g.:
// Create an ellipse 10px wide, 20px tall
let ellipse = Ellipse::new(Point::zero(), Size::new(10, 20));

// Create a circle 20px in diameter
let ellipse = Ellipse::new(Point::zero(), Size::new(20, 20));

Questions:

[ ] Should Circle become a simple wrapping struct to Ellipse, or should it be removed entirely? IMO Circle should be kept in.

Should Circle become a simple wrapping struct to Ellipse, or should it be removed entirely? IMO Circle should be kept in.

I think I agree with that.

Cool cool. Original comment updated.

While this definition of an Ellipse covers pretty much all the common use cases, it should be noted that it doesn't allow drawing ellipses with their major and minor axis not horizontal and vertical.
I don't know if this case is worth supporting, but it is at least good to clearly know the limitations of the chosen definition.


Upgrade to ledger-go 1.0.0 to fix support for Ledger Nano S+

Context
https://github.com/osmosis-labs/osmosis/pull/8872
Problem
There may be an issue with the latest Ledger software.
Proposal

[x] Upgrade to the latest Ledger Live application. Upgraded to Ledger Live 2.92.1
[x] Upgrade to latest Ledger firmware. Upgraded to Ledger Nano X - OS Version 2.4.0 (previously on 2.2.3)
[x] Test if it's still possible to use the Cosmos app and send TIA. Confirmed possible: https://www.mintscan.io/celestia-testnet/tx/9CA7A44F1C199383CCB5C2B1AD3285AAE4CA232647F9AC22C6EB92F236622EC1?height=3577088

We use https://github.com/celestiaorg/celestia-app/blob/818026ac446570605d3db582f2d261fb392e1973/go.mod#L258-L260 and the linked PR bumps https://github.com/zondax/ledger-go which we only depend on indirectly https://github.com/celestiaorg/celestia-app/blob/818026ac446570605d3db582f2d261fb392e1973/go.mod#L218

Based on this comment the issue may only impact Ledger Nano S+.
Inclined to wait for https://github.com/cosmos/ledger-cosmos-go to cut a new release that bumps their dependency of ledger-go to v1.0.0 so will monitor https://github.com/cosmos/ledger-cosmos-go/pull/70

I agree, I'm hoping for a bump in the ledger-cosmos-go app as well, and bump that

https://github.com/cosmos/ledger-cosmos-go/releases/tag/v0.14.0 was released an hour ago so we can bump to that here.


Checkbox in header

I want to add a checkbox in the v-pane-header without toggling the accordion when i click on the checkbox, How can i make that done !

By CSS z-index ;)

This didnt work for me. Could you provide a sample?

I fixed it finally :) Now you can add inactive attribute on v-pane-header element and create your custom toggle button, like so:
<v-accordion multiple>
  <v-pane ng-repeat="pane in panes">
    <v-pane-header inactive>
      {{ ::pane.header }}
      <button ng-click="$pane.toggle()">Toggle me</button>
    </v-pane-header>

    <v-pane-content>
      {{ ::pane.content }}
    </v-pane-content>
  </v-pane>
</v-accordion>


Added new CDN: PageCDN

Nice marketing move, but I don't think that's proper etiquette in open source community

Hi @praisedpk,
I would not mind linking your service, if it were appended to the list, rather than trying to shove the order...

Hi @cherniavskii Sorry you found my behavior inappropriate. I sent PRs to 10-15 libraries only that I thought could benefit the most from PageCDN due to the big size of the library and the better compression PageCDN offers.
@ghybs PageCDN uses brotli-11 compression level for resources hosted on Public CDN that makes leaflet.js around 4 kb smaller than the one hosted on other open-source CDNs. I see this as good for developers that use Leaflet, to have PageCDN as an option in Leaflet docs. The existing links are in no particular order, so I put p before u. But I think I made a wrong choice here.
I would thank you guys if you give PageCDN another consideration, since atleast there is some added benefit for developers without any cost.
Thanks.


WIP: linting, unit tests & pre-commit

This PR brings a light pre-commit configuration for development and runs both linters and unit tests for all commits. Should be merged only when everything passes.

Are these errors possibly caused by the usage of go 1.18? If so, fixes to this repo can be applicable to cluster-service-upcloud linting issues

Thanks for review @ajmyyra !


Solve piracies and ambiguities

Part of https://github.com/QuantumSavory/QuantumSavory.jl/pull/143

[x] The code is properly formatted and commented.

This should be okay I think

Do you mind adding a comment to the code explaining why these help or which ambiguity they address? I have to admit, looking at them, I am bewildered, and I will be pretty lost a year from know trying to understand why they were added.

Actually I tested everything again, and that part was unnecessary

I think this can be merged, let me know if it is okay @Krastanov

If you could release a new version of QuantumClifford now that this is merged it will be much appreciated because it will help testing the main pr

Certainly! I should have done that, my bad!


Add version: Google.AndroidStudio version <IP_ADDRESS>

Same as https://github.com/microsoft/winget-pkgs/pull/78868 but in two PRs

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.2 schema?

Microsoft Reviewers: Open in CodeFlow

Will need to be re-ran once https://github.com/microsoft/winget-pkgs/pull/78870 is accepted.

Needs re-running

@wingebot run

@wingetbot run

@wingetbot waivers Add Validation-Domain


Move search bar to header and make it responsive

@Neesh774 search 🔎 is not working and see screenshot

Fixed @danishzayan

Sorry, fixed search and fixed merge conflicts

@Neesh774 Gud work.. 😊 don't forget give to repo...


etcdmain: print out error and suggestion for fixing notify issue

/cc @eyakubovich @robszumski Please help to review :) Thanks!

LGTM


Offer a way to order some articles within a section

Some articles are related and flow better when in a certain order.
Some are on the same specific topic (e.g. Leyden), some on the general area (e.g. kotlin).
We tend to order them in the shownotes but the bot has no notion of ordering.
This is a generally hard UX problem. So finding the minimum viable solution would be best vs building something too complicated.
Agreed
Find a deterministic algorithm so that for two given runs of generate, the articles come in the same order.
I understand that technically the Slack API might not guarantee order, but if it does in practice that's good for me.
Proposition 1
Unless specified explicitly, articles are listed in the order they were pushed in the Slack channel
Proposition 2
Define a set of reactions lcc_1 to lcc_9.
For a given section, articles marked by lcc_n are ordered according to the value of n.
Articles without lcc_n reaction are listed after the ones with such reactions.
Conclusions
With these simple rules, I hope we can get a working solution that's simple enough to use.

The first proposition is already implemented. The messages are explicitly written in the show notes using a "chronological" order (order by the ts field, null last - ts is a unique, per-channel, timestamp - see also https://github.com/slackhq/slack-api-docs/issues/7#issuecomment-67913241).

Done. I will also add a way to sort replies of an articles.


socket timeout problems...

Hi Jonathan: We are using asyngular-server with socketCluster Android and js clients
We get a lot of socket timeout errors on server and I am not sure how to handle it:
{ SocketProtocolError: Did not receive #handshake from client before timeout at AGServerSocket._onClose (/usr/src/app/node_modules/asyngular-server/serversocket.js:956:17) at AGServerSocket.disconnect (/usr/src/app/node_modules/asyngular-server/serversocket.js:971:10) at AGServerSocket._handleHandshakeTimeout (/usr/src/app/node_modules/asyngular-server/serversocket.js:350:8) at Timeout.AGServerSocket._handshakeTimeoutRef.setTimeout (/usr/src/app/node_modules/asyngular-server/serversocket.js:120:10) at ontimeout (timers.js:436:11) at tryOnTimeout (timers.js:300:5) at listOnTimeout (timers.js:263:5) at Timer.processTimers (timers.js:223:10)

Client
We use protocol version 1 and the default socketCluster clients (Android and js).

How do I prevent this timeout error? is it a problem ?
I do not see where the socket timeout interval is setup right now in server.js when we deploy
to Kubernetes ?
and it seems maybe Android socketCluster client does not send pings by default ? we need to set it ?
in socketCluster js client, it seems by default it sends pings every 8 to 10 seconds ? I see #1 and #2 in chrome console in the messages for socket connection ? is there a settings for that? see below

Screenshot

@hervejegou There is no need to set the pingTimeout on the client side. You just set it on the server side and the client will use that same timeout. See https://asyngular.io/docs/api-ag-server/
The error you've posted can happen if the client is taking a long time to perform the handshake during the connections phase. It's not usually related to ping/pong (heartbeat) timeout (that seems to be working fine).
Is your socket subscribed to a lot of channels? Maybe it's trying to resubscribe to them all at once?
Or maybe you have a handshake middleware on the server which is holding up the connection for too long?
Or maybe your handshakeTimeout is too long? You can pass it as an option when attaching/creating the server. Default is 10000 (10 seconds).

@jondubois  Thank you.. I will experiment with handshakeTimeout and check on the android client if we have some delays during connection phase.


Make most assert! debug_assert!

-C debug-assertions can be passed to the compiler to have debug_assert! executed in optimized builds.

Closing as the resulting performance gain should be negligible.


Repeatedly shows error page

Hi Christian.
I am trying to customize your app and I'm getting redirected to error.html.erb repeatedly.
It says I need to purchase Advanced Shopify plan on the store but that's what I did a few weeks ago.
I deployed your app on my heroku account and installed it on my store(advanced plan).
Please let me know if anything needs to be configured for your app use.
Thanks,
Oliviu

@oliviu-rockstar  I'm in the same boat, and I'm on a Shopify Plus store.  Deployed to Heroku, ran migrations, created the app under a Shopify Partner account.
Did you ever find out what the issue was?

@oliviu-rockstar  Make sure you setup environment variables for APPLICATION_URL, SHOPIFY_CLIENT_API_KEY, and SHOPIFY_CLIENT_API_SECRET.  Then the app will work just fine.

If you want to submit a pull request to update the readme file with that information, I'll gladly merge it in.


Lock beamline control so that only one Python process can caput.

This needs to be a token that the run engine grabs when it starts and returns when it returns to idle.

@stuwilks: "Do not lock it in /tmp. Do it with a PV."

This could be accomplished using a collection of suspenders -- one per RunEngine instance --that are controlled by some hub that only allows one suspender to be un-tripped at time.
Incidentally, I believe we've already established that there is no way to keep some rogue RunEngine or some rogue caput from ignoring this constraint because ultimately EPICS does not care. This system would assume that all RunEngines are friendly RunEngines that install the necessary suspender.
Or, as others have said, there could be only one Special Process that is allowed to talk to the EPICS network.
I'm not aware that this is a priority for anyone; I'm just dropping the suggestion for Future Someone.

In theory, one could secure Channel Access connections the same way we secure HTTP connections. HTTP doesn't have a concept of security either...
https://h11.readthedocs.io/en/latest/basic-usage.html

So this is another good idea, we have 2 endstations with seperate control computers so being bale to lock one out but not the other would be very useful. However, we would want to be able to define which motors are locked and which aren't in the setup files.
In this way it combines nicely with the issue we have discussed regarding being able to define which 'plans' or RE processes are stopped by the suspenders and which are not.

I wonder if this is a use for channel access security. It is fairly easy to setup (just a bit laborious as you need to put it on all IOCs) but it allows you to control which user or host can read or write PVs. We have implemented this to control EPUs etc so that one branch cannot inadvertently operate the others gap. Same for endstation controls. If you like @awalter-bnl I could show you how to set it up (with John). @danielballan what do you think, this might be a more appropriate layer?

channel access security enables you to lock down read/write access to
pv's on a per host and/or user basis, but it can't distinguish between
multiple processes running under the same account on the same machine.
It's not real security, simple spoofing techniques easily work around it;
it's really just a layer of protection against inadvertent access.
That might acheive what you need?
but it won't lock it down to a single python process.
-matt

So we have different user accounts for the 2 endstations so this would probably resolve that issue.
we could just set the ARPES endstation motors to only be writeable from the ARPES endstation account. However when we swap from 1 branch to another it would be great to temporarily lockout PV access to the beamline PV's for the opposite branch account. We have a python plan that does the transfer, is it possible to have that change the read/write permissions?

Yes @cowanml I was not suggesting this as security - apart from the fact that it is called access security.
http://www.aps.anl.gov/epics/base/R3-14/12-docs/AppDevGuide/node9.html
If you look at what Andrew wants to do (as I understand it @awalter-bnl) it is making sure that when one branch is operating it can pass over the con to the other branch so users setting up on the other endstation can't advertently  change the optics etc.
If this is the case then this could be done by just using the IP addresses of the two control computers. There would then be a PV which would allow the con  to be passed between the two endstations. You could even only make the restriction to the xf??id? account so that a beamline scientist could always log into the computers (or do su - uname) to be able to write to PVs.
So, not security but just protecting from inadvertent control. I implemented this at CSX-{1,2} for exactly this reason and it works well.

@awalter-bnl if you have two distinct endstation computers you don't need (but could have) different accounts. I can stop by tomorrow and show you what I am describing here if you like.

@stuwilkins We have 2 accounts and 2 workstations so it sounds like your solution at CSX is exactly what we need. I am at the CFN retreat tomorrow but Friday or next week is fine to meet to discuss this implementation.

On Wed, 25 Oct 2017, Stuart Wilkins wrote:

Yes @cowanml I was not suggesting this as security - apart from the fact that it is
called access security.

didn't mean to imply you were, just felt compelled to add that
disclaimer since, as you point out, it's called "access security" and
we're discussing "locking" stuff...
...

So, not security but just protecting from inadvertent control. I implemented this at
CSX-{1,2} for exactly this reason and it works well.

yup, that's exactly what it's for and as long as there are distinct
computers (or accounts) involved it should solve the problem perfectly.
but you already knew that, so... in summary, "+1"
-matt

We can't solve this properly at the bluesky layer, because someone could always use another client (e.g. caput, CSStudio) to run around it. We could add locking anyway to the RunEngine, but this opens up a can of worms (managing what happens to a lock when a process crashes, etc.) without really solving the problem thoroughly.
Locking needs to solved per-Device and at a different layer. This is something bluesky project might someday address, but not in the bluesky library, so we are closing this issue.

Commenting further, EPICS Channel Access Security is used to provide some granularity to which EPICS clients (such as bluesky) can access (read and/or write) PVs.  Details of that security layer relevant to this issue are presented in section 3.3 Restrict Channel Access Requests to "Authorized" Clients.
IIRC, the available security restrictions are limited to user ID and IP address but not to specific client processes at that IP address.
I agree this issue should be closed.  The original concern would be advised to better identify the problem to be solved.


Fix uninitialized values in m2mreporthandler

Fixes following error

IOTCLT-285 - Valgrind error when using observe and send PUT REST request

@yogpan01 can you check this?

+1


Azure privatelink integration is impossible

Describe the bug
We are not able to use Azure keyvault privatelink integration due to hardcoded values in Azure backend, so we are able only to provide the name of keyvault so the path is different than privatelink endpoint
To Reproduce
kvpath = fmt.Sprintf("https://%s.vault.azure.net", kvpath)
Expected behavior
Be able to use privatelink endpoint for keyvault - .privatelink.vaultcore.azure.net
Screenshots/Verbose output

So i guess we would just need to accept an environment variable and override if provided. Feel free to put up a PR and we can review if this is a feature you need. Thanks!


ARIMA Forecasting

Writing an ARIMA model from scratch (plus an optimizer) seems to0 difficult - could just require that R is installed on the system and make a wrapper for R's auto.arima model

Followed https://cran.r-project.org/bin/linux/ubuntu/
Made a new directory r/
Ran R and then at the R prompt ran install.packages("fpp3") and said yes and yes for installing to a private location.

Failed with lots of "System has not been booted with systemd as init system (PID 1). Can't operate."
Trying to follow https://support.rstudio.com/hc/en-us/articles/360049776974-Using-RStudio-Server-in-Windows-WSL2

Still same issue, trying to install r packages with apt instead of install.packages

Can't get R packages to install on wsl still, so just use R.exe from Windows for now

Actually, don't even need to make calls to R, there is a great C library for time series analysis that we can include instead https://github.com/rafat/ctsa

Added with ctsa https://github.com/ianfr/adsl-cpp/commit/92185ab5b7d2b959aca9cd80142dd74c465e0da1


Heroku online application

I have changed some files to manage the heroku integration and also added a short description to configure heroku step by step

You deserves the :gift: (I have modified the README.md). Anyway, I have detected something weird that you should fix before merging your pull request. You must add a whitespace after the header mark in markdown.
#Installing Gradle on Windows 10

should be
# Installing Gradle on Windows 10

Please review your description.md because this happen in may places. Use the preview mode in GitHub for checking it.

Ouch, my bad. I used a plugin for the preview of Intellij IDEA and it showed me well. Changed :)

Done!


solve cicd issue

Pull Request template
Please, go through these steps before you submit a PR.
Why is this PR required? What issue does it fix?:
What this PR does?:
Does this PR require any upgrade changes?:
If the changes in this PR are manually verified, list down the scenarios covered::
Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs
Checklist:

[ ] Fixes #
[ ] PR Title follows the convention of  <type>(<scope>): <subject>
[ ] Has the change log section been updated?
[ ] Commit has unit tests
[ ] Commit has integration tests
[ ] (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
[ ] (Optional) If documentation changes are required, which issue on https://github.com/openebs/openebs-docs is used to track them:

PLEASE REMOVE BELOW INFORMATION BEFORE SUBMITTING
The PR title message must follow convention:
<type>(<scope>): <subject>.
Where:

type is defining if release will be triggering after merging submitted changes, details in CONTRIBUTING.md.
Most common types are:

feat      - for new features, not a new feature for build script
fix       - for bug fixes or improvements, not a fix for build script
chore     - changes not related to production code
docs      - changes related to documentation
style     - formatting, missing semi colons, linting fix etc; no significant production code changes
test      - adding missing tests, refactoring tests; no production code change
refactor  - refactoring production code, eg. renaming a variable or function name, there should not be any significant production code changes

scope is a single word that best describes where the changes fit.
Most common scopes are like:

data engine (localpv, jiva, cstor)
feature (provisioning, backup, restore, exporter)
code component (api, webhook, cast, upgrade)
test (tests, bdd)
chores (version, build, log, travis)

subject is a single line brief description of the changes made in the pull request.

Codecov Report

:exclamation: No coverage uploaded for pull request base (develop@9e0ac5b). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             develop    #266   +/-   ##
=========================================
  Coverage           ?   2.80%
=========================================
  Files              ?      61
  Lines              ?    4107
  Branches           ?       0
=========================================
  Hits               ?     115
  Misses             ?    3983
  Partials           ?       9

I think this change should use the script /buildscripts/test-cov.sh in the 'test' Makefile target, and then try to publish the codecov result without a token? Token is usually required when there are chances of a large number of github api calls, I think.

I think this change should use the script /buildscripts/test-cov.sh in the 'test' Makefile target, and then try to publish the codecov result without a token? Token is usually required when there are chances of a large number of github api calls, I think.

@niladrih , it's just like docker token to upload docker image.

Has been cherry-picked.


The search lands on 404 if no results and enterkey is pressed

@vvo @LukyVj Is there a reason why are we not using docsearch.js for the doc ?

Not that im aware of

Wanna help @rayrutjes @maxiloc?

@vvo @LukyVj Is there a reason why are we not using docsearch.js for the doc ?

It was never done, the search on this website was done before we released docsearch

Fixed in the new website.


Fix Turbo Mercure bridge dependency

Q
A

Bug fix?
yes

New feature?
no

Tickets
Fix https://github.com/symfony/ux/issues/198

License
MIT

I think the package should be able to handle both 1.x and 2.x without change. @weaverryan I guess this will require a dedicated tag right?

I think should only allow 2.0. Otherwise you will have one lib trying to use stimulus and another using @hotwired/stimulus is you have a mix of 1.0 and 2.0
@kbond this is the issue you were talking about (we saw this also with twig/live components) - our test suite needs to be enhanced to catch these :)

Thanks Tito!


Check that doctrine entities are not final

I create all my classes as final by default. This however can cause issues with Doctrine:
  [Doctrine\Common\Proxy\Exception\InvalidArgumentException]
  Unable to create a proxy for a final class "<entity class>".

What do you think about a rule to avoid final entities?

@ondrejmirtes Updated according to your suggestions. Also added a test case.

Hmm... there is already a similar pull request: https://github.com/phpstan/phpstan-doctrine/pull/84
Except I slightly disagree with the statement that entities shouldn't have final methods. I have a case where I'm actually using final method on purpose. But then again I could just disable that particular rule in my codebase.

Done. :-)

Should be okay now.

Thank you!

Tagged as 0.12.20.

@enumag thanks! 👍


Jeanette/scoot200 integration test

adding a go, protobuf-agnostic, client for the daemon (client_lib.go) and integration tests using the client, server, queueing runner, simple runner and os execer.

Is this all necessary prior to trying out daemon w/thrift, and even then it only appears useful for A/B testing and not after we choose one protocol over the other? Can you add a motivation paragraph to the summary (or arrange a VC if that format is better)?

Current coverage is 64.00% (diff: 53.08%)

Merging #160 into master will decrease coverage by 0.78%

@@             master       #160   diff @@
==========================================
  Files            63         65     +2
  Lines          2948       3067   +119
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1910       1963    +53
- Misses          911        960    +49
- Partials        127        144    +17

Powered by Codecov. Last update 195ee4d...d4c300b


Add pdoc github pages

Add pdoc github pages following steps in https://pdoc.dev/docs/pdoc.html

Codecov Report

Merging #113 (e6d31a7) into main (a24baec) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head e6d31a7 differs from pull request most recent head 53c3e12. Consider uploading reports for the commit 53c3e12 to get more accurate results

@@           Coverage Diff           @@
##             main     #113   +/-   ##
=======================================
  Coverage   97.10%   97.10%
=======================================
  Files           7        7
  Lines         518      518
=======================================
  Hits          503      503
  Misses         15       15

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


bee: 0.5.0 -> 0.5.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/ethersphere/bee/releases.
meta.description for bee is: "Ethereum Swarm Bee"
meta.homepage for bee is: "https://swarm.ethereum.org/"
meta.changelog for bee is: ""
Updates performed

Golang update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 0.5.1 with grep in /nix/store/s3iqv8akfc6cj0fdah4c4yc6095iw1xy-bee-0.5.1

directory tree listing: https://gist.github.com/eb21c871467c3a51998928725fbe97b8

du listing: https://gist.github.com/8f665b847a6aac63c6c59b62b96b2241

Rebuild report (if merged into master) (click to expand)

8 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 i686-linux rebuild(s)
2 x86_64-darwin rebuild(s)
2 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
bee
bee-unstable

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/s3iqv8akfc6cj0fdah4c4yc6095iw1xy-bee-0.5.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A bee https://github.com/r-ryantm/nixpkgs/archive/58fd3ff77554f2c3e245c3ad2178b399109878e0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/s3iqv8akfc6cj0fdah4c4yc6095iw1xy-bee-0.5.1
ls -la /nix/store/s3iqv8akfc6cj0fdah4c4yc6095iw1xy-bee-0.5.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package failed to build:

bee-unstable

1 package built:

bee

Maintainer pings
cc @attila-lendvai @kalbasit for testing.

Result of nixpkgs-review pr 114625 at 58fd3ff7 run on aarch64-linux 1

1 package failed to build:

bee-unstable

1 package built:

bee

1 suggestion:

warning: unused-argument
Unused argument: stdenv.
Near pkgs/applications/networking/bee/bee.nix:1:24:
  |
1 | { version ? "release", stdenv, lib, fetchFromGitHub, buildGoModule, coreutils }:
  |                        ^

please see instead: https://github.com/NixOS/nixpkgs/pull/113940


meson: switch from debug build type to plain.

This produces optimized binaries with assertion disabled.
To quote the documentation:

No extra build flags are used, even for compiler warnings, useful for distro
packagers and other cases where you need to specify all arguments by yourself

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

I noticed this when systemd-networkd crashes because of assertions, which are normally disabled in release builds.

weired. this was already set!


client.part() not firing

I call client.part(user.username); and no event is fired. I was able to reproduce this in both 1.1.20b and 1.1.19b.

Looks like it works in master. hm... I'll test with 1.1.18b.

Try the new beta version it should be okay now :smile:
Thanks!

part and join only seem to work if I set the twitch client to 1 in the options.

are you using the beta or master?

I am using the beta.

Hmmm... I just tried and it works for me. Are you using 1.1.22?
What does your code look like?

I am using the example code on the readme with this added...
client.addListener('join', function (channel, username) {
	console.log(username+' joined the channel');
});

client.addListener('part', function (channel, username) {
	console.log(username+' left the channel');
});

Doesn't that listen for a join or part event rather then fire the command? :confused:
https://github.com/Schmoopiie/twitch-irc/wiki/Command:-Join
https://github.com/Schmoopiie/twitch-irc/wiki/Command:-Part

Ah, looks like I mixed up client join/part with users joining/leaving.

Yep so did I. This may be a separate issue....

Looking for users joining/leaving so I can log times.

You would need to set tc: 1 for this but I do not recommend to do it this way if you are doing some kind of a use loyalty system because Twitch's plan is to deprecate TC (TWITCHCLIENT) and use IRCv3 tags instead, which doesn't show JOINs and PARTs (yet?). Also, keep in mind that JOINs and PARTs are sent every 30-60secs in batch. I would recommend to use the request module for node instead and query the Twitch chatters API (Undocumented) instead.

Thanks Schmoopiie. They have so much undocumented stuff, really frustrating sometimes. :)
I prefer the API chatters list over the part/join, this will work better with what I have planned :+1:


Remove hard-coded embassy special cases

https://trello.com/c/NTdQN6o3
This PR removes 8 (of 12) hard-coded special cases that appear on https://www.gov.uk/world/embassies. FCDO have approved these changes (see the Trello card for more information).
Each commit removes a hard-coded special case and adds a data migration that:

Modifies the data based on the requirements from FCDO
Fails fast if the data is left in an unexpected state after the change

⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠
Follow these steps if you are doing a Rails upgrade.

Thanks @BeckaL! I've addressed your feedback. Can you take another look when you have a mo, please?

Thanks, @BeckaL! :+1:


Fix projects/params poll

Fixes https://github.com/G-Research/fasttrackml/issues/1111.
Now polling for projects/params  in metrics tab and in params tab and stop polling when we are not in these tabs.

Looks good, and interesting fix! I hadn't thought of moving the logic into the view instead :)
Just wondering, is it alright if Scatters doesn't have this functionality?

Yep, Scatters doesn't need to fetch projects/params I think


Add Terms and Conditions page in footer section

Issue:
The footer section was missing a link to the "Terms and Conditions" page, which is essential for legal transparency and user guidance.
Changes Made:

Created a new "Terms and Conditions" page.
Added the link to the "Terms and Conditions" page in the footer section for easy access by users.

Files Modified:

Added new folder in app/customer: Updated the footer section to include the "Terms and Conditions" link.
New file: Created the Terms and Conditions page.

closes #751


Forge 1.19.4 crash on startup (Mixin apply failed)

Log here: latest.log

Balanced Enchanting is incompatible with Tax Free Levels.
You can simply remove it as it does more or less the same as Tax Free Levels minus all the compatibility stuff.
Lemme know if there's still any issue.

yeah so sorry that I made a very stupid mistake


Reconnect to parcel collection endpoint as first-party endpoints are registered or deleted

If you create a first-party endpoint after calling receiveMessages(), you won't receive messages bound for that first-party endpoint.

Calling receiveMessages does not affect parcel collection. We only fetch new parcels when the Gateway app notifies, one-off (CloseUponCompletion), and we fetch with all the first-party endpoints we have. So apparently there is no issue with the SDK. I'll check next the Gateway app, maybe it's not refreshing it's parcel collection when new endpoints are registered.

we fetch with all the first-party endpoints we have

That's the problem -- The nonce signers isn't actually updated as new first-party endpoints are registered, unless I'm missing something:
https://github.com/relaycorp/awala-endpoint-android/blob/0d39f7fb24701faedfda88df05c4efe09d6d7ece/lib/src/main/java/tech/relaycorp/awaladroid/messaging/ReceiveMessages.kt#L75-L95

That runs every time the gateway notifies the app to fetch new parcels. So if the new endpoint received some message, when it is notified for it, the nonce signers will be the new ones.

The Gateway app code also looks good, nothing should require a sync restart when a new endpoint is registered. So next I'll try to reproduce the issue with the Ping app.

Thanks @sdsantos! If you still can't reproduce it, let's put a pin on this whilst we try again. Maybe we just experienced a different issue or made a mistake.

Thanks @sdsantos! If you still can't reproduce it, let's put a pin on this whilst we try again. Maybe we just experienced a different issue or made a mistake.

Tried to reproduce it with the Ping app. I delayed creating the endpoint 5 seconds, but subscribed to receiveMessages immediately. The Pong was still received without a problem. So I suggest dropping this until it can be reproduced again with an app.

Sounds good! Will reopen this if that's the case.


Added README file

Added README file in example notebooks.

:books: Documentation preview :books:: https://jupysql--118.org.readthedocs.build/en/118/

there's an error in one notebook:
*** example_notebooks/demo_jupysql_databricks.ipynb ***
(404) https://github.com/ploomber/jupysql@alias

the alias branch no longer exists, can you update? dropping the @alias part should fix it

It's the notebook downloaded from Azure Databricks Subscription. I will try uploading local notebook and see how it happens.


Cpp: vector.ConstIterator should use an index

At the moment, we use a wrapper to provide range-like interface on std::vector:
https://github.com/bpfkorea/agora/blob/494f5bda4efa643087a3d95d2b92c5838bf79cb4/source/scpd/Cpp.d#L192-L213
However, the elephant in the room is resizing. If one modify the std::vector, a reallocation might happen, and the pointer won't be correct anymore.

Fixed by #679


Use memcached instead of session files for caching proof results

The use of memcached prevents the cumbersome handling of session files (cf. #838) and is even more efficient than session files (as gnatwhy3 calls can be prevented).
Tasks

[ ] Add memcached option to all calls of GNATprove
[ ] Add use of memcached to tests workflows
[ ] Remove session files from repository
[ ] Remove make targets and configurations related to session files

I'm curious, where does the memcached server live? And how do you make it "survive" between CI runs?

I'm curious, where does the memcached server live? And how do you make it "survive" between CI runs?

Unfortunately, there is no good solution within Github actions. Basically, we rent a cheap server on the internet and will do SSH port forwarding to make the remote memcached instance available.


[FEATURE REQUEST] finding subfolders is not efficient

Is your feature request related to a problem? Please describe.
I find that Feroxbuster does not prioritize to try to force browsing on the subfolders that it finds,  this makes finding all the subfolders quite time consuming compared with DirBuster...
Both DirbUster and Feroxbuster we fast enough to find: "http://<IP_ADDRESS>/xpto" , but while Dirbuster found easily "http://<IP_ADDRESS>/xpto/stuff" Feroxbuster took quite a long time to find it...
Describe the solution you'd like
Have more threads allocated to find subfolders of having that option...
Describe alternatives you've considered
running other Feroxbuster  instance to find subfolders of the folder found: "http://<IP_ADDRESS>/xpto/"
Additional context
N/A

Please delete this, it seems that the "not recursive" behaviour was a default from autorecon...

No sweat, glad it's sorted out!


[v1.16] CODEOWNERS: switch cilium/tophat to cilium/committers

Membership in tophat typically changes on a weekly basis, resulting in stale review approvals after a change-over. It's also a single source of failure in our review & merge machinery.
As tophat for stable branches is primarily meant to review for process conformity (backport criteria, correct formatting of the PR etc), pull in all of the committers instead to perform this important responsibility.
Note: if you can't find a single committer to approve your stable PR, it most likely shouldn't be merged.

i guess we need to somehow ensure all the committers apply the backporting criteria consistently https://docs.cilium.io/en/stable/contributing/release/backports/#id1
at some point i also wondered if the default code owners for stable branches should be @cilium/release-managers 💭 let's see how they feel about that.
another option might be to not remove people from the tophat team. we just need to configure review assignments right?

I don't love assigning ownership to committers mainly due to the bystander effect - a large group is responsible, so no-one is responsible.

I agree with the Joe's opinion here, but I guess it's a matter of configuring the review assignment? I'm fine with taking care of the review explicitly assigned to me, but otherwise, I guess I wouldn't take the responsibility of reviewing the PR out of my expertise out of 30 people. I would think there's sufficient people than me.

As a member of the tophat rotation, I'm really not looking forward to getting pinged for every single backport PR that gets created (as opposed to only getting pinged on my shift). If Yutaro's response is any indication, bystander effect will be huge, quite expected when blasting these out to 30 people.
IMO Joe's proposal is better, letting backport merges be coordinated by other backporters and/or release managers would be more effective. Release managers currently need to call for merges to be paused during branch preparation, and this likely causes some stress on their part if they feel like they're holding people up. Putting them in control of the whole process would cause less friction, if the workload is manageable, of course.

Something else that's probably worth noting is that the common case is there's already at least one committer reviewing backport PRs. If committers are the codeowner then this will naturally resolve the requirement for review. We don't necessarily have to directly assign a particular committer. In some ways this proposal as-is just makes the GitHub review system for stable branches more in line with how we actually review stable PRs - best effort review and sanity check from authors, then in the end committers (including backporters/release-managers) have rights to merge the PR.


Add new RSpec/SortMetadata cop

This cop sorts the RSpec metadata alphabetically:
  # bad
  describe 'Something', :b, :a
  context 'Something', foo: 'bar', baz: true
  it 'works', :b, :a, foo: 'bar', baz: true

  # good
  describe 'Something', :a, :b
  context 'Something', baz: true, foo: 'bar'
  it 'works', :a, :b, baz: true, foo: 'bar'

Before submitting the PR make sure the following are checked:

[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Updated documentation.
[x] Added an entry to the CHANGELOG.md if the new code introduces user-observable changes.
[x] The build (bundle exec rake) passes (be sure to run this locally, since it may produce updated documentation that you will need to commit).

If you have created a new cop:

[x] Added the new cop to config/default.yml.
[x] The cop is configured as Enabled: pending in config/default.yml.
[x] The cop is configured as Enabled: true in .rubocop.yml.
[x] The cop documents examples of good and bad code.
[x] The tests assert both that bad code is reported and that good code is not reported.
[x] Set VersionAdded in default/config.yml to the next minor version.

If you have modified an existing cop's configuration options:

[ ] Set VersionChanged in config/default.yml to the next major version.

Speaking of the name, I think we can refer to the upcoming Cop naming guidelines. I'd suggest RSpec/UnorderedMetadata.

Checked against real-world-rspec:
57497 files inspected, 230 offenses detected, 230 offenses autocorrectable

1 error occurred:
An error occurred while RSpec/SortMetadata cop was inspecting /Users/pirj/source/real-world-rspec/rspec-core/spec/rspec/core/metadata_spec.rb:51:14.

Hi @leoarnold, I’m a bit eager to merge this PR and get a release ready before I’m going on vacation – so I added a commit with @Darhazer’s code suggestion, plus support for multiline offenses (even if only the 1st line gets marked). I hope that’s ok 😅
Eventually, we’ll probably squash the two commits into just one before merging.

@bquorning I recognize that it is unclear "whose code" it is if you open a PR on somebody else's repository. Nevertheless I am very unhappy with such conduct. I don't understand why there is any objective requirement to rush this PR. And it kinda feels like you are putting pressure on other people solely based on constraints of your private life plans.

solely based on constraints of your private life plans.

Much open source work is to a certain degree constrained by the maintainers’ private life plans 😄
But @leoarnold, I am sorry for putting undue pressure on you. You are correct that there is no objective requirement to rush this PR. I guess I was getting carried away by this new cop being great and me feeling we were getting close to cutting a new release.

I am very unhappy with such conduct.

I am sorry to hear that, and please feel free to revert my changes.

@leoarnold Is there anything left you want to adjust in this PR before it gets merged?

@leoarnold I understand your frustration. Though personally I was more frustrated at the times when my PRs were ignored (one in PHPUnit will turn 6 this year, despite the efforts of the community to push it forward.
Realistically, if we'd merge your original PR, and then add the changes as a separate commit (as they are done) - the end result - code-wise - would be the same.
The difference is that you didn't got the chance to apply - or reply to - the suggestions yourself. I'm sorry about that. I hope you'd accept our - all of the rubocop-rspec maintainers - appology for this rushed decision.
I'm not merging this for the sole reason to give you the chance to respond (well, you could do that in a closed PR as well). Just responding with I'm happy for this PR to be merged would be enough. And I thank you for this great contribution :bow:

Also I just noticed there is no changelog entry for the new cop

As mentioned earlier, I really wish to see this pull request merged. But we have not heard back from the original contributor in several weeks, and I think moving forward now can’t be considered unnecessary rushing.
I am not going to repeat the mistake of pushing (let alone force-pushing) to a contributor’s fork of the repository. Instead, I consider opening a new pull request with @leoarnold’s commit and three other commits with subsequent changes. I hope this is an acceptable solution 🙏🏼

Superseded by #1378
@leoarnold I encourage you to revive the original part of this PR related to changing the style focus: true <-> :focus as a separate cop.


Kata Containers 1.3.0-rc0

qemu: refactor maximum vcpus supported in aarch64
Add ref counter for devices
versions: update kernel version 4.14.67
network: always assign sandbox networkNS
virtcontainers: wait until process exited before RemoveContainer
virtcontainers: revert apply devices constraints
vendor: update govmm to be compatible with qemu 2.8
network: Remove CNI docs
network: Move OCI hooks and network namespace creation out of virtcontainers
virtcontainers: delete out-of-date TODO
build: fix breaking 1.8.3 build
virtcontainers: apply devices constraints
kata-env: Fix amd64 VM container capable check
travis: Add variable needed to run static checks
virtcontainers: Apply memory constraints
builtin-proxy: print guest console output to logger
vendor: Update kata agent for gRPC tracing
Pass qemu --share-rw option for hotplugging disks
virtcontainers: storage: Rework resource storage
qemu/arm64: Detect host GIC version to configure guest GIC
virtcontainers: kata_agent: Move out a generic function
virtcontainers: qemu: Don't shutdown QMP from hotplug
Add virtcontainers trace support
vfio: Add configuration to support VFIO hotplug on root bus
vendor: update kata agent
network: fix vhost-user net creation
vc: Remove unused variable NumInterfaces
cli: kata-env: show if vsocks are supported
api: add sandbox hotplug network
CI: bump the CI travis's go to 1.10
cli: kata-env: add runtime path to output
Remove unused cni code
devices: add interface "sandbox.AddDevice"
versions: Update the commit for qemu-lite
tracing: Add initial opentracing support
cli: Make message of using initrd OR rootfs clearer
vendor: Update dep lock file for new format
network: Error out when host-networking is requested
virtcontainers: ppc64le: Add nvdimm to defaultQemuMachineOption
virtcontainers: qemu: x86: Support "virt" machine type

2cd2dd0 versions: update kernel version 4.14.67
2d13c46 test: add unit test for func MaxQemuVCPUs
c4ded6e qemu: refactor maximum vcpus supported in aarch64
a5478b9 virtcontainers: wait until process exited before RemoveContainer
4307982 virtcontainers: revert apply devices constraints
c518b1e device: use devicemanager to manage rootfs block
26f912e virtcontainers: Make qdisc of hotplug nics mq
5e849f1 network: always assign sandbox networkNS
e599ef4 device: don't persist ContainerPath of DeviceInfo
affd6e3 devices: add reference count for devices.
7f4b221 devices: share genericDevice logic among devices
af0c137 build: fix breaking 1.8.3 build
9a8b45f network: Remove CNI docs
e906095 virtcontainers: delete out-of-date TODO
35d7672 kata-env: Fix amd64 VM container capable check
414ddd1 travis: Add variable needed to run static checks
137769a virtcontainers: apply devices constraints
2677a5f virtcontainers: Apply memory constraints
3940189 builtin-proxy: print guest console output to logger
f841e89 vendor: update govmm to be compatible with qemu 2.8
97d280e virtcontainers: Don't handle the network in case of a factory
9c6ed93 hook: Move OCI hooks handling to the CLI
ec0fd1b virtcontainers: sandbox: Add new getter to retrieve netns
cb351dc network: Create network namespace from the CLI
44d2ec7 virtcontainers: network: Rename CNM to DefaultNetwork
7785352 vendor: Update kata agent for gRPC tracing
8b20c3f virtcontainers: storage: Separate storage interface from implementations
26f0430 virtcontainers: storage: Add a noop version of filesystem
f0e09c8 virtcontainers: kata_agent: Move out a generic function
a1787da virtcontainers: qemu: Don't shutdown QMP from hotplug
b438590 qemu/arm64: Detect host GIC version to configure guest GIC
d0679a6 tracing: Add tracing support to virtcontainers
6ddc9b4 network: Use correct logger
90970d9 tracing: Add trace spans to virtcontainers APIs
c200b28 tracing: Add context to virtcontainers API
f0073be tracing: Create trace function
a193366 tracing: Add subsystem tag to runtime
31e2925 vfio: Add configuration to support VFIO hotplug on root bus
49690b0 vendor: update kata agent
ee1ea36 network: fix vhost-user net creation
70edc56 disk: Pass the --share-rw option for hotplugging disks
b9e0ca3 vendor: vendor latest govmm
161e3a7 vc: Remove unused variable NumInterfaces
88d79f3 cli: kata-env: show if vsocks are supported
7beb309 test: add UTs for network hotplug
72df219 cli: add network commands
1a17200 virtcontainers: add sandbox hotplug network API
6666426 vendor: update govmm and agent changes
4850579 CI: bump the CI travis's go to 1.10
6e6be98 devices: add interface "sandbox.AddDevice"
474111c tests: Add a cleanup for the CNM tests
0d7b476 network: Remove unused cni vendored code
99fa758 network: Remove unused CNI code
f6bfb85 versions: Update the commit for qemu-lite
8b69c75 cli: kata-env: add runtime path to output
3a1bbd0 tracing: Add initial opentracing support
0ede467 tests: Add cli.Context helper functions
41d1c14 tests: Move assert closer to function call
bee8d66 cli: Make message of using initrd OR rootfs clearer
d9fa73c vendor: Update dep lock file for new format
2f3f375 network: Error out when host-networking is requested
ef3a7e8 virtcontainers: ppc64le: Add nvdimm to defaultQemuMachineOption
ef74bc5 virtcontainers: qemu: x86: Support "virt" machine type

ready to merge.

Hi - this issue is labelled release-gating but doesn't have an associated PR.
Please can you comment on the status of this item so we know if it could be
included in the upcoming 2.0 release?


Implement DM29

Add message handling and the associated tests required for the application to handle a DM29 message. #15

Reopening to allow testing to be added DTCModuleTest.