feat [IDP-1884] :  Added the release notes for v0.17.0

Harness Developer Pull Request
Added the release notes for v0.17.0
What Type of PR is This?

[ ] Issue
[x] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[x] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6582e33256d7340b0fd1a770--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6583d90eb2a7d7b1974e8bb4--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6583dce3e2582cb46fe19d19--harness-developer.netlify.app


Adds Pipe to late

when BC-eXtra-Pipes is added i get this

what version of bc / bc-extra-pipes is this?

same error we have been getting reported: https://github.com/BuildCraft/BuildCraftCompat/issues/80
only issue is this works perfectly fine in dev and on the test server so something else is messing with things, so far nobody has been able to provide more details


basic event layout, minor change in admin js

basic event.html layout
minor change in admin js to provide dynamic linkage

the layout of event page is excellent with images.
merge it as soon as possible so that everyone can work on it


Bug: When editing a project field, the Wrangling status disappears

Bug.
When editing a project field, the Wrangling status disappears. We have checked this with two statuses.
Steps to reproduce:

Open up a Project with a status 'Eligible' or 'in Progress', could be for other statuses as well
Change the Project label to something else. Ignore validation errors and Save.
Check the project Admin tab - Wrangling status is not only 'Not specified', but the whole box is missing (see screenshot)
Sometimes we see this on the first time, sometimes on the second time you go and Edit this project.

This actually happens when I create a project from scratch in backoffice.

Does it happen when your project has no submissions?

yep when I tested, but worth it to also check the scenario of having and existing submission and then editing project details.

as I didn't receive this error on the project with submission I'll try it with the clear project without any submissions
@gabsie @MightyAx

(1) I don't see the 'Wrangling status' field after creating a new project (at staging URL), is it a normal case? I assumed to see the 'N/A' for example (as in the other fields: Wrangling priority / Release number)

so far I don’t see an error either on 'staging' or on the 'dev'

[ ] @Mil-m to book meeting with @ofanobilbao for issue identification.

were we testing before in dev or prod? @ofanobilbao

@gabsie this was only in staging. It was only to clarify with Milena what the issues where. Once she observes the issues herself, then she might be able to start understanding why this is happening.

the bug couldn't be reproduced but a trace remains in the History tab
the status usually disappears after 12h, but in this case the status has remained
@ofanobilbao will add more details

Bug does not seem to be reproducible so moving to waiting.

@ofanobilbao to add some screenshots showing when this happened at least for a few examples

@ofanobilbao to add some screenshots and then close the ticket because it does not seem to be reproducible.

Find one specific example (others were the same) of a project that I added for that Atlas effort. I added the project, triaged it and labelled it Eligible and as per the history (another screenshot) I can see status was removed in about 12h.

2nd example of the same situation:

As per our discussion in stand-up earlier today, I am closing the ticket with all possible screenshots I took. In case we need to re-surface the ticket because the issue happens again. At the minute the errors seem to not be taking place. So the bug is either resolved or not reproducible anymore.


Cyclic structures

Why don't you support cyclic structures?:

Nested structures are supported, but reference cycles are not permitted.
HTTP-RPC documentation

You could use JSOG for dereferencing. Not allowing cyclic structures (like a student with a bidirectional many-to-many association to professors) is a very hard restriction for a RPC-framework.

Great question! The short answer is simplicity. In general, I've tried to avoid overloading the framework with features that could reasonably be handled at the application level.
However, I wasn't aware of JSOG. It's an interesting concept. It would require some additional processing to detect/reconstruct cycles as well as some additional bandwidth for the ID/ref data, but I can see the utility.
I'm not sure I'd want to automatically apply it to every response, but it might make sense as an opt-in feature. For example, if I know a method might returns a cyclic structure, I could annotate it and trigger JSOG handling of the result.

It may even reduce bandwidth and increase the referential integrity of an object tree (and an object graph too for sure) since objects that occur more than once in a request or a response are written only once and referenced by id at every other occurrence. For the resulting object graph is JSOG completely transparent. There is a Jackson extension library for JSOG, too (I don't know which library you use on the server side):
<dependency>
  <groupId>com.voodoodyne.jackson.jsog</groupId>
  <artifactId>jackson-jsog</artifactId>
  <version>1.1</version>
</dependency>

I wrote the JSON serialization code myself, so adding it to the server wouldn't be difficult. I also wrote the deserialization logic for the Java client, so that wouldn't be a problem either. But the iOS client uses NSJSONSerialization and the JavaScript client uses JSON.parse(), and I don't think either of those currently support JSOG.
That said, it might be OK to support JSOG on the server only in the short term and add client support later, if it makes sense. This implies that any JSOG-enabled server could only be accessed by a JSOG-aware client library, which, in this case, would need to be a 3rd-party library such as jackson-jsog, since the HTTP-RPC client libraries won't support it.
I'm still not sure what the best approach would be for enabling JSOG response processing, though. It could be done on a per-method basis via an annotation, at the service level via a configuration parameter, or even at the JVM level via a system property. Alternatively, it could be managed via a custom request header indicating that the client is JSOG-capable. Any thoughts?

I think you can do the JSOG-stuff in the iOS client in a pre-/post processing step (transparently). JSOG is more or less a one-hundred-liner. It is quite straightforward.
But if you want to be compatible to arbitrary clients there, will be a need for some kind of compatibility mode. JAX-RS for example offers @Consume and @Produce annotations on a per method (perhaps even per class, I'm unsure) basis where you can configure whether your server provides JSON or XML serialization (or both) and switches between the configured JSON (e.g. Jackson) and XML (e.g. JAXB) serialization implementation. The client may choose between them via the corresponding Accept-Header text/xml and application/json. Unfortunately, there is no MIME-Type for JSOG in the IANA registry of IETF.
A possible solution could be to send two MIME-types application/json and application/jsog+json. or two use a custom header as you said.

Yes, JSOG could be implemented via post-processing. And yes, there needs to be some sort of compatibility mode - that's what I meant by "enabling" JSOG. Not all clients will want or need it, so there needs to be a way to toggle it on/off.
I'm currently thinking that a header might be the best approach, since it would allow the caller to opt in. However, if a caller failed to request JSOG on a call that returns a cyclic data structure, it would result in an infinite loop on the server. So perhaps the service would need some way to flag a method as requiring JSOG as well. If a caller invokes a JSOG-required method but fails to specify the JSOG header, the server could simply return an error response.

if a caller failed to request JSOG on a call that returns a cyclic data structure, it would result in an infinite loop on the server

I think this cannot be decided completely upfront (at compilation time), so that flagging the method is IMHO inappropriate. In your current implementation an object to be returned could contain cycles in a reference either. So, independently from the support of cyclic structures it would be a good thing to recognize cycles during serialization and (if JSOG is disabled) raise a runtime exception. JSOG could easily be adapted to have a mode where it detects cycles (like a mark mechanism in a breadth-first search) instead of dereferencing.

That's true - the current implementation doesn't detect cycles and will go into an infinite loop if one is found. So detecting cycles at runtime rather than providing hints at compile time is probably a better approach.

Closing. The server component has been retired, so this issue no longer applies.

Retired?

Right. The project doesn't seem to be getting as much traction as I had hoped, so I was planning to phase out the server component and focus on the client. Are you currently using it? I really don't have any insight into usage. I opened up a Gitter channel a few weeks ago hoping to spur some discussion, but no one has posted anything yet. I tried something similar with Disqus a few months back - same result.
I've been thinking about restoring server support in a future update, in a slightly different but mostly compatible form. If enough people are actually using it, I'd consider doing that sooner rather than later.
For what it's worth, I moved all of the template stuff to a new project (everything is still in the development branch since it's not quite ready for release):
https://github.com/gk-brown/JTemplate
If I do update the server, it will most likely include this project as a dependency.

I was thinking about using it (we are currently using a relatively complex tool stack with hand-crafted support for cycles in the client) and wanted to see whether this project is better suited...


Check whether response is private?

This is more of a question to gauge your opinion before I do anything rash, like another pull request. I know right now I can do...
# is this a private direct message?
response.message.source.private_message?

to test if it is a private message sent directly to the bot. But maybe we could trickle this upstream closer to the response as well...
# is this a private direct message?
response.private_message?

Pros or cons?
Pro: developers who deal with separate private and public messages won't need to know about the intricacies of the Response object
Con: can't think of one right off the top of my head, I don't know this code well enough to judge if this is a terrible idea

I'd be happy to accept that in a PR. There are actually a lot of Law of Demeter violations in Lita's API because of all the interconnections between the robot, adapter, source, message, and room objects. In most cases that gap should be bridged by turning methods like foo.bar.baz into foo.baz when the meaning of the method is obvious in that context, and into foo.bar_baz (or something similar) when disambiguation is needed.
Perhaps also worth mentioning, though maybe not really relevant here, that Lita::Response may be broken apart into multiple objects in the future, since it's really more of a request object than a response object. It has elements of both, which makes it confusing (especially for new users) and not consistent with the way HTTP routes work.


bookshelf error on App Engine: "ImportError: cannot import name 'secure_filename' from 'werkzeug'"

Manifests in App Engine by showing:
502 Bad Gateway
nginx
Look in the logs, to see the whole traceback:
Traceback (most recent call last):
File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker worker.init_process()
File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 104, in init_process super(ThreadWorker, self).init_process()
File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 129, in init_process self.load_wsgi()
File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi self.wsgi = self.app.wsgi()
File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load()
File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load return self.load_wsgiapp()
File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp return util.import_app(self.app_uri)
File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 350, in import_app import(module)
File "/srv/main.py", line 22, in  import storage
File "/srv/storage.py", line 23, in  from werkzeug import secure_filename ImportError: cannot import name 'secure_filename' from 'werkzeug' (/env/lib/python3.7/site-packages/werkzeug/init.py)
The Fix seems to be to replace line 23 in storage.py:
from werkzeug import secure_filename
with:
from werkzeug.utils import secure_filename
I've already tested the fix. I'm going to propose a PR.

We see this in multiple labs right now. Here's one.
https://www.qwiklabs.com/focuses/6073?catalog_rank={"rank"%3A3%2C"num_filters"%3A1%2C"has_search"%3Atrue}&parent=catalog&search_id=4626184

Merge fixed issue in my testing.

it appears that a temporary workaround is to revert werkzeug to v0.16.1:
https://github.com/puckel/docker-airflow/issues/499#issue-561766591

You should use
from werkzeug.utils import secure_filename

@galeanodairo thats worked, change the from werkzeug import secure_filename tofrom werkzeug.utils import secure_filename. Thanks!

You should use
from werkzeug.utils import secure_filename

tx brother

You should use
from werkzeug.utils import secure_filename

Thanks.


[some particular issue](#12345) syntax unsupported

Hey there,
I am happy to see Documenter’s changelog.jl as a standalone package. So, I can open some first issue here :)
When reading the “Fixed” section of Documenter v1.0.0, I noticed some bad looking link probably due to the [text](#12345) syntax currently not being supported. Could we support that?
If there is any interest in that, I will draft a PR for this.

Sorry, I didn't notice this issue earlier. I'd say there's definitely interest in this. This is related to https://github.com/JuliaDocs/Documenter.jl/pull/2283.

Does [some particular issue][#12345] already work? If not, thats probably what should be implemented?

Re the Documenter PR, it looks like it doesn't. I.e. I don't think Changelog picks up the issue reference when it's written as [a bug in the Julia Markdown parser][JuliaLang/julia#29344]? I agree that this should be the syntax though.


question for models

Sorry, but I cannot find the dir 'models' in models/init.py, and then I cannot run main.py. Is it exists in the previous version?

Hi there. Our relative import paths assume that main should be launched from the project root directory. You should start from the root mammoth folder and then run main as python utils/main.py. By running it like this, everything should work.
Also make sure that the mammoth root directory is in your PYTHONPATH env variable.


add update to the es query for stats repo

Fix for #460
See comments on the Fix section, should be accountid as local es query on kibana confirms
Problem is the query needs to use accountid.
Here's what it returns if it uses the accountid in the stats query
"buckets": [
{
"key": "xxx2",
"doc_count": 61542
},
{
"key": "xxx3",
"doc_count": 11758
},
{
"key": "xxx4",
"doc_count": 1860
}
]
}
}

looks like this will never be fixed...


Faster getValidX

The code is simple and readable but not extremely fast when checking all the available squares for the different pieces. Functions execution should be benchmarked for future optimisation

bobby should simply validate moves, it's already fast enough


add telemetry code

add telemetry code
refactor existing code format

@SummerSun,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


chore: release 2.0.1

:robot: I have created a release *beep* *boop*
2.0.1 (2019-06-14)
Bug Fixes

deps: bump minimum deps and remove pify (#141) (26c3eeb)

This PR was generated with Release Please.

Release is at https://github.com/googleapis/nodejs-googleapis-common/releases/tag/v2.0.1


docker compose up error

Ubuntu 20.04.6 LTS
Docker version 25.0.3, build 4debf41
ui-1   |   You can now view your Streamlit app in your browser.
ui-1   |
ui-1   |   Network URL: http://<IP_ADDRESS>:8501
ui-1   |   External URL: http://<IP_ADDRESS>:8501
ui-1   |
api-1  | /usr/local/lib/python3.11/site-packages/beartype/_util/hint/pep/utilpeptest.py:311: BeartypeDecorHintPep585DeprecationWarning: PEP 484 type hint typing.Sequence[str] deprecated by PEP 585. This hint is scheduled for removal in the first Python version released after October 5th, 2025. To resolve this, import this hint from "beartype.typing" rather than "typing". For further commentary and alternatives, see also:
api-1  |     https://beartype.readthedocs.io/en/latest/api_roar/#pep-585-deprecations
api-1  |   warn(
api-1  | /usr/local/lib/python3.11/site-packages/pathway/io/http/_server.py:622: UserWarning: delete_completed_queries arg of rest_connector should be set explicitly. It will soon be required.
api-1  |   warn(
api-1  | Traceback (most recent call last):
api-1  |   File "/app/main.py", line 12, in <module>
api-1  |     app_api.run(host=host, port=port)
api-1  |   File "/app/api.py", line 32, in run
api-1  |     documents = input_data.select(texts=parser(pw.this.data))
api-1  |                                         ^^^^^^^^^^^^^^^^^^^^
api-1  |   File "/usr/local/lib/python3.11/site-packages/pathway/internals/udfs/__init__.py", line 194, in __call__
api-1  |     return self.executor._apply_expression_type(
api-1  |            ^^^^^^^^^^^^^
api-1  | AttributeError: 'ParseUnstructured' object has no attribute 'executor'
api-1 exited with code 1
ui-1   | 2024-03-16 15:29:33.898 Uncaught app exception
ui-1   | Traceback (most recent call last):
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
ui-1   |     sock = connection.create_connection(
ui-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
ui-1   |     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
ui-1   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/socket.py", line 962, in getaddrinfo
ui-1   |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
ui-1   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   | socket.gaierror: [Errno -2] Name or service not known
ui-1   |
ui-1   | The above exception was the direct cause of the following exception:
ui-1   |
ui-1   | Traceback (most recent call last):
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
ui-1   |     response = self._make_request(
ui-1   |                ^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
ui-1   |     conn.request(
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
ui-1   |     self.endheaders()
ui-1   |   File "/usr/local/lib/python3.11/http/client.py", line 1293, in endheaders
ui-1   |     self._send_output(message_body, encode_chunked=encode_chunked)
ui-1   |   File "/usr/local/lib/python3.11/http/client.py", line 1052, in _send_output
ui-1   |     self.send(msg)
ui-1   |   File "/usr/local/lib/python3.11/http/client.py", line 990, in send
ui-1   |     self.connect()
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 238, in connect
ui-1   |     self.sock = self._new_conn()
ui-1   |                 ^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 205, in _new_conn
ui-1   |     raise NameResolutionError(self.host, self, e) from e
ui-1   | urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPConnection object at 0x7f67dd4c0550>: Failed to resolve 'api' ([Errno -2] Name or service not known)
ui-1   |
ui-1   | The above exception was the direct cause of the following exception:
ui-1   |
ui-1   | Traceback (most recent call last):
ui-1   |   File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
ui-1   |     resp = conn.urlopen(
ui-1   |            ^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
ui-1   |     retries = retries.increment(
ui-1   |               ^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
ui-1   |     raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
ui-1   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='api', port=8080): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f67dd4c0550>: Failed to resolve 'api' ([Errno -2] Name or service not known)"))
ui-1   |
ui-1   | During handling of the above exception, another exception occurred:
ui-1   |
ui-1   | Traceback (most recent call last):
ui-1   |   File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 542, in _run_script
ui-1   |     exec(code, module.__dict__)
ui-1   |   File "/app/ui.py", line 25, in <module>
ui-1   |     response = requests.post(url, json=data)
ui-1   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 115, in post
ui-1   |     return request("post", url, data=data, json=json, **kwargs)
ui-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
ui-1   |     return session.request(method=method, url=url, **kwargs)
ui-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
ui-1   |     resp = self.send(prep, **send_kwargs)
ui-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
ui-1   |     r = adapter.send(request, **kwargs)
ui-1   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ui-1   |   File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
ui-1   |     raise ConnectionError(e, request=request)
ui-1   | requests.exceptions.ConnectionError: HTTPConnectionPool(host='api', port=8080): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f67dd4c0550>: Failed to resolve 'api' ([Errno -2] Name or service not known)"))

If Dockerfile is python3.10
# Use an official Python runtime as a parent image
FROM python:3.10-slim

# Set the working directory in the container
WORKDIR /app
# Copy the dependencies file to the working directory
COPY requirements.txt requirements.txt
# Install any dependencies
RUN pip install --upgrade -r requirements.txt
# Copy the content of the local repo
COPY . .

gives the same error

What version of pathway is actually to be used?

So building seemed to work for me (there might be some redundancy):
I started off with a clean ubuntu 22.04 server system on a i9 with nvidia graphics card (not needed: within proxmox)
Install python 3.11:
sudo apt update
sudo apt upgrade
sudo apt install --reinstall pkg-config cmake-data
sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev libsqlite3-dev wget libbz2-dev
wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tgz
tar -xf Python-3.11.0.tgz
cd Python-3.11.0/
./configure --enable-optimizations
make -j$(nproc)
sudo make altinstall
python3.11 --version
nano ~/.bashrc
add:
alias python3='/usr/local/bin/python3.11'
source ~/.bashrc
python3 --version
modified Dockerfile to
# Use an official Python runtime as a parent image
FROM python:3.11

RUN apt update && apt install build-essential

# Set the working directory in the container
WORKDIR /app
# Copy the dependencies file to the working directory
COPY requirements.txt requirements.txt

# Install any dependencies
RUN pip install --upgrade -r requirements.txt
# Copy the content of the local repo
COPY . .

then follow the readme for installing via the docker version
However I still get:
dropbox-ai-chat-api-1  | /usr/local/lib/python3.11/site-packages/beartype/_util/hint/pep/utilpeptest.py:311: BeartypeDecorHintPep585DeprecationWarning: PEP 484 type hint typing.Sequence[str] deprecated by PEP 585. This hint is scheduled for removal in the first Python version released after October 5th, 2025. To resolve this, import this hint from "beartype.typing" rather than "typing". For further commentary and alternatives, see also:
dropbox-ai-chat-api-1  |     https://beartype.readthedocs.io/en/latest/api_roar/#pep-585-deprecations
dropbox-ai-chat-api-1  |   warn(
dropbox-ai-chat-api-1  | /usr/local/lib/python3.11/site-packages/pathway/io/http/_server.py:622: UserWarning: delete_completed_queries arg of rest_connector should be set explicitly. It will soon be required.
dropbox-ai-chat-api-1  |   warn(
dropbox-ai-chat-api-1  | Traceback (most recent call last):
dropbox-ai-chat-api-1  |   File "/app/main.py", line 12, in <module>
dropbox-ai-chat-api-1  |     app_api.run(host=host, port=port)
dropbox-ai-chat-api-1  |   File "/app/api.py", line 32, in run
dropbox-ai-chat-api-1  |     documents = input_data.select(texts=parser(pw.this.data))
dropbox-ai-chat-api-1  |                                         ^^^^^^^^^^^^^^^^^^^^
dropbox-ai-chat-api-1  |   File "/usr/local/lib/python3.11/site-packages/pathway/internals/udfs/__init__.py", line 194, in __call__
dropbox-ai-chat-api-1  |     return self.executor._apply_expression_type(
dropbox-ai-chat-api-1  |            ^^^^^^^^^^^^^
dropbox-ai-chat-api-1  | AttributeError: 'ParseUnstructured' object has no attribute 'executor'

Digging further So I looked at
https://github.com/pathwaycom/llm-app
running the docker version seemed to work:
curl --data '{"user": "user", "query": "How to connect to Kafka in Pathway?"}' http://localhost:8080/
If I find time I will try to compare code with dropbox-ai-chat and maybe find the issue.

See bottom of this comment for the solution that worked for me.
So I decided to use poetry (as done in https://github.com/pathwaycom/llm-app) (again i might have redundancy here) and build without docker:
python3 -m venv pw-env && source pw-env/bin/activate
python3 -m pip install --user pipx
python3 -m pipx ensurepath
(open a new terminal)
python3 -m venv pw-env && source pw-env/bin/activate
pipx install poetry
poetry completions bash >> ~/.bash_completion
poetry init
(answer the questions with the packages from requirements.txt, but you can add later e.g. poetry add tiktoken if you forgot something)
python main.py
Alas: same error https://github.com/pathway-labs/dropbox-ai-chat/issues/5#issue-2190105798
however I tried first with
unstructured = {version = "0.10.15", extras = ["all-docs"]}
i removed that line for pyptoject.toml
and did
poetry add unstructured
which a.o. told me I had to set python to point to python3.11
so added
alias python=python3
to ~/.bashrc
then
source ~/.bashrc
and again:
python -m venv pw-env && source pw-env/bin/activate
python --version
gave Python 3.11.0
however
poetry add unstructured
still complained
So I did
poetry add unstructured["all-docs"]="^0.11.8
poetry update
deleted the poetry.lock file
poetry install
poetry run python main.py
Seemed that I had to recompile python (hahah), because it gave no modile _lzma
sudo apt-get install lzma
sudo apt-get install liblzma-dev
sudo apt-get install libbz2-dev
so I did recomplie python (see in a comment above how)
then again
poetry run python main.py
gave same error
AttributeError: 'ParseUnstructured' object has no attribute 'executor'
now I changed in pyproject.toml
python = "^3.11"
into
python = "3.11"
got an error that
because dropbox-ai-chat depends on both pathway (^0.8.3) and unstructured[all-docs] (^0.12.6), version solving failed.
changed pyproject.toml (copy parts from https://github.com/pathwaycom/llm-app) to
[tool.poetry]
name = "dropbox-ai-chat"
version = "0.1.0"
description = "AI-powered Dropbox search tool for private documents"
authors = ["many"]
license = "MIT"
readme = "README.md"

[tool.poetry.dependencies]
python = "3.11"
DateTime = "^5.4"
python-dotenv = "^1.0.1"
pathway = "=0.8.2"
openai = "^1.2.4"
requests = "^2.31.0"
streamlit = "^1.26.0"
unstructured = { extras = ["all-docs"], version = "^0.11.8"}
tiktoken = { version = "^0.6.0"}
litellm = "^1.18.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

then
poetry lock
poetry install
poetry run python main.py

server:
poetry run python main.py
UI:
streamlit run ui.py


Check opentracing.header_filter_span nil.

This fixes #10

Alternate fix in f384cef

@james-callahan This fix f384cef make a new issue #15


New version: MIRT v0.2.0

Registering package: MIRT
Repository: https://github.com/JeffFessler/MIRT.jl
Created by: @JeffFessler
Git reference: register/MIRT/v0.2.0
Version: v0.2.0
Commit: cc0e9d90bec07de2cffd54826a042fdb13b915c6
Reviewed by: @JeffFessler
Reference: https://github.com/JeffFessler/MIRT.jl/issues/4

https://github.com/JeffFessler/MIRT.jl/commit/09d84b9a1e2aacc5badafb7703fbf8d423166546#commitcomment-33542107


Mouse Move→Click→Drag gives link instead of horizontal scroll

Firefox 94.0.2-1
Chromium 93.0.4577.82-1
Chrome 96.0.4664.93-1
Reproducing: move mouse horizontal, press mouse button down to scroll
Expected: Horizontal scroll
Actual: Mouse "grabs" link, which can be dropped as new tab.

Afterwards the scroll is also stuck enabled until clicked again.
Also: In Firefox, scrolling seems to stop and gives above occurrence.

Thank you very much for your tip. I am aware of the problem and will investigate it a bit more in the next few days.

Hey @Urfoex the problem should be fixed now. Thank you!


Add preview section to reconfig modal

What does this PR do and why?
Adds funding cycle summary/preview to reconfig modal.
Big code changes in this PR are from making components for each funding parametre's Statistic with their tooltip explanations, risk warnings, etc. to be reused in both the 'Review and deploy' summary and here.
https://user-images.githubusercontent.com/96150256/167106408-00eae78b-03a9-446b-8dbd-8373baea8923.mp4
Also fixes 'Review and deploy' summary not taking full width in some cases:

Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] I have tested this PR in all supported browsers.

 This is cool 🥳

@johnnyd-eth just rebased to main, hope thats okay

Layout looks super good on desktop! Something to take note is we should consider enhancing the mobile view:

but definitely not what I consider a blocker

Layout looks super good on desktop! Something to take note is we should consider enhancing the mobile view:  but definitely not what I consider a blocker

Yeah pretty rare this would be used on mobile but I've improved it a bit:

@johnnyd-eth FUCK yes ser. Thank you for this. Approving, pls merge once translation conflicts are resolved ❤


[FEATURE] Make a reasonable thread pool size for listing file status

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the feature
Setting spark.hadoop.mapreduce.input.fileinputformat.list-status.num-threads=20 if unset in engine main might be a good choice
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

FYI https://github.com/apache/spark/pull/34792


Upgrade 0.15 cw20 bonding

Depends on https://github.com/CosmWasm/cosmwasm-plus/pull/304
Resolves https://github.com/CosmWasm/cosmwasm-plus/issues/307

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: orkunkl:x: uintYou have signed the CLA already but the status is still pending? Let us recheck it.


KAFKA-15951: MissingSourceTopicException should include topic names

This is minor changes!

Jira : https://issues.apache.org/jira/browse/KAFKA-15951
MissingSourceTopicException has field missingTopics to store missing topics.
The StreamsPartitionAssignor throws a MissingSourceTopicException. and it depends on result of repartitionTopics.missingSourceTopicExceptions().isEmpty(). thus, missingSourceTopicExceptions() must always contain an iterable MissingSourceTopicException, allowing for the aggregation and throwing of MissingSourceTopics.
Idea for improving more

StreamsRebalanceListener can throw MissingSourceTopicException as well. however, StreamsRebalanceListener cannot get missing topic List because StreamsRebalanceListener depends on ErrorCode AssignorError.INCOMPLETE_SOURCE_TOPIC_METADATA.code()
If StreamsRebalanceListener should include missing topic list when it throws MissingSourceTopicException as well, we can consider ThreadLocal to reach the target.
Skeleton

Set ThreadLocal to StreamThread as member field.
Put missing topics to ThreadLocal.
StreamsRebalanceListener can get missing topics through ThreadLocal..

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Gently ping, @cadonna !
I make new commit to apply your comments.
When you have some free time, please take a look. 🙇♂

Thanks for the PR. Made a pass.

Aye, i hope so. 👍

Gently ping, @mjsax .
Would you please take a look when you have some free time? 🙇♂

@mjsax thanks for your suggestion. it is very suitable 👍
I commit your suggestion. when you have free time, please take a look 🙇♂

There is a checkstyle error:
[2024-05-02T11:03:05.489Z] > Task :streams:checkstyleMain

[2024-05-02T11:03:05.489Z] [ant:checkstyle] [ERROR] /home/jenkins/workspace/Kafka_kafka-pr_PR-15573/streams/src/main/java/org/apache/kafka/streams/processor/internals/RepartitionTopics.java:113:45: '{' is not preceded with whitespace. [WhitespaceAround]

I took the liberty to commit a fix to retrigger the build.

Hi @mjsax !
Sorry, I missed to check checkStyle and thanks for your working instead of me 🙇♂

Thanks for the PR @chickenchickenlove! Merged to trunk.

Thanks for your guideline.
It was very helpful to me 🙇♂🙇♂🙇♂


Select with tab

Added tab to keydown() with same functionality as enter. When navigating in list with arrows and pressing tab, highlighted item should get selected.

@maren123 thanks for this. Can you exclude the changes in dist and I will build and include those with the release tag

@maren123 thanks for this. Can you exclude the changes in dist and I will build and include those with the release tag

Done

Thanks @maren123 - included in 2.3.0


Run CI tests for python 3.12 instead of the nonexistent 3.22

Heh, now it's slowest. Well, at least we're not losing coverage.


Apple Reminders: Bug fixes and use new Swift macro

Description

Fix a bug where it wasn't possible to remove the priority from a list item
Fix a bug where it wasn't possible to remove the due date from a list item
Fix a bug where it wasn't possible to change the due date from the menu bar
Migrate codebase to use new Swift macros

Closes #10259
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are

No, it doesn't. This will be addressed in another PR, but I don't know when it'll happen. I understand it's an important issue for you, so I encourage you to contribute to the extension in the meantime. I can provide some guidance if needed.

Will this pull request be merged soon? I can take a look at the other issue as soon as it has been merged.

I just submitted #10329.


fix(Avatar): add max-height to avatar element (#143)

After a lot of research, I believe this should be the least collateral solution to this problem. Any suggestion is welcome.

This is how it looks on my end now:


Can't find the integration anywhere

The problem
Installed according to readme. Clear cache. Still I can't find the integration anywhere...
Version of Easee integration having the issue?
v0.9.45
Version of Home Assistant Core having the issue?
2022.10.3
Anything in the logs that might be useful for us?
No

Additional information
No response

Well, you have probably missed something...

Install the integration via HACS
Restart Home Assistant
Add integration from Settings->Integrations -> Add integration (Clear browser cache - if needed)

Found it from the phone app


Crash on Windows when looading a dot file

Hi,
When I try to load a dot file (see attachment) with qvge (qvge-0.6.3-portable-64bit.zip) it crashes on Windows but it works on linux.
qvge_crash-obf.dot.txt

Hi, can you build v.0.7? Is it crashing as well?

Should have been fixed in 0.7 already.


Make buttons and emojis bigger on mobile phones

Buttons are fine, they were just displayed too small because of United Domains putting the web UI in a frame. I fixed that by using the other way of URL redirecting.
I increased the size of the emojis just a bit.


Use @border-radius-small for .label border-radius

border radius can be assigned the @border-radius-small value instead of defining it as 0.25em. Will be easier to update styles using variables.less.

Not sure whether that's right. @border-radius-small = 3px, whereas labels seem like they (might?) scale with the font size.
(Note when/if merging this: The commit message will need to be adjusted.)

Yup, not a correlation there because of the change of units.


broke

please bring it back, its just impossible without it

Has nobody seriously made a copy of this yet? I haven't maintained this for 4 years already.
I don't use Discord myself anymore, favoring more FOSS-friendly Matrix instead. So it's highly unlikely I'll ever return to maintaining this myself.
If there is a maintained version, tell me and I'll update the readme with a link to it.

man i wish


visible drop-target

It would be nice to have a visible drop-target rather than just open up a space. Depending on what's in your list, the space alone can be unclear at times. I'm thinking that we add an additional class "drop-target" onto the item before the space and style it with something like:
.sortable-item.drop-target:after
{
content: " ";
display: block;
width : 100%;
height : 50px;
border : 2px dashed yellow;
background-color: #FFF8F8;
opacity: 1;
}
In the case where the drop target is before the first item, we can add a different class "drop-target-before" and use :before CSS selector instead.
I've looked through the code and now have a reasonably good idea how most of it works. But some of it is still eluding me...
I can see how the dragged item updates its position. And that  it calls update on the group which in turn sets their positions. What I can't see is how the other item's position is calculated? Could you provide a few notes on how that works please?

Hey sorry for the slow reply (holidays). The other items get their positions from the group: https://github.com/jgwhite/ember-sortable/blob/master/addon/components/sortable-group.js#L108

I've pretty much worked out how it all works. Always nice understanding someone else's code and discovering a way to do things that you'd not thought of yourself! Also a good exercise for me to understand how an ember add-on is constructed. Whilst I've done a fair bit of ember now (all very complex apps), I've not tried to create an add-on before.
I've got something 80% working now. Although I've decided I can't use :after :before since I cannot set the height of those to match the dragged element. Instead I'm inserting a DIV. That causes the element below to be moved, which means, other than the initial translate, no other translate is required in update. Instead we just move the drop-target DIV.
I've got a small tweak to do now because the drop-target DIV does appears after where the item was originally, not where it is translated to. That's easy enough to do. Next job then is to modify my fixed drop-target height / width CSS and inject the size from the dragged item. I'll send you a preview when it's done.
Then I guess I need to do away with a hard coded DIV and look at a way to replicate the dragged item, clearing any content. So, for example, in the table example, I need to insert a TR tag as a drop-target.
One other enhancement I want to make is to set some meta data on the original item to indicate that it is being dragged. That's the original data being passed in, not the date we hold internally. This is important in my use case since we have varying sizes of items, and some are very large. So I want to render the item as a fixed size "drag helper" whilst it's being dragged. It's just easier than trying to drag sort an item that's the height of the screen! I also need that smart scrolling too since my lists are way larger than the screen...
I think I'll need to use something similar in order to inject a new item into the list for my "drag into a list" requirement... But I'll look at that in the New Year. Only got a couple of hours left to work today :-)

Fork here: https://github.com/ianbale/ember-sortable
Drop target working properly for vertical sorting. Horizontal messed up because it still has a fixed height/width for the drop-target rather than looking at the dragged item. Time to give up for the holidays though!

Awesome progress! Thanks so much for working on this. I also need to take a break from GitHub for the holidays but let's get this formalised into a pull request once the festivities have wound up. :santa:

@ianbale: Is this feature still being worked on? Would be a nice addition to this addon!

@jevanlingen : It kinda got hijacked and modified to be quite specific for the client job I mentioned.
That's not to say it couldn't be made a little more generic again without too much work. I'm happy to answer any questions about the code if you want to pm me, but it's unlikely that I will work on this again any time soon. Right now I'm not even working with Ember...

This project is now being maintained by the team at @heroku.
To get the project healthy and maintainable again, we're doing a bit of spring cleaning of older issues and pull requests. This issue hasn't been touched in a while so we're closing it. Thanks for all the great ideas and discussion.
Please feel free to re-open if you'd like to resume the conversation.


keras.layers.Conv2D with dilation_rate > 1 becomes SpaceToBatchNd-Conv2D-BatchToSpaceNd in tflite conversion

System information.

Have I written custom code (as opposed to using a stock example script provided in Keras): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.11.0
Python version: 3.8
GPU model and memory: NVIDIA 1080

Describe the problem.
tf.nn.convolution produces SpaceToBatchNd-Conv2D-BatchToSpaceNd pattern when we do tflite conversion with keras.layers.Conv2D (https://github.com/keras-team/keras/blob/master/keras/layers/convolutional/base_conv.py#L247-L263)
Describe the current behavior.
When converting functional model containing keras.layers.Conv2D which has dilation_rate > 1, the SpaceToBatchNd-Conv2D-BatchToSpaceNd appears and the pattern can be visualized with Netron.

Also, we get useless dilation_h_factor and dilation_w_factor because they are always 1.
We can fix it by replacing tf.nn.convolution with tf.nn.conv2d using custom layer but it is inconvenient if we want to further optimize the model with tfmot which encourages users to stick to built-in keras layers.
Describe the expected behavior.
keras.layers.Conv2D with dilation_rate > 1 should be converted as expected instead of SpaceToBatchNd-Conv2D-BatchToSpaceNd
Contributing.

Do you want to contribute a PR? (yes/no): yes
If yes, please read this page for instructions
Briefly describe your candidate solution(if contributing):

Just one single line modification:
class Conv2D(Conv):
    def __init__(
        ...
    ):
        super().__init__(
            ...
        )

    def convolution_op(self, inputs, kernel):
        ...

        return tf.nn.conv2d(  # => change from tf.nn.convolution
            ...
        )

Standalone code to reproduce the issue.
import tensorflow as tf

x = tf.keras.layers.Input((224, 224, 3))
y = tf.keras.layers.Conv2D(16, 3, 1, padding='same', dilation_rate=3, use_bias=False)(x)
y = tf.keras.layers.BatchNormalization()(y)
y = tf.keras.layers.ReLU()(y)

model = tf.keras.Model(x, y)
model.summary()
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
with open('model.tflite', 'wb') as f:
    f.write(tflite_model)

@james77777778 Could you please have a look at the gist in latest TF version and confirm the output?
Please feel free to raise the PR to fix the issue ?
Thank you!

@sushreebarsa
I ran the code you provided and verified that the issue still persists.

(visualization from netron)
I will try to raise the PR when I'm back from my vacation... :)


Conflicting definitions when they are identical

My use case is that I have IDL protocols generating multiple avsc files which overlap (default behavior for avro-tools-1.8.2.jar idl2schemata). When registering definitions it would be nice if the contents was checked for equality to determine if it's ok to proceed or not. Right now, any definition with duplicate FQN or aliases will abort the processing.
Relevant parts:
https://github.com/alanctgardner/gogen-avro/blob/master/types/schema.go#L71
https://github.com/alanctgardner/gogen-avro/blob/master/types/schema.go#L77

I have the same case, compile avsc files one by one will work out. however, my problem is gogen-avro will generate an identical file named "primitive.go" for every avsc file, thus the prev one will be replaced by  the later. Any idea?

@panxh @arienkock This issue should be fixed in master and in the new release 5.0.3, so you can now compile all your files at once without errors about the duplicate definitions. Thanks for your patience!

@actgardner this doesn't seem to work for enums in a different file but same namespace.
To reproduce:
file1.idl:
@namespace("myname")
protocol MyProtocol {
    enum type {
        type1,
        type2
    }
}

file2.idl
@namespace("myname")
protocol MyProtocol2 {
  record MyRecord {
      type mytype;
  }
}

For this scenario, IDL generates two avsc files, in the type.avsc file there's a "namespace: myname" attribute associated with type, but in the myrecord.avsc file there's no namespace attribute associated with type, so the deepequal check you added returns false and the error is still generated.


[C#] SinceVersion checks use type's version, not the field

When generating fields, the encodingToken is passed to the generate*Property() methods and the properties from the enclosing signalToken are ignored. In particular, using sinceVersion on a field has no effect. Any fields using primitive types will never output code from generateFieldNotPresentCondition(). Also, and misleadingly, fields using custom or composite types will output version checking code based on the sinceVersion specified in the type.

The sinceVersion on the type should be considered but it is overridden by the value on the field. The value from the field will be set in the IR based on the type if none is set on the field. The C# codes should use the value from the field. The Java and C++ codecs use the value from the field. In the past this was confused as the standard moved around on this topic.

@billsegall can you check if my change generates the correct code for you?


Reverse stack

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
Added the reverse stack code in the stack using linked list program. Improved the program.
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[x] I've edited the README.md and link to my code.

Related Issues or Pull Requests
Closes: #3546

Kindly review @HarshCasper :)


The Mapbox SDK is unnecessarily trying to fetch map tiles

Steps to reproduce

Run the PublisherExampleSwiftUI app and start publishing location for a trackable.
Look at the logs.

Observed behaviour
The logs contain lots of messages of the form 2022-10-05 14:23:17.809451-0300 PublisherExampleSwiftUI[76305:1046946] [Mapbox] [Error, nav-native]: Failed tile response:  dataset mapbox/driving-traffic version 2022_09_30-03_00_00 tile = 2/733190/0 status = Unauthorized, indicating that the Mapbox SDK was unable to download map tiles.
Expected behaviour
The Mapbox SDK should not be trying to download map tiles, given that we're not displaying a Mapbox map.
Notes
I don't know whether it's possible to tell the Mapbox SDK not to fetch map tiles.

The corresponding Jira issue is https://ably.atlassian.net/browse/SDK-2698


Replace use of django-guardian with fields on UserProfile.

As documented in https://github.com/zulip/zulip/issues/441, Guardian
has quite poor performance, and in fact almost 50% of the time spent
running the Zulip backend test suite on my laptop was inside Guardian.

As part of this migration, we also clean up the old API_SUPER_USERS
variable used to mark EMAIL_GATEWAY_BOT as an API super user; now that
permission is managed entirely via the database.

Fixes #441.

Automated message from Dropbox CLA bot
@timabbott, it looks like you've already signed the Dropbox CLA. Thanks!


[INLONG-7280][Manager] rename checkXXX to expectXXX in Preconditions

Fixes #7280

@fuweng11, for the problem of using unreasonable api for inspection in the implementation, let's modify the bill of lading separately, this bill is only for equivalent replacement

@gosonzhang please add more details about why we need to rename checkXXX to expectXXX.

@dockerzhang, the reason for the change has been explained in the issue[1]: checkxxx is just an action, not an expectation; changing the name to expectXXX will clarify what results are required

https://github.com/apache/inlong/issues/7280


update 'nested' to 'details'

Updating per https://github.com/json-schema-org/json-schema-spec/pull/1351
Ref: https://github.com/json-schema-org/json-schema-spec/issues/1347
Question:  Do I need to add an "edited" comment at the end to log the change?  What's the policy on the blog?

This post doesn't include any information on dropping detailed and renaming verbose -> hierarchical.  The focus seems to be more on the changes in structure than on the formats themselves.
I'm not sure that it's necessary (for this post) to update it to mention the name changes for the formats.

Maybe add a line somewhere that the structural changes here will be formats under new names (you can list the names or not) just so folks have a heads-up that the existing formats aren't expected to be changed (just deprecated/dropped)?  But really, I'm fine with you leaving it as-is if you prefer, so consider my "approved" to hold.  As far as I'm concerned you can merge this anytime as the spec change doesn't have to be in first.

I prefer waiting until the spec is updated.


[Question] How to copy alpha channel to RGB?

Hi @lovell, thanks for making such an awesome tool. Really appreciate all your work.
Problem
I'm using sharp for a gulp task that creates alpha masks for transparent PNGs. At first I used threshold to convert all the non-black pixels to white, but that crushes semi-transparent values and drops black values altogether:
image.clone().threshold(1).toBuffer((error, output, info) => { ... }

(Here's the full source code.)
For example, the following source.png should produce output-correct.jpg but currently produces output-incorrect.jpg:

source.png
✅ output-correct.jpg
❌ output-incorrect.jpg

Attempted Solutions
I thought I could manually update the values of toBuffer's output like this:
for (var i = 0; i < output.length; i += 4) {
  // Use alpha channel to define luminance
  output[i + 0] = output[i + 3];
  output[i + 1] = output[i + 3];
  output[i + 2] = output[i + 3];
  output[i + 3] = 255;
}

Error Messages
But I keep getting the following error:
$ gulp zorrosvg
[20:51:43] Starting 'zorrosvg'...

events.js:160
      throw er; // Unhandled 'error' event
      ^
Error: ✘ File `smoke4.png`: Input buffer contains unsupported image format

And depending on the image that error changes:
$ gulp zorrosvg
[20:57:50] Starting 'zorrosvg'...

events.js:160
      throw er; // Unhandled 'error' event
      ^
Error: ✘ File `smoke2.png`: Input buffer contains unsupported image format

(sharp:36492): GLib-CRITICAL **: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:36492): GLib-CRITICAL **: g_hash_table_insert_internal: assertion 'hash_table != NULL' failed

(sharp:36492): GLib-CRITICAL **: g_hash_table_lookup: assertion 'hash_table != NULL' failed
Segmentation fault: 11

Other Issues
I thought maybe it was similar to #680 and forcing the addition of an alpha channel would help, but that produced the same errors as above.
I also tried tweaking your joinChannel example but couldn't figure out how  it works exactly. I assume there's a way to join a layer of white pixels?
Thanks again for all your work. Appreciate the help.

Hello, if you need uncompressed pixel data then raw is your friend.
You can use it with greyscale and toColourspace('b-w') for single channel (greyscale) output.
For a RBGA input image, extractChannel(3) should give you the alpha channel.

Awesome, thanks! I'll try extractChannel(3) and close this for now. Will re-open if I have any more questions.

Just following up with what solved my problem:
Realized I was just confused about how to convert the raw buffer back to a sharp instance. @smremde's example helped me figure it out. The options.raw property was what I had missed:
sharp(buffer, { raw: { width: info.width, height: info.height, channels: info.channels } })

Here's what I ended up with:
sharp(file.contents).raw().toBuffer(function(error, buffer, info) {

  // Create a luminance mask based on the original image's alpha channel
  for (let i = 0; i < buffer.length / 2; i = i + 4) {
    let alpha = buffer[i + 3];
    buffer[i + 0] = alpha;
    buffer[i + 1] = alpha;
    buffer[i + 2] = alpha;
    buffer[i + 3] = 255;
  }

  // Compress as JPG
  sharp(buffer, { raw: { width: info.width, height: info.height, channels: info.channels } })
    .jpeg().toBuffer(function(error, finalBuffer, finalInfo) {

    // Pipe new file to gulp stream
    callback(null, new gutil.File({
      cwd: file.cwd,
      base: file.base,
      path: file.path.replace('.png', '.jpg'),
      contents: finalBuffer
    }));
  });
});

Full source code.


SSL/TLS for callback server

Is your feature request related to a problem? Please describe.
pyhomatic supports encrypted and authenticated connections to Homematic via SSL/TLS. There are several settings (e.g. port, ssl, verify_ssl, etc.). This ensures that credentials are secured from eavesdropping, as well as ensures authenticity and integrity.
However, it seems that the server that receives the responses from Homematic is always made available only via HTTP - without any encryption.
At the very least this will expose sensor/event data from Homematic to eavesdropping and other attacks.
Describe the solution you'd like
There hsould be an option to make the server hosted for receiving responses from Homematic (callbacks) available via SSL/TLS.
Describe alternatives you've considered
There is no alternative, SSL/TLS should be supported bi-directional.
Additional context
Not sure whether Homematic itself supports SSL/TLS in this capacity. If not, it needs to be requested / implemented.

The problem I see with this is that, due to the local nature, only self-signed certificates would be useful. Or a private CA. In both cases it would be required to add the CA/certificate to the trust of the CCU. I didn't find a way to do that within the CCUs UI, and doing it manually via SSH might not be the best solution as well.
So as a result of that the alternative would be, that the CCU simply doesn't verify the certificate of the XML-RPC endpoint it connects to. Of course that generally would provide an encrypted connection. But in terms of eavesdropping we wouldn't gain anything. After all anybody could MITM the connection with another certificate as it won't be verified for authenticity anyway. 🤷♂
I do see the point you're making, and I agree that it makes sense theoretically. But I doubt there would be a practical benefit.

The problem I see with this is that, due to the local nature, only self-signed certificates would be useful.

No, I'm using a public certificate from Let's Encrypt for my Homematic. And no, it is not exposed to the Internet. You can get certificates via other manes (DNSAPI, etc.) and upload them to your Homematic (Raspberrymatic).

I didn't find a way to do that within the CCUs UI, and doing it manually via SSH might not be the best solution as well.

That doesn't mean that there should not be support for it. We can request / add such a feature into Homematic. I'm uploading my certificate via SSH currently. @jens-maus seems to be quite active in this area.

But in terms of eavesdropping we wouldn't gain anything.

Well, in terms of eavesdropping (= passive) you gain a lot by only having encryption (without authentication).

After all anybody could MITM the connection with another certificate as it won't be verified for authenticity anyway. man_shrugging

You're right, but this requires an active attack (MitM), which is more difficult to do then just "listening in". It is still trivial to do for anyone with tools / skills, but its more than just capturing traffic via Wireshark. Having SSL/TLS - even without proper authentication - is still better than nothing, but obviously having proper certificate verification on both ends is even better.
Just consider the bulk data collection by NSA and other agencies. They were only able to collect everything, because it was trivial to do. I'm sure they can still actively attack specific targets, if they choose to do so (e.g. get rogue certificates from one of the many CAs), but they are no longer able to "collect everything", because most of the web is using HTTPS now.

I do see the point you're making, and I agree that it makes sense theoretically. But I doubt there would be a practical benefit.

There is a practical benefit, as we should assume anything non-encrypted to be "technical debt" and not simply accept it as "good enough". Proper SSL/TLS should be a required feature in all appliances like this, not something that is put there after the fact halfheartedly - like it currently is the case.

No, I'm using a public certificate from Let's Encrypt for my Homematic. And no, it is not exposed to the Internet. You can get certificates via other means (DNSAPI, etc.) and upload them to your Homematic (Raspberrymatic) without exposing them to the Internet. I'm doing this a lot for privately hosted appliances / assets.

Sure. But that then also requires a properly configured DNS. Which you probably have. But most users will have a generic router where they can't configure their local hosts that precisely.

That doesn't mean that there should not be support for it. We can request / add such a feature into Homematic. I'm uploading my certificate via SSH currently. @jens-maus seems to be quite active in terms of Homematic development / integration.

Well, I don't want to build something (for free) that only 2-3 people will be using as long it's not an official feature of the CCU. IF this gets added to the CCU (and not just RaspberryMatic), I would implement it. This of course is under the assumption, that with the current implementation the CCU would abort connections that don't pass the certificate verification.

Well, in terms of eavesdropping (= passive) you gain a lot by only having encryption (without authentication).

If authentication for the XML-RPC API isn't enabled (and the firewall open) on the CCU, any client can subscribe to read all the data. And if the firewall is closed, devices on the CCU can't be controlled. Fake events could be sent to the HA-side (because it doesn't require authentication), encryption doesn't help here as well. And keep in mind: HA --> CCU can be encrypted already. That's were currently credentials are transmitted (if authentication is enabled).
That being said, this all assumes your network has been compromised already. I agree that still it makes sense to encrypt. But if someone with malicious intentions is able to sniff, you actually have a bigger problem somewhere else.

You're right, but this requires an active attack (MitM), which is more difficult to do then just "listening in". It is still trivial to do for anyone with tools / skills, but its more than just capturing traffic via Wireshark.

If an attacker managed to get on your network, then this won't pose a problem for him.

Having SSL/TLS - even without proper authentication - is still better than nothing, but obviously having proper certificate verification on both ends is even better.

I agree, that the gathered information can have some value. But the effort to implement this for only a few people who would use this seems too high given all the obstacles that would need to be worked around. Adding TLS support might not be as easy as it sounds. The Server-module Python provides doesn't have support for that built in, so it would have to be applied to it manually, probably by wrapping the socket it the best case.

There is a practical benefit, as we should assume anything non-encrypted to be "technical debt" and not simply accept it as "good enough". Proper SSL/TLS should be a required feature in all appliances like this, not something that is put there after the fact halfheartedly - like it currently is the case.
Homematic seems to be a really crappy in terms of software development, privilege model and system integration. A somewhat secure setup should definitely use SSL/TLS (plus many additional things like putting the appliance outside of your actual LAN, etc.).

I agree with that as well. But again, this only makes sense if the CCU supports this in the first place.
One more thing: you also have to consider what would be accepted on the Home Assistant side. The only thing I can imagine to be accepted would be, that the certificate that is already being used for Home Assistant itself would also be passed to pyhomematic. That would already break the solution using a dedicated certificate where the hostname resolves locally, because typically these certificates contain the public hostname. So in my opinion the only real chance of this happening is, if the CCU doesn't verify certificates. And I have to say: I'd trust a CCU even less if it doesn't verify certificates.

Closing this issue. If at all, this feature should be implemented in hahomematic.


Add support for analytics.dashboards

Looks like the current version has support for analytics.reports, but not dashboards. I'll do my best to contribute.

Personally I don't have immediate need of dashboard API support, but you can cover it yourself and hopefully send pull-request.
By the way, the low-level Connection#request() can be used for any REST-based API calls in Salesforce without explicit support.


feat: Added Gateway Route Reports Integration tests for JAVA-SDK.

PR summary
The Integration Tests for JAVA-SDK needed to be updated to cover the new Gateway Route Reports APIs.
PR Checklist
Please make sure that your PR fulfills the following requirements:

[x] The commit message follows the Angular Commit Message Guidelines.
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type

[ ] Bugfix
[x] Feature
[x] Code style update (formatting, local variables)
[x] Refactoring (no functional changes, no api changes)
[x] New tests
[ ] Build/CI related changes
[ ] Documentation content changes
[ ] Other (please describe)

What is the current behavior?
All tests passing without the Route Report integration tests.
What is the new behavior?
All tests passing with the Route Report integration tests.
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information
N/A

:tada: This PR is included in version 0.23.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Circle should push images to Docker Hub

See: https://github.com/BIDS-Apps/example/blob/master/circle.yml#L45

Included the directives for pushing it to the hub.


Fix highlighting of option pragmas

This prevents the highlighter from coloring everything below as comments due to double-dash in leading OPTIONS pragmas:
{-# OPTIONS --without-K #-}
{-# OPTIONS --copatterns #-}

Prompted by @np.

It seems that I'm the only one watching this repo ._.
Hey @freebroccolo could you help us a bit

got fixed somehow ._.


Heroku deployment

Just started working on this starter.
How do you push this to heroku?
Do I have to create a prod-sever.js and configure there and then use that in a newly created Procfile?  Are there any other set ups i need to do?  Here is my current prod-server.js.
const express = require('express');
const path = require('path');
const port = process.env.PORT || 8080;

const app = express();

app.use(express.static('build'));

app.get('*', (req, res) => {
  res.sendFile(path.join(__dirname, 'build/client/index.html'));
});

app.listen(port, () => {
  console.log("Party started on port "+port);
})

Thanks

I used this template to start with a new application. To work with heroku I added some additional scripts in the package.json:
"preheroku-postbuild": "npm run clean:build", "heroku-postbuild": "webpack --config ./config/webpack.config.production.js --progress", "postheroku-postbuild": "npm run build:html", "heroku-serve": "http-server build/client -p ${PORT:=3003} --cors"
Also I added this buildpack to the heroku application settings
heroku/nodejs
Currently I'm looking to update the server since it only works with the root page.


Cannot do `initialize' in Term::ReadLine::Gnu at /usr/share/perl/5.22/perl5db.pl

Hi, I just installed the extension (in VS Code 1.33.0 @ Ubuntu 16.04, perl v5.22.1) and seeing the following error:
cd /home/user ; env PATH=/home/user/bin:/home/user/.local/bin:/home/user/bin:/home/user/.local/bin:/opt/OpenPrinting-Gutenprint/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin PERL5LIB= PERLDB_OPTS=RemotePort=localhost:41273 perl -d /home/user/file
Cannot do `initialize' in Term::ReadLine::Gnu at /usr/share/perl/5.22/perl5db.pl line 6840.
 at /usr/lib/x86_64-linux-gnu/perl5/5.22/Term/ReadLine/Gnu.pm line 807.
    Term::ReadLine::Gnu::AU::AUTOLOAD(Term::ReadLine=HASH(0x233dfd0)) called at /usr/lib/x86_64-linux-gnu/perl5/5.22/Term/ReadLine/Gnu.pm line 274
    Term::ReadLine::Gnu::new("Term::ReadLine", "perldb", IO::Socket::INET=GLOB(0x1f56470), IO::Socket::INET=GLOB(0x1f56470)) called at /usr/share/perl/5.22/perl5db.pl line 6840
    DB::setterm() called at /usr/share/perl/5.22/perl5db.pl line 1832
    DB::_DB__read_next_cmd(undef) called at /usr/share/perl/5.22/perl5db.pl line 2761
    DB::DB called at /home/user/file line 31
Cannot do `initialize' in Term::ReadLine::Gnu at /usr/share/perl/5.22/perl5db.pl line 6840.
END failed--call queue aborted at /home/user/file line 8671.
 at /home/user/file line 8671.
[u460L mysqld]$ cd /home/user/ ; env PATH=/home/user/bin:/home/user/.local/bin:/opt/OpenPrinting-Gutenprint/sbin:/opt/OpenPrinting-Gutenprint/bin:/home/user/bin:/home/user/.local/bin:/opt/OpenPrinting-Gutenprint/sbin:/opt/OpenPrinting-Gutenprint/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin PERL5LIB= PERLDB_OPTS=RemotePort=localhost:43575 perl -d /home/user/file
Cannot do `initialize' in Term::ReadLine::Gnu at /usr/share/perl/5.22/perl5db.pl line 6840.
 at /usr/lib/x86_64-linux-gnu/perl5/5.22/Term/ReadLine/Gnu.pm line 807.
    Term::ReadLine::Gnu::AU::AUTOLOAD(Term::ReadLine=HASH(0x1cf02d0)) called at /usr/lib/x86_64-linux-gnu/perl5/5.22/Term/ReadLine/Gnu.pm line 274
    Term::ReadLine::Gnu::new("Term::ReadLine", "perldb", IO::Socket::INET=GLOB(0x1908700), IO::Socket::INET=GLOB(0x1908700)) called at /usr/share/perl/5.22/perl5db.pl line 6840
    DB::setterm() called at /usr/share/perl/5.22/perl5db.pl line 1832
    DB::_DB__read_next_cmd(undef) called at /usr/share/perl/5.22/perl5db.pl line 2761
    DB::DB called at /home/user/file line 31
Cannot do `initialize' in Term::ReadLine::Gnu at /usr/share/perl/5.22/perl5db.pl line 6840.
END failed--call queue aborted at /home/user/file line 8671.
 at /home/user/file line 8671.

launch.json:
{
    "type": "perl",
    "request": "launch",
    "name": "Perl-Debug",
    "console": "integratedTerminal",
    "program": "${workspaceFolder}/${relativeFile}",
    "exec": "perl",
    "execArgs": [],
    "root": "${workspaceRoot}/",
    "inc": [],
    "args": [],
    "env": {},
    "stopOnEntry": false
}

Turns out the problem was with libterm-readline-gnu-perl module. After sudo apt remove libterm-readline-gnu-perl && sudo apt install libterm-readline-perl-perl it worked.
But now I'm observing that it doesn't stop on a breakpoint if started with "stopOnEntry": false.


Add default export

Most plugins in unified/remark/rehype use a default export, which allows them to be loaded when specifiers are unknown (e.g.m the engine).
Both a named and a specifier can be allowed, so this can be a minor

I prefer named exports, but I agree default exports fit better in the remark ecosystem. I’ve already got local breaking changes to make remark-mdx-frontmatter ESM only and compatible with the latest mdast changes.  So I can just drop the named exports in the next major release.

@remcohaszing need help with this one?

@remcohaszing need help with this one?

I appreciate the offer, but it's more a downstream issue I’m having with this.
I would like to combine this with the conversion to ESM. The main reason this is on hold, is because I use this in a code base where it needs to be imported using Jest, TypeScript, and ts-node.
Since I haven’t gotten that to work yet, I haven’t changed this yet. It's functional with named exports, but if someone really wants to use this as a default export (i.e. with unified-engine), I’ll convert to a default import..

Coincidentally, I just got bit by this again right now 😅😬

You can update the package and move to ESM while keeping on using the previous version of this package there though?

To be honest, I am in the named-export crowd 😄 just wanna to help

To be honest, I am in the named-export crowd :smile:

Me too :smile:
The only reason I want to convert my plugins to default exports is for consistency within the unified ecosystem.

just wanna to help

Cool! If you can convert this to use ESM, I’ll gladly accept a PR. Note TypeScript will need to be configured to output ESM as well.

I love named exports, I think both can be specified to get the best of both worlds?

Version 2.0.0 now uses a default export.


Check for binary in ./node_modules/.bin/ first

When you want to have a clean package, you must put dev tool list and version in devDependencies in your package.json, for example :
  "devDependencies": {
    "gulp": "^3.8.8",
    "bower": "^1.8.0"
  }

However, node-which which is used to check if the binary is present doesn't check in this folder since ./node_modules/.bin/ is unlikely to be in your path.
A workaround is to run with a prefixed export like this :
PATH=$PATH:./node_modules/.bin/ ./node_modules/.bin/gulp

But I think it would be a better solution to check in .node_modules/.bin/

https://github.com/slushjs/gulp-install/pull/51


integrate fbs test instead of pytest

Self explanatory.


Remove obsolete error handling in XPathContext

Resolves https://github.com/crystal-lang/crystal/issues/13036

Yeah, that's probably obsolete. But it's unrelated to this change.


Fix release_suffix behaviour

Fixes packit#1695
The actual behaviour table: https://gist.github.com/majamassarini/f2ebf6a78ca8af189208e77074a16dff
RELEASE NOTES BEGIN
Version and revision are no more updated in  SPEC files when the --no-bump flag is specified and None revisions are no more created.
RELEASE NOTES END

/packit build

/recheck


Missing sample code snippet - Webpack Dev Middleware

Issue description
Missing sample code snippet from Webpack Dev Middleware - Configuration section.
https://docs.microsoft.com/en-us/aspnet/core/client-side/spa-services?view=aspnetcore-2.2#webpack-dev-middleware
Also missing the code sample in the Routing  Configuration section.
https://docs.microsoft.com/en-us/aspnet/core/client-side/spa-services?view=aspnetcore-2.2#routing-helpers

Hello @dliabenow ... Take a look at the live topic. Code samples don't render on the GH previews of the docs.
https://docs.microsoft.com/aspnet/core/client-side/spa-services

My bad ..... I see it. I'll fix it up. Thanks for letting us know @dliabenow.


Add document list page

Add a list of document sorted by:

date made
most active
a - z
z - a

and a search feature

said search feature will be in the nav bar


looks like it wants to kill the bluetooth

Hello,
I'm trying to run the integration on my rasp4, I have installed the bluetooth and lib of bluetooth as you said, turned the bluetooth service on, added the bulbs inside the HA and I get this error, familiar to you?
021-02-08 19:59:21 ERROR (MainThread) [custom_components.awox.scanner] Failed: Command 'PATH=/usr/sbin:$PATH; rfkill unblock bluetooth' returned non-zero exit status 1. Traceback (most recent call last): File "/config/custom_components/awox/scanner.py", line 38, in async_find_devices bl = await hass.async_add_executor_job(init) File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/awox/scanner.py", line 34, in init return Bluetoothctl() File "/config/custom_components/awox/bluetoothctl.py", line 19, in __init__ subprocess.check_output("PATH=/usr/sbin:$PATH; rfkill unblock bluetooth", shell=True) File "/usr/local/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/local/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'PATH=/usr/sbin:$PATH; rfkill unblock bluetooth' returned non-zero exit status 1. 2021-02-08 19:59:22 ERROR (MainThread) [custom_components.awox.awox_mesh] Error fetching awox data: No device connected
Model: Eglo ESpot 225
Edit: I'm running HA on docker.

@gstefanov Have you found a solution to this? I have been experiencing the same issue, I'm not sure what I'm doing wrong. @fsaris
I'm running on CasaOS, so HA is inside of a docker container.
Executing the commands manually inside the container gives no errors:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.awox.scanner
Source: custom_components/awox/bluetoothctl.py:18
Integration: AwoX Mesh Beleuchtung (documentation, issues)
First occurred: 19:20:47 (12 occurrences)
Last logged: 19:26:33

Find devices process error: Command 'PATH=/usr/sbin:$PATH; rfkill unblock bluetooth' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/config/custom_components/awox/scanner.py", line 39, in async_find_devices
    bl = await hass.async_add_executor_job(init)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/awox/scanner.py", line 35, in init
    return Bluetoothctl()
  File "/config/custom_components/awox/bluetoothctl.py", line 18, in __init__
    subprocess.check_output("PATH=/usr/sbin:$PATH; rfkill unblock bluetooth", shell=True)
  File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'PATH=/usr/sbin:$PATH; rfkill unblock bluetooth' returned non-zero exit status 1.

Hey, no, I moved away from eglo stuff or bluetooth in general. I'm sorry I can't help you.

@fsaris Would you be able to say something about this?

No sorry, not sure what happens here.
I can recommend to switch to a dedicated esp32 device because currently the Bluetooth for devices that don't follow the BLE specs isn't working that great with HA. https://github.com/fsaris/EspHome-AwoX-BLE-mesh-hub


The server creation script needs some tuning

These are things that I can work around. The warnings about the future release seem to indicate the script may already be able to use the future version of the command?
Following warnings appeared with server creation script as well as an error. Maybe the script or instructions can make this even tighter. Here is the output I got:
VERBOSE: Importing alias 'List-AzureRmApplicationGatewayAvailableWafRuleSets'.
WARNING: The output object type of this cmdlet will be modified in a future release.
WARNING: The output object type of this cmdlet will be modified in a future release.
WARNING: The output object type of this cmdlet will be modified in a future release.
WARNING: The output object type of this cmdlet will be modified in a future release.
VERBOSE: Loading module from path 'C:\Program Files\WindowsPowerShell\Modules\AzureRM.Compute\5.2.0\Microsoft.
VERBOSE: Importing alias 'Get-AzureRmVmssVMDiskEncryptionStatus'.
WARNING: Since the VM is created using premium storage or managed disk, existing standard storage account, aud
itstoragelogs, is used for boot diagnostics.
New-AzureRmVM : Windows computer name cannot be more than 15 characters long, be entirely numeric, or
contain the following characters: ~ ! @ # $ % ^ & * ( ) = + _ [ ] { } \ | ; : . ' " , < > / ?. ErrorCode: InvalidParameter ErrorMessage: Windows computer name cannot be more than 15 characters long, be entirely numeric, or contain  the following characters: ~ ! @ # $ % ^ & * ( ) = + _ [ ] { } \ | ; : . ' " , < > / ?.
StatusCode: 400
ReasonPhrase: Bad Request
OperationID : 36b13e7a-34d7-4b42-b59f-ed45132a4f35
At line:43 char:2

New-AzureRmVM -ResourceGroupName $resourceGroup -Location $location  ...

 + CategoryInfo          : CloseError: (:) [New-AzureRmVM], ComputeCloudException
 + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.NewAzureVMCommand

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ad37c582-22e6-6c1a-522a-0683aa3cef2c
Version Independent ID: 0ffd6528-6f5c-107a-2a04-b19d0c807d15
Content: Quickstart - Encrypt a Windows IaaS VM with Azure PowerShell
Content Source: articles/security/quick-encrypt-vm-powershell.md
Service: security
GitHub Login: @mestew
Microsoft Alias: mstewart

@blouwagie Thanks for your feedback! We will investigate and update as appropriate.

Hey @blouwagie. Looks like you might be running version 5.2 for the AzureRM.Compute module (usually goes with AzureRM version 6.4). The scripts should work ok with that version aside from the warnings. I do know that there was an improvements to Get-AzureRmVmssVMDiskEncryptionStatus in 5.5 for Compute in case you are using scale sets in your environment.
I'm currently running AzureRM v6.13.1/AzureRM.Compute v5.9.1.  I get the FYI warning that the parameter EnableVmProtection was dropped from New-AzureRmVirtualNetwork. My VM still builds ok from the script since we aren't using the parameter. Let me save you a possible future headache by letting you know these scripts haven't been tested/reworked for the new PowerShell Az module yet. I definitely plan on updating the documentation once that process is complete.
Thanks,
Meghan

Hi Meghan,
Thank you for the response!
I am running version 6.4 of azureRM.
AzureRM 6.4.0   C:\Program
Files\WindowsPowerShell\Modules\AzureRM\6.4.0\AzureRM.psd1
I never got your scripts to work but found a great alternative discussed at
https://github.com/Azure/azure-powershell/issues/1833
The article links to this one that works:
https://github.com/Azure/azure-powershell/blob/master/src/ResourceManager/Compute/Commands.Compute/Extension/AzureDiskEncryption/Scripts/AzureDiskEncryptionPreRequisiteSetup.ps1
Regards
Bart
Bart Louwagie
Holmes, NY
<EMAIL_ADDRESS>845 202 0239 (google voice)
http://louwagie.com
https://www.linkedin.com/in/bart-louwagie/
(when conference call https://www.uberconference.com/bart40 is needed)
On Fri, Jan 4, 2019 at 4:36 PM Meghan Stewart<EMAIL_ADDRESS>wrote:

Hey @blouwagie https://github.com/blouwagie. Looks like you might be
running version 5.2 for the AzureRM.Compute module (usually goes with
AzureRM version 6.4). The scripts should work ok with that version aside
from the warnings. I do know that there was an improvements to
Get-AzureRmVmssVMDiskEncryptionStatus in 5.5 for Compute in case you are
using scale sets in your environment.
I'm currently running AzureRM v6.13.1/AzureRM.Compute v5.9.1. I get the
FYI warning that the parameter EnableVmProtection was dropped from
New-AzureRmVirtualNetwork. My VM still builds ok from the script since we
aren't using the parameter. Let me save you a possible future headache by
letting you know these scripts haven't been tested/reworked for the new
PowerShell Az module yet. I definitely plan on updating the documentation
once that process is complete.
Thanks,
Meghan
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/21668#issuecomment-451575619,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ASWZ9ZVV21YQuXl2TeDns4RveyKV5q4Yks5u_8lRgaJpZM4ZqAqb
.


Add stage for pathway name conventions

See also edits made by Eric Weitz; perhaps combine with adding PW ontology?

@DeniseSl22 : What about if we add a step to the last two steps in pathway curation tracks (purple and orange) to specify a name that follows the convention before upload? Seems more consistent than to have them upload something with an incorrect title just to change it later.
For example:

Add a step for changing the pathway name in PathVisio here: https://new.wikipathways.org/academy/stages/met-1-last/index.html (before the "save local file" step)
Add a similar step here: https://new.wikipathways.org/academy/stages/fig-met-1-last/index.html

Yes, great idea! I do think the https://new.wikipathways.org/academy/stages/met-1-last/index.html step would fit best, although it already has a lot of info people need to follow... And, we also don't have guidelines on what we want for the names of PWs, right? I could have a talk with @eweitz about this.

Hi @eweitz , thanks for contributing here.... If I'm honest, I have never seen this page with "pathway name guidelines" :see_no_evil: ; thank you for pointing it out! And, I believe this page is not completely covered in the Academy, @khanspers . So, we could check which elements would make nice additions.... but, some of these items will change with a new PV release, so we might want to only index which steps are missing, and then add them to the academy only after we know how to model this in PV4.

Added to met-1-last


pairing with a JavaScript library?

I'd love to use this with a JavaScript front-end. Any recommendations for having it work with a JavaScript client over HTTP?
Also, are you basing this library on CvRDTs (convergent replicated data types) or CmRDTs (commutative replicated data types)?

Right now there's no JS counterpart to Ditto. However, it should be possible to compile Ditto to WebAssembly with Rust's wasm32-unknown-unknown target, and one of my medium-term goals is to write a more ergonomic JS wrapper for the wasm output.
Ditto CRDTs are hybrids, that is, both CvRDTs and CmRDTs. You can replicate ops or changes as you see fit (via execute_op or via merge). Each form of replication comes with its own advantages and drawbacks. Op-based replication is typically more bandwidth-efficient but forces some restrictions on ordering (more in the README). State-based replication via state is typically less efficient, but it has no ordering restrictions.

I'm very much interested in helping to contribute a JS-WASM effort!


Schema Compare slow and then fails

Type: Bug
Using Schema compare on an object is quite slow compared to ADS version 1.44 (Which the extension no longer works for)
Previously clicking the "compare" button would show results within a few minutes, now it can take over 30 minutes. Once compare is complete and changes are selected choosing the "apply" option has no effect or output with the button becoming possible to press after a few minutes.
Expectation is, compare loads and then apply will update the project with the changes.
The --trustserver flag was testing to see if helped it at all but it did not. Same outcome without it.
Extension version: 1.21.0
Azure Data Studio version: azuredatastudio 1.46.1 (ba29842b81dec01177415e53948ca2168e69c3f8, 2023-10-02T18:14:22.887Z)
OS version: Windows_NT x64 10.0.22621
Restricted Mode: No
Preview Features: Enabled
Modes:

System Info

Item
Value

CPUs
12th Gen Intel(R) Core(TM) i7-1255U (12 x 2611)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
undefined

Memory (System)
31.64GB (18.50GB free)

Process Argv
--trust-server-certificate

Screen Reader
yes

VM
0%

@Whalee110 was schema compare also slow in ADS 1.45.1?
If needed, you can download older versions of the schema compare extension and manually install it, which will work with ADS 1.44 ex: https://sqlopsextensions.blob.core.windows.net/extensions/schema-compare/schema-compare-1.19.0.vsix

Thank you for that, that will help alot - The extension that upgraded though was SQL Projects - Do you have a link for that one too?
The Schema Compare extension did seem to break where it cut off most of the output window so I could only "see" 1 row at a time, but I believe that is more likely from extensions updates so I will give that a try.
If there are any logs I can provide for info on the above let me know! I couldn't find any info except the internal logs saying "check logs"

@Whalee110 If you are on 1.3.1 of SQL Projects and want to test with the older version, you can try: https://sqlopsextensions.blob.core.windows.net/extensions/sql-database-projects/sql-database-projects-1.2.0.vsix to access it. It is compatible with 1.44

An update that may help.
I downgraded ADS to 1.45.1 and downgraded all extensions to work, Schema Compare 1.19.0 and Database Projects 1.1.1 - This has no change in the performance in running Schema Compare within database projects still took well over 30 minutes and then would not function when attempting to "apply"
Downgrading ADS to 1.44.1 and Schema Compare 1.19.0 with Database Projects 1.2.0 completes the compare within a few minutes as expected but the "apply" is still not functioning in updating the project. I am going to attempt another clean install then try again.

So Schema Compare taking excessive time seems to be tied to ADS 1.45.1 +
Apply not working based on developer tool console seems to occur when there is an error during the compare selecting what to include etc. It then gives this output when you attempt to hit apply. (This is when I have loaded a .scmp files or started from scratch but have selected and deselected what you want to update)
 [[object Object]]Cannot read properties of undefined (reading 'sourceValue')
If I don't deselect any items the apply works as intended.

@Whalee110 as for the apply not working- this is currently a known issue where include/excluding changes takes some time in the background (there is no visual notification of if the include/exclude calculation has finalized). So, waiting out a little bit (a couple of mins may be depending on how big the schema is) and then trying Apply should ideally work.
Do let us know if that doesn't work. Thanks!

I am currently using 1.44, Database projects 1.1.1 and Schema compare 1.19.0. This combination functions as intended.
Database projects 1.3.1 / Schema compare 1.21.0 breaks the compare, within ADS 1.44 the compare window becomes incredibly small only showing at most 1 row at a time in the "Selection" and you cannot adjust it's size. (This may be due to the change in Schema compare that made the window resizable? - that was only other issue I saw similar to it)

Updating to 1.45 or 1.46 fixes that but then the compare only portion goes from a minute or so to take 30 minutes.
The apply does take a big chunk of time with no visual in both, so that itself isn't too bad but the 30 minute wait just for the compare is a huge impact.

The DacFx version used by schema compare was updated in insiders and some performance slowdowns were fixed - @Whalee110 can you try using the latest insiders release and see if it's any better?
If not, then there a couple other things we'll ask you to try to help pinpoint where this slowdown was introduced in the DacFx codebase.

Yep I will give the insider release a go, can't do right now because of work but I'll do asap and provide an update.
If there are tools or the like that would help point to the issue I can do that too :)

thanks for the update @Whalee110!
Here's the other tool to try out, in case insiders is still much slower than the 1.44 release. For some context, schema compare uses DacFx in the backend to do the comparison. The command line tool sqlpackage also uses DacFx, with the script action being similar to a schema compare generate script.

ADS release 1.44 is using a DacFx version just before the Sqlpackage 162.0.52 version
Current ADS insiders (1.47-insiders) is using the same DacFx version as SqlPackage 162.1.167

If you could try doing the sqlpackage script action with these two version of sqlpackage and see if there's any significant difference in the time it takes, then that would be very helpful. Also use the /df parameter to collect logs, so that this will log where time is spent. Note that sqlpackage only works on databases and dacpacs, not currently for sql projects.

Sorry I haven't provided update. We are currently undergoing a migration and strict timelines doesn't allow too much time to try the various packages / updates.
I will try get to it this week

We are currently experiencing the issue with our biggest project (schema compare hangs, and ADS crashes after consuming 12GB+ memory.
it there updates on planned release dates and/or possible workaround ?

thanks for the update @Whalee110!
Here's the other tool to try out, in case insiders is still much slower than the 1.44 release. For some context, schema compare uses DacFx in the backend to do the comparison. The command line tool sqlpackage also uses DacFx, with the script action being similar to a schema compare generate script.

ADS release 1.44 is using a DacFx version just before the Sqlpackage 162.0.52 version
Current ADS insiders (1.47-insiders) is using the same DacFx version as SqlPackage 162.1.167

If you could try doing the sqlpackage script action with these two version of sqlpackage and see if there's any significant difference in the time it takes, then that would be very helpful. Also use the /df parameter to collect logs, so that this will log where time is spent. Note that sqlpackage only works on databases and dacpacs, not currently for sql projects.

Apologies for the delay - I tried the insider version however there was no change in outcome. Still an excessive time to run the compare.
I will try doing it with logs when time allows!

My experience is with the latest version of ADS with the latest version of Schema Compare still doesn't works.
Years (!) ago out dev and production database compare took about several minute.
Now the same process still in progress in the las 6 hours. Not acceptable.
Version: 1.48.0 (system setup)
Commit:<PHONE_NUMBER>ef8254b7c22a5dc55af7f8a1dea93f
Date: 2024-02-27T00:05:08.293Z
VS Code: 1.82.0
Electron: 25.9.8
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22631

I am also on ADS 1.48.0 and schema compare 1.21.0, and I am getting an error "Apply schema compare changes failed....Error: index and length must refer to a location within the string. (Parameter 'length')" when I apply changes from the database to the project. This is covered in issue #22957. Is there a recommendation for an older version of schema compare that will work with ADS 1.48?

So recently I updated ADS and Database projects / Schema compare to test and it works again. Clicking apply still returns a build error but that has always been the case, it now completes in a reasonable time.
For those still with issues these are the versions I am using:
Azure Data Studio
Version: 1.48.1 (system setup)
Commit: 8825ec21e67a34ede00e4f02a4afc692eb6cab61
Date: 2024-06-13T21:07:19.253Z
VS Code: 1.82.0
Electron: 25.9.8
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22631
Extensions
Microsoft.sql-database-projects  1.4.2
Microsoft.schema-compare 1.21.0
Microsoft.agent 0.49.0
Microsoft.mssql 0.1.0
I reverted the change, but in case it was something that helped I did first use the experimental setting "affinity" for extensions - setting mssql and schema compare to 1


I/flutter (10738): Document number hash mismatch. If you think this is a mistake, please file an issue https://github.com/olexale/mrz_parser/issues

Hi,
thanks in advance,
I'm trying to read Mauritanian ID mrz with your plugin, But i got the error above,
So can you help me please.

Hi @henouni,
Thanks for raising this!
It may be that Mauritanian ID documents have special rules that are not covered. Can you elaborate more on the input and the expected/actual output?

Hey folks. I'm closing the issue. Feel free to re-open it with additional details.


 You can kill (or try to) spamming kill button at intro cutscene

fixed in 0.3.9

fixed in 0.3.9


Use grcov to generate lcov report

Codecov Report

Merging #395 (66946bd) into main (080efe7) will increase coverage by 2.39%.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##             main     #395      +/-   ##
==========================================
+ Coverage   89.79%   92.19%   +2.39%
==========================================
  Files          41       51      +10
  Lines        6557    14860    +8303
==========================================
+ Hits         5888    13700    +7812
- Misses        669     1160     +491

Impacted Files
Coverage Δ

crates/crochet_infer/src/unify.rs
91.52% <92.30%> (+0.08%)
:arrow_up:

crates/crochet_infer/src/lib.rs
95.93% <100.00%> (-2.54%)
:arrow_down:

crates/crochet_cli/src/main.rs
33.33% <0.00%> (-16.67%)
:arrow_down:

crates/crochet_parser/src/parse_error.rs
53.84% <0.00%> (-12.83%)
:arrow_down:

crates/crochet_infer/src/type_error.rs
60.41% <0.00%> (-3.22%)
:arrow_down:

crates/crochet/tests/fixture_tests.rs
72.97% <0.00%> (-2.96%)
:arrow_down:

crates/crochet_codegen/src/d_ts.rs
76.59% <0.00%> (-0.70%)
:arrow_down:

crates/crochet_parser/src/lib.rs
94.38% <0.00%> (-0.67%)
:arrow_down:

crates/crochet_infer/src/expand_type.rs
80.50% <0.00%> (-0.23%)
:arrow_down:

... and 43 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add iOS 15 iPad UI tests to engine

Xcode 13 beta supports XCTest simulating right clicking on iPadOS in UI tests.  See https://developer.apple.com/videos/play/wwdc2021/10208/?time=194
Add a Scenario app test scheme for iPad to run on iPadOS 15, allowing iPad input interactions like https://github.com/flutter/engine/pull/27019 to be testable.

Blocked by https://github.com/flutter/flutter/issues/85555