LVGL9.2 ESP32 IDF build Error

Hello! I ported eez-framework to esp32 idf architecture and compiled it with lvgl9.2, idf5.4.0 and it reported an error with two main parts:
The first part of the error is as follows:

error: invalid use of incomplete type 'lv_roller_t' {aka 'struct lv_roller_t'}
300 | size_t numPages = roller->inf_page_cnt;
The second part reports an error:
2, /eez-framework/src/eez/flow/components/lvgl.cpp:224:63: error: invalid conversion from 'lv_state_t' {aka 'short unsigned int'} to 'lv_obj_ flag_t' [-fpermissive]
224 | if (booleanValue) lv_obj_add_flag(target, flag);
| ^~~~
| ^~~~
| lv_state_t {aka short unsigned int}

components/eez-framework/src/eez/flow/components/lvgl.cpp:225:52: error: invalid conversion from 'lv_state_t' {aka 'short unsigned int'} to ' lv_obj_flag_t' [-fpermissive]
225 | else lv_obj_clear_flag(target, flag); | lv_obj_clear_flag(target, flag); lv_obj_flag(target, flag)
| ^~~~
| ^~~~
| lv_state_t {aka short unsigned int}
I can't fix the first part of the error, I don't know where it's caused by not setting the right settings, the second part of the error looks like the book data type doesn't match, I forced the type conversion so it won't report the error, forced the type code: if (booleanValue) lv_obj_add_flag(target, (lv_obj_flag_t)flag); else lv_obj_clear_flag(target, flag); ^~~~ | | lv_state_t {aka short unsigned int}
else lv_obj_clear_flag(target, (lv_obj_flag_t)flag);
Translated with DeepL.com (free version)
I compiled the cmake configuration as follows
`if(ESP_PLATFORM)
file(GLOB_RECURSE SOURCES
./src/eez/.cpp
./src/eez/.c
)
idf_component_register(
    SRCS ${SOURCES}
    INCLUDE_DIRS ./src ./src/eez/libs/agg
    REQUIRES lvgl__lvgl
)

target_compile_definitions(
    ${COMPONENT_LIB}
    PUBLIC EEZ_FOR_LVGL LV_LVGL_H_INCLUDE_SIMPLE)

target_compile_options(
    ${COMPONENT_LIB}
    PRIVATE -Wno-error=dangling-pointer)

else()
cmake_minimum_required(VERSION 3.12)
include_directories(
    ./src
    ./src/eez/libs/agg
)

file(GLOB_RECURSE SOURCES
    ./src/eez/*.cpp
    ./src/eez/*.c
)

ADD_LIBRARY(eez-framework STATIC ${SOURCES})

target_include_directories(eez-framework SYSTEM PUBLIC ./src ./src/eez/libs/agg)

endif()
`
Under lvgl 8.4 I followed this cmake and it works fine

You should update eez-framework. Or you don't need to use eez-framework at all if you enable this option in Settings - Build:

For more information check this.

You should update eez-framework.
Or you don't need to use eez-framework at all if you enable this option in Settings - Build:

For more information check this.

I did what you did and it still reports an error

You don't have the latest version of eez-framework! Your version:

Latest version:

https://github.com/eez-open/eez-framework/blob/master/src/eez/flow/components/lvgl.cpp#L222

You don't have the latest version of eez-framework! Your version:

Latest version:

https://github.com/eez-open/eez-framework/blob/master/src/eez/flow/components/lvgl.cpp#L222

Thank you very much for your help, I fixed it indeed did not download and update to the latest library, it is strange that I use vscode git to switch branches but can not switch to the latest, I directly download can go to the latest, it is possible that my network problem!
Here I got an error compiling as well, I got an error in component->type ! = defs_v3::COMPONENT_TYPE_USER_WIDGET_WIDGET || component->type >= defs_v3::FIRST_DASHBOARD_WIDGET_COMPONENT_TYPE
I added a bracket '()' and it worked, I don't know if it's a bug or just a difference in the compilation system that caused the error.

Your fix is not correct, it should be:

I fixed this in eez-framework.

Your fix is not correct, it should be:

I fixed this in eez-framework.

Your fix is not correct, it should be:

I fixed this in eez-framework.

Okay, I'll change it your way.

BTW Normally, this is not an error, but because of -Werror=parentheses, warning is promoted to the error.

We had report of similar case recently.
In C/C++ following is the same: a && b || c and (a && b) || c - and both are correct. Operator && has higher priority than ||. By habit, I never use parentheses for such cases.

顺便说一句，通常这不是一个错误，但是由于-Werror=parentheses，警告被升级为错误。

我们最近收到了类似案件的报告。
在 C/C++ 中，以下情况相同：a && b || c和(a && b) || c- 都是正确的。运算符 && 的优先级高于 ||。出于习惯，我从不对此类情况使用括号。
Ok, thank you very much for your reply, I've taken care of it now and can happily play with EEZ Studio!

BTW Normally, this is not an error, but because of -Werror=parentheses, warning is promoted to the error.

We had report of similar case recently.
In C/C++ following is the same: a && b || c and (a && b) || c - and both are correct. Operator && has higher priority than ||. By habit, I never use parentheses for such cases.

Ok, thank you very much for your reply, I've taken care of it now and can happily play with EEZ Studio!


Namespaces and sandboxing

Included here is the implementation, tests, and docs for namespaces and sandboxing.
Namespaces are an extension to the ability to pass true to server.services(true) in order to access all services (from the perspective of the root server).  Now you may also pass a plugin name as a string server.services('my-plugin') in order to access services from the perspective of that plugin.  In other words, calling server.services() inside my-plugin is the same as calling server.services('my-plugin') from any plugin; identically calling server.services() on the root server is the same as calling server.services(true) from any plugin.
Sandboxing is a mechanism to opt-out of services being available to all ancestor plugins/namespaces of the plugin that registered the service.  This essentially is the schmervice-y way of having services that are "private" within the plugin that registered them.
These features work together to enable better plugin encapsulation and greater testability, with the ability to not only access, but also mock-out services on a per-plugin basis.

Coverage remained the same at 100.0% when pulling 93e0fc94ede9d519742d863d7996b004567e592b on namespaces-and-sandbox into 6a409f198e21c014003482c667a92fdc22927987 on master.


Fix ensureDeviceLocale to work with API 23

Android 6.0 (API 23) uses 'persist.sys.locale' instead of 'persist.sys.language' and 'persist.sys.country' properties to store device language settings.
I have modified ensureDeviceLocale method to use setDeviceLanguage/Country or setDeviceLocale, depending on the API level.
This PR requires the new methods added in this appium-adb PR: https://github.com/appium/appium-adb/pull/143

I published your changes in appium-adb 2.5.0, and the tests here pass now.
Can you update package.json to make the minimum version of appium-adb be 2.5.0, and re-push?
Thanks for this work!

Updated!

Nice! Thanks!


Add possibility to run rosa-cross-dc deployment from KC repository an…

…d branch
Here is an example run of the changes in GHA: https://github.com/mhajas/keycloak-benchmark/actions/runs/7610623422/job/20724345702 for release/23.0 branch from upstream Keycloak.
Closes #522
Signed-off-by: Michal Hajas<EMAIL_ADDRESS>

it works, thanks for guiding me through this @mhajas.

task: [utils:patch-keycloak-operator-image] (test -n "image-registry.openshift-image-registry.svc:5000/kamesh-keycloak/keycloak-operator:latest" && KUBECONFIG="/Users/kameswararaoakella/.kube/config" kubectl patch deployment keycloak-operator -n kamesh-keycloak --type json -p '[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value": "image-registry.openshift-image-registry.svc:5000/kamesh-keycloak/keycloak-operator:latest"}]') || true


Fixed #30772 -- Optimized make_template_fragment_key().

Also on Trac: https://code.djangoproject.com/ticket/30772#ticket
The make_template_fragment_key function in django.core.cache.utils has the following (minor) issues:

Using urllib.quote for vary_on args, is not needed any more - it was originally added to make the unhashed strings safe to send to memcached and similar restricted systems.  But since the value is hashed, this is now adding nothing. (See https://github.com/django/django/commit/ebc1325721e43808cef4334edaffc23a43f86614#diff-702b69be0100a594fd6fea1e4ab2feb1).

Use of the MD5 hashing function is disallowed on certain (odd) systems, not being FIPS compliant. See (https://github.com/django/django/pull/10605).

Creates a string of all joined vary_on args to send to the hashing function, rather than using the hashlib .update() method - for very long strings, this can be slow and use a lot of memory.

Here is a version solving these, switching to SHA256, and speeding up the function quite a bit:
https://github.com/danthedeckie/django/tree/simplified_make_template_fragment_key
And here's the repo showing performance improvement:
https://github.com/danthedeckie/make_template_fragment_key_test
Which seems to be faster in every case.
The downside of this is that the cache key is now different from before.  The tests have been updated to the new values.
The current PR now sticks with MD5, but with a cleaned up function, following @claudep 's suggestion to do all hash function replacements as a separate PR.
Thoughts?

This should be reviewed soon by one of the Django fellows, just patient some days...

@danthedeckie Thanks for this patch :+1:. I removed some unrelated changes, restored test_proper_escaping() and added release notes.


[Security] Intrusion Detection and Prevention System.

Intrusion Detection and Prevention System.
Introduction
Intrusion Detection Systems (IDSs) will be obsolete very soon (if they aren't already). In its place is something much more capable, an Intrusion Prevention System (IPS). IPSs are not a new technology, they are simply an evolved version of IDS. IPSs combine IDSs and improved firewall technologies, they make access control decisions based on application content, rather than IP address or ports as traditional firewalls had done. Because IDS and IPS technologies offer many of the same capabilities, administrators can usually disable prevention features in IPS products, causing them to function as IDSs.
This document has been created for computer security staff and program managers, computer security incident response teams (CSIRT), and system and network administrators who are responsible for managing or monitoring IDPS technologies. This document does not assume that the reader has previous experience with any IDPS technologies, but it does assume that the reader has experience with information security.
Table of Contents

Introduction.
Definitions.
Uses of IDPS.
Key Functions of IDPS.
Types of IDPS.
Before evaluating IDPS products.
References.
Conclusion.

Definitions
Intrusions: - attempts to compromise the confidentiality, integrity, availability, or to bypass the security mechanisms of a computer system or network (illegal access).
Intrusion detection: - is the process of monitoring the events occurring in a computer system or network and analyzing them for signs of possible intrusions (incidents).
Intrusion detection system (IDS): - is software that automates the intrusion detection process. The primary responsibility of an IDS is to detect unwanted and malicious activities.
Intrusion prevention system (IPS): - is software that has all the capabilities of an intrusion detection system and can also attempt to stop possible incidents.
Uses of IDPS
It’s a dire fact that while every enterprise has a firewall, most still suffer from network security problems. IT professionals are acutely aware of the need for additional protective technologies, and network equipment vendors are anxious to fill in the gap.
Intrusion Prevention Systems have been promoted as cost-effective ways to block malicious traffic, to detect and contain worm and virus threats, to serve as a network monitoring point, to assist in compliance requirements, and to act as a network sanitizing agent.
IDPSs are primarily focused on: -
•	Identifying possible incidents, logging information about them, attempting to stop them, and reporting them to security administrators.
•	Identifying problems with security policies
•	Documenting existing threats
•	Deterring individuals from violating security policies.
Key Functions of IDPS

Recording information related to observed events: - Information is usually recorded locally, and might also be sent to separate systems such as centralized logging servers, security information and event management (SIEM) solutions, and enterprise management systems.

Notifying security administrators of important observed events: - This notification, known as an alert, may take the form of audible
signals, e-mails, pager notifications, or log entries. A notification message typically includes only basic information regarding an event; administrators need to access the IDPS for additional information.

Producing reports: - Reports summarize the monitored events or provide details on particular events of interest.
An IDPS might also alter the settings for when certain alerts are triggered or what priority should be assigned to subsequent alerts after a particular threat is detected.
IPSs respond to a detected threat by attempting to prevent it from succeeding. They use several response techniques:

The IPS stops the attack itself: -
Examples: -
•	Terminate the network connection or user session that is being used for the attack.
•	Block access to the target (or possibly other likely targets) from the offending user account, IP address, or other attacker attribute.
•	Block all access to the targeted host, service, application, or other resource.

The IPS changes the security environment: - The IPS could change the configuration of other security controls to disrupt an attack. Such as reconfiguring a network device (e.g., firewall, router, switch) to block access from the attacker or to the target, and altering a host-based firewall on a target to block incoming attacks. Some IPSs can even cause patches to be applied to a host if the IPS detects that the host has vulnerabilities.

The IPS changes the attack’s content: - Some IPS technologies can remove or replace malicious portions of an attack to make it benign.
Example: - an IPS removing an infected file attachment from an e-mail and then permitting the cleaned email to reach its recipient.

Most IDPSs also offer features that compensate for the use of common evasion techniques: -
Evasion: - is modifying the format or timing of malicious activity so that its appearance changes but its effect is the same. Attackers use evasion techniques to try to prevent IDPSs from detecting their attacks.
Example: - an attacker could encode text characters in a particular way, knowing that the target understands the encoding and hoping that any monitoring IDPSs do not. Most IDPSs can overcome common evasion techniques by duplicating special processing performed by the targets. If the IDPS can “see” the activity in the same way that the target would, then evasion techniques will generally be unsuccessful at hiding attacks.

Detection Methodologies

Signature-based: - compares known threat signatures to observed events to identify incidents.
This is very effective at detecting known threats but largely ineffective at detecting unknown threats and many variants on known threats.
Signature-based detection cannot track and understand the state of complex communications, so it cannot detect most attacks that comprise multiple events.
Examples: -
•	A telnet attempt with a username of “root”, which is a violation of an organization’s security policy
•	An e-mail with a subject of “Free pictures!” and an attachment filename of “freepics.exe”, which are characteristics of a known form of malware

Anomaly-based detection: - sample network activity to compare to traffic that is known to be normal.
When measured activity is outside baseline parameters or clipping level, IDPS will trigger an alert.
Anomaly-based detection can detect new types of attacks.
Requires much more overhead and processing capacity than signature-based.
May generate many false positives.
Example: - a profile for a network might show that Web activity comprises an average of 13% of network bandwidth at the Internet border during typical workday hours. The IDPS then uses statistical methods to compare the characteristics of current activity to thresholds related to the profile, such as detecting when Web activity comprises significantly more bandwidth than expected and alerting an administrator of the anomaly. Profiles can be developed for many behavioral attributes, such as the number of e-mails sent by a user, the number of failed login attempts for a host, and the level of processor usage for a host in a given period of time.

Stateful protocol analysis: - A key development in IDPS technologies was the use of protocol analyzers.
Protocol analyzers can natively decode application-layer network protocols, like HTTP or FTP. Once the protocols are fully decoded, the IPS analysis engine can evaluate different parts of the protocol for anomalous behavior or exploits against predetermined profiles of generally accepted definitions of benign protocol activity for each protocol state.
Problems with this type include that it is often very difficult or impossible to develop completely accurate models of protocols, it is very resource-intensive, and it cannot detect attacks that do not violate the characteristics of generally acceptable protocol behavior.
Example: - the existence of a large binary file in the User-Agent field of an HTTP request would be very unusual and likely an intrusion. A protocol analyzer could detect this anomalous behavior and instruct the IPS engine to drop the offending packets.

Types of IDPS

Network-based: - perform packet sniffing and analyze network traffic to identify and stop suspicious activity. They are typically deployed inline. Like a network firewall. They receive packets, analyze them, decide whether they should be permitted, and allow acceptable packets to pass through.
Allow some attacks, such as network service worms, e-mail. Borne worms and viruses with easily recognizable characteristics (e.g., subject, attachment filename), to be detected on networks before they reach their intended targets (e.g., e-mail servers, Web servers).
Most products use a combination of attack signatures and analysis of network and application protocols.
Network-based products might be able to detect and stop some unknown threats through application protocol analysis.
Some products allow administrators to create and deploy attack signatures for many major new malware threats in a matter of minutes. Although poorly written signature triggers false positives that block benign activity, a custom signature can block a new malware threat hours before antivirus signatures become available.
However, network-based products are generally not capable of stopping malicious mobile code or Trojan horses.
Deployment options
•	 Outside firewall
•	 Just inside firewall
Combination of both will detect attacks getting through Firewall and may help to refine firewall      rule set.
•	 Behind remote access server
•	 Between business units
•	 Between corporate network and partner networks
•	Sensors may need to be placed in all switched networks segment

Host-based: - are similar in principle and purpose to network-based , except that a host-based product monitors the characteristics of a single host and the events occurring within that host, such as monitoring network traffic (only for that host), system logs, running processes, file access and modification, and system and application configuration changes.
They often use a combination of attack signatures and knowledge of expected or typical behavior to identify known and unknown attacks on systems.
If a host-based product monitors the host’s network traffic, it offers detection capabilities similar to a network-based.
Host-based IDPSs are most commonly deployed on critical hosts such as publicly accessible servers and servers containing sensitive information.
Example: - attempted changes to files can be effective at detecting viruses attempting to infect files and Trojan horses attempting to replace files, as well as the use of attacker tools, such as rootkits, that often are delivered by malware.
Deployment options
•	 Key servers that contain mission-critical and sensitive information.
•	 Web servers.
•	 FTP and DNS servers.
•	 E-commerce database servers, etc.
•	 Other high value assets.
May also emplace these randomly to obtain probabilistic measure of hosts becoming compromised.

Network Behavior Analysis (NBA): - examines network traffic to identify threats that generate unusual traffic flows, such as denial of service (DoS) and distributed denial of service (DDoS) attacks, certain forms of malware (e.g., worms, backdoors), and policy violations (e.g., a client system providing network services to other systems).
NBA systems are most often deployed to monitor flows on an organization’s internal networks, and are also sometimes deployed where they can monitor flows between an organization’s networks and external networks (e.g., the Internet, business partners’ networks).

Wireless: - monitors wireless network traffic and analyzes its wireless networking protocols to identify suspicious activity involving the protocols themselves.
It cannot identify suspicious activity in the application or higher-layer network protocols (e.g., TCP, UDP) that the wireless network traffic is transferring.
It is most commonly deployed within range of an organization’s wireless network to monitor it, but can also be deployed to locations where unauthorized wireless networking could be occurring.
Organizations should consider using multiple types of IDPS technologies to achieve more comprehensive and accurate detection and prevention of malicious activity.
For most environments, a combination of network-based and host-based IDPSs is needed for an effective IDPS solution.
NBA technologies can also be deployed if organizations desire additional detection capabilities for DoS & DDoS attacks, worms, and other threats that NBAs are particularly good at detecting.
Wireless IDPSs may also be needed if the organization determines that its wireless networks need additional monitoring or if the organization wants to ensure that rogue wireless networks are not in use in the organization’s facilities.

Before evaluating IDPS products
Organizations need to understand the characteristics of their system and network environments, so that a compatible IDPS can be selected that can monitor the events of interest on the systems and/or networks.
Organizations should articulate the goals and objectives they wish to attain by using an IDPS, such as stopping common attacks, identifying misconfigured wireless network devices, and detecting misuse of the organization’s system and network resources.
Organizations should also review their existing security policies, which serve as a specification for many of the features that the IDPS products need to provide.
Organizations should determine if they require IDPSs or other specific system security resources.
Organizations also need to define specialized sets of requirements for the following: -
•	Security capabilities: - including information gathering, logging, detection, and prevention.
•	Performance: - including maximum capacity and performance features
•	Management: - including design and implementation (e.g., reliability, interoperability, scalability, product security), operation and maintenance (including software updates), and training, documentation, and technical support Life cycle costs, both initial and maintenance costs.
References
Abdullah A. Mohamed, “Design Intrusion Detection System Based On Image Block Computer and Communication Engineering, IACSIT Press, Vol. 2, No. 5, September 2013.
Denning, Dorothy E., “An Intrusion Detection Model,” Proceedings of the Seventh IEEE Symposium on Security and Privacy, May 1986, pages 119–131
Karen Scarfone & Peter Mell, “Guide to Intrusion Detection and Prevention Publication on Computer security, February 2007.
Lunt, Teresa F., “Detecting Intruders in Computer Systems," 1993 Conference on Auditing and Computer Technology, SRI International.
Conclusion
Before evaluating IDPS products, organizations should first define the general requirements that the products should meet. The features provided by IDPS products and the methodologies that they use vary considerably, so a product that best meets one organization’s requirements might not be suitable for meeting another organization’s requirements. Evaluators first need to understand the characteristics of the organization’s system and network environments and plans for near-term changes, so that an IDPS can be selected that will be compatible with them and able to monitor the events of interest on the systems and/or networks. This knowledge is also needed to design the IDPS solution. After gaining an understanding of the existing system and network environments, evaluators should articulate the goals and objectives they wish to attain by using an IDPS.

This topic appears a bit oversaturated. We focus more on unique projects. Feel free to suggest another topic. Thanks


Machine-readable output

Right now the output is designed for human consumption.
A --json flag would output a structured JSON output instead. One drawback I see is that it would make consumption a bit harder in simple cases (such as when the output is a single line).
I think --json would be most useful for biscuit inspect. The other commands are already designed to provide machine-usable output.
See #48

done in #48 and #51


Crash when called from Google app engine emulator (localhost).  (via ctypes)

Do you guys have any guess why I'm getting this crash?
Or if not, is there any way to fork this library in a way that does not depend upon ctypes?
Thanks for any thoughts you have ... I'm in quite a bind:
  File "/Users/dgaedcke/dev/gaeStdEndptTestcase/lib/apitools/base/py/credentials_lib.py", line 44, in <module>
    import fasteners
  File "/Users/dgaedcke/dev/gaeStdEndptTestcase/lib/fasteners/__init__.py", line 23, in <module>
    from fasteners.lock import locked  # noqa
  File "/Users/dgaedcke/dev/gaeStdEndptTestcase/lib/fasteners/lock.py", line 24, in <module>
    from fasteners import _utils
  File "/Users/dgaedcke/dev/gaeStdEndptTestcase/lib/fasteners/_utils.py", line 39, in <module>
    from monotonic import monotonic as now  # noqa
  File "/Users/dgaedcke/dev/gaeStdEndptTestcase/lib/monotonic.py", line 50, in <module>
    import ctypes
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 29, in <module>
    if int(_os.uname()[2].split('.')[0]) < 8:
ValueError: invalid literal for int() with base 10: ''

This is an issue with your Python environment, not this library.
There are other FFI implementations for Python, but you are probably better off debugging why ctypes doesn't work.


Still possible duplicate signal connection in ctk dicom table view

Cause of the bug:
If the dicom database is passed via the c'tor then the c'tor calls
setDicomDatabase, which calls setUpTableView, where the signals are
connected. Afterwards init is called by the c'tor, which also calls
setDicomDatabase, which then leads to the duplicate connection.
Solution:
The setUpTableview method is not really needed. All the connections can be
done in the init function, which is only called by the c'tor. Hence I removed the setUpTableview function.

:+1: Makes sense. Thanks for working on this.

Hi jc,
can this pull request be merged? We still need this fix for the MITK release.
Thanks in advance!

@AndreasFetzer All set. Thanks for the reminder.


Mccalluc/track component

Description
What was changed in this pull request?

Add HiGlassTrackComponent to API
Add demo
The demo did not work initially: I think when the call to launch is itself wrapped inside React, the order of execution change, and the component ref was not being set before exit... So I explicitly call createRef outside to avoid this reordering.
I had a test, but it didn't fail, even when the example failed, but I could try again.

Why is it necessary?

I want to be able to use HG to display a single track with React: The ref was being set only after launch returned.

Checklist

[ ] Unit tests added or updated
[ ] Documentation added or updated
[x] Example added or updated
[ ] Update schema.json if there are changes to the viewconf JSON structure format
[ ] Screenshot for visual changes (e.g. new tracks)
[ ] Updated CHANGELOG.md

I think the crux of this is
-  let component;
-
+  const ref = React.createRef();
   ReactDOM.render(
     <HiGlassComponent
-      ref={(c) => { component = c; }}
+      ref={ref}
       options={options || {}}
       viewConfig={config}
     />,
     element,
   );
-  return component;
+  return ref;

We can't assume that the ref function will have been evaluated when render returns... and although this change works for me, there are a lot of tests failing now. I'll look into them now, but feedback would be welcome... Is there a better idiom for what we were trying to do here? Or should I take different approach in my new single-track component?

yay! I'll merge this, but I do want to improve testing.

D'oh: I don't think this actually fixed anything:
HiGlassTrackComponent.componentDidMount() ->
   .initTrackViewer() ->
trackViewer() ->
viewer() ->
launch() ->
ReactDOM.render(<HiGlassComponent ...>)

The HiGlassComponent.render is still not being called before we ask for the api object.


Local timezone can break regrid operation

The following example works when the local timezone is GMT, but breaks if the local timezone is a different timezone (say US central time).
import pandas as pd
import holoviews as hv
from dateutil import parser
import datetime
import xarray as xr
import numpy as np
from holoviews.operation.datashader import regrid
hv.extension('bokeh')

time = [
 datetime.datetime(2018, 1, 9, 13, 35, 30),
 datetime.datetime(2018, 1, 9, 13, 36, 40),
 datetime.datetime(2018, 1, 9, 13, 37, 50),
 datetime.datetime(2018, 1, 9, 13, 39),
 datetime.datetime(2018, 1, 9, 13, 40, 10),
 datetime.datetime(2018, 1, 9, 13, 41, 20),
 datetime.datetime(2018, 1, 9, 13, 42, 30),
 datetime.datetime(2018, 1, 9, 13, 43, 40),
 datetime.datetime(2018, 1, 9, 13, 44, 50),
 datetime.datetime(2018, 1, 9, 13, 46),
 datetime.datetime(2018, 1, 9, 13, 47, 10),
 datetime.datetime(2018, 1, 9, 13, 48, 20),
 datetime.datetime(2018, 1, 9, 13, 49, 30),
 datetime.datetime(2018, 1, 9, 13, 50, 40),
 datetime.datetime(2018, 1, 9, 13, 51, 50)]

dim2 = np.arange(0,1000)
values, _ = np.mgrid[0:1000, 0:15]
xarr = xr.DataArray(values,
                       coords={'time': time, 'dim2': dim2},
                       dims=['dim2', 'time'])

image = hv.Image(xarr,  ['time', 'dim2'], ['val'])
regrid(image,  aggregator='mean')

@philippjfr To reproduce, switch the timezone from GMT to central time and any zooming will cause the data to disappear (and other weird artifacts have been observed as well e.g partial data display)

I haven't tried debugging this version, but in a related case the data was disappearing because datashader was being given a negative plot_width or plot_height, perhaps because data with and without timezone information was being subtracted.  I believe the code above works fine without regrid, so it's probably an issue in how datetimes are being handled in the hv datashader interface (given that datashader itself does not support datetimes).

Another weird error is that if you change the datetimes above from the first month to say the second month (so the first datetime is say datetime.datetime(2018, 2, 9, 13, 35, 30),) then you get an rtol exception error. I noticed this trying to change the month to other values too!

I believe the code above works fine without regrid

That is correct.


Bump the version of debian-iptables-amd64 to pick up some CVE fixes

Match the version used by kube-proxy from PR k/k #75845
https://github.com/kubernetes/kubernetes/pull/75845

/assign MrHohn


Autofill from script is having different parameters than output folder

Autofill from script is having different parameters that output. This blog says output folder but the autofill has inputtable and partitiontable as parameters. Please fix this problem.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fc24ebe8-5309-3143-ae92-41b6a5150cf6
Version Independent ID: d0e667a5-9160-73ce-2d29-156915b60a83
Content: Tutorial: On-demand clusters in Azure HDInsight with Data Factory
Content Source: articles/hdinsight/hdinsight-hadoop-create-linux-clusters-adf.md
Service: hdinsight
GitHub Login: @JasonWHowell
Microsoft Alias: jasonh

@dhangerkapil Thanks for the feedback! We are currently investigating and will update you shortly.

@dhangerkapil "Auto-fill" option looks for any parameters in the Hive script that require values at runtime and automatically fetch those parameters.
Users can manually also enter parameters here by clicking New button below Auto-fill option.
It seems everything looking good to me. Kindly let me know if that is not the ask and share more details for better understanding.


Client certificate validation via TrustManager interfaces not offloaded from event loop with OpenSSLEngine

We're using vertx.io, which recently delivered the ability to leverage the enhancement in netty/netty#8847 to allow blocking SSL operations to be offloaded from the event loop via vertx v4.3.8 see eclipse-vertx/vertx#4566.  We are doing client certificate validation with OCSP revocation checking, so certificate validation can be long running and we need to avoid performing this work on the event loop.  After we picked up the vertx changes, we tested using the JdkSslEngine and we saw the correct behavior in that all certificate validation operations were performed on the event loop; however, when we moved to OpenSSLEngine (which is a requirement for us going forward), we no longer see the certificate validation being offloaded.  Note that we do see some SSL operations being offloaded, but not the certificate validation.
As I understand it, the ssl engine provides a hint to netty about when operations could be offloaded, so its not clear whether this is a netty bug or whether this is expected behavior with the OpenSSLengine.
Expected behavior
When using OpenSslEngine and specifying a delegatedTaskExecutor to create an sslHandler, we expected to see TrustManager calls to validate a client certificate (specifically X509ExtendedTrustManager::checkClientTrusted) being executed by the delegated executor and not on the event loop.
Actual behavior
This is a stack trace of a case where the checkClientTrusted routine is being executed on an event loop.  note that this is using OpenSSLEngine and specifying an internal worker pool for the executor.  When we run the same test with the JdkSslEngine, we see proper behavior (see stack trace below).
Feb 13 12:54:14 -B control-path[13519]: [DEBUG] [] [TrustOptions|vert.x-eventloop-thread-0] Start Blocking Client Cert Validation
Feb 13 12:54:14 -B control-path[13519]: java.lang.Throwable
Feb 13 12:54:14 -B control-path[13519]: at CycTrustOptions$CycTrustManager.checkClientTrusted(TrustOptions.java:334)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.ReferenceCountedOpenSslServerContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslServerContext.java:276)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:779)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.internal.tcnative.SSL.readFromSSL(Native Method)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.readPlaintextData(ReferenceCountedOpenSslEngine.java:657)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1267)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1404)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1447)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:222)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1343)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1247)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1287)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
Feb 13 12:54:14 -B control-path[13519]: at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
Feb 13 12:54:15 -B control-path[13519]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
Feb 13 12:54:15 -B control-path[13519]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
Feb 13 12:54:15 -B control-path[13519]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
Feb 13 12:54:15 -B control-path[13519]: at java.base/java.lang.Thread.run(Thread.java:829)

This is the behavior we see when utilizing the JdkSslEngine.  It properly offloads the certificate processing to a worker thread.
Feb 13 13:10:05 -B control-path[1707]: [DEBUG] [] [TrustOptions|vert.x-internal-blocking-3] Start Blocking Client Cert Validation
Feb 13 13:10:05 -B control-path[1707]: java.lang.Throwable
Feb 13 13:10:05 -B control-path[1707]: at CycTrustOptions$CycTrustManager.checkClientTrusted(CycTrustOptions.java:334)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkClientCerts(CertificateMessage.java:682)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:411)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:375)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
Feb 13 13:10:05 -B control-path[1707]: at java.base/java.security.AccessController.doPrivileged(Native Method)
Feb 13 13:10:05 -B control-path[1707]: at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)
Feb 13 13:10:05 -B control-path[1707]: at io.netty.handler.ssl.SslHandler$SslTasksRunner.run(SslHandler.java:1787)
Feb 13 13:10:05 -B control-path[1707]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Feb 13 13:10:05 -B control-path[1707]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Feb 13 13:10:05 -B control-path[1707]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
Feb 13 13:10:05 -B control-path[1707]: at java.base/java.lang.Thread.run(Thread.java:829)

Steps to reproduce
Create an SSL Handler using OpenSSLEngine and specify a deletedTaskExecutor, then perform client certificate validation.
Minimal yet complete reproducer code (or URL to code)
I cannot provide a minimal reproducer as we have only investigated this with our proprietary code.  I'm looking into trying to create something minimal, but I don't have anything as of now.
The following vertx test does validate the offload of certain operations.  It is currently written to leverage the JdkSslEngine, but it is easy enough to modify to use OpenSslEngine.  If you change the engine, we do see that most operations are offloaded; however, this test doesn't do client certificate validation.
https://github.com/eclipse-vertx/vert.x/blob/462e3ea5203dd710e52a84bf8fba5f5a5dc5eb46/src/test/java/io/vertx/core/http/HttpTLSTest.java#L1655
Netty version
2.0.52.Final-linux
JVM version (e.g. java -version)
openjdk version "11.0.16" 2022-07-19
OpenJDK Runtime Environment (build 11.0.16+8-suse-150<IP_ADDRESS>-x8664)
OpenJDK 64-Bit Server VM (build 11.0.16+8-suse-150<IP_ADDRESS>-x8664, mixed mode)
OS version (e.g. uname -a)
cat /etc/os-release
NAME="SLES"
VERSION="15-SP2"
VERSION_ID="15.2"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP2"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp2"

Tagging @vietj  for visibility.

Will have a look.

We did some more investigation and wanted to pass along some more info.  it looks like the offload of trust manager interfaces were only done for BoringSSL, but we cannot take that path because we need a FIPS certified implementation.  its not clear whether OpenSSL cannot support this requirement or it just hasn't been done in tcnative.  A colleague of mine did find reference to OpenSSL supporting an async mode which might be helpful here.
adds cert validation offload for BoringSSL alone
https://github.com/netty/netty-tcnative/pull/435

@uncbailey1 this is because async cert verify was only added to OpenSSL 3 and only for the client side. We dont support this, so BoringSSL is your only bet atm


Delegate registration and verification to default to ng experience

Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the maintainers.
What Type of PR is This?

[ ] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Preview environment: https://ci.pr.harness.io/pr-596


Global mutable state in constraints

Is it safe to rely on a global mutable state (that will change results of constraint invocation during test) in constraints? For example:
BOOST_AUTO_TEST_CASE( event_is_generated_every_5_seconds )
{
    bool state = false;

    MOCK_FUNCTOR( f, void() );

    MOCK_EXPECT( f ).calls( [&]{ return !state; } ).once();
    MOCK_EXPECT( f ).calls( [&]{ return  state; } ).once();

    f();

    state = true;

    f();
}

Yes this is safe. In fact an even easier method is: MOCK_EXPECT( f ).once().returns(std::ref(state)); which simply returns the state.
See the test:
        mock::detail::function<int()> f;
        int i = 42;
        f.expect().returns(std::ref(i));
        i = 43;
        BOOST_CHECK_EQUAL(43, f());


pappl: 1.0.0 -> 1.0.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/pappl/versions.
meta.description for pappl is: "C-based framework/library for developing CUPS Printer Applications"
meta.homepage for pappl is: "https://github.com/michaelrsweet/pappl"
meta.changelog for pappl is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.0.1 with grep in /nix/store/nhm0c3iczg1p1r1lmy0bglpqkzd97479-pappl-1.0.1

directory tree listing: https://gist.github.com/9cc87e686f3ebff8a08a752bf921b2c6

du listing: https://gist.github.com/229719529031299157829c61d1fad77a

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
pappl

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/nhm0c3iczg1p1r1lmy0bglpqkzd97479-pappl-1.0.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A pappl https://github.com/r-ryantm/nixpkgs/archive/b690193721479211868744227ffa89ac2f54fbaf.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/nhm0c3iczg1p1r1lmy0bglpqkzd97479-pappl-1.0.1
ls -la /nix/store/nhm0c3iczg1p1r1lmy0bglpqkzd97479-pappl-1.0.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

pappl

Maintainer pings
cc @jonringer for testing.

PAPPL v1.0.1 is a general bug fix release

PAPPL v1.0.1 is a general bug fix release


License

Sorry to spam…
Which licenses do you think of ? I hope you think about open source… ;)

Well right now its a tie between BSD, MIT, and GPLv2 or later.
I was also thinking it would be nice to be able to relicense the code later for whatever reason but that would require me to also have a CLA(Contibuter License Agreement)...
http://oss-watch.ac.uk/resources/cla
If i did a CLA I would probably go with GPLv2 or later, issues with using BSD or MIT are that someone could drive by make it closed source and release a nice game change some stuff maybe make a large profit without any effort...

Yeah, that's why I prefer GPL too.

hmm another thought, license the "game" code under GPL and the engine side as BSD lol
Main thing I may want to reuse the engine side of things for something else later on (which may or may not end up being open source idk yet wouldnt be a music game or anything though)
So idk.

That was where the CLA would be useful, it would give me full rights over all contributed code allowing me to relicense it under something else later. It does adds a bit of extra overhead in order to contribute code though.

I'm not sure how you will feel about that though... any objections about it?

Well… The LGPL may be the best choice then.
Wikipedia : The main difference between the GPL and the LGPL is that the latter allows the work to be linked with (in the case of a library, 'used by') a non-(L)GPLed program, regardless of whether it is free software or proprietary software
So you may use some part of your code in non-free software BUT your modified code has to be open source. (I don't think that's terrible if it's not in a public Git repo ^^)
BUUUUUT if someone distributes a modified version of the game, it HAS to be LGPL. That's perfect.
Example : GTK/Qt use LGPL. You can use those toolkits to do proprietary software. But if you modify GTK to your needs, those modifications need to be open source.

Still after reading through this article might be a decent idea to have one anyways haha.
https://julien.ponge.org/blog/in-defense-of-contributor-license-agreements/
Idk about LGPL for the whole game. Plus if I wanted to reuse the code as LGPL I would need to refactor the engine side of things to support dynamic linking....
Technically I could separate the engine related stuff out into another repository, then leave just the game stuff here... Then license the two seperately.
Personally I don't care if someone takes the engine side of things and forks it to be proprietary. I only care about people forking the game and selling it off as their own or something to that nature.
So with that in mind, I could put the engine side of things as BSD licensed. The game as GPL v2 or later.

That sounds like a good idea to me.

Which idea, there are several over this issue?

About supporting dynamic linking… You don't need that. Some part of the code can be open source and another not and still be compiled as a whole binary.
And maybe a CLA is great. I never heard about it until now.

lol sorry reading from my phone :b But splitting the engine and game code. you could probably keep both in same repo and but under different folders and clearly state engine folder is bsd and game folder is gpl or whatever.

Yes, that could be a possibility.

and if code is is LGPL you cannot staticly link the code unless you release the rest of the binary unless its specifically stated that you can. which you may as well use GPL at that point

Yeah exactly

So I think i'm going to start looking into statically linking the engine side of things since it allows a bit more flexibility in all of this.

oops ment dynamically...

Sorry about the slowness on this. I have been busy with my new job. And have been implementing a open source patcher/launcher for a non-free game, because why not.... (Technically it could be adapted for other projects as well)

Ok so finally decided to go ahead ad do BSD 2 clause, just because its simpler to deal with in some aspects...


Add knobs for allowed modifications in ext_authz and ext_proc

Title: Add knobs to control which headers ext_authz and ext_proc servers can modify
Description:
Today, the ext_authz filter allows the remote server to request that any header,  including internal headers (prefixed with ":"), and x-envoy headers be modified.
The in-progress ext_proc filter does not allow these filters to be modified, but we expect that in some cases users will wish to support this.
We should give both of these filters configurable controls for this. One way would be to create a new enum and use it in the configuration of both filters.
I'd propose an enum with the following modes:

DEFAULT: In this mode, headers may not be modified if they are expected to cause problems later that cannot be resolved. That means that no x-envoy header may be set, and a few specific headers such as ":scheme" may not be set. However, other internal headers, including ":path", ":method", ":authority", and "host" may be set. This may cause routes to have to be recalculated, which each filter should account for, ideally without requiring the user to do anything specifically. (i.e. the filter should know that, if ":path" was changed, that the route cache might need refreshing). This will be the default mode of the ext_proc filter.
UNRESTRICTED: In this mode, any header may be set, including x-envoy, "host", and any internal headers. This will be the default mode for the ext_authz filter since this is how it works today.
RESTRICTED: In this mode, no internal headers may be set, the host header may not be set, and no x-envoy header may be set. This provides the maximum safety for embedding in other products or for multi-tenant environments.

If these three options are not sufficient, then an alternative would be to support a set of booleans like:

allow_internal_headers
allow_x_envoy_headers
allow_host_header

And if THAT is not sufficient then we could allow the user to specify an allow list and a deny list, complete with wildcards. However this might be overly complex.

@alyssawilk @htuch we were discussing this for possible usage of ext_authz in an embedded product.

@alyssawilk @htuch we were discussing this for possible usage of ext_authz in an embedded product.

Based on some other discussions, we might need another mode, like "DISABLED," which disallows all modifications and just lets the filter only decide whether to accept or reject.

Based on some other discussions, we might need another mode, like "DISABLED," which disallows all modifications and just lets the filter only decide whether to accept or reject.

The bools seem slightly better as they provide mechanism rather than policy, which is preferable in the API.
Arguably, having controls like this would also make sense for dynamic code like Lua/Wasm, since you could control what sandboxed code was capable of. But, if we design this right, and don't make the proto message restricted to ext_*, that can always be added later.

The bools seem slightly better as they provide mechanism rather than policy, which is preferable in the API.
Arguably, having controls like this would also make sense for dynamic code like Lua/Wasm, since you could control what sandboxed code was capable of. But, if we design this right, and don't make the proto message restricted to ext_*, that can always be added later.

As said on email earlier, I think this would be useful in a number of places.    If we do booleans let's encapsulate in a base level proto so we can apply it to other places we do header modifications.  I could imagine a cloud provider tacking on "no changing host" rules to ensure that client xDS config didn't change disallowed fields for per-route config for example.
Since I think bools are false by default, I'd be inclined to flip semantics and have
disallow_thing_a
disallow_thing_b
so by default everything is allowed and you filter by flipping explicit config, but you can also use Boolean and allow_thing.
cc @lizan and @PiotrSikora for wasm and @markdroth for gRPC thoughts (e.g. x-envoy: does gRPC have an equivalent?)
For naming, :scheme :method etc. are pseudo-headers (https://tools.ietf.org/html/rfc7540#section-<IP_ADDRESS> : I think what you were calling internal?) and we should comment it also applies to http/1.1 equivalents like host:
For x-envoy there's the header_prefix knob in Envoy bootstrap that lets Envoy change its prefix from x-envoy to x-foo so we'd want to make sure we comment and sanitize based on the actual header prefix if it's not the default:
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/bootstrap/v3/bootstrap.proto#envoy-v3-api-msg-config-bootstrap-v3-bootstrap

As said on email earlier, I think this would be useful in a number of places.    If we do booleans let's encapsulate in a base level proto so we can apply it to other places we do header modifications.  I could imagine a cloud provider tacking on "no changing host" rules to ensure that client xDS config didn't change disallowed fields for per-route config for example.
Since I think bools are false by default, I'd be inclined to flip semantics and have
disallow_thing_a
disallow_thing_b
so by default everything is allowed and you filter by flipping explicit config, but you can also use Boolean and allow_thing.
cc @lizan and @PiotrSikora for wasm and @markdroth for gRPC thoughts (e.g. x-envoy: does gRPC have an equivalent?)
For naming, :scheme :method etc. are pseudo-headers (https://tools.ietf.org/html/rfc7540#section-<IP_ADDRESS> : I think what you were calling internal?) and we should comment it also applies to http/1.1 equivalents like host:
For x-envoy there's the header_prefix knob in Envoy bootstrap that lets Envoy change its prefix from x-envoy to x-foo so we'd want to make sure we comment and sanitize based on the actual header prefix if it's not the default:
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/bootstrap/v3/bootstrap.proto#envoy-v3-api-msg-config-bootstrap-v3-bootstrap

See parameter sanitization in Proxy-Wasm capabilities (#13911).

See parameter sanitization in Proxy-Wasm capabilities (#13911).

@alyssawilk gRPC does not have its own equivalent of x-envoy headers; we generally have APIs for the application to configure things instead.  Although interestingly, as part of the xDS support we're adding, there are some cases where we're going to support some x-envoy headers for compatibility with Envoy. :/
Is the idea here that these knobs would be defined in some sort of wrapper that can be applied to any filter?  How can the settings of these knobs be enforced that way?  Or is the idea that we'd need each individual filter to support these knobs as appropriate?
CC @ejona86 @dfawley

@alyssawilk gRPC does not have its own equivalent of x-envoy headers; we generally have APIs for the application to configure things instead.  Although interestingly, as part of the xDS support we're adding, there are some cases where we're going to support some x-envoy headers for compatibility with Envoy. :/
Is the idea here that these knobs would be defined in some sort of wrapper that can be applied to any filter?  How can the settings of these knobs be enforced that way?  Or is the idea that we'd need each individual filter to support these knobs as appropriate?
CC @ejona86 @dfawley

My mental model was we'd have a proto of "what is disallowed" (everything being allowed by default) and different areas of the code base could add it over time to things like request_headers_to_add, or ext_proc.  There'd be a utility which took in that proto (or parsed config) and applied the checks, and that'd be added to sites with the API additions.
Totally open to options if there's a model which works better for grpc.  For example rather than x-envoy header we could do a generic "no headers with this prefix" and use x-envoy in the examples.

My mental model was we'd have a proto of "what is disallowed" (everything being allowed by default) and different areas of the code base could add it over time to things like request_headers_to_add, or ext_proc.  There'd be a utility which took in that proto (or parsed config) and applied the checks, and that'd be added to sites with the API additions.
Totally open to options if there's a model which works better for grpc.  For example rather than x-envoy header we could do a generic "no headers with this prefix" and use x-envoy in the examples.

It may be useful to add some controls that specify what should be done if the external filter attempts to modify a disallowed header.  Some options include:

emit some logging or stats
ignore the disallowed modifications
error out the request

Check out https://github.com/envoyproxy/envoy/pull/19141 which adds a proto to start to address this issue

As of https://github.com/envoyproxy/envoy/pull/19622, this has now been merged to main and is supported. See the new "mutation_rules" field in the filter configuration.


CloudQueue.PeekMessages(int) and similar don't tell max number to request

The max number of messages to request from a queue is 32, according to the REST API documentation. This number (32) is not documented in the .NET SDK.
E.g.
PeekMessages
GetMessages

This is a reference doc issue that requires a change to the underlying /// comments.
Opened new issue https://github.com/Azure/azure-storage-net/issues/650


Schedule open weather-tools developer meetings.

To foment better community support of these tools, let's schedule a weekly 30 minute meeting for weather-tools developers. If you are interested in contributing to this codebase, please let us know what time(s) you are available to meet here: http://whenisgood.net/weather-tools-meeting
Once a date is found, we'll update the contributing docs with the time and steps to attend the weekly meetings.

CC: @CillianFn @saveriogzz @pranay101

Hey @alxmrs did we finalise the date and time for a meet?

Thanks for following up. I'm going to announce the meetings soon – I'm waiting to hear back from a few more people.


Containerise Dependencies for Quickstart Script

/closes #387
What

[x] Adds docker image containing binaries needed for hack/quickstart-setup.sh
[x] hack/quickstart-setup.sh changed to use binaries in above image
[x] Adds GitHub action to build and push tools image to quay.io
[x] Update docs: change requirements in ocm-control-plane-walkthrough.md and getting-started.md

Verification
Run hack/quickstart-setup.sh locally and verify a successful install. Note: The docker build command currently runs in this script and should be removed before merging or when the tools image is available on quay.io.
Verify that make local-setup continues to work as expected.

@eoinfennessy do you want me to give this try locally?

@eoinfennessy do you want me to give this try locally?

Probably not much need to try it out locally yet... I'll try to get a few more of the binaries working first.

I haven't verified but docs updates look great. 👍

This seems fine, but i haven't tried it. Do you want to undo the changes in the quick start script that need removed before it can be merged?

@eoinfennessy I'm guessing the mgc-tools image is stiill private?
Unable to find image 'quay.io/kuadrant/mgc-tools:latest' locally
docker: Error response from daemon: manifest for quay.io/kuadrant/mgc-tools:latest not found: manifest unknown: manifest unknown.

Verified it locally. Just as a note, I had some issues building the tools image due to my local docker install so it might be good to add to the docs the minimum docker version required, as well as the fact that Buildx is required to suport heredoc when adding the kubernetes repo

Verified HEAD~1 there and worked good.

Verified it locally. Just as a note, I had some issues building the tools image due to my local docker install so it might be good to add to the docs the minimum docker version required, as well as the fact that Buildx is required to suport heredoc when adding the kubernetes repo

Yeah, I think buildx has been included in engine since Docker 19. Off-topic, we should look to see Podman works OK at some point too.

Verified HEAD~1 there and worked good.

Verified it locally. Just as a note, I had some issues building the tools image due to my local docker install so it might be good to add to the docs the minimum docker version required, as well as the fact that Buildx is required to suport heredoc when adding the kubernetes repo

Yeah, I think buildx has been included in engine since Docker 19. Off-topic, we should look to see Podman works OK at some point too.

It does not work with Podman, as there are some issues with heredoc.

If heredoc is causing trouble, maybe we can have a base image that has the repository added with a simple COPY, and build the tools image on top of it?

Verified it locally. Just as a note, I had some issues building the tools image due to my local docker install so it might be good to add to the docs the minimum docker version required, as well as the fact that Buildx is required to suport heredoc when adding the kubernetes repo

People really shouldn't be needing to build this image themselves, it's just because nobody has pushed it and the GH workflow isn't in place yet. So i don't really see this is a big issue. That said, kubectl isn't even being used inside the container from what i can tell, so you could just remove that whole bit from the Dockerfile.

It does not work with Podman, as there are some issues with heredoc. Not using it in Dockerfile works fine.

For the same reasons i wouldn't be too concerned about being able to build the image with podman, but it would be interesting to know if you could swap out docker for podman when running the tools via the container i.e podman run --rm -i --entrypoint="kustomize" quay.io/kuadrant/mgc-tools:latest build ....
@eoinfennessy I built the image as is and pushed it to quay.io/kuadrant/mgc-tools:latest, but it's far too big. We will need to reduce the size of the image by quite a bit. We could look into a multi stage build maybe and copy the binaries from a builder to the final image.

That said, kubectl isn't even being used inside the container from what i can tell, so you could just remove that whole bit from the Dockerfile.
Yeah, we're not using kubectl in the container (unless any of the other tools depend on it for some reason)? I'll remove this now.

@eoinfennessy I built the image as is and pushed it to quay.io/kuadrant/mgc-tools:latest, but it's far too big. We will need to reduce the size of the image by quite a bit. We could look into a multi stage build maybe and copy the binaries from a builder to the final image.
Will look into this. Is there a lightweight base image you'd recommend for the second stage?

That said, kubectl isn't even being used inside the container from what i can tell, so you could just remove that whole bit from the Dockerfile.

Yeah, we're not using kubectl in the container (unless any of the other tools depend on it for some reason)? I'll remove this now.

@eoinfennessy I built the image as is and pushed it to quay.io/kuadrant/mgc-tools:latest, but it's far too big. We will need to reduce the size of the image by quite a bit. We could look into a multi stage build maybe and copy the binaries from a builder to the final image.

Will look into this. Is there a lightweight base image you'd recommend for the second stage?

Small change, but could switch to ubi-minimal: https://catalog.redhat.com/software/containers/ubi8/ubi-minimal/5c359a62bed8bd75a2c3fba8
70MB->~35MB.
Multi-stage build like Mike suggested should shave a significant bit off

Small change, but could switch to ubi-minimal: https://catalog.redhat.com/software/containers/ubi8/ubi-minimal/5c359a62bed8bd75a2c3fba8

Thanks @jasonmadigan!
Latest Dockerfile.tools uses ubi8-minimal. It may be possible in future to further reduce the size of this tools image by using a ubi8-micro image, but I needed to use microdnf to install git, as copying over the binary from the build stage caused an error in the script that I haven't been able to resolve:
kustomize --load-restrictor LoadRestrictionsNone build github.com/kuadrant/multicluster-gateway-controller.git/config/quick-start/control-cluster
Error: git cmd = '/usr/bin/git fetch --depth=1 origin HEAD': exit status 128


Issue 44 - Improved performance of FileAncestryLinkManager.get_source_ancestors()

Improved performance of FileAncestryLinkManager.get_source_ancestors()

I'll make a note to look at it and create a new issue if needed. Thanks.


Fix readable_record_range

Currently Mobi.readable_record_range() returns range from 1..recods.len(). Find out where does the content actually start (probably idx 0), and check whether last record isn't always empty so that we can skip processing it.

This got resolved with bfa50f08ca743986905583ac610488839c637c93


Unsupported builder type

Hi,
when I trying to pass collection to Vuetable it's throw exception Unsupported builder type
$query = new Collection([
            ['name' => 'John Doe', 'email' =><EMAIL_ADDRESS>            ['name' => 'Jane Doe', 'email' =><EMAIL_ADDRESS>            ['name' => 'Test John', 'email' =><EMAIL_ADDRESS>       ]);

       return Vuetable::of($query)
           ->editColumn('name', function ($user) {
               return Str::lower($user['name']);
           })
           ->addColumn('urls', function ($user) {
               return [
                   'edit' => route('users.edit', $user['id']),
                   'delete' => route('users.destroy', $user['id']),
               ];
           })
           ->make();

What type of collection are you using? Because the package checks if the collection is an instance of Illuminate\Support\Collection

I tried both of Illuminate\Support\Collection and eloquent collection but not working with me.

I'll try it in my pc

What version of the package are you using, i think maybe you installed the 0.0.* and that version doesn't support collections. I have to fix the readme

yes, thank you fro you effort


PHP 5.3 Namespace compability

Just wondering if there is some plan on making it compatible with PHP 5.3 namespaces?
There are some issues with relative vs absolute scope (which can be overcome) but the main issue is with the factory calls (cant make a new_\My\Name\Space\Class function). Without a proper rewrite there are two quick ways to resolve this:

Translate the namespace separator to something else - in this case '_':

function create($classname) {
$classname = $this->scope->getImplementation($classname);
$funcname=strtr($classname,'\','');
if (isset($this->factory->{'new' . strtolower($funcname)})) {
return call_user_func($this->factory->{'new_'.$funcname}, $this);
}
if (method_exists($this->factory, 'new_' . $funcname)) {
return $this->factory->{'new_'.$funcname}($this);
}
return $this->createThroughReflection($classname);
}

Instead of checking for the existence of the 'new_' function call it and require the factory to have a __call catchall method that either handles the classname conversion or returns false.

@jaanush There are no imminent plans. However, if you are still using bucket, you can create a pull request with tests, then  will happily merge it in.


Add Meziantou.Framework.InlineSnapshotTesting

Thank you! This library is like the new level of automation.


MINOR: update docs note about spurious stream-stream join results

The existing docs note about spurious results from left and outer stream-stream joins is out-of-date as of https://issues.apache.org/jira/browse/KAFKA-10847.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Merged to 3.4 and cherry-picked to 3.3, 3.2, and 3.1 branches.
Can you also do a follow up PR again kafak-site.git -- otherwise the changes does only go live if we do a bug-fix what very unlikely for 3.3 and older.

Can you also do a follow up PR again kafak-site.git

https://github.com/apache/kafka-site/pull/507


GLSL->HLSL: Unsupported builtin in HLSL: 5 (BuiltInVertexId)

OpenGL VertexShader
#version 440 core
void main()
{
	// "fullscreen triangle" from http://de.slideshare.net/DevCentralAMD/vertex-shader-tricks-bill-bilodeau page 14
	gl_Position = vec4((gl_VertexID / 2) * 4.0 - 1.0, (gl_VertexID % 2) * 4.0 - 1.0, 0.0, 1.0);
}

OpenGL-style GLSL coming from glslang::GlslangToSpv
spirv_cross::CompilerHLSL compiler(spirv);
auto options = compiler.get_options();
options.shader_model = 40; // Shader Model 4.0
compiler.set_options(options);
...
output = compiler.compile(remap);

throws in CompilerHLSL::emit_builtin_variables()
Shouldn't BuiltInVertexId behave the same way as BuiltInVertexIndex?

Even after adding BuiltinVertexId cases to everywhere where BuiltinVertexIndex was handled,
causes builtin_to_glsl to assert because the CompilerHLSL::compile sets options.vulkan_semantics to true.
Am I doing something wrong, or shouldn't that be set for HLSL generation?


Installation issues...

Hi,
It was long time ago I had to do any PHP coding and lot has changed since then (I use NET in my everyday development) and now I have some small PHP project to attend to, so I decided to use VS Code to help me with PHP.
First thing I did is to install PHP Debug (thank U for this extension too) and made it work pretty fast. Now I can debug PHP code :)
During my PHP development sessions, I saw that there is no VS code PHP intellisense for MySQL objects (in specific mysqli), so I decided to try your other extension - PHP Intellisense and I failed :(
After I installed extension, I can see format code option in context menu, but no matter what I tried I could not see any intellisense regarding MySQL PHP objects.
Here are my settings:
"php.validate.executablePath": "C:/xampp/php/php.exe", "php.validate.run": "onType", "php.validate.enable": false
I tried "php.executablePath" too, without and in combination with "php.validate.executablePath" but it did not seem to work. As U can see, I am using XAMPP with PHP 7.0.13 (cli) (built: Nov  8 2016 13:45:28) ( ZTS ) (PHP is in my PATH too).
Based on all said in READ.me I was not sure if I need to install PHP language server in order to make intellisense work so I did not install it :)
Is there anything else that I can do/try in order to get PHP MySQL objects intellisense to work?
Thanks

Autocompletion is currently supported by the language server but not released in the VS Code extension yet because of https://github.com/Microsoft/vscode/issues/9003


Beginner friendly readme

I got some input on the readme structure from some new Django developers.
This PR does some minor touchups to the readme to be more beginner friendly.

@rmorshea Can we merge this before the release? I'd like the PyPI description to contain these modifications.


Updated audience validation for admin api tokens

no-issue
This is a temporary solution until we implement token permissions based on actions upon objects, for example using canThis

Cool that makes sense to me 👍

To clarify a little the why - we had lots of questions about what audience should be - should it include /ghost/? What about subdirectories.. should it be the full url? If I request a token for /ghost/api/v2/admin/posts/ am I allowed to request  /ghost/api/v2/admin/posts/id/ etc etc
It was p clear that the validation in place was too strict, or at least had too many unconsidered edge cases.
So, we talked about the purpose of audience, which is to narrow down what the token is for. The secret already ties a token to a specific site, so we use audience to tie a token to a specific API. This is futureproof against us having more APIs with similar auth in future.
Method and resource based restrictions can be added later, e.g. using claims.


Case sensitivity modifier for format specifiers

It would be very useful to be able to control the case sensitivity of some of the format specifiers such as weekday and month names. This is useful when implementing specs that recommend case insensitivity for consumers of a format, such as the HTTP caching spec. This example, starting with a deliberately lowercase s, fails to parse even though a cache ought to be able to accept it:
#[test]
fn parse_imf_fixdate() {
    let _ =
        PrimitiveDateTime::parse("sun, 06 Nov 1994 08:49:37 GMT", "%a, %d %b %Y %T GMT").unwrap();
}

I was unable to find documentation for the specifier syntax in 0.3, but I don't see anything in the code regarding case sensitivity. This capability is important for our ability to move from chrono to time.

The current docs for the format description in 0.3 is here. I still need to update the in-crate docs and finish the rest of the book, of course.
You are correct that it's not currently possible to match case insensitively. I think this is a reasonable addition behind a case_sensitive:{true|false} modifier. Off the top of my head, I'm pretty sure this would be quite simple to implement, as the underlying functionality already exists to an extent.

Thank you! Sounds great. No rush at the moment as I wrote a logos + lalrpop parser that builds an OffsetDateTime directly, but this seems like it'd be a nice feature to have for lighter-weight uses.

Actually going to go into quite a rabbit hole here. This request made me realize that the list of modifiers is not exhaustive, which requires the structs to be marked #[non_exhaustive]. Not an issue in and of itself, except that means users (including the macro) can no longer construct a modifier directly. So now I'm digging into a few patterns that have to be const-compatible 🤞🏽

Done in 7a05a42.


Missing graphql command in bal output

Description:
Missing graphql command in bal output
Steps to reproduce:
Try to run the bal command
The graphql command is missing
Affected Versions:
Ballerina Version: 2201.2.0-rc1.5
OS, DB, other environment details and versions:
Operating System: Ubuntu 20.04
Java SDK: 11

Closing this issue since the fix is merged


Adding a base_dir variable (#343)

Adding a base_dir variable to that shell scripts can be found when running remotely

set base_dir correctly in stf-run-ci/tasks/main.yml

Co-authored-by: Emma Foley<EMAIL_ADDRESS>Cherry picks changes from da1f402

test

test


Develop upstream sync 200812

Conflicts:

tensorflow/compiler/mlir/tools/kernel_gen/gpu_binary_creator.cc
tensorflow/compiler/xla/service/gpu/tests/gpu_noalias_test.cc
tensorflow/compiler/xla/service/mlir_gpu/kernel_lowering.cc
tensorflow/core/kernels/BUILD
tensorflow/python/BUILD
tensorflow/python/eager/def_function_xla_jit_test.py
tensorflow/python/kernel_tests/BUILD
third_party/gpus/rocm_configure.bzl

Can one of the admins verify this patch?

rest rocm-cpp please

jenkins retest rocm-cpp please

Jenkins retest rocm please

Retest rocm please

For record, ROCm has all pass on this PR here. It altered its result for unknown reasons.


Service Fabric solution does not build out of the box

You need define the build strucute. I beleive interfaces need to be build before the partionmap

This code builds fine on my environment and project build dependencies look correct. Can you please verify from your side if this is still a problem?


Using containerds default snapshotter labels instead of custom ones

Using containerds default snapshotter labels instead of custom ones.
Removing unused labels in config.go
https://github.com/awslabs/soci-snapshotter/blob/aed937970e4b35dac5bf65fd38ad4bc6208c0ef6/fs/config/config.go#L42-L46
This also changes the snapshotter to use the FromDefaultLabels()  as the source function solely instead of sourceFromCRILabels(). Currently there is no support for layer size labels within the snapshotters package in containerd. This is important since we need a layers size when creating the io.SectionReader for the layer in the layer resolver. In order to have kubernetes/CRI support in the future we must either push for layer size label support upstream or re-implement https://github.com/awslabs/soci-snapshotter/pull/234  which will allow us to obtain layer sizes via a network call.
I have left the CRI get sources implementation unchanged even though its not used. We can either choose to keep it or remove it once one of the two solutions listed above is implemented.
Issue #, if available:
Fixes: #458
Testing performed:
make test && make integration
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

LGTM, just need to address the check failures.


Do nothing with Application from LC process.

I've actually never seen this happen, though.

Still a good move. We don't want the onCreate() code to run for no reason.


Convert the CircleCI workflow to a GitHub Actions workflow

Summary
This pull request converts the CircleCI workflows to GitHub actions workflows. Github Actions Importer was used to convert the workflows initially, then I edited them manually to correct errors in translation.
The ${{ secrets.PYPI_PASSWORD }}, ${{ github.CIRCLE_REPOSITORY_URL }}, ${{ secrets.CIRCLE_CI_SSH_KEY }} and ${{ secrets.CIRCLE_CI_KNOWN_HOSTS }} settings will need to be set for the tasks to succeed.
Issues
setuptools.extern.packaging.version.InvalidVersion: Invalid version: 'main'

Full Error

Collecting IOHexperimenter>=<IP_ADDRESS> (from nevergrad==1.0.2)
  Downloading IOHexperimenter-<IP_ADDRESS>.tar.gz (245 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [27 lines of output]
Traceback (most recent call last):
File "/home/runner/work/nevergrad/nevergrad/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in
main()
File "/home/runner/work/nevergrad/nevergrad/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/runner/work/nevergrad/nevergrad/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 487, in run_setup
super().run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "", line 41, in
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/init.py", line 104, in setup
return distutils.core.setup(**attrs)
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 146, in setup
_setup_distribution = dist = klass(attrs)
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 318, in init
self.metadata.version = self._normalize_version(self.metadata.version)
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 354, in _normalize_version
normalized = str(Version(version))
File "/tmp/pip-build-env-soo31i8e/overlay/lib/python3.9/site-packages/setuptools/_vendor/packaging/version.py", line 200, in init
raise InvalidVersion(f"Invalid version: '{version}'")
setuptools.extern.packaging.version.InvalidVersion: Invalid version: 'main'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.

How did you test this change?
I tested these changes in a forked repo.
https://fburl.com/workplace/f6mz6tmw

Thanks! This inspired #1654 .


feat: look for config-files in git-repos

In my setup, I have workspaces for rather large projects that consist of multiple directories with multiple repositories. Without this change, typos-lsp/typos just looks for a config-file at the workspace-root and parent directories and thus will not find the relevant configuration files. With this change, it will first traverse any child-directory per workspace to find git-repositories (basically directories that contain a .git-directory or file). It will not traverse further by design.
Other repository-types can be added in follow-up PRs.
Please let me know, if a setting to enable/disable searching for repositories is desired.

One of the design goals of typos-lsp is to mimic the behaviour of typos ... do you think this deviates from how typos would handle the loading of config files?

I either manually run typos inside the actual working directory, usually a repository, run it as a pre-commit-hook or in a CI. All of these use case are on repository-level, not workspace-level. I wrote this change, because I kept getting warnings about typos which I disabled in a typos.toml, but they were not read, because I had multiple directories and repositories within a workspace, which annoyed me.

I can see it's useful for your use-case, and that typos doesn't have a concept of workspace-level. That's an IDE concept - which IDE do you use?
Currently typos-lsp defaults to working as the cli would, if the cli ran in the workspace's directory.


KKM: threads_exit_grp test fails with KKM on AWS occasionally

Happens once a while.
not ok 443 threads_exit_grp(glibc_dynamic): force exit when threads are in flight (exit_grp_test.fedora.dyn)
# (from function `assert' in file bats-assert/src/assert.bash, line 46,
#  in test file km-bats-7.glibc_dynamic, line 641)
#   `assert [ $status -eq 17 -o $status -eq 11  ]' failed with status 139
# start: 2022-07-18 09:11:43.911519623+00:00 threads_exit_grp(glibc_dynamic): force exit when threads are in flight (exit_grp_test.fedora.dyn)
# --- Test script output:
#
# -- assertion failed --
# expression : [ 139 -eq 17 -o 139 -eq 11 ]
# --
#
#   end: 2022-07-18 09:11:46.144535957+00:00 threads_exit_grp(glibc_dynamic): force exit when threads are in flight (exit_grp_test.fedora.dyn)
#
# --- Command line:
# km_with_timeout exit_grp_test.fedora.dyn
# --- Command output:
# Exit(11) from from a non-main thread
# ==32166== Thread 5 vcpu-3:
# ==32166== Invalid read of size 4
# ==32166==    at 0x4860593: pthread_sigqueue (in /usr/lib64/libpthread-2.30.so)
# ==32166==    by 0x114A4E: km_signal_vcpu_stop (km_cpu_init.c:403)
# ==32166==    by 0x114A4E: km_vcpu_not_paused (km_cpu_init.c:427)
# ==32166==    by 0x115936: km_vcpu_apply_all (km_cpu_init.c:378)
# ==32166==    by 0x115936: km_vcpu_apply_all (km_cpu_init.c:365)
# ==32166==    by 0x115B52: km_vcpu_pause_all (km_cpu_init.c:452)
# ==32166==    by 0x1405F9: km_vcpu_exit_all (km_vcpu_run.c:470)
# ==32166==    by 0x1405F9: km_vcpu_run (km_vcpu_run.c:659)
# ==32166==    by 0x48584E1: start_thread (in /usr/lib64/libpthread-2.30.so)
# ==32166==    by 0x49726C2: clone (in /usr/lib64/libc-2.30.so)
# ==32166==  Address 0x2d0 is not stack'd, malloc'd or (recently) free'd
# ==32166==
# ==32166==
# ==32166== Process terminating with default action of signal 11 (SIGSEGV): dumping core
# ==32166==  Access not within mapped region at address 0x2D0
# ==32166==    at 0x4860593: pthread_sigqueue (in /usr/lib64/libpthread-2.30.so)
# ==32166==    by 0x114A4E: km_signal_vcpu_stop (km_cpu_init.c:403)
# ==32166==    by 0x114A4E: km_vcpu_not_paused (km_cpu_init.c:427)
# ==32166==    by 0x115936: km_vcpu_apply_all (km_cpu_init.c:378)
# ==32166==    by 0x115936: km_vcpu_apply_all (km_cpu_init.c:365)
# ==32166==    by 0x115B52: km_vcpu_pause_all (km_cpu_init.c:452)
# ==32166==    by 0x1405F9: km_vcpu_exit_all (km_vcpu_run.c:470)
# ==32166==    by 0x1405F9: km_vcpu_run (km_vcpu_run.c:[659](https://github.com/kontainapp/km/runs/7386155103?check_suite_focus=true#step:3:668))
# ==32166==    by 0x48584E1: start_thread (in /usr/lib64/libpthread-2.30.so)
# ==32166==    by 0x49726C2: clone (in /usr/lib64/libc-2.30.so)
# ==32166==  If you believe this happened as a result of a stack
# ==32166==  overflow in your program's main thread (unlikely but
# ==32166==  possible), you can try to increase the size of the
# ==32166==  main thread stack using the --main-stacksize= flag.
# ==32166==  The main thread stack size used in this run was 8388608.
#
# Killed by signal. Timeout returns 139: /home/appuser/km/build/opt/kontain/bin/km --virt-device=/dev/kkm --km-log-to=stderr --copyenv=LD_PRELOAD  exit_grp_test.fedora.dyn
# ---

Problem seems to happen on valgrind runs.

This is modified code to identify the problem found on valgrind runs.
The vcpu is in STARTING state, vcpu_thread is null and we call pthread_sigqueue with NULL pointer which causes KM crash.
399 static inline void km_signal_vcpu_stop(km_vcpu_t* vcpu)
400 {
401    sigval_t val = {.sival_int = vcpu->vcpu_id | VCPU_STOP};
402
403    if (vcpu->vcpu_thread == 0) {
404       assert(vcpu->state != STARTING);
405    }
406
407    km_pkill_no_esrch(vcpu->vcpu_thread, KM_SIGVCPUSTOP, val);
408 }
698 #define km_pkill_no_esrch(thread, signo, val)
699    do {
700       int ret;
701       if ((ret = pthread_sigqueue(thread, signo, val)) != 0 && ret != ESRCH) {
702          km_err(ret, "pthread_sigqueue(" #thread ", " #signo ", " #val ") Failed ");
703       }
704    } while (0)
--- Command line:
km_with_timeout exit_grp_test.fedora.dyn
--- Command output:
Exit(11) from from a non-main thread
==423123== Thread 3 vcpu-1:
==423123== Invalid read of size 4
==423123==    at 0x48E1BDA: __vfxprintf (fxprintf.c:69)
==423123==    by 0x48E1CB9: __fxprintf (fxprintf.c:81)
==423123==    by 0x48A0479: __assert_fail_base (assert.c:64)
==423123==    by 0x48A0575: __assert_fail (assert.c:101)
==423123==    by 0x1142E2: km_signal_vcpu_stop.part.0 (km_cpu_init.c:404)
==423123==    by 0x1143DD: km_signal_vcpu_stop (km_cpu_init.c:407)
==423123==    by 0x1143DD: km_vcpu_not_paused (km_cpu_init.c:431)
==423123==    by 0x1143DD: km_vcpu_not_paused (km_cpu_init.c:426)
==423123==    by 0x115225: km_vcpu_apply_all (km_cpu_init.c:378)
==423123==    by 0x115225: km_vcpu_apply_all (km_cpu_init.c:365)
==423123==    by 0x115442: km_vcpu_pause_all (km_cpu_init.c:456)
==423123==    by 0x13FEC9: km_vcpu_exit_all (km_vcpu_run.c:470)
==423123==    by 0x13FEC9: km_vcpu_run (km_vcpu_run.c:659)
==423123==    by 0x48F5E2C: start_thread (pthread_create.c:442)
==423123==    by 0x497A7D3: clone (clone.S:100)
==423123==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==423123==
==423123==
==423123== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==423123==  Access not within mapped region at address 0x0
==423123==    at 0x48E1BDA: __vfxprintf (fxprintf.c:69)
==423123==    by 0x48E1CB9: __fxprintf (fxprintf.c:81)
==423123==    by 0x48A0479: __assert_fail_base (assert.c:64)
==423123==    by 0x48A0575: __assert_fail (assert.c:101)
==423123==    by 0x1142E2: km_signal_vcpu_stop.part.0 (km_cpu_init.c:404)
==423123==    by 0x1143DD: km_signal_vcpu_stop (km_cpu_init.c:407)
==423123==    by 0x1143DD: km_vcpu_not_paused (km_cpu_init.c:431)
==423123==    by 0x1143DD: km_vcpu_not_paused (km_cpu_init.c:426)
==423123==    by 0x115225: km_vcpu_apply_all (km_cpu_init.c:378)
==423123==    by 0x115225: km_vcpu_apply_all (km_cpu_init.c:365)
==423123==    by 0x115442: km_vcpu_pause_all (km_cpu_init.c:456)
==423123==    by 0x13FEC9: km_vcpu_exit_all (km_vcpu_run.c:470)
==423123==    by 0x13FEC9: km_vcpu_run (km_vcpu_run.c:659)
==423123==    by 0x48F5E2C: start_thread (pthread_create.c:442)
==423123==    by 0x497A7D3: clone (clone.S:100)
==423123==  If you believe this happened as a result of a stack
==423123==  overflow in your program's main thread (unlikely but
==423123==  possible), you can try to increase the size of the
==423123==  main thread stack using the --main-stacksize= flag.
==423123==  The main thread stack size used in this run was 8388608.
timeout: the monitored command dumped core

Killed by signal. Timeout returns 139: /data/f36-kontain/km/build/opt/kontain/valgrind/bin/km --virt-device=/dev/kkm --km-log-to=stderr --copyenv=LD_PRELOAD  exit_grp_test.fedora.dyn
---