Si.memLayout not returning anything on Linux

Expected behavior and actual behavior.

On windows

Steps to reproduce the problem.

Get linux
Execute the following

Hi, can you try the following command on your linux machine:
dmidecode -t memory

maybe you need admin privileges here to get a valid output.
Thank you in advance

With Admin Privileges:
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.8 present.

Handle 0x0041, DMI type 16, 23 bytes
Physical Memory Array
	Location: System Board Or Motherboard
	Use: System Memory
	Error Correction Type: None
	Maximum Capacity: 32 GB
	Error Information Handle: Not Provided
	Number Of Devices: 4

Handle 0x0042, DMI type 17, 40 bytes
Memory Device
	Array Handle: 0x0041
	Error Information Handle: Not Provided
	Total Width: 64 bits
	Data Width: 64 bits
	Size: 4096 MB
	Form Factor: SODIMM
	Set: None
	Locator: ChannelA-DIMM0
	Bank Locator: BANK 0
	Type: DDR3
	Type Detail: Synchronous
	Speed: 1600 MHz
	Manufacturer: Samsung
	Serial Number: 37F11DEB
	Asset Tag:<PHONE_NUMBER>
	Part Number: M471B5173QH0-YK0
	Rank: 1
	Configured Clock Speed: 1600 MHz
	Minimum Voltage: 1.35 V
	Maximum Voltage: 1.5 V
	Configured Voltage: 1.35 V

Handle 0x0044, DMI type 17, 40 bytes
Memory Device
	Array Handle: 0x0041
	Error Information Handle: Not Provided
	Total Width: Unknown
	Data Width: Unknown
	Size: No Module Installed
	Form Factor: DIMM
	Set: None
	Locator: ChannelA-DIMM1
	Bank Locator: BANK 1
	Type: Unknown
	Type Detail: None
	Speed: Unknown
	Manufacturer: [Empty]
	Serial Number: [Empty]
	Asset Tag:<PHONE_NUMBER>
	Part Number: [Empty]
	Rank: Unknown
	Configured Clock Speed: Unknown
	Minimum Voltage: Unknown
	Maximum Voltage: Unknown
	Configured Voltage: Unknown

Handle 0x0045, DMI type 17, 40 bytes
Memory Device
	Array Handle: 0x0041
	Error Information Handle: Not Provided
	Total Width: 64 bits
	Data Width: 64 bits
	Size: 4096 MB
	Form Factor: SODIMM
	Set: None
	Locator: ChannelB-DIMM0
	Bank Locator: BANK 2
	Type: DDR3
	Type Detail: Synchronous
	Speed: 1600 MHz
	Manufacturer: Samsung
	Serial Number: 37F11DE4
	Asset Tag:<PHONE_NUMBER>
	Part Number: M471B5173QH0-YK0
	Rank: 1
	Configured Clock Speed: 1600 MHz
	Minimum Voltage: 1.35 V
	Maximum Voltage: 1.5 V
	Configured Voltage: 1.35 V

Handle 0x0047, DMI type 17, 40 bytes
Memory Device
	Array Handle: 0x0041
	Error Information Handle: Not Provided
	Total Width: Unknown
	Data Width: Unknown
	Size: No Module Installed
	Form Factor: DIMM
	Set: None
	Locator: ChannelB-DIMM1
	Bank Locator: BANK 3
	Type: Unknown
	Type Detail: None
	Speed: Unknown
	Manufacturer: [Empty]
	Serial Number: [Empty]
	Asset Tag:<PHONE_NUMBER>
	Part Number: [Empty]
	Rank: Unknown
	Configured Clock Speed: Unknown
	Minimum Voltage: Unknown
	Maximum Voltage: Unknown
	Configured Voltage: Unknown

Thank you!
To be more explicit, the command returns result but does si.memLayout() also with admin privileges anyway returns noting (empty array)?

Hi again, did you manage to test si.memLayout() with admin privileges? Seeing the output of the underlying command, this should return correct values. Can you confirm that?

Closing it for now as I cannot reproduce it any more ...


Not served over HTTPS

pearsearch.org is not currently served over HTTPS.
This is due to GitHub bug #156: Add HTTPS support to Github Pages including custom domains.
The failure to use HTTPS puts visitors to the website at increased risk of various attacks. This might be less of a concern if pearsearch.org did not expect visitors to execute JavaScript, but it does.
Resolving #5 would potentially also resolve this issue, by migrating pearsearch.org (and indeed the rest of the PeARS project's code hosting) off GitHub and onto a host that does support HTTPS for custom domains.
Thanks!

Yes Yes.  Moving pearsearch.org into our own server is already in our TODO list :)

As an interim solution (i.e. not requiring the investment of time, attention and money required by self-hosting), you could still potentially solve this by resolving #5 :)

Sorry we're being slow with this -- most of PeARS team is currently either travelling or battling with urgent neural networks unrelated to PeARS ;) But yes, we have #5 on our discussion list for the next meeting. Thanks!

The site uses HTTPS now.


Login

Hello everyone!
This pull request introduces login in our app, which consists in a modal that is opened when the user arrives on the page and that contains a username/password form. The credentials are verified thanks to a login correspondences file, stored in the repository but encrypted with the Simple Crypt library. To open it, an environment variable LOGIN_CORRESPONDENCES_KEY is required, which corresponds to the password used for the encryption of the file (this allowed to add this variable in the secret configuration variables on Heroku).
A new callback manages the login process, ie. checks the user's credentials and if these are validated, closes the modal. The callback also updates the center and zoom attributes of the map object, so that the user ends up with a view of his department only. It also fetches site devices data from the API which will probably prove useful later on, when dealing with the alerts flow.
This summarises the main changes to homepage.py and main.py. As for alerts.py, I added a function which allows to build the vision angle of a camera based on various inputs (location of the device, yaw, opening angle, etc).
NB: in main.py, I also commented the callback updating the zoom and center attributes of the map object based on clicks on the alert banner, to avoid any conflict with the login callback and because - if I am not mistaken - we are not keeping the banner in the next steps.
Let me know if you have any comments or questions of course, here are a few screenshots of how it renders:

There seems to be an error with Docker when trying to install a dependency (pycrypto) of the Simple Crypt library. I will look into it as soon as possible!

Hi @pechouc nice PR, thank you very much. I've done some small comments.
There is an issue with the callback update_alert_data as you can see on this screen. I've investigated a bit and its linked to the fact that now the response = api_client.get_ongoing_alerts().json() returns an empty json (and the corresponding dataframe has no columns ...).

To prevent this issue, you can change the callback to:
@app.callback(
    Output("store_live_alerts_data", "data"),
    Input('interval-component-homepage', 'n_intervals')
)
def update_alert_data(interval):
    """
    The following function is used to update the div containing live alert data from API.
    """
    # Fetching live alerts where is_acknowledged is False
    response = api_client.get_ongoing_alerts().json()
    all_alerts = pd.DataFrame(response)
    if all_alerts.empty:
        raise PreventUpdate
    live_alerts = all_alerts.loc[~all_alerts["is_acknowledged"]]

    return live_alerts.to_json()

Otherwise I've tested a bit, seems great to me ! 😄

Agreed, debugged to the db reset 😅


nixos/mastodon: add assertion for only allowing one sidekiq scheduler queue

Description of changes
This was meant to be included in https://github.com/NixOS/nixpkgs/pull/202408, but apparently I forgot to push it before merging.
This assertion makes sure, that there can be only one Sidekiq scheduler queue present, as upstream seems to be very cautious about it. See also this discussion: https://github.com/NixOS/nixpkgs/pull/202408#discussion_r1041082412
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

@SuperSandro2000 Is there a reason you approved, but didn't merge? Sometimes I get confused about the dos and dont's. Would you rather have a second person approve and merge this or would you like me as the maintainer to do the merge? I find myself self-merging urgent Mastodon updates sometimes, because the review capacity for Mastodon on NixOS is pretty low in general.

Didn't have much of a reason

Doesn't seem to work as intended. Test values are from the example.
# cat mastodon.nix | grep sidekiqProcesses -A17
    sidekiqProcesses = {
      all = {
        jobClasses = [ ];
        threads = null;
      };
      ingress = {
        jobClasses = [ "ingress" ];
        threads = 5;
      };
      default = {
        jobClasses = [ "default" ];
        threads = 10;
      };
      push-pull = {
        jobClasses = [ "push" "pull" ];
        threads = 5;
      };
    };
# nixos-rebuild switch
building Nix...
building the system configuration...
error:
       Failed assertions:
       - There must be one and only one Sidekiq queue in services.mastodon.sidekiqProcesses with jobClass "scheduler".
(use '--show-trace' to show detailed location information)

@jollheef Ouch, thanks for letting me know. Care to review the fix? https://github.com/NixOS/nixpkgs/pull/227338


Update Docs and Manifests for v0.19.0

Added v0.19 references to README
Update k8s manifests with v0.19.0 references

/assign @damemi @ingvagabund


Subtree memorization / reactive templates

Allows templates to be created to skip diffing on static parts of the html. The templates can be changed when the program is running to allow for hot reloading. The dynamic parts of the templates contain a index that maps dynamic nodes/attributes in the template to the captured dynamic nodes/attributes. This allows dynamic parts to be moved for hot reloading.
The templates allow for fine-grained diffing within a node. It will only diff attributes within a node that are dynamic, and only the dynamic parts of a formatted text node.

[x] Create DomEdits for templates
[x] Create structure for templates

[x] Allow static/dynamic attributes in the same element

[x] Allow Dynamically created or Statically created versions of templates

[x] Allow skipping to any point in the template based on what part was edited

[x] Handle volatile values in templates

[x] Create templates with the rsx macro
[x] Diff templates
[ ] Handle templates in native-core
[x] Handle templates in web interpreter

[x] Render static templates

[x] Handle updating templates

[ ] Handle templates in desktop
[ ] Use templates in hot reloading
[ ] Update documentation
[ ] Use keys in the template to provide a key for the template?

This is awesome!
I think it might be worth writing down some of the architecture and approach if you have anything in mind.

Here is what I am thinking for the architecture:
Diouxs side:

A new vnode variant VNode::NodeRef that contains a reference to a template and the dynamic context to hydrate the template.
Each dynamic part of the template contains a index that points to the dynamic part to use so that the templates created by the hot reloading cli can move the dynamic parts around.

Renderer side:
New DomEdits:

CreateTemplate{template_id}
CreateTemplates starts the creation of a template. A template node is created through normal DomEdit messages, but must be allocated on a stack that is different from the normal vnode stack because the memory is managed differently.
FinishTemplate{}
FinishTemplate finishes the template creation.
CreateTemplateRef{id, template_id}
Create a reference to a template. When the reference is on the top of the stack, any ids in messages refer to ids in the template stack instead of the vnode stack

Open questions:

How should the templates be stored to allow for heap allocated and &'static variants?
a. The only solution I can think of is an enum with static and allocated variants, but it would result in a lot of duplication.
Is there a better way to structure the DomEdits?

The web implementation is mostly complete (🎉), so I thought it might be a good time for more notes:
I forked and updated the js framework benchmark to work with this branch, and we preform better than react for all but one of the performance related benchmarks:

This also improves memory usage compared to the 0.2 results, but it is probably the area we most need to improve:

A very breaking change to attributes:
This PR changes attributes to be represented as an associated constant instead of a function, to allow them to be used at compile time. For the most part this shouldn't effect existing user code, but it would break custom renderers.

Everything appears to work. I checked all the examples, and they work correctly. Ssr, tui, liveview, web, desktop, and hot-reloading all seem to work.
I don't love the way the mutations are structured right now, but they work. If there is any better ideas, I'm open to suggestions. I think shifting to a mutations trait instead of struct might be nice so that different platforms can store them differently. Eg: The ffi friendly mutations could be stored differently form the native-core mutations meant for use in rust, which is stored differently from the binary encoding used for web, etc.
Here is the final performance results:

@Demonthos Note that cargo install --git https://github.com/DioxusLabs/cli (the only current recommended way to install the CLI) is now broken with:
  no matching package named `dioxus-rsx-interpreter` found


Re-design and new website

Adds support for learning cryptography and comes with a brand new design.

[x] New re-designed UI Components
[x] New Playground
[x] Update API
[x] Update API documentation
[x] New logo

All the commits are already merged into the branch v2, so closing this PR.


Fix doc links and add build job to check the documentation

The PR fixes unresolved doc links and adds a build job to check the documentation on rust nightly with denied
intra_doc_link_resolution_failure warnings.
Related: #202
What about the use of intra-doc links instead of urls to the rendered files?

What about the use of intra-doc links instead of urls to the rendered files?

This has sadly not landed in nightly yet, but when it does we should switch to it.

The docs.rs is using nightly to build the docs anyway and cargo +nightly doc --open makes it easy if local/offline access is needed.

I think we should avoid intradoc links until the crates are on crates.io/docs.rs. I use local docs when I develop my stuff (like my bot), so until everyone can switch off of depending on the repo I think we should avoid a nightly dependency.


use hungarian notation for laravel

Use the hungarian notation for the Framework. Debugging of web applications would be easier for developers. (https://en.wikibooks.org/wiki/PHP_Programming/Alternative_Hungarian_Notation)

Oh my god please, don't... (I know that Laravel will never switch)
This notation is so 80s...

I'm in the middle of an investigation.  Whomever references that WikiBook needs to be put in Internet jail.


feat: support Preset#inject negative override / subtraction

Sometimes an environment might want to undo any kind of inject configuration provided by the previous preset, because this environment guaratees that an API is provided natively and doesn't need to be supported via injection.
An example is Cloudflare's workerd, which natively support Blob and global when the nodejs_compat flag is turned on.

I'm fine with false. I'll update later today.


fix: rename Characterisitc to Characteristic

close #149

I think I'm going to go ahead and fix other typos while I'm at it in this PR.


Scaling SaaS model (one DbContext, database-per-tenant) with EF Core?

There doesn't seem to be any good guidance on how to set up and maintain a database-per-tenant model for EF Core.

How to create/ migrate new tenant dbs programmatically?
How to manage connection strings?
How to select a specific tenant at runtime?

Sharding seemed to be supported in .NET 4.x but no support in .NET Core or beyond. How is EF Core used in a scalable, sustainable manner in a SaaS environment with tenant data isolation?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb29a2b6-c401-287f-2da9-1aeb3c633b41
Version Independent ID: bb29a2b6-c401-287f-2da9-1aeb3c633b41
Content: Multi-tenancy - EF Core
Content Source: entity-framework/core/miscellaneous/multitenancy.md
Product: entity-framework
Technology: entity-framework-core
GitHub Login: @JeremyLikness
Microsoft Alias: jeliknes

Sharding seemed to be supported in .NET 4.x

Can you elaborate what you're referring to here?

Sure, there's support for elastic tooling in .NET 4.x (see here) which seems to support programmatically scaling database "shards" in Azure, but is not supported for .NET Core or .NET 5.x/ 6.x+ (see the issues page)
The project also does not seem to be receiving any official support at all at this point, which makes very few options available for programmatically scaling out a SaaS database-per-tenant model.

@jeremy-holovacs-sp that seems to be an Azure SQL-specific library, that specifically works at the ADO.NET layer, and so probably isn't relevant for working with EF Core. It indeed doesn't seem to have received much attention recently (last commit in 2020), but that project doesn't really have anything to do with us...
Regardless, our docs do contain some information on the database-per-tenant model - see here and below - which isn't to say there isn't room to add more.
All that isn't to say we shouldn't look into improving our guidance for using EF Core with database sharding

@roji That's understood; I was offering the Azure project as an example of a dynamic, scalable SaaS database-per-tenant model that I feel is tragically underrepresented for EF; if the project were still actively supported we'd probably have been able to use that instead, but it's not. We use EF Core against Azure, but are running into serious scaling challenges because of this limitation and we're having to get... creative... in our horizontal scaling efforts. It's very frustrating.
I have seen the documentation you referred to, but that really doesn't solve the main problem we have. We need to dynamically add databases, maintain a store of connection strings, and programmatically run migrations at scale. The example documentation is very static, and requires storing connection string data for a finite number of databases in a configuration file, which is... messy, among other things. It has to be maintained at design-time.
I'd really like a more enterprise-ready scaling solution. I would hope to see some guidance/ support for this in the future.

Hi @jeremy-holovacs-sp
I too found the elastic tooling code, which has a great set of features, but isn't supported any more. Version 3 of my AuthPermissions.AspNetCore library provides both straight sharing and a hybrid version, just like elastic tooling (did) do. I should say that some elastic tooling features, like stopping access to a tenant while it is moved, aren't in yet but I have some ideas on how to do that.
As well as the articles that @roji mentioned you might like to see the ASP.NET Core community standup I did a a month ago about Multi-Tenant applications. This gives you a overview of the features (and my approach) to multi-tenant - and as Jon Galloway said, there a lot of other libraries that help with multi-tenant apps.

@JonPSmith That's fantastic, I'll take a look at this. Thanks!


update app version

belongs to #39
updating version for new release to app store

Will wait till app is approved on App Store Connect before merging


Investigate protection against multi-access lockup

A thread on the Pi Camera forum indicates it may be possible to guard against lockup of the camera when multiple processes attempt to access it:

While fumbling with vcdbg syms and vdbg dump 0xdec02020 (the address of cam_alloc_alloc_count), I found that when the camera is not being used, the value of cam_alloc_alloc_count is 0, and when in use, it is 1.
Now to find an appropriate place to implement a test for this to prevent crashing the camera.

This should be added to PiCamera.__init__ but looks like it'll require a bit more header translation, hence it won't make it into 0.5, but should be added before 1.0.

Hello Dave,
I'm using your livestream on the raspberry PI. Everything is working fine. But if I want to make a picture wiht the following command: raspistill -o image.jpg during the live stream is running I get the following error message:
mmal: mmal_vc_component_enable: failed to enable component: ENOSPC
mmal: camera component couldn't be enabled
mmal: main: Failed to create camera component
mmal: Failed to run camera app. Please check for firmware updates
Can you help me?
Is it possible to create pictures or videos during the livestream?
Thank you!

It is possible, but only from within the same process; i.e. if one process (pistreaming or some picamera script) is using the camera, another process (like raspistill or raspivid) can't. However, the camera can produce JPEGs while recording video as shown in this recipe so if you can adjust your script to do both, you should be good to go.

Thank you Dave. In what file I have to add this code?
import picamera
with picamera.PiCamera() as camera:
camera.resolution = (800, 600)
camera.start_preview()
camera.start_recording('foo.h264')
camera.wait_recording(10)
camera.capture('foo.jpg', use_video_port=True)
camera.wait_recording(10)
camera.stop_recording()
Thanks!

Well, it's not quite a matter of just copying and pasting that code somewhere - you need to understand what it does and then use that technique in whatever script you're working with. So, firstly when you say you're using a "livestream" what script is that exactly?


Menu entry swapper for escort

Fixes issue #5544

I have an open request for this at the moment #5383

This is a niche feature that is best suited for the plugin hub, and is currently covered by the Menu swapper extented plugin there. See runelite/plugin-hub#42


fix HF inference benchmark

pipe.model.model_times()) is only for deepspeed inference.
So, introducing under if condition.

Hi @satpalsr, we recently moved the DeepSpeed Benchmarks and this fix was partially implemented. Could you please open a PR in the DeepSpeedExamples repo to include the changes to printing the model times?
#3157 will remove the duplicate benchmark scripts
Thank you!

Thanks for the info @mrwyattii. Moved here https://github.com/microsoft/DeepSpeedExamples/pull/262


AKS - Waiting for Operator to register custom resource definitions....

I am testing the installation in a Cluster with Azure AKS. In a bash terminal I installed azure cli, got the aks credentials and runthe installer. He stops at the item below
*>>> Waiting for Operator to register custom resource definitions......
K8s: 1.11.3
Version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T17:53:03Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Nodes:
NAME                       STATUS    ROLES     AGE       VERSION
aks-agentpool-21009078-0   Ready     agent     1d        v1.11.3
aks-agentpool-21009078-1   Ready     agent     1d        v1.11.3
Thanks,
Ramon
------------------------ LOG ---------
root@LAPTOP:/temp/prometheus-kubernetes# ./deploy
Check for uncommitted changes
WARNING! You have uncommitted changes! Please commit your changes now, then press ENTER
M assets/alertmanager/alertmanager.yaml
M manifests/alertmanager/alertmanager.yaml
M manifests/grafana/grafana-datasources.cm.yaml
M manifests/grafana/grafana.de.yaml
M manifests/kube-state-metrics/kube-state-metrics-rbac.yaml
M manifests/kube-state-metrics/kube-state-metrics.de.yaml
M manifests/node-exporter/node-exporter.ds.yaml
M manifests/prometheus-operator/prometheus-operator-cluster-role-binding.yaml
M manifests/prometheus-operator/prometheus-operator.yaml
M manifests/prometheus/prometheus-k8s-rbac.yaml
M manifests/prometheus/prometheus-k8s-service-monitor-alertmanager.yaml
M manifests/prometheus/prometheus-k8s-service-monitor-kube-state-metrics.yaml
M manifests/prometheus/prometheus-k8s-service-monitor-node-exporter.yaml
M manifests/prometheus/prometheus-k8s-service-monitor-prometheus.yaml
M manifests/prometheus/prometheus-k8s.yaml
[ENTER]:
Enter desired namespace to deploy prometheus [monitoring]:
Creating monitoring namespace.
Error from server (AlreadyExists): namespaces "monitoring" already exists

AWS
GCP
Azure
Custom
Please select your cloud provider:3
Deploying on Azure
Are you deploying in Azure Container Engine (AKS)?
Y/N [N]: Y
Remove rules for self hosted clusters

Setting components version
Enter Prometheus Operator version [v0.23.1]:
Enter Prometheus version [v2.3.2]:
Enter Prometheus storage retention period in hours [168h]:
Enter Prometheus storage volume size [40Gi]:
Enter Prometheus memory request in Gi or Mi [1Gi]:
Enter Grafana version [5.2.2]:
Enter Alert Manager version [v0.15.1]:
Enter Node Exporter version [v0.16.0]:
Enter Kube State Metrics version [v1.3.1]:
Enter Prometheus external Url [http://<IP_ADDRESS>:9090]:
Enter Alertmanager external Url [http://<IP_ADDRESS>:9093]:
Do you want to use NodeSelector  to assign monitoring components on dedicated nodes?
Y/N [N]: Y
Make sure you label the nodes correctly. The default label is: beta.kubernetes.io/monit=prometheus
Node label key [beta.kubernetes.io/monit]:
Node label value [prometheus]:
Do you want to set up an SMTP relay?
Y/N [N]: N
Do you want to set up slack alerts?
Y/N [N]: N
Removing all the sed generated files
Deploying Prometheus Operator
serviceaccount/prometheus-operator unchanged
clusterrole.rbac.authorization.k8s.io/prometheus-operator unchanged
clusterrolebinding.rbac.authorization.k8s.io/prometheus-operator unchanged
service/prometheus-operator unchanged
deployment.apps/prometheus-operator unchanged
Waiting for Operator to register custom resource definitions....................................................................................................................
................

Hi,
commit your changed to work, after that change variable:
STORAGE_CLASS_TYPE=Managed
to
STORAGE_CLASS_TYPE=managed-standard
Kubernetes new version changed the name to managed-standard


[BUG] one-time view message does not work now

when receiving the one time view message the message is not displayed, this is necessary for the functionalities I am performing, is there any way to enable it?
the payload received is this:
{
"key": {
"remoteJid"<EMAIL_ADDRESS>"fromMe": false,
"id": "TEST"
},
"messageTimestamp":<PHONE_NUMBER>,
"pushName": "USER",
"broadcast": false,
"messageStubType": 2,
"messageStubParameters": [
"Message absent from node"
]
}

Duplicate! Why didn't you read other issues?


Cant disarm from Home Assistant

I can enable arm and night_arm but any attempt to disarm does not work
Service call from Home Assistant is pretty basic:
service: alarm_control_panel.alarm_disarm
data:
code: "XXXX"
target:
entity_id: alarm_control_panel.g4s_alarm
I have tried:

double and triple checked the password.
Changed tag passcode in G4S and adjusted the service call in Home Assistant
created a tag without a passcode in G4S and run the service from HA without a passcode.
running the service without "" around code value
disarming from the little numpad on the entity.

No warning or error is thrown and Log is empty. even with debugging enabled from the integration.
I even get the little checkmark on the button when clicking "run service" in the HA "Services" menu.
But the alarm is never disarmed.
Im running the beta version that fixed this issue: https://github.com/PTST/G4S-HomeAssistant/issues/4
I will be happy to provide additional info.

Im not sure what is wrong with the Home Assistant integration in my situation, but i took the liberty to look into the code and was able to fire a post request from Postman to G4S servers that actually disarmed the alarm. But i never used the pin code. I used the PanelID the state info (systemstatus/getState).

Hi @hausner
The API does not actually need the pin to disarm it.
What happens in the code is that we get a list of all users and their pins and then check locally whether that pin is matching a known user pin before allowing sending of the disarm request.
Are you sure that a pin is actually set for the user?
If you in postman makes a post request to
https://mit.g4severhome.dk/ESI.API/API/systemstatus/getState

with the body
{
	"panelId": "{{ panel_id }}",
	"username": "{{ username }}",
	"password": "{{ password }}"
}

then in the received object you nedd to go the "users" property and check that one of the users in the array has a property called accessCode with the pin you used in home assistant set

hmm. The accesscode under all user properties are set to null.
The only place the accesscode is set is under "stateDevices".  So for example on a device named "Martins chip"

Are you able to disarm your alarm without using the chip? So by just entering your passcode on the alarm panel?

My user settings are set like this in the app. But be warned that this will alter you settings to allow disarm without a chip

hmm. i only have the "Chips" section, no "Bruger adgangskode".
Any idea how to add that?

Unfortunately not. But i'm just now testing a new version that also checks the chip access codes. It should be ready for release later today.

Please try upgrading to v0.2.4, this version should fix your issue

Sorry for the delay.
Tried the 0.2.4 with no luck though. Still cant disarm from HA.
I see that i have a chip in "stateDevices" with "Type": 201 which i guess should match the DeviceType.ACCESSCHIP (could find it in the code to check though).
That state device has the "accessCode":  value set to the correct pin that im using in HA. But still no luck
snippet:
"accessCode": "xxxx",
"key": null,
"isTampered": null,
"hasLowBattery": null,
"hasSupervisionFault": null,
"Id": 4,
"Type": 201,
"Name": "Tag-4",
"ParentDeviceId": null,
"PanelId": 10004608

There seemed to be a parsing error in the underlying api library.
I have fixed it and released a new version of the integration, please try the 0.2.5 version just release

yeah, works now.
Thanks a lot. Really appreaciate it


Max Email Height Issue in Windows Outlook 2013+

I've come across a maximum height issue for the entire email body in Win Outlook 2013+ and was hoping someone may have previously seen a workaround?
I know there is a max-height for images and the text-boundary concern which forces the use of small vertically stacked containers to work around.
I've come across the problem in both hand coded and mjml generated builds.
Examples of two different templates below both showing the issue in 2013+ on Windows:
MJML generated example:

Hand-Coded example previously encountered (excuse the inconsistencies, that's why I switched):

Even more perplexing is that the limit seems arbitrary, in the examples above the MJML example is 3146px tall in browser whereas the hand-coded example is 10000px+.
Could potentially be a litmus bug. I've asked them to look into it too, I haven't the physical machines to test this theory.

Identified as a Litmus bug, closing issue.


thrift_test loses test line numbers

I don't know if it's easy to fix this and it shouldn't hold us back I don't think, but I thought I'd make an issue for discussion's sake (maybe someone has an idea for a quick fix).  The thrift_test macro makes it impossible to run, e.g., mix test test/thrift/generator/binary_protocol_test.exs:599 because the line numbers get lost.  Maybe it's as simple as adding location: :keep to the defmacro?

It's a pretty complex macro. It predates the compile.thrift integration. Would it be possible to convert the thrift_test stuff to simple fixtures now?

Rule 1: Don't Write Macros

Metaprogramming Elixir, Chris McCord

I think I would prefer that.  Every time I've tried to use macros to automate tests, it's come back to bite me eventually :/   It would be a fairly large undertaking to convert everything, though :(
One compromise might be to change the macro so that it's intended to be used either as an annotation or inside the test case - i.e., either
@thrift_test "idunno what would go here"
test "the test case" do
  # test assertions
end

or
test "the test case" do
  @thrift_test "idunno what would go here"
  # test assertions
end

Or better yet, if it could all just be put in the setup/setup_all blocks somehow and anything specific to a particular test could just be a function call?

I would also like us to rely less on disk-based file generation where possible. We should be able to write the majority of our tests in terms of in-memory, AST-based assertions.

It would be really handy if there was still an option to write the generated code to disk - I've found that to be super handy for debugging.

My $0.02: When debugging, I've found it incredibly helpful to have a file with the generated code out there.
We should not conflate the macros being difficult to debug with having artifacts on disk.
If I were to re-do this, I'd leverage describe blocks and setup functions. It would be cool to do:
thrift_file "filename.thrift",  contents: """
 // thrift contents
"""

describe "using filenae.thrift" do
  setup :generate_filename

  test "doing something"  do
    # in here, you have everything inside of the above thrift file
  end

Basically, the thrift_file function builds a function that can be used in a setup callback in a describe block. It'd also generate on_exit teardown fns.

I'm fine with there being an option to write the generated files to disk, but I don't like the fact that so many of our tests are dependent upon disk-based code generation. (The mix and generator tests are an obviously exception.)

@jparise can you clarify why you have this objection? Of all the things, it doesn't seem like it's causing problems.

@jparise I'm fine with an option too, as it's similar to what we have now (our option doesn't clean up the files).

I have picked up this issue. I just have a problem with containers.


Figure out how to do styling now that AmbientTextStyle and AmbientContentColor are gone

In alpha07, these were moved from foundation into material libraries.
We can no longer automatically style text, instead we need to provide hooks for converting rich text style into whatever ambients you need to for your design systems.
Another possible option is providing our own mini design system, with our own text components that directly consume the rich text themes.

Hi Zach, have you considered making a "basic" version of RichText? Similar to the foundational BasicText. I.e:
@Composable
public fun BasicRichText(
  modifier: Modifier = Modifier,
  textStyle: TextStyle = TextStyle.Default,
  richTextStyle: RichTextStyle? = null,
  children: @Composable RichTextScope.() -> Unit
)

Then design systems could easily add their own RichText composable that sets the correct TextStyle (which includes a color). And if you want to keep the convenience of having a composable that works out-of-the-box for Material, then we could also add a richtext-material module that sets up the textStyle using LocalTextStyle and LocalContentColor.
(For context, we're hoping to use this library for its markdown support, but at the moment there's no way for us to set the default TextStyle since we don't use Material in our app)

Basic Rich Text sounds like an oxymoron to me. I would think a better solution would be to build the necessary styling primitives into this library, since it is basically a design system itself. I haven't thought much about how to do that though, and how to make it flexible enough to use from other design systems.
Curious what Siyamed and Sean think about this.

Ahoy!
First, I'd love to get input on the current API design here, as well as share our thinking.
Splitting theme aware and themed composables
We made an intentional choice in Compose to make BasicTextField theme agnostic. It's expected that real-world usage of compose will have companies (and design libraries) define their own theme system as they will need features that don't fit 1-1 with the Material system. In the simplest case, this may be adding a few extra brand colors, but some systems and designers will likely define an entire alternative typography system.
Looking at this library, it feels very foundational and provides general features that will be useful even in non-material apps, so I'd suggest taking a similar approach. Specifically, this would look like
@Composable fun BasicRichText(...) // theme-agnostic composable that's responsible for just displaying text
@Composable fun RichText(...) // Material-theme integration that reads the correct styles then calls BasicRichText

Then, a developer that's working in a Fancy design system can define FancyRichText in their app code to call BasicRichText with the appropriate styling.
A few notes that we found as we did this split:

Functionality related to displaying rich text should be defined as features on BasicRichText
Functionality related to the design system only should be defined in RichText

Integrating with a theme from a design system
Avoiding design-related ambients in BasicRichText will make it easy to integrate with alternative design systems. Then, even if you do introduce type dependencies in the formal arguments that differ from the types used in the alternative design system (e.g. they use FancyTextStyle), a theme wrapper like the Material RichText proposed above would be the only place that does the type translation.
Defining your own theme system
This one is a bit more of a judgement call - and is something that could make sense given the size of the library. Here's some things to consider

Can the styling be defined accurately using only MaterialTheme?
Will developers see it as a second theme system just to use this library while the rest of their app uses another theme?
Is there a clear advantage to the styling information you can provide in RichTextTheme (e.g. bullet shapes etc)

In this case, I personally think this will be easier to use by integrating with MaterialTheme. However, if there's a clear reason to provide an alternative system (e.g. bullet shapes would be a good reason) then it might make sense to define a new theme and exposing integrations for both. Material for devs that don't need the rich formatting, and the custom system for devs that do. You can also build an adapter from MaterialTheme->RichTextTheme to avoid duplication of colors etc.

Thanks for the detailed writeup Sean! I need to play with this to prove to figure out the details. Unless @halilozercan beats me to it, since he seems to have the right idea and clearly has a real use case to drive it.

I wish I could beat you to it but 3 months long curfew has just been lifted in Turkey so I'll be outside as much as I can, before the inevitable return of curfew 😄 Btw, thanks a lot for this writeup Sean! It is a great summary of theming and styling philosophy of Compose, which got me confused after the BasicText transition.

https://github.com/halilozercan/compose-richtext/pull/2
PR is not worthy of a review for now. Hence, it's targeted to my fork. I would appreciate just a skim through of the new ideas. If we land on a common ground for the new changes, I can make the PR ready for review in short time.

I have a question about this part of @objcode's comment:

Avoiding design-related ambients in BasicRichText will make it easy to integrate with alternative design systems. Then, even if you do introduce type dependencies in the formal arguments that differ from the types used in the alternative design system (e.g. they use FancyTextStyle), a theme wrapper like the Material RichText proposed above would be the only place that does the type translation.

If the decision is for this library to make its own theme, then why would using our own theme composition locals make Material integration trickier? The RichText -> BasicRichText wrapper would still be the only place where that conversion would have to happen, regardless of whether we used locals or something else.
Also, not 100% sure what you mean by "formal arguments" here - as in function arguments instead of implicit locals?

#45 resolves this issue. I'm going to keep this issue open for a little while longer for potential discussion.


Revert upgrade to python 3.6

Sadly we have to revert the update to python 3.6 for now as the used gevent version 1.1 is to old (at least 1.2 is needed for python 3.6, see https://github.com/gevent/gevent/issues/903) but our old web3 v3 needs gevent < 1.2.
To solve this in the future we need to upgrade web3 to v4.

Just out of curiosity: Why hasn't this been caught by travis?

The problem only shows up when calling the push notification API. We don't have tests for that.


Import spec-diffing code that @jamaalbwells built for apigee/registry.

This partially addresses #120 by restoring code that had been removed in https://github.com/apigee/registry/pull/868. Except for the proto location, everything was moved as-is, and all tests pass.
I think we'll want to follow-up with PRs that move this into a pkg directory and integrate it into registry-experimental for further evaluation.

@seaneganx FYI


Annotation result column shows to much information

Problem:
When working on a labeling project and wishing to review the annotation results, the associated column not only displays the annotation result, but also the entire content of the JSON file from the annotation result section. In the Annotation result column, only the label itself should be visible, without the annotation ID or the type of annotation, as these details are not necessary. The inclusion of unnecessary information in the annotation result column can lead to confusion, particularly in the context of a classification task. In the case of object detection or semantic segmentation, an additional column displaying the coordinates may prove to be beneficial.
Additionally, the presence of extra information in the annotation result column complicates the filtering process, as the entire content, including the ID and type, is taken into consideration, thereby diminishing the ease of filtering.
Solution:
To solve the problem, the annotation result column should only show the label itself.
For object detection or semantic segmentation an additional column for the coordinates could be included.

This cache seems to require a manual update not only after the set of labels is changed, but also when the data is changed.
I have to say, it's not really useful, given such constraints 😞 Is there really no way to just render selected labels in the column?


How to train new SRL model ?

Hi, nlpnet is cool, but i cannot find good tutorial on how to train my custom Semantic Role labeling model (language other than english).
I read docs http://nilc.icmc.usp.br/nlpnet/training-srl.html and tried to reproduce, but first problem i faced is that there is no such file nlpnet-train.py.
I need to get new SRL model from text, so i tried to run in console:
python train_srl.py srl pred --gold train_google.txt --data srl-model/
where train_google.txt - is txt file with raw text, train_srl.py is file in \Lib\site-packages\nlpnet\srl
But nothing happens, no errors, no new folder created
Am i right?

Hi, you have to use nlpnet-train.py; train_srl.py doesn't train any models.
When you install nlpnet, nlpnet-train.py is copied from the bin/ directory to the Scripts folder of your Python installation or /usr/local/bin. If in Windows, are you sure it is in your PATH environment variable?

Found it, close


Add changelog and github issue template

What type of PR is this?
Documentation changes: Please include [skip ci] in your commit message as well
/kind documentation
[skip ci]
What does this PR do / why we need it:
We want to do manual changelog management for better release notes.
Which issue(s) this PR fixes:
Action item in retro
PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[X] Documentation

[X] I have read the test guidelines

How to test changes / Special notes to the reviewer:

CI running tests 😞  for docs

CI running tests disappointed for docs

Because we have filters set for .adoc and not for .md, IIUC. @anandrkskd something you can help with by opening a PR on openshift/release repo?

/override ci/prow/unit
/override ci/prow/v4.7-integration-e2e
/override ci/prow/psi-kubernetes-integration-e2e


Add batch export, update dependencies and add support to GoPro MAX files

Changes:

Batch Export
New UI
Support for GoPro MAX
Code Refactor
Updated dependencies

Sweet! Thanks!


Fixed uppercases in XML-parser

Fixed issue with uppercase letters in XML-parser. Interface expected the values as they are written in german.

lgtm


Build xpi at time of release

that way the end user can skip the building step and only have to upload the finished xpi to mozilla.

whoops i forgot that you upload a zip to amo not an xpi

@reesericci I think though that a zip can be built using GitHub Actions, sounds like a good option.

@reesericci I think the zip I was providing with each release is not important anymore since it can be directly installed by cloning the repo.
I'll update the installation instructions and move to download latest release source zip rather than attaching a build during release.

@reesericci I think the zip I was providing with each release is not important anymore since it can be directly installed by cloning the repo.
I'll update the installation instructions and move to download latest release source zip rather than attaching a build during release.

just make sure it has the manifest.json so we don't have issue #34 come up again.

@reesericci As in the latest release, I have not added any custom zip. It's just the one provided by GitHub which has the source code as of the release date. That one surely contains the mainfest.json file because the source contains it, so should be cool!


Clean addresses

Clean addresses, removing unnecessary trailing whitespace.
Remove any whitespace prior to a newline. This catches trailing
spaces, as well as carriage returns.
The carriage returns cause issues when flattening countries to use one
outcome per path, as when converted to an erb template, the carriage
returns do not show up in the output, causing a discrepancy between
the new and old test artefacts.

Squashing the two commits would be ideal IMHO

I've squashed, and merged :)


Stops outputting sound when sound device is switched

When the sound output device is switched (such as plugging in headphones), sometimes eqMac2 will not output sound on the new device. Manually selecting (or re-selecting) the new device in System Preferences forces eqMac2 to re-evaluate the situation and it seems to fix things temporarily.
iMac 17,1 running High Sierra (10.13.1).

Have observed the same thing on 10.10.5 after plugging in my speakers back in to the headphone jack (MBP 11,5). Cannot hear any sound on any device when this occurs. Although I fixed it by changing outputs in the eqMac2 window to something else (DisplayPort), then changing back to built-in output.

Hey everyone. Excuse me for the lack of updates. I have been really busy on the new version of eqMac. I am happy to announce an open beta release.
You can download it here: https://eqmac.app
There are a lot of known bugs that I could not fix just yet:

Driver not installing from the first try. Sometimes need to restart eqMac a couple times.
Complex Device switching logic is not perfect (for example when switching off the device that's currently playing, plugging in new device etc.)
UI takes long to load (a lot can be done to improve that)
UI dimensions are not synching properly
There's a noticeable audio delay in the playback, so not suitable for video yet (I know where the problem lies just haven't had the time to fix yet)
Sometimes the playback doesn't start, in that case please try to switch the output to the device you want and eqMac should try and build the audio pipeline again.

So as you can see there's still some work to be done but at least it's a completely modern tech stack that I am fully willing to work on: 100% Swift, User Space driver and a Web UI, instead of C++, Kernel Drivers. As I said, should work on Catalina and should not have any pesky ear drum blasting volume bugs (at least I didn't come across any).
I'm open to feedback and issue reports here: https://github.com/bitgapp/eqmac
That repo will become the main repo for eqMac and this repo will be archived eventually.
And yes it's called just eqMac, not eqMac3 :) And it will stay that way forever.

Hey everyone 👋
I will be open sourcing the new version of eqMac, therefore, I am closing this issue and repository as I will be transfering nodeful/eqMac2 repo to bitgapp/eqMac
Please bear with me for the next couple of days as I setup that new repo. The eqMac2 code will be living in the old-eqMac2 branch if anyone is interested in referencing it for whatever reason.
Please try out the new version and report bugs to the new repo.
Thanks for using eqMac.

Moving the discussion of this issue to #227


[BUG] string length not enforced in client or server

Hi,
First off, trying out eRPC and impressed by what I see so far. Great work.
I have a command with a maximum of string length associated with the parameter. The max length (@max_length) is not properly enforced in either client nor server:

On client side the generated code simply writes the string with writeString(strlen(), (const char*)str, (const char*)), no validation is done
On server side it actually looks worse:

codec->readString() is used, populating size and pointer (uint32_t str_len,  char * str_local)
Allocation is done with the @max_length+1
If the read string is not NULL and allocation was successful memcpy() is used with the length of the read string (possible buffer overflow if string sent exceeds @max_length)

The issues I see:

Client side can and should avoid sending content that violets the @max_length
No validation of size, possible buffer overflow(?)

Thank you!

HI @taldinj, you may be correct that there is missing validation. Thank you for your input. Once i will have time i will improve this behavior.

PR #328 is for solving this issue

As @taldinj mentioned, we have here security issue od buffer overflow. so I will be happy to push this change to 1.10.0

@amgross Thank you for your effort in our project ;)


Intern unscoped global field name strings

Memory profile analysis shows ~120k duplicates of the string "$n" amounting to ~6MB waste (on the test suite).
Further digging revealed that this happens through the "$n" call-helper (null check) via:

SJSGen#genCallHelper
VarGen#globalVar
VarGen#globalVarIdent
VarGen#genericIdent

A more invasive (but efficient) alternative would be to move the prefix to the call sites. As a result, no runtime interning would have to be performed as the strings would end up being part ot the static constant pool.
Discovered while working on #4906.

@sjrd WYDT? I've been toying with the idea of having all the fields in VarGen (maybe even in an AnyVal). This would give us some more compile time safety and a point of documentation (what the fields actually contain).

Common wisdom goes against the use of the global intern mechanism. Not sure whether this is justified. But it does feel wasteful to construct at run-time "$" + field, only to intern it right away, when we in fact have a known finite set of such fields.

I've been toying with the idea of having all the fields in VarGen (maybe even in an AnyVal). This would give us some more compile time safety and a point of documentation (what the fields actually contain).

This seems like a much better alternative, indeed.

Closing in favor of #4911.


Implement ungridded dim mirroring and complete ungridded dim parsing

This PR implements mirroring of ungridded dims in MAPL3. In addition it also completes the parsing of the ungridded dim from the yaml file since that had not been completed; only the extent had actually been read from the file,  the name, the units, if the user specified coordinates rather than the extent, none of that was being parsed.
I did change the constructor for the ungriddedDim class so it would be easier to use when parsing the yaml file.
Types of change(s)

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Trivial change (affects only documentation or cleanup)

Checklist

[ ] Tested this change with a run of GEOSgcm
[ ] Ran the Unit Tests (make tests)

Description
Related Issue

It was a different variable.   Since it was not part of your change – it did not properly identify.   But … as it turns out it was because GitHub was showing the 1st half of the procedure above and the latter half of the procedure below and the 2 had nearly identical names …
From: Ben Auer @.>
Date: Tuesday, July 16, 2024 at 9:27 AM
To: GEOS-ESM/MAPL @.>
Cc: Clune, Thomas L. (GSFC-6101) @.>, Review requested @.>
Subject: [EXTERNAL] [BULK] Re: [GEOS-ESM/MAPL] Implement ungridded dim mirroring and complete ungridded dim parsing (PR #2907)
CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.
@bena-nasa commented on this pull request.

In generic3g/OutputInfo.F90https://github.com/GEOS-ESM/MAPL/pull/2907#discussion_r1679397601:

@@ -223,7 +223,7 @@ function make_ungridded_dim(info, n, rc)

   type(UngriddedDim) :: make_ungridded_dim

   integer, intent(in) :: n

   type(ESMF_Info), intent(in) :: info

 integer, optional, intent(out) :: rc

 integer, optional, intent(out) :: rc

 integer :: status

 character(len=:), allocatable :: key

I didn't make this particular file, from previous PR. But that rc is used, there's lots of _RC in the procedure.
—
Reply to this email directly, view it on GitHubhttps://github.com/GEOS-ESM/MAPL/pull/2907#discussion_r1679397601, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABPP7YFPW2ALIGR7WY6UIELZMUNVXAVCNFSM6AAAAABK5GTYIKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCOBQGI2DENRTG4.
You are receiving this because your review was requested.Message ID: @.***>


Updating dnabot from version 3.1.0 to 4.1.0

Hello! This is an automated update of the following tool: dnabot. I created this PR because I think the tool's main dependency is out of date, i.e. there is a newer version available through conda.
I have updated dnabot from version 3.1.0 to 4.1.0.
Project home page: https://github.com/brsynth/DNA-BOT/releases
For any comments, queries or criticism about the bot, not related to the tool being updated in this PR, please create an issue here.

local test passing


Adds repr to Message

Context
What is the purpose of this PR? Is it to

[x] add a new feature
[ ] fix a bug
[ ] update tests and/or documentation
[ ] other (please add here)

Fixes https://github.com/pytorch/torchtune/issues/1639
Updating __repr__ configures the output for __str__ as well. This PR diffs a little from the initial proposal by printing out the list of items. The content type is not printed out, since it can be inferred from the __repr__ itself. i.e. strings are printed directly and Image uses the __repr__ from PIL.
Changelog

Adds __repr__ to Message.

Test plan

[x] run pre-commit hooks and linters (make sure you've first installed via pre-commit install)
[x] add unit tests for any new functionality
[ ] update docstrings for any new or updated methods or classes
[x] run unit tests via pytest tests
[ ] run recipe tests via pytest tests -m integration_test
[ ] manually run any new or modified recipes with sufficient proof of correctness
[ ] include relevant commands and any other artifacts in this summary (pastes of loss curves, eval results, etc.)

UX

[ ] I did not change any public API
[ ] I have added an example to docs or docstrings

awesome awesome, thanks for adding this!


Add poetry scripts for common commands

This PR adds poetry scripts with some common commands that I always forget and need to look up 😄
Please let me know if you're open to this idea, otherwise I can close the PR.
It currently adds the following scripts (we can always add more or change as we go):

poetry run docs - build docs
poetry run test - runs tests

Can accept specific directories/file and will run only them, for example: poetry run test tests/caching

poetry run lint - runs pylint from pre-commit

Can accept specific directories/files and will run only them, for example: poetry run lint starlite/cache/**

poetry run fmt - runs black, isort, prettier, blacken-docs from pre-commit

Can accept specific directories/files and will run only them, for example: poetry run fmt starlite/cache/**

poetry run type_check - runs mypy from pre-commit

Can accept specific directories/files and will run only them, for example: poetry run type_check starlite/cache/**

poetry run all_checks - runs all pre-commit checks

Can accept specific directories/files and will run only them, for example: poetry run all_checks starlite/cache/**

PR Checklist

[x] Have you followed the guidelines in CONTRIBUTING.md?
[x] Have you got 100% test coverage on new code?
[x] Have you updated the prose documentation?
[x] Have you updated the reference documentation?

tox can also be used to abstract commands away like this too, e.g. see the refactor branch of example app that I'm working on: tox -e lint, tox -e test etc etc. Also, we already have tox.ini in the project.

I'll close this PR and try to use tox for this. If it doesn't work we can use a Makefile


MergeInto is failing when trying to upsert 70GB data onto a 25TB Delta store

MergeInto is failing when trying to upsert 70GB data (size on S3) onto a 25TB Delta store (size on S3)
The failure is happening in the findTouchedFiles() spark stage and we see that our Driver is running out of memory before this stage can finish. We tried with a r5.24xlarge node (768GB RAM) for our driver (spark deploy mode = client). I think the reason is that the current Delta store has 25TB spread across 43,500 files and this Upsert is probably touching most of the files. So in this Spark stage containing 50k tasks, each of the tasks return about 7MB of data for the touchedFilesAccum and the driver is unable to hold this much data and is running OOM. On doing a Heap dump analysis of Driver, I see the following for class histogram:

We are using the following setup for our data pipeline:
EMR 5.33.0
Spark 2.4.7
Delta core 0.6.1
We have 8 r5.24xlarge machines as core nodes and 1 r5.24xlarge machine for the master node
Current Delta store size: 25 TB (after bootstrapping with initial data via a single commit)
Upsert data size: 70 GB
No partition used since our data has no logical partitioning and is a simple 5 column table with billions of rows
I even tried to do the same Upsert above with the following setup, but notice the same behavior:
EMR 6.4.0
Spark 3.1.2
Delta core 1.0.0
We have 8 r5.24xlarge machines as core nodes and 1 r5.24xlarge machine for the master node
Please suggest how we can proceed. One thing I am trying is to reduce the number of partitions (spark.sql.shuffle.partitions) so that the overall data returned to the Driver will be smaller, I haven't gotten this to succeed yet and I think this is not a scalable solution since reducing partitions means I need to use bigger executors. Please let me know if any further info is required.

This is definitely a hard problem to solve. Here are some initial thoughts.

Each task is returning 7 MB in the touchedFilesAccum because each task is probably finding matches across a large fraction of the 43500 files, and therefore returning the names of many files. This accum is a basically a distributed Set accumulation to find the set of files that have matches. Finding this list of files is computed as follows
Inner join the table and upsert data by the merge condition. This shuffle data of all the files across all the reduce tasks.
Then in each reduce task, we collected the unique set of file names in touchedFilesAccum that have matching data.
Every task's set of file names in touchedFilesAccum is returned to the drive for the final set union.

My guess it that there is a very large overlap between the the 7MB worth of files names returned by tasks. After merging them across tasks, its not a lot of data. But internal data structures in Spark keeps track of the touchedFilesAccum data of every task, so 50k tasks with 7MB of accum data each = at least 350 GB in memory.
You are right that the most likely way to avoid this blowup is to reduce the number of tasks, say by 10x. Each task will still probably still return around 7-10 MB (because of large overlap of unique files seen by each task). You have a cluster of 8 r5.24xlarge machines = 768 cores. So properly use the cluster's parallelism, try running 5k tasks. Expected memory usage = 5k * 10MB = 50GB. If that does not work because too much memory spilling in the tasks, then up it 10k.
If you are still having trouble, here are the next steps. Can you give us more information on stage's metrics? Total sizes and distribution across tasks for task times, accum size, Shuffle size written and read, etc. Screenshots of the SQL plan
showing all the metrics can help a lot.

I have gotten this to succeed for now with lesser number of shuffle partitions (so I will close this ticket). I was wondering if there is a reason why Accumulators were used to get this set of Touched files and not a pure Spark action.

I am glad you got it to work!
Accumulators were used originally because our initial testing had showed that accumulator to do the set was faster in finding the files than using the Spark job. But in hindsight, we clearly did not test it at the scale and settings that you are running it. Setting the shuffle partitions in using Apache Spark  has always been a dark art, and many things can tip over like this when not configured correctly. Now that we are hitting this, we definitely want to improve this. Would you be interested in brainstorming on how to convert the accum into a spark action? and maybe contribute it?

Sure thing! I can do that. I am off for my holidays now though, shall we connect after Jan 1st when you're available? (I am based out of India btw)

Sure! happy to chat about this in the new year.
On Fri, Dec 24, 2021 at 10:11 AM ganesh-ravi-srib @.***>
wrote:

Sure thing! I can do that. I am off for my holidays now though, shall we
connect after Jan 1st when you're available? (I am based out of India btw)
—
Reply to this email directly, view it on GitHub
https://github.com/delta-io/delta/issues/874#issuecomment-1000870241,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFB5LC4CD5PDWXTKLUJCBLUSSER7ANCNFSM5KNWLG4A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Hi @tdas , I am back from my vacation. Please let me know when you want to chat about this

Alright! I am back from vacation as well. The challenge in this problem is that we want to run a single Spark job that computes 2 things

The list of touched files
The frequency/count of how many times a unique target line matches, so that we can check for duplicate matches and throw error.

The way the current code works is by use Dataframe operations to scan target files + join + groupBy + aggregate to do 2, and rely on accumulators to do 1. I think there may be a way to compute both  1 and 2 in the groupBy + aggregate.
Here is the code - https://github.com/delta-io/delta/blob/master/core/src/main/scala/org/apache/spark/sql/delta/commands/MergeIntoCommand.scala#L335-L366
Let's brainstorm about this together.

Hi @tdas , I have gone through that piece of code and now understand how touchedFilesAccumulator and the >1 target row count check works. Both of the above are currently being calculated from the same inner join without using any caching or persisting, the first using accumulators and the second using groupBy and aggregate.
I wonder how we could do the both in the same inner join without using a persist/cache. Please let me know your idea for this.

Actually maybe we could swap the two. In the sense, we could use Accumulators to find the duplicate target row matches and use Spark distinct to find the touched files. This will help because this way the data returned to the driver will be of much smaller size.
The Accumulator for finding duplicate target row matches can be a hashmap accumulator of type <Long, Boolean> where long represents the monotonically_increasing_id and the Boolean represents has_duplicates.
I am not entirely sure how Accumulators are internally implemented in Spark though. Considering the current touchedFilesAccumulator - will each task send the complete list of touched files to the Driver and the driver will add them to the hashset? Or will there be a hashset associated with each task and after the completion of the task this hashset will be sent to the driver and finally after the stage completes, the driver will merge all hashsets?

Hi @tdas , please let me know your view


'f' hotkey doesn't work in types view

Version and Platform (required):

Binary Ninja Version: 3.1 - dev

Bug Description:
the 'f' hotkey "cycle float size" doesn't work in types view

Fixed in 3.1.3549


Fix tests from last year

Some tests from last year are still relevant but not supported yet. It could be helpful to take a look and fix the ones we want to keep

We will change so much that most of the tests are not needed anymore


Fix issue with scroll full height using incorrect class name

Resolves https://github.com/gregnb/mui-datatables/issues/580#issuecomment-529796114

Coverage remained the same at 76.408% when pulling aa843055e6d2adae0254b94a5090eec6b8bb0ee9 on bugfix/scroll-full-height into 3032a7c22cdbbafbc53409d797a179215a40f6ad on master.


SpriteFont drawing is unexpectedly CPU intensive in Debug builds

While upgrading an old Dx9 project to Dx11 with DirectXT I noticed everything was going smoothly for performance until I started drawing fonts.  CPU usage went up by a factor of 4 and made in impossible to maintain 30 FPS.
The profiler says the main culprit is DirectX::SpriteFont::Impl::FindGlyph function call.  Is this expected behavior or a bug?  I'm not doing anything too crazy, drawing perhaps 10-15 different pieces of text on the screen.

Are you profiling an optimized Release build or a debug build?

It is a Debug build.

Please profile against an optimized build.  It is normal and expected that STL usage in debug builds is much slower due to all the range checks and consistency validation included by the CRT.

Thanks for the quick replies guys.  With full optimizations turned on, font drawing is no longer a problem.  However, while using a debug build (which is most of development), performance is REALLY bad.  Would it be possible to have a version of SpriteFont that caches the glyph look ups?  I suppose I could render to texture as a work around, but this seems like a reasonable feature request.

"Optimizing" debug performance is quite difficult, especially when leveraging the STL. I'll make a note here to look into it as I've gotten some comments from others on this one. The performance seems to be mainly in the use of the C++ lambda which in non-optimized code is a function call instead of getting inlined. Fixing this would likely require major rewriting of the existing code.
Another option would be for you to change the project settings in your copy of the DirectX TK vcxproj to use optimizations for the SpriteFont.cpp module in the Debug configuration.

Or just use the Release configuration of DirectXTK?
Unless you are debugging inside DirectXTK itself, it is reasonable to build your own code for debug but other libraries that you depend on with full optimizations.

@shawnhar That's a great point guys, I hadn't considered just using the optimized version of DXTK.

Sorry to necro-post, but I just got back to this project and have a question about using a optimized build in a debug project.  Could I get some pointers on how to accomplish this?  I've tried just linking against a optimized .lib of DXTK but get a boat load of CRT mismatch errors.  How can I actually use an optimized version of DXTK in an otherwise debug build?


Add WhoAmIRequest Aggregated Virtual REST API

This change adds a new virtual aggregated API that can be used by
any user to echo back who they are currently authenticated as.  This
has general utility to end users and can be used in tests to
validate if authentication was successful.
Signed-off-by: Monis Khan<EMAIL_ADDRESS>Fixes #402
Release note:
The pinniped concierge now supports a WhoAmIRequest API to echo back the current authenticated user.

Codecov Report

Merging #439 (4c87b30) into main (f228f02) will increase coverage by 0.09%.
The diff coverage is 79.62%.

@@            Coverage Diff             @@
##             main     #439      +/-   ##
==========================================
+ Coverage   77.63%   77.72%   +0.09%
==========================================
  Files         109      111       +2
  Lines        5499     5567      +68
==========================================
+ Hits         4269     4327      +58
- Misses       1063     1072       +9
- Partials      167      168       +1

Impacted Files
Coverage Δ

internal/groupsuffix/groupdata.go
0.00% <0.00%> (ø)

internal/concierge/server/server.go
48.96% <76.92%> (+6.62%)
:arrow_up:

internal/registry/whoamirequest/rest.go
95.65% <95.65%> (ø)

cmd/pinniped/cmd/kubeconfig.go
92.34% <100.00%> (ø)

cmd/pinniped/cmd/login_oidc.go
88.23% <100.00%> (ø)

cmd/pinniped/cmd/login_static.go
94.91% <100.00%> (ø)

internal/config/concierge/config.go
82.35% <100.00%> (ø)

internal/config/supervisor/config.go
76.92% <100.00%> (ø)

internal/groupsuffix/groupsuffix.go
100.00% <100.00%> (ø)

internal/registry/credentialrequest/rest.go
100.00% <100.00%> (ø)

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f228f02...4c87b30. Read the comment docs.


PIP install fails on README.md

DEPRECATION: --no-install, --no-download, --build, and --no-clean are deprecated.  See https://github.com/pypa/pip/issues/906.
Downloading/unpacking eveimageserver==0.1.0
Downloading eveimageserver-0.1.0.tar.gz
Running setup.py (path:C:\Users\regner\AppData\Local\Temp\pycharm-packaging0.tmp\eveimageserver\setup.py) egg_info for package eveimageserver
Traceback (most recent call last):
File "", line 17, in
File "C:\Users\regner\AppData\Local\Temp\pycharm-packaging0.tmp\eveimageserver\setup.py", line 21, in
long_description=open('README.md').read(),
IOError: [Errno 2] No such file or directory: 'README.md'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 17, in
File "C:\Users\regner\AppData\Local\Temp\pycharm-packaging0.tmp\eveimageserver\setup.py", line 21, in
long_description=open('README.md').read(),

IOError: [Errno 2] No such file or directory: 'README.md'

Needs a manifest.in


[AB3: Tracing Code]: Clarify what Event Driven Mechanisms is

Current:

Problem:
Students might not understand what it means by “various event-driven mechanisms” and may be confused on the choice of placing the breakpoint where the UI transfer controls to the logic component.
Proposed:
Add a paragraph after the highlighted part:
This is because, unlike linear command-line programs, GUI applications respond to user actions (like clicks and keystrokes) and other events, triggering various event listeners and handlers. This non-linear flow of control can make it difficult to trace and debug the sequence of operations, as the program reacts dynamically to different events, creating an obscure execution path.

I think that will be a good solution as it does not make the whole tutorial lengthy and students who are unsure can refer to the additional information.

I think that will be a good solution as it does not make the whole tutorial lengthy and students who are unsure can refer to the additional information.

👍
Closing this issue.


Populate Tour section

Choose images!
First image (The synapse image? Image of the molecules that our drugs target? Magic mushrooms!? MRI/EEG equipment?) - tell about our work, esp. aims and methods
Second image (some sort of network) - Collaborations (and maybe stress that we're always looking for collaborators!)
Third image (Photo of us? Or something more global, meaning we're going from a neuron/synapse in the first slide, then onto a network and finally, to something even broader!?..) - Something catchy about our work (making a difference + help us to make a difference) (add e.g. depression/anxiety) and then a button (e.g. "Reach Out" or "Get in Touch") that leads to the contact page

The text is not displayed properly on desktop (only 1) and mobile devices (all 3)...
Issue is with .hero-lead {
max-width: 768px;
font-size: 1.35rem;
}
???


Update code-coverage.ts for kts file

add file extension check on the build.gradle file for kotlin support
Task name: Gradle
Description: I added a check for build.gradle.kts
Documentation changes required: (Y/N) N
Added unit tests: (Y/N) N
Attached related issue: (Y/N) <Gradle Kotlin DSL results in warnings in log #9340>
Checklist:

[ ] Task version was bumped - please check instruction how to do it
[ ] Checked that applied changes work as expected

@csecio When you added check for kts, generated report on code coverage tab?

Any plan to merge this fix?


fix EncodeBC1 floating point exception

Floating point exception sometimes happens when calling HDRColorALerp() in EncodeBC1() which lerp the uninitialized (and unused) alpha value.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: poulpi sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Can you provide a bit more information about the underlying conditions of why you felt this was a problem?
FP-specials are not enabled by default, and turning them on in code outside your control is problematic to begin with.

A more direct fix might also be to just initialize the HDRColorA struct so that .a is 0...

Please see if the changes in this commit resolve your issue... If not, feel free to reopen.

We are using FP exceptions to help us tracks bugs, and we use DirectXTex at runtime as with need texture compression on the fly.
The commit you proposed do not fix it. The problematic value is created l494 before the OptimizeRGB(), and crash in HDRColorALerp() l578.
Initializing the rgba values in the HDRColorA class constructor may be the best fix ?

Can you give it one more try with  this commit? I found another case I missed.
BTW, can you provide the snippet of code you are using to enable FPs exceptions for your debug builds? I can see about adding that to my DirectXTex test suite.

The commit works, thanks you.
Here the code we use to set or unset floating points exceptions:
#define FLOAT_EXCEPTION_FLAGS (_EM_ZERODIVIDE | _EM_OVERFLOW | _EM_INVALID | _EM_DENORMAL)
unsigned int x86_cw;

set:
#if defined (_M_IX86)
	_clearfp();
	__control87_2( _EM_UNDERFLOW|_EM_INEXACT|_EM_DENORMAL|_EM_ZERODIVIDE, _MCW_EM, &x86_cw, NULL );
#elif defined(_M_X64)
	_controlfp_s(&x86_cw, 0, FLOAT_EXCEPTION_FLAGS);
#endif

unset:
#if defined (_M_IX86)
	_clearfp();
	__control87_2( _MCW_EM, _MCW_EM, &x86_cw, NULL );
#elif defined(_M_X64)
	_controlfp_s(&x86_cw, FLOAT_EXCEPTION_FLAGS, FLOAT_EXCEPTION_FLAGS);
#endif

Thanks.
Note that I'm not sure your _M_IX86 case does what you think it does. Unless you use /arch:IA32, the Visual C++ 32-bit compiler is going to use SSE/SSE2 and not x87 for floating-point computations, and it appears from the code above you are not setting the SSE/SSE control word in x86 processes. You should probably just use _controlfp_s for all platforms.

Updated test suite in this commit


Typo "Azure Cosmos explorer"→"Azure Cosmos DB Explorer"

https://docs.microsoft.com/en-us/azure/cosmos-db/cassandra-support

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Bootstrap Waypoint Images

Challenge http://freecodecamp.com/challenges/waypoint-use-the-bootstrap-grid-to-put-elements-side-by-side has an issue. Please describe how to reproduce it, and include links to screenshots if possible.
I am not able to view any images which are added with the instructions for illustration of the concept.
On clicking the image it goes to a evernote page which also shows an error.
Not sure though if it is happening only for me.
Please verify it.
Thanks,
Nimit

Evernote appears to be down right now. It's weird that Evernote is being used as an image host at all, though...

Yea I cannot see any images on there now. I thought earlier it was happening for only a single lesson but its on all of them.


added consultation history with date time and status and changed design of page

@dhairyagothi  kindly assign me this

assigned @Krishna100604


⚠ Bofrost.de has degraded performance

In 689ba20, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1593 ms

Resolved: Bofrost.de performance has improved in 7616c98.


Added RUF010 rule - use f-string conversions

This rule will check if an f-string contains anything that can be replace with a conversion (e.g. str(foo) to foo!s)
Sorry if there are any glaring Rust mistakes, this is my first time with the language

Will merge this after cutting the release.


Update Node.js to active LTS version

What
Update the project to use the latest active Long Term Support (LTS) version of Node.js for development, testing, and deployment.
Why
This project currently uses Node.js version 14, which has been in maintenance since last October (2021-10-19) and won't receive new features.
Who needs to work on this
Developer
Who needs to review this
Developer, Tech lead
Done when

[ ] Update .nvmrc
[ ] Update package-lock.json to use lockfile v2

We've encountered some issues with this change on Mac M1 devices. They appear to related to node-sass, which we use in our tests and with gulp-sass.
gulp-sass will not support M1 chips with the node-sass compiler.
We've managed to get this working on M1 machines by deleting node_modules, then running:
nvm use
npm uninstall node-sass -g && npm cache clean -force && npm install node-sass
nvm use
npm install

Current theories are that the issue might be related to installing with Node v14, then installing this change, and something going awry in the switch. We're going to test installing straight from this branch and see if a clean install works.
We're then going to decide whether:

We stick to Node 14 for a while
We push the Node 16 update and document these steps for devs to troubleshoot any issues
We look at switching to dart-sass

Did you try npm rebuild after swapping out the Node.js version out of interest?

Did you try npm rebuild after swapping out the Node.js version out of interest?

We did, but no luck.

As a little after-the-fact question: Given there's some faff involved in updating from v12 to v16, would it be useful to add that as something in the changelog/release notes?

@querkmachine Are there general issues with updating from v12 to v16?

@domoscargin I meant v14, sorry.

Ah, right!
I'd lean towards not including something in the release notes, since this shouldn't affect anybody but devs/contributors.
But maybe something in the changelog with a "only for contributors" type warning?