The Primer/SystemArgumentInsteadOf linter doesn't report on marketing utility classes

Summary
We have utilities for marketing typography that come in the form of:

Headings: .h0-mktg to .h6-mktg
Font styling:.f0-mktg to .f6-mktg

While reviewing a GitHub internal PR, I noticed that our application enabled this particular cop, but didn't report on one of the components that included an instance from this utility family:
            <%= render Primer::Beta::Button.new(classes: "f4-mktg") %>

I created a new test in test/lib/rubocop/system_arguments_instead_of_class_test.rb and observe that the assertions fail:
  def test_marketing_class
    investigate(cop, <<-RUBY)
      Primer::BaseComponent.new(classes: "f4-mktg")
    RUBY

    assert_equal 1, cop.offenses.count
    assert_equal "Primer/SystemArgumentInsteadOfClass: Avoid using CSS classes when you can use System Arguments: https://primer.style/view-components/system-arguments.\n", cop.offenses.first.message
  end

## Terminal Output
# Running:

.F

Failure:
RubocopSystemArgumentInsteadOfClassTest#test_marketing_class [/workspaces/view_components/test/lib/rubocop/system_arguments_instead_of_class_test.rb:66]:
Expected: 1
  Actual: 0

Questions:

Should it report on marketing utility classes?
If so, would the team be open to me cutting a PR for this? 😸

Hey @francisfuzz, thanks for bringing this to our attention! A couple of questions:

How often are people using these marketing classes?
Where does the styling live for these classes? Is the entire set of them known?

Should it report on marketing utility classes?

I'm going to say yes. I thought about potentially writing a separate linter for this, but since we have marketing stuff in PVC already, adding them to the existing set of utility classes seems ok.

would the team be open to me cutting a PR for this?

Absolutely! The place to put them is custom_utilities.yml.

How often are people using these marketing classes?

For our internal application within the github org (can share on request 😉 ), my back-of-the-napkin grep script counted about ~195 instances of these marketing classes. I'm not sure how many people are using these marketing classes beyond our app!

Where does the styling live for these classes? Is the entire set of them known?

From what I can tell, it's in our app's components and views folders (in the context of an HTML+ERB file). The entire set of them can be known with some updates to my script. 😸

I'm going to say yes. I thought about potentially writing a separate linter for this, but since we have marketing stuff in PVC already, adding them to the existing set of utility classes seems ok.

would the team be open to me cutting a PR for this?

Absolutely! The place to put them is custom_utilities.yml.

@camertron - Wonderful, thank you! I've assigned myself to this issue and will set aside some cycles to work on this. 👍

Where does the styling live for these classes? Is the entire set of them known?

@camertron - I re-read your questions again and I think that I misinterpreted what you were asking, sorry!
I'm wondering if you meant where those respective headings and font stylings are defined? In that case, I believe this is it: https://github.com/primer/css/blob/main/src/marketing/type/typography.scss

@camertron - Re-thinking this after a chat with my colleague @smashwilson 💭
First, I'm wondering if there's a guideline of sorts for when to use these marketing utilities? Were these created solely for the GitHub Marketing team for a specific purpose, or can any GitHub team use these utilities to incorporate in their github-owned applications?
Also, assuming that it is okay for other GitHub teams to use these marketing utilities, is there a reason that they don't have system argument representation yet? Perhaps we should implement those in the system arguments, before implementing the changes in the linter -- does that order make sense?
Last, is the goal for the system arguments to represent all of primer/css, or only a subset?

my back-of-the-napkin grep script counted about ~195 instances of these marketing classes

Ok, it sounds like they're definitely worth supporting as system arguments then 😄

I'm wondering if you meant where those respective headings and font stylings are defined? In that case, I believe this is it:

That does mostly answer my question. Are you looking to support only the typography classes though? I see a bunch of other stuff in that utilities directory.

First, I'm wondering if there's a guideline of sorts for when to use these marketing utilities? Were these created solely for the GitHub Marketing team for a specific purpose, or can any GitHub team use these utilities to incorporate in their github-owned applications?

That I'm not sure of 😓 I think in general, GitHub-owned applications should use Primer components first and non-marketing utility classes second. I believe those marketing utility classes were created specifically for landing pages, etc, so I would try to avoid using them outside of a marketing context. We are also always up for discussing new utility classes or updates to components to achieve styling goals.

Also, assuming that it is okay for other GitHub teams to use these marketing utilities, is there a reason that they don't have system argument representation yet? Perhaps we should implement those in the system arguments, before implementing the changes in the linter -- does that order make sense?

Yes, that order makes sense 👍 although as I said above, we should only use them in marketing contexts.

Last, is the goal for the system arguments to represent all of primer/css, or only a subset?

For now, system arguments only support a subset of utility classes. We add missing ones whenever there's a need.

That does mostly answer my question. Are you looking to support only the typography classes though? I see a bunch of other stuff in that utilities directory.

@camertron - Personally speaking, I'd like to keep the focus narrow and only support those typography classes. To your point, maybe the other stuff could wait until someone surfaces the need for it? 😉

That I'm not sure of 😓 I think in general, GitHub-owned applications should use Primer components first and non-marketing utility classes second. I believe those marketing utility classes were created specifically for landing pages, etc, so I would try to avoid using them outside of a marketing context. We are also always up for discussing new utility classes or updates to components to achieve styling goals.

Roger that: I can adhere to that approach for sure! 👍

Based on our discussion thus far, I think the next best steps are to:

Open a separate issue for tracking work for implementing the marketing typography classes in this repo
Keep this issue open until that gets implemented

Is that appropriate, or should I do something different to move this forward? 💭

Having another thought about this after chatting with my colleague @jalafel!
One of the topics they shared was considering the ongoing work that's happening with Primer React in conjunction Primer Brand. Might there be any planned or upcoming work that involves Primer Brand also being implemented for Primer View Components, too? 💭
Maybe taking that context into consideration may steer the direction of this work, where it may be wiser to hold off and see where that goes? 😅

Based on our discussion thus far, I think the next best steps are

Yes, those are good next steps :) If you're up to it, we would also welcome a PR since it's unlikely we'll be able to pick this up until next year.

Might there be any planned or upcoming work that involves Primer Brand also being implemented for Primer View Components, too?

I haven't heard anything, but maybe? @lesliecdubs, can you weigh in here?

👋 Hi! Thanks for tagging me in here @camertron.

One of the topics they shared was considering the ongoing work that's happening with Primer React in conjunction Primer Brand. Might there be any planned or upcoming work that involves Primer Brand also being implemented for Primer View Components, too? 💭

My understanding is that the Primer Brand library (implemented in React) is GitHub's official design system for marketing pages and experiences, and that forward-looking investments in marketing infrastructure will be happening in React, as per the future of GitHub's UI platform. cc'ing @raytalks in case she has anything to add here
Given this direction I would generally be in favor of removing and/or discouraging use of marketing utilities within Primer CSS and PVC, as these styles should now be handled by the Primer Brand library.
That said, I'm not super familiar with what the linter Primer/SystemArgumentInsteadOf does, so if there's a compelling reason to consider adding this for marketing utilities in the meantime - feel free to share more and we can keep discussing!

My understanding is that ... forward-looking investments in marketing infrastructure will be happening in React

Right, that makes sense 👍 Given 1) the relatively high number of existing usages of the marketing classes, and 2) how uncomplicated it is to add them to custom_utilities.yml, I still say let's do it.

I'm not super familiar with what the linter Primer/SystemArgumentInsteadOf does

It ensures any CSS classes passed to a component, eg. render(FooComponent.new(classes: "f4-mtkg")) are passed as system arguments instead, eg: render(FooComponent.new(f_mktg: 4)). The linter already enforces this style for many of Primer CSS's utility classes.

Given 1) the relatively high number of existing usages of the marketing classes, and 2) how uncomplicated it is to add them to custom_utilities.yml, I still say let's do it.

@camertron should we consider adding these classes to Primer CSS instead of PVC, since that's where we're trying to consolidate most Rails styles these days? Or is there not an easy way to add the requested linter to PCSS? Also wanted to make sure you were of https://github.com/primer/view_components/pull/2477, as it seemed like the outcome from that work might have an effect on the direction we might want to take here (cc @jonrohan).

We talked about this at our latest PVC sync meeting and I discovered we're https://github.com/primer/view_components/pull/2477 enforcement of utility classes as system arguments. You can still use them, but we will stop enforcing them. Let me know if that changes anything on your end, @francisfuzz.

@lesliecdubs & @camertron - Happy New Year! 🎉 Thank you both for providing such rich context to my questions. Admittedly, I'm still letting these updates settle in my mind, and I've set aside time tomorrow to make a decision for where-and-how to move this forward. 👍

My apologies: this totally slipped my radar and that's on me. I am sorry! 😞
After evaluating against the following context, I've decided to close this issue and leave the behavior alone, so folks can continue to optionally use marketing classes where it makes the most sense for their use case.
The first reason is related to what Leslie had shared: I don't think Marketing-related content should be introduced in Primer View Components, given the team's move to consolidate that area into Primer Brand. I think there are likely to be folks who disagree with my view, and they're more than welcome to take up that work if they so choose! 🙇
The second reason is related to https://github.com/primer/view_components/pull/2477 along with the example Cameron shared about using the marketing classes. It got me thinking: "how are these actually defined under the hood?" 💭
I dove in primer/css and was curious to see how the font size stylings were defined for those types.
Here's the "plain" flavor:
/* Set the font size to #{$h4-size} */
.f4 {
  font-size: var(--h4-size, $h4-size) !important;

  @include breakpoint(md) {
    font-size: var(--h4-size, $h4-size) !important;
  }
}

And here's the marketing flavor:
.f0-mktg,
.f1-mktg,
.f2-mktg,
.f3-mktg,
.f4-mktg,
.f5-mktg,
.f6-mktg {
  font-family: $font-mktg;
  font-feature-settings: $mktg-font-feature-settings;
  font-weight: $font-weight-normal;
}

They're not exactly one-for-one, and understandably so based on previous design decisions!
My humble take, based on these findings, is to go forward with leaving this alone, such that we give folks flexibility to choose whatever makes the most sense for their use case, which feels like a better developer experience than introducing the potential friction that comes with a new linter change.


Garage Door Openers/Kwikset 910 locks

Hello,
I know there was an open zwavejs issue about barrier operators not returning the proper states to Home Assistant. After updating to 1.2.1 with zwavejs 6.4, what do I need to do in order to update the device for HA? I thought re-interview node might do it, but that doesn't seem to update the MQTT payload.
Separately, way back in the day there was an openzwave fix for the Kwikset 910s that used the alarm level to indicate locked or unlocked. I saw the 910 mentioned in zwavejs release docs. Do I need to do something similar to update the lock information now that I've upgraded to 1.2.1?
Thank you in advance.

OK, I see that the barrier operator fix isn't implemented yet.

@drjjr2 I have a Kwikset 910 and am having some issues. Can I pick your brain on a few things?

Ok, I was able to come up with a similar looking script in my homebridge MQTT plugin (Just last night actually! It was a glorious moment,) but I only noticed the 21, 22, 24, and 25 states. What are 18 and 19? Locked or unlocked from the keypad?

Heads up, I just upgraded to v3.0.2 and the lock status is now properly reported. But the alarm types are not begging updated so my binary sensors quit working. Not a big deal because now I can just use the lock state like the old days.

But the alarm types are not being updated so my binary sensors quit working

Could you better explain this?

But the alarm types are not being updated so my binary sensors quit working

Could you better explain this?

Certainly.
Prior to switching to zwavejs2mqtt, the lock state for my Kwikset 910s was properly reported in Home Assistant when I manually locked or unlocked the device. After the switch that state did not get updated but I was able to create the sensors mentioned earlier in the thread to use the alarm type to determine the state of the lock.
After updating to 3.0.2, I no longer see the alarm type change. However the lock state is now correct. The only message I see in the log is
2021-03-28 12:34:35.830 INFO ZWAVE: Node 53: value updated: 98-0-currentMode 255 => 0
2021-03-28 12:34:35.830 INFO ZWAVE: Node 53 CC Notification {
type: 6,
event: 2,
label: 'Access Control',
eventLabel: 'Manual unlock operation',
parameters: undefined
}

@AlCalzone ?

That's the result of https://github.com/zwave-js/node-zwave-js/pull/2045, which fixes https://github.com/zwave-js/node-zwave-js/issues/1713
Lock/Unlock operations are not state, they are events. They have always been handled as events for devices that support Notification CC V2+. Now we do the same for V1 devices where a mapping from alarmType/Level to notification events is defined.

However the lock state is now correct

Yeah, use that or listen for the events if you need to distinguish between keypad/RF/manual lock operations.

Should the alarmtype and alarmlevel entities be removed? MQTT still thinks they are a thing for my Kwikset 910 locks, but if they do nothing do we need them?

MQTT still thinks they are a thing for my Kwikset 910 locks

Maybe something retained from previous versions?

If I delete the store, delete the messages using MQTT Explorer, re-interview and then rediscover the node, they show up again.
{
  "type": "sensor",
  "object_id": "notification_alarmtype",
  "discovery_payload": {
    "value_template": "{{ {}[value_json.value] | default(value_json.value) }}",
    "icon": "mdi:alarm-light",
    "state_topic": "homeassistant/Front_Door_Lock/113/0/alarmType",
    "json_attributes_topic": "homeassistant/Front_Door_Lock/113/0/alarmType",
    "device": {
      "identifiers": [
        "zwavejs2mqtt_0xefa976a8_node53"
      ],
      "manufacturer": "Kwikset",
      "model": "Smart Code with Home Connect Technology (910)",
      "name": "Front_Door_Lock",
      "sw_version": "3.37"
    },
    "name": "Front_Door_Lock_notification_alarmtype",
    "unique_id": "zwavejs2mqtt_0xefa976a8_53-113-0-alarmType"
  },
  "discoveryTopic": "sensor/Front_Door_Lock/notification_alarmtype/config",
  "values": [
    "113-0-alarmType"
  ],
  "persistent": false,
  "ignoreDiscovery": false,
  "id": "sensor_notification_alarmtype"
}

We cannot be sure that there isn't a value that we can't translate. For these values, we need the alarm type and level to exist.


feat(headless,atomic): fix issues for commerce did you mean + handle query trigger for commerce

Add a new search-actions-thunk-processor file for commerce search actions (which mimics what we have for search use case)
It is in charge of processing the search response specifically (not product listing) since there can be a mismatch between what Headless think was the query executed (what the user submitted in the search box) and what was actually executed in the backend (query correction/query trigger)
When there is a mismatch, we need to reconcile the search boxes (query set slice) and the query itself (query slice) by dispatching actions. Otherwise it causes bug where the end user will see that did you mean component says "query foo was executed" but the search box will display bar.
Since it can be a complicated piece of code to push into the thunk function itself, the processor file is there to isolate and make it easier to test the behaviour.
https://coveord.atlassian.net/browse/KIT-3224

I think it could be theoretically possible to get in an infinite loop with the combination of automatic query correction + query trigger.
E.g.,

"foo" query is executed
did you mean corrects query to "bar" and automatically executes
configured query trigger replaces "bar" with "foo" and frontend executes execute, looping back to 1.

This is a total edge case, but maybe we should consider protecting ourselves against this if this is not already done.


Test support for TW light

Can anyone confirm this is working? I even fail pairing Osram Lightify bulbs with the current (1.17.1) version of the Wemo iOS app.


[PE-D][Tester C] Help window does not display the right URL link.

Bug Description
For the help feature, the displayed and copied URL does not match Moddie's user guide page. Severe bug as if the user has just the application, there is no way for them to access the user guide.
Steps to Reproduce

Type help.
Click copy URL
Paste in browser

Expected
Link to https://ay2324s2-cs2103-f15-4.github.io/tp/UserGuide.html
Actual
Links to https://se-education.org/addressbook-level3/UserGuide.html
Screenshot

Suggestions for Improvement
Update the URL in the help window. I personally think it is quite a big bug (which is easy to rectify, so its a low hanging fruit), since it completely breaks the help feature and renders it useless.

Labels: severity.High type.FunctionalityBug
original: gerteck/ped#5

Duplicate issue


Constant folding sincos and matrix-from-floats

Constant folding of the matrix constructor from 16 floats
Constant folding of sincos.
Noticed that the implementation of the vector version of sincos did not use fast_sincos underneath, as the other versions did, so fixed that.
Along the way, it was helpful to change RuntimeOptimizer::turn_into_new_op() so that it takes all three arguments, instead of operating by keeping the result the same as before. It's only used in a handful of places, but had to touch those to add the extra argument.

LGTM


Amazon Games authentication not working

Check for existing issue
Please use issue search in the repository if the bug is something really obvious there's a high change it was already reported.
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
If it's not clear from the description how to reproduce the issue please add clear steps to reproduce the behavior.
Screenshots
If applicable, add screenshots to help explain your problem.
Diagnostics ID
Please generate diagnostics package from "About Playnite" menu and attach identifier you received. If the issue is a crash bug, you can generate diag. package directly from crash dialog.

Just to report again, I tried to log in today once more and it finally let me in. It asked me for my user and password twice (the second time, I had to fill in a security captcha in addition to my user and password). I copypasted the same password both times. On the second attempt, it finally triggered my 2FA and let me in. This was before updating to 8.12, so it might not be a bug anymore? Currently on 8.12 now and syncing my Amazon library at the moment.
Thanks for your hard work!

I will be changing http user agent in Playnite 9, which should hopefully make this more reliable (it at least seems to fix issues with Google sign-in).

Thanks. I apologize for not following up sooner. I wasn't able to find this thread (or one other) prior to receiving an e-mail today. Hopefully, the HTTP update corrects the issue, as I've been unable to authenticate my Amazon account since originally posting.
I'm not receiving the error that the other two individuals are.  When I attempt to authenticate via Playnite, I log into the account, which then has me verify my credentials twice, with the addition of a CAPTCHA code (letters and numbers). It then loads the Amazon website homepage into the modal window.

@loficat it did that with mine, too - it eventually closed on its own and let me in.

@loficat it did that with mine, too - it eventually closed on its own and let me in.

I just tried it again. I went through the two logins and then the homepage loads... Unfortunately, it remains. I've had it open for over 10 minutes. I don't think it's going to work for me. I'm glad it did for you. It has only authenticated the first time, during which I only had to enter my credentials once and then the modal window closed.

Please reproduce the issue again with this build of Amazon plugin and generate diag. package that was originally requested in issue template.
Replace files from the archive in <playnite install dir>\Extensions\AmazonGamesLibrary\.

Thank you for the new files. Unfortunately, they didn't work (for me). Something has changed, as the process only requires two login attempts (one without and one with the CAPTCHA). The Amazon homepage then loads (logged out). I also attempted the authentication after logging into Amazon via Edge prior to authenticating via Playnite with no success.
Normally, after going through the two (or three) logins, Amazon will send a notification to my phone, which I then have to authorize. Honestly, while I appreciate security, it's a pain. It's not something I opted in for.
If there's anything else I can do to help with this, please let me know. I appreciate your assistance.

I have not made any changes in that debug build, only added new debug messages into logs. If you don't send me diagnostics package generated with that special build, then I can't help you.

Here you are:
f3b76748-ce71-4f5d-ac3e-eea89ec90fb9

Can you make a video of the issue? It should definitely work according to logs, I can see that authentication worked and redirection url was called with access token (which is the moment when the login window should automatically close).

Also, try updating to 8.12 to see if it helps. It has newer version of web view component we use for authentication dialogs.


docs: Suggestion - Provide consistency in the description of setup.cfg

Summary of changes
Hi, I moved the metadata description from setup.py to setup.cfg.
I found it desirable to have consistent indentation in the documentation.

Python programmers are familiar with 4 space indentation. I also prefer consistent indentation in the setup.cfg documentation. What do you think?
Thanks to the maintainers of the setuptools source code and documentation.
Pull Request Checklist

[ ] Changes have tests
[ ] News fragment added in changelog.d. See documentation for details

Works for me. Thanks for the contrib!


Adding Penbook's dev log

Hi, and thanks for your submission!
Is there a blog available on the site? All I saw when I visited the link was an email sign up. I can see there are post titles in the RSS feed but I'm not sure it's all set up right? Am I missing something?

Ohhh now I see what's happening... I completely missed the link to go to the posts. That's pretty extreme 😬 I must admit if I landed there as a reader and wasn't able to see any content that I was signing up for, I would just close that window (in fact, that's what I did).


Disable test which fails intermittently

Purpose
This PR disable intermittently failing test till we remove the MB subscriptions
Fixes #14792 temporarily
Approach

Describe how you are implementing the solutions along with the design details.

Samples

Provide high-level details about the samples related to this feature.

Remarks

List any other known issues, releated PRs, TODO items or any other notes related to the PR.

Check List

[ ] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

This is also failing:
[ERROR] testExternalTimeQuery(org.ballerinalang.stdlib.streams.BallerinaStreamsV2TimeOrderWindowTest)  Time elapsed: 0.019 s  <<< FAILURE!
org.ballerinalang.util.exceptions.BLangRuntimeException:
error: {ballerina/task}TaskError {message:"Cannot initialize the Task Listener/Scheduler."}
at ballerina/task:init()
ballerina/task:__init(listener.bal:30)
ballerina/task:__init(scheduler.bal:22)
ballerina/streams:ballerina/streams.(snapshot.bal:41)
at org.ballerinalang.stdlib.streams.BallerinaStreamsV2TimeOrderWindowTest.testExternalTimeQuery(BallerinaStreamsV2TimeOrderWindowTest.java:47)
Can you please disable that as well?

This is also failing:
[ERROR] testExternalTimeQuery(org.ballerinalang.stdlib.streams.BallerinaStreamsV2TimeOrderWindowTest)  Time elapsed: 0.019 s  <<< FAILURE!
org.ballerinalang.util.exceptions.BLangRuntimeException:
error: {ballerina/task}TaskError {message:"Cannot initialize the Task Listener/Scheduler."}
	at ballerina/task:init(<native>)
	   ballerina/task:__init(listener.bal:30)
	   ballerina/task:__init(scheduler.bal:22)
	   ballerina/streams:ballerina/streams.<init>(snapshot.bal:41)
	at org.ballerinalang.stdlib.streams.BallerinaStreamsV2TimeOrderWindowTest.testExternalTimeQuery(BallerinaStreamsV2TimeOrderWindowTest.java:47)

Can you please disable that as well?

Sure

Codecov Report

Merging #14806 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #14806      +/-   ##
============================================
+ Coverage     64.25%   64.25%   +<.01%
  Complexity      544      544
============================================
  Files          2283     2283
  Lines        100178   100178
  Branches      13274    13274
============================================
+ Hits          64372    64374       +2
+ Misses        30768    30767       -1
+ Partials       5038     5037       -1

Impacted Files
Coverage Δ
Complexity Δ

...va/org/ballerinalang/channels/ChannelRegistry.java
94.59% <0%> (-2.71%)
0% <0%> (ø)

...ain/java/org/ballerinalang/net/websub/hub/Hub.java
69.53% <0%> (-1.57%)
0% <0%> (ø)

...rg/ballerinalang/net/http/WebSocketDispatcher.java
89.52% <0%> (-1.05%)
0% <0%> (ø)

...a/org/ballerinalang/bre/bvm/GlobalMemoryBlock.java
79.41% <0%> (ø)
0% <0%> (ø)
:arrow_down:

...e/src/main/java/org/ballerinalang/bre/bvm/BVM.java
77.62% <0%> (+0.03%)
0% <0%> (ø)
:arrow_down:

...ang/net/http/actions/websocketconnector/Close.java
72.72% <0%> (+2.27%)
0% <0%> (ø)
:arrow_down:

...a/org/ballerinalang/bre/bvm/WorkerDataChannel.java
84.56% <0%> (+2.46%)
0% <0%> (ø)
:arrow_down:

...rinalang/runtime/threadpool/ThreadPoolFactory.java
66.66% <0%> (+4.76%)
0% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c822adf...eb2a1d4. Read the comment docs.

@gimantha when we say Fixes #14792 temporarily the issue will get closed when we merge the PR. Better to change the description.

@gimantha I updated the description as we need to merge this PR immediately to continue with the builds of other PRs.


Question about geoarrow memory layout and inter-language communication

Dear geoarrow and arrow experts, I am trying to understand the memory usage and layout of a geoarrow object, e.g. a MixedGeometryArray object especially when passing its memory block between different languages. I created a minimal example in Rust:
let wkt_geoms = ["LINESTRING(-71.150281 42.248729,-71.150837 42.249113,-71.151144 42.24932)"];
let mut builder = StringBuilder::new();
wkt_geoms.iter().for_each(|s| builder.append_value(s));
let arr = builder.finish();
let crs = r#"THAT EXTREMELY LONG PROJJSON STRING FOR EPSG:4326"#; // is it really necessary to provide such a long string instead of only "EPSG:4326"?
let meta = Arc::new(ArrayMetadata {crs: Some(Value::String(crs.to_string())), edges: Some(Edges::Spherical)});
let geom_arr = MixedGeometryArray::<i32>::from_wkt(&arr, Default::default(), meta, false).unwrap();
let s = geom_arr.num_bytes();
println!("geom_arr.num_bytes(): {s}");

And I got this output:
Finished dev [unoptimized + debuginfo] target(s) in 1.58s
  Running `target/debug/geoarrow-rs-test`
geom_arr.num_bytes(): 52

Then my questions are basically around this 52 bytes:

is that really true that the created MixedGeometryArray variable geom_arr only occupies 52 continuous memory? The number makes more or less sense to me: 6 float64 number + some metadata is roughly 52 bytes, which is really condensed.
are the geotype, i.e. LINESTRING in this example and the srid, i.e. EPSG:4326 in that super long PROJJSON format also included in this 52 bytes?
is there some way to store these 52 bytes into a Vec<u8> so that I can see those binary stuff? with std::io::Write maybe?
where can I get the starting address of these 52 bytes?
if I want to "share" this memory block to another language, e.g. C or Python, and the other language would probably need to persist this binary chunk onto a disk so that in the future it can restore it back into the memory, what would be the best way for transferring/sharing these 52 bytes?

I have some potential solutions to 5:
a). via Arrow FFI (as Kyle kindly suggested in another ticket, and sorry that I'm still somehow stuck here...) - this is hard :/ and I try to avoid unsafe code blocks
b). write the raw 52 bytes memory chunk into a Vec<u8> and send it to the other language
c). via Arrow's Buffer
d). via geoarrow's RecordBatch and GeoTable
Any suggestions or hints will be very appreciated. Thanks a lot!!

let crs = r#"THAT EXTREMELY LONG PROJJSON STRING FOR EPSG:4326"#; // is it really necessary to provide such a long string instead of only "EPSG:4326"?

The spec currently defines PROJJSON as the way to materialize CRS. In particular, PROJJSON has some nice benefits around being able to interpret the CRS without access to PROJ.
The CRS is stored only once per column in a table, so it's very little overhead.

is that really true that the created MixedGeometryArray variable geom_arr only occupies 52 continuous memory? The number makes more or less sense to me: 6 float64 number + some metadata is roughly 52 bytes, which is really condensed.

Yes.

are the geotype, i.e. LINESTRING in this example and the srid, i.e. EPSG:4326 in that super long PROJJSON format also included in this 52 bytes?

No. Though here the API is mixed, that metadata is stored on the Arrow field, separately from the array buffers themselves.

is there some way to store these 52 bytes into a Vec<u8> so that I can see those binary stuff? with std::io::Write maybe?

Well, you can export to an Arrow Arc<dyn Array> (here a UnionArray) and traverse its buffers. But those 52 bytes aren't guaranteed to be a single Vec<u8>. You can refer to the Arrow Dense Union docs, which defines how it's laid out in memory.

where can I get the starting address of these 52 bytes?

There are a few different ways, but I really don't think you want this.

if I want to "share" this memory block to another language, e.g. C or Python, and the other language would probably need to persist this binary chunk onto a disk so that in the future it can restore it back into the memory, what would be the best way for transferring/sharing these 52 bytes?

If you're working with another language that can access the same memory space, e.g. C or Python, Arrow FFI is the publicly recommended approach. You shouldn't need to write any unsafe blocks yourself I don't think. The upstream Arrow crate should manage the unsafe parts for you.
Otherwise you want to use Arrow IPC, which will maintain the field metadata as well. See arrow-ipc or write_ipc_stream

Thanks very much @kylebarron for your very detailed reply! I am kinda looking for a way to bundle the array buffer and metadata together (geometry + srid), have them stored in a continuously memory chunk and share it with C (or Python). The separation of FFI_ArrowArray and FFI_ArrowSchema bothered me a bit from the C side. I actually only need a persistable and restore-able memory block (as condensed as possible) without the necessity of interpreting its internal details in C. Anyway, I will continue to try with arrow::ffi. Meanwhile, I will also try geoarrow's write_ipc_stream, which requires to create the GeoTable with RecordBatch if my understanding is correct. I hope the GeoTable way won't bring too much extra overhead. Will post here some updates if I've got any success. Thanks again!

have them stored in a continuously memory chunk and share it with C

If you don't want to use FFI, use Arrow IPC. This is what it's there for. You can serialize to a single Vec<u8> which stores both the schema and all array chunks.

The separation of FFI_ArrowArray and FFI_ArrowSchema bothered me a bit from the C side

Why did it bother you? You can describe many chunks of an array with only a single schema object, which is much more efficient than copying the schema with every array chunk.

the GeoTable way won't bring too much extra overhead.

The overhead is... on the order of two vec allocations?

well, regarding the overhead concern I mentioned, in most of my cases there will be only one single simple OGC geometry object without any extra column or attribute. I've considered simply using ewkb/wkb bytes, but gave up because it's too sparse (122 Bytes for the above example linestring) and started trying with geoarrow. In the future, there might be some use cases containing a table- or subset-like data package, but unsure yet at the moment. Anyway, I think I will go with Arrow IPC and GeoTable

I've considered simply using ewkb/wkb bytes, but gave up because it's too sparse

I'm surprised that EWKB would be too sparse... it's designed to be as compact as possible. It's not even aligned on 8 bytes.
Anyways, I think the question has been answered and I'll close this.


WIP: Custom thumbnails

Related to #25

Tabs are introduced for the options
Custom image is downloaded and set

TODO:

[ ] good grief the CSS :crying_cat_face:
[ ] resize image while saving

It looks like ass at the moment, but it works.

If I remember correctly, the CSS for the dials expects a fixed image size. So you will have to write the loaded image to a canvas with that size and adjust the size so it's not distorted. Maybe for setting an image it would be better to right click a dial and tweak the image in an overlay. You don't have much control over how the settings menu looks like. Not a huge fan of the async/await you used because it doesn't fit the rest of the code (they weren't available when I wrote the extension, if I started over now, I would probably use it too...)

Nothing new in more than half a year, I'll close this for now.


Add a prettier configuration option for captureWhitespace

Is your feature request related to a problem? Please describe.
{% capture var %} is technically whitespace sensitive. It captures its contents to a variable. Because of this and because we should not to change the semantic meaning of the file with default options, we were forced to use "paste-as-is" logic for the contents of capture tags.
It has been reported by partners that this level of strictness is not always necessary. You can turn that feature off by using a white-space: normal comment above the capture tag, but some folks would want this by default and the other option on-demand.
@stijns96 proposed having a prettier config that switch the default for that. I like that idea.
Describe the solution you'd like

Add a new prettier configuration setting that changes the css-whitespace property of the capture tag. Something like this:

"captureWhitespace": "strict" | "ignore"

Reusing the nomenclature for htmlWhitespaceSensitivity.

strict (default), whitespace is considered significant and will be maintained as pre
ignore, whitespace will not be considered significant. Same as {% # white-space: normal %} before capture tags

Additional context
Breadcrumbs:

Where we define our configuration variables
Where we would want to switch to a different default for capture tags depending on the config value. Note that that function doesn't accept options like the CssDisplay one above, we should change that.

Closed in https://github.com/Shopify/theme-tools/pull/355


Missing implementation on type 'SQLitePCL.SQLite3Provider_e_sqlite3'

This appears to be very similar to #211 but this one has to do with sqlcipher. I am using SQLitePCLRaw.bundle_sqlcipher and I get the same message. Error: NUTC303B:Internal Compiler Error: Method 'instance System.Int32 SQLitePCL.ISQLite3Provider.sqlite3_bind_blob(System.IntPtr, System.Int32, System.Byte[], System.Int32)' is missing implementation on type 'SQLitePCL.SQLite3Provider_e_sqlite3' from assembly 'SQLitePCLRaw.provider.e_sqlite3' while loading type 'SQLitePCL.SQLite3Provider_e_sqlite3'. while computing compilation roots.
I tried the suggestion in #211 but that did not work in my case and I was wondering if there is some other package I should use. This issue only arises when compiling with the native toolchain. It compiles in debug or release with the native toolchain disabled.
I do have a sample app if that would be helpful.
Thank you advance for any advice.
Walter

If you have a sample app to repro the problem, yes, I would like to see it.  Thanks.

Hmmm.
I confirm that I can reproduce the problem with the sample app you sent.
But I am not yet sure what to do about it...
I'll dig deeper.

Thank you very much for taking a look at this!

Eric, any thoughts yet on how to resolve this or maybe an idea for a workaround?

Sorry for the delay.  Crazy week.
According to the results of some Google searches, in previous times when this error message appeared, it was a bug in the .NET Native code generator.  I can't be sure that the same is true this time, but I suspect it, partly because it is happening on a case that might be a bit odd, an overloaded method in an interface, with the overloads sharing the same first N params (IIRC).
I would like to gather some more evidence to back up this theory.
After that, either I need to workaround the issue or get the the relevant team to address it.  The workaround would be problematic, because it would probably involve a compatibility-breaking change.
The ball remains in my court.

Absolutely no worries. I definitely understand being too busy!
My many thanks for the update and please let me know if there is any way I can help.
I did, just for grins, try to compile after updating to 15.8.4 hoping it was a compiler/code generation issue and it was the same result as before.
With security being so important and this being the best way to get encryption support for SQLite in UWP maybe you could suggest to the appropriate team to put in a unit test in for this before future updates to the .NET Native. code generator.

Any further info on this? The problem still persists.

Not yet, but I'm currently working on another release, and hoping to give this some attention as part of that cycle.

I'm sorry I've allowed issue this to dangle in the wind.
Let's throw up a bat signal to @bricelam
Brice, I've got an issue here that I suspect is a bug in .NET Native, but I don't have enough evidence for an indictment, much less a conviction.
Nonetheless, it appears this is happening due to a call from EF Core, so it's possible the issue is (or should be) on your radar screen.
If you have other customers experiencing this problem, it looks like it would be at the intersection of UWP, sqlite, and blobs.  Have you by chance seen this?
Like I said, my guess is that the .NET Native toolchain is losing track of one of the overloads of sqlite3_bind_blob.  In part, this guess is based on googling and seeing that previous appearances of a similar error message did turn out to be bugs of that flavor.  Also, I don't see how this problem could occur any other way, but I'm trying to stay open minded on that.  :-)

@divega Know anyone on the .NET Native team we can get to help us dig into this?

@MichalStrehovsky has been super helpful in the past with .NET Native issues.

Hmph.
That's good info.  Disturbing, but good.
Thanks.

OK, apologies to everyone here -- I should have caught this much sooner.
The thing that should have been glaringly obvious from the beginning is that this is a sqlcipher project, so SQLite3Provider_e_sqlite3 isn't supposed to be there at all.  So why are we getting an error message about it?
The library project in this solution is referencing Microsoft.AppCenter.Crashes, and that is bringing in sqlite-net, which is bringing in an old version of bundle_green, which is causing build problems because bundle_sqlcipher is there too.
The easiest workaround is to just remove Microsoft.AppCenter.Crashes.  The solution builds just fine without it.

Eric, excellent news and short term we can do that. Longer term we would like to keep using AppCenter.Crashes along with encrypting our own SQLite databases so should we file an issue with the AppCenter SDK team and see if something can be done to make this compatible when used together? Any other ideas or suggestions?

See Microsoft/AppCenter-SDK-DotNet#569

I don't know the difference between your various versions but the docs for Microsoft.Data.Sqlite.Core prescribe your SQLitePCLRaw.bundle_winsqlite3. And this one is in conflict with AppCenter, resulting in this error message. If I use SQLitePCLRaw.bundle_green instead, it works (at least I didn't find any difference in functioning). You might be able to tell whether this is a viable alternative or not but if it is, the docs might need to be changed.

I don't know the difference between your various versions but the docs for Microsoft.Data.Sqlite.Core prescribe your SQLitePCLRaw.bundle_winsqlite3. And this one is in conflict with AppCenter, resulting in this error message. If I use SQLitePCLRaw.bundle_green instead, it works (at least I didn't find any difference in functioning). You might be able to tell whether this is a viable alternative or not but if it is, the docs might need to be changed.

@deakjahn Those docs were written by the team who creates winsqlite3.dll 😉 Of course they'd prescribe it...

@deakjahn Those docs were written by the team who creates winsqlite3.dll 😉 Of course they'd prescribe it...


Add SmiteshP/nvim-navbuddy

Checklist:

[x] The plugin is specifically built for Neovim, or if it's a colorscheme, it supports treesitter syntax.
[x] The lines end with a .. This is to conform to awesome-list linting and requirements.
[x] The title of the pull request is Add/Update/Remove `username/repo`  (notice the backticks around `username/repo`) when adding a new plugin.
[x] The description doesn't mention that it's a Neovim plugin, it's obvious from the rest of the document. No mentions of the word plugin unless it's related to something else.
[x] The description doesn't contain emojis.
[x] Neovim is spelled as Neovim (not nvim, NeoVim or neovim), Vim is spelled as Vim (capitalized), Lua is spelled as Lua (capitalized), Tree-sitter is spelled as Tree-sitter.
[x] Acronyms should be fully capitalized, for example LSP, TS, YAML, etc.

Changed the location of nlsp-settings plugin because symbols-outline.nvim, aerial.nvim and nvim-navbuddy all three provide similar functionality and I though it would be good if they were grouped together in the list.

Thanks for the PR! Can you also change the description of the item you moved to not mention that it's a plugin?

@tversteeg Changed the description 👍🏽


Utils functions - part 2

Add:
_Mattie functions for working with the Fiber tree
_Initial test for the utils functions
Change:
_traverseWith implementation.

Add:
_Mattie functions for working with the Fiber tree
_Initial test for the utils functions
Change:
_traverseWith implementation.

_Mattie <- citation required


Add Dark Mode

Adding a dark mode here would not be as much a priority as adding dark mode on the Stuyspec website, but it would certainly be nice to have a dark mode for the CMS as well.

stuyspec/stuyspec.com#330


Upgrade to node 12.4, and use smtp-server

Updated the library to use smtp-server, bluebird promises, and node 12.4.
The travis tests no longer run the code beautifier, just the mochaTest command.
Made more of the options needed for the smtp server as command line options.

Awesome! Your last commit says that all the tests pass, however the Travis build failed, any idea?
Let me review this one thoroughly.

There are a few things I am not sure how the defaults should be handled.  smtp-connection defaults to requiring AUTH, I added a parameter to be able to turn it off but I am not sure if it should be off by default.
As for the tests, I just can't get them to fail like the server is.  I am running on OSX with node 0.12.4, and npm 2.10.1  Any ideas?

+1

Ok, I'll dig into why the tests fail and I'll should be able to merge it begining of next week.

@DavidTanner @dapriett Guys, quick update. Sorry for the delay, I had a baby last week, needless to say did not had much time ;). I should be able to look at this very soon.
@DavidTanner Would you mind updating the pull request with your latest commits? By the way I think that the tests are failing because .travis.yml is running the tests against node 0.10.

I just updated the engine version in the package.json, but the travis config is set to use 0.12.4

Now using node v0.12.4
$ node --version
v0.12.4
$ npm --version
2.10.1
$ nvm --version
0.23.3

it won't work with node 10

@DavidTanner @dapriett Merged and published on npm. Mailin v3.0.0 needs node ~0.12.0, mailin ^2.0.0 is still compatible with older node versions.
@DavidTanner Thanks again !


Unable to compile

I'm getting this error when I try to compile
matplotlibcpp.h:177:29: error: cannot convert ‘wchar_t**’ to ‘char**’

Hello,
I'm not really sure about what I'm going to say, but I think it's because you use Python2 and not Python3. When I link Python2 I have this error and not with Python3 by the way.
But nevermind, you can replace the line 177 PySys_SetArgv(argc, const_cast<wchar_t **>(argv)); by those two lines:
char ** argm = (char **)(argv);  PySys_SetArgv(argc, argm);
At least, it worked for me when using Python2.

I got the same error.
The readme example looks like it's supposed to link to python 2.7
g++ minimal.cpp -std=c++11 -I/usr/include/python2.7 -lpython2.7


Coding guides for ControlNet

Looking for more coding guides with control net. I don't want to use Automatic1111 and most guides out there are just for Automatic1111. I been following some of the diffusers library guide, but their examples are quite limited.
I been trying as of late to use img2img with openpose, depth maps, and canny. I been trying to generate an initial image, then feed the most recently created image back into the img2img; this way, I can have an animation get generated. I create the openpose/depth map images I create from blender.

please use the discussion section for general questions!
look here: https://huggingface.co/docs/diffusers/using-diffusers/controlnet

thanks

Do you know of anything that kinda just is raw controlnet code usage? I use huggingface, but man, a lot of things they code has been non performant. They only recently started caring about performance.


Jacobians

I've added Jacobians of exp and log as a step forward towards full support of robotics stuff. What do you think about this approach?
The tests compare results against a finite difference computation on a sphere.

While this looks good to me in general, the tests are currently failing.
Also great that you directly provided a very general allocation for the Jacobian.

Maybe basis_domain and basis_codomain would fit better with what we have in Manopt.jl already?

I must have missed that naming, yes I think your new proposals fit petter into Manopt and I would prefer them.


Use Vite

This PR configures Storybook to use the Vite builder.
Caveats

Some dev dependencies that are implicit can’t be found by Vite, so they're hoisted directly to devDependencies along with iterall (which pnpm recommends as an alternative to having to --shamefully-hoist) to fix Storybook’s build.

Vite is causing many Playwright tests to take longer than 5s. Working with the community on Discord to resolve. Till then, putting this in draft state.


Drag and drop in big trees takes a long time to initialize

With big trees (i.e. with more than 500 nested items) drag and drop does not work. Strange enough, it seems to start working after just waiting for a long time.
For example, I generated a 3-level deep tree in which each item has 8 children. That tree contains 8^3 = 512 items in total. Drag and drop in such a tree does not work after the demo page has loaded. Dragging the top-most item simply freezes the dragged handle. Yet after waiting for about 10 seconds, drag and drop becomes responsive again. I tested this in Chrome 36.0.1985.125 on Ubuntu 13.10.
The issue becomes even more noticeable when the model of each item gets a 'collapsed' boolean that is passed to the ui-tree-node directive with
    data-collapsed="item.collapsed"

The bigger the tree, the longer one has to wait before drag and drop becomes responsive. In a tree with 1000 items, it already takes several tens of seconds.

there is problem in rendering the tree and take more time and the page get irreponsive it may affect performance issue


Update licenses.html to reflect the change of the home page

Because the Apache 2.0 license has been removed from the home page, thus update licenses.html to reflect the change.

The travis build failure is caused by Eclipse che switched to EPL2.0; Fixed in another patch (https://github.com/github/choosealicense.com/pull/599).

Thanks!


bug: SharePoint permissions fetching should be opt-in

Describe the bug
I running the SharePoint connector from the cli without specifying any permissions options, but getting an error about not being able to obtain the permissions token
To Reproduce
Run SharePoint connector without permissions options:
PYTHONPATH=. ./unstructured/ingest/main.py \
  sharepoint \
  --download-dir /tmp/sharepoint-demo/download \
  --num-processes 6 \
  --strategy fast \
  --preserve-downloads \
  --reprocess \
  --output-dir /tmp/sharepoint-demo/result \
  --verbose \
  --client-cred "$SHAREPOINT_CRED" \
  --client-id "$SHAREPOINT_CLIENT_ID" \
  --site "$SHAREPOINT_SITE" \
  --path "Shared Documents" \
  --recursive

Expected behavior
Permission fetching should not be attempted.
Screenshots
  --recursive
2023-10-26 08:40:20,854 MainProcess DEBUG    options: {'download_dir': '/tmp/sharepoint-demo/download', 'num_processes': 6, 'strategy': 'fast', 'preserve_downloads': True, 'reprocess': True, 'output_dir': '/tmp/sharepoint-demo/result', 'verbose': True, 'client_cred': '*******', 'client_id': '*******', 'site': 'https://030rx-admin.sharepoint.com/', 'path': 'Shared Documents', 'recursive': True, 'skip_infer_table_types': None, 'pdf_infer_table_structure': False, 'ocr_languages': None, 'encoding': None, 'fields_include': ['element_id', 'text', 'type', 'metadata', 'embeddings'], 'flatten_metadata': False, 'metadata_include': [], 'metadata_exclude': [], 'partition_by_api': False, 'partition_endpoint': 'https://api.unstructured.io/general/v0/general', 'api_key': None, 're_download': False, 'download_only': False, 'max_docs': None, 'embedding_api_key': None, 'embedding_model_name': None, 'chunk_elements': False, 'chunk_multipage_sections': False, 'chunk_combine_under_n_chars': 500, 'chunk_new_after_n_chars': 1500, 'work_dir': '/Users/ryannikolaidis/.cache/unstructured/ingest/pipeline', 'raise_on_error': False, 'permissions_application_id': None, 'permissions_client_cred': None, 'permissions_tenant': None, 'max_retries': None, 'max_retry_time': None, 'files_only': False}
2023-10-26 08:40:20,856 MainProcess INFO     running pipeline: DocFactory -> Reader -> Partitioner -> Copier with config: {"reprocess": true, "verbose": true, "work_dir": "/Users/ryannikolaidis/.cache/unstructured/ingest/pipeline", "output_dir": "/tmp/sharepoint-demo/result", "num_processes": 6, "raise_on_error": false}
2023-10-26 08:40:22,368 MainProcess INFO     Running doc factory to generate ingest docs. Source connector: {"processor_config": {"reprocess": true, "verbose": true, "work_dir": "/Users/ryannikolaidis/.cache/unstructured/ingest/pipeline", "output_dir": "/tmp/sharepoint-demo/result", "num_processes": 6, "raise_on_error": false}, "read_config": {"download_dir": "/tmp/sharepoint-demo/download", "re_download": false, "preserve_downloads": true, "download_only": false, "max_docs": null}, "connector_config": {"client_id": "8ed640cc-c6de-4ff1-8c18-2abcecb1e547", "client_credential": "b+OfndJ4wrirwOOi3eT4/9F+dto2sGsCmj7X8CvWRJ4=", "site_url": "https://030rx-admin.sharepoint.com/", "path": "Shared Documents", "process_pages": true, "recursive": true, "permissions_config": {"application_id": null, "client_cred": null, "tenant": null}}}
--- Logging error ---
Traceback (most recent call last):
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/connector/sharepoint.py", line 66, in get_permissions_client
    permissions_connector = SharepointPermissionsConnector(self.permissions_config)
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/connector/sharepoint.py", line 398, in __init__
    self.initialize()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/connector/sharepoint.py", line 401, in initialize
    self.access_token: str = self.get_access_token()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/utils.py", line 179, in wrapper
    return func(*args, **kwargs)
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/connector/sharepoint.py", line 418, in get_access_token
    return response.json()["access_token"]
KeyError: 'access_token'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryannikolaidis/.pyenv/versions/3.10.11/lib/python3.10/logging/__init__.py", line 1100, in emit
    msg = self.format(record)
  File "/Users/ryannikolaidis/.pyenv/versions/3.10.11/lib/python3.10/logging/__init__.py", line 943, in format
    return fmt.format(record)
  File "/Users/ryannikolaidis/.pyenv/versions/3.10.11/lib/python3.10/logging/__init__.py", line 678, in format
    record.message = record.getMessage()
  File "/Users/ryannikolaidis/.pyenv/versions/3.10.11/lib/python3.10/logging/__init__.py", line 368, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/./unstructured/ingest/main.py", line 11, in <module>
    main()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/./unstructured/ingest/main.py", line 7, in main
    ingest_cmd()
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/core.py", line 1666, in invoke
    rv = super().invoke(ctx)
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/ryannikolaidis/.pyenv/versions/unstructured/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/cli/base/src.py", line 41, in src
    runner.run(**options)
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/runner/sharepoint.py", line 63, in run
    self.process_documents(
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/runner/base_runner.py", line 57, in process_documents
    process_documents(
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/processor.py", line 86, in process_documents
    pipeline.run()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/pipeline/pipeline.py", line 51, in run
    json_docs = self.doc_factory_node()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/pipeline/interfaces.py", line 66, in __call__
    self.result = self.run()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/pipeline/doc_factory.py", line 10, in run
    docs = self.source_doc_connector.get_ingest_docs()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/connector/sharepoint.py", line 377, in get_ingest_docs
    permissions_client = self.connector_config.get_permissions_client()
  File "/Users/ryannikolaidis/Development/unstructured/unstructured/unstructured/ingest/connector/sharepoint.py", line 70, in get_permissions_client
    logger.error("Couldn't obtain Sharepoint permissions ingestion access token:", e)
Message: "Couldn't obtain Sharepoint permissions ingestion access token:"
Arguments: (KeyError('access_token'),)

Environment Info
NA
Additional context
None

Came up to this in #1774 and had a quick fix already
Also adding opt-in functionality now 👍


Error: 505 when I try to install

Oneplus One (A0001)
Android: 5.1.1 (Cyanagen OS 12.1-YOG4PAS3JL)
Not Rooted
I'm getting the following when I try to install:

You probably have an app installed that defines the same custom permission as K-9 Mail. But newer Android versions don't allow two apps to define the same permission.
Closing this because it's not a bug in K-9 Mail.

See also: https://commonsware.com/blog/2014/08/04/custom-permission-vulnerability-l-developer-preview.html


Is there an offline version for this?

I would like it if this markdown editor can be used as a standalone application for Windows, Mac, Linux, Android and iOS. I really love how fully featured this online application is (to the point that I believe it could replace word processors in some cases for my workflow), but I dislike that it depends on a constant connection to be used, and that it can't be used on mobile. Would you be able to fulfill this?

It only relies on an Internet connection the very first time you open it. Once fully loaded, it's available offline.


Rework UI

So now that I am using it more. I feel that the quick panel is not the best option to show the compass. It blocks the view when navigating through files, especially files with a small number of lines.
I am rethinking the UI to be something like this

It shows down below while still showing the preview above without blocking it.
Let me know your thoughts.

I love this, similar to Doom Emacs.

I've done something similar with terminus, fzf, and ripgrep. I can live grep for words in the active folders. It's a little hacky right now but it seems it could be much nicer.

Cool! I actually have something similar to that already. If I can manage to convert the UI to this, we can totally create things like this fuzzy grep. The current implementation is using a quick panel which is bound to its limitations.

@diocletiann What the heck... Can you share the code for what you've done here?

@diocletiann What the heck... Can you share the code for what you've done here?

Sure. I (poorly) adapted one of the examples from https://github.com/junegunn/fzf/blob/master/ADVANCED.md#using-fzf-as-interactive-ripgrep-launcher. What you see is a custom Terminus command that runs the shell script below.
#!/bin/dash

RG_PREFIX="rg --column --no-heading --smart-case --color always --colors 'match:fg:red' --colors 'path:none'";
: | fzf \
	--ansi \
	--scroll-off=3 \
	--layout=reverse \
	--disabled \
	--bind "change:reload:sleep 0.2; $RG_PREFIX {q} || true" \
	--print0 \
	--delimiter : |
	cut -d ':' -f 1-3 |
	xargs -0 -I {} subl {} && exit


Minimal Admin API

We need an admin (superuser) API. Users table needs a boolean "is_superuser" or similar. Admin APIs are things like creating a new community, etc.
Waiting for a list of minimal absolutely required API calls from @ItsiW.

Requires #1373

Aapeli:

Get a user's email address based on username or user id (for support/moderation stuff)
Create a community (there's some code in the tests, create_community I think. I was thinking that the community's polygon would be supplied as a geojson object from something like https://geojson.io/)

Ban a user
Delete a user (policy team is still figuring out what else to do than set is_banned, but that's the first thing)


Expose a stand-alone DML generator and entity source code generator

Right now, DML script are displayed in logs file during unit testing. It would be more convenient for expose a DML generator and entity source code generator.
It requires refactoring the PersistenceManagerFactory internals to clearly separate the entity parsing/statement preparing/schema validation/DML generation/runtime

Fixed in Achilles 4.0.0


code and ckpts

Hi,
Thanks for your amazing work ?
could you update us about the time of release code and ckpts?

Hi, thanks for your attention, we will release our code and ckpts next month.

@lzw-lzw any update？

Waiting for updates

We have released our code, and we will release our dataset and model soon.

Thank you, @lzw-lzw
I’m also looking forward to seeing the dataset and model weights soon! 😃


Update Blog “yadokawa”

Automatically generated by Netlify CMS

👷 Deploy Preview for koyomiy processing.

Name
Link

🔨 Latest commit
7ad8c76eb46fb994073895cacde4cd0dd9abc78a

🔍 Latest deploy log
https://app.netlify.com/sites/koyomiy/deploys/62d25e3050d4c1000854dfb0


Creation of .nfo file

Would be great if a .nfo file for each downloaded movie is created.
This should be compatible for Kodi https://kodi.wiki/view/NFO_files#Metadata_nfo

Cool Idea :)
I have no time these days but will note it down for later this month.

I had a first look and have some questions. Did you have contact with .nfo files already?
They support movies and tv shows (and music stuff). The problem here is, that mtv_dl does not know if a download is a "serial" tv show or a movie. Also mtv_dl is missing most of the information required for tv shows. The only way we could use this is to provide a movie-.nfo for all downloads.
Would this confuse Kodi? What information would you expect to get with this file (more than title and description)?

Please have a look to the 'nfo' branch for how this could look like.

Looking at the code seems exactly to fit my issue!
Unfortunately testing needs to wait three weeks. I'll report than.

Sure, no problem. I will wait :)

I did some testing and the .nfo files are generated as expected.
Unfortunately Kodi ignores these files.
I've created some .nfo files manually (with the same xml tags and contents) and these are used by Kodi. I tried to approximate the manual written ones to the generated ones, to figure out, what is stopping Kodi from using the generated ones (maybe something with encoding, newlines, ids, ...). But no success yet. I'll need some more time and report than.

Sure, no problem. I could also try to spot the difference if you give me the files, but I cannot test then.


Academicpages website not appearing in google search result

Hi everyone,
My academic pages website is up and running but it does not appear on google search. Can anyone please suggest ways to improve the visibility of the website on google?
Any help or guidance is much appreciated.
Thanks in advance.

Mine took about a week to show up after following instructions here: https://support.google.com/webmasters/answer/34397?hl=en

It takes a week or two for Google to find your website and start crawling after you follow the instructions from the link that I provided above. It is unfortunately necessary to be patient. You cannot get Google to find your website immediately. I don't know anything about an html file.

It takes a week or two for Google to find your website and start crawling after you follow the instructions from the link that I provided above. It is unfortunately necessary to be patient. You cannot get Google to find your website immediately. I don't know anything about an html file.

I used the link that you posted and then I entered my URL and clicked verify. Next, I get some options like to upload an HTML file on my website in order to get verified..

It takes a week or two for Google to find your website and start crawling after you follow the instructions from the link that I provided above. It is unfortunately necessary to be patient. You cannot get Google to find your website immediately. I don't know anything about an html file.

I used the link that you posted and then I entered my URL and clicked verify. Next, I get some options like to upload an HTML file on my website in order to get verified..

Just upload the HTML file to the master file (the file where README.md is)

It takes a week or two for Google to find your website and start crawling after you follow the instructions from the link that I provided above. It is unfortunately necessary to be patient. You cannot get Google to find your website immediately. I don't know anything about an html file.

I used the link that you posted and then I entered my URL and clicked verify. Next, I get some options like to upload an HTML file on my website in order to get verified..

Just upload the HTML file to the master file (the file where README.md is)

Can you explain exactly what you mean? To upload into the GitHub repository the HTML file that I get from Google for verification?

It takes a week or two for Google to find your website and start crawling after you follow the instructions from the link that I provided above. It is unfortunately necessary to be patient. You cannot get Google to find your website immediately. I don't know anything about an html file.

I used the link that you posted and then I entered my URL and clicked verify. Next, I get some options like to upload an HTML file on my website in order to get verified..

Just upload the HTML file to the master file (the file where README.md is)

Can you explain exactly what you mean? To upload into the GitHub repository the HTML file that I get from Google for verification?

Yes. You get an HTML file from google. Then you upload the file to the GitHub repository. Make sure you can open the file via the following link:
yourname.github.io/googlefilename.html


a waring under jdk11

[ ] I have searched the issues of this repository and believe that this is not a duplicate.

Environment

Dubbo version: 3.1.1
Operating System version: windows10
Java version: 11.0.2

Steps to reproduce this issue
no steps ,just running
Expected Behavior

Actual Behavior

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.alibaba.com.caucho.hessian.io.Hessian2Input (file:/D:/repository/org/apache/dubbo/dubbo/3.1.1/dubbo-3.1.1.jar) to field java.lang.Throwable.detailMessage
WARNING: Please consider reporting this to the maintainers of com.alibaba.com.caucho.hessian.io.Hessian2Input
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

This error will not affect any functionality in JDK11. The reason for the error is that JDK will limit reflection in future versions. Regarding the support of JDK 17, we will provide native support based on fastjson2 in 3.2.0.

这个不影响正常使用的，只是jdk版本的一种警告。我已经在生产环境使用了jdk11


[jsonschema] remove validation exception catching

Codecov Report

Merging #36 (8eb793f) into main (b6b6d9d) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #36   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            3         3
  Lines          192       191    -1
  Branches        37        37
=========================================
- Hits           192       191    -1

Flag
Coverage Δ

unittests
100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

target_s3_jsonl/__init__.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6b6d9d...8eb793f. Read the comment docs.


Handle React Elements and Circular References

don't recursively traverse React elements with owners
warn (don't crash) on other recursive calls

I've just enabled travis. Closing this PR and re-opening to kick travis.


refactor(ContentGroupBaseCard): Add margin between description and metadata on small cards

Overview

PR Checklist

[ ] Related to designs:
[ ] Related to JIRA ticket: [ABC-123]
[ ] I have run this code to verify it works
[ ] This PR includes unit tests for the code change
[ ] This PR includes testing instructions tests for the code change
[ ] The alpha package of this PR is passing end-to-end tests in all relevant Codecademy repositories

Testing Instructions

Don't make me tap the sign.
PR Links and Envs

Repository
PR Link
PR Env

Monolith
Monolith PR
Monolith Env

Portal
Portal Link
Portal Env

Another Repo
Another Link
Another Env

📬Published Alpha Packages:@codecademy/gamut-kit@0.6.287-alpha.520e5f.0@codecademy/gamut-labs@35.6.2-alpha.520e5f.0<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://63fcf2fc53d8c21005f03c96--gamut-preview.netlify.appDeploy Logs


Module 'System' has no exported member 'String'

I created a function which returns string in one of the entity EndPoint class. after that i am getting
error
Module 'System' has no exported member 'String'
while rebuilding application.
Application runs. But i wan to know
where i can make changes for this error.
regards
Shailesh

Endpoints can't return string, must be something derived from ServiceResponse

It works fine.
Thanks
regards
Shailesh


Feat/connectors

Finally brings some documentation about Connectors.
Details
The overview of the connectors section contains introduction elements to properly define the connector notion, details their responsibilities, and introduce the Axone SDK exploring the features it covers.
The Storage subsection has been added to guide the implementation of a storage proxy connector with code samples.

Summary by CodeRabbit

New Features

Introduced comprehensive guides for connecting external resources and building storage provider connectors using the Axone protocol.
Added a section on the Axone SDK, detailing its features and providing a link to the SDK documentation.

Documentation

Updated terminology in the governance document for consistency.
Removed outdated connector category documentation.
Added new JSON files to categorize connectors effectively.

Bug Fixes

Eliminated redundancy by removing the outdated overview document on connectors.

@coderabbitai resolve


Copy pasting an image from 4K laptop screen results in error (2mb size limit)

Issue:
I used to be able to take a partial or full screenshot and paste it into Excalidraw to then annotate.
I can no longer do this. When attempting to, I see the below:

Notable information:

Zoom was reset to 100%
Width of screenshot was full width (3840px)
Method of screenshot is by using "Windows + Shift +S"
The copied image does not appear on the clipboard when clicking "Windows + V"

Exceptions:
Opening this copied image into Snipping Tool and then clicking the in-built copy function within that, then allows me to paste within Excalidraw.

Are you working locally when this occurs?  If not, this might involve the 1MB Firebase storage limits.

Working locally, yes.
I've not experienced the issue again.
It may be have been linked to the zoom level being up high when the image
was pasted. I always ensure the zoom is reset before starting a new image.
Thanks
On Wed, 28 Dec 2022, 00:59 DanielJGeiger, @.***> wrote:

Are you working locally when this occurs? If not, this might involve the
1MB Firebase storage limits.
—
Reply to this email directly, view it on GitHub
https://github.com/excalidraw/excalidraw/issues/5941#issuecomment-1366292383,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALYE4TS362OCVEJYZBFLNALWPOGIXANCNFSM6AAAAAASRVGVVM
.
You are receiving this because you authored the thread.Message ID:
@.***>


Possible for a CustomElement to manipulate a template part?

From within a CustomElement I would like to be able to manipulate a template part provided by the consumer of the CustomElement. For example I'd like to be able to add classes to the elements within the template part so the consumer doesn't have to.
Anyway to accomplish this?

One way I've manged to do this is by specifying @skipContentProcessing on the customer element, modifying the contained template and then injecting/compiling back into the custom element at the correct location during bind().
Would this be the recommended way or is there a better way to go about it?

It sounds like @processContent was designed for cases like this, so you might want to take a look at that. You can manipulate the content before the aurelia template compiler deals with it?

@csaloio are you still looking for a solution for this issue? If so, could you provide some more info about your use-case? I agree, manually compiling the template sounds a little heavy-handed.

I haven't worked on this in some time, but the scenario was for a datagrid custom element that had user supplied columns. The consumer of the control specified each column like this: (syntax may be outdated now):
<cs-grid src.bind="rows" multiple cs-grid.ref="grid">
   <template replace-part="columns">

${i.id}

${i.firstname + ' ' + i.lastname}

   </template>
</cs-grid>

As the author of the grid control it is useful to be able to modify the divs specified by the consumer within the template part for example to add css classes so the consumer does't have to.

You can accomplish this with the processContent hook.


⚠ Bofrost.es has degraded performance

In 81b48d4, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1553 ms

Resolved: Bofrost.es performance has improved in d222ef8 after 19 minutes.


Firebase login

Hi,
Can I use firebase login (firebase auth) with matrix sdk login?
I have done firebase login and I have firebase user. How do you I connect my firebase user with riot (matrix-sdk)?

it is not implemented yet.

@vishnubenny
The login is designed to work with the displayname/username/phone number + password.
It will require a server update.

@ylecollen
Is this is the same integration in web too? Is there any other auth available in web?

right, it is the same.
If the registration params are not supported, a webview is opened.


No support for readDir?

Great module, but i'm not seeing files when I read dir. I tried some stuff like
prepareVirtualFile(path, undefined, { isDirectory: true )

Thank you for your interest in this module!
Yes readDir is not supported yet.


iohook reported error on nwjs: Uncaught Error: Module did not self-register.

When I installed the iohook-v0.2.0 module successfully in the nwjs SDK package using "npm install iohook -- save", but when I double-clicked to run nw.exe, the console reported the following error:
Uncaught Error: Module did not self-register.
at Object.Module._extensions..node (internal/modules/cjs/loader.js:749:18)
at Module.load (internal/modules/cjs/loader.js:613:32)
at tryModuleLoad (internal/modules/cjs/loader.js:552:12)
at Function.Module._load (internal/modules/cjs/loader.js:544:3)
at Module.require (internal/modules/cjs/loader.js:650:17)
at require (internal/modules/cjs/helpers.js:20:18)
at Object. (C:\E\nwjs-sdk-v0.30.5-win-x64 - iohook-test2\node_modules\iohook\index.js:16:21)
at Module._compile (internal/modules/cjs/loader.js:702:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:719:10)
at Module.load (internal/modules/cjs/loader.js:613:32)
My expectation is that when I run nw.exe, iohook will work and output my keyboard and mouse behavior as required.
But after running the console reported the above error and iohook could not be used.
The following is a screenshot of my code, including the error output after running:

This problem makes me unable to use iohook, and the next work can not be carried out. I've spent two weeks looking for solutions online, and the results have not changed. I hope you can offer some help.
My running environment:
nw.js:0.30.2/0.30.5/0.33.2 (all three versions have tried and reported the same error).
node.js:10.0.0;
npm:5.6.0;
yarn:1.9.4 (also tried to use "yarn add iohook", but still reported the same error).
Win10-64;
Thank you!

Hi there, in order to use it with NW, you will have to compile iohook yourself for this target. Sadly I can't help you with that, I don't know the corresponding target.

I am sorry that I failed to reply to you in time because I had a business trip some time ago.
This is my first contact with nwjs, so how do I compile it?
Do I need to upload the project files to you?

Closing this as this was related to a previous, outdated iohook version. Feel free to reopen it.


npm cache does not speed up npm install

Description:
Consider the following builds:

3rd build without npm cache
3rd build with npm cache

These two repositories are almost identical, with the only difference being the latter repository caches npm via the setup-node GitHub Action, whereas the former one does not. In other words, the only difference between the repositories is at the .github/workflows/main file:
name: Build Pipeline
on: push

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '11'
          # Following line is present only on the latter repository
          cache: 'npm'
      - run: npm install
      - run: npm run build

Although the build at "setup-node-with-cache" successfully uses npm cache (as evident by the output of the Run actions/setup-node@v2 step), run time of Run npm install step is almost the same as the corresponding step of the build at "setup-node-without-cache".
Isn’t the run time of Run npm install step of the build at "setup-node-with-cache" supposed to be significantly shorter than the corresponding step of the build at "setup-node-without-cache", since it is supposed to use the cached npm packages? Am I missing something here?
Action version:
2
Platform:
Ubuntu
Runner type:
Hosted
Tools version:
Node 11 and default npm that comes with it.
Repro steps:

3rd build without npm cache
3rd build with npm cache

Expected behavior:
The npm install step of the build that uses the cached npm to be quicker.
Actual behavior:
The npm install step of the build that uses the cached npm is (virtually) the same as the non-cached version.

Another question is, what is the point of including package-lock.json in the cache key? That is, as far as I can tell, if package-lock.json changes, the cache will be invalid and hence, all packages will be fetched over the network from scratch.
However, I would say this is pointless, since if package-lock.json changes between builds, most likely only one or two new packages are added or removed, compared to the previous build. In other words, most of the packages from the previous build are still being used, even though package-lock.json is changed. Hence, the cache would actually still be useful even if package-lock.json is changed. However, since the cache will be invalidated (since it depends on package-lock.json), we won't be able to benefit from it.
So, what's the point of including package-lock.json in the cache key?

Hello @ugultopu. Thank you for your report. The action setup-node caches saves cache from the global cache directory. The benefit from the caching comes for medium and large size dependencies. Besides, some dependencies are built in runtime, that is why it takes more time.
We use a package-lock.json file, because it contains the exact versions of all dependencies. If some dependencies change the cache can be outdated, that is why we use package-lock.json to track it. Moreover, we use package-lock.json file because it is of the mostly used package's tree dependency file. You can override this behaviour with cache-dependency-path. We have a feature request with adding restore keys to the action.
For now I'm going to close the issue, because it's an expected behaviour. If you have any concerns feel free to ping us.

Hi @dmitry-shibanov ,
i used  - name: Get npm cache directory
id: npm-cache-dir
run: |
echo "::set-output name=dir::$(npm config get cache)"
- uses: actions/cache@v3
id: npm-cache # use this to check for cache-hit ==> if: steps.npm-cache.outputs.cache-hit != 'true'
with:
path: ${{ steps.npm-cache-dir.outputs.dir }}
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: install dependencies
run: npm install
  - name: build app
    run: npm run build --if-present
    env:
      CI: ''

this file however cache has been stored in action cache but after i run pipeline it is install dependencies from scratch it gets 1 minute and doesnt use stored cache.

please can you explain how we can use stored cache for future runs?


Suggested change

Just having a play with this and noticed an issue which should be relatively easy to work around, but thought I'd mention it here before offering a pull request in case there's a reason for the current design...
The docs suggest this as a way to mark that the index page has been visited @mark('index:visited', current_user.id) but if the user is not logged in then current_user doesn't have an id attribute.
Might it be better to pass just current_user and let flask-bitmapist decide whether to act on it if it finds an id attribute.
(As an aside, is it possible to make this work with an authenticated redis? I've been trying to pass the password in using the BITMAPIST_REDIS_URL but without any luck)

@drcongo thanks. That sounds like a good idea. If you want to issue a PR for it that would be great!
I've created a separate issue for the authenticated redis bug (#11).


Implement table.insert and table.remove

Implement table.insert and table.remove, supporting metamethods
For plain tables (no index/newindex/len metamethods), optimize inserts and removes that work entirely within the array portion of the table; this is the normal use-case of table.insert and table.remove, so it should be worth the minor added complexity.

This now includes a reasonable set of tests, though the edge cases of the optimized case may be difficult to trigger explicitly.

This is really good work, thank you so much!
We talked about this a bit in chat, but there's some API changes I want to make to the raw table portion. Mostly, I don't want RawTable to have an API which only makes sense to use as part of the stdlib, so I'd like to change it to be more useful generally.. I'd like it to go ahead and correctly insert / remove and shift even when there are indexes in the map portion of the table.
This change is very good though, so I want to go ahead and merge it, and those changes can be done in a separate commit.
Thank you again!


ACCUMULO-4310 - User manual should mention accumulo-site.xml property…

… in instructions to enable native maps

+1

+1

Test comment

Test comment

Test comment

BTW, it's possible to give a +1 with the icon to "Add your reaction", rather than create separate comments. It keeps things nice and rolled up, instead of potentially long-scrolling +1 vertical columns.

Test comment

@mikewalch  what branch did you intend this to go in?

master and 1.8 should be sufficient but adding it to more (like 1.7) would be great.

1.7 and later use the same mechanism to build the docs, so it should be trivial to add it to all 3 of those. 1.6 is a bit annoying, because of the docs using a LaTeX build.


Driver isn't being installed reliably on Windows

I haven't tested on a wide range of machines or anything, but I believe Wix is going to be far more robust in the area of driver installation. Whereas we had NSIS running an external utility to install the driver, and then failing to reliably show its output, Wix has driver installation built-in natively, and can tell you exactly why the driver installation failed if that happens.
I have a huge part of the work to moving to Wix done already, which can be seen here (comparison). The installer I built from that repo succeeded in installing the driver in a VM where the NSIS installer failed to do so. Still lots to be done though, like a proper installation GUI and possibly fixing the build for other platforms.
Wix is more complicated than NSIS. It is a wrapper for the Windows Installer system, and so it produces .msi files rather than .exe files (which can be wrapped using its bootsrapper tool Burn). Unlike NSIS installers, which executes a series of steps, .msi files tell Windows "Here is precisely what I want everything to look like. You handle it." This results in a system where you have a huge XML configuration file where you have to give every single file a globally unique ID, and tell it exactly where it should end up in the file system. You also have to be careful about managing the GUIDs between updates. In exchange, you get flexibility, better reliability, and better integration with Windows, allowing for things like automated repair and shared libraries between applications. Wix is very stable, Microsoft Office even uses it for its installer.
I'm looking forward to hearing what you guys think about the possibility of switching, as well as the code I've written in that branch.

I haven't had a chance to review that code, but I did want to say that I'm totally open to moving to Wix. If managing the GUID's is tricky I think we should have some documentation about how to do that.

I'll make sure to write some, after I figure out the intricacies myself :P

The other thing that will be tricky about maintaining the installer is that nearly all of the files it is copying belong to Electron, not our code (I configured electron-packager to put almost all of our code in a single .asar file to make installer maintenance as easy as possible). What we should probably do is lock down the electron version, and update it manually, verifying that new versions don't add or remove any new files.

Initial UI is implemented. You can try it out here: https://github.com/NoahAndrews/qmk_firmware_flasher/releases/tag/v0.6.0

You can see a list of things that I know need to happen with the new installer here: https://github.com/NoahAndrews/qmk_firmware_flasher/issues.
There's a fair amount of research about WiX in there too. There's a lot to learn.


[type: refactor] shade for opentelemetry plugin in agent

Codecov Report

Merging #2748 (df482e3) into master (fe6f340) will decrease coverage by 0.10%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2748      +/-   ##
============================================
- Coverage     60.18%   60.08%   -0.11%
+ Complexity     4408     4402       -6
============================================
  Files           681      681
  Lines         19613    19613
  Branches       1743     1743
============================================
- Hits          11805    11785      -20
- Misses         6814     6829      +15
- Partials        994      999       +5

Impacted Files
Coverage Δ

...yu/sync/data/http/refresh/AbstractDataRefresh.java
70.00% <0.00%> (-13.34%)
:arrow_down:

...apache/shenyu/admin/utils/ShenyuDictH2Trigger.java
63.63% <0.00%> (-9.10%)
:arrow_down:

...ache/shenyu/admin/utils/PluginHandleH2Trigger.java
63.63% <0.00%> (-9.10%)
:arrow_down:

...che/shenyu/sync/data/http/HttpSyncDataService.java
77.94% <0.00%> (-5.89%)
:arrow_down:

...henyu/plugin/grpc/resolver/ShenyuNameResolver.java
58.51% <0.00%> (-5.32%)
:arrow_down:

...henyu/admin/service/impl/UpstreamCheckService.java
63.49% <0.00%> (-0.80%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe6f340...df482e3. Read the comment docs.


About the testing of novel views and novel projections

Hi, thanks for the great work. I am very interested in this repository and the paper. However, I could not find details in the repository concerning the division of the training and test sets, as well as the metrics calculated for the test set. Could you please provide the images and camera parameters corresponding to the 36 novel views and novel projections described in the paper?  Wish your reply!

Hi,

division of the training and test sets

I was slightly panicking initially because I didn't find this split anywhere in the data, but then I realized it was hard coded in the code here:
https://github.com/yoterel/nepmap/blob/1c8557add2140275a3de8cbaa3d0c06040e7fdef/train.py#L323
Please manually exclude these indices from training (that is what I did), and use them for testing.
Note that indices start from 0.
Just as an example, if index 74 appears, you would just go to "0074.png" and that would be an image belonging to the test set.
To find the corresponding novel projection, you can use the file called "img2tex.json" which maps each image to a projection image in the projectors folder (the file can be found for each scene).

metrics calculated for the test set
The paper shows the PSNR metric, which is computed using standard formula (see page 6 of paper, last paragraph):

$$10 \cdot log (\frac{max(Image)}{Loss_{img}})$$
where $Loss_{img}$ is the standard L2 loss over all the pixels (see equation (4) in the paper)

Could you please provide the images and camera parameters corresponding to the 36 novel views

Yes, they are right there in the dataset folder, see the file called "transforms.json", which exists for every scene. (go to the entries corresponding to the image in the test set to find the camera info in blender format or opencv format). For example, if the index is 74, go to "0074.png" in transforms.json, and extract the camera info.
I apologize this isn't automated. If you by any chance write code to do this programmatically feel free to submit a pull request I would appreciate that !

closing issue, please reopen if the issue persists.

Thank you. But I feel confused here:
_https://github.com/yoterel/nepmap/blob/1c8557add2140275a3de8cbaa3d0c06040e7fdef/train.py#L57_
I noticed that the images used for testing seem to be read from another folder named xxx_random. I guess maybe the test images are only read from such folders？Based on this situation, is it still necessary to manually exclude those frames used for testing from the training set?

You are absolutely right that the xxx_random were used and not the original, which I forgot about.
Sorry for the confusion. You can forget what I said about excluding them from the training set.


Add enabled_metrics attributes to autoscaling_group

PR o'clock
Description
Added enabled_metrics attribute to the worker groups configuration to optionally enable the collection of CloudWatch metrics.
Documentation: https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#enabled_metrics
Checklist

[x] terraform fmt and terraform validate both work from the root and examples/eks_test_fixture directories (look in CI for an example)
[ ] Tests for the changes have been added and passing (for bug fixes/features)
[ ] Test results are pasted in this PR (in lieu of CI)
[x] I've added my change to CHANGELOG.md
[x] Any breaking changes are highlighted above

Can smth similar be done to node groups?


Form-data with file upload?

I'm trying to upload a file with form-data, but I can't find the correct way to do it. This is what it looks like in Retrofit:
MultipartBody.Part.createFormData("files[]", fileName, requestBody);

Is there a way in Fuel to do this?

You can do with source like this
.upload("/post")
.source { request, url ->
   File(currentDir, fileName)
}
.name { "files" }
.responseString { req, res, result -> }

Hope this answer your question :)

Sadly that doesn't fix my issue, the form data is still not set

I think you can do .upload("/post, params=YourFormData)
Put your params in the List<Pair<String, Any>> format. I may be mistaken though if this works without a file

Update: You can do this, see my response in #165

Closing as inactive, please reopen if you need more help on this question.

I tried this, but the resulting multipart body still collect separate part for each file. The name of the part is "files1", "files2", etc.
What I need (and seams the originator of this topic) the multipart with single part named "files" and containing the list of files.
How can we do THAT?


[RHOAIENG-12577] - enable modelCar by default

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Type of changes
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Feature/Issue validation/testing:
Please describe the tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

[ ] Test A

[ ] Test B

Logs

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

[ ] Have you added unit/e2e tests that prove your fix is effective or that this feature works?
[ ] Has code been commented, particularly in hard-to-understand areas?
[ ] Have you made corresponding changes to the documentation?

Release note:

Re-running failed tests

/rerun-all - rerun all failed workflows.
/rerun-workflow <workflow name> - rerun a specific failed workflow. Only one workflow name can be specified. Multiple /rerun-workflow commands are allowed per comment.

Validated with devFlag pointing to the PR,s branch:
    kserve:
      devFlags:
        manifests:
          - contextDir: config
            sourcePath: ''
            uri: 'https://github.com/spolti/kserve/tarball/RHOAIENG-12577'


Fix interactive crop in latest mpv

[tmpwmhndbqq] Lua error: /tmp/tmpwmhndbqq.lua:134: attempt to call field 'get_osd_resolution' (a nil value)

Getting the same