Add Scalar / Datum support to compute kernels

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
When implementing analytics, users often want to do operations like array + constant or array + array
The Rust implementation of Arrow is strongly typed 🙌  , including the compute kernels, but often data is passed around in Arc<dyn Array> / ArrayRef so the types can be dynamic. This then places the burden on the user of the library to determine the exact types of the inputs in order to call the appropriate kernels
Let's take for example, trying to compare an array to find all elements that contain the number 5 and your input an array may have Int64 or Int32 values
In the current version of arrow, in order to do so you would need to write code like
use arrow::compute::eq_scalar;

fn find_5(array: ArrayRef) -> Result<BooleanArray>{
  match array.data_type() {
    DataType::Int64 => eq_scalar(array.as_any().downcast_ref::<Int64Array>().unwrap(), 5),
    DataType::UInt64 => eq_scalar(array.as_any().downcast_ref::<UInt64Array>().unwrap(), 5),
    ...
  }
}

This ends up being macroized and is non ideal because the user had to do dynamic type dispatch anyways, so there is no runtime performance benefit to strongly typed kernels
Describe the solution you'd like
It would be nice to be able to call a single function and let the rust library dynamically (at runtime) pick the correct kernel to call.
As described and suggested by @jorgecarleitao  and @nevi-me  in https://github.com/apache/arrow-rs/pull/984#issuecomment-994093090 this could go all the way and take the form of following the C++ Datum model
fn eq(left: Datum, right: Datum) -> BooleanArray {
...
}

Where Datum looks something like
enum Datum {
  Scalar(ScalarValue),
  Array(ArrayRef)
}

And ScalarValue which would look something like ScalarValue from DataFusion https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/scalar.rs or Scalar from arrow2 https://github.com/jorgecarleitao/arrow2/tree/main/src/scalar
Describe alternatives you've considered
The alternative is a continued proliferation of individual kernels such as eq, eq_utf8, etc
Additional context
There is a lot of additional discussion on https://github.com/apache/arrow-rs/pull/984
cc @matthewmturner @Dandandan @Jimexist @houqp

I support adding higher level easy to use API for non-performance critical code, but it would be good to still keep the specialized kernels around so users can still leverage them when type info is available at the call site.

but it would be good to still keep the specialized kernels around so users can still leverage them when type info is available at the call site.

The reason I was saying the specialized kernels might not be totally useful was,  my mind, that  the overhead of a type dispatch could be almost entirely amortized -- like if we are comparing 1000 elements in an array, an extra check on the array's type might not be a huge deal
I don't see anything wrong with leaving all the specialized kernels available too (maybe it would help inlining 🤔 )

I believe this overlaps with https://github.com/apache/arrow-rs/issues/2842

So I've been playing around with this and the major challenge is avoiding a huge amount of API churn / boilerplate
Take the signature
add_dyn(a: &dyn Array, b: &dyn Array) -> Result<ArrayRef>
Its not clear how to convert this to a Datum based model. One option would be
add_dyn(a: Datum<'_>, b: Datum<'_>) -> Result<ArrayRef>

Where Datum is something like
enum Datum<'a> {
    Array(&'a dyn Array),
    Scalar(&'a dyn Scalar)
}

But this has a couple of issues

Callsites now have to explicitly wrap there arguments in Datum
There is no way to return a scalar

Making Datum a trait doesn't help here either, because the specialization rules prevent blanked implementations for both T: Scalar and T: Array.
Another option would be to make the methods generic, with impl Into<Datum>, but this also has downsides of

Runs into same blanket impl issues as deriving Datum trait
Kernels resulting in significant additional codegen

Taking a step back I had a potentially controversial thought, why not just treat a single element array as a scalar array?
This would have some pretty compelling advantages:

No changes to type signatures necessary
Unary kernels like casting just work with no modification
Complete type coverage for no effort

The obvious downside is the representation is not very memory efficient. I think the question boils down to what is the purpose of the scalar representation, is it:

To allow more efficient kernels where one side is known to be a scalar, e.g. scalar comparison, etc...
Provide an efficient type-erased representation for row-oriented operations like grouping
Provide efficient scalar operations

My 2 cents is that 2. is a use-case better served by the row representation, and 3. is beyond the scope of a vectorized execution engine, and therefore 1. is the target for this feature. As such I think this is perfectly acceptable approach. The overheads of the slightly less efficient representation will be more than outweighed by the costs of the dynamic dispatch alone.
What do people think?

Taking a step back I had a potentially controversial thought, why not just treat a single element array as a scalar array?

For what it is worth I think this is what DuckDB does (at least this is how I interpret this slide from the  22 - DuckDB Internals (CMU Advanced Databases / Spring 2023) lecture

I think the biggest potential downside of the "use a single row to mean a scalar value" could be that it is confusing as an API. The two input array sizes have to match except when one of them (would it always be the right argument?) had a single row, in which case it would have a special fast path 🤔 But it wouldn't be clear from the call signature if there was a special fast path or not.
enum Datum<'a> {
    Array(&'a dyn Array),
    Scalar(&'a dyn Scalar)
}
``

What if we made a 'new' set of kernels (that just dispatched to the existing ones, if they existed)
This seems like the ideal user experience from my perspective:
let scalar = Int64Scalar::new(42);
let array = Int64Array::from(....);

// Call the "add" kernels in a new module
let result: Datum = arrow::compute::datum::add(&scalar, &array);

Maybe the types could look like this:
enum Datum<'a> {
    Array(&'a dyn Array),
    Scalar(&'a dyn Scalar),
    OwnedArray(Arc<Array>),
    OwnedSclar(Arc<Scalar>),
}

impl Into Datum<..> for Int64Scalar {
..
}

fn add (left: impl Into<Datum>, right: impl Into<Datum>) -> Datum<'static> {
..
}

Perhaps?

dispatched to the existing ones

The existing ones don't support scalars with logical types, that's the whole issue 😅. We have to change the way we support scalars, and I'm leaning towards the one that doesn't involve bifurcating the API...

except when one

There aren't a huge number of binary kernels, so I think we could get pretty good coverage quite quickly.
Is this not something that could be overcome with a bit of documentation, from the UX side I can see an argument that having to treat scalars differently is actually a worse UX...

This runs into the blanket impl issue, there isn't a way to provide blanket impls for the concrete types which then forces peculiar things like (&array as &dyn Array)

I was imagining implementing impl Into<Datum> explicitly for all the concrete types...

For what it is worth I think this is what DuckDB does (at least this is how I interpret this slide from the 22 - DuckDB Internals (CMU Advanced Databases / Spring 2023) lecture

If the array is already in Flat format, it needs to construct an extra Selection vector to iterate the array, maybe it's a little overhead than iterating the array itself.
We used an enum to represent the vector, it's hard to add support for another dictionary vector.
#[derive(Debug, Clone, PartialEq, EnumAsInner)]
pub enum ValueRef<'a, T: ValueType> {
    Scalar(T::ScalarRef<'a>),
    Column(T::Column),
}

The main drawback is code flood, so we use the code generator to help write vectorized methods, the generated file  is ~6000 LOC .


locations_to_reach with worldname containing space

(Required) What is your Quests version / build number (do not say "latest")?
3.05
(Required) Please describe the problem and how to replicate it. Any console errors?
World Name contains a space, causes a parsing error.
Our world name is "Towns 1" with a space.    Adding a location to reach of 126.0, 64.0, 98.0 causes a parsing error.
loccations-to-reach:
- Towns 1 126.0 64.0 98.0
Reloading quests gives the following error:
[20:19:48 ERROR]: [Quests] Towns 1 126.0 64.0 98.0 inside mobs-to-kill: inside Stage 1 of Quest Meet Mag is not in proper location format!
Note; I also filed the bug report for creating events with mob spawners crashing while trying to setup spawn locations, it maybe the same issue as its the same world.

The fix for #100 should have resolved this issue. Hope to take a look soon

Related to #124
Fixed in 1b2eb65a105a34ec18db75b964f8b0fcd4258d7e


Dragging items in inspector causes scrollbar drag issues (4.0)

Godot version:
v4.0.dev.custom_build.41e902886
hash:41e9028868e49669de83c82ba20fd9dcef0d1b8b
OS/device including version:
Windows 10
Issue description:
When you have enough items in the Filesystem, if you try to drag a up or down, the scrollbar will glitch.
For example:
-the scrollbar starts dragging with it the item
-the scrollbar resets its position each time you move the mouse (while pressed)
-when you release the item the scrollbar keeps following the mouse (even though it's not pressed anymore)
Steps to reproduce
Create enough items in the filesystem so that the scrollbar appears.
Then drag items up and down in FileSystem (enough so that the scroll bar has to move)
This project has enough items so the scrollbar appears:
Drag.zip
https://user-images.githubusercontent.com/47060866/103465635-cfba3c80-4d0b-11eb-8b37-0f8d7e2888a9.mp4

Duplicate of https://github.com/godotengine/godot/issues/38665.

Duplicate of https://github.com/godotengine/godot/issues/38665.


NumberFormatException after failed checkout

Hello, we have a few crashes with the following ST:
Caused by java.lang.NumberFormatException: s == null
       at java.lang.Integer.parseInt(Integer.java:577)
       at java.lang.Integer.valueOf(Integer.java:801)
       at com.sumup.merchant.controllers.CheckoutFlowController.sendCancelCheckout(SourceFile:184)
       at java.lang.reflect.Method.invoke(Method.java)
       at org.greenrobot.eventbus.EventBus.invokeSubscriber(EventBus.java:507)
       at org.greenrobot.eventbus.EventBus.postToSubscription(EventBus.java:434)
       at org.greenrobot.eventbus.EventBus.postSingleEventForEventType(EventBus.java:411)
       at org.greenrobot.eventbus.EventBus.postSingleEvent(EventBus.java:384)
       at org.greenrobot.eventbus.EventBus.post(EventBus.java:265)
       at com.sumup.merchant.ui.Fragments.MobilePaymentFragment.sendCancelCheckoutRequest(SourceFile:347)
       at com.sumup.merchant.ui.Fragments.MobilePaymentFragment.access$1500(SourceFile:84)
       at com.sumup.merchant.ui.Fragments.MobilePaymentFragment$2.onClick(SourceFile:560)
       at androidx.appcompat.app.AlertController$ButtonHandler.handleMessage(AlertController.java:167)
       at android.os.Handler.dispatchMessage(Handler.java:107)
       at android.os.Looper.loop(Looper.java:237)
       at android.app.ActivityThread.main(ActivityThread.java:7948)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1075)

From our logs this crash seems to happen after retrying a previously failed payment. In all our crashes we clearly see that there are two attempt to start a payment before this exception is thrown.
SDK Version: 3.2.2
Android Version: we are receiving crashes on Android 9 & 10

@danielebart
I will make a ticket for our developers and come back to you with an update soon.

@Mike-Lopez Hey, have it been already fixed? Can you update us with a progress on this matter?

Hello,
My apologies for the late reply.

Please provide me with your merchant ID or registered email address
with SumUp?

Once I get this information, I will look further into the issue and get
back to you.
Best,
Mike
On Mon, Sep 21, 2020 at 7:25 AM Milszym<EMAIL_ADDRESS>wrote:

@Mike-Lopez https://github.com/Mike-Lopez Hey, have it been already
fixed? Can you update us with a progress on this matter?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sumup/sumup-android-sdk/issues/134#issuecomment-695906118,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APYALARLJZEA3LNJCVUWNLLSG3PTVANCNFSM4OSUJVCA
.

--
Michael Lopez
Technical Implementation Associate
Sales and Partners

@Mike-Lopez, Hey, thanks for the reply. I'd rather send it via email. Can you give me your email please?

Please email<EMAIL_ADDRESS>

Ok sure, thanks I have emailed you with the data that you have asked for.


External apps integration

We should rework the current approach so that the browser extension integration could've been possible in a more secure and easy way.
We need to add one request:

/auth -- authenticates the extension or the external app. All other requests must be sent with an auth token received.

Other requests are basically the same as what we have in the browser API.

How to make it compatible with service workers ? Tokens will not be valid after restart.

Well, the extension will have to store token in the browser storage


Revisit the indexes.json file format to see if the format can be simplified or if can use an existing golang parser.

@themandalore mentioned that this is the format that other projects use this format(Chainlink....) so on the plus side this should mean that people are familiar with this format. Lets see if can use some existing golang module to remove the need for a custom parser and sync with some user of the miner to agree on the format.
@mikeghen maybe you can also comment?
		"json(https://api.binance.com/api/v1/klines?symbol=BTCUSDT&interval=1d&limit=1).0.4",

could maybe be something like:
"URL": "https://api.binance.com/api/v1/klines?symbol=BTCUSDT&interval=1d&limit=1"
"type": "json",
"jsonPath":".0.4"

The format seems to be similar to https://goessner.net/articles/JsonPath/

ducpliate of https://github.com/tellor-io/telliot/issues/252 so will continue the work these.


Fix detector display by replacing println with print

This fixes the following display:
julia> using ADTypes, SparseConnectivityTracer

julia> backend = AutoSparse(AutoForwardDiff(), sparsity_detector=TracerSparsityDetector())
AutoSparse(dense_ad=AutoForwardDiff(), sparsity_detector=TracerSparsityDetector()
)

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 36.62%. Comparing base (702b2d0) to head (f7290a0).

Files with missing lines
Patch %
Lines

src/adtypes_interface.jl
0.00%
2 Missing :warning:

:exclamation:  There is a different number of reports uploaded between BASE (702b2d0) and HEAD (f7290a0). Click for more details.
HEAD has 3 uploads less than BASE

Flag
BASE (702b2d0)
HEAD (f7290a0)

4
1

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #201       +/-   ##
===========================================
- Coverage   91.05%   36.62%   -54.44%
===========================================
  Files          47       46        -1
  Lines        2080     1794      -286
===========================================
- Hits         1894      657     -1237
- Misses        186     1137      +951

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Thanks!


Support "selected" state icons in TabBarIOS.Item

TabBarIOS doesn't yet support showing a different icon when its tabs are selected. Is anyone working on this? I might work on this in the near future otherwise.

It does support it, selectedIcon prop. I just documented it much better, going out in the next sync

You're right. I was looking at propTypes and it wasn't there but I see it in render().

Yeah, this component was in dire need of cleanup/documentation


internal errors always rendered in console, notification is opt-out

motivated by #151.
Also change the recently introduced Errors To Console option to an opt-out setting for notifications for those kinds of errors.
julia side: https://github.com/JunoLab/Atom.jl/pull/34
cc: @dlfivefifty

Ok, thinking about it, this change seems reasonable. But just as a note, we probably shouldn't focus too much on streamlining the Internal Error experience, as opposed to preventing them ;)
In this case we really need to catch display errors and treat them as user errors (I just keep putting it off).

Ye, sure, but nicer backtraces are valuable for us as well :)
Also, I'm not sure how or even if we can distinguish between internal and user errors in all possible cases.

Fair point.
You can catch all user errors if you define them as "errors that occur during user-triggered eval/display". You might be thinking of e.g. a bug in Gadfly display as an internal error, but I just define us as users of our own system in that case.

I'll merge this then, unless you have any objections?

Please do


CSS var: stepped process

CSS var for stepped process

@louismaximepiton Reverted the creation of --#{$prefix}stepped-process-max-items via https://github.com/Orange-OpenSource/Orange-Boosted-Bootstrap/pull/1281/commits/c94be7ba24d8132e1da58170a90e0c1659a5c602 because of @for $i from 1 through $stepped-process-max-items {. Indeed you won't be able to change the behavior of this loop by changing only the CSS var and having the value stored in this CSS var doesn't seem very useful in terms of reuse. We can discuss about that when you're back if I missed something and you want it to be reintroduced.


codec/color: fix segfault in freerdp_image32_copy

Code path for dstBytesPerPixel == 3 moved src and dst pointers beyond their respective buffers.

+1


How do I use this?

This looks really cool, but how do I install it?

You download the latest release, "the file name is Custom.Gmod.Bot.zip."
Then you go to your Gary's mod directory and unzip the file in you addons folder
And that's it.
I'm working on putting this addon on the steam workshop because of how confusing it is to install the addon right now.

Thank you


IS_METRIC

IS_METRIC
Modified speed using IS_METRIC
CSLCSpeed
put/get the params_memory CSLCSpeed as float, using unit MS instead of MPH
speedSetPoint
In create_gm_acc_spam_command_ms:
When MPH checking value are: 20MPH & 16MPH
When KPH checking value are: 28KPH & 24MPH (tested in Baby Enclave)
Known Issue
The cruise speed on the dashboard is dancing between 40-45KPH when using MPH, 38-39KPH when using KPH. Might be related to the rounding difference.

Used in https://github.com/garrettpall/openpilot/commit/b4ecb39eaee40bf284b0d55c98ef02bf9087f54f


UniFi - Handle wired bug on restart

Description:
On restart wired bug timer will be set meaning something being affected by wired bug might erroneously get marked as home.
Related issue (if applicable): fixes #30162
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist

If the code communicates with devices, web services, or third-party tools:

[x] The manifest file has all fields filled out correctly. Update and include derived files by running python3 -m script.hassfest.
[x] New or updated dependencies have been added to requirements_all.txt by running python3 -m script.gen_requirements_all.
[x] Untested files have been added to .coveragerc.

If the code does not interact with devices:

[x] Tests have been added to verify that the new code works.

Test failure is a flakey test -- merging!

Thanks @andrewsayre


feat: Replace serde_json with sonic_rs for poem crate

Refactored poem crate, replaced at some places usage of serde_json with the more faster json serde crate sonic_rs.
I've locally benchmarked a simple poem service like this
use poem::{handler, listener::TcpListener, post, web::Json, Route, Server};

#[handler]
fn echo(val: Json<serde_json::Value>) -> Json<serde_json::Value> {
    val
}

#[tokio::main]
async fn main() -> Result<(), std::io::Error> {
    let app = Route::new().at("/echo", post(echo));
    Server::new(TcpListener::bind("<IP_ADDRESS>:3000"))
      .run(app)
      .await
}

With the help of locust I've got a 20% decrease of response time, with the following benchmark.
from locust import FastHttpUser, task
import json

# Opening JSON file
f = open('large-file.json')
data = json.load(f)

class HelloWorldUser(FastHttpUser):
    @task
    def hello_world(self):
        self.client.post(
            "/echo",
            json=data
        )

large-file.json
Issue
https://github.com/poem-web/poem/issues/813

How about be a feature?
Someone may not care about the performance.

@attila-lin updated it with adding a new feature flag sonic-rs.
Should I also put some notes in somewhere in docs about this new feature ?

@attila-lin
So sorry for the delay, forgot about this PR.
I've updated corresponded README.md files and rust docs.

I'd like to merge this. Any suggestions? @sunli829

does sonic-rs only work with nightly Rust?

@sunli829 no, it works on stable one

error[E0658]: use of unstable library feature 'result_option_inspect'
  --> /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/sonic-rs-0.3.12/src/reader.rs:73:21
   |
73 |         self.peek().inspect(|_| {
   |                     ^^^^^^^
   |
   = note: see issue #91345 <https://github.com/rust-lang/rust/issues/91345> for more information


Call wifi.sta.config call with a table instead of two strings

Seems that the api has been updated since this script was last updated.
See https://nodemcu.readthedocs.io/en/master/en/modules/wifi/#wifistaconfig

I can confirm this, should be merged.


How to generate the interaction picture after scaffolding?

Hi,
How to generate the interaction picture after scaffolding?
I want to ask is there file/files in the output available to make the interaction picture? Or do I need to align the HiC reads to the scaffold and then make the picture by my self?

Hi,
Currently, we don't have a code to generate the interaction heatmap. You can use juicer to align reads to the scaffolds and then use the .hic file generated by juicer to visualize the heatmap in juicebox.

It's Ok. Thank you!


fix: signing public keys

Issue being fixed or feature implemented
There was an issue with signing public keys that has now been resolved.
What was done?

How Has This Been Tested?

Breaking Changes

Checklist:

[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have added or updated relevant unit/integration/functional/e2e tests
[X] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[ ] I have assigned this pull request to a milestone

What was the issue?


Any way to use Option[Ping]?

I'm trying to write a handler that can take an option, ie receive either a json string representing Ping, or a null/blank (which I'm actually going to use in a scheduler to fire every minute just to keep the lambda warm.) I have:
class PingPongHandler extends Lambda[Option[Ping], Pong] { ...
When I invoke with { "inputMsg": "whatever" }, it's happy, and the handler receives a Some. However, I can't figure out what to invoke it with to generate a None. I've tried passing null which is how circe encodes a None, but that just gives circe decoding failure. So does "null", "", {} etc. Is there some reason that options woudn't work with this library? Thanks!

Hm... I think it should work fine. In fact, I've just added a test (482a992), which checks that.

Ok, so I've implemented a test case similar to yours and it passes the tests fine. But these scalatests are working by running PingPongHandler.handle in the local development JVM. The problem I was having is after deployment to aws. I was trying to test it with
serverless invoke -f pingPongOptionFunc -d "null"
(from powershell serverless cli) and that gave the error:
{
    "errorMessage": "Attempt to decode value on failed cursor: DownField(inputMsg)",
    "errorType": "io.circe.DecodingFailure$$anon$2",
    "stackTrace": []
}

  Error --------------------------------------------------

  Invoked function failed

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     win32
     Node Version:           6.11.2
     Serverless Version:     1.26.1
`

I also tried putting the string null in a file and using --path instead but that gave the same result. Same with sticking in extra quotes. However, if I invoke it on a schedule from serverless.yaml using this:
functions:
  pingPongOptionFunc:
    handler: handler.PingPongOptionHandler::handle
    events:
      - schedule:
          rate: rate(1 minute)
          input: "null"
          enabled: true

then I can check the logs and it appears to be behaving correctly. (And this is my actual use case.)
So perhaps this behavior is due to the way the serverless cli handles the string "null"., in which case it's probably outside the responsibility of the aws-lambda-scala library.

Yes, you're right, it seems to work perfectly from aws cli. It was just the serverless "feature" turning null -> {} that was throwing me off. Thanks.


Add a markdown document for wer

store in athena/doc/wer_are_we
we can reference to https://github.com/syhw/wer_are_we, but we add more information related to Mandarin Chinese corpus and github opensource results other than paper

This benchmark will arise a more attractive thing around athena

@neneluo let's work together on this

@neneluo let's work together on this

Sure.


⚠ Jenkins Plugins Artifactory has degraded performance

In 05af994, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1014 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 2aa634c after 12 minutes.


Support for background images

Support background image on canvas for the options argument.

This looks promising. Would love to see this merged.

@ccampean -- Is there a reason you closed this request? Looks like a great feature.

Same question here @ccampean, I need this feature too


Complete Freestanding Bundles support in all platforms

Adds support for finding the bundle path for implicitly loaded bundles (for which we only know the executable path).
Unifies Windows and other platforms implementation.
Enables Freestanding Bundles tests.

(This simplifies and minimizes the changes proposed at #1570)

@swift-ci please test

@compnerd @millenomi
(In a follow-up PR I would merge this commit that includes the SPI commented in #2083)

@swift-ci test

@spevans Could you restart the tests? Thanks!

@swift-ci test

@millenomi This is be ready for review.

(Updated with bundle changes for Android from #2120)

@pvieito could you rebase & squash this to eliminate the merge commit, thanks

@spevans Done!

@swift-ci test

@swift-ci test

@millenomi can we merge this?

@parkera Who can/should review this? Thanks!

@pvieito would you like to resolve the conflict so that this could be reviewed? Thanks!

@MaxDesiatov Unfortunately, I have requested a review in several occasions for this PR with no response from the maintainers, so I am not longer updating it.

Well that's really unfortunate...

Ping @pvieito, mind rebasing now?

The Swift project moved the default branch to main and deleted master branch, so GitHub automatically closed the PR. Please re-create pull request with main branch.
More detail about the branch update - https://forums.swift.org/t/updating-branch-names/40412


Only one color in umap after adata.obs.sort_index(inplace=True)

Please make sure these conditions are met

[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of anndata.
[ ] (optional) I have confirmed this bug exists on the master branch of anndata.

Report
After I run adata.obs.sort_index(inplace=True) then,
sc.pp.highly_variable_genes(
    adata,
    n_top_genes=2500,
    subset=True,
    layer="counts",
    flavor="seurat_v3",
)
sc.tl.pca(adata)
sc.pp.neighbors(adata, n_pcs=30, n_neighbors=20)
sc.tl.umap(adata, min_dist=0.3)
sc.pl.umap(
    adata,
    color=["gse"],
    frameon=False,
    save='_10x_gse.png'
)

I get the following umap which only has a single color:

Versions
session_info        1.0.0
-----
argcomplete         NA
asttokens           NA
backcall            0.2.0
colorama            0.4.6
cython_runtime      NA
dateutil            2.8.2
decorator           5.1.1
exceptiongroup      1.1.3
executing           1.2.0
gmpy2               2.1.2
h5py                3.10.0
jedi                0.19.1
mpmath              1.3.0
natsort             8.4.0
numexpr             2.8.7
numpy               1.24.4
opt_einsum          v3.3.0
packaging           23.2
pandas              2.1.1
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
prompt_toolkit      3.0.39
ptyprocess          0.7.0
pure_eval           0.2.2
pygments            2.16.1
pytz                2023.3
scipy               1.11.3
six                 1.16.0
stack_data          0.6.2
sympy               1.12
torch               2.0.0
tqdm                4.66.1
traitlets           5.11.2
typing_extensions   NA
wcwidth             0.2.8
zoneinfo            NA
-----
Python 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:49:32) [GCC 12.3.0]
Linux-3.10.0-1160.108.1.el7.x86_64-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-28 09:04

Thanks for opening the report.
I don't think that you are actually only getting one color in your UMAP. I think that's the colors are now basically random because the obs dataframe is no longer aligned to the rest of the object. Your code is sorting obs, but leaving the rest of the object in the original order.
I think you should be doing something more like:
adata = adata[adata.obs_names.sort_values(), :].copy()

Does this resolve your issue?

Thanks. When I use adata = adata[adata.obs_names.sort_values(), :].copy() and I try to save it with adata.write_h5ad I get the following error:
ValueError: Output dtype not compatible with inputs.

Could you share the full traceback? Can you also share environment info with anndata imported?

/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/anndata/_core/anndata.py:1906: UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
  utils.warn_names_duplicates("obs")
/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/anndata/utils.py:260: UserWarning: Suffix used (-[0-9]+) to deduplicate index values may make index values difficult to interpret. There values with a similar suffixes in the index. Consider using a different delimiter by passing `join={delimiter}`Example key collisions generated by the make_index_unique algorithm: ['AAACGGGGTTCATGGT-1-1', 'AAAGCAAAGATGCCTT-1-1', 'AAAGCAAAGATGCCTT-1-2', 'AAAGCAAAGATGCCTT-1-3', 'AACACGTCAGCCTTGG-1-1']
  warnings.warn(
Traceback (most recent call last):
  File "/scratch/cs/csb/users/malonzm1/scripts/pan-autoimmune/scripts/scvi/combine_10x_0.py", line 17, in <module>
    adata = adata[adata.obs_names.sort_values(), :].copy()
  File "/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/anndata/_core/anndata.py", line 1592, in copy
    X=_subset(self._adata_ref.X, (self._oidx, self._vidx)).copy()
  File "/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/functools.py", line 888, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/anndata/_core/index.py", line 177, in _subset_spmatrix
    return a[subset_idx]
  File "/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/scipy/sparse/_index.py", line 78, in __getitem__
    return self._get_arrayXslice(row, col)
  File "/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/scipy/sparse/_csr.py", line 330, in _get_arrayXslice
    return self._major_index_fancy(row)._get_submatrix(minor=col)
  File "/scratch/work/malonzm1/.conda_envs/scvi-env/lib/python3.9/site-packages/scipy/sparse/_compressed.py", line 708, in _major_index_fancy
    csr_row_index(M, indices, self.indptr, self.indices, self.data,
ValueError: Output dtype not compatible with inputs.
srun: error: fn3: task 0: Exited with exit code 1
srun: launch/slurm: _step_signal: Terminating StepId=29260470.0

session_info        1.0.0
-----
argcomplete         NA
asttokens           NA
backcall            0.2.0
colorama            0.4.6
cython_runtime      NA
dateutil            2.8.2
decorator           5.1.1
exceptiongroup      1.1.3
executing           1.2.0
gmpy2               2.1.2
h5py                3.10.0
jedi                0.19.1
mpmath              1.3.0
natsort             8.4.0
numexpr             2.8.7
numpy               1.24.4
opt_einsum          v3.3.0
packaging           23.2
pandas              2.1.1
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
prompt_toolkit      3.0.39
ptyprocess          0.7.0
pure_eval           0.2.2
pygments            2.16.1
pytz                2023.3
scipy               1.11.3
six                 1.16.0
stack_data          0.6.2
sympy               1.12
torch               2.0.0
tqdm                4.66.1
traitlets           5.11.2
typing_extensions   NA
wcwidth             0.2.8
zoneinfo            NA
-----
Python 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:49:32) [GCC 12.3.0]
Linux-3.10.0-1160.108.1.el7.x86_64-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-05 04:17

When reporting your environment with session-info, anndata needs to be imported for it and its dependencies to show up. E.g. you need to run something like:
import anndata, session_info; session_info.show(html=False, dependencies=True)

I think this is a related to:

https://github.com/scipy/scipy/issues/20182

Which you can work around by making sure each sparse matrix has an indptr and indices of the same dtype. E.g.
X_int64_idx = adata.X.copy()
X_int64_idx.indptr  = X_int64_idx.indptr.astype(np.int64)
X_int64_idx.indices = X_int64_idx.indices.astype(np.int64)
adata.X = X_int64_idx

Any idea how you ended up with those arrays having a different dtype? What size is this array?

I did run import anndata, session_info; session_info.show(html=False, dependencies=True).
No idea. The array is 3428130 × 36601.

I did run import anndata, session_info; session_info.show(html=False, dependencies=True).

That's odd. It should be showing the version of anndata in the session info report. Could be worth reporting this to session-info.

Once you get over np.iinfo(np.int32).max (2,147,483,647) non-zero entries in your matrix, theintptr and indices of your array start needing to be 64 bit to accommodate all the values. Let me know if the work around I proposed fixed this issue for you.

Can you share the code and possibly h5ad?

Also you have to upcast the indices and indptr before doing the indexing. As you can see in the first line of your traceback (assuming this is the same)
Traceback (most recent call last):
  File "/scratch/cs/csb/users/malonzm1/scripts/pan-autoimmune/scripts/scvi/combine_10x_0.py", line 17, in <module>
    adata = adata[adata.obs_names.sort_values(), :].copy()

The error is coming from the subset operation, not the indexing operation.
So, the code should be more like:
# Upcast indices and indptr
X_int64_idx = adata.X.copy()
X_int64_idx.indptr  = X_int64_idx.indptr.astype(np.int64)
X_int64_idx.indices = X_int64_idx.indices.astype(np.int64)
adata.X = X_int64_idx

# Re-order
adata = adata[adata.obs_names.sort_values(), :].copy()

# Write
adata.write_h5ad(filename='temp.h5ad')

It worked. Thanks!


Fix notebook authentication

Makes gcloud authentication in Google notebooks more instructive for users.
Currently, if you are not authenticated locally, and try to use the colab authentication in Jupyter notebooks, they fail, as colab's auth is expected to run in a container, e.g. stores the credentials to / or wherever $DATA_LAB_ROOT is pointing to.

Keeps the colab based auth only in case colab is the execution environment. Otherwise it assumes gcloud based authentication ("advanced usage").

Also, I added empty outputs for each of the Google notebooks so that the devsite pipeline doesn't break on trying to executing them. After https://github.com/quantumlib/Cirq/issues/3323 we will be able to run these notebooks against a test environment automatically.

Previews in colab:
https://colab.sandbox.google.com/github/balopat/Cirq/blob/fix_notebook_authentication/docs/tutorials/google/start.ipynb
https://colab.sandbox.google.com/github/balopat/Cirq/blob/fix_notebook_authentication/docs/tutorials/google/reservations.ipynb
https://colab.sandbox.google.com/github/balopat/Cirq/blob/fix_notebook_authentication/docs/tutorials/google/colab.ipynb

Only comment is that there seems to be a lot of repetition. I wonder if we should put a standard "authenticate_for_notebook" type function in our code base somewhere. This could be done later though.

I agree! It could be part of a cirq.google.ipython package. It would be useful for users as well as if the number of google colabs grow, but for this three we should be good for now. It would also require to switch over to cirq-unstable until we release the functionality.


Guide: What is a good first issue?

Outline:

What is a good first issue?
For project maintainers: how to create good first issues?
For contributors: how to find an issue to get started?

The content should be live at: /guide/what-is-good-first-issue

Good first issues are the issues created by people on their projects. These issues are very easy to solve.

@nilesh212 Would you be willing to contribute to the guide?

will you plz elaborate on it?

@sanket-deepsource I would be willing to contribute to the guide!

@sanket-deepsource Would love to contribute to the guide

@sanket-deepsource i would too

@sanket-deepsource i would too

would love to conrtribute

would love to conrtribute

@sanket-deepsource A good first issue is an issue created for the sole purpose of facilitating newbies/beginners to jump-start their open journey.
To get started, one can use keywords related to what they already know, such as "python", or  with keyword "issue" .

Given below is an example of search terms used to find issue:

And lastly, I would like to contribute :)

@sanket-deepsource A good first issue is an issue created for the sole purpose of facilitating newbies/beginners to jump-start their open journey.
To get started, one can use keywords related to what they already know, such as "python", or  with keyword "issue" .

Given below is an example of search terms used to find issue:

And lastly, I would like to contribute :)

can someone englighten me about all this. I am new to open source and looking forward towards this. So, if anyone could guide me. :)

Can anyone please show me a good first issue, so that I can solve it. I'm a newbie to open source. Please help.

Can anyone please show me a good first issue, so that I can solve it. I'm a newbie to open source. Please help.

Hi @Chaitu17 , you have to go in a Tracker Issue page and use this filter of search:
is:issue is:open label:"good first issue"
Read the description of issue, if you think to can resolve write some comment in the issue and ask to take in charge it, then resolve and create a PR to the repository :)


precision

have you compared the moiu between your cc.py and the original code on cityscapes ???
becauese i have test your cc.py,and .....

have you compared the moiu between your cc.py and the original code on cityscapes ???
becauese i have test your cc.py,and .....

I have checked the mIoUs, but I dont know what is the measure of moiu.

can the miou reach 80％------------------ 原始邮件 ------------------
发件人<EMAIL_ADDRESS>发送时间: 2020年1月10日(星期五) 晚上6:06
收件人<EMAIL_ADDRESS>抄送: "swjtulinxi"1335452106<EMAIL_ADDRESS>主题: Re: [Serge-weihao/CCNet-Pure-Pytorch] precision (#1)
have you compared the moiu between your cc.py and the original code on cityscapes ???
becauese i have test your cc.py,and .....
I have checked the mIoUs, but I dont know what is the measure of moiu.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

can the miou reach 80％------------------ 原始邮件 ------------------

I load the same checkpoint using our pure PyTorch implementation and the official cc_attention, the mIoU results are almost the same, because the calculation is the same. If you have a ckpt with a testing code of 80％ of the original cc_attention, my implementation can get the same results.

CKPT of 30000/60000, using official inplace-abn
Testing with

official  CUDA cc_attention

My CC.py

i see ,thank you  for yor code,beacuse i can not use the original code, your code can work on my computer,i don't know why.

the original code is not like yours,i don't understand his code ,for example , _ext. is for what, i can't understand
def _check_contiguous(*args):
if not all([mod is None or mod.is_contiguous() for mod in args]):
raise ValueError("Non-contiguous input")
class CA_Weight(autograd.Function):
@staticmethod
def forward(ctx, t, f):
# Save context
n, c, h, w = t.size()
size = (n, h+w-1, h, w)
weight = torch.zeros(size, dtype=t.dtype, layout=t.layout, device=t.device)
    **_ext**.ca_forward_cuda(t, f, weight)

    # Output
    ctx.save_for_backward(t, f)

    return weight

@staticmethod
@once_differentiable
def backward(ctx, dw):
    t, f = ctx.saved_tensors

    dt = torch.zeros_like(t)
    df = torch.zeros_like(f)

    _ext.ca_backward_cuda(dw.contiguous(), t, f, dt, df)

    _check_contiguous(dt, df)

    return dt, df

the original code is not like yours,i don't understand his code ,for example , _ext. is for what, i can't understand

They use CPP extension with CUDA, so you may have some problems of compatibility(兼容性). The official inplace-abn is  also a CUDA implementation.

What op sys are you using, Linux or Win?

win

This repo uses Synchronized-BatchNorm-PyTorch as cross gpus BatchNorm, it costs more gpu memory. I have tested it for batchsize of 4, the result is about 67, so I think the Synchronized-BatchNorm-PyTorch may have some problems or the training hyperparameters are not good for batchsize of 4. I use inplace-abn under Linux, and I will implement inplace-abn using pure PyTorch in the future.

i just one GPU,so i use the  nn.batchnorm2d,

the paper said ,the batchsize shoud be 12 or higer when we training ,otherwise it will affect the result

inplace-abn can save some GPU memory for training, but the official CUDA  inplace-abn may not support Windows well. If this repo is helpful for you, you can star it.


Revert "Fix balena-engine logo image"

Reverts balena-os/balena-engine#319

@balena-ci I self-certify!


new option for notification controller rate limit interval

Signed-off-by: Kevin De Keyser<EMAIL_ADDRESS>
What this PR does / why we need it:
add a new option to notification controller to set rate limiting of messages.
Which issue this PR fixes
Special notes for your reviewer:
did not add an example to values.yaml as it is optional
Checklist

[x] DCO signed
[ ] Chart Version bumped
[x] helm-docs are updated
[x] Helm chart is tested
[ ] Update artifacthub.io/changes in Chart.yaml
[ ] Run make reviewable

Please add it to the values.yaml, because the chart sets a value (5m) and that should be clear to the users. That this option exists and whats its value. It's the same as loglevel.
You also missed the points to bump the chart version (make update.chartversion chartyamlpath=./charts/flux2/Chart.yaml  semvertype=minor)
There is no flux app 0.25.4 upstream.
There is a 0.26.0 release which we want to check first. So I would like to postpone you PR.

I see you have additionalargs now:
https://github.com/fluxcd-community/helm-charts/blob/main/charts/flux2/templates/notification-controller.yaml#L44
We can close this PR!


use bool in assert

Changes related to dart-lang/sdk#30326

In 2.0, the first argument to assert() must be an expression that evaluates to true or false.

@Hixie

Unfortunately, LGTM. This makes me sad, though.
cc @munificent

Me too. I can't believe after all these years we still don't have a nice way to write debug-only code. :(


Single bookmark without search results

Prerequisites
affected version: 1.16.2
make sure to check this before filing an issue:

[x] I am running the latest version
[x] I checked the documentation and found no answer
[x] I checked to make sure that this issue has not already been filed

Expected Behavior
Bookmarks should be always present in search view
Current Behavior
Single bookmark isn't shown. Only if bookmark's count >= 2
Steps to Reproduce
Open file;
Create 1 bookmark

done with 1.7.2


Is there any Information about Fixed focus range of RGB sensor?

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
{ D400 }

Firmware Version
<IP_ADDRESS>

Operating System & Version
{Win 10

Platform
PC

SDK Version
{ legacy / 2.. }

Language
{C/C#/labview/nodejs/opencv/pcl/python/unity }

Segment
{Robot }

Issue Description
<Describe your issue / question / feature request / etc..>
Hello sir,
I'm using Realsense D435i for Robotics application.
I'll use it to get variety of information in indoor environment.
and sometimes, I should do very delicate task too, like OCR.
for this purpose, I thought that I need to find out the focus range of RGB sensor, but in every datasheet, it only says It is "Fixed Focus" but what range would be great to observe objects.
so my question is " is there any information about fixed focus range of rgb sensor?"

and I tried to calculate appropriate working distance of rgb sensor.
like this
focal length x FOV / sensor size = Working Distance
refferd from https://www.ni.com/ko-kr/support/documentation/supplemental/18/calculating-camera-sensor-resolution-and-lens-focal-length.html
is it right approch?
I hope to hear your anser soon.
regards.

Actually it is more about what the hyper focal distance is. I don’t have the exact number in front of me but the image is in focus from roughly 30cm to infinity.

Thanks very much @agrunnet for your advice!
@suho0515  Do you reuire further assistance with your question, please?  Thanks!

You are very welcome @suho0515 - thanks for the update!

@suho0515 I will give you more details. The way to calculate the hyper focal distance of an imaging system is H= focal length * focal length / (circle of confusion * F-number)
The circle of confusion is how many pixels of blur you can tolerate. Let's set it to 2 pixels.  Since each pixel is 1.4um, this means 0.0028mm.
For D435 the color imager has focal length of 1.91mm.
Finally the F# is also a lens spec. It is 2.
With this the hyperfocal distance is 65cm.
However, the definition of hyper focal distance is that the "image is in focus from HALF hyper focal distance to infinity".
So that means ~33cm to infinity.

@suho0515 I will give you more details. The way to calculate the hyper focal distance of an imaging system is H= focal length * focal length / (circle of confusion * F-number)
The circle of confusion is how many pixels of blur you can tolerate. Let's set it to 2 pixels. Since each pixel is 1.4um, this means 0.0028mm.
For D435 the color imager has focal length of 1.91mm.
Finally the F# is also a lens spec. It is 2.
With this the hyperfocal distance is 65cm.
However, the definition of hyper focal distance is that the "image is in focus from HALF hyper focal distance to infinity".
So that means ~33cm to infinity.

sorry for late answer,
and Really appriciate for your details.
It is absolutely helpful, and the answer what I was looking for.
Now the Issue is completely resolved.
Thanks again @agrunnet and @MartyG-RealSense

You are very welcome @suho0515 - it's great to hear that the advice provided by @agrunnet was helpful to you.  Thanks for the update!

Hello @MartyG-RealSense @agrunnet,
I want to know what the focal length of the color imager and F# are for L515. Thank you very much.

Hi @JZhu1208   The table below is taken from the data sheet document for the L515 camera.


Add HE localization for 972

For https://github.com/googlei18n/libphonenumber/issues/1557
Since this is the first entry for language HE, does it need to be added to a list somewhere?

This change is

@SumathiSundara I saw you changed the title to iw -- was that a mistake? https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes says Hebrew is he.

Android says:
"Locale's constructor has always converted three language codes to their earlier, obsoleted forms: he maps to iw, yi maps to ji, and id maps to in. This continues to be the case, in order to not break backwards compatibility."
So they see "iw" as the "canonical".
We currently use iw too: https://github.com/googlei18n/libphonenumber/tree/master/resources/geocoding/iw
I think we should put it there, until someone says it doesn't work...

Could you merge the latest changes from master? I don't have permissions to do this (since the commits would be in your fork). Then we should be able to merge.
Thanks a lot for the quick turnaround!

Was that what you needed?

Is that what you need?

Yep, perfect!

Very nice, thanks!

Fixed in release 8.3.0.


This browser won’t be supported starting Sept. 1, 2021.

Description
Some of my slacks services reports the following message at the bottom part of the screen that I wanted to share with you. Do you think that Ferdi will continue to be able to work with the Slack servers ?
This browser won’t be supported starting Sept. 1, 2021. Update your browser or get the latest app to keep using Slack. Learn more

@cmoulliard - please upgrade to the latest nightlies found here - i am using slack on Ferdi (nightly) - and don't see this issue.
Also a request: when logging tickets, it would be a good idea to post which version of Ferdi you are using , which OS, etc. There's a template for such items in GH.

Why are you closing the ticket ?
Here is the info requested
Ferdi : Version 5.6.0-beta.2 (5.6.0-beta.2.2329)
MacOs: BigSur 11.4

@cmoulliard - we are free to reopen the ticket. As I mentioned above, this is not an issue with the latest code (not yet released). Please upgrade to the nightliy version (which is quite stable) and you can get the latest version of Chromium that Ferdi is built with.

With one of the PRs that got merged in over the past week or so, the user will also be able to specify their own string for the user-agent - which is how Slack determines whether the browser is supported or not. Feel free to explore that option as well for your usecase.

Many thanks for the clarification. If this ticket will be fixed with the upcoming 5.6 release, then you could perhaps add here the link to the PR or milestone to keep us informed  ;-)

This is the PR that merged the latest version of electron 13.1.4 - of course, your issue might have been fixed in an earlier version as well. Please try the latest nightly and close this issue once you can confirm this.

For what it's worth: I just updated to the 5.6.0-nightly.66 and don't immediately see the notification. Thanks!


TrojanDropper.Dapato.acbs

Which version are you using?
v0.12.2
Which operating system are you using?
Windows (+Mac + Linux - no viruses)
Question
There is trojan detection alert on VT:
https://www.virustotal.com/gui/file/f3bec6c7374b18ae8cc89ea8b9b97022d41bf6014a68e78f6d65f04d857fd43e/detection

And one more report for nightly version (rtsp-simple-server_v0.12.2-1-g0f11dd5_windows_amd64.zip) -  https://www.virustotal.com/gui/file/165cda3d13efefbc1c88ca9c34513141b9d750d9bd6b79cdbcc7d43cf9ffc6ce/detection


Set bundle version to 1.0

The bundle version was $(CURRENT_PROJECT_VERSION) this breaks Carthage versions when validating an App Store version of an app as the version will actually be blank.

@neakor could you give this a review.

@neakor can you merge this, I can't.


MWPW-120863 - Modal Button Focus Bug

Fixes accessibility issue in the modal: When the modal closes, the focus should return to the button that opened the modal

Resolves: MWPW-120863
Test URLs:

Before: https://main--milo--adobecom.hlx.page/drafts/methomas/modal?martech=off
After: https://methomas-modal-button-focus--milo--adobecom.hlx.page/drafts/methomas/modal?martech=off

@vhargrave I've removed the "verified" label because our QE needs to take a look at this first. If there's an assignee on the PR, they should be the one to verify it. Thanks!


Feature Request: Options for Customizing 'Section' background color

Hi,
The purple background color makes it difficult to distinguish the text. (See below image).
I can't find a way to change background color of a section in git log
Could you develop it?

Please set StyleSectionLine in ov.yaml(https://github.com/noborus/ov?tab=readme-ov-file#32-config).
https://github.com/noborus/ov?tab=readme-ov-file#71-style-customization
StyleSectionLine:
  Background: "slateblue"

Try changing to below
StyleSectionLine:
  Background: "gray"
  Foreground: "aqua"
  Underline: true

@noborus Thx, What a nice tool!

ov can be colored depending on the style, but the only thing that changes the color is the Dim setting.
Therefore, I think that the git color specification or the terminal emulator's palette is having an effect.


(PUP-9477) Make call() load functions from any loader

Before this the call() function was only capable of calling functions defined in puppet core.

CLA signed by all contributors.

There was an unrelated appveyor error that apparently has to do with generation of passwords, and supposedly goes away when running again... so - running again.


Remove old 'drop constraint' and 'add column'

Closes https://github.com/crate/crate/issues/13302, one of the pre-requisites for https://github.com/crate/crate/issues/4457

Hi @seut, could you please take a look?
In https://github.com/crate/crate/pull/13708/commits/cdce64fbfaa0b4b1e3309da67c83ae8e4d73f3cf I simply removed old code + update mapping (as I thought we need only updateSettings for the ALTER TABLE SET scenario).
Apparently, there is a 1 case where we change setting but still have to update mapping (column policy) - pushed a fixup in https://github.com/crate/crate/pull/13708/commits/24a300804debbcecfe069d86be76bcc58423e459
What's important, even with getting back updateMapping, AlterTableClusterStateExecutor doesn't copy existing metadata anymore - I used same approach as we used for new add column approach.
Didn't reuse code from the AddColumnTask since this case is even simpler - we don't merge constraints, we put only "dynamic" key to the mapping and what we have in AddColumnTask not really fits here.


user defined formats

This is the code that allows user-defined formats to be implemented as separate libraries.
Fixes #177.
This requires that user-defined format writers see and use the functions in nc4internal.h. That motivates the split of hdf5 code, which greatly reduces the complexity and volume of the code exposed in nc4internal.h.
However, it is always going to be true that writers of user-defined formats are going to be looking at netCDF internals and will need to develop a programmer-level knowledge of them, in order to implement a new binary format.
Also fixes #1006.

Seeing a build failure on my local machine not caught on appveyor.  See http://cdash.unidata.ucar.edu/viewBuildError.php?buildid=2610.  Figuring out the discrepancy.

My immediate guess is that those symbols were not properly __declspec'd.

OK, I see what is going on here.
Let me ponder what to do.

I'm addressing it on the consolidation branch, primarily it's a matter of using the appropriate EXTERNL macro.

Done.

@WardF this PR now ready to go. In response to comments from last time, I've added much documentation and tests. Also note that magic numbers are now supported for user-defined formats.

Taking a look, will merge shortly, happy Monday all.

Thanks for the PR.  One thing that's been bothering me for a while (in general) is the extent to which internal non-user-API functions are being exported, typically for the sake of testing.  It's not a high priority at this point, certainly, but it's something I'm chewing on and will probably look to address down the road.

Is there some way we can at least get a list of those "external for testing only"
function?

@DennisHeimbigner the functions are all in include/ncdispatch.h and include/nc4dispatch.h which have an EXTERNL macro.
Without this EXTERNL macro, user-defined libraries would not function as windows DLLs. That is, user-defined libraries would only work on unix platforms.

We may be talking at cross purposes. I was referring to Ward's comment:

One thing that's been bothering me for a while (in general) is the extent to which internal non-user-API functions are being exported, typically for the sake of testing

I know so some such as the one I added temporarily to support debugging for the new
nc_get/put_vars
This may be a hard list to compile because they are often stuck deep in various
internal files.

It's my thought that, with a few exceptions I'm sure, anything that isn't public api as denoted by nc_ should not be exported.  I'm sure there are exceptions, but for the most part that is how I would begin looking at it.  It's low priority, so can be worried about later.


Float/IntVariable clamped issue

Generate new FloatVariable or IntVariable over
right click -> Create -> Variables -> Float/Int
and then it is not possible to set a other Float/IntVariable as Min/Max Clamped Value, just numbers.

System:
Windows 10 64 bit, 16 GB RAM, SSD,
Unity 2020.1.0a17, 2D project template

I was re-implementing this feature, unfortunately it's not possible with the current codebase.
When the screenshot in the wiki was taken, clamped variables were completely separate classes. When it was merged into the current class structure, the reference type was dropped - at the time probably by accident. The problem is that in order to serialize the clamped values as a reference, they have to be passed as a generic argument. The reference, in turn, needs to know what type of variable it has to serialize. This creates a cyclic serialization, which prevents me from re-implementing the feature.
I'm keeping this open as it would be possible if we were to switch to a custom serialization scheme #100 , that doesn't rely on Unity's restriction. In this particular case, being able to serialize generics should solve it, as we only need to pass the base type (float instead of FloatUnityEvent, FloatReference, etc.) as a generic argument.


why the exe file is missing?

I can't find the exe file everywhere,no matter how hard I tried.the bat file won't work with out the exe.

It's here? https://github.com/0xbb/gpu-switch/releases


Added goto value feature

Fixes #652
Manually tested to ensure the goto feature works on integer columns and string columns

[ ] If go to is initiated via shortcut or the sidebar menu, and there is a table row cursor (a selected row) populate the initial values for the row index with that row number
[ ] If go to has a row index, then populate the go to value with the value of the selected column matching that row index, if the column selected dropdown is changed, pre-fill the new matching value for that row index.

I am good with splitting the reminder of the work out into a separate ticket so this can ship.

for the issue with excess space, I think resetting lastTop should really be taken care of in a IrisGridBottomBar. Thoughts?
Yes probably.

seekRow function in IrisGrid was restructured into a switch for better style


Accessibility migration

🐞 Problem
Our ansys-sphinx-theme is not compliant with accessibility guidelines, see:

WAVE Report: https://wave.webaim.org/report#/https://sphinxdocs.ansys.com/

Notice that some of the features we use are inherited from the pydata-sphinx-theme. Thus, any accessibility improvements we make could be moved at some point to this other theme.
📝 Solution
After discussing privately, here is a list of the issues that need to be addressed for making our theme more accessible:

[ ] Color related

[ ] Stop using Ansys gold and use near black
[ ] Same applies for header links
[ ] Hyperlinks must be vibrant blue

[ ] Top navigation bar

[ ] Top navigation bar needs to reflect "normal", "active" and "hover" states
[ ] Activate navigation bar section needs to be underline
[ ] Navigation bar toggler dropdown requires to be visible in both light and dark theme

[ ] Left sidebar

[ ] Fix styling when hovering and for active section
[ ] Fix visibility when using a narrow browser

[ ] Right sidebar

[ ] Fix styling when hovering and for active section
[ ] Fix margin (this needs further discussion)
[ ] Fix titles

[ ] Breadcrumbs

[ ] Remove bottom border

[ ] Body content

[ ] Remove maximum width restriction on container
[ ] Impose auto margin

👥 Teams
This topic was originally raised by @timdansys and discussed with documentation and PyAnsys team. Pinging here all people related with this issue: @PipKat @MaxJPRey @RobM-official @Revathyvenugopal162
🔗 References
Acesibility guidelines: https://www.w3.org/WAI/standards-guidelines/wcag/

Linking here the pull requests which partially addressed this issue: #90 and #93.

Marking the addressed problems in #90 and #93.

not me! I think you wanted @RobPasMue

[ ] Admonitions theme.

[ ] color for admonitions

Already created an issue on this last point: #100

I think #121 this is also relevant here...


[code-quality]: Remove imports from __init__.py files

Fixes https://github.com/embedchain/embedchain/issues/216
Still keeping the App related imports in __init__.py file for now but we will change that later.
Tested locally by running tests.
(base) ➜  embedchain git:(remove-init-imports) make lint
python3 -m ruff .
(base) ➜  embedchain git:(remove-init-imports) make test
python3 -m pytest
====================================================================================== test session starts ======================================================================================
platform darwin -- Python 3.11.3, pytest-7.4.0, pluggy-1.2.0
rootdir: /Users/deshrajyadav/Projects/embedchain
plugins: anyio-3.7.1
collected 2 items

tests/test_embedchain.py ..                                                                                [100%]

======================================================================================= 2 passed in 1.54s =======================================================================================

This PR is stale now since we have incorporate the similar changes in a different smaller PRs. Hence, closing this one.


changed .raw elements in dashboard to .keyword

@gingerwizard can you sanity check this?
With the current version of kibana this wasn't working, since the element names switched from .raw to .keyword
I will try to get the blog (https://www.elastic.co/blog/building-cloud-sandbox-with-sample-data-v2) updated as well, but even if people figure out how to get here this won't load properly

LGTM


Update inference sample make file.

Description
Current CPP inference examples on OSX cannot be executed. They failed to load libmxnet.so file.
This PR will fix rpath issue for OSX.

put all output files into build folder.
set rpath location for osx build.

Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[X] Changes are complete (i.e. I finished coding on this PR)
[X] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[X] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[X] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@mxnet-label-bot add [pr-awaiting-review]

@mxnet-label-bot add [C++, Example]
@frankfliu could you re-trigger CI?

@frankfliu The CI still seems to fail on this PR. Can you have a look at it ?

@frankfliu Can you please take a look at the build failure so that we can move forward with this PR?

@frankfliu gentle ping to take a look at CI failure.
cpp-package:
Inception-BN-0126.params

Inception-BN-symbol.json

synset.txt

./unit_test_inception_inference.sh: line 30: build/inception_inference.log: No such file or directory

grep: build/inception_inference.log: No such file or directory

FAIL: inception_inference FAILED to identify the image.

./unit_test_inception_inference.sh: line 33: [: ==: unary operator expected

build.py: 2019-03-08 17:52:11,312Z INFO Waiting for status of container c0f2c67b4b86 for 600 s.

build.py: 2019-03-08 17:52:11,505Z INFO Container exit status: {'StatusCode': 1, 'Error': None}

build.py: 2019-03-08 17:52:11,505Z ERROR Container exited with an error 😞

build.py: 2019-03-08 17:52:11,505Z INFO Stopping container: c0f2c67b4b86

build.py: 2019-03-08 17:52:11,507Z INFO Removing container: c0f2c67b4b86

build.py: 2019-03-08 17:52:11,553Z CRITICAL Execution of ['/work/runtime_functions.sh', 'integrationtest_ubuntu_gpu_cpp_package'] failed with status: 1

Terminated

@frankfliu gentle ping to take a look at CI failure.

@frankfliu Gentle ping...

@frankfliu Any update on this PR?

@frankfliu Could you please update your finding on this PR and drive this to merge? Thanks!

@frankfliu Gentle ping...

gentle ping to rebase and update, thanks!


0.9.3 release

Awesome, thanks!
Did you use a script for the changelog? If so, can you put up a PR for it?


added port scanner & gitignore & readme typos

Created a port scanner function to the code that scans the target IP address ports. Also fixed couple of typos + added .gitignore
#4

It is not working

Hmm.. is python3 network_scanner.py -t [IP address range] meant to get range is some way? I managed to use that to get the single IP address open ports

Or you meant the pip3 is not working?


Prevent addresses from being added to the URLs

Though the playground isn't intended for use with sensitive information, we should still avoid including addresses in URLs to avoid any privacy risks.
The recent addition of analytics that are including URLs in the event parameters exacerbates this risk so it should be prioritized.
Here's an example of what's being passed when there are params included in the URL (there is some masking but that is from a third-party UI and doesn't mean they don't have the full address in their system):
analytics.page({
  path: '/wallet/reference/wallet_requestpermissions/',
  referrer: '',
  search: '?requestPermissionsObject[eth_accounts][account]=0***',
  title: 'MetaMask developer documentation',
  url: 'https://docs.metamask.io/wallet/reference/wallet_requestpermissions/?requestPermissionsObject[eth_accounts][account]=0***'
});

At the same time, we want to balance privacy against the developer experience.
@BelfordZ recommends adding a "Copy" button to allow developers to save their input data from the playground in case they need it again later.

just to clarify, the copy button would

set the url params to include the request params
copy the url into your clipboard


adding excluding option in the project setting for excluding notes from  folder/subfolders  from the project

What would you like to be added?
would it be possible to extend this feature https://github.com/marcusolsson/obsidian-projects/issues/93 to exclude a folder or pattern like
subfolder-with-excluded-notes*?
or even better, to use a regex pattern to exclude specific files (at this moment the exlusion of folders together with subfolders will be enough)
Why is this needed?
Currently I try to store all project related folders in one folder with subfolders, sometimes some of the files  doesnt make sense to be included in the project structure created by the plugin,
for example I want to track only files with milestones and meeting notes but I want to exclude folder with project-related notes  (definition, drafts, et cetera) from  tracking (espacially when I have many of the notes)

I have the same requirement. I am used to putting attachments in a subfolder at the same level as the note, which causes the attachments to mix into the project. I hope the author can develop this feature as soon as possible!🥺

Or at the most basic level, could you add an option to place restrictions on formatting? I often put the notes text folder in the same project folder as the attached images folder, and then import the sub-parent folder into the Projects plugin, which results in image files often mixed in between normal markdown items in table and gallery views, which is a bit annoying. I hope you can optimize this issue soon, it will reduce a lot of repetitive work for me, thank you!🥲


Invalid mapping...

Loading bootprint-openapi 0.16.0
Loading bootprint-json-schema 0.8.6
Loading bootprint-base 0.7.2

/usr/local/lib/node_modules/bootprint/node_modules/q/q.js:155
                throw e;
                ^
Error: Invalid mapping: {"generated":{"line":5,"column":17487},"source":"/usr/local/lib/node_modules/bootprint-openapi/node_modules/bootstrap/less/type.less","original":{"line":1,"column":null},"name":null}
    at SourceMapGenerator_validateMapping [as _validateMapping] (/usr/local/lib/node_modules/bootprint/node_modules/less/node_modules/source-map/lib/source-map-generator.js:277:13)
    at SourceMapGenerator_addMapping [as addMapping] (/usr/local/lib/node_modules/bootprint/node_modules/less/node_modules/source-map/lib/source-map-generator.js:101:12)
    at SourceMapOutput.add (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/source-map-output.js:88:42)
    at Element.genCSS (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/tree/element.js:42:12)
    at Selector.genCSS (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/tree/selector.js:104:21)
    at Ruleset.genCSS (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/tree/ruleset.js:389:21)
    at Directive.outputRuleset (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/tree/directive.js:111:22)
    at Media.genCSS (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/tree/media.js:34:10)
    at Ruleset.genCSS (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/tree/ruleset.js:414:18)
    at SourceMapOutput.toCSS (/usr/local/lib/node_modules/bootprint/node_modules/less/lib/less/source-map-output.js:126:24)

Appears to be an issue with the 2.7.0 release of less.
https://laracasts.com/discuss/channels/elixir/elixir-ccompilation-fails-on-new-page

Appearently this is due to https://github.com/less/less.js/issues/2896
I'll change the dependencies to use a ~2.6 version

Sorry about that. 2.7.0 has been unpublished and 2.7.1 is published.

Thanks for the personal notice. And thanks for fixing so quickly.

Thanks @matthew-dean and @nknapp - confirmed, we're all good.

👍


Codeunit 7000 "Sales Price Calc. Mgt." event on procedure GetSalesLineLineDisc

Describe the request
Hello, would it be possible to add OnBeforeValidateSalesLine event on the procedure GetSalesLineLineDisc to codeunit 7000 like this:
procedure GetSalesLineLineDisc(SalesHeader: Record "Sales Header"; var SalesLine: Record "Sales Line")
...
if PAGE.RunModal(PAGE::"Get Sales Line Disc.", TempSalesLineDisc) = ACTION::LookupOK then begin
            SetCurrency(SalesHeader."Currency Code", 0, 0D);
            SetUoM(Abs(SalesLine.Quantity), SalesLine."Qty. per Unit of Measure");
            ...
            SalesLine.TestField("Allow Line Disc.");
            IsHandled := false;
            OnBeforeValidateSalesLine(SalesLine, TempSalesLineDisc, IsHandled); //New Event
            if not IsHandled then
                 SalesLine.Validate("Line Discount %", TempSalesLineDisc."Line Discount %");
end;

OnAfterGetSalesLineLineDisc(SalesLine, TempSalesLineDisc);

[IntegrationEvent(false, false)]
local procedure OnBeforeValidateSalesLine(var SalesLine: Record "Sales Line"; var SalesLineDiscount: Record "Sales Line Discount"; var IsHandled: Boolean)
begin
end;

Internal work item: AB#538768

Availability update: We will publish a fix for this issue in the next update for release 24.
Build ID to track: 21676.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 21691.


Allow experience flag to be toggled

Closes N/A
Code Changes

[x] Enable privacy experience flag to be user modifiable

Steps to Confirm

Run the app and check out the toggle!

Pre-Merge Checklist

[x] All CI Pipelines Succeeded
Documentation:

[x] documentation complete, PR opened in fidesdocs
[x] documentation issue created in fidesdocs

[x] Issue Requirements are Met
[x] Relevant Follow-Up Issues Created
[x] Update CHANGELOG.md
[x] For API changes, the Postman collection has been updated

Description Of Changes

Failing test passes locally... a problem for another time :)


Need to configure AppGateway to return required security response headers

Currently, the mandatory HTTP response headers are not returned when a path is not found or JWT header is missing or invalid in a request. Need to return the following mandatory headers from App Gateway:
"X-XSS-Protection"
"X-Content-Type-Options"
"X-Frame-Options"
"Cache-Control
"Expires"
"Strict-Transport-Security"
This feature may be required or custom rules can be added.

This feature/code has fully migrated to GitLab.


Edit UG

add definition of date range to ToC
edit add command's title to be consistent with features list
change find command's description

Codecov Report

Merging #217 (234a069) into master (b520f27) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #217   +/-   ##
=========================================
  Coverage     71.26%   71.26%
  Complexity      643      643
=========================================
  Files            90       90
  Lines          1970     1970
  Branches        239      239
=========================================
  Hits           1404     1404
  Misses          480      480
  Partials         86       86

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b520f27...234a069. Read the comment docs.


TS-4249: Deprecate stale_while_revalidate plugin.

FreeBSD build successful! See https://ci.trafficserver.apache.org/job/Github-FreeBSD/576/ for details.

Linux build successful! See https://ci.trafficserver.apache.org/job/Github-Linux/472/ for details.

👍 - Looks good

Yeh, I like this, major drag we have to keep it for another full year. :-/.
:+1:


Feature/geometry data points filter for master

This filter reproduces our heuristic intended for estimating radio signal attenuation through point clouds (GPS and UGF signals).

Can one of the admins verify this patch?

ok to test

Jenkins has some error related to be unable to read some output log file (java exception in the end of the console output). Seems to me more like Jenkins configuration problem than the actual build and test problem...

that's a recurring problem and why we want to transfer ownership

I'm a user of this package and would like to help reviewing the changes. I will proceed to that.

ok to test

@kubelvla Would it be possible to add the unit tests for this new filter?
Additionally, will you add it to the documentation within the scope of this PR?

For the unit tests, I'll think about what should actually be tested and then ask one of our students to help me. The bibliography will be updated as soon as the paper is out, now waiting for review.

Hi @kubelvla,
Thank you for addressing the comments.
I understand. I would propose to merge as soon as possible if you and the author approves. Hence, can we open an issue about creating tests and documentation for this filter?

@YoshuaNava I'll open the issue with the biblio and unit test for myself.

@kubelvla Looks good to me.

@kubelvla What is the status of this PR? Do you have further implementations planned?

@YoshuaNava Under the condition I made no mistake when implementing the filter, there should be no changes to this filter.

@kubelvla Is this PR ready for merging?

If there is no objection, I'll merge it tomorrow.

@pomerlef No objection from my side.


[ENHANCEMENT] add video prop - "isPrivate"

Is your feature request related to a problem? Please describe.
For some applications it required to know if the video is private
Describe the solution you'd like
Add isPrivate property.
Describe alternatives you've considered
🤷♂
Additional context
https://github.com/CodeDotJS/youtube-playlist/blob/459eb95c05d438ffc42c4716c07a7b4bb94c9cc2/index.js#L18

I probably won't add this anytime soon.

I will add this when I started rewriting for 2.0.0.
As for now, I'm just focusing on bug fixes, but PR for new feature will be accepted wink wink.

This will be available in v2.0.0 (which is still in progress), as well as isDeleted

This will be available in v2.0.0 (which is still in progress), as well as isDeleted.
Although I might need to combine them both (isPrivateOrDeleted) since the only way to check whether the video is deleted or private (as far as I know at least) is by checking the title ("[Private video]" / "[Deleted video]" for english), and Youtube can sends different language depending on the location / settings, so there's no way to check unless if I cover the whole available language from Youtube to check which probably isn't gonna be efficient.


Unable to search in indices with attachments (mapper-attachment plugin) NEST v5.2.0

NEST/Elasticsearch.Net version: v5.2.0
Elasticsearch version: v5.2.2
Description of the problem including expected versus actual behavior:
We have our node hosted on Elastic Cloud and have many indices with PDF attachments stored (using mapper-attachment plugin)
Searching in these indexes, now returns error: Attachment fields are not searchable
Steps to reproduce:

Create index with attachments using mapper-attachment
Index PDF file(s)
Search for text within PDF file

We were able to do this with NEST v2.4.4
This issue started appearing since we upgraded to NEST v5.2.0 and ElasticCloud v5.2.2
Searching in non attachment indices works fine
Debug Information:
Invalid NEST response built from a unsuccessful low level call on POST: /au_tcg/_search?ignore_unavailable=true
Audit trail of this API call:

[1] BadResponse: Node: https://excluded.us-west-2.aws.found.io:9243/ Took: 00:00:01.0400595

ServerError: ServerError: 400Type: search_phase_execution_exception Reason: "all shards failed" CausedBy: "Type: query_shard_exception Reason: "Attachment fields are not searchable: [content]""
OriginalException: System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at Elasticsearch.Net.HttpConnection.d__9`1.MoveNext() in C:\code\elasticsearch-net\src\Elasticsearch.Net\Connection\HttpConnection.cs:line 204
Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
Cheers,

Could you provide an example of the kind of search you are performing?

Hi @russcam ,
I'm doing a simple keyword search (e.g: 'exemption') from a PDF file (DCR - A4 Copy paper.pdf)
PDF files like above, are indexed using the below code:
var result = await elasticClient.CreateIndexAsync(indexName, index => index
                .Mappings(mappings => mappings
                    .Map<IndexModel>("book_content", map => map
                       .Properties(properties => properties
                            .Attachment(att => att
                                .Name(name => name.content)
                                .FileField(fileField => fileField
                                    .Name("content")
                                    .TermVector(TermVectorOption.WithPositionsOffsets)
                                    .Store()
                                )
                           )
                        )
                    )
                )
                .Settings(settings => settings
                    .NumberOfReplicas(0)
                    .NumberOfShards(noOfShards)
                )
            );

var result = await elasticClient.IndexAsync(indexModel, i => i
                        .Index(indexName)
                        .Type("book_content")
                        .Id(Guid.NewGuid().ToString())
                        .Refresh(Elasticsearch.Net.Refresh.True)
                        //.Refresh()
                        .Ttl("1m")
                    );

Type:
public class IndexModel
    {
        public string id { get; set; }
        public string name { get; set; }
        public string content { get; set; }
    }

Below is the mapping:
{
  "book_content": {
    "properties": {
      "next": {
        "type": "text",
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        }
      },
      "id": {
        "type": "text",
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        }
      },
      "name": {
        "type": "text",
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        }
      },
      "content": {
        "type": "attachment",
        "fields": {
          "date": {
            "type": "date"
          },
          "keywords": {
            "type": "text"
          },
          "content_type": {
            "type": "text"
          },
          "author": {
            "type": "text"
          },
          "name": {
            "type": "text"
          },
          "language": {
            "type": "text"
          },
          "title": {
            "type": "text"
          },
          "content": {
            "term_vector": "with_positions_offsets",
            "store": true,
            "type": "text"
          },
          "content_length": {
            "type": "integer"
          }
        }
      }
    }
  }
}

I don't have issues with indexing. Its just the search in attachment indices that stopped working.
Surprisingly, I managed to get search hits from attachment related indices using Postman client.

@thammadi can you also provide the search that you're doing?


Update the Chevron component

Description
This PR updates the api of the Chevron component, changing the classNames property that took an object of shape { wrapper: string, svg: string }, to the className property, which takes a string.
Note:
This PR uses https://github.com/Ensembl/ensembl-client/pull/787 as a base. It can either be merged into the show-hide branch, or merged independently after https://github.com/Ensembl/ensembl-client/pull/787 has been merged.
Deployment URL(s)
http://update-chevron.review.ensembl.org
Views affected

Entity Viewer
Species page
Help page
Contact form

@azangru Looks good so approved the PR. However, there are some conflicts that need to be resolved before merging.


Document pull-request -o

We have already documented the option in the man page, but when you do hub pull-request --help, the internal hardcoded help is displayed which you edited here. Thanks, but I believe that we need to fix documentation by consolidating the two help sources, not by trying to keep them up-to-date separately. We'll see what can be done here.

Please excuse the drive-by suggestion, but maybe calling exec man would be a solution?

Yes, it might for non-Windows platforms. But we would also have to split the big man page into man pages for individual commands.


Enable use of subsystems in get_components

I am opening this issue for discussion to incorporate the notion of subsystems in PSY.
The issue arrises from the fact that for large systems it is strategic to split the system in sub-systems to solve for effectively. Although we have areas and load zones, those definitions conflict with administrative definitions critical for reporting and can be different than the desired split of a system for modeling purposes as is the case in NTP.
Further, it could strategic to split a system in ways that aren't relevant to those administrative definitions for example in Balancing Areas according to their slack bus. These in almost every real data set do not correspond to the same area as defined in the PSSe files.
For the reasons above I am proposing not to use the name Area or Region but subsystem.
The proposed approach is to enable an internal map to System such that we can reference the UUID of a component to a "sub system"  in a dict. E.g., Dict{String, Vector{UUID}}  and enable an interface as follows `get_components(ThermalStandard, sys, "SubSystemID").
The proposed interface through get_components() would be the minimally invasive approach to incorporate this feature in PowerSimulations.jl
cc. @claytonpbarrows

The current interface to get components in an Area or LoadZone is PSY.get_components_in_aggregation_topology(subtype, system, aggregator) where aggregator is an AggregationTopology instance. Maybe we can refactor this to be just another method of get_components too? And then for consistency we should either refer to subsystems by instances of a subsystem struct or refer to these existing aggregators by string (or both).

The challenge here as mentioned above is the fact that even aggregation topologies can be part of subsystems too.


HTTP connections created by Gradle should use TCP keep-alive

Expected Behavior
HTTP Build cache client should work for long builds where the build cache is behind a proxy that terminates idle connections.
Current Behavior
In between usages of the HTTP connections, they may be terminated by a MITM or the server. The client doesn't see this until it goes to use the connection, upon which the request fails and the build cache is then disabled for the rest of the build.
Context
Raised here: https://github.com/gradle/gradle/issues/6018#issuecomment-414687655
Note: this won't completely eliminate failures due to dead connections, but should significantly reduce them by the keep alive mechanism keeping the connection alive (i.e. preventing idle) and probing the connection regularly to test whether it is still alive.

Fixed via https://github.com/gradle/gradle/pull/6564.
@ldaley Can you confirm this works now?

@wolfs is there any chance of back-porting this fix to 4.X ?
I can confirm the fix works on a local custom gradle build of 4.9.

@josecoll We probably won't be back-porting this feature to the 4.x series. We only do bugfix releases.
Is there something in particular which would block you from upgrading to 5.0 (i.e. do you have some deprecation messages in your build)?


Notion cannot display the graph chain of TMDB

Do the author know the reason or solution？

Graph chain ?

Graph chain ?

Sorry for the incorrect expression, as shown in the picture below, the image does not display

Previously, it could have been displayed normally, could it be due to the network environment in China? I have already enabled VPN.

Ohh wierd, its not working for me as well, seems like notion changed something with their images

I also think so, but I tried to copy the link of the image to the notion, which can be displayed.

Hello. I have the same problem, do u have a correction date preview?
Thanks for you work on this. <3


Update google-cloud-bigquery to 2.1.10

Updates com.google.cloud:google-cloud-bigquery from 2.1.9 to 2.1.10.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.google.cloud", artifactId = "google-cloud-bigquery" } ]

labels: library-update, semver-patch

Superseded by #122.


Update to version 2.1.0

Description

This PR updates to version 2.1.0
Related Issue

Fix #121
Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Close this PR just before next release deploy


Reduce the amount of allocations and code for options [v2]

Improvements to how we deal with struct sol_flow_node_options.
The second patch ensures that we simply pass thru the options from the spec if nothing is being exported, as discussed in #335.

@ibriano could you take another look? Patches 1 and 2 are unmodified. Patch 3 got some code changes due to the addition of named_opts.
This is rebased, deals with JS (currently doesn't uses options) and with simplectype.

Replaces #421.

Still looks good

Applied in 12b9675d431b68b4e38c89df51249ea9e04cbddd and parents.


[BEAM-2284] Completely removes OldDoFn from Beam.

While doing this, noticed that WindowingInternals can also be moved into Dataflow worker and deleted from Beam. Will do that separately.
R: @tgroh

Run Dataflow ValidatesRunner

Run Dataflow ValidatesRunner

retest this please

Coverage increased (+0.5%) to 70.729% when pulling 045cce79902a7b1ca500c249b6b1910b092f9cfc on jkff:remove-old-dofn into 0acf3620d90a970f4024ae95b909c3bb4ef827ad on apache:master.

Ugh flaky tests.

retest this please

Coverage increased (+0.6%) to 70.74% when pulling 045cce79902a7b1ca500c249b6b1910b092f9cfc on jkff:remove-old-dofn into 0acf3620d90a970f4024ae95b909c3bb4ef827ad on apache:master.

run Dataflow ValidatesRunner

Coverage increased (+0.5%) to 70.748% when pulling be626a412fe76707396539444e824f905ccd88d9 on jkff:remove-old-dofn into 88ec00860032078dcd33e07d94642912f6aa7bf7 on apache:master.

CC: @kennknowles


Add missing config

The configuration for test is not passed via the engine
Overview
TL;DR
It seems to be a missing config, it used to check for process.env.Test but when compileOnceOnly has been introduce, it hasn't been added to the engine
Cool Spaceship Picture

LGTM


Add a config to set all columns to nullable

KCBQ currently translates the Kafka connect required/nullable settings for fields into BigQuery fields as-is. That is, if a field is required in Kafka connect's schema, it will be REQUIRED in BigQuery. The same is true for NULLABLE fields.
It would be useful to be able to tell KCBQ to ignore these settings and always set BigQuery fields to have NULLABLE for all fields. This will give us a bit more flexibility in the BigQuery pipeline, since it's difficult to migrate tables from REQUIRED to NULLABLE in BigQuery, and upstream systems occasionally want to do this. I acknowledge that when upstream systems do this, it's an incompatible operation for downstream consumers, but in cases where KCBQ is the only consumer, making it a bit more resilient to the change (vs. having to re-bootstrap the data to change the schema).
I suggest exposing this option via a config. To make things compatible for existing connectors that are already running, it would be nice if the config worked such that it gracefully only set NULLABLE for all fields that it doesn't already know about. That is, if a table already exists in BQ with required fields, it leaves those as they are, but any new fields that are added are always nullable.

We recently had an issue with this, having a source schema go from nullable to required kind of breaks in the case when using something like CDC via Debezium, because you can't update a nullable to required

I believe this was resolved by config parameter allBQFieldsNullable just after the previous comment.


Bug/visibility issues

Am I free from my mortal coil?
Am I finally done debugging visibility?
Tune in next week on DBZ

Can this be closed now that we have the foaf fix?


[Bug]: Aborting sync because of net::ERR_HTTP2_PROTOCOL_ERROR

What happened?
I cannot perform sync between Linux (Fedora 39 KDE) and Windows 10 using OneDrive. I've made initial sync from Linux today, but I can't manage to sync with Windows or even to sync into completely empty Vault.  The error is net::ERR_HTTP2_PROTOCOL_ERROR and these are my logs:
plugin:remotely-secure:180 remotely-secure-1702494364014: abort sync, triggerSource=manual, error while syncing

and
AggregateError:
    Error: ML/Probability & Statistics for Engineers & Scientists/standard normal distribution.md: net::ERR_HTTP2_PROTOCOL_ERROR
        at eval (plugin:remotely-secure:112:51897)
    Error: ML/Probability & Statistics for Engineers & Scientists/Hypergeometric Distribution.md: net::ERR_HTTP2_PROTOCOL_ERROR
        at eval (plugin:remotely-secure:112:51897)
    Error: ML/Deep Learning with PyTorch Step-by-Step/Deep Learning with PyTorch Step-by-Step.md: net::ERR_HTTP2_PROTOCOL_ERROR
        at eval (plugin:remotely-secure:112:51897)
    Error: ML/Introduction to Probability, Statistics, and Random Processes/Bernoulli Trial.md: net::ERR_HTTP2_PROTOCOL_ERROR
        at eval (plugin:remotely-secure:112:51897)
    Error: ML/Introduction to Probability, Statistics, and Random Processes/Binomial Formula.md: net::ERR_HTTP2_PROTOCOL_ERROR
        at eval (plugin:remotely-secure:112:51897)
    Error: too many errors, stop the remaining tasks
        at Xre (plugin:remotely-secure:112:52082)
        at async Rx.doActualSync (plugin:remotely-secure:180:4114)
        at async Rx.syncRun (plugin:remotely-secure:180:2769)
    at Xre (plugin:remotely-secure:112:52138)
    at async Rx.doActualSync (plugin:remotely-secure:180:4114)
    at async Rx.syncRun (plugin:remotely-secure:180:2769)

I've deleted few similar errors to cencor note names.
I also had promlems with previous version of add-on (remotely save) but can't name the exact errors.
What OS are you using?
Windows, Linux
What remote cloud services are you using?
OneDrive for personal
Version of the plugin
0.4.28
Version of Obsidian
v1.4.16
Using password or not

[ ] Yes.

Ensure no sensitive information

[X] I ensure that no sensitive information is submitted in the issue.

Odd, I haven't seen this error - looking into our OneDrive sync it looks like we're using fetchUrl from Obsidian, and I don't see an option to override the HTTP protocol.
From googling it might be related to some bug in Windows - there's very little out there for that exact error.
Is your Windows 10 machine on the latest Windows update?

I've updated my Windows 10 and reloaded and it still downloads the next updates. I've tried to sync to test and it still drops errors. After I've decided to delete folder with PDF files (nearly 1GB of them) from OneDrive and it synced successfully after it.
I previously had problems with length of PDF files' names (more than 128 characters couldn't be copied into Android) - maybe this is a reason why it was failing.
The smartphone on Android also finally synced as well.

Appreciate the update. Odd we did some large file testing and didn't see this error but I will close the issue if it's working g now in favor of #42