[Feature]: Operator can install Korifi without needing HNC

Blockers/Dependencies
No response
Background
As an Operator
I want fewer dependencies
So that I can more easily run Korifi
Acceptance Criteria
GIVEN I have docker and kind installed
WHEN I use the deploy-on-kind.sh script
THEN I see that HNC no longer gets installed
AND I can still use the product to create orgs and spaces and cf push
Dev Notes

A separate story will cover the updates to installation instructions.
#769 covers the spike work we did to try this out. That work did not include watching existing roles, rolebindings, and secrets to propagate changes. It also didn't include the finalizer we will need to delete namespaces when a CFOrg or CFSpace gets deleted.
Proposed implementation is covered in more detail in this proposal document
Consider switching Eirini namespaced roles to a ClusterRole so that we don't have to propagate them. We will still need the rolebinding to be created in new namespaces, but shouldn't need to watch for updates.
We will need to watch the root namespace for the appearance of new RoleBindings

Changes that need to be done:

Orgs

[ ] Propagation of resources at creation time
[ ] Propagation of added/changed/deleted resources
[ ] Finalizer deletion of managed namespaces

Spaces

[ ] Propagation of resources at creation time
[ ] Propagation of added/changed/deleted resources
[ ] Finalizer deletion of managed namespaces

[ ] remove installing HNC from deploy-on-kind script

[ ] Remove code that waits for HNC in API

[ ] Remove code that waits for HNC in tests

In order to switch the eirini roles to ClusterRoles we may need to make changes to eirini-controller. That might be more appropriate to do as a separate story.

A/C passed


RemoveFromNavigation and then popping PageModel causes ReverseInit being called on removed FreshBasePageModel, instead of the one being popped

Let's say we have FirstPageModel and SecondPageModel in the current page model navigation stack and we are on ThirdPageModel. Then if we call:
CoreMethods.RemoveFromNavigation<SecondPageModel>();
await CoreMethods.PopPageModel(data);

the FirstPage and FirstPageModel (at PopPageModel call) are resolved correctly, but I would expect that the ReverseInit is called on FirstPageModel, which doesn't happen. Instead it is called on SecondPageModel, that was removed in the RemoveFromNavigation call.
The problem lies in RemoveFromNavigation<T>() implementation that does not rewire the link of PreviousPageModel.
Below is the proposed solution which works for us. It keeps the track of the next page model as we are looking for a match in linked list of previous page models. As the match is found, it links the previous and next node of removed one.
/// <summary>
/// Removes specific page/pagemodel from navigation
/// </summary>
public void RemoveFromNavigation<TPageModel> (bool removeAll = false) where TPageModel : FreshBasePageModel
{
    FreshBasePageModel nextPageModel = null;
    foreach (var page in this._currentPage.Navigation.NavigationStack.Reverse().ToList())
    {
        if (page.BindingContext is TPageModel)
        {
            page.GetModel()?.RaisePageWasPopped ();
            this._currentPage.Navigation.RemovePage (page);

            bool isPreviousPageModelMatch = typeof(TPageModel) == nextPageModel?.PreviousPageModel?.GetType();
            if (isPreviousPageModelMatch)
            {
                nextPageModel.PreviousPageModel = nextPageModel.PreviousPageModel.PreviousPageModel;
            }
            if (!removeAll)
                break;
        }
        nextPageModel = nextPageModel == null ? _currentPageModel : nextPageModel.PreviousPageModel;
    }
}

Please let me know what do you think. If you agree with the solution I can send a pull request.

@theberserker Great to see you using the RemoveFromNavigation. That looks like a good solution, I would accept a PR. Thanks

Opened a pull request here: https://github.com/rid00z/FreshMvvm/pull/105

I'm facing this issue as well and still have not found a way of fixing it. I am trying to pass parameters so I really need the reverseInit to work properly.. Any suggestions ?

I'm facing this issue as well and still have not found a way of fixing it. I am trying to pass parameters so I really need the reverseInit to work properly.. Any suggestions ?


chore(security): add talisman

:tada: This PR is included in version 1.39.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


here some errors, what's wrong with this?

   gulp.task('connect',function(){
    	connect.server({
    		root: [outputDir],
    		port: 8000,
       	    livereload: true,
    		// open: { browser: 'Google Chrome'}
   	   });
    });

I just confused this ,and can not understand why !
I following the below video tutorial , but some errors occured!
(May be it out of date?)

Get started with gulp Part 6: LiveReload and web server

youtube video
I'm looking forward to hearing from your  reply as soon as possible!

@AveVlad
AveVladb
@maximilianschmitt
maximilianschmitt
@justinmchase
justinmchase

thanks for your time, in advance!

holy shit!!

you show me the error code!

It should be like this!

gulp.task('connect',function(){???});

gulp.task('connect',function(){
  connect.server({
  root: [outputDir],
  //port: 8080,
     livereload: true
  //open: { browser: 'Google Chrome'}
  //error ?
  });
 });

Is there still a question on this, or can I close it?


Update app-email-alert-api-db-admin for Training environment

Add backend to build app-email-alert-api-db-admin in the Training environment.
Add parameters to select which domain to use with the DNS records (Training
does not use the stack domain). app-email-alert-api-db-admin doesn't have
external ELB/DNS record.

We've discovered this app is no longer being used. It appears it was never migrated to AWS


stats.dart

This issue was originally filed by @financeCoding

Link to your package code:
https://github.com/financeCoding/stats.dart/
http://financecoding.github.com/stats.dart/
Please remember to also check:
Conform your packages to the pub package layout.
Ensure you have a valid pubspec.yaml file.
Pick an OSS license and correctly add it to your package. If you don't have
one, you can always use Dart's BSD-style license.
Host your source code in a public location so we can retrieve it.
Add dartdocs so people can easily use your libraries.
Update your README to contain a getting started guide.

This issue has been moved to dart-lang/pub#329.


First attempt at backwards optimization use

Backwards optimization or input optimization determines an optimal input for a fixed model and fixed target prediction. By seeding the optimization with an existing input, we can see how that input might be optimized to achieve the desired prediction (e.g., a missed extreme), revealing the “the dominant pattern of variability the neural network looks for when making its decisions” ( https://doi.org/10.1029/2019MS002002).
This new notebook demonstrates this for an extreme temperature prediction that was missed by 3 times more loss than the data set average. It reveals what the model would've liked to see, uses EG to see what was important in the optimal input, and contextualizes that with some extra exploratory data analysis. Overall, it appears to lend itself to science-backed information and possible next steps for the use-case.

merging, no real review needed, and we can always iterate on it

merging, no real review needed, and we can always iterate on it


Can not install realtime-talib and use BitVision

After installing BitVision, I found I can not install realtime-talib

And I can not use BitVision.

Python version: 3.7.4
Node version: 10.16.3

@Ericchen159 I faced the same issue, try installing Ta-lib manually and see if you get any error. are you using Linux? this may help you:
(Troubleshoot Ta-Lib)[https://github.com/mrjbq7/ta-lib]

@Ericchen159 I faced the same issue, try installing Ta-lib manually and see if you get any error. are you using Linux? this may help you:
Troubleshoot Ta-Lib


Removed merge conflicts in #353

Greetings @jakeherp thanks for this. What we do is to update the PR with the conflicts directly there https://github.com/reactjs/es.reactjs.org/pull/353. I'll close this but if you want to join Spanish translations you could check the open issues like the Blog for instance and start collaborating https://github.com/reactjs/es.reactjs.org/issues 🚀


AKCORE-9: Add support for describe share groups in bin script.

➜  ccs-kafka git:(AKCORE-9-cli) ✗ bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --describe --group group-share-x                    [23:16:10]

GROUP_ID                                   COORDINATOR_NODE                           OFFSETS
group-share-x                              1                                          topic-share-y-0=>168, topic-share-x-0=>138

➜  ccs-kafka git:(AKCORE-9-cli) ✗ bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --describe --group group-share-x --state            [23:16:21]

GROUP_ID                                   COORDINATOR_NODE                           OFFSETS                                    STATE
group-share-x                              1                                          topic-share-y-0=>170, topic-share-x-0=>138 Stable

➜  ccs-kafka git:(AKCORE-9-cli) ✗ bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --describe --group group-share-x --members          [23:16:26]

MEMBER_ID              GRP_INSTANCE_ID        CLIENT_ID              HOST                   ASSIGNMENT             TARGET_ASSIGNMENT
Y3zeJ4SoT5G84FydoiZA3w                        console-share-consumer /<IP_ADDRESS>
3aNXfh5iTV-JzcSZKYWYPg                        console-share-consumer /<IP_ADDRESS>             topic-share-x:0
Vj_MosWcT6OAYPai-PK-Ww                        console-share-consumer /<IP_ADDRESS>             topic-share-y:0

➜  ccs-kafka git:(AKCORE-9-cli) ✗ bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --describe --group group-share-x --offsets          [23:16:32]

GROUP_ID                                   COORDINATOR_NODE                           OFFSETS
group-share-x                              1                                          topic-share-y-0=>177, topic-share-x-0=>138

Would have been good if there would be a response from the terminal in description which shows the output. It would be then easy to see the print statements and details.

@smjn I needed to make a change to DescribeShareGroupsHandler. Once that was made, this code works fine and I can use kafka-share-groups.sh to display a share group.

Overall, lgtm, but I did not manage to get it to display a share group. There's something wrong in the stack somewhere I think.

Found the issue - Will raise commit


[9.x] Adds support for flattening associative arrays

Added a new helper method called Arr::flattenAssoc(), similar to existing Arr::flatten(), but works for multi-level associative arrays.
Receives the array to flatten and an optional depth (defaults is infinity).
Uses PHP's built-in array_reduce() and returns a single-level associative  array.
Can transforms this:
['a' => 1, 'b' => 2, 'c' => [
    'd' => 3, 'e' => 4, 'f' => [
        'g' => 5, 'h' => 6
        ], 'i' => 7
    ], 'j' => 8
];

into this:
[
  "a" => 1
  "b" => 2
  "d" => 3
  "e" => 4
  "g" => 5
  "h" => 6
  "i" => 7
  "j" => 8
]

Works great on results of an eloquent query (after ...get()-> toArray()):
[
  "name" => "Caroline Casanova"
  "email" =><EMAIL_ADDRESS>  "primary_address" => [
    "postal_code" => "07550-778"
    "address" => "Avenida Inácio"
    "number" => "156"
    "complement" => "Apto 5895"
    "province" => "do Norte"
    "state" => "Maranhão"
    "country" => "Palaos"
    "primary_address" => 1
  ]
]

[
  "name" => "Caroline Casanova"
  "email" =><EMAIL_ADDRESS>  "postal_code" => "07550-778"
  "address" => "Avenida Inácio"
  "number" => "156"
  "complement" => "Apto 5895"
  "province" => "do Norte"
  "state" => "Maranhão"
  "country" => "Palaos"
  "primary_address" => 1
]

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.


Add ability to cancel in progress publish

Currently if a publish is started there is no way for us to cancel it. Ideally the user should be able to cancel a publish if they see something is incorrect in the summary or a publish step.
### Acceptance Criteria
- [ ] Add a button to cancel the in-progress publish

This needs work outside of the publisher. I wonder if we should close it for now and revisit it when/if the work for cancelling inflight deployments on connect happens. We do allow people to dismiss the tracking in the extension now, though the deployment will still proceed on the server.

I wonder if we should close it for now and revisit it when/if the work for cancelling inflight deployments on connect happens.

This sounds like the right move unless we want to pick-up cancellation on Connect soon. If we do, adding the hold label would be preferred to keep this on the board.


Does it work on mkdocs-material?

since there are so many plugins to choose from, I am just wondering whether this plugin will work on mkdocs material smoothly. (currently I cannot test, so I am asking to confirm)
Thanks.

It does, see the README


Windows - " instead of '

testing on Windows; from a bash shell is fine, but the Windows command console requires double quotes in examples like:
gron 'https://api.github.com/repos/tomnomnom/gron/commits?per_page=1'
more an FYI for your docs

Ah, good spot! I didn't know that. Fix incoming :)


Update index.md to check polkaflow first delivery and completed

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix: encrypt sub-command broken by commit 1766cf5 (IDFGH-14062)

input files arg for encrypt sub-command needs to be parsed as a list before passing to the generate() function
resolves #10

Thank you a lot, but we already noticed this and made the patch 😄 Should be live in a few moments.


fetchPolicy config not working for queries

Environment

Operating System: Darwin
Node Version:     v16.18.0
Nuxt Version:     3.0.0
Nitro Version:    1.0.0
Package Manager<EMAIL_ADDRESS>Builder:          vite
User Config:      runtimeConfig, css, vite, components, build, modules, proxy, serverMiddleware, apollo, alias
Runtime Modules<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Build Modules:    -

Describe the bug
I tried to change the fetch policy for apollo in the nuxt.config.ts to cache-first
    defaultOptions: {
      query: {
        fetchPolicy: 'cache-first',
      },
  ...
}

This setting has no effect on the queries, which are always executed with  no-cache
Expected behaviour
Queries should use fetch Policy from defaultOptions.
Reproduction
I use<EMAIL_ADDRESS>and<EMAIL_ADDRESS>Nuxt config:
 apollo: {
    defaultOptions: {
      query: {
        fetchPolicy: 'cache-first',
      },
    },
  ...
}

Query
const { data } = await useAsyncQuery(query)

Additional context
In the https://github.com/nuxt-modules/apollo/blob/v5/src/runtime/composables.ts#L51 the queries get set to no-cache, how can I overwrite this? Is there a way that I can use apollo cache for queries?
Logs
No response

1 Same issue and cannot see the Apollo dev tools either.

I'm oddly having the opposite problem and no-cache is not respected and always hits the cache

Just want to add I'm seeing this generally work for useQuery but not at all for useAsyncQuery


Add Nong Song

Song Name
Metamorphosis
Artist Name
Ok
Source
Youtube
Youtube Link
https://www.youtube.com/watch?v=vce0S8IWq1Q
Direct File Link
No response
Song ID
3421
Extra
No response

Nice

Nice


Fix nil issue with nav-flash & ivy

Raised on discord, without this causes issues with ivy in most cases. (sorry about the dirty commits)

True, yeah I'll change. I did try but it wasn't having it and I didn't expect it to bomb out.

All good! Apologies, haven't had to do a rebase for a while.

No worries. Thanks for the help!


Warmup Trigger not working with Node.js Function Apps

I am trying to add a Warmup Trigger to my function app, but I doesn´t get executed during scale-out.
For testing purposes I have created a Function App with an Elastic Premium App Service Plan.
In the function app, I added the warmup trigger based on the examples from the documentation.
To trigger a scale-out operation of the function app, I manually increased the "Always ready instances" of the function app.
I tested the following scenarios:

.NET Function App

Warmup Function has been triggered successfully during scale-out operation

Node V3 & V4 Function App

Warmup Function has not been executed

Further information:

in the Filesystem Logs I was able to see the following log messages, which at least indicate to me, that the function app should be able to work with the warmup trigger:

Initializing Warmup Extension.
"WORKER_WARMUP_ENABLED": "1"
Found the following functions:Host.Functions.httpTrigger1Host.Functions.warmupTrigger1
Stopping the listener 'Microsoft.Azure.WebJobs.Extensions.Warmup.Trigger.WarmupTriggerBinding+NullListener' for function 'warmupTrigger1' (this happened after reducing the number of instances)

Functions seem to be successfully loaded:

warmup.js
const { app } = require('@azure/functions');

app.warmup('warmupTrigger1', {
    handler: (warmupContext, context) => {
        context.log('Function App instance is warm.')
    },
});

host.json
{
  "version": "2.0",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  }
}

@lorenzadomat I remember that I stumbled over this on certain doc-pages and just had a lookup in the source...Does this start to work when you explicitly name your function 'Warmup' (and maybe change the trigger-name also to 'WarmupTrigger')?
Over at https://github.com/Azure/azure-functions-host, there is some explicit name-check regarding warmup in place (not sure if this is responsible here, though)
From the docs, this matches for .NET functions:

isolated
in-process

All other samples for the different languages use the same naming, despite the one for 'JavaScript on V4'...maybe this is just a missing doc / typo?

@earloc seems to work after renaming the function. Thanks for the hint 👍🏻


Configurable response parsing

Hi @zverok!
Came across your projects through one of your blog posts and it seems we have several project ideas in common. I've been developing DSLs for configuring pipelines for data processing, and in that project, my intention is to build up an abstract data model that can utilize various implementations when running each job.
One of the aspects of the pipeline is to define sources, and an API DSL such as TLAW is exactly what I've envisioned. However, there are some aspects of TLAW that are not very configurable (such as processing the response). Specifically, a use case is to stream the response through the pipeline of transforms and so on.
I'd be interested in leveraging the DSL work you've done to define the API client, and abstract out some aspects such as response processing.
Are you open to this kind of contribution?
Thanks!
~Joel

@joelvh In fact, I am! TLAW is in search-and-experiment stage, and while using it myself for various cases I've also grown a suspicion that response processing is either not as useful as I've hoped, or at least should be "opt-in" thing. Probably something like Rack's/Faraday's "middleware", or something. Probably even that request construction DSL is what TLAW is about, and this response processing trickery could be separated to completely another gem (and in fact, it can be done as a Faraday middleware, probably).
So yeah, I am pretty open to any contributions coming from practice!

OK, great. I will work with what you've established with the DSL (which is really awesome!) and probably abstract out aspects related to instantiating and composing the API client behavior.
Simply forking the repo, running bundle install and then bundle exec rspec spec shows 3 tests failing.
What's the proper way to run the tests, or is Ruby 2.5 not supported?
Thanks.
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's default settings.
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's default settings.
....No need to use its_call without arguments, just it {} will work
F...................................................................No need to use its_call without arguments, just it {} will work
FNo need to use its_call without arguments, just it {} will work
F..........................No need to use its_call without arguments, just it {} will work
..No need to use its_call without arguments, just it {} will work
..No need to use its_call without arguments, just it {} will work
.....................................

Failures:

  1) TLAW::API.define () should to send TLAW::DSL::APIWrapper.new and to send #<InstanceDouble(TLAW::DSL::APIWrapper) (anonymous)>.define
     Failure/Error:
       is_expected.to send_message(DSL::APIWrapper, :new).with(api_class).returning(wrapper)
         .and send_message(wrapper, :define)

     NoMethodError:
       undefined method `call' for nil:NilClass
     # ./spec/tlaw/api_spec.rb:11:in `block (3 levels) in <module:TLAW>'

  2) TLAW::Namespace instance #<endpoint> () should to send #<(unnamed endpoint class): call-sequence: some_ep(foo: nil); docs: .describe>.new and to send #<InstanceDouble(#<(unnamed endpoint class): call-sequence: some_ep(foo: nil); docs: .describe>) (anonymous)>.call
     Failure/Error:
       is_expected
         .to send_message(endpoint_class, :new).returning(endpoint)
         .and send_message(endpoint, :call).with(foo: 'bar')

     NoMethodError:
       undefined method `call' for nil:NilClass
     # ./spec/tlaw/namespace_spec.rb:131:in `block (4 levels) in <module:TLAW>'

  3) TLAW::Namespace instance #<endpoint> initial params () should to send #<(unnamed endpoint class): call-sequence: some_ep(foo: nil); docs: .describe>.new and to send #<InstanceDouble(#<(unnamed endpoint class): call-sequence: some_ep(foo: nil); docs: .describe>) (anonymous)>.call
     Failure/Error:
       is_expected
         .to send_message(endpoint_class, :new).with(apikey: 'foo').returning(endpoint)
         .and send_message(endpoint, :call).with(foo: 'bar')

     NoMethodError:
       undefined method `call' for nil:NilClass
     # ./spec/tlaw/namespace_spec.rb:140:in `block (5 levels) in <module:TLAW>'

Simply forking the repo, running bundle install and then bundle exec rspec spec shows 3 tests failing.

Ugh, sorry, I forgot to push latest commit (pushed now) when I updated my experimental saharspec RSpec addon. Spec should work on the latest commit, checked on Ruby 2.5 too.

@zverok I didn't see you had some commits on master that I needed to rebase. Tests pass now. Thanks!


Display label and placeholder when Select value is empty (vnext)

I'm trying to display placeholder value in a Select if value is not selected. When user selects value it should disappear and the value should replace placeholder. However label should be displayed all the time. I can achieve it in TextField but not in Select.
Here is my try however it doesn't display anything in placeholder.
const errorObj = mapError(props);

    return (<FormControl error={errorObj.error} fullWidth={props.fullWidth} margin={props.margin} required={props.required}>
        <InputLabel htmlFor={props.label} shrink>{props.label}</InputLabel>
        <Select input={<Input id={props.label} placeholder={props.placeholder} margin={props.margin} />} {...props as any} >
            {props.children}
        </Select>
        {errorObj.helperText && <FormHelperText>{errorObj.helperText}</FormHelperText>}
    </FormControl>)

Duplicate of #8813


Implement NumPy sorting routines

xref https://docs.scipy.org/doc/numpy/reference/routines.sort.html
Extracted from #70:

[x] np.sort
[x] np.lexsort
[x] np.argsort
[x] np.msort
[x] np.sort_complex
[ ] np.partition
[ ] np.argpartition

All of these should be relatively doable, though it's worth noting that I don't think XLA currently has efficient partitioning schemes.

@shoyer and @srvasude - Can I help any of these features into JAX?
P.S. I'm a beginner in terms of Open Source Contributions, so checking with you both before getting my hands dirty with this.

Hi @Vaibhavs10 - this is a bit complicated, because sorts must be implemented in terms of XLA operations, which are exposed in JAX in https://github.com/google/jax/blob/master/jax/lax/lax.py.
Of the ones listed above, I think lex_sort and sort_complex would be doable using XLA's sorting of multiple inputs with a custom comparator, but unfortunately lax.sort doesn't yet support passing a custom comparator: that would have to be added as a precursor to this work.

Actually, it looks like np.sort_complex would be a fairly good place to start. lax.sort() already handles complex input correctly as far as I can tell. What would need to happen is to define jax.numpy.sort_complex similarly to sort here: https://github.com/google/jax/blob/c1aeb8b3fe94f2a064ead43d8d9a1c35982d11d5/jax/numpy/lax_numpy.py#L3141-L3151
with modifications that make it follow the semantics of np.sort_complex (things like: real inputs result in complex outputs, etc.
And then add a test of sort_complex similar to the test of sort here: https://github.com/google/jax/blob/c1aeb8b3fe94f2a064ead43d8d9a1c35982d11d5/tests/lax_numpy_test.py#L2569-L2589
If you want to try this out, that would be great! If you need, feel free to open a draft pull request and tag me if you have questions.

Thanks @jakevdp I’ll look at this tomorrow in my day time and raise a PR hopefully by weekend.

Is this issue still open regarding partitioning schemes?

Yes, I don't think anyone has worked on partitions yet. It's not trivial to do, because they're not (yet?) implemented in XLA
It may be possible to implement using scans (similar to how jnp.searchsorted is implemented), but in the end it might actually be faster to implement partitioning via a full sort, since XLA knows how to do this efficiently.

Yes, I don't think anyone has worked on partitions yet. It's not trivial to do, because they're not (yet?) implemented in XLA
It may be possible to implement using scans (similar to how jnp.searchsorted is implemented), but in the end it might actually be faster to implement partitioning via a full sort, since XLA knows how to do this efficiently.

can you please tell me how to get started with implementing this. I am a beginner to open source.

Hi - thanks for the interest! Are you asking how to get started with open source github contributions in general, or how to get started with JAX in particular?

Hi - thanks for the interest! Are you asking how to get started with open source github contributions in general, or how to get started with JAX in particular?

Thanks for the quick reply. I want to get started with implementing the partition and argpartition functions.

OK - as mentioned above, I don't know if there's really any good way to do that at the moment. XLA has no partial sorts, so the only option would be to implement e.g. a partial quicksort from scratch using control flow operators. A similar approach for implementing binary search was used for jnp.searchsorted, which you can see here, although sorting is somewhat more complicated. Since lax control flow operations tend to be fairly slow, I'm not sure if it would be worth going down that route; it may be better to just wait and hope that XLA implements partial sorts at some point in the future.

OK - as mentioned above, I don't know if there's really any good way to do that at the moment. XLA has no partial sorts, so the only option would be to implement e.g. a partial quicksort from scratch using control flow operators. A similar approach for implementing binary search was used for jnp.searchsorted, which you can see here, although sorting is somewhat more complicated. Since lax control flow operations tend to be fairly slow, I'm not sure if it would be worth going down that route; it may be better to just wait and hope that XLA implements partial sorts at some point in the future.

Got it. would love to have a shot at some other beginner issue. Can you suggest something?

It looks like the number of good first issue issues has dwindled recently... there may be a few routines in #70 that would be a good. np.insert could be an interesting one, also np.polyfit and np.polydiv

Just want to mention that argpartition is especially useful when it comes to implementing top-N accuracy metrics.

For that application, you might check out lax.top_k and lax.approx_max_k.

oooh thank you @jakevdp!!


[gardening] Prefer Array(seq) over seq.map { $0 }

See also https://github.com/apple/swift-corelibs-foundation/pull/867.

@swift-ci please smoke test


training error

when I change the sparseconv to normal conv, errors occurred below:
Traceback (most recent call last):
File "pytorch/train.py", line 647, in
fire.Fire()
File "/home/users/wenyong.zheng/.local/lib/python3.6/site-packages/fire/core.py", line 127, in Fire
component_trace = _Fire(component, args, context, name)
File "/home/users/wenyong.zheng/.local/lib/python3.6/site-packages/fire/core.py", line 366, in _Fire
component, remaining_args)
File "/home/users/wenyong.zheng/.local/lib/python3.6/site-packages/fire/core.py", line 542, in _CallCallable
result = fn(*varargs, **kwargs)
File "pytorch/train.py", line 402, in train
raise e
File "pytorch/train.py", line 249, in train
ret_dict = net(example_torch)
File "/home/users/wenyong.zheng/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "../second/pytorch/models/voxelnet.py", line 679, in forward
voxel_features, coors, batch_size_dev)
File "/home/users/wenyong.zheng/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "../second/pytorch/models/voxelnet.py", line 331, in forward
ret = scatter_nd(coors, voxel_features, output_shape)
File "../torchplus/ops/array_ops.py", line 20, in scatter_nd
ret[slices] = updates.view(*output_shape)
RuntimeError: tensors used as indices must be long or byte tensors

the dense model is not tested, i will fix this tonight.

@traveller59 I also have this problem，do you solved this problem？


Update Dataverse guides to include frontend security advices, practices and procedures

Overview of the Feature Request
When including the SECURITY.md file, based on the same as the Dataverse core repository, I realized that it will be necessary to update the security-related Dataverse guides with new Frontend topics. In particular:

https://guides.dataverse.org/en/latest/installation/config.html#securing-your-installation
https://guides.dataverse.org/en/latest/developers/security.html

Possibly there exist other pages that will need to be updated as well.
Note: This issue will be worked on in the future, when the new frontend is standardized. For this reason, it is not assigned to any MVP milestone yet.
What kind of user is the feature intended for?
Dataverse frontend developers
What inspired the request?

Including SECURITY.md for #2

What existing behavior do you want changed?
N/A
Any brand new behavior do you want to add to Dataverse Frontend?
N/A
Any related open or closed issues to this feature request?

#2

GitHub added a new feature AFTER we created the SECURITY.md file in the main repo: the ability to report security issues privately in the repo itself: https://github.blog/changelog/2022-11-09-privately-report-vulnerabilities-to-repository-maintainers/ . We might want to try using this.
Our default is to ask people to email<EMAIL_ADDRESS>which goes into a private ticket tracker (RT, Request Tracker).


Bug in example in 17.4.6, Interface member access

The first example in this section has a bug. Here is that example and surrounding narrative:

Interface members are accessed through member access (§11.7.6) and indexer access (§<IP_ADDRESS>) expressions of the form I.M and I[A], where I is an interface type, M is a method, property, or event of that interface type, and A is an indexer argument list.
For interfaces that are strictly single-inheritance (each interface in the inheritance chain has exactly zero or one direct base interface), the effects of the member lookup (§11.5), method invocation (§<IP_ADDRESS>), and indexer access (§<IP_ADDRESS>) rules are exactly the same as for classes and structs: More derived members hide less derived members with the same name or signature. However, for multiple-inheritance interfaces, ambiguities can occur when two or more unrelated base interfaces declare members with the same name or signature. This subclause shows several examples, some of which lead to ambiguities and others which don’t. In all cases, explicit casts can be used to resolve the ambiguities.
Example: In the following code
interface IList
{
    int Count { get; set; }
}

interface ICounter
{
    void Count(int i);
}

interface IListCounter : IList, ICounter {}

class C
{
    void Test(IListCounter x)
    {
/*17*/  x.Count(1);             // Error
/*18*/  x.Count = 1;            // Error
/*19*/  ((IList)x).Count = 1;   // Ok, invokes IList.Count.set
/*20*/  ((ICounter)x).Count(1); // Ok, invokes ICounter.Count
    }
}

the first two statements cause compile-time errors because the member lookup (§11.5) of Count in IListCounter is ambiguous. As illustrated by the example, the ambiguity is resolved by casting x to the appropriate base interface type. Such casts have no run-time costs—they merely consist of viewing the instance as a less derived type at compile-time.

The only compilation error reported is for Line 18:

Library.cs(18,9): error CS1656: Cannot assign to 'Count' because it is a 'method group'

If I disable that line, no error results from Line 17, in any csc compiler version since V1!
BTW, this example has been in the Ecma spec since V1. Perhaps this case was an error in a pre-V1 release when the first MS spec was first written!

Good find. And eek!

VS 17.11.2 accepts x.Count(1); and calls the method member, which I think is the right thing to do, so there is no error. Here's why:
12.5 Member lookup|12.5.1 General states:

A member lookup is the process whereby the meaning of a name in the context of a type is determined. A member lookup can occur as part of evaluating a simple_name (§12.8.4) or a member_access (§12.8.7) in an expression. If the simple_name or member_access occurs as the primary_expression of an invocation_expression (§<IP_ADDRESS>), the member is said to be invoked.

From that I deduce that x.Count(1); is an invocation_expression, and that member Count is being invoked.

If a member is a method or event, or if it is a constant, field or property of either a delegate type (§20) or the type dynamic (§8.2.4), then the member is said to be invocable.

This indicates that the method count is invocable, and the property Count is not.
Further down, in the bullet list, we have:

Next, if the member is invoked, all non-invocable members are removed from the set.

which causes the property to be removed, leaving only the method, making the use unambiguous.
If my analysis is correct, we need to remove lines 17 and 19, and adjust the description accordingly.
Lines 18 involves the following spec text, with the final bullet prevailing:

Finally, having removed hidden members, the result of the lookup is determined:

If the set consists of a single member that is not a method, then this member is the result of the lookup.
Otherwise, if the set contains only methods, then this group of methods is the result of the lookup.
Otherwise, the lookup is ambiguous, and a binding-time error occurs.

@BillWagner A compiler-message/doc issue: The line 18 error is

Library.cs(xx,yy): error CS1656: Cannot assign to 'Count' because it is a 'method group'

however, the online help text for CS1656 says:

Cannot assign to 'variable' because it is a 'read-only variable type'

which is quite different.
Also, the situation this example presents is nothing like the two examples shown, so maybe this scenario could be added there. If you agree, I can write up some words for that and submit a PR; let me know.


Compliance with AWS QuickStart architecture requirements

Source: https://aws-quickstart.github.io/reqs.html#min

[ ] Multi-AZ architecture
[x] Support for the majority of AWS Regions
[ ] New VPC and existing VPC deployment options
[ ] Product instances in private subnets
[ ] NAT gateways for outbound internet access from private subnets
[ ] Marketplace AMIs whenever possible; no prebaked AMIs
[ ] AMI mappings; no hardcoded AMIs
[x] User-friendly parameter labels and groups
[ ] CIDR block lockdown for external admin access
[x] Security groups with principle of least privilege
[ ] No software bits with deployment
[x] No hardcoded passwords
[x] No sensitive data in EC2 instance user data or other clear text
[ ] No use of <IP_ADDRESS>/0 for open remote management access
[x] No resources created automatically outside stack

Also worth taking a look at https://www.cloudconformity.com/conformity-rules/

Finished at 567ba214877b1b81155b82ea134d527812edb363


Describe use cases and possible requirements for "enabling self-service of logical clusters"

A key component of large Kubernetes clusters is shared use, where the usage pattern might vary from externally controlled (via gitops / existing operational tools) to a permissive self-service model.  The most common partitioning model in Kubernetes is namespace, and the second most common model is cluster.  Self-service is currently limited by the set of resources that are namespace scoped for the former, and by the need to parameterize and configure multiple clusters consistently for the later. Cluster partitioning can uniquely offer distinct sets of APIs to consumers. Namespace partitioning is cheap up until the scale limits of the cluster (~10k namespaces), while cluster partitioning usually has a fixed cost per cluster in operational and resource usage, as well as lower total utilization.  Once a deployment reaches the scale limit of a single cluster, operators often need to redefine their policies and tools to work in a multi-cluster environment. Many large deployers create their own systems for managing self-service policy above their clusters and leverage individual subsystems within Kubernetes to accomplish those goals.
Explore the goal

logical clusters + api composition + policy should allow organizations to provide teams "chunks of capacity" that they can use for workloads and services in a more comprehensive way than a single Kubernetes cluster.

Output

an investigation doc

Hoping to have a google doc soon to share, been canvassing various public projects, user stories I've heard, ecosystem reports, and asking questions ilke "how can self-service/tenancy for teams be abstracted like containers abstracted linux processes and kube abstracted rollouts"

@smarterclayton bump on question from @ncdc ☝

Looks like it does; closing.


Allow to use #[serde(getter = "...")] on non-remote-wrapper types

Restrict getters only to #[serde(remote = "...")] types looks like artificial restriction with no rationale explaining. This PR eliminates it.
Closes #2777

Why not? It is currently possible, but you need to use hacks. If in the end there is no difference, why should we put a spoke in wheel?
I could understand you if it was a completely new attribute, but it is already there and works, just for some reason it is limited in use for no reason.


[Help may not issue]Connection is not subscribed and is idle

Hi,
Thanks for your library.
Im facing some issues with streaming only, the rest working just fine ! (great work though)
I tried this
betfair_socket = trading.streaming.create_stream(unique_id=1) betfair_socket.start(async=False) betfair_socket.authenticate() betfair_socket.subscribe_to_markets(market_filter={'eventTypeIds':['1'],                                                    'inPlayOnly': True,                                                    'marketTypes':["MATCH_ODDS"]},                                     market_data_filter={'fields': ['EX_BEST_OFFERS']})
and I'm ending up with this result:
`1 {"op":"connection","connectionId":"006-051016081016-209520"}
None {"op":"status","statusCode":"FAILURE","errorCode":"TIMEOUT","errorMessage":"Connection is not subscribed and is idle: 15000 ms","connectionClosed":true,"connectionId":"006-051016081016-209520"}
`
Any help pls where im messing up ?
Once more thanks again for your effort.
Regards

Hi,
Glad its working for you, the reason you are being closed is because async needs to be True if you are not creating a separate thread for it in order to handle the socket:
`betfair_socket = trading.streaming.create_stream(unique_id=1)
betfair_socket.start(async=True)
betfair_socket.authenticate()
betfair_socket.subscribe_to_markets(market_filter={'eventTypeIds':['1'], 'inPlayOnly': True, 'marketTypes':["MATCH_ODDS"]}, market_data_filter={'fields': ['EX_BEST_OFFERS']})
while True:
time.sleep(0.5)
`
The idea being that you can create a separate thread if required to catch any errors, without this the your script blocks at .start due to the while loop that is polling the socket.
Just so you know, I am not using streaming in production and still lacking on test coverage.

Thanks ,
Sadly im getting another msg [NOT_AUTHORIZED]
So i suppose my AppKey, needs an upgrade or somthing :)
Anyway thanks a lot !
Regards

You need to register, send an email to Neil<EMAIL_ADDRESS>


Add an option to extend list of group members

Hi,
I wasn't able to find any way of adding another user to an existing group with already defined memberships resource. New memberships only overwrite the list of members for a group.
This way we cannot contribute to a single group with users from different projects.
I was expecting something similar to an exhaustive property which is present in group roles resource or user roles resource.

Example?

apiVersion: user.keycloak.crossplane.io/v1alpha1
kind: User
metadata:
  name: user
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    enabled: true
    realmId: acmecorp-00001
    username: user
    initialPassword:
      - temporary: false
        valueSecretRef:
          key: credentials
          name: initial-user-global-acmecorp-00001-admin-input-credentials
          namespace: keycloak-system
---
apiVersion: role.keycloak.crossplane.io/v1alpha1
kind: Role
metadata:
  name: role
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    name: role
    realmId: acmecorp-00001
---
apiVersion: group.keycloak.crossplane.io/v1alpha1
kind: Group
metadata:
  name: group
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    name: group
    realmId: acmecorp-00001
---
apiVersion: group.keycloak.crossplane.io/v1alpha1
kind: Roles
metadata:
  name: group-roles
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    realmId: acmecorp-00001
    groupIdRef:
      name: group
    roleIdsRefs:
      - name: role
---
apiVersion: group.keycloak.crossplane.io/v1alpha1
kind: Memberships
metadata:
  name: membership
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    realmId: acmecorp-00001
    groupIdRef:
      name: group
    members:
      - user

This creates user with assigned group with single role. All good.
But if later somthing like this comes:
apiVersion: user.keycloak.crossplane.io/v1alpha1
kind: User
metadata:
  name: another-user
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    enabled: true
    realmId: acmecorp-00001
    username: another-user
    initialPassword:
      - temporary: false
        valueSecretRef:
          key: credentials
          name: initial-user-global-acmecorp-00001-admin-input-credentials
          namespace: keycloak-system
---
apiVersion: group.keycloak.crossplane.io/v1alpha1
kind: Memberships
metadata:
  name: another-membership
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    realmId: acmecorp-00001
    groupIdRef:
      name: group
    members:
      - another-user

The user another-user is given the group membership, however the user user loses it. The another-membership resource overwrites the existing list of group members.

does something like that work in terraform? 🤔

Frankly, I have no clue (not a terraform specialist).
I was hoping that the functionality is very similar to assigning roles directly to users with Roles.user.keycloak.crossplane.io resource having the exhaustive flag. Now reading the docs again it claims that "manually" added roles aren't removed with it, so now I'm not really sure that even this does what I expected from it.

Hm, i'll check that out and tell you asap 😄
On Mon, Jul 8, 2024, 09:30 vladimirblahoz @.***> wrote:

Frankly, I have no clue (not a terraform specialist).
I was hoping that the functionality is very similar to assigning roles
directly to users with Roles.user.keycloak.crossplane.io
http://Roles.user.keycloak.crossplane.io resource having the
exhaustive flag. Now reading the docs again it claims that "manually"
added roles aren't removed with it, so now I'm not really sure that even
this does what I expected from it.
—
Reply to this email directly, view it on GitHub
https://github.com/crossplane-contrib/provider-keycloak/issues/134#issuecomment-2213240379,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC3JPMJCTGYA3AF2E77JMB3ZLI52JAVCNFSM6AAAAABKQF3QGOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJTGI2DAMZXHE
.
You are receiving this because you commented.Message ID:
@.***>

I just checked that this (the situation with roles) works:
apiVersion: user.keycloak.crossplane.io/v1alpha1
kind: User
metadata:
  name: user
  labels:
    username: user
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    enabled: true
    realmId: acmecorp-00001
    username: user
    initialPassword:
      - temporary: false
        valueSecretRef:
          key: credentials
          name: initial-user-global-acmecorp-00001-admin-input-credentials
          namespace: keycloak-system
---
apiVersion: keycloak.ataccama.com/v1alpha1
kind: Role
metadata:
  name: mmm-role-1
  namespace: default
spec:
  identifier:
    tenant: acmecorp-00001
    label: role1
    component: mmm
---
apiVersion: keycloak.ataccama.com/v1alpha1
kind: Role
metadata:
  name: mmm-role-2
  namespace: default
spec:
  identifier:
    tenant: acmecorp-00001
    label: role2
    component: mmm
---
apiVersion: user.keycloak.crossplane.io/v1alpha1
kind: Roles
metadata:
  name: user-roles1
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    realmId: acmecorp-00001
    roleIdsRefs:
      - name: mmm-acmecorp-00001-role1
    exhaustive: false
    userIdRef:
      name: user
---
apiVersion: user.keycloak.crossplane.io/v1alpha1
kind: Roles
metadata:
  name: user-roles2
spec:
  providerConfigRef:
    name: keycloak-default
  forProvider:
    realmId: acmecorp-00001
    roleIdsRefs:
      - name: mmm-acmecorp-00001-role2
    exhaustive: false
    userIdRef:
      name: user

So apparently Roles resource is able to append the roles and you can have as many of them as you want targeting the same user as long as the exhaustive flag is set to false

Well,  https://registry.terraform.io/providers/mrparkers/keycloak/latest/docs/resources/group_memberships is designed to be  a single source of truth it seems.  As long as we do not implement our own controllers and rely on upjet, that's it's not possible to change that behaviour. --> i'll note that, as i'm in the process of writing some selfmanaged controllers for some resources

If you say,  "This way we cannot contribute to a single group with users from different projects" i'm wondering what your IAM structure is when using the keycloak provider and how you manage the roles.

If you, e.g, use something like ArgoCD as gitops tool to sync everything into your cluster,  you could combine that with a little helm wrapper and supply  values.yaml files from different repositories easily.  (See here

If you don't use gitops tools.  you might be able to write a pipeline that scrapes all the information from different project repos and creates the resource you need / build a hook that triggers some umbrella repository

Thanks for the investigation!
First of all now that I have tested that the direct roles assignment works, we have a workaround and that's absolutely fine for now (since it sounds like quite a lot of work to make this work using Memberships).
We are using ArgoCD as gitops tool, so I'll check that link if that is an interesting solution for us 👍
🙇♂


added new helper function to calculate capital_asset_pricing_model

Created a new helper function that can calculate the capital asset pricing model by taking risk_free_interest_rate, beta_of_security, and expected_market_return as inputs.

Made changes to both endpoint.md and main.py files

Congrats @ShamanthVallem 🎉, your PR got merged. To be eligible for the swags from ClueLess. Follow these steps.

Star this repository. ⭐
SignIn / SignUp in [ClueLess Website](https://www.clueless.tech/)
Go to the profile section from the top right-hand corner.

Comment your profile URL under this comment tagging the person who merged your PR.
That’s it. Follow the leaderboard!
And don’t forget to share your achievement with your friends.


CI/CD Docker

Description
Refactors the CI/CD files to use the new official Docker image. The image is currently hosted as jammont/isofit on dockerhub. We can leave this as is temporarily until we have an official isofit account on Dockerhub that joins their open source program.
Additionally, this PR updates the tensorflow package to be installed via pip instead of Conda. It is recommended by the tensorflow documentation to use pip as well as there were sudden issues with the micromamba in the docker container installing isofit. Instead of resolving why it couldn't, we should just follow best practices and install via pip:

Note: Do not install TensorFlow with conda. It may not have the latest stable version. pip is recommended since TensorFlow is only officially released to PyPI.

Changes

✨ New addition
🐛 Resolved a bug
🗒 Documentation

Description

:sparkles:
Replaced container in example workflows to use new official image

:sparkles:
Updated tensorflow to be pip installed in both the conda-environment.yml and Dockerfile

All of the checks are failing as the currently hosted isofit image only supports linux/arm64 whereas github uses linux/amd64. These should all be passing once the amd64 image is posted. In the meantime, the files are ready for review.

@jammont This looks great! Thanks for implementing this huge step forward. No suggested edits from my side. As you mentioned it above, are you working on getting the amd64 image posted?

The git workflows are now working and passing. This is now ready for merge.
And yes, both AMD64 and ARM64 images are posted at this time. More to come soon.

Thanks, @jammont, this is excellent!

Looks great.  Thanks James!


Convert to es5

I also love new cool es6 syntax. But in my apps i really need to exclude node_modules from babel parse. Parse node_modules decrease speed of build.
Can you convert this to es5? Or if I do, will you merge PR?

One option would be to keep the sources in es6 but add a prepublish script which automatically converts the code to es5.
@slonoed In the meantime you could define your babel step like this to prevent loading all of node_modules (assuming you're using webpack, but I guess other tools support similar patterns):
{test: /\.jsx?$/, loader: "babel?experimental", include: [path.resolve(__dirname, "src"), path.resolve(__dirname, "node_modules/react-chartist")]},

Sorry for the inconvenient, I'll compile it to es5 and publish it again.

Hi @slonoed I included the es5 code inside the dist/index.js, and set it as the main entry file, you can just require it in your code now.
By the way the version is 0.5.0. Let me know if it works for you, thanks.

Great, works for me.

@razagill Thanks for the feedback. I'll close this issue then.


Feature scrolling with j and k

To navigate a website with the keyboard, scrolling is essential.

See this file.


Spike: Consider delaying attestation production if block validation is happening

https://discord.com/channels/697535391594446898/697539289042649190/1032740792126681108
i have a proposal for teku that would improve validator performance especially when paired with the sister project besu which sometimes suffers from slow block import times
but even with other EL nodes i think this small change is even beneficial for reaching consensus
in the scenario where CL (teku) already has the block, it has confirmed that it is valid from a CL point of view but EL (besu) is still busy validating but the attestation deadline passes - teku should WAIT for engine_newPayloadV1 to return instead of rushing out the attestation

It's ok to be optimistic about EL validity if CL is valid, proposer only has one attempt at this without getting slashed and would harm exclusively himself if EL payload is not valid
If we're optimistic about EL validity then the rushed attestation will most certainly be for the wrong block_root and result in lowered rewards (possibly even a penalty if the previous block had low votes and leads to us also getting the source wrong)
There are 4 full seconds to get the attestation to the aggregator - in my experience that's more than enough and it's totally fine to eat into that a little
Even if this leads to missing the aggregation deadline a late (1 block) but correct block_root vote yields higher rewards than a timely but wrong vote
The consensus model plans for votes coming in late and handles it gracefully - I think this and the reward structure shows that it favors correct votes over timely (rushed) votes

please consider, thanks

I #6595 should be implemented first, so when we can decide to apply the delay only for block that will meet the late block threshold, otherwise we risk to vote on a block that have high change to be reorged out but the next proposer.


The Xray APIs should not use ArtifactoryClientBuilder

Describe the bug
Following the merge of https://github.com/jfrog/jfrog-client-go/pull/211 we should refactor the code, so that the new Xray APIs introduced in this PR do not use the ArtifactoryClientBuilder. We should either create a generic builder for all JFrog services, or alternatively create client APIs for Xray.
See the following comment as part of the above PR:
Now that ArtifactoryClientBuilder also needs to be used for Xray, we should rename it to HttpClientBuilder.
We should also move the httpclient package, which is currently located under the artifactory directory, to be one level up, next to the artifactory dir.
rthttpclient should be renamed to httpclient, or if the the httpclient is already taken as an alias in the file, maybe jfroghttpclient.


Update index.html

added favorite site list

I have no clue what is going on

I have no clue what is going on


Modules with no react exports are treated as fast-refresh boundaries

Describe the bug
In Dan's original comment describing fast-refresh (https://github.com/facebook/react/issues/16604), he says only files that only export components should be treated as boundaries.  But, it seems that the isRefreshBoundary function in @vitejs/plugin-react is much more lax.  It seems to only check that export names are capitalized.  But, they don't need to be functions, they can be exported objects.  See the reproduction for an example of how this causes problems.
One approach we could take which might be more robust, would be to perform a runtime check using isLikelyComponentType from the react-refresh runtime library.  That has more robust checks than I think we can accomplish statically in the AST.  https://github.com/PepsRyuu/rollup-plugin-react-refresh/blob/master/index.js is an example of a project that takes roughly this approach.
Reproduction
https://stackblitz.com/edit/vitejs-vite-b4yvhl?file=src/not-react-components.jsx
System Info
N/A

Used Package Manager
npm
Logs
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

I'm willing to attempt a PR to use isLikelyComponentType if it's agreed that's a viable option by the core team.

I've hit a snag, unfortunately.  I was able to add a conditional import.meta.hot.accept() which checks if all the exports are react components, using isLikelyComponentType, and that part worked great.  But I found that changes were still not bubbling up to the parent, and tracked down the reason:
https://github.com/vitejs/vite/blob/71eaacfde79a0044fcd3654afbbf4278fbac9bcb/packages/vite/src/node/plugins/importAnalysis.ts#L399-L419
It appears that the import analyzer only looks for the string import.meta.hot.accept() in the source code, and does not consider that it might be conditional.  So, even when my if block would prevent the self-accept from happening, vite still does not allow the change to bubble to a parent and be dealt with there.
I tried to calculate a list of "acceptable" exports and use import.meta.hot.acceptExports (which would be really cool to do!), but again hit a problem because it's looking for a static list of export names, not a dynamic variable calculated at runtime.
I'd love to hear any thoughts from @patak-dev or @aleclarson (or anyone else!) about what could potentially be done here.


Replace customers in a user list request

There is an example how to add customers to a specific user list (https://github.com/googleads/google-ads-java/blob/50a65751c0b714cadcdd7c3ff1f3eebc65090f6d/google-ads-examples/src/main/java/com/google/ads/googleads/examples/remarketing/AddCustomerMatchUserList.java), but how to replace customers fully?
Looks like we should do something like this:

Run a job to wipe a user list with a single operation:

OfflineUserDataJobOperation.newBuilder().setRemoveAll(true).build()

Add new users to the user list with operations:

UserData ud = ...;
OfflineUserDataJobOperation.newBuilder().setCreate(ud).build()

Is it true? And how do offline jobs execute in google backend? Can be the situation where the second job executes earlier than the first?

These jobs could execute in parallel, so you should wait for the first job to complete before starting the second.
In terms of which request body to set here I'm not exactly sure. I'd suggest reaching out on our forum where our support team will be able to point you in the right direction.

Yep, that is exactly what I did in the first place. They suggested to reach you for this question: https://groups.google.com/g/adwords-api/c/Gqum3kL6qpk/m/ZR1yvZ6zAgAJ

Oh I see. Yes, so your syntax above is correct.
OfflineUserDataJobOperation.newBuilder().setRemoveAll(true).build()

This will remove all users from the user list. Note that this only supports customer match user lists.

One additional note: I just confirmed that if your goal is to remove all members and add new members, then you can do so in a single job by ordering the operations in the job as:

The operation with removeAll = true
The create operations

The first operation will mark all existing members in the list for removal, but will not mark the members from the subsequent create operations for removal.
Thanks

@jradcliff Thank you for a solution but there is an obstacle.
If we want to remove all members and add new, for example, 199,999 members, then we have to split adding in two batches (api can process only 100k operations in a single request).
So,
the first batch contains 1 removeAll operation and 99,999 members to add after that (100k operations)
the second batch contains 100k members to add
If the api doesn't support ordering by time, then they execute in parallel and it will be complete mess.

I think there may be a way to satisfy your use case using the external_id attribute of OfflineUserDataJob, but I need to confirm the behavior. I'll investigate and get back to you once I have an answer.
Thanks,
Josh

I've got good news: the 100k limit applies to each AddOfflineUserDataJobOperations request, not to the job overall (I'll update our documentation accordingly). This means you do not need to split the upload into multiple jobs. Instead, you can do this sequence of steps:

AddOfflineUserDataJobOperations where the first operation has removeAll = true and remaining 99,999 ops are create.
AddOfflineUserDataJobOperationswith 100,000 more create operations.
AddOfflineUserDataJobOperationswith 100,000 more create operations.
...repeat until all operations added to the job.
RunOfflineUserDataJob to start the job and apply all operations.

I think that covers your use case, but let feel free to reopen this issue if you have additional questions.
Thanks,
Josh

Unfortunately, this solution doesn't work.
I don't know to whom I should write about a strange behaviour, so let's start from here :).
I made the exact same sequence of steps and launched the job to apply all operations.
After a day it looked like it worked. I could get mine 'offline_user_data_job' and 'user_list'.
Also I saw that the user_list has size_for_display = 130k and size_for_search = 80k, but the offline_user_data_job was in the RUNNING status.
Next day I checked again offline_user_data_job's for the user_list and the user_list itself and it surprised me:

size_for_display is 130k, but size_for_search is 0 now.
there are two offline_user_data_job's now for the user_list. one (mine) is still in status RUNNING and the second is in status SUCCESS.

Is it expected?

Sure, ids:
6651197514 (RUNNING)
6664062721 (SUCCESS)

It looks like your job may be stuck in the running state. I've reached out to the folks who own this service and will get back to you.
b/184946908

Hi, @nwbirnie, Is there any update?

Hey @awant yes there is. We fixed a bug where uploads to mobile user lists were not completing. You should now see that both jobs you mentioned are in status SUCCESS.
However, looking at the userlist in question it would appear that the same anomaly with the size is present. It appears very large on display but has < 100 users on search. I'm guessing that this isn't expected? If not, could you try the upload again? It's possible that something went wrong that may now be fixed.

Also did you previously see the same (or similar) userlist have a larger size on search?


Verify Certificate Transparency SCT signatures

Provides functions for verifying SCT (Signed Certificate Timestamp) signatures.

Comments addressed, but won't build until PR #668 is merged now.

That was merged, just forgot to mark closed.

I've rebased this, now that the previous PR has been merged. It has one issue, which is that ct_test now depends on a config file "test/ct/log_list.conf", but I can't see an obvious way to specify that in build.info. Therefore, if doing a unified, out-of-source build, the test fails because it can't find that file.

@levitte, can you help with the dependency?

I'll have a look

Ah.  When running test/ct_test, you need to set the environment variable CTLOG_FILE with the path to the appropriate log_list.conf.  That need to be done in test/recipes/80-test_ct.t, and would probably mean something like this:
diff --git a/test/recipes/80-test_ct.t b/test/recipes/80-test_ct.t
index 616bb17..e81d5de 100644
--- a/test/recipes/80-test_ct.t
+++ b/test/recipes/80-test_ct.t
@@ -2,5 +2,7 @@

 use OpenSSL::Test::Simple;

+$ENV{OPENSSL} = srctop("test", "ct", "log_list.conf");
+
 simple_test("test_ct", "ct_test");

Thanks, I've made that change to the next PR (it turns out that while "log_list.conf" is added in this PR, it isn't used by tests until the next PR, #665).

a bunch of style nits and a couple of bigger questions.  also need docs.  i'll re-review later today or tmorrow when you post a new commit.  ping me here :)

Comments addressed, except for docs. Am I ok adding those as a separate PR?

Yes, as long as you promise to do so :)

Absolutely, scout's honour :)

some trivial whitespace issues, but this looks real good.  thanks!  let me know when you fixed those and i will plus-one.  (and then you can nudge @benlaurie to do the same :)

I've addressed the last of the comments now.

 Is this good to be merged? I'll rebase #665 once it is and then that can be reviewed.

Yes, it will be merged shortly.  We just had some other things in the way first :)

Thanks, #665 is now rebased and ready for review.


chore(scanner): Use db init bundles

Description
Add a workflow that runs one loop of the updater to populate a PSQL DB and dump its content into a "database init bundle," which is then copied into the Scanner V4 DB image so that the initialization container can populate the databases before marking the DB POD "Ready."
The change is optional, so DB image builds that don't want to include the init bundle will work as before.
This will speed up the time for the vulnerability matcher to start successful scans when the DB does not exist (fresh install or when deleted).
Tests show that the time comes to ~15m on SSD installations, while on disks, it can take up to 1h.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change
Testing Workflow
https://github.com/stackrox/stackrox/actions/runs/7823520401
Manual

make db-image SCANNER_IMAGE_DB_INIT_BUNDLE_ENABLED=true
podman push quay.io/jvdm/stackrox-dev:scanner-v4-db (use whatever repo you have)
Deploy Stackrox with V4 enabled. Use SSD storage:scannerV4:
  db:
    persistence:
      persistentVolumeClaim:
        storageClass: "ssd-csi"

Replace DB images.
Inspect logs and services health.

Init script start/end:
     state:
       terminated:
         containerID: cri-o://a11d918110e799fa5ab86188bc0aad802b0c245e9dd64723a3b80d7795b4297c
         exitCode: 0
         finishedAt: "2024-02-08T05:36:28Z"
         reason: Completed
         startedAt: "2024-02-08T05:26:04Z"

Total load time: 10m 24s
Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

/test ocp-stable-scanner-v4-qa-e2e-tests


Yatin: Watch (?-i:Cpoop70VAkc)(?# youtube)

Yatin requests the watch of the watch_keyword (?-i:Cpoop70VAkc)(?# youtube). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

(?-i:Cpoop70VAkc)(?# youtube) has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ


Renaming nodes in a scene inherited by others can cause corruption and crashes.

Godot version
f8818f85e6c43cdf1277e8ae85eba19ca0a003b0
System information
Windows 11
Issue description
Current renaming a node in a scene which is inherited by others can completely corrupt scenes which inherit it which then subsequently lead to the engine crashing.
Steps to reproduce
Open the example project.
Open BaseScene.tscn
Rename the BaseNodes to something else
Open TestScene.tscn
Minimal reproduction project
scene_composition.zip

I think I need to break this into two problems:

The editor already attempts to recover nodes where the parents have gone missing, so the first priority should be figuring out why this causes a crash.
Probably a more longer term question that will require proposals and planning, I would say a later task should be figuring out if we can not only not make it crash, but allow inherited/instanced scenes with nodes placed in-between them 'recover' from name changes, whether this is by taking advantage of the fact that we track dependencies, modifying the files directly, or introducing some kind of UID system for nodes.


[CSLD-119] Enable swagger & client doc autobuild by travis

My only purpose is to update https://cardanodocs.com/technical/wallet/api/
And this PR seems to be the easiest way to do that

Afaik haddock is not generated because it took too long. This will need to wait on https://github.com/input-output-hk/cardano-sl/pull/1192

I omitted haddock and remained executables-based documentation builders, hope them to be very fast

Okay, I've just found out that travis job limit is already fully depleted, chances that this will pass are phantom


Crash in libopencv_objdetect.so when detectObject are called multiple times

Use detectObject function on ubuntu node js to do face detection from usb camera.
The sample code is in https://github.com/drejkim/face-detection-node-opencv
detectObject() function need 1 secs to process one 640x480 image for face detection on raspberry pi.
However for frame rate higher than 1, detectObject is called multiple times before first image finish.
That will cause libopencv_objdetect.so crash.
Analyze ram dump as below call stack.
#0  0x761c6758 in ?? ()
   from /usr/lib/arm-linux-gnueabihf/libopencv_objdetect.so.2.4
#1  0x761c708e in cv::HaarDetectObjects_ScaleImage_Invoker::operator()(cv::Range const&) const () from /usr/lib/arm-linux-gnueabihf/libopencv_objdetect.so.2.4
#2  0x761c7798 in cvHaarDetectObjectsForROC(void const*, CvHaarClassifierCascade*, CvMemStorage*, std::vector<int, std::allocator<int> >&, std::vector<double, std::allocator<double> >&, double, int, int, CvSize, CvSize, bool) ()
   from /usr/lib/arm-linux-gnueabihf/libopencv_objdetect.so.2.4
#3  0x761b159e in cv::CascadeClassifier::detectMultiScale(cv::Mat const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > >&, std::vector<int, std::allocator<int> >&, std::vector<double, std::allocator<double> >&, double, int, int, cv::Size_<int>, cv::Size_<int>, bool) ()
   from /usr/lib/arm-linux-gnueabihf/libopencv_objdetect.so.2.4
#4  0x761a6f8e in cv::CascadeClassifier::detectMultiScale(cv::Mat const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > >&, double, int, int, cv::Size_<int>, cv::Size_<int>) ()
   from /usr/lib/arm-linux-gnueabihf/libopencv_objdetect.so.2.4
#5  0x766d8c94 in AsyncDetectMultiScale::Execute() ()
   from /home/qcttsw/face-detection-node-opencv/server/node_modules/opencv/build/opencv/v3.0.0/Release/node-v11-linux-arm/opencv.node

Looks access an invalid address in HaarDetectObjects_ScaleImage_Invoker::operator().
Please kindly help to provide solution for the issue.
Many thanks.

Which version of OpenCV do you use? I can see some calls from 2.4 and opencv/build/opencv/v3.0.0/Release node module. Shouldn't node module version correspond to installed OpenCV library version?

General note:
Most part of OpenCV algorithms is not reentrant. If you want to use same algorithm in multiple threads, so you should clone / copy / create algorithm with same parameters and use they separately.
Node.js note:
We don't maintain OpenCV bindings for Node.js, so we can't fix reentrant usage of CascadeClassifier object or something else (real problem is unknown).
We can take a look how to resolve this issue in case if you provide minimal reproducer with C++ API.

My opencv version is 2.4.9.
However if there is no known solution,I will build debug c library and debug.
Thanks.
Mark


[Enhancement ] New Configurable Trigger

It would be nice if the bot would only respond when users are having a conversation with it.
How it could work in practice:

Bot owner sets reply_percent to something really low, like -aiuser percent 0.01
Users trigger a conversation via @X.XX'ing the bot, a random message, or reply_percent as per normal.

Bot may continue to reply to new messages if no more than conversation_reply_time seconds have passed since the last message.

conversation_reply_time = 60 (setting to zero disables the feature)
conversation_reply_percent = 0.5 (50% chance to trigger a reply within the conversation_reply_time time frame

This feature request is basically asking for 2 configurable variables for a new way to trigger responses.

Great suggestion, makes sense!
will implement in the future..

implemented  by  e583d57f7f444188a4c5e1f94fa14658a0de84c6
currently it works off a timer off the last bot message (if there's too much side effects, i'll swap it to working off the last message instead as suggested)
the commands are [p]aiuser trigger conversation_reply_percent and [p]aiuser trigger conversation_reply_time


Support for multiple data types

Hi,
I want to use ND4j for a project of mine but my understanding right now is that you cannot have an Integer NDArray and a Double NDArray at the same time. Is it true that all tensors have to be of the same data type? If yes, what prevents you from supporting multiple data types and casting when needed based on the Ops used? And, do you have any plans to add support for multiple data types in the future?
Thanks,
Anthony

You can sorta do it but why do you want multiple data types? We handle casting everything to the proper data type for you anyways.

How can you do it though? An example setting where it could be useful is when you a set of instances, each having a set of features (Double precision floats) and a set of labels (could be Boolean, for binary, or 8-bit unsigned integers, for example), in the context of machine learning. Then it would be wasteful to represent those labels as a tensor of doubles, which is the precision of the features.

Numpy is an example of a library that can handle such a case and which also uses BLAS.

Don't get me wrong please, but type cast for array literally means this sequence:

create new array to match new data type length
loop through original array and convert each element to new data type
destroy original array eventually

Are you 100% sure that your savings on 32 labels (average batch size), will be worth this hassle?
We do support data type conversion currently. But i just don't see real benefits here.

I see your point but please consider the following use case:
We train a neural net using the cross entropy loss. We can implement this loss function efficiently by using int-valued labels to index on the double-valued predictions tensor (containing the probabilities for each label). I think this would be much less efficient had we been using double-valued labels, but please correct me if I'm wrong.
Note also that similar examples could be constructed for Natural Language Processing (NLP) applications where we often deal with discrete data structures.

No, it doesn't work that way. All BLAS functions use floating point data types, so even if you're using INT somewhere in python, it's only at your level. When things go down to BLAS - your INT value becomes discrete FLOAT value. Or DOUBLE. Or HALF.  That's due to simple fact: There's just no integer data types in BLAS specs. Float. Double. Complex. Now new data type gets popularity: HALF precision, aka float16, aka fp16, aka half1

I see. So, what happens when you index a tensor with an int (i.e., to obtain the ith value of a vector)?

You get back a view with the underlying data type being float or double.

That's true but i (i.e., the index to the vector) was provided and used as an integer. It was never converted to a double value. So, back to my example, what if you use values of one int-valued tensor (i.e., the labels tensor) as an index to another tensor that's double-valued (i.e., the predictions tensor). It would be too inefficient in this case to also store the labels as a double-valued tensor as they're only used as indices into another tensor.

No. It would be inefficient to handle dtype conversion all the way through the pipeline. And converting ints to doubles/floats once - is quite ok. Lesser evil, you know :)

I agree but my point in this case is that if both data types are supported at the same time, then no conversion would be necessary at any point. You would directly use the integer labels as indices in the predictions tensor that contains double values and you would get back another double-valued tensor.
The sparse cross entropy function in TensorFlow is implemented this way and it is much more efficient than the cross-entropy function using one-hot labels (which is what I would need to do given the current data type constraint of ND4j).

And what stops you from doing the same with double data type? How big values we're speaking about? Are they big enough to loose integral part precision?

How would you index an array with a double value? You would need to cast it to an integer at some point to obtain a memory address eventually, right? So not supporting an int tensor in this case has two disadvantages: (i) more memory consumption, (ii) eventual cast to an integer.

Nobody cares about those 32 labels in double dtype, seriously. And in case of float, memory footprint will  be equal to integer datatype, 4 bytes per element.

I've explained that above: it's more efficient to cast once (as index), then doing type checks & occasional type casting through the whole math pipeline.

That's why everyone out there eventually have everything in the same data type with floating point values.

It's not true that everyone out there uses floats and that there is no performance impact. The TensorFlow sparse cross-entropy example I gave above is a good example of where data types matter and you gain a lot of efficiency by using them appropriately. And the same example applies to Theano, Torch, MXNet, and most major deep learning frameworks out there.

Where you've got idea of "no performance impact"? I didn't say that.

Point me to the code of op that has chance of going into BLAS pipeline with mixed data type in any framework.


chore(release): v0.10.3 🎉

:robot: I have created a release beep boop
0.10.3 (2024-05-25)
Bug Fixes

deps: update aws sdk dep (#14) (071c4b8)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/ehmpathy/simple-dynamodb-client/releases/tag/v0.10.3 :sunflower:


Failed to parse source map error when using with source-map-loader

Using @susisu/promise-utils with source-map-loader yields a warning (see below) when building.
I tested it with utgwkk/202204017-sketch-webpack-sourcemap repository.
Library version

@susisu/promise-utils: 0.1.1
webpack: 5.72.0
source-map-loader: 3.0.1

Output of build
% ENABLE_SOURCEMAP=true yarn build
yarn run v1.22.10
$ webpack
assets by status 403 bytes [cached] 1 asset
./src/index.js 225 bytes [built] [code generated]
<EMAIL_ADDRESS>1.07 KiB [built] [code generated] [1 warning]

WARNING in<EMAIL_ADDRESS>Module Warning (from ./node_modules/source-map-loader/dist/cjs.js):
Failed to parse source map from<EMAIL_ADDRESS>file: Error: ENOENT: no such file or directory, open<EMAIL_ADDRESS> @ ./src/index.js 2:0-48 4:35-42

1 warning has detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.

webpack 5.72.0 compiled with 1 warning in 285 ms
Done in 0.79s.

Directory structure in node_modules
It looks like src/index.ts is not found.
% tree ./node_modules/@susisu/promise-utils
./node_modules/@susisu/promise-utils
├── LICENSE
├── README.md
├── lib
│   ├── index.d.ts
│   ├── index.d.ts.map
│   ├── index.js
│   └── index.js.map
└── package.json

1 directory, 7 files

@utgwkk Thanks for reporting! The repro really helped.
I have just published<EMAIL_ADDRESS>which includes src/index.ts in the distribution package.
Can you try this out?
Hmm... Maybe all of my libraries are poor.
https://zenn.dev/suin/scraps/9f454023cefb67

Upgrading to v0.1.2 solved the issue! Thank you!
% yarn add @susisu/promise-utils
yarn add v1.22.10
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency.
info Direct dependencies
└─<EMAIL_ADDRESS>info All dependencies
└─<EMAIL_ADDRESS>Done in 6.11s.

% env ENABLE_SOURCEMAP=true yarn build
yarn run v1.22.10
$ webpack
asset main.70ba59efcfb368af548e.js 403 bytes [emitted] [immutable] [minimized] (name: main) 1 related asset
./src/index.js 225 bytes [built] [code generated]
<EMAIL_ADDRESS>1.07 KiB [built] [code generated]
webpack 5.72.0 compiled successfully in 439 ms
Done in 1.02s.


Update repositories.txt

Closing as duplicate of https://github.com/arduino/library-registry/pull/2357


Drop PHP 8.0 support due to end of life and lack of security updates (#128)

BTW It might worth removing PHP 8.0 from composer.json


Should require struct/union prefix when casting

We're allowed to skip the struct/union prefix when casting as a shorthand at the moment, but it can cause problems:
test.h:
struct Foo
{
  int x;
};

typedef union
{
  int a;
  int b;
} Foo;

This script allows some invalid field accesses:
#include "test.h"

BEGIN
{
  ((Foo*)0)->a;
  ((Foo*)0)->b;
  ((Foo*)0)->x;
}

This is what should be required:
#include "test.h"

BEGIN
{
  ((Foo*)0)->a;
  ((Foo*)0)->b;
  ((struct Foo*)0)->x;
}

@brendangregg thoughts?


Adding version override.

I need the ability to override the version to be different than either the project or nuspec version. This allows me to use a custom version which allows us to programmatically create snapshot versions before release.

Hi !
Supposedly you can already do the same with:
nuget {
    nuspec {
        metadata() {
            delegate.version '1.0.0'
        }
    }
}

That doesn't work out for you ?

Scratch that, I missed the "different than the nuspec version" part, that's what I get for commenting too early :)

I didn't know you could upload packages to nuget with a version different than the nuspec one, may I ask how it works and how you manage snapshots like that ?

Our gradle build sees that it is a non release build and creates a unique version by appending a timestamp to the form of "-SNAPSHOT-" which is then used at the nuget version. Since nuspec is simply a template to build the nupkg we would rather configure the version outside of the nuspec otherwise we have to handle updates to the nuspec in version control which is more complicated than specifying a version in gradle plugin. Note that this means the nuspec version doesn't have to match the assembly version in the csproj file.

Ok, but then in the nupkg, the version from your nuspec is no longer there (only remains the version you passed as argument) no ?

Any fields you specify on the command that override a value in the nuspec, such as version, are written into the nuspec stored in the nupkg. So the version shows up correctly in the nupkg

But then in that case my earlier comment is still valid, if you do:
nuget {
    nuspecFile = 'foo.nuspec'
    nuspec {
        metadata() {
            delegate.version '1.0.0'
        }
    }
}

then it'll achieve the same, no ?

I am rather new to groovy but unless I am mistaken the method getNuspec looks for a nuspecFile and if it exists returns this files content and does not look at any information in the nuspec provided in the gradle build. I also tried testing this and the version used is the one from the nuspecFile.

Right, the entire nuspec block is ignored if you provide a file, my bad.
Shouldn't we instead change that and make it so that when you specify both, it takes the nuspec file as a base and enriches it with the provided block ?
(if it can be achieved) ?

I'll take a look into that, I would also like a way to modify the in gradle nuspec after evaluation.

My one concern though is when i want to call nuget directly on a csproj but change the version. That would get interesting because in that case a nuspec doesn't exist at all.

That's now implemented, I'll close this PR !
Thx


Trying to do Bootloader update OTA (IDFGH-13262)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report
We have a large number of devices deployed during late 2017 and early 2018 using very old boot loaders.
Our firmware and hardware has been very reliable and the last upgrade of these devices was done in early 2019.
We are currently investigating a process to try and upgrade these boot loaders remotely using a modified OTA.
None of these devices have any level of flash encryption enabled
Steps we believe are required are as follows:

Check if new bootloader is available
If available, download as "bootloader.bin" to LittleFS storage
If download and file write successful, read existing flash to LittleFS as "bootloader.old"
Write "bootloader.bin" to bootloader at address 0x1000
If write successful, reboot using new boot loader
If not successful, try restore "bootloader.old" to 0x1000 then reboot

At a later stage, when all devices have been upgraded to latest bootloader and firmware we would want to start a process to enable bootloader checksum on these device and automatically calculate and update the initial checksum in the field
Steps 1 to 3 work successfully, we can check the boot loaders files on LittleFS, correct size and contents.
Steps 4 and 5 run correctly, no errors reported. When we check read the flash with esptool and check the new bootloader written all looks correct, offsets and lengths.
Before the bootloader update ROM startup output was as follows:
ets Jul 29 2019 12:21:46
rst:0xc (SW_CPU_RESET),boot:0x1e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:6156
ho 0 tail 12 room 4
load:0x40078000,len:16112
load:0x40080400,len:4
load:0x40080404,len:3712
entry 0x400805dc

After bootloader update ROM startup shows the following repeatedly:
ets Jul 29 2019 12:21:46
rst:0xc (SW_CPU_RESET),boot:0x1e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:6144
load:0x00000000,len:1634885484

Based on the very different 'load' information I assume that I have failed to update some pointers and/or sizes somewhere.
Any suggestions.

Hi @ammaree!
Could you clarify some points.

If you read out the newly written bootloader and then run image_info for this file do you see that it is correct and the same as on the host?

esptool.py read_flash 0x1000  <SIZE>  path/to/readout_new_bootloader.bin
esptool.py image_info path/to/readout_new_bootloader.bin

Does this new bootloader work well if you write it directly to 0x1000 using esptool?
Do not you forget to erase the 0x1000 to CONFIG_PARTITION_TABLE_OFFSET, right?

Will read back and check, let you know.
New bootloader, like all previous, are 100% standard, nothing modified so should work if written by esptool
Just so I understand areas to be erased before, are you saying:
0x0000 to 0x0FFF, I believe unused?
and 0x1000 to 0x7FFF, normal bootloader?

Just so I understand areas to be erased before, are you saying:
0x0000 to 0x0FFF, I believe unused?
and 0x1000 to 0x7FFF, normal bootloader?

Normally we consider the bootloader size as = (CONFIG_PARTITION_TABLE_OFFSET - ESP_BOOTLOADER_OFFSET). The bootloader must not exceed the partition table location. If signature block(s) are appended, they should be erased too. To guarantee that you have to erase 0x1000 - CONFIG_PARTITION_TABLE_OFFSET.

Erase did the trick, I "assumed" it would be part of the flash write operation.
Thanks for quick response.

I am glad to hear that it was resolved.
FYI, I am working on Bootloader OTA. It will have two modes:

Power-unsafe OTA bootloader update. I guess it could be similar to your implementation.  I have expanded the OTA APIs to download the new bootloader.
Power-safe OTA bootloader update. A new chip version will have some changes in the ROM bootloader which will be able to select a recovery bootloader in case the primary bootloader fails to load.

I close this issue. Thanks.

@KonstantinKondrashov
Last question. In order to change the spiffs and/or coredump partition sizes the MD5 checksum becomes a problem.
Currently I am scanning for the checksum indicator (right at the end) and clearing the values to all 0xFF before writing the changed partition table back.
Ideally I would like to recalculate and update the checksum going forward, especially for old devices where the bootloader has been updated.
Can you please provide some info on the area to be included in the checksum and the algorithm used, hopefully a standard IDF API?
Thanks

Hi @ammaree!
MD5 for the partition table is used if the CONFIG_PARTITION_TABLE_MD5 is enabled.
There are ROM APIs that are used to calculate MD5. You can have a look code in IDF:

https://github.com/espressif/esp-idf/blob/master/components/esp_partition/partition.c#L88-L91
https://github.com/espressif/esp-idf/blob/master/components/bootloader_support/src/flash_partitions.c#L41-L54

#include "esp_rom_md5.h"

md5_context_t context;
unsigned char digest[16];
esp_rom_md5_init(&context);
esp_rom_md5_update(&context, (unsigned char *) partition_table, num_parts * sizeof(esp_partition_info_t));
esp_rom_md5_final(digest, &context);

@KonstantinKondrashov
Sorry, hopefully last question...
Been looking for documentation on the esp_rom_md5_???? APIs but cannot find much, any link that will help?
esp_rom_md5__init is self explanatory.
esp_rom_md5_update will update the MD5 context with the calculated value from the range specified, either the whole partition table as 1, of in steps of an entry at a time.
esp_rom_md5_final I would prefer not to make assumptions on...
Primarily interested in verifying a) calculating a digest and b) verifying an existing digest...
Thanks

@ammaree
The description of ROM MD5 functions. An example of their usage for the partition table was in the previous message.
https://github.com/espressif/esp-idf/blob/master/components/esp_rom/include/esp_rom_md5.h#L74-L106
You may look at the mbedtls APIs for MD5, here is an example:
https://github.com/espressif/esp-idf/blob/master/examples/storage/spiffsgen/main/spiffsgen_example_main.c#L38-L79
Sorry, I guess that is all that we have for now. There is not a particular doc for that. You can try to search for more examples in the esp-idf repo by "md5".

Hi @KonstantinKondrashov
Thanks for all the info above, got the process for auditing, changing and updating partition table working.
Also got safe & reliable download and updating of bootloader working.
We have however now come across the complication of matching
#versions/configs of partition tables, essentially with or without MD5; with
#versions of boot loaders, essentially again whether they ignore and/or handle and/or expect an MD5 record in the partition table.
The majority of old devices were deployed Feb to May 2018 and those are the ones where we are unsure of the bootloader & partition table versions and configurations. We assume they all use IDF v2.1.1 based bootloaders with a couple of later replacements using IDF 3.0 and 3.1 bootloaders
To get all devices to a common minimum standard we foresee the following steps.

Deploy "audit" firmware to report( via syslog) all info on the existing partition table as well as the existing bootloader.

Armed with this info develop and deploy firmware to update all partition tables to a similar structure, primarily coredump and littlefs related. If MD5 is present, update it. If not present leave as is, do not add. This functionality we already have, tested and working.

The develop and deploy firmware to do the bootloader upgrade. This step we need help on since the mixture of devices with/without MD5 must be accommodated transparently. Probably deploy a new bootloader with MD5 support but at same time time, immediately after OTA of new bootloader, also add MD5 to partition table if not there, the reboot.

Regarding 1 above:
Our current thought is build and deploy new "audit" firmware version that does an audit of the partition table and logs that info to our syslog host. We would like to also audit the installed boot-loader and report all info relating to the version and options it was build with, specifically MD5 support but also other items relevant to later doing a boot-loader upgrade.
Auditing the partition table we have done and working, what remains is the bootloader audit.
Core questions are:
A: Is there any information, hints, APIs or similar that we can use to extract this info programmatically from existing bootloaders?
B: If not, what are the safest build options to use for building this special "audit" firmware and an associated new bootloader that will work on the widest range (all combinations) of partition tables?
Thanks again for your assistance.
André

@KonstantinKondrashov
May I again ask for some advice on the questions (especially A) in the message above, really need to try finalise the bootloader audit.


Added Times to Logs

Added Times to Logs Should be Tested

We need to spend more time figuring out how RIOLog works before trying this. We may just be better off not logging timestamps to RIOLog, and only to the USBLogger


sonicguo - bug dash 4/26

Description
<C# sample> Subscriber client connection ended immediately
https://azure.github.io/azure-webpubsub/getting-started/publish-messages/csharp-publish-message#setup-subscriber
namespace subscriber
{
class Program
{
static async Task Main(string[] args)
{
if (args.Length != 2)
{
Console.WriteLine("Usage: subscriber  ");
return;
}
var connectionString = args[0];
var hub = args[1];
        // Either generate the URL or fetch it from server or fetch a temp one from the portal
        var serviceClient = new WebPubSubServiceClient(connectionString, hub);
        var url = serviceClient.GetClientAccessUri();

        using (var client = new WebsocketClient(url))
        {
            client.MessageReceived.Subscribe(msg => Console.WriteLine($"Message received: {msg}"));
            await client.Start();
            Console.WriteLine("Connected.");
        }

        Console.Read();
    }
}

}

Repro Steps

Follow the steps to create the subscriber: https://azure.github.io/azure-webpubsub/getting-started/publish-messages/csharp-publish-message#setup-subscriber
run the subscriber with the command:  dotnet run ""
In the console, it appears Connected.
Send messages from the publisher, the subscriber doesn't receive messages.
Check the status from the Live Trace Tool. It appears the subscriber disconnected immediately after connected.

The same issue happens with the python sample https://github.com/Azure/azure-webpubsub/tree/9de1e1ee90fa998f7eed4fa0e68163a24f859108/samples/python/pubsub
Repro Steps:

Follow the steps to create the subscriber and publish: https://github.com/Azure/azure-webpubsub/tree/9de1e1ee90fa998f7eed4fa0e68163a24f859108/samples/python/pubsub
Send and receive messages successfully.

Publish:

Subscriber:

No message trace from the Live Trace Tool.

message trace from server to client is now not included in live trace. @wanlwanl consider adding this part?

the python one is duplicate with https://github.com/Azure/azure-webpubsub/issues/115


Append the ports for the Operator vs replacing

The last change to the Operator service overwrites the ports from the Service vs appending them.

Why do you want to access the Prometheus Operator without TLS?
@paulfantom my understanding is that this was intentional, right?