A way to access the current offset of a consumer

Current version i am using: 1.1.x
Currently, the only way to get the current offset of a topic-partition is by implementing ConsumerSeekAware and receiving an idle container event via ConsumerSeekAware#onIdleContainer, and accessing the assignments.
/**
 * If the container is configured to emit idle container events, this method is called
 * when the container idle event is emitted - allowing a seek operation.
 * @param assignments the new assignments and their current offsets.
 * @param callback the callback to perform a seek.
 */
 void onIdleContainer(Map<TopicPartition, Long> assignments, ConsumerSeekCallback callback);

https://docs.spring.io/spring-kafka/reference/htmlsingle/#_current_positions_when_idle
It would be helpful to provide a method directly on the KafkaMessageListenerContainer to access the topic-partition offsets on demand rather than on partitions assigned (or idle events), similar to the approach of https://github.com/spring-projects/spring-kafka/pull/402. This method would simply take advantage of position(TopicPartition) on the apache kafka Consumer.
Reason for this request: When implementing an AlwaysRetryPolicy, the container can continually receive the same message that causes an error, making it not idle. Since the container is no longer idle but the offset is not moving, it can be difficult to detect that the consumer is stuck. Having this would allow someone to track the offset and perform some action if the offset hasn't increased.
I've been digging through the issues looking for a duplicate of this request but haven't seen any specific to this, so apologies if it's been requested before. Thanks!

Starting with version 2.0 there is ConsumerAwareMessageListener to consider on the matter: https://docs.spring.io/spring-kafka/docs/2.0.0.RC1/reference/html/_reference.html#message-listeners
And @KafkaListener method can be supplied with the Consumer<?, ?> argument:
@KafkaListener(id = "qux", topics = "annotated4", containerFactory = "kafkaManualAckListenerContainerFactory",
				containerGroup = "quxGroup")
public void listen4(@Payload String foo, Acknowledgment ack, Consumer<?, ?> consumer) {

Let us know if that is OK with you.
Right, there is nothing to do with that in the previous versions.
But I think your idea about KafkaMessageListenerContainer.position(TopicPartition) API makes sense.
@garyrussell , WDYT?

Thanks for the quick response @artembilan!
The @KafkaListener approach would definitely work. My current thought is I'd like to have some class, separate from the @KafkaListener method's class, to have access to and monitor the offsets. Using the @KafkaListener approach doesn't really support that. Having the offsets available via the listener container follows the approach of the metrics API I have referenced above.
To expand on the API, it could look something like:
public Map<TopicPartition, Long> positions()

Well, I'm afraid we can't provide such an API on the Container level, since KafkaConsumer.position() is one of those methods which can be called only from a single thread:
public long position(TopicPartition partition) {
        acquire();
...
    /**
     * Acquire the light lock protecting this consumer from multi-threaded access. Instead of blocking
     * when the lock is not available, however, we just throw an exception (since multi-threaded usage is not
     * supported).
     * @throws IllegalStateException if the consumer has been closed
     * @throws ConcurrentModificationException if another thread already has the lock
     */
    private void acquire() {
        ensureNotClosed();
        long threadId = Thread.currentThread().getId();
        if (threadId != currentThread.get() && !currentThread.compareAndSet(NO_CURRENT_THREAD, threadId))
            throw new ConcurrentModificationException("KafkaConsumer is not safe for multi-threaded access");
        refcount.incrementAndGet();
    }

That's how the ConsumerAwareMessageListener is better solution since we don't call any other consumer operations during listener process, therefore that KafkaConsumer is free for calling from the end-user code.
Does it make sense to you?

Hi @artembilan
Yeah I noticed this the other day when attempting to implement this locally. I couldn't think of any good clean approaches to get around the issue. Using ConsumerAwareMessageListener makes sense and will work for me when I upgrade to that version.
Thanks for the analysis and the feedback!


disable_referential_integrity does not work with PostGIS tables

We use the [PostGIS extension] (http://postgis.net/) and the rgeo AR gem.  We use truncation for our js tests as well as our before(:suite) strategy.
The problem is that the PostGIS extension creates a table spatial_ref_sys that is owned by the user who installs the extension (namely, a super user).
Running bundle exec rspec fails with the following error:
/Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `async_exec': PG::InsufficientPrivilege: ERROR:  must be owner of relation spatial_ref_sys (ActiveRecord::StatementInvalid)
: ALTER TABLE "spatial_ref_sys" ENABLE TRIGGER USER;ALTER TABLE "filtered_tracked_points" ENABLE TRIGGER USER;ALTER TABLE "processed_tracked_points" ENABLE TRIGGER USER;ALTER TABLE "raw_tracked_points" ENABLE TRIGGER USER;ALTER TABLE "resource_intersections" ENABLE TRIGGER USER;ALTER TABLE "schema_migrations" ENABLE TRIGGER USER;ALTER TABLE "users" ENABLE TRIGGER USER;ALTER TABLE "feature_types" ENABLE TRIGGER USER;ALTER TABLE "features" ENABLE TRIGGER USER
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `block in execute'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:473:in `block in log'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activesupport-4.2.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:467:in `log'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:154:in `execute'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:23:in `rescue in ensure in disable_referential_integrity'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:20:in `ensure in disable_referential_integrity'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:20:in `disable_referential_integrity'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/active_record/truncation.rb:235:in `clean'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/base.rb:40:in `clean_with'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/configuration.rb:91:in `block in clean_with'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/configuration.rb:91:in `each'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/configuration.rb:91:in `clean_with'
    from /Users/<user>/Development/<company>/<project>/spec/support/database_cleaner.rb:3:in `block (2 levels) in <top (required)>'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/example.rb:333:in `instance_exec'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/example.rb:333:in `instance_exec'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/hooks.rb:357:in `run'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1559:in `block in run_hooks_with'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1559:in `each'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1559:in `run_hooks_with'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1525:in `with_suite_hooks'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:109:in `block in run_specs'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/reporter.rb:62:in `report'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:108:in `run_specs'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:86:in `run'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:70:in `run'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:38:in `invoke'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/exe/rspec:4:in `<top (required)>'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/rspec:23:in `load'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/rspec:23:in `<main>'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/ruby_executable_hooks:15:in `eval'
    from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/ruby_executable_hooks:15:in `<main>'

The error comes from the call to disable_referential_integrity because it calls all tables, ignoring any tables that might have been excluded via the truncation options.
Our database_cleaner.rb file:
RSpec.configure do |config|
  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation, {except: %w[spatial_ref_sys]})
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation, {except: %w[spatial_ref_sys]}
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.append_after(:each) do
    DatabaseCleaner.clean
  end
end

One workaround is to execute the tests as a db super user. Another would be to change the ownership of the spatial_ref_system table. Neither of these are very desirable.
Is there a way to skip tables in the disable_referential_integrity call?

Similar issue here, just that rspec is completely hanging so I needed to do this trick and found DatabaseCleaner.clean_with(:truncation, { except: %w[spatial_ref_sys] }) as the line causing the problem.
Here's the complete database_cleaner.rb file:
RSpec.configure do |config|
  config.before(:suite) do
    DatabaseCleaner.strategy = :transaction
    DatabaseCleaner.clean_with(:truncation, { except: %w[spatial_ref_sys] })
  end

  config.prepend_before(:each) do
    DatabaseCleaner.start
  end

  config.append_after(:each) do
    DatabaseCleaner.clean
  end
end

Maybe this isn't an issue with database_clenaer after all, couldn't this be an issue with the pg gem itself? See https://bitbucket.org/ged/ruby-pg/issues/75/hang-in-pgconn-block-while-waiting-for.

I believe this is a problem with ActiveRecord as mentioned here.
I've fixed this for my problems by adding the following monkey patch in test_helper.rb
module ActiveRecord   module ConnectionAdapters     module PostgreSQL       module ReferentialIntegrity # :nodoc:         def supports_disable_referential_integrity? # :nodoc:           false         end       end     end   end end


fix(api): container registry dockerhub 🐛

Summary
Users trying to use our Terraform resource to manage Docker Hub integrations are failing with:
lacework_integration_docker_hub.this: Creating...
╷
│ Error:
│ [POST] https://account.lacework.net/api/v2/ContainerRegistries
│ [400] Error validating integration. Please verify the configuration details entered.
│
│ with lacework_integration_docker_hub.this,
│ on dockerhub_integration.tf line 1, in resource "lacework_integration_docker_hub" "this":
│ 1: resource "lacework_integration_docker_hub" "this" {

This change is fixing a few issues in the api package that interacts with container registry integrations.
Signed-off-by: Salim Afiune Maya<EMAIL_ADDRESS>How did you test this change?
Wrote tests 😄
Issue
https://lacework.atlassian.net/browse/LINK-1089

Purposely wrote a test that uncovers the issues:
=== FAIL: api TestContainerRegistriesNewDockerhub (0.00s)
    container_registries_dockerhub_test.go:55:
                Error Trace:    /codefresh/volume/go-sdk/api/container_registries_dockerhub_test.go:55
                Error:          Not equal:
                                expected: "index.docker.io"
                                actual  : ""

                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1 +1 @@
                                -index.docker.io
                                +
                Test:           TestContainerRegistriesNewDockerhub
                Messages:       wrong container registry domain


add: default arg to write_to_textfile

What
Add REGISTRY as the default arg of write_to_textfile
Why
write_to_textfile is one of the easiest way to bring the power of custom metrics to the users' Prometheus, so it should be very nice to have its interface simple.
Other functions like make_wsgi_app, start_http_server already have it as default value, so it seems fair to add REGISTRY here as default.
before
from prometheus_client import write_to_textfile, REGISTRY

while True:
    do_whatever()
    write_to_textfile(TMP_PATH, REGISTRY)

after
from prometheus_client import write_to_textfile

while True:
    do_whatever()
    write_to_textfile(TMP_PATH)

Your example is not a valid use for the textfile collector, it's something to be written once at the end of a batch job and always with a custom rather than the default registry.

If you find otherwise, you should probably be scraping the process normally.

I think I got what you've meant. So having everything as HTTP server is the Google way and the philosophy is the same for Prometheus. Makes sense.
But, well, I want to keep the number of the exposed ports/servers as small as possible.  So I prefer doing this way, especially when I run this for internal network monitoring using ssh and expect.
(also, cron always makes a mess, after a few years later)
Anyway, that's just MY way and not the prometheus/client_python way. I deeply agree with that.
So thank you, I close this PR.


How to get the text layer, font color, size, orientation

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

You know you need to fill all that stuff in right? :/


10.40 Gathering materials does not work

Gathering materials quite often does not work.

??

Sometimes gathering materials from trees, cars, etc. on version 10.40 does not work.


Broken syntax highlighting when the default import is named "from"

TS and JS Grammar Extension version:  latest
Code
import from from './from';

Visual output

If the name assigned to the default export is "from" then it gets highlighted improperly.

Confirmed, they both get:
keyword.control.from.tsx
meta.import.tsx
source.tsx

Given how tmlanguage grammar works and how we have things set up the effort to fix this is too much compared to how many people are running into this.


change accordion & side-nav docs to use  instead of

@alex-page nice pickup, I didn't even notice the purple link 😅. Should be fixed now 👍


Advisory Council should display members with sleep apnea

A lot of advisory council members are people with sleep apnea, somehow display this

Not in a negative way! Doesn't need to specifically mention who.

This issue is covered by #244 Advisory Council Text Update


fail to start

installing IPFS-Desktop-Setup-0.13.2 successfully and starting the app right after.
Specifications

OS: win32
IPFS Desktop Version: 0.13.2
Electron Version: 9.3.2
Chrome Version: 83.0.4103.122

Error
Error: spawn UNKNOWN
    at ChildProcess.spawn (internal/child_process.js:394:11)
    at spawn (child_process.js:549:9)
    at execFile (child_process.js:233:17)
    at module.<computed> (electron/js2c/asar.js:163:31)
    at C:\Users\Matthias\AppData\Local\Programs\IPFS Desktop\resources\app.asar\src\ipfs-on-path\index.js:85:5
    at new Promise (<anonymous>)
    at runWindows (C:\Users\Matthias\AppData\Local\Programs\IPFS Desktop\resources\app.asar\src\ipfs-on-path\index.js:84:10)
    at run (C:\Users\Matthias\AppData\Local\Programs\IPFS Desktop\resources\app.asar\src\ipfs-on-path\index.js:108:12)
    at firstTime (C:\Users\Matthias\AppData\Local\Programs\IPFS Desktop\resources\app.asar\src\ipfs-on-path\index.js:79:21)
    at module.exports (C:\Users\Matthias\AppData\Local\Programs\IPFS Desktop\resources\app.asar\src\ipfs-on-path\index.js:56:3)

@Matthiasc - suggesting that it may be better to take this issue to the forums at discuss.ipfs.io for more active troubleshooting, as core team members are diverted at the moment and may not be able to troubleshoot quickly. Thank you!

I am afraid the only fix here is to find IPFS Desktop/config.json and manually set:
"ipfsOnPath": false,

We will be deprecating this feature (https://github.com/ipfs-shipyard/ipfs-desktop/pull/1768), if you want ipfs on your PATH   follow https://docs.ipfs.io/install/command-line/


Finalizing the notebook apis

Issue for tracking finalization (or deprecation) of the various notebook API proposals
Key
⬆  —  Waiting for extensions (upstream)
Tasks

[ ] ⬆  https://github.com/microsoft/vscode/issues/146831
[x]  https://github.com/microsoft/vscode/issues/147248
[x] https://github.com/microsoft/vscode/issues/149181
[x] https://github.com/microsoft/vscode/issues/147264
[x] #155245
[x] https://github.com/microsoft/vscode/issues/146838
[X] https://github.com/microsoft/vscode/issues/149271
[X] https://github.com/microsoft/vscode/issues/147414
[X] https://github.com/microsoft/vscode/issues/146781

At this point, we've finalized or removed the core notebook APIs. This work has included finalizing proposals for editing and notebook editors, as well as removing all the main proposals that saddled us with significant technical debt
The remaining api proposals are as follows:

notebookCellExecutionState — This is an advanced API but it also needs more thought as the current proposal has known gaps. With the limited number of use cases we know of, we are not sure finalizing it is worth it.

notebookControllerAffinityHidden — New API. @IanMatthewHuff Can you please determine if this should be finalized?

notebookControllerKind — https://github.com/microsoft/vscode/issues/168535 Small. Unclear if others beside Jupyter need this. Will follow up in linked issue

notebookDeprecated — https://github.com/microsoft/vscode/issues/146831. Minor seeming but kind of annoying to get right. We are tracking this in the linked issue

notebookKernelSource — New api. @rebornix can you please determine if this API should be finalized

notebookLiveShare — This will likely be a proposal forever. LiveShare's use case is pretty unique and we don't think other extensions should use this API. It would be nice if we could move LiveShare off of this proposal but we don't know how to do this. The proposal is also not adding much debt to our code

notebookMessaging — The proposal name is not accurate since most parts of notebook messaging have already been finalized. For the remaining parts, NotebookRendererScript may be replaced by #163511. It's not clear if notebook controller messaging is needed at all

notebookMime — https://github.com/microsoft/vscode/issues/126280. Will follow up in linked issue

As I believe the original goal of this issue has largely been achieved, I'm going to close this issue. I will following up on the linked issues


Zeroize mem / proj status re safe-oqs?

All,
I'm in the early stages of writing my project, which is in Rust, and relies on ML-KEM / Kyber (both, one for FIPS world, the other for non-fips).
safe-oqs Q
I originally incorporated a fork of liboqs-rust, safe-oqs because it had updated the submodule of liboqs to include the recently released ML-KEM.  However, I now have a few questions, and @Scarjit and @joernheinemann don't have issues open on their fork...
@thomwiggers should liboqs-rust just pull in the relevant changes from safe-oqs?  I looked at the diff between main branches, there's not much there.  Just updated liboqs commit and exposing ML-KEM / ML-SIG.  The rest is mostly renaming their crate
Real Q re Zeroize
Outside of that, my real question is this: Does liboqs-rust zeroize sensitive buffers on Drop?  I see OQS_MEM_secure_free(), and a build check to ensure it's used over free(), but I don't see the Drop trait implemented on the Rust side in the macros.  Drop only appears to be implemented for Kem and Sig, which, afaict, are just handles.  Additionally, that Drop implementation is calling OQS_KEM_free(), which does not securely zero the memory.
Would it make sense to just #[derive(Zeroize, ZeroizeOnDrop)] from the zeroize crate?

@jac-cbi Hello,
sorry forgot to enable the issues. Now it is enabled.


(bug) Google cloud storage retriever only gets the file in the first attempt

Observed behavior
It was not possible to use an instance of gcstorageretriever.Retriever{} since it is only able to get the file on the initial bootstrap of go-feature-flag instance. After feature flag client is up and running, the internal flags cache is no longer updated when gcstorageretriever is used.
Expected Behavior
gcstorageretriever.Retriever{} should be able to the flags when the file stored in google cloud storage changes.
Steps to reproduce
Using the following code(just an exaple):
ffcli, err := ffclient.New(ffclient.Config{
  PollingInterval: 3 * time.Second,
  Retriever: &gcstorageretriever.Retriever{
      Options: []option.ClientOption{option.WithoutAuthentication()},
  Bucket: "my-bucket",
  Object: "flags.yaml",
  }
})
defer ffcli.Close()

After the bootstrap, we are able to get current flags data, for example using the following func:
flags := ffcli.AllFlagsState(user)

But, if the file is updated in google cloud storage, go-feature-flag is not able to retrieve the current config.
Moreover, We can see several log traces that refer to the following line of code(saying file not found)
fmt.Errorf("unable to read from GCP Object %s in Bucket %s, error: %s", retriever.Bucket, retriever.Object, err)

@jferrl thanks for reporting this issue.
I am sorry that it is not working as expected 😓
I am currently on PTO so I am not able to check this before next week.
In the mean time if you want to work on resolving this issue it will be much appreciated.

@jferrl thanks for reporting this issue.
I am sorry that it is not working as expected 😓 I am currently on PTO so I am not able to check this before next week.
In the mean time if you want to work on resolving this issue it will be much appreciated.

No worries! I'm working on the fix!
Thanks! 😄

Resolves by #311


Enhance Traceflow e2e test

Enable intra-Node e2e test for kind
Test case enhancement

/test-e2e

Codecov Report

Merging #1397 into master will decrease coverage by 19.55%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #1397       +/-   ##
===========================================
- Coverage   64.39%   44.83%   -19.56%
===========================================
  Files         159       74       -85
  Lines       12681     5589     -7092
===========================================
- Hits         8166     2506     -5660
+ Misses       3660     2805      -855
+ Partials      855      278      -577

Flag
Coverage Δ

#integration-tests
44.83% <ø> (-0.04%)
:arrow_down:

#kind-e2e-tests
?

#unit-tests
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/agent/proxy/service.go
0.00% <0.00%> (-100.00%)
:arrow_down:

pkg/ovs/ovsconfig/default.go
0.00% <0.00%> (-100.00%)
:arrow_down:

pkg/agent/proxy/types/groupcounter.go
0.00% <0.00%> (-95.00%)
:arrow_down:

pkg/agent/proxy/endpoints.go
0.00% <0.00%> (-88.58%)
:arrow_down:

pkg/agent/querier/querier.go
0.00% <0.00%> (-87.50%)
:arrow_down:

pkg/apis/controlplane/register.go
0.00% <0.00%> (-85.72%)
:arrow_down:

pkg/agent/proxy/types/types.go
0.00% <0.00%> (-84.62%)
:arrow_down:

pkg/agent/flowexporter/connections/conntrack.go
0.00% <0.00%> (-84.22%)
:arrow_down:

pkg/agent/types/networkpolicy.go
5.88% <0.00%> (-82.36%)
:arrow_down:

pkg/apis/stats/register.go
0.00% <0.00%> (-81.82%)
:arrow_down:

... and 125 more

/test-e2e

/test-all

It seems the new IntraNode test does not cover the case where traffic is dropped because of a NetworkPolicy. Is that intentional?

Ignore this. I misread the diff.

/test-all

/test-all

/test-all

/test-all

/test-all


fix typos.

What this PR does / why we need it:
Correct some spelling errors in the document
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
fix typos.
Special notes for your reviewer:
Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: The Boy Scout Rule was considered and applied if needed
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[x] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to the kubevirt-dev was considered

Release note:

something has wrong.


Increment versions for face releases

Increment package versions for face releases

API change check
API changes are not detected in this pull request.

/check-enforcer override


iOS DateTimePicker ignores the passed date in props

Environment

System:
OS: macOS 10.15.4
CPU: (8) x64 Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
Memory: 906.57 MB / 16.00 GB
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 12.16.1 - ~/.nvm/versions/node/v12.16.1/bin/node
Yarn: 1.22.4 - ~/.yarn/bin/yarn
npm: 6.14.5 - ~/.nvm/versions/node/v12.16.1/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 13.4, DriverKit 19.0, macOS 10.15, tvOS 13.4, watchOS 6.2
IDEs:
Android Studio: 3.6 AI-192.7<IP_ADDRESS>92135
Xcode: 11.4.1/11E503a - /usr/bin/xcodebuild
npmPackages:
react: 16.9.0 => 16.9.0
react-native: 0.61.5 => 0.61.5
Platforms

iOS
Versions

iOS: 13.4.1
react-native-modal-datetime-picker: 8.6.0
react-native-community/datetimepicker: 2.4.0
react-native: 0.61.5
react: 16.9.0

Description

iOS DateTimePicker ignores the passed date props . So that when the picker is opened it shows the correct date props selected , but if I select "Confirm" it passes the current date to the onConfirm function
Reproducible Demo

Video for the case
Below is the code
const minimumDate = getTomorrow();

const pickDateEnd = selectedDate => {
  setShowDateEndPicker(false);
  if (selectedDate) {
    itinerary.dateEnd = selectedDate;
  }
};

<DateTimePickerModal
  isVisible={showDateEndPicker}
  mode="date"
  minimumDate={itinerary.dateBeg || minimumDate}
  date={itinerary.dateEnd || minimumDate}
  onConfirm={pickDateEnd}
  onCancel={() => setShowDateEndPicker(false)}
/>

@HaidarZ 👋
Could you please follow the reproducible step instructions of the issue template? Thanks!

Let us know how to reproduce the issue. Include a code sample or share a project that reproduces the issue.
Follow the guidelines for providing a minimal example: https://stackoverflow.com/help/mcve
Please notice that WE WON'T SUPPORT ISSUE REPORTS THAT DON'T HAVE A MINIMAL REPRODUCIBLE EXAMPLE.


numpy error

Colab should be updated to fix this; however, requires restarting the runtime before running the last cell. Closing unless this problem reappears.


Configure Business Rules, Workflow and Process Visibility

Tutorials: https://developers.sap.com/tutorials/cp-starter-ibpm-employeeonboarding-2-configure.html
At:
STEP 5:
Download and import workflow sample application:
3. Click on the Dev Space to go into the workspace and from File menu choose Open Workspace:
If you do not have any dev-space, then click Create Dev Space. In the New Dev Space page, enter any dev space name, select SAP Fiori kind and Workflow Management extension. Finally click Create Dev Space to create a space in your business application studio.
Here, the Workflow management has been renamed to "Workflow Module". Please update.
And screenshot needs to be updated too:
Expected:

Actual:

In further steps:
If you have a dev-space but Workflow Management extension is not added then click on Edit button, select Workflow Management and Save Changes.
Here also, same update has to be implemented along with the screenshot update:
Expected:

Actual:

Labels to be added:
Scenario label: SCPTest-eboard
Sprint Label: SCPTest-2303A
Landscape Label: trail2(Trial US East (VA) (AWS) ) & Trial3 (AP Region)
Priority: Medium
Thanks and Regards,
Farozan

@ArchanaShukla can you please respond to this user's feedback?


Question: when I use some js functions, jerry_run_simple() return false

I want to put the jerryscript engine on the target board and interactively send the JavaScript code and execution results between the target board and the PC terminal via serial communication.
I am using the NUCLEO_F446RE board, and I have succeeded in building the jerryscript engine to target mbed os 5.
Then I followed Step 1 of the tutorial below and got 'Hello World!' output.
Https://github.com/janjongboom/jerryscript/blob/master/docs/03.API-EXAMPLE.md
<main.cpp>
#include <string.h>
#include "mbed.h"
#include "jerry-api.h"
Serial pc(USBTX, USBRX)
int main (int argc, char *argv[])
{
const jerry_char_t script[] = "print ('Hello, World!');";
size_t script_size = strlen ((const char *) script);
bool ret_value = jerry_run_simple (script, script_size, JERRY_INIT_EMPTY);
return (ret_value ? 0 : 1);
}
The problem is that when I use function codes such as setInterval(), DigitalOut() instead of the print() function, jerry_run_simple() return false.
Ex.
const jerry_char_t script [] = "setInterval ( function () {print ('a') }, 1000);
I want to know why this is happening and what should I do.
Thanks in advance. #

SetInterval and friends are provided by the environment, not the JS engine. As far as I know mbed5 provides these functions. Perhaps @janjongboom @thegecko could help.

@limjunsung I haven't looked at the underlying issue yet, but does it work properly when you run jerry_run? I'm using that for the REPL I was building and that worked without issues...
See https://github.com/ARMmbed/mbed-js-repl/blob/master/repl/repl.h#L248

There are only the standard builtins available when you are using jerry_run_simple. This function is init a new environment at start and clean it up at the end.


Sorry new to Github ... I have requested the test tokens to be sent to my wallet, everything looks set up fine with the wallet but getting an error message from faucet

Issue description
Briefly describe the issue.
Wallet version
Which version of wallet are you using?

wallet version (version number, commit, or branch):

System specification
What is your operating system?
Mac/Win/Linux (+Version and or Distro if possible)
Is it a repeatable issue?
Y/N
Steps To reproduce the issue
Explain how the maintainer can reproduce the issue.

Expected behaviour
Describe what you expect to happen.
Actual behaviour
Describe what actually happens.
Errors
Paste any errors that you see from the Error Log in the Firefly menu or share screenshots of the issue.

F5 the page / keep requesting, it will work eventually.

Thanks
just trying to log into faucet and getting bad gateway message!!

They are currently fixing / improving it as we speak (based on Discord messages)

Thanks again, problem solved and tokens have arrived, can we try and send each other some?

atoi1qp9wrcqjr6qw270adt3x9mdzrd5xlqc7swwvn3srsgg4kt68ppyqz7540zy

atoi1qz7gp4yak8wm9kk6r6uhmphxcp75fm7saj7vr7umfn7gdcvmaejj5hu488f
Much appreciated, seemed that transaction was very fast

assume it arrived OK

Hi, the faucet should be working now


Update SkiaSharp to 2.88.8

What does the pull request do?

Updates to latest released version of SkiaSharp.

You can test this PR using the following package version. 0.10.999-cibuild0020645-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

You can test this PR using the following package version. 0.10.999-cibuild0020649-beta. (feed url: https://nuget.avaloniaui.net/repository/avalonia-all/index.json) [PRBUILDID]

@maxkatz6, FYI newest Skia version produces strange artifacts in images, I created an issue for it - https://github.com/mono/SkiaSharp/issues/2084


All ladder opponents appear as Anonymous in my matches tab

When I go to my matches tab, every opponent appear as 'Anonymous':

Edit: If in the meanwhile one of my matches is simulated, the opponents' names are replaced with the correct ones.

Probably related to 0c0f075

Probably some bug or race condition in the loadNames logic.


Content not changing on Link click but it does on history change.

Sorry to post this here, but I'm looking to fix this and didn't get it through the docs or the example in the source code.
I've this code (started with root in /frontend for migration reasons):
class App extends Component {
  render() {
    const links = [
      '/frontend',
      '/frontend/season',
      '/frontend/episode'
    ].map(l =>

        <Link to={l}>{l}</Link>

    );
    console.log('render');
    return (

App Container

        {links}
        {this.props.children}

    );
  }
}
App.propTypes = {
  children: PropTypes.node
};
function mapStateToProps(state) {
  return {
    routerState: state.router
  };
}
connect(mapStateToProps)(App);

const rootReducer = combineReducers({
  episode,
  router: routerStateReducer
});

const store = compose(
  reduxReactRouter({ createHistory})
)(createStore)(rootReducer,initialState);

class Root extends Component {
  render() {
    return (

        <ReduxRouter history={history}>
          <Route path="/frontend" component={App}>
            <Route name="episode" path="episode" component={EpisodeApp} />
            <Route name="season" path="season" component={SeasonApp} />
          </Route>
        </ReduxRouter>

    );
  }
}

React.render(
  <Provider store={store}>
    {() => <Root/>}
  </Provider>
  , document.getElementById('root'));

The thing is that when I press the links nothing changes and the App doesn't re-renders its children, but when I go back and forth using the browser navigation it does work. Where am I screwing this up?
Thanks a lot!

Update:
Replace this line:
<ReduxRouter history={history}>

by this one (so removing history object):
<ReduxRouter>

Make it work, not sure why.


The rated wind speed power of 5MW wind turbine is calculated as 4.6MW

The power of semi-submersible wind turbine is 4.6MW at rated wind speed（11.4m/s）, no pitch, roll and yaw. Is the power at rated wind speed equal to 5MW?

Other questions in the forum also have related issues such as https://github.com/OpenFAST/openfast/issues/1634, attributed to the version update, but other reason was given.

Dear @sdu20951,
Are you referring to the OpenFAST model of the NREL 5-MW baseline wind turbine atop the OC4-DeepCwind semisubmersible?  It sounds like you have disabled the platform degrees of freedom, at least in roll, pitch, and yaw; is that correct?  What other simulation settings have enabled?
Best regards,

Dear @sdu20951,
Are you referring to the OpenFAST model of the NREL 5-MW baseline wind turbine atop the OC4-DeepCwind semisubmersible? It sounds like you have disabled the platform degrees of freedom, at least in roll, pitch, and yaw; is that correct? What other simulation settings have enabled?
Best regards,

dear jjonkman
The modified file is as follows，Other files in the example are not modified.

Dear @sdu20951,
Do you reach rated power if you increase the wind speed a bit higher than 11.4 m/s?  If so, I would guess this difference is attributed to changes in the aerodynamics model between the old FAST v7 and newer OpenFAST versions, as discussed in the forum topic (https://forums.nrel.gov/t/nrel-5mw-reference-wt-power-curve/2359) cross-referenced in the issue you cite.
Best regards,

Dear @sdu20951
I've had the same problem as you before. Have you solved the problem and could you explain the cause of the problem for me? Thank you in adcance.


Download and understand control datasets

Understanding the schema and format of the control datasets so as to craft optimal prompts for zero-shot and fine-tuned eval of closed source VLMs on control tasks.

TODO

Identifying what each action value means.
How to update the environment and get the accumulated rewards during the inference stage?

Action definitions

DM Lab (Discrete) => Need more investigation...

https://github.com/google-deepmind/deepmind-research/blob/master/rl_unplugged/dmlab_r2d2.ipynb

ALE Atari (Discrete)

https://gymnasium.farama.org/environments/atari/complete_list

Baby AI (Discrete)

https://minigrid.farama.org/environments/babyai

MuJoCo (Continuous)

https://gymnasium.farama.org/environments/mujoco

DM Control Suite (Continuous)

https://github.com/google-deepmind/dm_control/tree/main/dm_control/suite

V-D4RL (Continuous)

Extended from DM Control Suite

Meta-World (Continuous)

https://arxiv.org/pdf/1910.10897
https://huggingface.co/datasets/jat-project/jat-dataset/viewer/metaworld-peg-unplug-side
https://github.com/Farama-Foundation/Metaworld/blob/08489b46faffe3bfdeaa1287fa108260e74e194d/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L558

Procgen (Discrete) => Need more investigation...

https://github.com/openai/procgen/blob/5e1dbf341d291eff40d1f9e0c0a0d5003643aebf/procgen/env.py#L155

Contact to each author?

DM Lab (First authors of RL Unplugged)

Caglar Gulcehre: https://www.caglarg.com
Ziyu Wang: https://www.linkedin.com/in/ziyu-wang-433b4614
Alexander Novikov: https://www.linkedin.com/in/alexander-novikov-b0a968a6/?originalSubdomain=uk
Tom Le Paine: https://tomlepaine.github.io

Procgen (First author of Procgen)

Karl Cobbe: https://www.linkedin.com/in/kcobbe


Access devtools protocol inside puppeteer

Hi,
I'm trying to get all styles for element on page and for that i want to use devtools-protocol, but i cannot find any way to access this function and many other. I would be very grateful for help :)
Windows 10
Chrome: 63.0.3239.84 (64b)
puppeteer: 0.13.0
node: 8.9.3

Here is an explanation for your problem #31 .
https://github.com/GoogleChrome/puppeteer/issues/31

@Everettss thank you this is exactly what i've been looking for :D ❤


added interface for confirmed and forecasted dr events

and modified corresponding services to publish on BW accordingly for both PG&E and SCE namespaces

Is this ready for pulling in? I noticed you made some commits recently


Fix appveyor example with regards to Python3

Changelog: Fix appveyor example with regards to Python3

[ ] Refer to the issue that supports this Pull Request.
[x] If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.

The appveyor example caused errors with on Appveyor VS2017 image where both Python2 and Python3 are installed and Python3 is the default one. The problems seems to be in the poor support of Python2 while Python3 works well.
Python2 was selected by default because it was in PATH while Python3 weren't regardless of env variables.
The fixes to PATH in these PR should fix it.
I tested it with one of packages with just slightly different PATH. But PYTHON_HOME was still in the first place. It worked properly.
I'd also update the matrix with VS2019. But it's another story :)

[ ] I've read the Contributing guide.
[ ] I've followed the PEP8 style guides for Python code.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I guess there is a lot of stuff to change after Python2 deprecation. so my PR is rather meaningless. It's better to change all at once.
So I closed it.


[BEAM-918] Allow users to define the storage level via pipeline options

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[X] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[X] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[X] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[X] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

R: @amitsela

First "ugly" attempt, base for discussion.

@jbonofre now I understand your question from yesterday 😉
How about having STORAGE_LEVEL a static member in.. somewhere ? EvaluationContext / SparkRuntimeContext ? not sure..
Also, you can avoid null by setting a default of: StorageLevel.MEMORY_ONLY which is Spark's default, and call persist() only.

New approach updating the Dataset interface and both BoundedDataset and UnboundedDataset.

@jbonofre looking good!
Added a few comments.

Rebased and updated according to your comments.

LGTM pending Jenkins&Travis. Thanks!

Good point. I added a test at least to check that the runner is not broken by the change. Let's see if we can add a "hook" to get the RDD storage level (but not so easy as we don't have "RDD -> PCollection" way ;)).

Added a test on the storage level.

Rebased and updated according to Amit's comments.

It generally LGTM, but I'm wondering if we can avoid adding test transformations in TransformTranslator and avoid JUnit assertions in non-test packages...
@jbonofre mind if we keep this open for a few days, while exploring the possibility to use TestSparkRunner with an overriding translators for test ? where we could naturally add your (and future) evaluators ?

No problem at all. Let me try to create a TestSparkRunner with a extend of evaluators dedicated to test.

There is already a TestSparkRunner that delegates to SparkRunner.
You'd probably need to "delegate" translators as well.

Good point, let me try something there. Thanks !

As discussed, I propose a two steps:

I'm updating this PR to use PAssert instead of using Assert directly in the evaluator
I will prepare another PR to use the TestSparkRunner with a custom TransformTranslator (we have another Jira about that).

Updated the PR to avoid Assert in the TransformTranslator. Instead, the StorageLevelPTransform provides a PCollection<String> containing the storage level description. Then, we can use PAssert in the test.

Jenkins fails on a couple of unused imports.

Yes, my bad: I did a mvn clean verify but forgot mvn clean install -Prelease,apache-release.

LGTM pending tests.
@jbonofre are you merging ? if so, you can probably squash this into 1 or 2 commits..

@jbonofre not sure what asfbot tries to run... but Jenkins seems satisfied.

Yeah I gonna squash and merge. Thanks !

Waiting the end of travis to merge.

Test passed, merging.


store defined with function and ref as state variable not working correctly

Reproduction
Please visit Stackblitz to check the 4 different (d, fp, fap, fnp) simple stores I created.
Actual behavior
Check the console: fp and fnp are not console logged during the subscription
  store.$subscribe((mutation) => {
    console.log('mutation', mutation.events.newValue);
  });

Check the localstorage: fp is not persisted
  if (options.persist) {
    for (const persistKey of options.persist) {
      console.log(persistKey);
      store.$state[persistKey] = useLocalStorage(
        store.$id,
        store.$state[persistKey]
      );
    }
  }

and
  { persist: ['myStateVar'] }

The difference between fp and fap is:
fp contains Ref
fap contains RemovableRef from @vueuse/core
Expected behavior
fp and fnp should callback the subscription (seems to be related when the var is a ref)
d, fap, fp should all be persisted (when overwriting the ref in the plugin with useLocalStorage seems not to fix the issue of fp)

@posva
Thank you for your quick reply and help! I fixed it and applied it on my stackblitz test environment.
However a new issue now raised:
import { defineStore, acceptHMRUpdate } from 'pinia';
import { ref, computed } from 'vue';

export const useFPStore = defineStore(
  'fp',
  () => {
    const myStateVar = ref('init fp');

    const myComputed = computed(() => {
      const fp = useFPStore();
      console.log('myStateVar.value from fp', myStateVar.value);
      console.log('fp.myStateVar', fp.myStateVar);
      return myStateVar.value;
    });
    return {
      myStateVar,
      myComputed,
    };
  },
  { persist: ['myStateVar'] }
);

if (import.meta.hot) {
  import.meta.hot.accept(acceptHMRUpdate(useFPStore, import.meta.hot));
}

By assigning v to store[persistKey] I lose unfortunately the reference to
myStateVar.value in my computed getter, due to the override in plugin.
(the value stays to 'init fp' for myStateVar.value)
In your documentation you pass (state) to each getter. Is there a similar API maybe I can use by defining my store with a function ?
Or is the best way, to inject the current state in the computed by myself with const fp = useFPStore();

PS:
I know I can also have the useLocalStorage since the beginning, but this would make my code less flexible...
I prefer doing it by plugin.
And of course another idea, which I could do for my persistPlugin, is to subscribe to the mutations and do the getItem, setItem etc myself, but this is more boiler code, and looks less nicer...


Make preprocessor optional

I have trouble using the preprocessor on my computer (Ubuntu 16.04, works on my Laptop but not on my workstation, still haven't figured out why) and therefore can not use the whole Test Suite.
Please make the preprocessor optional, because it is not an essential part of the Test Suite.

Is the problem the new preprocessor? Does the old one (for import statements) work for you?

I've removed the new preprocessor from the main mjtest version.

The old preprocessor works for me. Thank you!


Update/Add buildah

Tool name
buildah
Tool license
Apache License 2.0
Add or update?

[ ] Add
[X] Update

Desired version
1.37.0
Approximate size
No response
Brief description of tool
Current version is pretty old and missing many bugfixes/features
/usr/bin/buildah version
Version:         1.23.1
Go Version:      go1.17
Image Spec:      1.0.1
Runtime Spec:    1.0.2-dev
CNI Spec:        0.4.0
libcni Version:
image Version:   5.16.0
Git Commit:
Built:           Thu Jan  1 00:00:00 1970
OS/Arch:         linux/amd64
BuildPlatform:   linux/amd64
URL for tool's homepage
https://buildah.io/
Provide a basic test case to validate the tool's functionality.
buildah version

Platforms where you need the tool

[ ] Azure DevOps
[X] GitHub Actions

Runner images where you need the tool

[X] Ubuntu 20.04
[X] Ubuntu 22.04
[X] Ubuntu 24.04
[ ] macOS 12
[ ] macOS 13
[ ] macOS 13 Arm64
[ ] macOS 14
[ ] macOS 14 Arm64
[ ] macOS 15
[ ] macOS 15 Arm64
[ ] Windows Server 2019
[ ] Windows Server 2022

Can this tool be installed during the build?
No response
Tool installation time in runtime
No response
Are you willing to submit a PR?
I would but I don't know how to provide a tool update PR

Hi @ramonsmits , Thanks for providing the issue .We will update once validated . thanks

@vidyasagarnimmagaddi FYI:
When I do:
sudo apt-get update
sudo apt-get -y install buildah

It outputs:
buildah is already the newest version (1.23.1+ds1-2).

Which is strange.. I think it misses a repository?

According to docs it should be part of Ubuntu

The buildah package is available in the official repositories for Ubuntu 20.10 and newer.

Source: https://github.com/containers/buildah/blob/main/install.md#ubuntu

Hi
We understand that 1.37.0 is the most recent version of Buildah, and the image presently has Buildah 1.33.7. We will continue to update the system with the most recent stable versions, so please bear with us till next week. We will keep you updated.
ref link : https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md

Dear @ramonsmits
The exact cause is still being looked into,  please bear with us.

Dear @ramonsmits
Thanks for your diligence, we understand that you required the latest version of Buildah tool,
this version is   pinned for update and Ubuntu has 24.04 has Buildah 1.33.7 version, this version is pinned, and Ubuntu 24.04 has Buildah 1.33.7 version, we recommend to use this, and to inform you that there are many reasons that few selected tools may not be updated regularly, such as dependency conflicts, regularity, performance related, etc.
The workaround is to use the latest version in your runtime, and we will continue to monitor and update the latest version in the following days.


clickhouse and poller settings

TL;DR
Added support for configuring parallel pollers and asynchronous inserts in ClickHouse storage.
What changed?

Added a new configuration option poller.parallelPollers to control the number of parallel pollers.
Introduced asyncInsert configuration for ClickHouse storage in main, staging, and orchestrator settings.
Updated the Poller to use the configured number of parallel pollers.
Modified ClickHouse connection setup to enable async inserts when configured.
Fixed error messages in InsertBlockData function to correctly reflect insert operations.

How to test?

Update your configuration to include the new poller.parallelPollers setting.
Set asyncInsert: true in the ClickHouse configuration for main, staging, or orchestrator storage.
Run the application with these new settings and monitor performance.
Check logs for debug messages showing executed queries.
Verify that data is being inserted correctly with the new parallel polling and async insert features.

Why make this change?
This change aims to improve performance and scalability of the application by:

Allowing multiple pollers to work in parallel, potentially increasing the rate of data collection.
Enabling asynchronous inserts in ClickHouse, which can improve write performance for large datasets.

These optimizations should lead to faster data processing and reduced latency, especially when dealing with high volumes of blockchain data.

#110  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @iuwqyir and the rest of your teammates on  Graphite

what's the reason to have asyncInserts?

To not overload Clickhouse with too many inserts


V2Flag overwrites existing flags

Zemurin
So, what probably happens, is we create a flag for C09, then detect it's mex, and save the flag as mex
just order needs to be reversed
Iohannes
That's what happens for sure
The converter should go:
-create a flag for C09
-IF C09 becomes MEX, scrap it and keep MEX flags
-IF C09 gets a new Xxx tag, use the generated flags for that Xxx tag

eh it's even more convoluted
as englishname.tga takes priority over TAG.tga


[nrf fromtree] modules: hal_nordic: Update nrfx to have doc fixes

Update the hal_nordic module revision, to have Doxygen doc
fixes that solves warnings about non-existing CHANGELOG.md file
and nrfx_atomic Doxygen group.
This is needed for following PR: https://github.com/nrfconnect/sdk-nrf/pull/4722

@gmarull could you check whether this PR fixes your PR https://github.com/nrfconnect/sdk-nrf/pull/4722 ?

Can confirm the fixes solve the problem, thanks

Marking as DNM for now - let's wait for the upstream PR to get merged: https://github.com/zephyrproject-rtos/zephyr/pull/36018


Feature/fake data front end

This is to be merged into master once the #31 has been merged. Leaving as WIP until that time

:rocket: deployed to http://profiles-pr-38.dev.c2s.nhschoices.net

:rocket: deployed to http://profiles-pr-38.dev.c2s.nhschoices.net

:rocket: deployed to http://profiles-pr-38.dev.c2s.nhschoices.net

:rocket: deployed to http://profiles-pr-38.dev.c2s.nhschoices.net

:rocket: deployed to http://profiles-pr-38.dev.c2s.nhschoices.net

:rocket: deployed to http://profiles-pr-38.dev.c2s.nhschoices.net


Documentation should reference more recent Docker Compose command

The current documentation states references docker-compose up when it should use docker compose up instead. More details here.
The text to be changed is in the Preparing to Build the Application section of the README.adoc:

You can now run docker-compose up to start the RabbitMQ service. Now you should have an external RabbitMQ server that is ready to accept requests.

Should instead be:

You can now run docker compose up to start the RabbitMQ service. Now you should have an external RabbitMQ server that is ready to accept requests.

Additionally, there is another reference to docker-compose up that should be changed to docker compose up in the Setting up the RabbitMQ Broker:

Manually run the compose.yaml file with docker-compose up

Should instead be:

Manually run the compose.yaml file with docker compose up


ci: speed up the PR builds for Windows+CMake

Use a Debug build for faster PR builds, a future change will enable
a Release build for post-submit builds. Also parallelize the unit
tests it saves maybe 30 seconds, but I will take it. Prepare the
ground work for running the integration tests for this build too.
Part of the work for #3489

This change is

I think this takes us from about 45m (sometimes 50m, sometimes 70m!) to about 35m. Not enough, but I will take whatever I can get.


push live

Docs Build status updates of commit 20a0614:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

AdaptiveCards/resources/media.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Fixed pooling command printing

Command is now printed with args in a driver-accepted format, rather than old non supported one.
Fixes #1647.

@atamazov I can't set you as a reviewer, but here we are.

@DrizztDoUrden  I built your branch and tried to test locally:
/root/MIOpen/build# ./bin/MIOpenDriver pool -M 0 -n 2 -c 2 -H 1 -W 5 -y 1 -x 1 -p 0 -q 0 -v 1 -u 1 -m avg -F 1 -t 1
MIOpenDriver pool -M 0 -n 2 -c 2 -H 1 -W 5 -y 1 -x 1 -p 0 -q 0 -v 1 -u 1 -m avg -F 1 -t 1
Input Flag: n Not Found !

anything I might have missed here?

@jun that line won't be parced by the library at all since the command line args update. The correct line would be like this:
/root/MIOpen/build# ./bin/MIOpenDriver pool -M 0 --input 2x2x1x5 -y 1 -x 1 -p 0 -q 0 -v 1 -u 1 -m avg -F 1 -t 1

My fix doesn't change the arguments, it changes the printout to the correct format. You could use short key instead of input, but i don't remember which one, should be in --help if you care, but that makes it unreadable for anyone unfamiliar with the syntax.

@junliume

MIOpenDriver pool -M 0 -n 2 -c 2 -H 1 -W 5 -y 1 -x 1 -p 0 -q 0 -v 1 -u 1 -m avg -F 1 -t 1
Input Flag: n Not Found !

anything I might have missed here?

Please look at #1313 for more info about the new syntax.

@junliume

MIOpenDriver pool -M 0 -n 2 -c 2 -H 1 -W 5 -y 1 -x 1 -p 0 -q 0 -v 1 -u 1 -m avg -F 1 -t 1
Input Flag: n Not Found !

anything I might have missed here?

Please look at #1313 for more info about the new syntax.

Thanks, @atamazov !
These lines are grepped from the debug output from https://github.com/ROCmSoftwarePlatform/MIOpen/issues/1642
So I guess something need to be changed in the debug output?
BTW~ #1642 is still an urgent issue :(

@junliume We are working on #1642 together with @DrizztDoUrden

@junliume As it only changes log lines not even used in the tests, I don't think we need to spend CI time for the rest of the tests, but the PR is no hurry either.

@junliume this has passed the CI


Fix example to work

what

Change example to work

why

It is not work because of alias invalid.

references

From error message,

| Error: Invalid provider configuration alias
│
│ An alias must be a valid name. A name must start with a letter or underscore and may contain only letters, digits, underscores, and dashes.

The changes don't seem to reflect the description. Feel free to update and we will reopen.


Your website SSL certificates have expired

Just a heads up that your SSL certificates on the kunkalabs.com domains have expired and your customers are getting security warnings when visiting the websites.

Thanks so much for the heads up - fixed now!


bugfix ipcServerImpl invalid iterator

Summary: Fix a bug in ipcServerImpl where a invalid iterator could be used
Type: Fix
Test Plan: UT/ CT
Jira: NO-JIRA

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/48577/

Sanity tests for ['BCM Reference Refapp2'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/48581/


Make Query1(String text, Term[] args) in Query.java public?

Hi,
Class Query.java has a useful public method to convert a string with args into a Term:
private static Term Query1(String text, Term[] args)

Would be very nice to have this as a public method as it is useful to get the term itself, not just the whole query. And maybe rename it to queryBuilder or something?
Can this be done? It is a small change and would help, otherwise I need to replicate that code in my framework.
Thanks in advance,
Sebastian

mmm yes you are right, Util.textParamsToTerm(String text, Term[] params) hsa that already. In fact, Query1 uses it (well its non params version) and then injects the params into the ? placeholders...
Now this is interesting... It seems to me we would like to get rid of Query1 and use the already existing tool in org.jpl7.Util.
As you said Query1 seems to be "careful" and transform a text Atom into a Compound, but  textParamsToTerm should handle or we should make it to handle that.
In the end, we would like to write Query without using Query1, that is ,we want:
	public Query(String text, Term[] args) {
		this(Util.textParamsToTerm(text, args));
	}

so the strange case would be that text is not a compound text with placeholders ?, but just an atom and you do pass arguments...
presumably you should have as many arguments as ? in text. There must be an error otherwise...

OK all tested. and is good:

No need to make Query1 public as there is  Util.textParamsToTerm(text, args) available.
In fact Query1 is still needed to allow building queries in two ways:

passing the name of the functor, and the args separately. A Compound is built form the two.
passing a string that is the full query but with placeholders ?, where the args will be "injected"
To do that Query1 generates a corresponding Term as needed depending on input of text.

Query1 has been renamed to buildQueryTerm
Have added standalone test org.jpl7.test.standalone.QueryBuilder to test all possible ways of building a Query and all possible exceptions that can be thrown.

All this is in commit 2507f1b73c06a46646ec994d6eab62687b85ea69


Mock Cluster Config Provider

Issue Description
In order to complete adequate testing of #159 Status Pusher, several mock providers must be made.  The scope of this card is to create a mock cluster config provider.  The real provider is being handled in #145.
Acceptance Criteria (How do we know when this is done?)
Functioning well enough to complete testing on #159.
Related Tickets
#159
#145

Waiting for use in #159 tests


Create event website

Description
Create an Event Website for KCD Austria.
Additional information
Tooling
CNCF recommends using Bevy platform for "Website and ticket support".
Examples

KCD Munich 2022
KCD Munich 2023
KCD France 2023
KCD Zurich 2023

The request has been put in to create the group, i will let you know whe it is ready so we can start working on the content


clicked on Read our tutorial and got the Issue reporter - Spyder has encountered and internal problem!

Description
What steps will reproduce the problem?

Opened Spyder
was prompted to get latest update
Did that using
returned to Spyder and clicked on "tutorial" in the Usage dialog box New to Spyder? Read our tutorial
then got this internal error, issue reporter
Traceback
  File "C:\Apps\Anaconda\lib\site-packages\spyder\plugins\help.py", line 752, in handle_link_clicks
  File "C:\Apps\Anaconda\lib\site-packages\spyder\plugins\help.py", line 746, in show_tutorial
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Apps\\Anaconda\\lib\\site-packages\\spyder\\utils\\help\\tutorial.rst'

Versions

Spyder version: 3.3.6
Python version: 3.7.4
Qt version: 5.9.6
PyQt5 version: 5.9.2
Operating System: Windows 10

Dependencies
pyflakes >=0.6.0  :  2.1.1 (OK)
pycodestyle >=2.3 :  2.5.0 (OK)
pygments >=2.0    :  2.4.2 (OK)
sphinx >=0.6.6    :  2.2.0 (OK)
rope >=0.9.4      :  0.14.0 (OK)
jedi >=0.9.0      :  0.15.1 (OK)
nbconvert >=4.0   :  5.6.0 (OK)
pandas >=0.13.1   :  0.25.1 (OK)
numpy >=1.7       :  1.16.5 (OK)
sympy >=0.7.3     :  1.4 (OK)
cython >=0.21     :  0.29.13 (OK)
qtconsole >=4.2.0 :  4.5.5 (OK)
IPython >=4.0     :  7.8.0 (OK)
matplotlib >=2.0.0:  3.1.1 (OK)
pylint >=0.25     :  2.4.2 (OK)

Hi @MikeD-Respond after updating, did you restart Spyder? Also, what command did you use to update Spyder? (seems like the command is missing in the issue description). Thank you for the feedback!

Closing due to lack of response.


Remove version: igoogolx.lux version 0.3.0+1

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

igoogolx.lux prVersion 0.3.0+1 is less than ManifestVersion 0.4.2


Upgrade to cardano-api-<IP_ADDRESS>

Changelog
- description: |
    Upgrade to `cardano-api-<IP_ADDRESS>`
# uncomment types applicable to the change:
  type:
  # - feature        # introduces a new feature
  # - breaking       # the API has changed in a breaking way
  - compatible     # the API has changed but is non-breaking
  # - optimisation   # measurable performance improvements
  # - improvement    # QoL changes e.g. refactoring
  # - bugfix         # fixes a defect
  # - test           # fixes/modifies tests
  # - maintenance    # not directly related to the code
  # - release        # related to a new release preparation
  # - documentation  # change in code docs, haddocks...

Context
Additional context for the PR goes here. If the PR fixes a particular issue please provide a link to the issue.
How to trust this PR
Highlight important bits of the PR that will make the review faster. If there are commands the reviewer can run to observe the new behavior, describe them.
Checklist

[ ] Commit sequence broadly makes sense and commits have useful messages
[ ] New tests are added if needed and existing tests are updated. See Running tests for more details
[ ] Self-reviewed the diff

This PR has been absorbed by #463. May you close it @newhoggy ?


enh: add application to create auto constrast of image

Add application from IRTK: https://github.com/BioMedIA/IRTK/blob/master/applications/autocontrast.cc
Auto contrast an image.

I think I have addressed all the comments here. Let me know if it is ready to be merged.


API routing?

How to setup routing so I can communicate with ASP.NET Web API's in /api/* - anything else should go to Angular.

Also same problem -- when running via VS without debugging, cannot get port designation to work, and ng serve opens as NON-administrator, which breaks CLI application (see open issue 9)

Make sure app.UseMvc() is placed before app.UseNgProxy() and app.UseNgRoute() in Statup.Configure(). and also make sure you specified "baseHref": "/" in .angular-cli.json. You can find instructions in the Readme_AngularCLI.txt file installed by the item template.

I had app.UseMvc() afterwards. Thanks. Will try again.


Export TsigProvider functions (Resolves #1296)

Export TsigProvider functions to allow use when writing custom DNS server handlers.
Context in #1296
When you define a custom DNS handler there is no way to specify a custom TsigProvider.

TsigGenerate and TsigVerify pass hard-coded the default TsigProvider and the tsigGenerateProvider and tsigVerifyProvider are unexported.

The only place I saw that you can define the TsigProvider is in the DNS client which is unrelated to the DNS handler for receiving traffic.

why can't you use co.TsigProvider ?

why can't you use co.TsigProvider ?

As far as I understand the code this is only when we initiate a connection as a client.
How can I do that if I am the server inside the ServeDNS handler function?
func (server *DNSServer) ServeDNS(writer dns.ResponseWriter, query *dns.Msg) {
  ...
  reply.SetTsig(tsig.Hdr.Name, tsig.Algorithm, 300, time.Now().Unix())
  data, _, err = dns.TsigGenerate(reply, tsigSecret, tsig.MAC, false)
  writer.Write(data)
}

After digging more into the code, the problem seems to be that the server does not support TsigProvider. So this means that you have to directly use the TsigGenerateProvider function and then with the writer to use Write instead of WriteMsg.

I'm going to close this PR as this issue is mostly a design issue on TsigProviders for server/xfr rather than implementation


Create DEV Helm Repository for Tractus-X

This PR implements:

a GO binary, pulling released Helm charts from GH Org repositories, build DEV Helm repository
GH Workflows for building the GO binary and the Helm repository
README.md file which will be shown in Helm repo root (→ GH page)
README.md in Git repo root filled with content.

The Helm repository will be available at https://eclipse-tractusx.github.io/charts/dev
Things to discuss/ ToDo's for next iteration

Is it best practice to build a binary and push it back to the repo and use this binary with a workflow?
Create a GH release for binary helmRepoIndex, which makes it reusable in other repositories as well and adjust GH worflow.
Implement Stable Helm Repository
Create UnitTest
Authenticated github client isn't necessary for workflow execution. This was integrated during local testing and hitting the GH API rate limit. Either remove, or add argument to Go binary to use it or not.

Go binary helmRepoIndex

get a list of all GH Org repositories
for each repository:

check if GH Pages are enabled, if not continue
If a index.yaml in repos GH Page is available download
merge downloaded  index.yaml with local one
delete downloaded index file

GH Workflows
Build Go - helmRepoIndex
Build the Go binary and stores it at .github/bin.  .github/bin directory is created if not presend. Go binary is pushed back to repository to make it available for the second workflow.
Build Helm Repository - DEV
This workflows fetches from all GH Org repositories their local Helm repository file (index.yaml) and merges it into the central charts/dev/index.yaml. No released Helm charts are fetched and stored localy.
Updated charts/dev/index.yaml is pushed back to repository.

Code and repo overall looks good to me. Licence checking has to be done, before merging this PR

Currently, a lot of licences are not recognized, so we will need to re-check that on a regular basis


[Cherry-Pick} Added Configuration to enable/disable predicates

https://github.com/volcano-sh/volcano/issues/185

/lgtm
/approve

/lgtm
/approve


cxxopts: 3.0.0 -> 3.1.1

Description of changes
Updated to 3.1.1, removed a Nix-specific patch as it has been merged upstream. Also patched python3Packages.chiapos to comply with the new exception naming scheme.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 232786 run on x86_64-linux 1

14 packages built:

avizo
chia
chia-dev-tools
chia-dev-tools.dist
chia.dist
cxxopts
libtree
nwg-panel
nwg-panel.dist
pamixer
python310Packages.chiapos
python310Packages.chiapos.dist
python311Packages.chiapos
python311Packages.chiapos.dist


Replace mirror-cn profile with Apache officially suggested dlcdn

Why are the changes needed?

Replace mirror-cn profile with Apache officially suggested dlcdn.
mirror-cn is only suite for CN, while dlcdn is much wiser.
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

Codecov Report

Merging #1970 (a0756b4) into master (770499c) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head a0756b4 differs from pull request most recent head a325f82. Consider uploading reports for the commit a325f82 to get more accurate results

@@            Coverage Diff            @@
##             master    #1970   +/-   ##
=========================================
  Coverage     60.36%   60.36%
  Complexity       69       69
=========================================
  Files           305      305
  Lines         14859    14859
  Branches       1927     1927
=========================================
  Hits           8969     8969
- Misses         5111     5113    +2
+ Partials        779      777    -2

Impacted Files
Coverage Δ

...rg/apache/kyuubi/engine/trino/TrinoStatement.scala
67.46% <0.00%> (-2.41%)
:arrow_down:

...mon/src/main/scala/org/apache/kyuubi/Logging.scala
39.72% <0.00%> (-1.37%)
:arrow_down:

.../org/apache/kyuubi/operation/KyuubiOperation.scala
61.33% <0.00%> (-1.34%)
:arrow_down:

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
96.36% <0.00%> (+0.14%)
:arrow_up:

...ala/org/apache/kyuubi/session/SessionManager.scala
85.95% <0.00%> (+0.82%)
:arrow_up:

...n/scala/org/apache/kyuubi/KyuubiSQLException.scala
93.97% <0.00%> (+1.20%)
:arrow_up:

...ache/kyuubi/operation/KyuubiOperationManager.scala
95.00% <0.00%> (+2.50%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 770499c...a325f82. Read the comment docs.


[Enhancement] cloud native support external table

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Problem Summary(Required) ：

support creating cloud native table as external table.
Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[x] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

[FE PR Coverage Check]
:disappointed: fail : 3 / 82 (03.66%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/leader/LeaderImpl.java
0
65
00.00%
[867, 1019, 1020, 1021, 1022, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1084, 1085, 1086]

:large_blue_circle:
com/starrocks/catalog/ExternalOlapTable.java
0
14
00.00%
[102, 142, 150, 151, 162, 174, 175, 176, 278, 298, 302, 306, 354, 355]

:large_blue_circle:
com/starrocks/catalog/Table.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/planner/OlapTableSink.java
2
2
100.00%
[]

@Mergifyio backport branch-3.0


PluralSight download ERROR

Hello.
Since 01/03/2017 the following error I started to have error to download the videos of the site of PluralSight. Today I updated the version of youtube-dl, installed in Ubuntu, and the following error was shown:
_youtube-dl -i -v -u user -p password -o "%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s" --all-subs "https://app.pluralsight.com/library/courses/windows-server-2012-mcsa-70-410"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-i', u'-v', u'--sleep-interval', u'60', u'--max-sleep-interval', u'100', u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'-o', u'%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s', u'--all-subs', u'https://app.pluralsight.com/library/courses/windows-server-2012-mcsa-70-410']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.03.02
[debug] Python version 2.7.6 - Linux-3.4.0+-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: none
[debug] Proxy map: {}
[pluralsight:course] windows-server-2012-mcsa-70-410: Downloading course JSON
[download] Downloading playlist: Windows Server 2012 R2 (70-410) MCSA and the 70-410 Exam
[pluralsight:course] playlist Windows Server 2012 R2 (70-410) MCSA and the 70-410 Exam: Collected 13 video ids (downloading 13 of them)
[download] Downloading video 1 of 13
[pluralsight] Downloading login page
[pluralsight] Logging in as<EMAIL_ADDRESS>ERROR: Unable to log in; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 761, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 426, in extract
self.initialize()
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 384, in initialize
self._real_initialize()
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 60, in _real_initialize
self._login()
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/pluralsight.py", line 100, in login
raise ExtractorError('Unable to log in')
ExtractorError: Unable to log in; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
**ERROR: '**NoneType' object has no attribute 'copy'
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 772, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 950, in process_ie_result
extra_info=extra)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 845, in process_ie_result
new_result = info.copy()
AttributeError: 'NoneType' object has no attribute 'copy'
What could be going wrong?
Thank you!

And....I can login by website.

Add --dump-pages and post the output.

Hi.
The dump file is attached.
youtube-dl_issue_12345_dump_pages.txt

How To Get a Free Full Plural Sight Subscription / Download Courses automatically from PluralSight
https://andrewramzyblog.wordpress.com/2016/04/25/how-to-get-a-free-full-plural-sight-subscription-download-courses-automatically-from-pluralsight/

This is what pluralsight have said to you:

Great, resolved!
Thank you!

Hi @rgmorato
How did you solved this one.
I did log out and log in again, but nothing happened
Thanks


fix: switchChain-handler and signTypeData

Summary

The parts will be implemented in later Pull Requests
Related Links
#74 #80

Asana:
Mockup:
Checklist

[X] Pasted Asana or Mockup link.
[X] Tagged labels.

Prerequisite/Related Pull Requests

Screenshot/Gif

Why need prod release now?


Added power support for the travis.yml file with ppc64le. and update go versions for package:  xff

Added power support for the travis.yml file with ppc64le. This is part of the Ubuntu distribution for ppc64le.
This helps us simplify testing later when distributions are re-building and re-releasing.
updated the go version go:1.13, 1.14 and 1.15

kindly respond and merge the code with power architecture. For more info tag @gerrith3


gilbraltar does not work anymore with GCC 11 & GCC 12.

With a deep look, the help of @TheLortex and @pitag-ha. many beers and a MirageOS retreat, we figure out that the compilation of variadic arguments with GCC 11 and GCC 12 produces a bad assembly for a RPi4. At least, that the only real difference we noticed.

See #21, powered by the MirageOS retreat, beers and friends.


wip - Ensure that all scorecard tests are checked and with changes made in the PR

Description of the change:

all e2e test with are using the dev tag image

add test:dev scorecard images used to be built in the CI

add custom test for ansible/helm to ensure that this test will work with them

cleanup scorecard and ensure that these images and binaries will be generated in the place and at the same way ansible/helm. see:

fix targets to be able to generate the binaries that are copied to the images on them and in this way check the PR with the changes performed.

remove the old bin/entrypoint standard in favour of the scorecard follow up the same ansible/helm approach to define the user/dir that should be used.

Motivation for the change:

Closes: #3839

Checklist
If the pull request includes user-facing changes, extra documentation is required:

[ ] Add a new changelog fragment in changelog/fragments (see changelog/fragments/00-template.yaml)
[ ] Add or update relevant sections of the docs website in website/content/en/docs

The issue was solved in another pr.
Closing this one.


Make environment variable population more generic

Fixes #122

@lukehoban @lblackstone sorry for the noise. Could you take another look? I went ahead with the approach I proposed in https://github.com/pulumi/pulumi-kubernetes-operator/pull/125#discussion_r601134885


feat: mux with select

Awesome work and great test. To test the performance, you can use something like this below to initialize the plonky2 logger, which will print out the number of gates and rows in the circuit. The less gates the better :)
let mut builder_logger = env_logger::Builder::from_default_env();
builder_logger.format_timestamp(None);        builder_logger.filter_level(log::LevelFilter::Trace);
builder_logger.try_init()?;

for smaller value <100, random_access lighter, for larger value, random_access would overflow


AccessViolationException with SSL configuration (consumer)

I'm running Confluent.Kafka client (0.9.4) with .Net framework 4.5.1. While it works fine without SSL, consumer client throws AccessViolationException with SSL configuration:
{
            {"group.id", kafkaGroup},
            {"client.id", clientId},
            {"bootstrap.servers", kafkaServer},
            {"enable.auto.commit", true},
            {"security.protocol", "ssl" },
            {"ssl.key.location", "sslkey/client.keystore.jks"},
            { "ssl.key.password", "testtest" },
            { "auto.commit.interval.ms", 5000},
            { "default.topic.config", new Dictionary<string, object>()
                {
                    { "auto.offset.reset", "smallest" }
                }
            }

        }

looks like you need to review your configuration to me:
https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka
it's not good for us to be ever throwing an AccessViolationException though so I'll label this as bug and put it in the backlog.
please carefully review your config and update here if you are still seeing problems.

@mhowlett Double checked. If I comment "ssl.key.location" and run the code it work perfectly. If "ssl.key.location" is added code throws exception during consumer creation.
Stacktrace (Changed some sensible data to ***)
at Confluent.Kafka.Impl.LibRdKafka.NativeMethods.rd_kafka_conf_destroy(IntPtr conf)
   at Confluent.Kafka.Impl.LibRdKafka.conf_destroy(IntPtr conf)
   at Confluent.Kafka.Impl.SafeKafkaHandle.Create(RdKafkaType type, IntPtr config)
   at Confluent.Kafka.Consumer..ctor(IEnumerable`1 config)
   at Confluent.Kafka.Consumer`2..ctor(IEnumerable`1 config, IDeserializer`1 keyDeserializer, IDeserializer`1 valueDeserializer)
   at ***.KafkaConsumer..ctor(ILogger logger, String kafkaServer, String kafkaGroup, String clientId, String topic, Action`1 callbackFunc) in ***\KafkaConsumer.cs:line 41
   at ***.KafkaConsumerTest.UnitTest_Connection() in ***\KafkaConsumerTest.cs:line 52

Can you try specifying the full path to the keyt?

@edenhill same

Hi. I have (kind of) the same issue, but when I'm creating a publisher:
Exception thrown: 'System.AccessViolationException' in Confluent.Kafka.dll)
InnerException=null.
I'm trying to connect to a cloudkarafka instance using SSL and the certificates they supply.
Any idea on what could be happening? Thanks!
(Sorry if i should have got this to another thread.)

Can you provide some context? (kafka version, ok, configuration, some code snippets...)
You may want to try upgrading librdkafka.redist to the latest prerelease (0.11.0-RC2) as some changes have been made on SSL (no need to upgrade confluent.Kafka, just add reference to librdkafka.redist package) The version is still prerelease and should not be used in prod yet, but you may verify if this solve your issue

Thank you for the recommending 0.11.0-RC2, but it did not fix the issue for me.
I have sent this issue also to cloudkarafka guys, but no response yet, so i'm still hoping to find a solution with you.
Some more details:
Cloudkarafka has given me the certificates in a single file called "env" with no extension, that is structured like this:

CLOUDKARAFKA_CA="-----BEGIN CERTIFICATE-----[key]-----END CERTIFICATE-----"
CLOUDKARAFKA_CERT="-----BEGIN CERTIFICATE-----[key]-----END CERTIFICATE-----"
CLOUDKARAFKA_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----[key]-----END PRIVATE KEY-----"

(where [key] is the long scrambled char string)
I copied the [key] values into their own specific files, and added them to the producer configuration (to ssl.ca.location, ssl.key.location, ssl.certificate.location) as specified on their documentation page(https://www.cloudkarafka.com/docs-nodejs.html - i used the node.js guide because they told me it is the most similar to .net in this case)
This configuration, leads me to the 'System.AccessViolationException' when instantiating the Producer.
Stack trace:

at Confluent.Kafka.Impl.LibRdKafka.NativeMethods.rd_kafka_conf_destroy(IntPtr conf)
at Confluent.Kafka.Impl.LibRdKafka.conf_destroy(IntPtr conf)
at Confluent.Kafka.Impl.SafeKafkaHandle.Create(RdKafkaType type, IntPtr config)
at Confluent.Kafka.Producer..ctor(IEnumerable1 config, Boolean manualPoll, Boolean disableDeliveryReports)    at Confluent.Kafka.Producer2..ctor(IEnumerable1 config, ISerializer1 keySerializer, ISerializer1 valueSerializer, Boolean manualPoll, Boolean disableDeliveryReports)    at Confluent.Kafka.Producer2..ctor(IEnumerable1 config, ISerializer1 keySerializer, ISerializer`1 valueSerializer)
at TestCloudKarafka.Program2.Main(String[] args) in D:\Tut\confluent-kafka-dotnet-master\TestCloudKarafka\Program2.cs:line 50
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()

I need to specify now, that setting an empty string value instead of these certificates, i get "SSL handshake failed", but the Producer manages to get instantiated.

Issue resolved on my side, thank you for your quick involvement!
The issue was quite silly, for the separated certificates i saved into different files, by mistake i left the start&ending quotes inside the file. So the format of the pem was not quite correct.
Thank you, keep on the good work!

Given the AccessException, it actually come from
public static SafeKafkaHandle Create(RdKafkaType type, IntPtr config)
        {
            var errorStringBuilder = new StringBuilder(512);
            var skh = LibRdKafka.kafka_new(type, config, errorStringBuilder,
                    (UIntPtr) errorStringBuilder.Capacity);
            if (skh.IsInvalid)
            {
                LibRdKafka.conf_destroy(skh.DangerousGetHandle());
                throw new InvalidOperationException(errorStringBuilder.ToString());
            }

LibRdKafka.conf_destroy(skh.DangerousGetHandle()); throws (which seems normal as skh is invalid)
I'm not sure what we must do given conf in this case, rdkafka doc tate

conf is an optional struct created with rd_kafka_conf_new() that will be used instead of the default configuration.
The  conf object is freed by this function on success and must not be usedor destroyed by the application sub-sequently.

but when looking at rdkafka.c, it is also destroy when method fail if I'm not mistaking
@bjcryss @themoah fix on #263, you can take the nuget artefact here for test : https://ci.appveyor.com/project/mhowlett/confluent-kafka-dotnet/build/1.0.84
You should get an InvalidOperationException with proper error
(note that you wil have to target librdkafka.redist 0.11.0-RC2, RC1 had a bug)

closing as this was resolved, thanks @treziac !


Feature Request: Defining CEF index

Can we make it so indexes can be defined in the splunk_metadata file for CEF like syslog?

This is already a feature https://splunk.github.io/splunk-connect-for-syslog/main/sources/base/cef/

How? I looked through that webpage and only see where to change sourcetype. The only mention I see of index is main.


[Spike] Enable team to know how to use migrated data in calculation of total production

Description
As a team
We want to look at migrated data
So that we understand how we can use migrated data in the current calculation of total production
Acceptance criteria

[ ] Team are able to come up with recommendations on how we can use migrated data in the current calculation of total production

@seblapela
@BjarkeMeier @lasrinnil


Update OS causes crash

To Reproduce
Steps to reproduce the behavior:
0. OS is just installed and account with iCloud linkage setup

Go to Settings -> Update OS

Click on Update OS (10.14.1)

OS update fails - core dump appears to be produced - "shutdownStall" file. Log is binary not text.

Update would complete by normal process.
Is this a bug or am I not choosing the right Clover image for the boot?

Is this reproducible? Sometimes shutdown/restart doesn’t work.

Sadly no. it seems to occur randomly. I think it is related to speed of reading of the cdrom image file. (I had it on a network drive which seemed to cause an different type of clover boot).
I still can't reliably reproduce. And the system has now updated. (I think it is associated with OS updates the require special preboot to permit update of OS specific libraries/files).


Issue 3167: Missing values that are NaNs are not supported

Fixed #3167
This PR fixes #3167. However, the fix in this PR is internally replace NaN's with std::infinity. This is a much simpler fix than proposed in #3167, only touching two files, but may not prove to be as robust.
Note, this PR also fixes an unreported bug: The NetCDFCFCollection class did not handle the case where a single NetCDF
file used by the "_FillValue" and "missing_value" attribute name to indicate missing values.

@clyne - Can we make this a normal PR instead of a draft at this point?

@clyne - Can we make this a normal PR instead of a draft at this point?

Indeed. Thanks for catching. Done.


Sysmon Event ID 9: RawAccessRead - Add target file

Hi Sysmon team,
Is there a way to add in the file being targeted by an EID 9 RawAccessRead event? There seems to be quite a bit of legitimate activity happening from processes such as svchost.exe. Attackers regularly inject into svchost.exe which makes this not a very useful detection or log with the limited data available.
If there was a way to determine the file being accessed, it would allow for high fidelity filtering/alerting for files such as ntds.dit on domain controllers. This would also just be generally way more helpful because even if you see a tool such as RawCopy.exe accessing a file, investigators still have no log of what file the process accessed. To avoid an assumption, filtering on the target file would be great as well.
I have no idea how Sysmon does the monitoring but one solution may be to compare which data runs/clusters the raw file read accessed (if that data is available?) and look those up against $MFT data.
Thanks to @jwillyamz for the idea!
Thank you, sysmon team, for considering this request.

Can you please share a log with the Svchost raw access read events?
From: davidpany @.>
Sent: Monday, February 7, 2022 6:46 AM
To: MicrosoftDocs/sysinternals @.>
Cc: Subscribed @.***>
Subject: [MicrosoftDocs/sysinternals] Sysmon Event ID 9: RawAccessRead - Add target file (Issue #495)
Hi Sysmon team,
Is there a way to add in the file being targeted by an EID 9 RawAccessRead event? There seems to be quite a bit of legitimate activity happening from processes such as svchost.exe. Attackers regularly inject into svchost.exe which makes this not a very useful detection or log with the limited data available.
If there was a way to determine the file being accessed, it would allow for high fidelity filtering/alerting for files such as ntds.dit on domain controllers. This would also just be generally way more helpful because even if you see a tool such as RawCopy.exe accessing a file, investigators still have no log of what file the process accessed. To avoid an assumption, filtering on the target file would be great as well.
I have no idea how Sysmon does the monitoring but one solution may be to compare which data runs/clusters the raw file read accessed (if that data is available?) and look those up against $MFT data.
Thanks to @jwillyamzhttps://github.com/jwillyamz for the idea!
Thank you, sysmon team, for considering this request.
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/sysinternals/issues/495, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAVEZVGXT46HINKZNPATKLTUZ7LMLANCNFSM5NXVAC6A.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message ID<EMAIL_ADDRESS>

Hey Mark,
Here is a ZIP of an EVTX for you with a couple minutes of background data and some tests with C:\Users\FEMandiant\Desktop\RawCopy64.exe.
sysmontest.zip
For reference, here is a pivot table of Image Paths and counts recorded:
Image Path                                                                                Count of Image Path C:\Users\FEMandiant\Desktop\RawCopy64.exe	                                                       9 C:\Windows\System32\CompatTelRunner.exe	                                                       18 C:\Windows\System32\svchost.exe	                                                                        474 C:\Windows\System32\usocoreworker.exe	                                                                48 C:\Windows\SystemApps\Microsoft.Windows.Cortana_cw5n1h2txyewy\SearchUI.exe	12 System	                                                                               30 Grand Total	                                                                               591
Let me know what else I can do to support.

What service(s) is running in that process?
From: davidpany @.>
Sent: Monday, February 7, 2022 12:14 PM
To: MicrosoftDocs/sysinternals @.>
Cc: Mark Russinovich @.>; Comment @.>
Subject: Re: [MicrosoftDocs/sysinternals] Sysmon Event ID 9: RawAccessRead - Add target file (Issue #495)
Hey Mark,
Here is a ZIP of an EVTX for you with a couple minutes of background data and some tests with C:\Users\FEMandiant\Desktop\RawCopy64.exe.
sysmontest.ziphttps://github.com/MicrosoftDocs/sysinternals/files/8018393/sysmontest.zip
For reference, here is a pivot table of Image Paths and counts recorded:
Image Path Count of Image Path C:\Users\FEMandiant\Desktop\RawCopy64.exe 9 C:\Windows\System32\CompatTelRunner.exe 18 C:\Windows\System32\svchost.exe 474 C:\Windows\System32\usocoreworker.exe 48 C:\Windows\SystemApps\Microsoft.Windows.Cortana_cw5n1h2txyewy\SearchUI.exe 12 System 30 Grand Total 591
Let me know what else I can do to support.
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/sysinternals/issues/495#issuecomment-1031877183, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAVEZVHQRC6QVZJAZUROVXDU2ARZLANCNFSM5NXVAC6A.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID<EMAIL_ADDRESS>

For the testing I just opened up cmd.exe and ran the command: C:\Users\FEMandiant\Desktop>RawCopy64.exe /FileNamePath:C:$MFT
For reference, RawCopy is from https://github.com/jschicht/RawCopy. In real life investigations though the attacker has injected their malware into svchost.exe and accessed ntds.dit which is what we are trying to look for in the real world.
Let me know if I'm misunderstanding what you are looking for though.

So the svchost raw access events aren’t from a svchost service, but from injected code?
There’s a ton of complexity reversing raw blocks to files, which is out of scope for Sysmon.
From: davidpany @.>
Sent: Monday, February 7, 2022 1:27 PM
To: MicrosoftDocs/sysinternals @.>
Cc: Mark Russinovich @.>; Comment @.>
Subject: Re: [MicrosoftDocs/sysinternals] Sysmon Event ID 9: RawAccessRead - Add target file (Issue #495)
For the testing I just opened up cmd.exe and ran the command: C:\Users\FEMandiant\Desktop>RawCopy64.exe /FileNamePath:C:$MFT
For reference, RawCopy is from https://github.com/jschicht/RawCopy. In real life investigations though the attacker has injected their malware into svchost.exe and accessed ntds.dit which is what we are trying to look for in the real world.
Let me know if I'm misunderstanding what you are looking for though.
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/sysinternals/issues/495#issuecomment-1031942298, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAVEZVB2BMCG7W3PPH2EYQLU2A2JPANCNFSM5NXVAC6A.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID<EMAIL_ADDRESS>

In my test data data uploaded, the process RawCopy64.exe performed the raw access events. In real world scenarios, attackers could perform raw access events with their own processes or after they've injected into another process.
it makes sense that reversing the raw blocks back to the file that was accessed would be challenging. Unless there is some Windows API or something that does a lookup, I imagine you'd have to parse out the entire $MFT to identify the data blocks until you get a file path that matches. I'm no Windows programmer though so it could be way worse than that.
As a starting point, is it possible to resolve the Device # to the letter of the drive and list out the raw data blocks accessed? As a forensic analyst, I could then parse the $MFT to find what clusters the file used.