No transparency and border (maybe usage error?)

Hi,
I'm pretty sure this is not a bug. However, perhaps you've got an idea what might be wrong here: http://www.daemon.de/idisk/gpx/Niendorf,Klausdorf,Gahlendorf,Burg.gpx.html ?
The elevation graph displays and works but it's black and there's no transparent window behind it. I'm not sure what I did wrong.
Thanks in advance,
Tom

Solved, I forgot to include the leaflet-elevation.css :)


Feature: You can share events by clicking "copy" on a link

Each event view has their unique link, which you can send to friends and make them see the same thing

Having "search params"-like url changing event modals


Custom stats

Pulling in @vigneshshanmugam 's additions to this feature branch.

@vigneshshanmugam I've merged this but when this feature lands in master, would you mind signing the CLA by clicking the CLA badge above?

Wohoo awesome...Signed the license already :)


NIFI-10713 Add Deprecation Warning for PGP in EncryptContent

Summary
NIFI-10713 Adds a deprecation warning to the EncryptContent Processor as part of custom validation when the Processor is configured with PGP properties.
OpenPGP support is implemented in the following Processors, providing better performance and additional features:

EncryptContentPGP
DecryptContentPGP
SignContentPGP
VerifyContentPGP

The warning mentions EncryptContentPGP and DecryptContentPGP as those Processors provide direct replacement for the PGP capabilities in EncryptContent.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 8
[ ] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

Would it be useful at this time to remove "OpenPGP", "PGP", "GPG" from the Tags annotation, to discourage finding EncryptContent when a user searches for PGP processors?

Would it be useful at this time to remove "OpenPGP", "PGP", "GPG" from the Tags annotation, to discourage finding EncryptContent when a user searches for PGP processors?

Thanks for the suggestion @mosermw, that's a good idea!


CRS-0: create skeleton Spring boot project

Create skeleton Spring boot project for CRS.
Acceptance criteria:
Spring boot project created.

Spring boot project created.


Create wrappers for "Semantic UI"

https://github.com/nd0ut/semantic-ui-rails
http://http://semantic-ui.com/

Still no plans to support rails generate simple_form:install --semantic_ui now that all the cool kids are using semantic ;) ?

I'd like to chime in and say that this would make my life so much easier since I use semantic-ui instead of Bootstrap or Zurb in all my projects.

Until this is added to as initializer to SimpleForm, you can refer to this wrapper that I have created.
https://developers.supportbee.com/blog/integrating-rails-simple-form-with-semantic-ui/
Till now, it's been working perfectly well for us.


fermipy doc failing during fit.

Describe the bug
This line is failing in the Fermipy documentation fitting:
https://github.com/threeML/threeML/runs/2092912814?check_suite_focus=true#step:9:111
To Reproduce
Run the fermipy_lat.md doc in the docs folder
Expected behavior
the fit should not fail?
Screenshots
See link above to actions.
Desktop (please complete the following information):
github actions.
Additional context
hey... the fermipy seems to be working again!

Hm. I've been poking around a bit. This is what I've found so far:

It's not a problem with the fit/minimizer. ThreeML's FermipyLike.get_log_like() (which is essentially a wrapper for Fermipy's gta.like.logLike.value() ) returns a nan value, even before the first fit step.
The problem goes away if we consider smaller search radii for the LAT catalog (18 degrees or smaller). As a workaround, use ra, dec, table = lat_catalog.search_around_source("Crab", radius=18.0) in the second cell.
I suspect something goes wrong with one of the sources when translating from 4FGL to astromodels and back to fermipy. I'll try to see if I can figure out which one (or if there's a different problem).

Ok. I think I tracked the issue down to one source, PSR_J0622p3749.
Calling model.remove_source("PSR_J0622p3749") before assigning the model makes the fit work.
 So something is wrong with this source, but I'm stumped on what exactly the problem is. @omodei any ideas?

I wonder if having 0 predicted flux in some energy bins is a problem. Maybe we can try setting it to some small number, e.g. 1e-300, instead?

👊
Docs are building now, still getting some conda errors.

Thanks!


Docs: fix typos on refactor/all-the-things

This PR contains a:

[ ] bugfix
[ ] new feature
[ ] code refactor
[ ] test update
[x] typo fix
[ ] metadata update

Motivation / Use-Case

Some typos.  Done using github-spellcheck-cli.
Breaking Changes

N/A
Additional Info
N/A

Thanks for the enthusiasm but we're not accepting third-party contributions to that branch at the moment.


chore(ci): added back openapi trait test

Altough the feature is deprecated it is good we verify with testing it still works as expected

Release Note
chore(ci): added back openapi trait test

@hernanDatgDev fyi, this test would keep validating openapi trait in the deprecated way.


PermissionError when running hello_quantum on local_qiskit_simulator and local_clifford_simulator

Attempting to run hello_quantum.py on local_qiskit_simulator and local_clifford_simulator results in a PermissionError exception.
Expected Behavior
When using local_qasm_simulator, the following gets printed to stdout:
      -- Ignoring SSL errors.  This is not recommended --
      WARNING: There's no connection with IBMQuantumExperience servers.
                   cannot test I/O intesive tasks, will only test CPU intensive tasks
                   running the jobs in the local simulator
      The backends available for use are:
      ['local_clifford_simulator',
       'local_qiskit_simulator',
       'local_unitary_simulator',
       'local_qasm_simulator']

      COMPLETED
      {'counts': {'00': 512, '11': 512}}

Current Behavior
When using local_qiskit_simulator and local_clifford_simulator, the following gets printed to stderr:
      concurrent.futures.process._RemoteTraceback:
      """
      Traceback (most recent call last):
        File "/usr/lib/python3.5/concurrent/futures/process.py", line 175, in _process_worker
          r = call_item.fn(*call_item.args, **call_item.kwargs)
        File "/home/anton/CK_TOOLS/lib-qiskit-gcc-6.3.0-linux-64/build/qiskit/_jobprocessor.py", line 54, in run_backend
          return backend.run(q_job)
        File "/home/anton/CK_TOOLS/lib-qiskit-gcc-6.3.0-linux-64/build/qiskit/backends/_qiskit_cpp_simulator.py", line 87, in run
          result = run(qobj, self._configuration['exe'])
        File "/home/anton/CK_TOOLS/lib-qiskit-gcc-6.3.0-linux-64/build/qiskit/backends/_qiskit_cpp_simulator.py", line 158, in run
          stdin=PIPE, stdout=PIPE, stderr=PIPE) as proc:
        File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
          restore_signals, start_new_session)
        File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
          raise child_exception_type(errno_num, err_msg)
      PermissionError: [Errno 13] Permission denied
      """

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "../hello_quantum.py", line 56, in <module>
          print(result.get_data("bell"))
        File "/home/anton/CK_TOOLS/lib-qiskit-gcc-6.3.0-linux-64/build/qiskit/_result.py", line 204, in get_data
          raise exception
        File "/home/anton/CK_TOOLS/lib-qiskit-gcc-6.3.0-linux-64/build/qiskit/_jobprocessor.py", line 99, in _job_done_callback
          result = future.result()
        File "/usr/lib/python3.5/concurrent/futures/_base.py", line 398, in result
          return self.__get_result()
        File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
          raise self._exception
      PermissionError: [Errno 13] Permission denied

Possible Solution
N/A.
Steps to Reproduce (for bugs)
Please see how to reproduce the problem via Collective Knowledge - QISKit here.
Context
The program is virtually unmodified. The only significant change consists in reading the backend name from the environment:
backend = 'local_qasm_simulator'
if 'CK_IBM_BACKEND' in os.environ:
    backend = os.environ['CK_IBM_BACKEND']

Your Environment

Version used: Master
Environment name and version: Python 3.5.2
Operating System and version: several Ubuntu 16.04 systems (including x86_64 and aarch64).

/cc @fvella

@psyhtest can you please try this and let me know if it works?
If you installed with PIP:
sudo chmod +x qiskit/backends/qiskit_simulator
If you installed from source:
sudo chmod +x out/src/qiskit-simulator/qiskit_simulator

Looking at the linked issues on the description, it seems you are manually compiling the simulator, and patching the _qiskit_cpp_simulator.py to look for the simulator  on a custom path? The custom changes seem to be in the right direction, but looking at the traceback there might be an issue with the executable permissions of the file (chmod), or might also require further patching to look for the binary in the right place by the time run() is executed (it is a bit more convoluted, as it can be overridden by the configuration for a particular qobj).
Without being too familiar with ck, perhaps a good first step would be to check the value of executable when the function is invoked at https://github.com/QISKit/qiskit-sdk-py/blob/c8f04404cacc1096264b9f6fa119e82d72030a59/qiskit/backends/_qiskit_cpp_simulator.py#L133, and check if that file exists in your environment and has the right permissions?
Please note as well that we are in the process of making significant changes to the C++ simulator (#351) for the upcoming release, and in the short future we will move towards including the simulator as a shared library in a more pip-conventional way (compiling the simulator manually should still be an option, but not the one used by the "official" installation procedures). While we try to cater to all configurations and setups, we cannot unfortunately commit to supporting all existing integration possibilities - and this particular one seems a bit out of the usual scope. I'd personally be happy to lend a hand at https://github.com/ctuning/ck-qiskit/issues/2 and continue the discussion there!

No worries at al, @psyhtest - I peeked a bit in the linked issues in your repository by that time, and was glad to learn that it helped solve your issue, and it also seems it made sense to put the work on hold until the 0.5 release was out! :+1:
I'd personally be happy to try to find ways of easing collaboration with your Quantum Collective Knowledge project, which seems an exciting initiative and it feels it has a number of points where we can cooperate.  Nevertheless, we will provide a more official answer after discussing with the team - thanks!

Thanks @diego-plan9! Let's keep in touch.


Prevent the mentor from applying for himself

Purpose

The purpose of this PR is to fix #
Goals

Approach

Screenshots

Checklist

[x] This PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.
[ ] I have read and understood the development best practices guidelines ( http://bit.ly/sef-best-practices )
[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation

Related PRs

Test environment

Learning

@anjula-sack I think a mentor of a certain program shouldn't be allowed to apply as a mentee for the same program at all, just preventing the mentor from applying to himself implies that he can be a mentee of another mentor, which is absurd imo.


If you have a 'foreign' folder in the output folder, no scenarios can be run

FloodAdapt version checks

[X] I have checked that this issue has not already been reported.
[X] I have checked that this bug exists on the latest version of FloodAdapt.

Reproducible Example
I zipped a few output folders to give to a student. So I now have a folder in my output folder called "Hurricane_Ian_Scenarios". when I tried to run a scenario, I got the error message that the scenario couldn't be run. The error message in VS Code was:
[Errno 2] No such file or directory: 'D:\_CODE\FloodAdapt-GUI\Database\charleston_practitionerGroup_Oct2023\input\scenarios\Hurricane_Ian_scenarios.zip\Hurricane_Ian_scenarios.zip.toml'
So it is checking that every output folder has associated input folders.

Current behaviour
If there's a 'foreign' folder in the output folder, scenarios will not run.
Desired behaviour
If there is a foreign folder in the output folder, FloodAdapt will recognize it as foreign (doesn't match any of the scenario names) and ignore it.
Task list

Decide who picks this issue up
Identify the code that is doing this check before running a scenario
Add in an exception for folder names that do not coincide with a scenario name

Additional context
No response

I am not sure if really see this as an issue. Why should we allow for having other folrders in the output?

Well it is allowed haha. I zipped some output folders to share with a student and left the zip file in the folder and then FloodAdapt wouldn’t run.
Sent from my phone

From: Panos Athanasiou @.>
Sent: Wednesday, November 15, 2023 10:54:58 PM
To: Deltares-research/FloodAdapt @.>
Cc: Kathryn Roscoe @.>; Author @.>
Subject: Re: [Deltares-research/FloodAdapt] If you have a 'foreign' folder in the output folder, no scenarios can be run (Issue #290)
Caution: This message was sent from outside of Deltares. Please do not click links or open attachments unless you recognize the source of this email and know the content is safe. Please report all suspicious emails to @.***" as an attachment.
I am not sure if really see this as an issue. Why should we allow for having other folrders in the output?
—
Reply to this email directly, view it on GitHubhttps://github.com/Deltares-research/FloodAdapt/issues/290#issuecomment-1813323069, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AS4ZATVB3EBOTSYKWXTBGYLYEU23FAVCNFSM6AAAAAA7LBNL5CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJTGMZDGMBWHE.
You are receiving this because you authored the thread.Message ID: @.***>
DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

yeah, that's what I am saying. There tons of things that can go wrong if someone starts to delete or add files in the folder structure. Should we do something for all these potential scenarios? I think we should just make clear that a user should not mess up with the file and folder structure.

Solved by adding more documentation


Make IngestSegmentFirehoseFactory splittable for parallel ingestion

To use an ingestSegment firehose with parallel ingestion, specify
taskGranularity in the firehose's spec. There is no default; index_parallel will
warn and fall back to sequential ingestion inside the supervisor if
taskGranularity is not set.

I recognize that this is a PR without a matching issue, so it's more of a "proposal in the form of code" than a ready-to-merge pull request, and I will not feel like my time writing the code has been wasted if this change is unwanted. The PR currently has no tests; if the design of this feature is accepted, I'll write some.

Note: there's a comment in the PR about how Granularity.getIterable can return intervals which extend beyond its input interval.  Here are the uses of that function in non-test code in Druid:

druid-indexing-hadoop, GranularityPathSpec: trims the output of that iterable
druid-processing, IntervalChunkingQueryRunner: does some processing which I believe is equivalent to trimming the output of that iterable
druid-processing, GroupByUniversalTimestamp: probably should be using granularity.bucketStart instead
druid-processing, QueryableIndexStorageAdapter.CursorSequenceBuilder: appears to trim the output of that iterable (inside the apply method)
druid-processing, IncrementalIndexStorageAdapter: most uses of the interval (in initializing timeStart and cursorIterable) trim it to the actualInterval, but the time field doesn't. Is that a bug?
druid-server, UniformGranularitySpec: does not trim the intervals.  Because batch ingestion uses this field to throw away data not in the given explicit intervals, this means that the interval used to throw away data may be less precisely (larger) than the specified one.

This suggests that it would be helpful to have a Granularity.getTrimmedIterable() method which would allow most of the current callers of Granularity.getIterable() to be simplified. There are two calls in my list that wouldn't be able to use getTrimmedIterable, and I don't know Druid well enough to say whether that is due to bugs or is intentional.  If we do conclude that these are both bugs, I don't know the Druid compatibility rules well enough to know if it would be OK to just change the semantics of Granularity.getIterable() rather than add a second variant.

@glasser thanks for working on this! I have one comment.
As in other distributed systems like Hadoop or Spark, it's important to evenly distributed work in parallel indexing because the entire ingestion is completed when the longest task is finished. It means, each sub task is supposed to process almost same amount of data.
In this PR, it looks that input segments are split based on taskGranularity. But, in practice, it's possible that each time chunk has different amounts of data. What do you think about splitting based on the segment size, so that each task process a similar size of segments?

Good question. I was kind of imagining you would set taskGranularity equal to your output segmentGranularity so that each subtask would write one segment. You're right that things can get unbalanced though.
Are you imagining that the split implementation would query the segments metadata to learn all the segment sizes and the user would specify bytes per split?  Would we try to not divide any input segments but just chunk them together?
This seems like a reasonable option to desire but I kind of feel like people might still want to get started with the simpler "I know my peons can handle an hour of data, just split by hours" anyway... so implementing one of these options doesn't necessarily stop from implementing the other later.

Are you imagining that the split implementation would query the segments metadata to learn all the segment sizes and the user would specify bytes per split? Would we try to not divide any input segments but just chunk them together?

Yes, this is exactly what I want. The task can ask the coordinator to get the segment metadata. Tasks use CoordinatorClient when they talk with the coordinator, so you may want to add a new method to it which calls /druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?simple (http://druid.io/docs/latest/operations/api-reference.html#datasources). Its result is like below:
{
  "2019-02-11T23:00:00.000Z/2019-02-12T00:00:00.000Z": {
    "size": 21459255,
    "count": 2
  },
  "2019-02-11T22:00:00.000Z/2019-02-11T23:00:00.000Z": {
    "size": 24510542,
    "count": 2
  }
}

This seems like a reasonable option to desire but I kind of feel like people might still want to get started with the simpler "I know my peons can handle an hour of data, just split by hours" anyway... so implementing one of these options doesn't necessarily stop from implementing the other later.

My feeling is that maxInputSegmentBytesPerTask is simpler than taskGranularity because , with taskGranularity, people should think about how many segments are in each time chunk and each task can handle it. However, for maxInputSegmentBytesPerTask, they can set it to whatever a task can handle. I think we can provide a default, so that people even don't have to think about it in most cases.
If taskGranularity is better than maxInputSegmentBytesPerTask in some cases, I'm fine with adding it. But, I don't think of anything. Do you have something in your mind?

I guess my logic is we run Kafka indexing service making one segment per task so handling one segment per task feels doable and simple :) But of course KIS can be configured with max bytes as well. I like the point that we can have a default max bytes here.
What are you imagining this would do for single segments larger than the max bytes setting?

(er, single intervals, not single segments - I'm bad at terminology here.)

For an interval larger than the max bytes setting, I think each subTask should process a subset of segments, so that the large interval can be processed in parallel.
For a segment larger than the max bytes setting, probably it's fine to process with one subTask.

OK. And I guess it doesn't really matter how segments are assigned between tasks — all the locking and segment allocation works no matter how much overlap there is, right?
Since this will require the subtasks to understand how to have specific segments named rather than just intervals, perhaps the task itself should be extended to allow you to specify a list of segments as an alternative to an interval in the first place?

OK. And I guess it doesn't really matter how segments are assigned between tasks — all the locking and segment allocation works no matter how much overlap there is, right?

Would you tell me more about what kind of overlap you're thinking? I think it's fine if each task processes a disjoint subset of input segments.

Since this will require the subtasks to understand how to have specific segments named rather than just intervals, perhaps the task itself should be extended to allow you to specify a list of segments as an alternative to an interval in the first place?

I think we can extend IngestSegmentFirehoseFactory to accept a list of segments. In IngestSegmentFirehoseFactory, I think the interval should be empty if segments are specified. This segments should also be integrated with the intervals of GranularitySpec. If intervals are given, the task should read only the segments overlapping with the intervals.

Would you tell me more about what kind of overlap you're thinking? I think it's fine if each task processes a disjoint subset of input segments.

I was just stressing about what happens about combining the results of processing two segments from a given time chunk on different subtasks, but I think the answer is that everything just works fine.
I have a few further questions about overlapping intervals.
(a) It looks like the /druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval} API only shows us segments that are fully contained inside the given interval. I think that's not what we want — we also want to include partially covered intervals (filtering the rows we read from them of course). Do you know if this happens to be accessible somewhere in the API? Would it be reasonable to add an includeOverlapping query parameter to that API which uses theInterval::overlaps instead of theInterval::contains for its filter.
Alternatively, it seems like we could just use the same SegmentListUsedAction that the connect method uses. Seems like this might be simpler and more clearly equivalent to the unsplit case.
In either case it looks like the client in question isn't immediately accessible. I think I would need to add an injection for CoordinatorClient directly into IngestSegmentFirehoseFactory, but it looks like several services use IndexingServiceFirehoseModule without binding CoordinatorClient. Would I want to add a binding for CoordinatorClient directly to IndexingServiceFirehoseModule?
Or if using SegmentListUsedAction I would have to make sure to call setToolbox in the supervisor task like it is currently called in the sub task.  That seems simpler than using CoordinatorClient.
(BTW I notice that when IndexTask calls setToolbox on its firehose factory, it knows how to recurse through CombiningFirehoseFactory, but the parallel sub task does not do that.  I think I could add a TaskToolboxConsumingFirehoseFactory interface implemented by IngestSegmentFirehoseFactory and CombiningFirehoseFactory to simplify this and make it more consistent.)
(b) I don't really know what happens when we have two segments with overlapping intervals and distinct versions.  Eg v1 @ 1:00-3:00, v2 @ 2:00-4:00.  Is the data from 1:00-2:00 in the first interval supposed to be visible, or is the entire segment overshadowed?  If it's the former, then specifically, when IngestSegmentFirehoseFactory calls
      final List<TimelineObjectHolder<String, DataSegment>> timeLineSegments = VersionedIntervalTimeline
          .forSegments(usedSegments)
          .lookup(interval);

will the overshadowing still work if we're in a subtask where we're only passing the first segment in usedSegments, or does this calculation have enough context to understand that part of the segment should be ignored even though the segment that overshadows it is not in usedSegments in this particular subtask?

@glasser good questions!
For (a), thank you for reminding me about SegmentListUsedAction. I think it's the best option and you can set taskToolBox and use it. For TaskToolboxConsumingFirehoseFactory, I guess you want to extract IndexTask.setFirehoseFactoryToolbox() to IngestSegmentFirehoseFactory. Please go for it if you think it's simpler.
For (b), the first segment should be partially overshadowed. The part of v1 @ 1:00-2:00 should be available for input. VersionedIntervalTimeline.lookup() is responsible for doing this. If usedSegments are the segments of v1 @ 1:00-3:00 and v2 @ 2:00-4:00, lookup() would return 2 TimelineObjectHolder of v1 @ 1:00-2:00 and v2 @ 2:00-4:00. However, if usedSegments is v1 @ 1:00-3:00, lookup() just returns a TimelineObjectHolder for the same segment.
So, you may want to adjust intervals of the granularitySpec for sub tasks as well to prune out the overshadowed parts. I'm not sure how this problem is currently being handled though.

Oh, I found it. In IngestSegmentFirehoseFactory, it creates WindowedStorageAdapter which filters out data outside of the interval of TimelineObjectHolder. So, you don't have to worry about it.

But doesn't that holder come from the timeline, so it only works if the timeline was constructed from the full set of segments?
I think what this means is that IngestSegmentFirehoseFactory.connect needs to always call SegmentListAction/VersionedIntervalTimeline on the full original interval even in a subtask.  However it will only call fetchSegments on the selected segments, and it should skip elements of timelineSegments that aren't in the selected segments.
This implies that instead of segments being an alternate option for interval, interval always needs to be specified.
Alternatively, the split operation needs to provide both the full list of segments and the split-specific segment list to each split firehose factory.  (Or at least include an extra list of overlapping segments.)
(In other news, my TaskToolboxConsumingFirehoseFactory idea runs into trouble because CombiningFirehoseFactory is in druid-server which doesn't have access to druid-indexing-service's TaskToolbox type. I suppose the interface could declare setTaskToolbox(Object) and let IngestSegmentFirehoseFactory do a typecast?)

Is your concern that different sub tasks can process overlapping segments of different versions, so that both of them produce data of overlapping intervals which can cause the incorrect result? I think it's valid.
To avoid this, we can add another restriction in split generation: each split should contain all overlapping segments. For example, suppose we have 3 segments of v1 @ 1:00-3:00, v2 @ 2:00-4:00, and v2 @ 4:00-6:00. Then, we need 2 sub tasks each of which processes (v1 @ 1:00-3:00 and v2 @ 2:00-4:00) and (v2 @ 4:00-6:00), respectively. Each sub tasks should be able to generate the valid timeline from the given subset of sub tasks.
I think this restriction is acceptable because this kind of overlapping won't be common. What do you think?

That sounds reasonable, with this caveat: you previously said:

For an interval larger than the max bytes setting, I think each subTask should process a subset of segments, so that the large interval can be processed in parallel.

and I think what you meant by that is that if there are a bunch of segments with the same interval and version (different partition num) that it would be OK to split them up across sub tasks.
So I think the algorithm would be something like: list the segments for the whole interval as a timeline.  Select the first segment, and take the set of all segments that overlap it, transitively. If this set of segments has more than one interval, then all of those segments are constrained to go in the same subtask.  Otherwise, each of the segments in this set (all of which are for the same interval) may go in their own subtask.
We've now partitioned the full set of segments into subsets that have to go together.  We can then divide the whole list up into subtasks. This is the https://en.wikipedia.org/wiki/Bin_packing_problem. Rather than try to optimally solve it or even use the first-fit problem, I would just use the greedy algorithm that goes down the list of subsets and greedily assigns them to subtasks, since this is more likely to get segments of the same interval onto the same subtask and thus lead to fewer output segments.

@glasser sounds nice! Are you planning to update your PR?

Yes, now that I think we have a concrete algorithm!
I did just pull out the observation I had above about the special-case setToolbox code in IndexTask not existing in ParallelIndexSubTask into https://github.com/apache/incubator-druid/pull/7063

Thanks! I'll take a look it too.

What do you think a good default value for maxInputSegmentBytesPerTask is? I was thinking of just copying what the Kafka indexing service uses for its default segment size limit, but that's a row count, not bytes.

So I think the algorithm would be something like: list the segments for the whole interval as a timeline. Select the first segment, and take the set of all segments that overlap it, transitively. If this set of segments has more than one interval, then all of those segments are constrained to go in the same subtask. Otherwise, each of the segments in this set (all of which are for the same interval) may go in their own subtask.

OK, I think this was overcomplicating it. An easier way to solve the overlapping segment problem is just to make the task specify directly to the subtask a List<WindowedSegment>, where a WindowedSegment is a class that holds a segment ID (String) and a List<Interval>.  This can be read off directly from the full timeline when calculating splits.
The sub-task won't need to recalculate a timeline at all: it'll just set up some WindowedStorageAdapters to read the appropriate parts of the segments.

I'll work on this once #7089 is merged.
I think I can also change CompactionTask to use the new "tell IngestSegmentFirehoseFactory what segments to use directly" API.

@jihoonson OK, I have a basic implementation here. No new unit tests and haven't gotten all existing tests passing yet, but if you have a chance to take a basic look at the design that would be helpful! Will try to finish up the tests in the next couple days, but am curious if there are any major changes I'd need to make.

@glasser thanks! I'll take a look.

Thanks @jihoonson. It's much simpler using TimelineObjectHolder and considering segmentIds mode to be an internal detail.
It's probably worth another review pass now, though I still need to do unit tests. I'll wait to see what you think of #7171 before making the integration tests work.

Note that I'm holding off on cleaning up the integration tests here until https://github.com/apache/incubator-druid/pull/7171 is merged. I should write unit tests (eg of the splitting algorithm) though.

I've added a unit test that I feel pretty good about, which takes advantage of the existing useful test cases in IngestSegmentFirehoseFactoryTimelineTest.
I have two questions about the impact on CompactionTask from this change that I had added as FIXME comments earlier. The last commit now removes those comments (so that the PR could be in theoretically-mergeable state) but I'd still appreciate feedback on those questions.

Most of the TeamCity errors are in untouched code. Two of them are in new code from me but I'm not sure what changes would be useful (see my comments).

It was probably because of the changed Intellij version which is reverted back now. I restarted TemCity.

Good, TeamCity passed this time.


[Website] Update macOS to 0.71

Description
Why
Update website to list the latest version of React Native macOS.
Microsoft Reviewers: Open in CodeFlow

Not sure if I need to add a new versioned doc for rnm-getting-started.md or not. Working on it...

Not sure if I need to add a new versioned doc for rnm-getting-started.md or not. Working on it...

If you edit rnm-getting-started.md but not the versioned copies that will change the copy under next (e.g. https://microsoft.github.io/react-native-windows/docs/next/rnm-getting-started) but not any previous versions (e.g. https://microsoft.github.io/react-native-windows/docs/0.67/rnm-getting-started).
Since this comment refers to "what is latest", it seems like all copies of rnm-getting-started should be updated?

[ ] \react-native-windows-samples\website\versioned_docs\version-0.61
[ ] \react-native-windows-samples\website\versioned_docs\version-0.62
[ ] \react-native-windows-samples\website\versioned_docs\version-0.63
[ ] \react-native-windows-samples\website\versioned_docs\version-0.64
[ ] \react-native-windows-samples\website\versioned_docs\version-0.66
[ ] \react-native-windows-samples\website\versioned_docs\version-0.67

Not sure if I need to add a new versioned doc for rnm-getting-started.md or not. Working on it...

If you edit rnm-getting-started.md but not the versioned copies that will change the copy under next (e.g. https://microsoft.github.io/react-native-windows/docs/next/rnm-getting-started) but not any previous versions (e.g. https://microsoft.github.io/react-native-windows/docs/0.67/rnm-getting-started).
Since this comment refers to "what is latest", it seems like all copies of rnm-getting-started should be updated?

[ ]  \react-native-windows-samples\website\versioned_docs\version-0.61
[ ]  \react-native-windows-samples\website\versioned_docs\version-0.62
[ ]  \react-native-windows-samples\website\versioned_docs\version-0.63
[ ]  \react-native-windows-samples\website\versioned_docs\version-0.64
[ ]  \react-native-windows-samples\website\versioned_docs\version-0.66
[ ]  \react-native-windows-samples\website\versioned_docs\version-0.67

Thanks for the info. I'm pretty sure nothing about getting started has changed per release (It's all just react-native-macos-init + run-macos).. so I'm tempted to just remove all the versioned versions?

Thanks for the info. I'm pretty sure nothing about getting started has changed per release (It's all just react-native-macos-init + run-macos).. so I'm tempted to just remove all the versioned versions?

Yep, you can delete the old ones and remove the need for versioned copies.

I removed references of "What the latest version is" from old docs, as well as removing the 0.67 version docs given we never released a 0.67.
Rather than adding a new 0.71 page, I just updated the "next" version. My understanding is since a 0.71 version doesn't exist, then the "next" version will be used in the 0.71 page?

Looks like I don't have write access. Could someone else merge this for me please? :)


Catppuccin Mocha Theme bug

Catppuccin Mocha Theme: white and black colors are mixed, you have to switch them. Didn't check other color variants.
This is how it should be:
"palette": [
"#585b70",
"#f38ba8",
"#a6e3a1",
"#f9e2af",
"#89b4fa",
"#f5c2e7",
"#94e2d5",
"#a6adc8",
"#45475a",
"#f38ba8",
"#a6e3a1",
"#f9e2af",
"#89b4fa",
"#f5c2e7",
"#94e2d5",
"#bac2de"
],

This appears to happen in other dark variants as well. I've replaced the palette like yours and it looks as it should now.
Mentioning your issue over at the tilix repo for reference.


[build][Suggestion] Enable C++11 support;

I was hoping to see if others want to enable using C++11 standard during compilation. Its trivial to enable, however I don't want to start using C++11 features to update the code base until others are okay with it.

I'm game, if nothing else just to get familiar with it. What are the
pros/cons?
On Mon, Feb 16, 2015 at 12:16 PM, Gregory King<EMAIL_ADDRESS>wrote:

I was hoping to see if others want to enable using C++11 standard during
compilation. Its trivial to enable, however I don't want to start using
C++11 features to update the code base until others are okay with it.
—
Reply to this email directly or view it on GitHub
https://github.com/ValveSoftware/vogl/issues/198.

There are couple of features that I find useful:
(1) auto, decltype - compiler determined type declaration, common usage for auto:
std::vector<int> my_vector;
auto pos = my_vector.begin();//auto pos will be of type std::vector<int>::iterator

int f = 6;
decltype(f) has_the_same_type = 7;

(2) nullptr - equivalent to 0 or NULL but not convertible to integral types (except bool)
(3) std::hash and std::unordered_map - standard containers (note: hash algorithm is implementation dependent)
(4) lambdas - anonymous functions
(5) std::thread (and concurrence features) - standardized interface for thread, mutex types (linux gcc defaults to posix threads and mutex), atomics, futures and promises
(6) static_assert, constexpr - compile time checks
(7) delete, default, override, final within class definitions - allows for control of compiler construction of class members and inheritance.  delete, final can be used to guarantee that an object cannot be used for inheritance.
(8) improved random facilities
(9) better smart pointers (auto_ptr is deprecated): std::shared_ptr provides reference counted smart pointer
Disadvantages:
(1) Older compilers may only have some of the features implemented. I believe GCC 4.8 and clang 3.4 have full support, unsure about VS.clang support, gcc support, VS support
(2) auto can make reading the code more difficult
(3) some features can be implementation dependent
There a few other features (but I haven't used them much) and so I will decline to comment (for example, R-Value references and std::move ...).

How much porting would be involved or will the current C++ just work? I.e.
is C++ a superset of C++? I was thinking the conversion would be listed
with disadvantages but maybe not...
On Mon, Feb 16, 2015 at 2:16 PM, Gregory King<EMAIL_ADDRESS>wrote:

There are couple of features that I find useful:
(1) auto, decltype - compiler determined type declaration, common usage
for auto:
std::vector my_vector;
auto pos = my_vector.begin();//auto pos will be of type std::vector::iterator
int f = 6;
decltype(f) has_the_same_type = 7;
(2) nullptr - equivalent to 0 or NULL but not convertible to integral
types (except bool)
(3) std::hash and std::unordered_map - standard containers (note: hash
algorithm is implementation dependent)
(4) lambdas - anonymous functions
(5) std::thread (and concurrence features) - standardized interface for
thread, mutex types (linux gcc defaults to posix threads and mutex),
atomics, futures and promises
(6) static_assert, constexpr - compile time checks
(7) delete, default, override, final within class definitions - allows for
control of compiler construction of class members and inheritance. delete,
final can be used to guarantee that an object cannot be used for
inheritance.
(8) improved random facilities
(9) better smart pointers (auto_ptr is deprecated): std::shared_ptr
provides reference counted smart pointer
Disadvantages:
(1) Older compilers may only have some of the features implemented. I
believe GCC 4.8 and clang 3.4 have full support, unsure about VS.clang
support http://clang.llvm.org/cxx_status.html, gcc support
https://gcc.gnu.org/projects/cxx0x.html, VS support
https://msdn.microsoft.com/en-us/library/hh567368.aspx
(2) auto can make reading the code more difficult
(3) some features can be implementation dependent
There a few other features (but I haven't used them much) and so I will
decline to comment (for example, R-Value references and std::move ...).
—
Reply to this email directly or view it on GitHub
https://github.com/ValveSoftware/vogl/issues/198#issuecomment-74580834.

is C++ a superset of C++?
should have been
is C++11 a superset of C++?

On Mon, Feb 16, 2015 at 4:00 PM, Lawrence Love<EMAIL_ADDRESS>wrote:

How much porting would be involved or will the current C++ just work? I.e.
is C++ a superset of C++? I was thinking the conversion would be listed
with disadvantages but maybe not...
On Mon, Feb 16, 2015 at 2:16 PM, Gregory King<EMAIL_ADDRESS>wrote:

There are couple of features that I find useful:
(1) auto, decltype - compiler determined type declaration, common usage
for auto:
std::vector my_vector;
auto pos = my_vector.begin();//auto pos will be of type std::vector::iterator
int f = 6;
decltype(f) has_the_same_type = 7;
(2) nullptr - equivalent to 0 or NULL but not convertible to integral
types (except bool)
(3) std::hash and std::unordered_map - standard containers (note: hash
algorithm is implementation dependent)
(4) lambdas - anonymous functions
(5) std::thread (and concurrence features) - standardized interface for
thread, mutex types (linux gcc defaults to posix threads and mutex),
atomics, futures and promises
(6) static_assert, constexpr - compile time checks
(7) delete, default, override, final within class definitions - allows
for control of compiler construction of class members and inheritance.
delete, final can be used to guarantee that an object cannot be used for
inheritance.
(8) improved random facilities
(9) better smart pointers (auto_ptr is deprecated): std::shared_ptr
provides reference counted smart pointer
Disadvantages:
(1) Older compilers may only have some of the features implemented. I
believe GCC 4.8 and clang 3.4 have full support, unsure about VS.clang
support http://clang.llvm.org/cxx_status.html, gcc support
https://gcc.gnu.org/projects/cxx0x.html, VS support
https://msdn.microsoft.com/en-us/library/hh567368.aspx
(2) auto can make reading the code more difficult
(3) some features can be implementation dependent
There a few other features (but I haven't used them much) and so I will
decline to comment (for example, R-Value references and std::move ...).
—
Reply to this email directly or view it on GitHub
https://github.com/ValveSoftware/vogl/issues/198#issuecomment-74580834.

C++11 is a nearly a superset of C++98/03 (it has a few language extension/new keywords, so using certain features of C++11 can be guaranteed to not compile if forced into C++98/C++03 compliant compiler, I haven't come across much C++98/C++03 (primarily collisions with new extensions/other features) code that doesn't properly compile when using C++11 - further there are a few deprecated features, including auto_ptr<> and throw specifications). This covers most of the features (with small code samples): C++11 FAQ;
To enable using C++11: you can add -std=c++11 (CXXFLAGS). You'll notice a few warnings when compiling with clang (for example- related to the code already and referencing C++98):
(1) warning: variadic macros are incompatible with C++98,
(2) warning: commas at the end of enumerator lists are incompatible with       C++98 [-Wc++98-compat-pedantic,Parse Issue]
(3) warning: extra ';' outside of a function is incompatible with       C++98 [-Wc++98-compat-pedantic,Parse Issue]
(4) warning: 'long long' is incompatible with C++98 [-Wc++98-compat-pedantic] (this is still valid, provided the compiler supports it, but its non-standard :+1: )
C++ stackoverflow question;
In addition clang/llvm provides tools to migrate to using C++11 idioms.clang-moderize. Documentation of -std compliance mode for clang.

Thanks, that helps.  I had looked at that Stackoverflow link but your
specific answer is good to know. And thanks for the other links, the FAQ
and the fact that clang offers a migration tool.
On Mon, Feb 16, 2015 at 5:19 PM, Gregory King<EMAIL_ADDRESS>wrote:

C++11 is a nearly a superset of C++98/03 (it has a few language
extension/new keywords, so using certain features of C++11 can be
guaranteed to not compile if forced into C++98/C++03 compliant compiler, I
haven't come across much C++98/C++03 (primarily collisions with new
extensions/other features) code that doesn't properly compile when using
C++11 - further there are a few deprecated features, including auto_ptr<>
and throw specifications). This covers most of the features (with small
code samples): C++11 FAQ http://www.stroustrup.com/C++11FAQ.html;
To enable using C++11: you can add -std=c++11 (CXXFLAGS). You'll notice a
few warnings when compiling with clang (for example- related to the code
already and referencing C++98):
(1) warning: variadic macros are incompatible with C++98,
(2) warning: commas at the end of enumerator lists are incompatible with
C++98 [-Wc++98-compat-pedantic,Parse Issue]
(3) warning: extra ';' outside of a function is incompatible with
C++98 [-Wc++98-compat-pedantic,Parse Issue]
(4) warning: 'long long' is incompatible with C++98
[-Wc++98-compat-pedantic] (this is still valid, provided the compiler
supports it, but its non-standard [image: :+1:] )
C++ stackoverflow question
http://stackoverflow.com/questions/6293075/is-c11-c0x-a-complete-super-set-of-c03
;
In addition clang/llvm provides tools to migrate to using C++11 idioms.
clang-moderize http://clang.llvm.org/extra/clang-modernize.html.
Documentation of -std compliance mode for clang
http://clang.llvm.org/docs/UsersManual.html#differences-between-various-standard-modes
.
—
Reply to this email directly or view it on GitHub
https://github.com/ValveSoftware/vogl/issues/198#issuecomment-74602948.

@kingtaurus, there are even more goodies from C++11. If we would move to C++11 as the default, we can loose the pthread dependency. This means that the entire vogl_threading_pthreads.h, vogl_threading_win32.h can be removed in favor of a generic one.


Debug symbol upload for Android fails with Gradle (works with CLI, and on desktop (Windows) build)

From Discord
OS: Windows
Build: Android build with Gradle
Sentry can`t automatically upload debug symbols for Android, but it automatically uploads symbols for Windows builds. The Android building process returns an error: Invalid org token (HTTP status 401). However, I can manually upload debug symbols for Android using the console sentry CLI. I used the same token for automatic Windows debug symbol upload and for manually uploading debug symbols for Android using the CLI.
I have added screenshots with errors, corrected upload debug symbols for Windows, and manually uploaded symbols for Android.

Task failed uploadSentryProguardWMappingsRelease

The error message is "Invalid org token so I suspect gradle is picking up a token from somewhere like env var (SENTRY_AUTH_TOKEN) or sentry.properties`.
The error message comes from here: https://github.com/getsentry/sentry-cli/blob/6d0729dafaf0ce928fcb591f4c701a051c98ddf4/src/commands/login.rs#L76-L91
So does this mean Sentry rejected the token or the format of the token is invalid? (Or could be either?)

I have "sentry.properties" in my project folder, but they were generated by Sentry Unreal pugin, when I set up token, project name, organization name in plugin engine settings. When I changed vales its regenerated in folder. The values the same like in plugin engine settings.
After my builds on Android fails,  i used senty CLI for uploading android debug symbols. And they saved my valid token somewhere in PC, when i first log in. But, after that, I tryied upload adnroid debug symbols automaticaly. and got error again. I do not see SENTRY_AUTH_TOKEN in enviroment vairables on my PC.

@Rostislav98 Which UE plugin version do you have? How do you use Sentry - SaaS / self-hosted?
It looks similar to #502.
The Sentry Android Gradle plugin which's used in Unreal for symbol upload also had this issue (https://github.com/getsentry/sentry-android-gradle-plugin/issues/694) so we've upgraded it recently to get a corresponding fix (#633) and make things work out of the box. It should be available starting from 0.20.0.

Updating plugin resolved the symbol upload issue.


Properly check expressions in class extends clause

Fixes #4431.

Can we add the test from the other pull request?
errorInClassExpressionInHeritageClause01.ts
class Foo extends class { reset() { return this.y; } } {
    constructor() {
        super();
    }
}

Copied test from #4496.

:+1: don't forget to get this into release-1.6


fix warning of ReactDOM.render

Warning: ReactDOM.render is no longer supported in React 18. Use createRoot instead. Until you switch to the new API, your app will behave as if it's running React 17.

this is error on the storybook, and doesn't resolve it


refactor(ast): add a container macro to AST nodes

The most ergonomic implementation would be a terse attribute macro #[arena].
We could also add to that macro a bunch of the boilerplate attrs which are crowding oxc_ast. It could also allow #[serde] attrs to be used without #[cfg_attr].
I know that macros can obscure meaning, and explicit code with no magic is a virtue, but personally I find the level of attributes in oxc_ast is now so high that it makes it hard to see the wood for the trees. I think this:
#[arena]
#[serde(tag = "type")]
pub struct Program<'a> {
  #[serde(flatten)]
  pub span: Span,
  // ...
}

is much easier to grok than:
#[derive(Debug, Hash)]
#[cfg_attr(feature = "serde", derive(Serialize), serde(tag = "type", rename_all = "camelCase"))]
#[cfg_attr(all(feature = "serde", feature = "wasm"), derive(tsify::Tsify))]
pub struct Program<'a> {
  #[cfg_attr(feature = "serde", serde(flatten))]
  pub span: Span,
  // ...
}

If the macro's implementation is simple enough, and we added a comment to the top of each file saying // See 'arena' macro in 'oxc_macros' crate for attributes which are added to AST types, then that'd make it easier for people new to the codebase to understand it swiftly.
I do have one further motivation: A macro of this sort would also be very useful for AST transfer, which needs to mutate enums extensively (see current impl here).
Of course the big argument against an #[arena] macro is if it'd have a noticeable impact on compile times. If it's going to hurt that measure considerably, then probably a bad idea. Do you think that'd be the case?

Shrinking derive macros
Readability wise it's 💩, but everything is googleable if it's all laid out. New contributors will have a hard time navigating things if everything is hidden away.
Sorry I'm in the "less / no macros" camp 😅

I do agree with you in general about macros, but in the case of oxc_ast, I just think it's getting out of hand with the sheer volume of #[cfg_attr]s everywhere. It's becoming seriously hard to see what the hell is going on - and I say that as someone who's been heavily immersed in OXC's codebase for 2-3 months now.
These #[cfg_attr]s are almost the same for all the AST types. Worse, there are some differences - but those differences are wrapped in so much boilerplate, that it's hard to see them.
In my opinion, the reason macros make codebases hard to follow is not that macros are a bad thing in themselves, but lazy development practices:

Lack of documentation - #[macro] with no comment in the file where it's used saying why it's there, what it does, or where to find the code for the macro.
Macros too "clever" - convoluted macro code which makes it completely impossible to understand what a macro expands to, and no comments to explain it.

OXC's development practices are not lazy, and I think we can avoid these pitfalls.
So, yes, there are serious problems with how macros are often used, and in general it's best to avoid them. But I do think a pragmatic approach which recognises that sometimes they're appropriate is helpful. And in my view this is one such case.
I'm arguing fairly hard on this, but that's just my personality! Please do feel free to disagree.

How about this for a compromise?
/// Dummy derive macro which doesn't actually derive anything,
/// but allows `#[serde]` attrs on the item, without deriving `Serialize`
/// and without `#[cfg_attr]`.
#[proc_macro_derive(SerdeSupport, attributes(serde))]
pub fn serde_support(_input: TokenStream) -> TokenStream {
    // Return an empty token stream
    TokenStream::new()
}

Then this:
#[derive(Debug, Hash)]
#[cfg_attr(feature = "serde", derive(Serialize), serde(tag = "type", rename_all = "camelCase"))]
#[cfg_attr(feature = "wasm", derive(tsify::Tsify))]
pub struct Program<'a> {
    #[cfg_attr(feature = "serde", serde(flatten))]
    pub span: Span,
    // ...
}

becomes this:
#[derive(Debug, Hash, SerdeSupport)]
#[cfg_attr(feature = "serde", derive(Serialize))]
#[cfg_attr(feature = "wasm", derive(tsify::Tsify))]
#[serde(tag = "type", rename_all = "camelCase")]
pub struct Program<'a> {
    #[serde(flatten)]
    pub span: Span,
    // ...
}

Nothing is hidden away in a macro, but the #[cfg_attr] boilerplate is separated from the #[serde] tags which contain the meaningful information.

👍 go for it


Update Composer dependencies (2021-12-17-00-05)

Visual regression test passed!
View the visual regression test report


Define git_service config getter for Github actions

This is my attempt at fixing the issue https://github.com/google/mdbook-i18n-helpers/issues/173.
I am not very familiar with this codebase but I think it might fix the issue.
My Github workflows implements an unusual technique for codecov uploads which I described in this thread.
It used to work in codecov-action v3 and stopped working in v4. The error I am getting is Commit creating failed: ["Service not found: none"].
I suspect this is due to missing implementation of _get_git_service in github_actions.py.

I just discovered https://github.com/codecov/feedback/issues/263
Based on https://github.com/codecov/feedback/issues/263#issuecomment-1929753219, my PR might fix the issue.

@drazisil-codecov could this PR fix the issue that you mentioned in this comment?

@drazisil-codecov could this PR fix the issue that you mentioned in this comment?

It might, but as you observed , that doesn't help github_enterprise. However, "enterprise" in this context refers to self-hosted, or a URL that isn't github.com. I think it would had it be aware of if a URL was also passed, that should be good.
Also tests, please , if possible.
I'll refer to @thomasrockhu-codecov since I don't really touch the action code that much.

Thanks for response, @drazisil-codecov.
I will wait for confirmation from @thomasrockhu-codecov on the approach. If this approach is correct I can add tests as well. Did not want to invest too much work into this since I wasn't sure if this is the right fix.


Add flashing of loaders and cluster with rcar_flash

Flashing of cluster_wrapper by rcar_flash takes about five minutes, but it requires modified flash_writer. So, this commit adds ulcb_32_no_limits_for_flash.mot which has removed the check for 64MB limit, and is renamed to avoid possible confusion with original
AArch32_Flash_writer_SCIF_DUMMY_CERT_E6300400_ULCB.mot.
Example of usage:
sudo ./rcar_flash.py --conf rcar_flash_bin.yaml -c -f -b h3ulcb_4x2 cluster

For reference, here is the diff of changes made to the flash_writer v1.0.12:
diff --git a/dgmodul4.c b/dgmodul4.c
index cbe4d21..240a1c4 100644
--- a/dgmodul4.c
+++ b/dgmodul4.c
@@ -718,13 +718,6 @@ void XLoadSpiflash0_2(uint32_t mode)
 			return;
 		}
 	}
-	else if( (gSelectQspiFlash==QSPI_BOARD) || (gSelectQspiFlash==HYPER_FLASH) ){
-		if(S25FL512_END_ADDRESS < PrgSpiStatAdd){
-			PutStr("Address Input Error",1);
-			PutStr("Range of H'000_0000 ~ H'3FF_FFFF",1);
-			return;
-		}
-	}

 // WorkMemory CLEAR (Write H'FF)
 	PutStr("Work RAM(H'50000000-H'53FFFFFF) Clear....",1);		//
@@ -776,19 +769,6 @@ void XLoadSpiflash0_2(uint32_t mode)
 			return;
 		}
 	}
-	else if( (gSelectQspiFlash==QSPI_BOARD) || (gSelectQspiFlash==HYPER_FLASH) ){
-		if(PrgSpiEndAdd>S25FL512_END_ADDRESS){
-			PutStr("Program over size Error",1);
-			PutStr(" SpiFlashMemory Stat Address : H'",0);
-			Data2HexAscii(PrgSpiStatAdd,str,4);
-			PutStr(str,1);
-			PutStr(" SpiFlashMemory End Address  : H'",0);
-			Data2HexAscii(PrgSpiEndAdd,str,4);
-			PutStr(str,1);
-			return;
-		}
-	}
-

 	MaskSectorSize= 0xFFFC0000;

I have successfully tested the flashing of the cluster_wrapper.bin with the following command:
sudo ./rcar_flash.py --conf rcar_flash_bin.yaml flash -c -f -b h3ulcb_4x2 -s /dev/ttyUSB0 -p <path_to_loaders> cluster

Whoops, sorry, missed it earlier.
Acked-by: Volodymyr Babchuk<EMAIL_ADDRESS>

Ack is added. Tested. Rebased on the latest head of the camera-sharing-main-v2 branch.
Ready to be merged.


gbm train process with valid res is not the same with valid predict from train model

Please search your question on previous issues, stackoverflow or other search engines before you open a new one.
For bugs and unexpected issues, please provide following information, so that we could reproduce on our system.
Environment info
Operating System:
CPU:
C++/Python/R version:
Error Message:
gbm train process with valid res is not the same with valid predict from train model
Reproducible examples
import lightgbm as lgb
import re
import pickle
import operator
from sklearn.metrics import roc_auc_score
import numpy as np
import gc

data = np.random.rand(500, 10)  # 500 entities, each contains 10 features
label = np.random.randint(2, size=500)  # binary target

valid_data = np.random.rand(100, 10)  # 500 entities, each contains 10 features
valid_label = np.random.randint(2, size=100)

model_path = 'model'

def train():
    train_dataset = lgb.Dataset(data, label=label)
    test_dataset = lgb.Dataset(valid_data, label=valid_label)

    param = {'objective':'binary','task': 'train','boosting_type':'gbdt', 'num_leaves':31,'metric': {'auc','binary_logloss'},
     'learning_rate': 0.05,
    'feature_fraction': 0.8,
    'bagging_fraction': 0.8,
    'bagging_freq': 5,
    'verbose': 0,
    'lambda_l1':2,
    'bagging_seed':1993
    }
    # if os.path.exists(model_path):
        # bst = lgb.train(param, init_model = model_path, train_set  = train_dataset, num_boost_round=500,valid_sets=[valid_dataset],early_stopping_rounds = 100)
    # else:
    bst = lgb.train(param, train_set  = train_dataset, num_boost_round=2,valid_sets=[test_dataset],early_stopping_rounds = 10)
    bst.save_model(model_path)

def valid():
    bst = lgb.Booster(model_file=model_path)
    pres = bst.predict(valid_data)
    print(roc_auc_score(valid_label,pres))

if __name__=="__main__":
    train()
    valid()

Steps to reproduce
1.run the script above ,and get auc of valid dataset
2.comment train()
3.two valid auc res are different

@summersunshine1 Have they a large gap ?
I think the slight difference is not a problem.
There may is some floating point precision issues between cpp and python in AUC calculation.

when valid() function call after train() successively,the valid function result is fixed and is the same with eval result in train(),but when i just call valid() ,the valid results is seriously different at every time.

@summersunshine1
when you comment the train and re-run this script, and valid data (generated randomly) is changed.
As a result, it will have the different result, right ?
To check it, you should use the same valid data for the test.

sorry, it's my fault, i close the issue


Performance issues - Cloudflare is throttling weserv.nl

Hi guys, don't know how to check that but is there any ongoing issue?
It seems terribly long to load an image when using the proxy today.
@andrieslouw @kleisauke

We are seeing increased response times, even leading to timeouts sometimes, looking into this.

I confirm that the performance for servers located in the EU is also very low. It could also be a problem on the cloudflare side - https://www.cloudflarestatus.com/

Yes, thanks, we can confirm this, CloudFlare is really slow at this time. We just tried with a new server, and putting CloudFlare in front of a simple image, slows down image loading significantly. It almost looks like it's throttling to a few kb/s. Ping times are perfect, just the throughput is really slow at this moment. This causes numerous slowdowns, as a lot of origins are also behind CloudFlare.

For future reference, CloudFlare finally confirms issues:

@andrieslouw I guess there is not much we can do but wait till the routing is resolved. Thanks for investigating.

We have asked Cloudflare for comments on this issue, and will keep you updated. At this moment we are still seeing slowdowns in throughput.

We're 99% sure that Cloudflare is throttling the domain images.weserv.nl and even other subdomains of weserv.nl. The domain wsrv.nl seems to be unaffected. Will keep you updated.

Just confirming that we see the same problem (from denmark / finland)

Can we safely replace images.weserv.nl per wsrv.nl?

Can we safely replace images.weserv.nl by wsrv.nl?
Yes, it's the same infrastructure. It was build to be the future address of the service, and is in use for a while now. But we'll try our best to get the original images.weserv.nl up to speed again, and solve this issue. wsrv.nl will stay, and is unaffected from what we can see.

Based on what I read on Cloudflare Discord, the throttle is put on websites that are on the free plan and are using "excessive" bandwidth. Would it be of any help to upgrade from the free plan or are Cloudflare willing to help out?

We have upgraded, but it is to no avail yet. Response from Cloudflare is taking time unfortunately.

Is it just me, or is it loading much faster now?

Our monitoring shows requests are still taking 1.05 seconds on average (with 95% in 0.52 - 2.90). Normally this is around 0.52 seconds (with 95% in 0.15 - 0.75).
I'm afraid it is still slow, we're evaluating on how to proceed. The slowness is deliberately caused by Cloudflare, and so far they're only pointing at section 2.8 of their TOS. Without assistance from Cloudflare, I don't see any alternatives at this time.
Apologies for the inconvenience, it is hard to see this happening to the project, I would love to have better news or answers for you all.

For reference, this is section 2.8 of Cloudflare's terms of services:
https://www.cloudflare.com/terms/#28-limitation-on-serving-non-html-content

The ToS is a bit weird since Cloudflare has introduced R2 and Cloudflare Images. Does it go against their ToS if the images are stored on R2 or delivered through Cloudflare Images?
Would it make any sense to look into other solutions such as AWS CloudFront and maybe start selling API keys for users who want to use weserv to pay for the egress bill?

From what I understand, the R2 and Images are exempt by the supplemental terms: https://www.cloudflare.com/supplemental-terms/
So anything non-HTML would require the R2 or Images/Stream according to the latest TOS. Which is probably commercially motivated.
It's not the intention of this service to accept payments, or go commercial. We have never done so. That's not the spirit of an internet service. We don't even sell ads, and the only logo's and names we mention on our project are from the ones that help us. Names like nginx, libvips, GitHub, Cloudflare, Hetzner, OVH, Scaleway, and many others. If applicable, we privately fund small costs where needed (servers, licenses, open source development support to others), to help in creating a better internet.
Be reminded that the free service processes 6 million requests/images per hour on average, with ~400TB ingress (our servers) and ~250TB egress (Cloudflare) per month. I think it would be cheaper to just rent out servers in different locations global, and build our own CDN based on DNS geolocation.
Commercial services are also not 1-on-1 compatible in offering, they would require a good rewrite of almost everything. Apart from that, let's look at the money: Cloudflare Images you're looking at $2/million requests (assuming 90% cache ratio), Cloudflare R2 is around ~$4.5/million, AWS Cloudfront is ~$4.3/million. So you're looking somewhere around $9k up to $20k/month.
Any cloud-related offering is always crazy expensive, when compared with 1 Gb/s unmetered/1 CPU/32GB RAM for ~$50/month, which will almost handle the same amount of traffic, including the all the image operations (but just from 1 location).
I think a cheap 10 Gb/s unmetered server that just does what Cloudflare does (nginx-proxying) is around $200/month all-in now, and this will handle 3000 TB/month in and out (again, single location).
With scale, around $30/month should be the cost of the mentioned 6 million reqs/hr / 400TB/month CDN offering, so the paid cloud services designed for this operate at a 300x - 700x markup, ballpark figure.
You can always decide to use the source code, and do the roll-out you mention on any cloudservice, we'd welcome this, and appreciate all feedback. It's how we learn, how we improve the code.
With cloud offerings you sure pay for more than just some hardware, fibre and electricity. However, those are the only 3 things our project actually needs to work. The project itself is just a piece of love, which is free, as it should.

Hey guys, since last week we can see performance problems with resizer :

we are using https://images.weserv.nl/?url=
Out app could be sending multiple request (but nothing excessive like more than 1k in 10s ), but we didn't have problems so far.
Could it be connected to throttling? Should we try running our own instance of resizer or is it temporary?

@andrieslouw I am just curious if you had a more positive response from Cloudflare on this issue?

@tompodsiadlo: Please try using https://wsrv.nl/?url=, it should perform better.
@raramuridesign: No, still not yet.

@andrieslouw I have successfully switched to the other domain mentioned.
Questions

will this not have the same issue as the previous domain?
can cloudflare not be switched off on images.weserv.nl?
or will this cause too much stress on the current server?

M.

@raramuridesign:

We don't know, it is hard to get answers from Cloudflare.
No, unfortunately, this will also impact performance enormous now. This is however something we're evaluating, as said in the comment about building out our own CDN.
It will cause long response times for everyone outside Europe, and the bandwidth on peaks is not yet there.

@andrieslouw I am on the discord channel from Cloudflare, if you chat with them there - you might get to the right person quicker. Just a thought.

@raramuridesign: Ah, you probably saw the messages from @kleisauke in that Discord already, on 2 nov 2022 around 17:37 UTC. There wasn't a lot of response, we even shared the ticket numbers.

I was just curious to know if there has been any update. If you were to build your own CDN, would you physically host your own server, or utilize AWS or any other cloud provider? Would love to hear some rough ideas on your plans going forward :)

I was just curious to know if there has been any update. If you were to build your own CDN, would you physically host your own server, or utilize AWS or any other cloud provider? Would love to hear some rough ideas on your plans going forward :)

My guess is any cloud provider (especially AWS) would become too expensive.

@kleisauke This is great news... well done for persisting!

Does this also apply to wsrv.nl?

wsrv.nl is and was unaffected by all of this. But everything is working as it should again.

Hello,
It seems that this issue may be occurring again. Is anyone else experiencing slow loading?

I just filled a support ticket at Cloudflare. It seems to be occurring again (with the same symptoms, wsrv.nl is not affected).

Cloudflare's engineering team also removed this secondary throttle from the weserv.nl zone.
I'll close, please open a new issue if this occurs again.


typeof nil should be undefined

In JS typeof undefined is "undefined", but in Lua it is "nil". It also probably should be extracted to a helper.

It also probably should be extracted to a helper.

Agree - the current transformed version is pretty gnarly.


[docs] add jsdocs for public apis/interfaces

I did see this plan in Github Project a few days ago.
When I started to integrate orcild into my project, I found it important to have a well explained docs.
Could you please add some examples on how to add jsdocs, I can help to add others

Github Project makes sense when many people are working on it but in my case, I just use a local todo text file and didn't update the plan on GitHub, and the plan on GitHub was running out-of-sync and out-of-date. So I removed it.
JSDocs (or TSDocs) I guess will reflect the documentation, so almost the same info that is on the website will be in code comments.
Example how kysely does it: they write text, have links in it, and add code examples.
I'm not writing these comment docs because it would be tiresome, and I have many other more interesting plans for improving the ORM, but surely such docs are useful and helpful for users, and sooner or later they should be added.

I agree that comment docs can be tiresome, especially when syncing it with the code and documentation, I seldom write it in my own project, too.
But it do save time when I want to explore the api usage. Looking through the documentation is OK, but not as quick as cmd + click.
Anyway, if you need any help, I can write these kind of docs.

Of course, I'll be very grateful for your help!
If you really like the ORM and going to use it in the future, you could learn it deeper by reading the website and writing corresponding docs in the code. And later you will be more familiar with the code and could make a feature or a bug fix. I'll be happy to have a collaborator.
Make sure your editor can do grammar checks.

Yes, I quite like this ORM, I have used lots of ORMs , and orchid is the one I'm waiting for years in nodejs ecosystem.
I'm integrating it into my project and refactoring my old queries, seems to work nicely.
There is no need to feel awkward because it's a natural for me to learn and dive deep into the library by contributing docs or fixing bugs.
By the way, I'm glad to see this project to be more mature and recognized by more people


APIv2 Purchase Credit Memos missing

There is no implementation for the Purchase Credit Memos in APIv2 (other purchase documents are implemented)

There is no available purchase credit memo API as of now. We have it in our backlog but it is not planned for this release. If this is urgent for you, you can always create your own custom API for purchase credit memo.
Please check out Developing a Custom API
for the general custom API walkthrough and APIV2SalesCreditMemos for inspiration.

It's not only the API, you need to

create the aggregator table
the aggregator codeunit
duplicate complete dimensionlines api, because it relates on an non extensible enum
duplicate the attached document files api (like document lines)

I agree, that's why it is in our backlog and it is not something we can provide in short term.


Add params object to pullChanges() and delta() (support for path_prefix on /delta calls)

The /delta API call allows you to pass a few different options that have not yet been implemented. This commit moves the params object from a variable to a function parameter. At the moment only the path_prefix parameter is supported.
This change should be backwards-compatible for existing callers of the pullChanges method

Merge this! :rage:


metrics

Please consider adding some middleware logging/metrics to determine perform and/or any issues with routing to services

Do you have any specific preferences for the underlying mechanism?
Do you think that some kind of Application Insights integration is a good idea or should we use a different more generic mechanism?
Maybe should we use some basic logging instead?

Application Insights would be a nice addition; however I feel that it should be a pluggable logging/metric component; maybe something similar to the kestrel event tracing using EventSource given that the middleware maybe deployed on premise and could possibly log private/confidential information

If you have some ideas for logging implementation, I think it can make a valuable contribution to this project. It would be really great, if you could create a PR.

This issue was moved to aspnet/Home#2486


Add a maintainer

Thanks for this plugin! It looks like there are a few issues and PRs that could use attention though. Is it possible to add a maintainer in order to get the PRs sorted out? Or perhaps to think about merging this + PRs into one of the other fs modules? I'd be glad to volunteer.
Ref: #9 #15 #20 and timsuchanek/react-native-fileupload#1

we may be better off merging this android code into one of these:
https://github.com/aroth/react-native-uploader
https://github.com/wootwoot1234/react-native-transfer
https://github.com/booxood/react-native-file-upload
This is the only one with Android support that I've found. And the other ones seem more actively maintained. This one seems to have missing features on iOS, such as the params option.
From my standpoint, I just need Android support + progress reporting for android. @aroth's package seems like the most stable and maintained, but perhaps is missing a few newer things other forks have. It also has the best presentation in the readme, I think the most stars, etc. I could be wrong, but that seems to be the one all these related packages should coalesce around. @wootwoot1234's repo also seems to be gaining steam though.

I'd be interested in helping make this happen as part of my react-native-uploader lib.
https://github.com/aroth/react-native-uploader

there's a lot of dangling forks and pull requests, but it seems you could clean house by bringing it all together under one roof.

I'm happy to combine my project too.  I thought React Native started supporting uploading, is that only with iOS?

FWIW, I just started a new job that moves away from RN a bit. I'm forever appreciative of and indebted to y'all for putting this stuff together, but might struggle to pull off high-level actual-problem-solving thought on this at the moment. Glad to help if it's tedious straightforward stuff thought 😄


Automated pipx nightly injection

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
I'm looking to use cron to inject the nightly YT-DLP version into my tubeup install. This command works well:
pipx inject tubeup "yt-dlp[default] @ https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/download/2<IP_ADDRESS>825/yt-dlp.tar.gz"
Is there any URL that will automatically get the nightly tar.gz file? Since the filename is dated, I can't seem to find a way to have a static link to the most recent nightly built.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
No response

https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest/download/yt-dlp.tar.gz

https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest/download/yt-dlp.tar.gz

You're awesome thanks!


Have terminal.TTY commands prefer standard paths

Fix for #143, #212, #259.
Tested on MacOSX and Ubuntu.  No Windows box to determine
standard path for stty and tput.

Red build looks like a flake, so it should be fine.
Could you explain to me, as someone who knows nothing about tput (and very little about bash) what this is doing? I trust you that it works but I'd love to understand a bit more about the code I'm merging ^_^

Merged as https://github.com/lihaoyi/Ammonite/commit/6a0e7eefe20f1a3f6cf1342d71d79e3dfcf59bdc

Sorry Haoyi,  wasn't paying attention to this.
On both Mac and Linux the tput and stty commands that come with the system are found on system paths /usr/bin/tput and /bin/stty.  (I don't have any knowledge on tput and stty beyond what the man pages says about them so can't comment further there.)  With bash any command that is not provided by rooted or relative path is searched for on the directories of the user's PATH environment variable with first-found being used.  On Macs using homebrew for gnu's coreutils the package will suggest placing a path containing tput and stty before the system defaults.  This patch attempts to first locate tput and stty along system paths and if they cannot be found falls back to letting the system find them using the user's PATH (i.e., pre-patch behavior).
As an aside I'm not sure why the coreutil's stty doesn't handle -g and reading the results back in.  The flag and behavior seem to be there.


[BUG] xlsx-renderer: When iterates on empty collection it leaves garbages

Using template with nested loops, the expected file is generated correctly however few renderer variables (in this case !# END_LOOP item and !# FINISH) are not removed from the generated file.

template.xlsx
vm.json.txt
result.xlsx

(@Siemienik EDIT): Reproduce:
sxr --model vm.json template.xlsx > result.xlsx

@Siemienik I have notice another issue, if the array in the view model data is empty, the variables of the loop section are not removed and they are just placed in the generated file.
I'm attaching the source and result files:
Result-vm-empty-data-array.xlsx
vm-empty-data-array.json.txt
template.xlsx

Yesterday I added a utility which will helps me debug this issue (#134 #151). For now nothing blocks me and I start to fixing this bug ;)

@Siemienik It seems like the issue is caused by empty arrays on which we try perform the loop.

Any update on this issue?

Thanks for refresh this topic, will investigate soon


run commands on comments

Added ability to parse comments when it's triggered by a issue_comment event.
Updated documentation showing how to execute the action's code.
Added composite action to trigger the commands when passed to a matrix and push changes to the branch.
Resolves #1 and #2

And do you have a failed run log when you dont configure this safe directory at all?On 2 May 2024, at 19:33, Javier Bullrich @.***> wrote:
@Bullrich commented on this pull request.
In run/action.yml:

   # Here it would get the script from previous step

   run: |

     echo "$COMMAND" >> command.sh

     chmod +x command.sh

     ./command.sh

     rm command.sh

   env:

     COMMAND: ${{ inputs.command }}

 - name: Report failure

   if: ${{ failure() }}

   shell: bash

   run: echo "Command failed!"

 - name: Check changes

   shell: bash

   run: |

     git config --global --add safe.directory /__w/${{ github.event.repository.name }}/${{ github.event.repository.name }}

Here is tge test run: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/8927469091/job/24520953847
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because your review was requested.Message ID: @.***>
This communication and any attachments are confidential.


Release v0.4.1

/cc @link2xt

I just tested, it compiles on Haiku, with 3 tests still failing.

Released.


Bugfix: The balance of the feature was miscalculated and was considering every subscriber

The issue is that when a feature ticket is assigned to the subscriber, and if there are more than 1 subscriber for the same ticket, the package loads the balance as summed up for all the feature tickets.
The whereHas has the scope for who is the subscriber, but the with misses the scope, and because of that, when the feature tickets for a particular feature is loaded, it loaded summed up charges for the tickets assigned to anyone.
Example:
I assign 5 charges as a feature ticket when a user confirms their account:
$user->giveTicketFor('document', null, 5);

So this happens for each user who registered and confirmed their account.
Now when we see how much balance for each user has using:
$user->balance('document')

If the system has only 1 user, it will return 5.
If there are more than one users, say n users, it will return 5 * n.

Hey, @khatriafaz! Thanks for reporting the bug and bringing a solution! I added a test case to cover this scenario and I'm gonna publish a release right now! 🚀

@lucasdotvin thanks for considering my solution.


Generate both WebApi and HttpClient

Hi @Jalalx , I think we can generate both WebApi and HttpClient by source generators, like a RPC.

Can you elaborate? Or a code sample for what you mean?

Firstly, add a service class or interface, like what in your readme but should be implemented:
    public partial class ToDoService
    {
        [HttpGet("todos/{id}")]
        public partial async Task<ToDoItem> GetToDoItemByIdAsync(int id)
        {
                 return await _db.GetById(id);
        }
    }

Then, generator the WebAPI and HttpClient:
WebAPI :
public class ToDoController: ApiController
{
    private ToDoService _toDoService;

    pubilc ToDoController(ToDoService toDoService)
    {
         _toDoService=toDoService;
    }

     public Task<ToDoItem> GetToDoItemByIdAsync(int id)
     {
          return _toDoService.GetToDoItemByIdAsync(id);
     }
}

The HttpClient just like current design.

Yes, that is possible but developers prefer to implement web api part manually so that they can have full control over the details.
Supporting web api generation is out of current projects scope.

I just think it makes more sense that the user only needs to care about the business and not manage the configuration of the api.

Thank you again for providing feedback. Feel free to fork the project and implement the desired feature.


Add NodePathTester tests

I've also got an interest in reducing how much this package does in projects without ESLint installed, or in windows that aren't even JavaScript projects. Ideally I'd like to kill the worker once it's clear it's not needed, but for now all I'm interested in doing is not sending a job to the worker when we know what the outcome will be.

OK, I implemented the worker-killing after all. I needed JobManager to have a better handle on the worker life-cycle anyway. This lets us defer worker creation until the first linting job, and to go into “sleep mode” eagerly, since we can easily restart the worker if we think we'll need it.


CORS on geocoding due to 301 redirection on Photon

The Photon API moved and no more working due to 301 redirection without CORS.
Report of https://github.com/osm-fr/infrastructure/issues/137#issuecomment-832183119

That's been fixed here: 27a30beda33adc119e22f11741bf71a5b1e1f92e
I just published MapContrib v1.18, @frodrigo you can update the main instance whenever you want


NIFI-10981 Ensure NarAutoLoader starts after provider retrieves NARs,…

… and ensure the auto-loader attempts to process any NARs available before starting to watch for new files

Summary
NIFI-00000
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[ ] Apache NiFi Jira issue created

Pull Request Tracking

[ ] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[ ] Pull Request based on current revision of the main branch
[ ] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 8
[ ] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

Thanks @bbende. The change makes sense to me. Ensure that we get everything downloaded AND loaded before startup, not just wait until we get it downloaded. +1
System test is helpful too, to ensure that things continue to work as expected in the future. Thanks! Will merge to main.


Fix mail message about Free Plan conditions

The mailing template contained improper information about credits being replenished monthly.

@weban or @shairyar please deploy this change ASAP


feat: adds dedicated Local Testing section to Tools

Note: This will remain a DRAFT PR until we are done ensuring era-test-node works as intended and the documentation introduced in this PR is updated with any changes done.
What 💻

Adds dedicated section for "Local Testing" in Tools.
Offers an overview of various local testing alternatives available for developers, outlining when and how to utilize each, as well as their distinct functionalities.
Splits out Dockerized local testing into own page
Splits out In-memory node local testing into own page

Why ✋

With the addition of era-test-node which provides developers with forking capabilities and additional features, providing clear and concise guidance on the use and implementation of these testing options has become essential. This update aims to aid developers with instructions and examples for effective usage.

Evidence 📷

Run: yarn dev:docs and visit http://localhost:8080/docs/

Let's also add the network details in each page (docker and in-memory). Something like:
### Network details
L1 RPC: http://localhost:8545
L2 RPC: http://localhost:3050
Network Id: 270

@dutterbutter missing "Prerequisites" in the in-memory node page?

@uF4No @idea404 @MexicanAce opening this up for an additional review. I have added the requested changes, and added essential warnings indicating its alpha stage.


No change comment

"Change in policy not found." No change comment to be added. Example IDs: OXCGRT_449010, OXCGRT_449004

@todowede - this phrase just needs to be added to config/no_update_phrase/OXCGRT.csv

Thank @OIQuinn - this is added in #149


Render as an image issue

Describe the bug
When exporting a circuit as a png or jpg or any image format, the labels are not included in the rendered image even if "full-circuit view" is selected
To Reproduce
Steps to reproduce the behavior:

Export any circuit as image which has any element whose label is very long and on the extreme left or right side.
Select "full-circuit view" instead of "current view" and export.

Expected behavior
The label must also be included in the image when rendered in "full-circuit view"
Screenshots

Device Information [optional]:

OS: Windows 10
Browser: Chrome
Version: 89

Are you working on this issue? (Yes/No)
No

@humble-barnacle001 can I work on this issue .

I'm not working on it so you can ofc

okay @humble-barnacle001

@ZadeAbhishek  Are you working on it?

@ZadeAbhishek @tachyons can i work on this


default fee recipient is no longer experimental

removed the X in the flag name

Thank you @msagansk! LGTM


feat: sitemap oppty enhacement

Please ensure your pull request adheres to the following guidelines:

[x] make sure to link the related issues in this description
[x] when merging / squashing, make sure the fixed issue references are visible in the commits, for easy compilation of release notes

Wiki: https://wiki.corp.adobe.com/display/AEMSites/SEO+|+Sitemap+auto-detect
Ticket: https://jira.corp.adobe.com/browse/SITES-24615
Example for https://www.spacecatest.com
 {
    "auditResult": {
        "details": {
            "issues": { // this contains URLs that don't have 200 statusCode per sitemap
                "https://www./{baseURL}.com/sitemap-en.xml": [
                    {
                        "url": "https://www./{baseURL}.com/test",
                        "statusCode": 301
                    }
                ]
            }
        },
        "reasons": [
            {
                "value": "Sitemaps found and checked." // sitemap audit run successfully
            }
        ],
        "scores": {},
        "success": true, // status of the sitemap audit
        "paths": { // this contains URLs that have 200 statusCode per sitemap
            "https://www./{baseURL}.com/sitemap-ja.xml": [
                "https://www./{baseURL.com/jp/test",
                ... (more urls with 200 status OK)
                "https://www./{baseURL.com/jp/endtest"
            ],
            "https://www./{baseURL}.com/sitemap-en.xml": [
                "https://www./{baseURL.com/test",
                ... (more urls with 200 status OK)
                "https://www./{baseURL.com/endtest"
            ]
        },
        "url": "https://www./{baseURL}.com"
    }

:tada: This PR is included in version 1.33.0 :tada:
The release is available on:

v1.33.0
GitHub release

Your semantic-release bot :package::rocket:


Keyboard type should not allow character and special symbols when type is Numberpad?

I am creating mobile number field and setting keyboard type as NumberPad.  Then following scenario occurs:
When running with iPad keyboard we are able to enter special characters. (iPad only)
We can paste the text which is copied earlier(iPad and iPhone).
How to find out each and every character typed? callbacks not calling
I am able to do this without your library by reading this tutorial https://programmingwithswift.com/numbers-only-textfield-with-swiftui/
see image here:

I don"t have clear idea where to implement this in your class files. It would much better with those features.
Hope you got my points, Thanks for your great work.

Please refer to new version from v4.0.1
I have added a new property for disabling text field editing action.
.addDisableEditingAction([.paste]) /// Disable text field editing action. Like cut, copy, past, all etc.
Thank you very much and sorry for my late response.


Move runAsyncMain into devcmd package

Thanks so much for your contribution! I'll look through the changes and add some comments :)

Thanks so much for your contribution! I'll look through the changes and add some comments :)


Update dependencies, drop support for Ruby 2.5 add Ruby 3.0

Drop support for Ruby 2.5
update dependencies
Add support for Ruby 3.0

Coverage decreased (-0.6%) to 96.917% when pulling 580adc0dc93801a5697fce57893d61fd66f9e962 on olbrich:update-dependencies-2021.05.02 into 17b85439e3718a4beca6a29544b34cfa0b310177 on devise-security:master.


fix typo error

fix typo error in MetricsContainer:
WORFLOW_TYPE -> WORKFLOW_TYPE

Pull Request Test Coverage Report for Build 4274

1 of 2 (50.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 69.54%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

client/src/main/java/com/netflix/conductor/client/telemetry/MetricsContainer.java
1
2
50.0%

Totals

Change from base Build 4269:
0.0%

Covered Lines:
12472

Relevant Lines:
17935

💛  - Coveralls


DatabaseMetadata.getColumns() is not returning all columns of table

Driver version
10.2.1.jre8
SQL Server version
Microsoft SQL Azure (RTM) - 12.0.2000.8
Client Operating System
Linux x64.
JAVA/JVM version
1.8.0_332-b09
Table schema
Create a table called 'SOH' with following columns:
SalesOrderID        bigint
RevisionNumber      tinyint
OrderDate           datetime
DueDate             datetime
ShipDate            datetime
Status              tinyint
SalesOrderNumber    nvarchar    25
CustomerID          int
ShipToAddressID     int

run sp_columns_100 'SOH', 'dbo';
DatabaseMetaData.getColumns() runs this to get the metadata
Then ALTER TABLE add three columns:
    OnlineOrderFlag     bit
    SalesOrderNumber    nvarchar    25
    PurchaseOrderNumber nvarchar    25

run sp_columns_100 'SOH', 'dbo';
the three new columns are not shown
Problem description
DatabaseMetadata.getColumns() is not returning all columns of table after table has been altered
Expected behavior
DatabaseMetadata.getColumns() must return the new columns of table after table has been altered.
Actual behavior
Incomplete list of columns are returned (it does not return new columns that were added to the table)
SalesOrderID        bigint
RevisionNumber      tinyint
OrderDate           datetime
DueDate             datetime
ShipDate            datetime
Status              tinyint
SalesOrderNumber    nvarchar    25
CustomerID          int
ShipToAddressID     int

Error message/stack trace
No error occurs, it just returns incomplete data.
Any other details that can be helpful
Refer: https://github.com/microsoft/mssql-jdbc/issues/1859.
JDBC trace logs
NA

HI @www-viveksharma, I'm still unable to repro the issue when following your steps. These are the steps I performed in SSMS with the same SQL db version.
1. Create the table
create table SOH (
SalesOrderID        bigint,
RevisionNumber      tinyint,
OrderDate           datetime,
DueDate             datetime,
ShipDate            datetime,
Status              tinyint,
SalesOrderNumber    nvarchar(25),
CustomerID          int,
ShipToAddressID     int
)

2. Queried the table, here I see all 9 columns
sp_columns_100 'SOH', 'dbo';

3. Altered the table with 3 new columns
alter table SOH add OnlineOrderFlag bit, AccountNumber nvarchar(25), PurchaseOrderNumber nvarchar(25);

4. Queried the table again, however I do see the 3 new columns in addition to the previous 9
sp_columns_100 'SOH', 'dbo';

Is there anything missing from what I did? This is all done on a fresh environment.

The difference I see in the new columns is that when I run:
SELECT DOMAIN_CATALOG
  FROM INFORMATION_SCHEMA.COLUMNS c
 WHERE TABLE_SCHEMA = 'dbo'
   AND TABLE_NAME = 'SOH'
 ORDER BY ORDINAL_POSITION ;

The DOMAIN_CATALOG shows as NULL for existing columns, but for the new 3 columns added, DOMAIN_CATALOG = Adventureworks...

can this issue be related to some system indexes not being refreshed after the alter table was done ?

Let me know if you want me to run some other query and you want to look at that result

@tkyc any updates on this ?

Like I mentioned before, since it isn't a direct driver issue (because we're working off of SSMS/Db clients rather than through the driver), your best bet is to contact the Azure SQL team.

GRANT REFERENCES ON TYPE::dbo.aliastype TO user
solved the problem for me


Catalog class with loading from HiPSCat files

Basic catalog class with basic DataFrame representation methods wrapping Dask DataFrame.

Closed in #12