report correction

rough_draft.pdf

I am reviewing the document , and I found out 1 major issue so I am writing is ASAP. We are missing the "robustness"  experiment which is even there in the title.
For robustness the experiment to do is  adding noise to the labels  like the one which was done in my paper .
So basically the experiment for adding different noise to the labels , retraining the models with the 4 loss function or atleast (the Gaussian and cauchy)  .  Do it only for 1 dataset whichever trains faster.
I know this doesnt fit in your plans but we were discussion this before the break but missed the point after the break. But think about it, its in the title and we started for checking the robustness.
I would say you try these experiments till 15th and submit which is possible. I will send the other review ASAP .

There is one another option which might take less time than previous approach . Which is checking the robustness of the inference and not in training .
In this you can add noise to the image by using the different augmentation techniques (https://pytorch.org/vision/stable/transforms.html) so here you dont need to train the model  just change the transform for the test dataloader and check the outputs.

The output we expect all the predictions to deteriorate, so nothing surprising , but at-least we can learn which loss function is deteriorating more.

So by doing this you can atleast check the mark that you have looked something into robustness of the model.
@nandhini30

Annotations
(12/22/2022, 8:49:31 PM)

[ ]  “.” (Mathivanan, p. 1 page=13&annotation=CMXDP9RV)) check spaces

[ ]  “.M” (Mathivanan, p. 1 page=13&annotation=TSY8JNTP))

[ ]  “Motivation In this research, th” (Mathivanan, p. 1 page=13&annotation=QRP47A9N)) This below what you have written needs to be last section of the Chapter 1.
Basically this is not the motivation .
Motivation :
Why does one need uncertainty estimation ? Why does anyone need to check the robustness of the uncertainty estimation ?
Very little "why keypoint regression ? "
Try to answer this 3 questions : And give multiple answers to the questions.
Dont dump this section move to the last part of the chapter 1

[ ]  “robustness capacity” (Mathivanan, p. 1 page=13&annotation=EG78NQEF)) There is no experiment for the robustness capacity .. As mentioned in the issue.
The robustness experiment is missing .
So there is no case of improvement.
Add this point after you do 1 of the robustness experiment.

[ ]  “is improved by” (Mathivanan, p. 1 page=13&annotation=NCEZGGTK))

[ ]  (Mathivanan, p. 1 Add
\ A new loss function Cauchy loss function [cite by blog] was compared on the keypoint estimation

[ ]  “robustness capacity” (Mathivanan, p. 1 page=13&annotation=H88GHUSA)) The uncertainty estimation capability of different loss function is evaluated using complex regression task Keypoint detection.

[ ]  “Problem Statement” (Mathivanan, p. 2 page=14&annotation=GGFWNMGQ)) why is this in points.
make it a paragraph

[ ]  (Mathivanan, p. 2 This is problem statement section :
So the end paragraph should start with should be :
In this RnD we look into the problem of ..........................
NOTE : It can also be a mathematical formulation of the problem. Just copy paste the equations from my papers problem statement (maybe change the alphabets if possible) . Make sure in the begining of the paragraph you mention the below paragraphs are taken from the paper.
Dont copy the sttaements only the equations and cite.

[ ]  “The challenge of estimating uncertainty increases when there are noisy labels or outliers in the training data. The ability of learning algorithms to properly learn uncertainty by ignoring the outliers is known as robust uncertainty estimation.” (Mathivanan, p. 2 page=14&annotation=93ZS5M6H)) So you now the definition.
But the experiments are missing .
So either do the experiments or remove this line .

[ ]  (Mathivanan, p. 3 Move that contributions section here :

[ ]  (Mathivanan, p. 8 3 pages and little less. Please add more here .
Anyways write a conclusion atleast on which datasets were selected and which uncertainty estimation method was selected and WHY. WHY is important

[ ]  “et.” (Mathivanan, p. 10 page=22&annotation=7ZUZXY7K)) citation

[ ]  “s.” (Mathivanan, p. 10 page=22&annotation=AB2Q63DF)) citation

[ ]  “es.” (Mathivanan, p. 11 page=23&annotation=SU373NDS)) citation

[ ]  “l.” (Mathivanan, p. 11 page=23&annotation=9ESTC2ZS)) citation

[ ]  “d” (Mathivanan, p. 12 page=24&annotation=M6HS7NN4)) citation

[ ]  (Mathivanan, p. 12 copy paste the derivation from the paper of the loss function
Put the code side by side from pytorch .. same comment for all below

[ ]  “Laplace negative log likelihood” (Mathivanan, p. 12 page=24&annotation=SQGCXBNN)) citation

[ ]  (Mathivanan, p. 12 copy paste the derivation from the paper of the loss function

[ ]  “Cauchy negative log likelihood” (Mathivanan, p. 12 page=24&annotation=UI53QL3G)) citation

[ ]  (Mathivanan, p. 12 copy paste the derivation from the blog of the loss function

[ ]  “Evidential loss” (Mathivanan, p. 13 page=25&annotation=5U29K4T6)) citation

[ ]  “β(1+” (Mathivanan, p. 13 page=25&annotation=3NWWKTTA))

[ ]  (Mathivanan, p. 14 Please paraphrase my blog on how to get the upper and lower her .
Just on top of the section write the below section comes from the blog here .

[ ]  (Mathivanan, p. 16 Dont do this :
1 page 3 images .
Just pust the 3 images side by side .
Professors get really angry when we see that just to increase page numbers images are being posted

[ ]  “Metrices” (Mathivanan, p. 19 page=31&annotation=W67QX9KU)) metric.
Which metric ..
Why is it not on the graph .. What is value .. Just put the metric name

[ ]  (Mathivanan, p. 19 Plot it straight please ..
it easily fits horiontally

[ ]  (Mathivanan, p. 20 which value ??
make the plot smaller half a sheet is enough ,, what is there in this graph that a whole page has to be taken

[ ]  “metrices” (Mathivanan, p. 20 page=32&annotation=9EJEBU9T)) metrics .
Which metric ..
Why is it not on the graph .. What is value .. Just put the metric name

[ ]  “Metrices” (Mathivanan, p. 21 page=33&annotation=QVTTU98L)) metrics .
Which metric ..
Why is it not on the graph .. What is value .. Just put the metric name

[ ]  (Mathivanan, p. 21 Make it straight
What is value

[ ]  “metrices” (Mathivanan, p. 22  metrics .
Which metric ..
Why is it not on the graph .. What is value .. Just put the metric name

[ ]  (Mathivanan, p. 23 Make the metric which is lowest bold .
Add caption and explain what the table . Wat is better in the mteric lower better or higher better and conclude also ..
Cauchy works better etc
Make 2 tables :
1 for Biwi and 1 for Face .
There is nothing we can compare in between BIWI and FACE so they shoudl be separte .

[ ]  (Mathivanan, p. 24 How is this most confident ..
All the blue mismatches the yellow ..
When you dont se the yellow its a match .. Here all mismatch
Mayeb you are plotting wrong ..

[ ]  (Mathivanan, p. 24 For a single dataset
All most confident images and all the least confient images from all 4 loss function fit in a SINGLE page .
in the following order
(Page 1)
# Most confident
3 x gaussian x face
3 x laplace x face
3 x evidential x face
3 x cauchy x face
# Least confident
3 x gaussian x face
3 x laplace x face
3 x evidential x face
3 x cauchy x face
Big caption explaining each row
(Page 2)
# Most confident
3 x gaussian x BIWI
3 x laplace x BIWI
3 x evidential x BIWI
3 x cauchy x BIWI
3 x gaussian x BIWI
3 x laplace x BIWI
3 x evidential x BIWI
3 x cauchy x BIWI
similar 2 pages for most error and least error
order the images vertically for 1 loss function
gaussian laplace evidential cauchy
gaussian laplace evidential cauchy
gaussian laplace evidential cauchy

[ ]  (Mathivanan, p. 24 How is this most confident ..
This is the most error image the face it selcted the wrong face totally

[ ]  (Mathivanan, p. 25 How is this least error ..
This is the maximum error

[ ]  (Mathivanan, p. 25 Again here (a ) looks all match ..
but b and c are many mismatch .

[ ]  (Mathivanan, p. 33 A report should be read like a story .
This section doesnt fit anywhere .
My suggestion drop RQ2 and just move this section in state-of-art .
And every section conclude something .
So what was the goal of this section and write a conclusion paragraph in the end on what do you conclude from the state of the art.

Uploading rough_draft_reviewed.pdf…

Major comments :

Missing robustness experiment, as explained in the first 2 issues solution for it .
Each section , chapter please write a single para conclusion of the section
Reduce all the image size , try to fit more images in single page to do the comparison. I have writen how you can do in the report. You can make the collage in any other image tool like inkscape or any image tool u know and just import 1 single image in latex.
The paragraph describing the graph or the images try to keep it near to the image, probably in the same page.
The plots . put the metric name in the plot and in the caption.
Cite Cite and Cite .. Just keep on citing on every statement, in caption everywhere.

Finally, do reconsider on the decision to submit early. The report doesnt reflect the work you have done . Some robustness experiment is required to justify the title . basically the current contribution you have written is completely not true there is no robustness test. On the contrary there is no explanation of what is robustness and which experiment explains robustness.  If you are able to address all the concerns mentioned above and the robustness experiment please submit otherwise also submit its a descent report .
Dont get demotivated by the comments, you have done good amount of work. The report holds major percentage of the grade, so its a big deal.

rough_draft_reviewed.pdf

Robustness_Results.pdf
I have uploaded the results of data augmentation techniques (gaussian blur, random rotation and random invert). The entropy plots were plotted and I have interval score values too. Is this enough for robustness study?


References with Prefetching

After hundreds of man--hours finding the most optimal way to add support for prefetching, I've come up with this.
Let me know if you need more info. This is a massive PR, and I'm happy to explain any part of it.
References
This function with return a new manager which will return each item in the database with its references
The references are returned as a prefiltered manager, which will only return the items which are related to the item
For example:
for (final (group,refs) in await groups.withReferences().get()) {
  final usersInGroup = await refs.users.get();
  /// Is identical to:
  final usersInGroup = await users.filter((f) => f.group.id(group.id)).get();
}

Prefetching
The keen among you may notice that the above code is extremly inefficient, as it will run a query for each group to get the users in that group.
This could mean hundreds of queries for a single page of data, grinding your application to a halt.
The solution to this is to use prefetching, which will run a single query to get all the data you need.
For example:
for (final (group,refs) in await groups.withReferences((prefetch) => prefetch(users: true)).get()) {
   final usersInGroup = refs.users.prefetchedData;
}

Note that prefetchedData will be null if the reference was not prefetched.
Symmetrical Type Converters
This PR fixes a bug where referenced filters which didn't share the exact same type would not act correctly.
The Manager API now ignores references whose type converters arrent the same. The Tests were updated to support this.
Internals
The is much going on, if we go through this step by step, it should come clear
References
When withReferences, we create a new manager which has it's ActiveDataclass generic updated to the DataclassWithReferences generic. The ActiveDataclass is the class which is returned by any of the get/watch functions.
When any of these get/watch method are called, we apply the _withReferenceMapper if the  ActiveDataclass is a DataclassWithReferences.
What is returned from a withReferences call is a record, with the item, along with an object which has getters for prefiltered managers.
for (final (group,refs) in await groups.withReferences().get()) {
    group; // Original Item
    refs; // Prefiltered Managers
    final usersInGroupManager = refs.users  // Prefiltered Manager
    final usersInGroupManager = refs.users  // Prefiltered Manager
}

Prefetching
When withReferences is called with a prefetcher, we create a class called PrefetchHooks and add it to the manager. This class will be different depening on what was prefetched.
groups.withReferences((prefetch) => prefetch(users: false, admin: true))

Would create a PrefetchHooks which was configured to only prefetch admin.
This PrefetchHooks class has 2 jobs:

Add joins to the query before it is executed
Execute a 2nd query to get reverse references and store it's result in each TypedResult object.

When a referenced manager is created that has prefetch data in it's TypedResult object, it will be loaded into the prefetchedData field of the manager.
$$TodosTableTableProcessedTableManager get todos {
    final manager = $$TodosTableTableTableManager($_db, $_db.todosTable)
        .filter((f) => f.category.id($_item.id));

    final cache = $_typedResult.readTableOrNull(_todosTable($_db)); // Loads the prefetched data from the TypedResult
    return ProcessedTableManager(
        manager.$state.copyWith(prefetchedData: cache));
  }

Hacks
Loading in multiple items into a single TypedResult object wasn't currently supported.
 This item is only meant to act as a key for the TypedResult object, and is never actually used to read the data. Although, nothing bad would happen if it was.
More work:

[ ] Docs
[ ] Tests

@simolus3 Ready for review!

Let me know if I should break up this PR into parts to make it digestible.

Good point. I'll add samples of generated code to the doccoments.

Good point on a centralized comment explaining everything

Take a look at the doccomment

I updating some deocs regarding streams. I don't really like how it works.
Any ideas?

Maybe we could make a version of the watch function which allows us to pass in additional tables to watch. (to trigger new event when a reverse references table has an updates [which is not prefetched using a join, but another query])
Won't be very efficient, because it will rerun all the queries, even though it only needs to re-run the 2nd one

Either way my docs aren't clear. I'm need a fresh set of eyes on these new streaming docs

@simolus3
Update:
Adding support for streams is not as complex as I thought.
I won't have time to work on it today, but should be done over the next couple of days.
Doesn't add any significant complexity.
Hold off on merging

Yeah I'm in favor of option 3 of course :) We're recommending something similar for a current approach shared on the documentation website.

Also need to work on doing everything in a single transaction to for race conditions.
It may seem like I am remembering new things last minute, but I'm just going through my TODOs, which have all of this laid out.

@simolus3 Can I add rxdart to dependencies?

@simolus3 Things got slightly more complex, but I think it's still pretty clear
Lmk if anything isn't clear and i'll update the doccomment.
I've got no idea on how to use a transaction with a stream, so I'm going to need your help on that...

Apart from relatively minor things (not in these sense that we shouldn't fix them, just that they won't impact the overall implementation much), this looks good to me. So I agree with the API, design and the general implementation.

The problem with marking the original query to watch all referenced tables
is that ALL the queries will run again if anything changes.
Now only the reference table query is ran again.
Do you think you’ll be able to find a way to get transactions on streams
working?
On Wed, Jul 17, 2024 at 5:38 PM Simon Binder @.***>
wrote:

Apart from relatively minor things (not in these sense that we shouldn't
fix them, just that they won't impact the overall implementation much),
this looks good to me. So I agree with the API, design and the general
implementation.
—
Reply to this email directly, view it on GitHub
https://github.com/simolus3/drift/pull/3086#issuecomment-2234365509, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ASDJ623VUKG7EMSKYXKIVRLZM3P5LAVCNFSM6AAAAABKU43SLGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZUGM3DKNJQHE
.
You are receiving this because you authored the thread.Message ID:
@.***>

I was going through the possibilities of what a race cond could be:
Example groups.withReferences((prefetch)=> prefetch(users: true)).watch()

A groups is added in between queries:
Result: The group won't be in the final result, no error will occur.
A group is deleted in between queries,
Result: No users will be returned for this now-non-existent group.

There will be a single tick where info is out of data, but no errors will occur

@simolus3 I'm going to make this a bool on prefetch(inTransaction: false)
Complicates the code further, but I think this is valid solution to this issue.

@simolus3 If you're good with this I'm going to proceed

Adding tests for this won't be trivial.

I think it could be possible with a mocked executor or when manually invalidating stream queries. Definitely not easy though, let me know if you need more ideas on that.

Result: The group won't be in the final result, no error will occur.

I'm still not sure I like that - stream queries should not emit stale data. If we know that data is going to be out-of-date while a query is still running, we should ignore the results and run the query again (which is also what we're doing for single queries). We should find a way to provide these guarantees for stream queries as well (e.g. by not emitting a completed snapshot if the upstream groups table has changed while we're processing an earlier change by re-running the users query).

There will be a single tick where info is out of data, but no errors will occur

You left out the case where a group was updated, couldn't that emit an inconsistent snapshot if columns from the groups table are referenced in users? The principle of never emitting stale data should still apply though.

Is this impossible to get right?

I think the only fully sound solution is to run each fetch of the stream in a transaction. That can be inefficient, and we should be transparent about this caveat when mentioning streams with pre-fetches, but it's the only way I can come up with that provides consistency guarantees.
Even if we don't know about an update, or learn about an update that happened between the main and a prefetch query too late, we'd emit inconsistent rows.
In 58aa118fea323b9dd9b76c4f533af0d772c09fcc, I've explored a relatively simple change for this. You would create a StreamQueryFetcher that has a null key and fetchData: get. readsFrom must be all tables then. This makes sure that, if an update happens while we're fetching data, the (read-only) transaction is thrown away and we'd restart.

I'm confused. I was under the impression that we are already doing everything in a transaction:
I've added a private top level function called _watchWithAdditionalTables and am using a transaction for the referenced prefetches, I'm not sure why anything else needs to be added?

Have we thought about the following?

Cache the previous stream result, so we can emit the old result if we detect something has changed
How do we detect if something has changed? We have a UUID for each table, which updates when any mutating action is performed on it. We store this UUID on the db instance, along with each result each stream.

class db {
  final tableIds = {"table_name":"y37423483729..."}
}
/// The final combined stream which may contains inconsistent data
stream.listen((v){
  final typedResult, previousTypedResult ,tableUuid,tableName = v;
  if (db.tableIds[tableName] != tableUuid){
    return previousTypedResult;
  } else {
    return typedResult;
  }
});

class UUIDIterceptor extends QueryInterceptor {
  //... Listen for mutating queries and update db.tableIds
}

We can then use an CombineStream, where we have the best performance.
The above implementation should work, however It doesn't look super neat.
@simolus3 , what are your thoughts?

The above implementation should work, however It doesn't look super neat.

Wouldn't we still run the query then and only decide that we can cache the interpretation of results afterwards?
I also think this would be pretty hacky and unreliable. We cannot be 100% accurate about when a stream must update (a second connection to the database could also be made outside of drift for example). But I strongly feel that the actual results emitted by a stream must always be a consistent snapshot. And the only way to ensure that when multiple queries are involved is to use a transaction and invalidate everything when any involved query changes. By the way, I don't think this would be unacceptably slow either. We're doing the same for single queries with joins for instance, so I don't think users would be surprised that everything making up the query is re-run when an input table changes.

I was expecting such a response.
I completely agree.
What are our next steps to finish this PR (besides tests)?
On Tue, Jul 23, 2024 at 10:51 AM Simon Binder @.***>
wrote:

The above implementation should work, however It doesn't look super neat.
Wouldn't we still run the query then and only decide that we can cache the
interpretation of results afterwards?
I also think this would be pretty hacky and unreliable. We cannot be 100%
accurate about when a stream must update (a second connection to the
database could also be made outside of drift for example). But I strongly
feel that the actual results emitted by a stream must always be a
consistent snapshot. And the only way to ensure that when multiple queries
are involved is to use a transaction and invalidate everything when any
involved query changes. By the way, I don't think this would be
unacceptably slow either. We're doing the same for single queries with
joins for instance, so I don't think users would be surprised that
everything making up the query is re-run when an input table changes.
—
Reply to this email directly, view it on GitHub
https://github.com/simolus3/drift/pull/3086#issuecomment-2245472765, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ASDJ623SSSWELYUR36JP2BTZNZUXTAVCNFSM6AAAAABKU43SLGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVGQ3TENZWGU
.
You are receiving this because you authored the thread.Message ID:
@.***>

What are our next steps to finish this PR (besides tests)?

So implementation wise we still need to change the watch implementation to migrate from asyncExpand (which thus runs prefetch queries independently) to something actually ensuring each emit happens in a transaction. Given the small size you can probably cherry pick 58aa118fea323b9dd9b76c4f533af0d772c09fcc into this PR (it's not on develop yet since I wanted your feedback on whether that approach would work for you).
Given that the prefetch hooks already expose the list of additional tables, I think an implementation could simply be:
@override
Stream<List<$ActiveDataclass>> watch(
    {bool distinct = false, int? limit, int? offset}) {
  // Build the initial query once so that we know which tables it selects from
  // (todo if there's a better way to get the tables we should use it)
  var baseSelect =
      $state.prefetchHooks.withJoins($state).buildSelectStatement();
  var context = GenerationContext.fromDb($state.db);
  baseSelect.writeInto(context);

  return $state.db.createStream(QueryStreamFetcher<List<$ActiveDataclass>>(
    readsFrom: TableUpdateQuery.onAllTables([
      ...context.watchedTables,
      ...$state.prefetchHooks.explicitlyWatchedTables
    ]),
    fetchData: () async {
      return get(distinct: distinct, limit: limit, offset: offset);
    },
  ));
}

Now comes the fun part, TESTS!

@simolus3
Hmmm, something isn't quite working with the new CustomQuery feature.
Just using it plainly is throwing an error I can't seem to nail down.
Would you be able to take a look at this please?
Pull the branch and run this test:
dart test ./test/manager/manager_prefetch_test.dart  --chain-stack-traces  --plain-name "watch"

Seems to be an issue with a couple of fields on QueryStream which are not updated to use the Rows generic.
I don't understand much of this code, so I don't feel confident fixing this...

Thanks for the contribution!! :tada: :heart:


Add datalake-store/resource-manager/readme.typescript.md

Autogenerated

Automation for azure-sdk-for-python

Automation for azure-sdk-for-js

Can one of the admins verify this patch?

Automation for azure-sdk-for-node

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-go


Proposal for Flagger to join Flux project and CNCF

The Flagger authors are requesting for Flagger to join the fluxcd org.
Flagger extends Flux functionality with progressive delivery strategies like Canary Releases, A/B Testing and Blue/Green and it was specifically designed for GitOps style delivery.
Since the inception of the GitOps Toolkit, it’s clear that fluxcd/ will become more of a family of GitOps related projects. The Flagger maintainers looks forward to making use of the toolkit components and simplifying Flagger this way. Consolidating the code-bases and thinking in terms of a “Flux Family of Projects” and writing up the roadmap accordingly should benefit both communities as a whole.
Background information:

Approval of Flagger maintainers (Stefan Prodan and Takeshi Yoneda)
Approval from Weaveworks to transfer Flagger and its copyright to fluxcd org (and thus, CNCF)
Upcoming roadmap for Flagger includes GitOps Toolkit integration
License, dependencies and copyright review can be found here

We would like to present at one of the next Flux meetings and state our intent in more detail. We are looking forward to your questions and a good debate.

+1 great idea

Flagger is a great fit for the Flux project. I'm :100: on board with this.

As a maintainer, I am completely +1 :1st_place_medal:

+1 without any doubt

➕1⃣


Error after run ns-process-data

Hi,
I  ran ns-process-data images --data /images/ --output-dir images_with_depth/
and ns-process-data images --data /images/  --use-sfm-depth  --output-dir images_with_depth/
Then this core dump error happen on both situations.
I ran this on cuda/11.8 RTX4090.

??????????????????????????????????????????????  ?? ?? ?? ERROR ?? ?? ??  ???????????????????????????????????????????????
Error running command: colmap feature_extractor --database_path devPhase_01_nerfstudio_with_depth/colmap/database.db
--image_path devPhase_01_nerfstudio_with_depth/images --ImageReader.single_camera 1 --ImageReader.camera_model OPENCV
--SiftExtraction.use_gpu 1
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
QObject::moveToThread: Current thread (0x55ab6b85b070) is not the object's thread (0x55ab6b867670).
Cannot move to target thread (0x55ab6b85b070)
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in
"/home/kkennethwu/anaconda3/envs/nerfstudio_iccv/lib/python3.8/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may
fix this problem.
Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland,
wayland-xcomposite-egl, wayland-xcomposite-glx.
  Don't know how to solve it

Hi,
Try this command before running it.
export DISPLAY=:0.0

Hi, Try this command before running it.
export DISPLAY=:0.0

HI, I've tried it but it didn't work.
However, when I use cpu to run it it works.
ns-process-data images --data /images/  --use-sfm-depth  --output-dir images_with_depth/ --no-gpu

This is an issue with the colmap install with gpu support. Of you need the gpu support, I would try installing colmap from source.

pip uninstall opencv-python
pip install opencv-python-headless
```

Fixes the problem

For me this is not related with colmap, and you probably don't get the error anymore with `--no-gpu` because you don't use opencv the same way. I think that the error is link with opencv which use Qt which has some library missing; that's way I unisntall opencv-python to use opencv-python-headless


Create wiki page with details on how to test

As volunteered by @fzipi.

Created https://github.com/coreruleset/coreruleset/wiki/Testing-the-ruleset
Let me know if that works for you!

I'm closing this one. If you feel there are additional questions or stuff not covered, please let me know.


[ARM][Jetson] Make failed on Jetson TX2 (Jetpack 3.3, CUDA 9.0)

Description
Failed to compile on Jetson TX2 with Jetpack 3.3
INFO: nvcc was not found on your path
INFO: Using /usr/local/cuda-9.0/bin/nvcc as nvcc path

Killed
Makefile:471: recipe for target 'build/src/operator/tensor/ordering_op_gpu.o' failed
make: *** [build/src/operator/tensor/ordering_op_gpu.o] Error 137

Environment info (Required)
----------Python Info----------
('Version      :', '2.7.12')
('Compiler     :', 'GCC 5.4.0 20160609')
('Build        :', ('default', 'Nov 12 2018 14:36:49'))
('Arch         :', ('64bit', 'ELF'))
------------Pip Info-----------
('Version      :', '18.1')
('Directory    :', '/usr/local/lib/python2.7/dist-packages/pip')
----------MXNet Info-----------
No MXNet installed.
----------System Info----------
('Platform     :', 'Linux-4.4.38-tegra-aarch64-with-Ubuntu-16.04-xenial')
('system       :', 'Linux')
('node         :', 'tegra-ubuntu')
('release      :', '4.4.38-tegra')
('version      :', '#1 SMP PREEMPT Thu May 17 00:15:19 PDT 2018')
----------Hardware Info----------
('machine      :', 'aarch64')
('processor    :', 'aarch64')
Architecture:          aarch64
Byte Order:            Little Endian
CPU(s):                6
On-line CPU(s) list:   0-5
Thread(s) per core:    1
Core(s) per socket:    3
Socket(s):             2
Model name:            ARMv8 Processor rev 3 (v8l)
CPU max MHz:           2035.2000
CPU min MHz:           345.6000
Hypervisor vendor:     horizontal
Virtualization type:   full
L1d cache:             32K
L1i cache:             48K
L2 cache:              2048K
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0069 sec, LOAD: 0.4568 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0080 sec, LOAD: 0.4829 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0490 sec, LOAD: 1.0248 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0112 sec, LOAD: 0.3875 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0302 sec, LOAD: 0.7067 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.2152 sec, LOAD: 0.6906 sec.

Build info (Required if built from source)
Compiler:
gcc 5.4.0
Build config:
cp make/crosscompile.jetson.mk config.mk
USE_CUDA_PATH = /usr/local/cuda-9.0
Error Message:
INFO: nvcc was not found on your path
INFO: Using /usr/local/cuda-9.0/bin/nvcc as nvcc path
Running CUDA_ARCH: -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=[sm_70,compute_70] --fatbin-options -compress-all

...

/usr/local/cuda-9.0/bin/nvcc -std=c++11 -Xcompiler -D_FORCE_INLINES -O3 -ccbin g++  -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=[sm_70,compute_70] --fatbin-options -compress-all -Xcompiler "-DMSHADOW_FORCE_STREAM -Wall -Wsign-compare -O3 -DNDEBUG=1 -I/home/nvidia/incubator-mxnet/3rdparty/mshadow/ -I/home/nvidia/incubator-mxnet/3rdparty/dmlc-core/include -fPIC -I/home/nvidia/incubator-mxnet/3rdparty/tvm/nnvm/include -I/home/nvidia/incubator-mxnet/3rdparty/dlpack/include -I/home/nvidia/incubator-mxnet/3rdparty/tvm/include -Iinclude -funroll-loops -Wno-unused-parameter -Wno-unknown-pragmas -Wno-unused-local-typedefs -DMSHADOW_USE_SSE=0 -DMSHADOW_USE_F16C=0 -I/usr/local/cuda-9.0/include -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 -DMSHADOW_DIST_PS=0 -DMSHADOW_USE_PASCAL=0 -DMXNET_USE_SIGNAL_HANDLER=1 -DMXNET_USE_OPENCV=0 -fopenmp -DMXNET_USE_OPERATOR_TUNING=1 -DMXNET_USE_LAPACK -DMSHADOW_USE_CUDNN=1 -I/include -I/usr/include/aarch64-linux-gnu/ -I/home/nvidia/incubator-mxnet/3rdparty/cub -DMXNET_USE_NCCL=0 -DMXNET_USE_LIBJPEG_TURBO=0" --generate-dependencies -MT build/src/operator/tensor/broadcast_reduce_op_value_gpu.o src/operator/tensor/broadcast_reduce_op_value.cu >build/src/operator/tensor/broadcast_reduce_op_value_gpu.d
/usr/local/cuda-9.0/bin/nvcc -c -o build/src/operator/tensor/broadcast_reduce_op_value_gpu.o -std=c++11 -Xcompiler -D_FORCE_INLINES -O3 -ccbin g++  -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=[sm_70,compute_70] --fatbin-options -compress-all -Xcompiler "-DMSHADOW_FORCE_STREAM -Wall -Wsign-compare -O3 -DNDEBUG=1 -I/home/nvidia/incubator-mxnet/3rdparty/mshadow/ -I/home/nvidia/incubator-mxnet/3rdparty/dmlc-core/include -fPIC -I/home/nvidia/incubator-mxnet/3rdparty/tvm/nnvm/include -I/home/nvidia/incubator-mxnet/3rdparty/dlpack/include -I/home/nvidia/incubator-mxnet/3rdparty/tvm/include -Iinclude -funroll-loops -Wno-unused-parameter -Wno-unknown-pragmas -Wno-unused-local-typedefs -DMSHADOW_USE_SSE=0 -DMSHADOW_USE_F16C=0 -I/usr/local/cuda-9.0/include -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 -DMSHADOW_DIST_PS=0 -DMSHADOW_USE_PASCAL=0 -DMXNET_USE_SIGNAL_HANDLER=1 -DMXNET_USE_OPENCV=0 -fopenmp -DMXNET_USE_OPERATOR_TUNING=1 -DMXNET_USE_LAPACK -DMSHADOW_USE_CUDNN=1 -I/include -I/usr/include/aarch64-linux-gnu/ -I/home/nvidia/incubator-mxnet/3rdparty/cub -DMXNET_USE_NCCL=0 -DMXNET_USE_LIBJPEG_TURBO=0" src/operator/tensor/broadcast_reduce_op_value.cu
Killed
Makefile:471: recipe for target 'build/src/operator/tensor/ordering_op_gpu.o' failed
make: *** [build/src/operator/tensor/ordering_op_gpu.o] Error 137
make: *** Waiting for unfinished jobs....

Steps to reproduce
sudo apt-get update
sudo apt-get -y install git build-essential libatlas-base-dev libopencv-dev graphviz python-pip
sudo pip install pip --upgrade
sudo pip install setuptools numpy --upgrade
sudo pip install graphviz jupyter

git clone https://github.com/apache/incubator-mxnet.git --recursive
cd incubator-mxnet

cp make/crosscompile.jetson.mk config.mk

# Edit config.mk with vim
USE_CUDA_PATH = /usr/local/cuda-9.0

# Edit the Mshadow Makefile after line 122
MSHADOW_CFLAGS += -DMSHADOW_USE_PASCAL=1

make -j $(nproc)

What have you tried to solve it?
Changed CUDA path due to nvcc not found

As of today (March 27th) I have similar errors on Jetson TX2, but with JetPack 3.2.1.
The action stated as "# Edit the Mshadow Makefile after line 122 <MSHADOW_CFLAGS += -DMSHADOW_USE_PASCAL=1>" does not solve the issue.
When I do it and move to  $make -j $(nproc) , I still get errors.
I'm still attempting to break through my similar error, however I noticed something on your error log.
 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=[sm_70,compute_70] --fatbin-options -compress-all
This error may occur because you didn't limit your installation to your TX2 CPU and GPU.
One way around it might be removing all other architectures from you Makefile, i.e., 53 and 62 only.  The aarchs in your -gencode should be these two.
There are yet more bullet points to address, such as: USE_SSE must be set to 0 on msshadow.mk.
And yet more, I'll update as I go.
Cheers.


[sc-28951] Action to downlaod schema.gql and check it into the repo

🤔 Why?

We need to have the latest API schema in order to make sure codegen isn't targeting outdated schema. In order to do that, the Metaphor app repo will upload the latest API schema as a GH artifact, and here we need to download it.
🤓 What?

Added a GH action that is triggered by:

Manually, with a run_id as input
By the workflow implemented in https://github.com/MetaphorData/app/pull/9759.

This action also creates a PR to check the downloaded schema.gql into the repo.
🧪 Tested?

Not yet, no artifact has been uploaded yet.

not gonna do this, we are not using gh artifact for this


[type:fix] fix license

Make sure that:

[ ] You have read the contribution guidelines.
[ ] You submit test cases (unit or integration tests) that back your changes.
[ ] Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

Codecov Report

Merging #3868 (69d4c19) into master (7909785) will decrease coverage by 0.34%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #3868      +/-   ##
============================================
- Coverage     64.76%   64.41%   -0.35%
+ Complexity     6099     6070      -29
============================================
  Files           898      898
  Lines         24729    24729
  Branches       2240     2238       -2
============================================
- Hits          16015    15930      -85
- Misses         7307     7399      +92
+ Partials       1407     1400       -7

Impacted Files
Coverage Δ

...n/alibaba/dubbo/cache/AlibabaDubboConfigCache.java
47.12% <0.00%> (-31.04%)
:arrow_down:

...controller/ShenyuClientHttpRegistryController.java
77.77% <0.00%> (-22.23%)
:arrow_down:

...ruptor/RegisterClientServerDisruptorPublisher.java
52.94% <0.00%> (-11.77%)
:arrow_down:

...henyu/plugin/grpc/resolver/ShenyuNameResolver.java
51.06% <0.00%> (-11.71%)
:arrow_down:

...a/org/apache/shenyu/common/utils/VersionUtils.java
67.85% <0.00%> (-10.72%)
:arrow_down:

.../apache/shenyu/admin/model/dto/BatchCommonDTO.java
50.00% <0.00%> (-7.15%)
:arrow_down:

...apache/shenyu/admin/model/dto/AuthPathWarpDTO.java
57.14% <0.00%> (-7.15%)
:arrow_down:

...org/apache/shenyu/admin/model/dto/AuthPathDTO.java
58.82% <0.00%> (-5.89%)
:arrow_down:

...rg/apache/shenyu/plugin/rewrite/RewritePlugin.java
66.66% <0.00%> (-5.56%)
:arrow_down:

...nyu/plugin/motan/cache/ApplicationConfigCache.java
52.00% <0.00%> (-5.00%)
:arrow_down:

... and 27 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Move to Compose v2 (a.k.a "Local Docker Compose")

I saw that docker-compose support was recently added; i suggest to install the newer implementation based on go.
Considering that it is based on the "cli plug-in mechanism" we can add the ability to install external cli plug-ins as packages or wait for it to become part of the base docker cli installation.

I think we can consider this issue outdated since the API has changed a lot.

Closing this one after discussion with Tom


Profiler not visible anymore

since upgrade to official nugets?
reported by @sebllll in chat.

yes, i think the switch to the official nuget was exactly the moment it disappeared.

yes, i think the switch to the official nuget was exactly the moment it disappeared.


[Improvement] Driver communication layer

driver communication layer that exchanges commands and events with ovs and ovsdb, will optimize the most of ACA cost to best use up ovs and ovsdb's performance capacity

Port libfluid into Alcor repo for connection management, doing unit testing for ovsdb.

Build ovs performance test tool (like 2k, 5k, 10k and more flow-mod rules to ovs via the long running channel) on top of libfluid threading-model

Fixed cmake to integrate lib-fluid ovs helper and existing ACA vsctl vconn helper, both are using a unified set of openvswitch include headers and library.


Support multi-arch container images

Currently, the Dockerfile can only build x86_64 images, due to the following lines:
https://github.com/parca-dev/parca-agent/blob/8338e7725d3cda39f1f5ae2c79be2d6370fd1c97/Dockerfile#L33-L40
On the latest Mac this location is actually aarch64-linux-gnu for example. Changing it in the Dockerfile to the correct location makes docker build correctly produce an image.

Dup of https://github.com/parca-dev/parca-agent/issues/54?

Technically that issue only mentions arm64, but I am fine by closing this if you don't mind changing your title maybe?

Changed.
Side note: arm64 is the same as aarch64 :)

Side note: arm64 is the same as aarch64 :)

Yes, I meant for other arch as well, hence multi-arch.


upgrading AWS Redshift DB from Flyway 3.2.1 to 4.0 failed - couldn't upgrade metadata table

What version of Flyway are you using?
3.2.1
What database are you using (type & version)?
AWS Redshift
What operating system are you using?
Linux
What did you do?
(Please include the content causing the issue, any relevant configuration settings, and the command you ran)
Changed my code to use Flyway 4.0. On calling Flyway#migrate, Flyway tried to upgrade and failed (details below).
What did you expect to see?
Expected to see successful Flyway init.
What did you see instead?
Flyway init failed:
2016-03-03 23:09:14,251 INFO  [main] o.f.c.i.m.MetaDataTableImpl - Upgrading metadata table "public"."schema_version" to the Flyway 4.0 format
...
org.flywaydb.core.internal.dbsupport.FlywaySqlScriptException:
Script failed
SQL State  : XX000
Error Code : 500310
Message    : Amazon Invalid operation: cannot insert/update into table after dropping non-nullable column;
Line       : 20
Statement  : UPDATE "public"."schema_version" SET "version_temp"="version"
at org.flywaydb.core.internal.dbsupport.SqlScript.execute(SqlScript.java:117)
at org.flywaydb.core.internal.metadatatable.MetaDataTableImpl.upgradeIfNecessary(MetaDataTableImpl.java:79)
at org.flywaydb.core.Flyway.execute(Flyway.java:1356)
at org.flywaydb.core.Flyway.migrate(Flyway.java:917)

@nathanvick Can you have a look at this? Thanks!

Can you tell me what version of Redshift is being used? The error is not familiar to me, but it makes me wonder if it is an old version of Redshift. I have been dropping not null columns and then updating data for a couple years.
http://docs.aws.amazon.com/redshift/latest/dg/r_VERSION.html
PS: I tested this with flyway 4 (command line), upgrading from flyway 3.2 (command line), and it worked for me,  using both the Postgres jdbc driver and the Redshift jdbc driver.

Hmmm. Not aware there was a choice of Redshift version, it's SaaS. See below for version info nonetheless. I'm invoking Flyway through the API rather than the command line.
Which version of the Redshift driver are you using? On the download page  there is a JDBC 4.0 driver and a 4.1 driver. We are using version <IP_ADDRESS>.1007. I see there is a more recent driver version available, will see if the newer driver works any better.
Sounds like not much more you can do on your side. Thanks for looking into it,
Walter
select version; returns:
PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.1036

It is possible to refuse  major version upgrades...
"Amazon Redshift provides a setting, Allow Version Upgrade, to specify whether to automatically upgrade the Amazon Redshift engine in your cluster if a new version of the engine becomes available. This setting does not affect the database version upgrades, which are applied during the maintenance window that you specify for your cluster. Amazon Redshift engine upgrades are major version upgrades, and Amazon Redshift database upgrades are minor version upgrades. You can disable automatic version upgrades for major versions only. For more information about maintenance windows for minor version upgrades, see Maintenance Windows."
http://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html
...but I don't think that's the case here. Unfortunately, I can't confirm what version of Redshift I was using because I accepted a new position at a different company, and my last day was today.
I'm still confused as to why you hit this error and I didn't.  Perhaps the best solution is to change tge migration to the following.

Rename old metadata table (so as to lock it)
Create new metadata table
Copy data to new metadata table
Drop old metadata table

This AWS forum thread cannot insert/update into table after dropping non-nullable column looks relevant. Because the AWS forums require a login, I'm including the content below. Will try including a commit in the upgrade script and see if that helps.
"Hi, I was running a migration against my Redshift database and found a weird behavior- running all the migrations in one shot fails with the message
cannot insert/update into table after dropping non-nullable column
while running them individually succeeds. It turns out that two of our migrations modify a column on the same table- by adding a temp column, copying data to the temp column, and dropping the old column. For some reason, dropping the column then puts some kind of restriction on the db that we can't add data to that table until we do a commit. Running a commit manually gets things to work again, but we don't understand the purpose of this restriction, and couldn't find anything in the docs to explain exactly what is going on (running the same script against postgres works)"

Ok, thanks for the details from the forum post. Please let me know how the commit workaround goes.
However,  I still think creating a new table would be a better solution because if the migration fails for any reason, it can all be rolled back, whereas committing midway would prevent that.

That is a good point. I'll try your suggestion. Having trouble building Flyway because of MS SQL jar, which I understand can't be mavenized publicly because of licensing issues, but should be able to locally install it.

I could run the script standalone but if I do it as part of Flyway I'll be able to submit a pull request.

Possible for me to test a change to flyway-core/src/main/resources/org/flywaydb/core/internal/dbsupport/redshift/upgradeMetaDataTable.sql without building Flyway at the top level? I've manually installed MS SQL Server jar and the Oracle driver jar, now the build is failing on the IBM jar, fearing that it's going to take a while to work through all these proprietary drivers that I don't actually need for this test. Tried building flyway-core by itself but that has the same problem.

You can build Flyway with -DskipTests and run the Redshift tests in your IDE

I made a suggested fix here: https://github.com/nathanvick/flyway/commit/4f0b1d852434ad02722107321961a776e6e81443
I hope to be able to test it out sometime this week.

Thanks! Just tried it & there is a problem. I was able to fix the problem but not sure my fix is generally acceptable. Nathan, see my comment on your commit for details.

@nathanvick Can you submit a PR for this? Thanks!

@alexfontaine,
Sure, I'll try to get to it this week.


Archive option doesn't seem to be deleting anything for some reason

It will say it went through like 2403 tweets but none of them actually get deleted. Only the non archive version deletes tweets and gets rate limited after a bit of time

I am not even getting a box or anything to upload the tweet.js file

Yeah, I can confirm that this method doesn't work. It says "Deletion complete" and lists like two retweets that it claims to have deleted, but it didn't. And it completely misses all normal tweets. The old method works fine.

Still not working. When I tried the archive method it would get stuck at 917 or something, the page would become unresponsive, I'd force close Chromium and then none of my tweets would be deleted.


Openmetrics integration stops scraping metrics whenever datadog-agent is restarted

Hello, I would like to ask for help to a debug a problem with openmetrics integraton.
TL'DR after each restart of datadog agent we loose metrics from the integration. We don't see any errors that would point us to some direction where the problem could be.

Output of the info page
===============
Agent (v7.35.2)
===============
  Go Version: go1.17.6
  Python Version: 3.8.11
  Build arch: amd64
  Agent flavor: agent
  Check Runners: 4
  Log Level: DEBUG

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Host Info
  =========
    bootTime: 2022-06-24 10:09:22 UTC<PHONE_NUMBER>000)
    kernelArch: x86_64
    kernelVersion: 5.4.170+
    os: linux
    platform: ubuntu
    platformFamily: debian
    platformVersion: 21.10
    procs: 285
    uptime: 1h59m58s
    virtualizationRole: guest

Additional environment details (Operating System, Cloud provider, etc):
Steps to reproduce the issue:

create kubernetes deployment and configure openmetrics check

        ad.datadoghq.com/app.check_names: '["openmetrics"]'
        ad.datadoghq.com/app.init_configs: '[{}]'
        ad.datadoghq.com/app.instances: |-
          [{
            "prometheus_url": "http://%%host%%:8080/metrics",
            "namespace": "test",
            "max_returned_metrics": 200000,
            "metrics": [ "*" ],
            "send_distribution_counts_as_monotonic": true,
            "send_distribution_buckets": true,
            "send_histograms_buckets": true,
            "tags": ["pod_name:%%kube_pod_name%%"]
          }]

apply deployment and watch metrics in datadog metrics explorer (they should appear after a while)
rollout restart datadog agent daemon set
metrics won't be scraped anymore
restart application
metrics will get back

Describe the results you received:
unavailable metrics after DD upgrades or rollout restarts
Describe the results you expected:
scraping of prometheus endpoint would work even after restart of DD
Each hole in the picture is caused by DD restart - reproducible 100% of time.

Thank you in advance for any kind of help.

Hi @kirecek, thank you for reaching out. Can you please reach out to our Datadog Support Team to open a support ticket so that we can investigate this issue more thoroughly? Our support team will be better able to assist in case we need additional information from you.


[admin] merge to live

Docs Build status updates of commit 2967b2c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.publish.config.json
:white_check_mark:Succeeded

officedocs-dev-office-pia/_zip/msdn.4.5.2.zip
:white_check_mark:Succeeded

officedocs-dev-office-pia/docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Filenames of videos, thumbnails, preview can be too long thus workflow fails

Describe the bug
The filenames of any process file can be too long, therefore tools like ffmpeg etc. fail with error and the whole workflow might fail.
2021-06-09T15:53:12,055 | INFO  | (EncoderEngine:461) - /srv/opencast/workspace/mediapackage/b39f3e94-81c0-48ac-9a5f-abb07a208c2c/bf15e33c-0eec-4855-bd92-ca30514723e5/2021_06_09_13_36___WS13_1_Rare_species_may_act_as_problem_solvers_in_critical_situations_of_the_bacterial_co_occurence_network_in_cystic_fibrosis_airways___Marie_M*****_P***__presentation__OC_Studio__8c349465-3440-4563-b2ea-4d8952b6d4f9-preview-composite.mp4: File name too long
Here so many informations will be concatted together, which the system itself would not need: date, title of the recording, name of the presenter type of the video (presenter/presentation), roomname plus uuids etc.
Seems to me, that these are for human readability only.
My opinion of naming and handling files, internally work with uuids only and use human-readable naming only for downloads and respect filesystem specific conventions (pull human-readable data from db & truncate some info e.g. title).
To Reproduce
Steps to reproduce the behavior:

Go to 'Add Event'
Enter Title of at least 150 characters approx.
Enter Presenter(s) of at least 20 characters approx.
Start Studio Upload Workflow
Check opencast.log as the webfrontend yields unspecific error details

Environment Information

OS: Debian Stretch
Browser any
Opencast version with minor release <=8.11

Since this is related to Opencast Studio which has a separate issue tracker, would you mind reporting this at
https://github.com/elan-ev/opencast-studio

I would be in favor if Opencast would resolve this for the clients using the ingest and upload APIs (either reject or truncate). It makes integrations unnecessary complex otherwise, see for example

https://github.com/elan-ev/studip-opencast-plugin/issues/372
https://github.com/elan-ev/studip-opencast-plugin/issues/489
https://github.com/Opencast-Moodle/moodle-block_opencast/issues/204

And at Moodle, there is no report, yet, but that will probably change today.

If we're munging the names, why not rename the file to a UUID, and then store the input filename as an asset property?

That would just move the problem to a different location. Once you distribute the files, you would then end up with the same problem unless you also build the UUID +  storing the filename separately into the distribution channels, which then means that this could cause issues when users try downloading the files (that's the reason why this was already fixed in Studio). So you likely will end up with a lot of work and a complex system for little gain. I would rather just limit the filenames to ~75 characters and be done with it. Much simpler and I doubt anyone will complain that they can't have a novel as a filename. Let me propose a patch…


Remove root from config

Attempt to solve https://github.com/modulz/stitches/issues/832
We think the problem is coming from https://github.com/modulz/stitches/blob/canary/packages/core/src/createStitches.js#L34
In our config, we are passing the root. When we memoize root, we encounter the circular error.
And we might not need root any longer. We removed the root from the config in this PR.
When it was used, it was used to keep a reference to the document hosting all of the stylesheets. We can’t recall whether it was needed for the React library or for an earlier version of the CSSOM functionality.

The root config options is required for shadowRoot support and should not have been removed! (cc. https://github.com/modulz/stitches/issues/628 & https://github.com/modulz/stitches/discussions/747)
The real bug is on the following line, invoked by the createMemo() function:
https://github.com/modulz/stitches/blob/0ebaf9f988871ac0d8d5f2b72f2a8042e0d1b56f/packages/core/src/utility/createMemo.js#L1
Will open a PR to fix now.

Opened a PR here: https://github.com/modulz/stitches/pull/1004


[Feedback]: German translations (for scenarios & compartments etc.) & Ability to rename Scenarios

Feedback
German translations (for scenarios & compartments etc.) & Ability to rename Scenarios.
Depends on work by HPC.

Related Issues:

#158

We need to temporarily hack translations for backend data into the frontend. This has the highest priority.
An idea is that we can also use crowdin for translations that are missing.

Haven't used that yet. Maybe we can have a meeting to discuss the possible options?
I think we might also be able to use the existing translations via the json files even though it means the translated compartments would be hardcoded for now.
Depending on the implementation it might be easy for everyone to quickly push translation changes into the development build.

We might as well define an API endpoint in the frontend and just generate the translations in there with the help of some translation system similar to the generator I used for the migrations during the coding week.
This way we can easily remove it later and fall back onto the API once the funcionalities are in place while already using the proper channels in the frontend.


GraphQL language parser

The plan is to use the CPP GraphQL parser. It can be wrapped as a PHP extension, so it saves us the trouble of writing our own parser. There is even a ready-made wrapper for it, but some tests seemed to fail while I tried to build an extension with the latest version of the parser.

We decided to give developers the option to use the CPP parser, but we are also going to provide a shim so that this library can be used without any dependencies.

This is done for now, but we are still missing #9 before we can close this issue.
Currently, we should be able to build the AST for any query you can throw at the parser.


Remove guest records in the next round of sync for all users.

Sync to get the latest version deployed.
Remove manually just one record with partition key UsersSyncData from UserData.

Send a message to all => Guests should NOT receive the message. All guest users should be removed forever from the UserData table automatically.

IMO this simple fix can resolve all potential issues with current customers and requires minimum effort from your and the customer side @aosolis what do you think?

Closed as we have the better solution in #450


Minor error in documentation

The readme says ...
calendarConfig.allDateFormats.moment.title = 'ddd D MMM';

... but i think it should be like this:
calendarConfig.titleFormats.day = 'ddd D MMM';
calendarConfig.titleFormats.week = 'Week {week} of {year}';

That's intentional - titleFormats is a getter only that retrieves the config from the allDateFormats object. See the source here to see what I mean.

okay i see, but than the .day is missing - right?
calendarConfig.allDateFormats.moment.title.day = 'ddd D MMM';

Oh yeah you're correct, fixed now, thanks!


Question: farrow-pipeline & Building system

Is it possible to implement a building system tool library based on farrow-pipeline?
https://mp.weixin.qq.com/s/Zg-snOR7BG6l6DojqzO3Yg

store: context
task: combination between pipelines
build: pipeline.run or other forms

I think the answer will be No!
Since the internal structure behind farrow-pipeline is a list of middlewares, but the structure behind a build-system should be a graph or tree to manage the dependences graph/tree, the graph for circular references, the tree for non-circular references.
But farrow-pipeline may still be useful for Task. Any plain function equipped farrow-pipeline can be divided into the combination of some smaller peaces of middlewares, such like the resolver function in GraphQL.


Update scipy

Pull Request Description
What issue does this change request address?
#1882
What are the significant changes in functionality due to this change request?
Updates scipy

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[x] 1. Review all computer code.
[x] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[x] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[x] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[x] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[x] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[x] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[x] 8. If an analytic test is changed/added is the the analytic documentation updated/added?
[x] 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

Job Test Ubuntu 18 PIP on d4f4cf1 : invalidated by @joshua-cogliati-inl
updated civet

Job Test Ubuntu 18 PIP on 32811e8 : invalidated by @joshua-cogliati-inl
restarted civet

checklist passed, and PR can be merged.


Where did all the changes I made vanish to?

I practically rewrote this is many large places as it was a total mess, and yet the contributions show that I contributed 2 lines. Did you copy/paste my code so you'd get the credit for it? I know for sure they're in the codebase.

@ChangePlaces check the commit you will find your name there.
You didn't git config to your local machine that's why your username not showing properly.
I always respect my contributors.

@ChangePlaces Did you get it?


How many active users and developers?

Survey from an outsider:
I was wondering who is using this project and not in secret?
How many developers are actively supporting this platform?
How much commercial interest is out there?
P.S. Adding a quick disclaimer that I am just an interested outsider on this project.

I have not contributed code to this project but I am actively using it. We are building a tvOS and Amazon FireStick app. I am a part of a small lean team inside a larger company. After a couple days of R&D we decided to go with this project as opposed to You.I TV.
I appreciate all the hard work to keep this project going. If I can find ways to help contribute in the future I will.

At Playlister our Apple TV app uses this project. We have 3 - 5 engineers working on the project and have been in production for about a year and a half. The rest of the platform is React and React Native so React Native was the natural choice for our Apple TV app. We recently moved from the core repo to this project.

We build a TV App with React Native (TADAAM: https://tadaam.be). After they announced that the tvOS support could be dropped in the next releases we switched to this repo.
The good news is that @douglowder did a great job setting all this up. One of our own custom components (the FocusGuide) was also added and merged. That would never have happened in the Core repo. So please contribute and help us out.

Also building OTT application with React Native.

Thanks to all for the comments, nice to see that the tvOS support is alive and in active use. Any objections to keeping this issue open?
A quick disclaimer is that I am just an interested outsider in this project.

Yes I’ll keep this open!

At JunctionTV, we have built the CONtv and Dove Channel apps using React Native for the last two years after switching from native. Now we are starting to build TV apps also. We will be using this repo for our TV apps.
Thanks @dlowder-salesforce and the contributors for all the hard work in maintaining this repo.

i am using this package and it's awesome. and i am building tvOS for my company

Yes many thanks to @dlowder-salesforce et al., we are using this repo to support AppleTV on our video-on-demand platform here at Shift72

🤚We are using this repo to develop an androidTv and tvOS app ,  we are developing an app similar to netflix, which includes many flatlists, video component and navigations. Till now i have suffered any issues.
Thanks to @dlowder-salesforce

I'm using this library to build an app to android tv an apple tv

The NFL app on tvOS, Android TV and FireTV are all built with this fork!

Using this to build an app line plex/emby for Android TV 😊

Using this to build android tv  project 😄

We build a TV App with React Native (TADAAM: https://tadaam.be). After they announced that the tvOS support could be dropped in the next releases we switched to this repo.
The good news is that @dlowder-salesforce did a great job setting all this up. One of our own custom components (the FocusGuide) was also added and merged. That would never have happened in the Core repo. So please contribute and help us out.

Wow this is amazing! This gives me hope.  I am trying to figure out how to get the fork using this project set up but your project looks amazing!

@esegebart you can do
react-native init TestTVApp --template=react-native-tvos

and that will set up a new project using this repo.
This is from 2020, but the fundamentals discussed in this article are generally still up to date:
https://douglowdercode.medium.com/react-native-on-the-big-screen-part-1-introduction-c2734db18289

@esegebart you can do
react-native init TestTVApp --template=react-native-tvos

and that will set up a new project using this repo.
This is from 2020, but the fundamentals discussed in this article are generally still up to date:
https://douglowdercode.medium.com/react-native-on-the-big-screen-part-1-introduction-c2734db18289

Fantastic!!! I knew I had seen this page before but couldn't find it on here. I got it set up and about to give it a whirl! Thank you so much!

Hi 👋 , I recently started using this repo. This is great 😊 . I am building a TV app using this package. Sometimes when doing development work, problems arise and get stuck if there is no information to read. So I want to know, is there any forum where we can discuss issues and improvements related to this package? Please let me know.

@avishka-codimite we can use Twitter for this, maybe the community https://twitter.com/i/communities/1509407040095068166 would be a good place to start. If there is enough interest, I can set up a discussion repo in the react-native-tvOS workspace.

@douglowder thank you. yeah, I think it better if have a discussion section 👍

@douglowder @avishka-codimite I would be interested in a discussion section! I am stuck on a few things.

Discussions were created by @douglowder on https://github.com/react-native-tvos/react-native-tvos/discussions
@esegebart Feel free to create a discussion and I'll be happy to help!

@douglowder Thank you so much!!!


docker: Docker-CSI Southbound Bridge

This patch introduces a Docker-CSI compatibility layer called the Docker-CSI Southbound Bridge. The Docker module has been removed from REX-Ray, and Docker support is now enabled via the CSI module. All CSI module instances will, unless explicitly disabled, multiplex incoming requests to either a CSI service or the Docker Volume API. The Docker Volume API receiver satisfies requests by using a CSI client.
This is a work-in-progress and is not-yet-complete:

[x] Cursory testing looks good, but still need to complete the detailed testing for each Docker API operation
[x] Update CSI-VFS dependency to revision b386662
[x] Before merging this PR, CSI-VFS should be tagged instead of depending on its revision directly (tagged as v0.1.7)
[ ] Tag CSI-NFS and update RR (assigned to @codenrhoden)
[ ] Tag CSI-BlockDevices and update RR (assigned to @codenrhoden)
[x] Update the CSI-libStorage bridge

[x] Add name to VolumeInfo.Id.Values returned from the CSI-libStorage bridge
[x] Add targetpaths to VolumeInfo.Metadata.Values returned from the CSI-libStorage bridge. Hopefully @codenrhoden can assist with this

[x] Docker bridge sets value in outgoing context that is received by libStorage bridge
[x] libStorage bridge looks at above value and returns mount info if value is true

[x] Perform an initial List call to cache the volume name-to-ID mappings
[ ] Create

[ ] Configure sensible defaults for a CSI create call OR
[ ] Parse the Docker options into the CSI CreateVolume options
[x] CreateVolume
[x] Add volume to the cache
[ ] Better logging
[ ] Testing

[ ] List

[x] ListVolumes
[x] Cache ListVolumes result for future name-to-ID mappings
[ ] Better logging
[ ] Testing

[ ] Get

[x] Implement
[ ] Better logging
[ ] Testing

[ ] Remove

[x] NodeUnpublishVolume
[x] ControllerUnpublishVolume
[x] DeleteVolume
[x] Remove volume from the cache
[ ] Better logging
[ ] Testing

[ ] Path

[x] Implement
[ ] Better logging
[ ] Testing

[ ] Mount

[x] GetNodeID
[x] ControllerPublishVolume
[x] NodePublishVolume
[x] Update volume's state in the cache
[ ] Better logging
[ ] Testing

[ ] Unmount

[x] NodeUnpublishVolume
[x] ControllerUnpublishVolume
[x] Update volume's state in the cache
[ ] Better logging
[ ] Testing

[x] Capabilities

Fixes #1037

Codecov Report

Merging #1055 into master will decrease coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1055      +/-   ##
==========================================
- Coverage   34.19%   34.16%   -0.04%
==========================================
  Files          36       36
  Lines        2901     2901
==========================================
- Hits          992      991       -1
- Misses       1806     1807       +1
  Partials      103      103

Impacted Files
Coverage Δ

libstorage/api/types/types_localdevices.go
84.93% <0%> (-1.37%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b110206...4320a0f. Read the comment docs.

cc @clintkitson @codenrhoden @cduchesne
The Problem
NOTE: This problem does not apply to native CSI plug-ins.
In 0.10.0 we introduced the CSI-libStorage, northbound bridge. This enabled  CSI clients to use all of the storage platforms supported by libStorage. At this  point in time the Docker module in REX-Ray was still accessing libStorage  drivers directly: ex. integration, storage, etc. In other words, this was the model:
CSI Client    -> RR CSI Endpoint    -> libStorage bridge -> libStorage drivers

Docker Client -> RR Docker Endpoint -> libStorage drivers

0.10.x Docker Mount Operation
A Docker Mount operation looked like this:
Docker Client -> Mount -> RR Docker Endpoint -> libStorage Integration Driver --
                                                                               |
              <- targetPath ----------------------------------------------------

The libStorage integration driver would create a directory to mount the volume once it was attached (if block storage) or mount the volume directly if it was shared storage.
The path to the mounted volume was returned to Docker. This point is key.
0.10.x CSI NodePublishOperation
A CSI NodePublishVolume operation looked like this:
CSI Client -> NodePublishVolume[targetPath] -> RR CSI Endpoint -> libStorage Bridge -> libStorage Integration Driver

Ultimately a CSI NodePublishVolume sat on top of our standard workflow. @codenrhoden  wrote very elegant code to then bind mount the mounted volume to the requested targetPath.
0.11.x Docker Mount Operation
Here's where it gets tricky for existing libStorage platforms:
Docker Client -> Mount -> RR Docker CSI bridge -> NodePublishVolume[targetPath] -> CSI Endpoint -> libStorage Integration Driver --
                                                                                |
              <- targetPath -------------------------------------------------------------------------------------------------------

Because REX-Ray's Docker endpoint now talks to CSI instead of libStorage directly,  the RR Docker->CSI bridge is the CSI client, or a CO. This means to perform a  NodePublishOperation the Docker endpoint must explicitly provide a targetPath.
The Bug
This one is on me. When writing the Docker->CSI bridge I knew I needed to generate  a targetPath. At first I thought I needed to locate it outside the normal mount area defined by libstorage.integration.volume.operations.mount.path. Then I thought, "No, I'm overthinking it." Well, I wasn't.
The libStorage->CSI Bridge is bind mounting the private mount path to the supplied targetPath. And this is happening:
$ mount -o bind /var/lib/rexray/volumes/MyVolume/data /var/lib/rexray/volumes/MyVolume

In the end, the mount table will look like this:
SOURCE                                 MOUNT POINT
/dev/xvda1                             /var/lib/rexray/volumes/MyVolume
/var/lib/rexray/volumes/MyVolume/data  /var/lib/rexray/volumes/MyVolume

Here's the thing, the above technically works. You can bind mount a path back over itself. Except it screws up idempotency. The first unmount succeeds,  but successive, idempotent unmount attempts fail because the targetPath no longer exists since the first unmount removed the targetPath due to ref counting. And the idempotency provider for the libStorage->CSI bridge expects the targetPath to exist or an error is returned:
var errMissingTargetPath  = errors.New("target path not created")

// IsNodePublished should return a flag indicating whether or
// not the volume exists and is published on the current host.
func (d *driver) IsNodePublished(
	ctx xctx.Context,
	id *csi.VolumeID,
	pubInfo *csi.PublishVolumeInfo,
	targetPath string) (bool, error) {

	_, err := os.Stat(targetPath)
	if os.IsNotExist(err) {
		return false, errMissingTargetPath
	}

The Solution
Here's where I'm at with this:

I don't think the libStorage-CSI bridge's idempotency provider should necessarily return an error if the provided targetPath does not exist. I need to confirm this with @codenrhoden, but ultimately shouldn't this just mean the volume is not published on the node, and we should return false without an error?

To maintain backwards compatibility with environments that have already deployed REX-Ray we cannot change where REX-Ray's Docker integration driver mounts volumes. However, I suggest we introduce a new property, rexray.csi.mount.path, with a default value of /var/lib/rexray/csi/volumes, and use it as the root location for volumes mounted via REX-Ray's libStorage->CSI bridge. This means that if Docker wants to mount a volume named MyVolume, the system's mount table would look like this:

SOURCE                                 MOUNT POINT
/dev/xvda1                             /var/lib/rexray/volumes/MyVolume
/var/lib/rexray/volumes/MyVolume/data  /var/lib/rexray/csi/volumes/MyVolume

So going forward, new volumes created by the Docker->CSI bridge would be presented to Docker with a mount path rooted at  /var/lib/rexray/csi/volumes. However, the above change would also be backwards compatible with existing deployments.
Please provide your thoughts and comments below.

Thanks for the write-up @akutz.  It really helped me understand.

I don't think the libStorage-CSI bridge's idempotency provider should necessarily return an error if the provided targetPath does not exist. I need to confirm this with @codenrhoden, but ultimately shouldn't this just mean the volume is not published on the node, and we should return false without an error?

It was considered an error because the CO shouldn't ever be sending us a targetPath that does not exist. However, I see no problem with the idempotency provider punting in this scenario and allowing the storage plugin to determine what to do in that scenario. If it's a publish request and the path doesn't exist, it is an error. If it's an unpublish request and the path doesn't exist, it's clearly not published there so the storage plugin can also check if it needs to remove it's private mount, etc...  So yeah, your proposal sounds fine.

I suggest we introduce a new property, rexray.csi.mount.path, with a default value of /var/lib/rexray/csi/volumes

This makes sense to me. We will need to update all the docker plugins to set this as a propogated mount path as well. Seems like a clean solution to me. I didn't see this problem coming, that's for sure. In a way, we would now be using /var/lib/rexray/volumes as our private mount area, which is very consistent with what we've done in other drivers. 👍

Re #2 these plugins would also need an updated propagated path to Docker.

I would also be concerned with the root path parameter and ensure it still targeted /var/lib/rexray/volumes/MyVolume/data and /data.

I am unclear on what both of you are referring to regarding propagating a path to Docker. The path is returned by our Docker endpoint. It's fine for Docker to consume volumes that exist in different areas on the filesystem. Old Docker-created volumes will be from our private area. New volumes will be created in the new area. Not sure what you two are seeing here...

@akutz https://github.com/codedellemc/rexray/blob/master/.docker/plugins/cinder/config.json#L181

Hi @clintkitson,
Thank you for the clarification. FWIW, your comment about /data is inconsequential because a volume mounted at /var/lib/rexray/csi/volumes/VOLUME_NAME would actually be bind mounted to /var/lib/rexray/volumes/VOLUME_NAME/data.

Totally get that, but just making sure that the /data is still configurable after change, sounds like you have that covered.

Hi @codenrhoden,

It was considered an error because the CO shouldn't ever be sending us a targetPath that does not exist

That makes sense. I only considered the function in regards to Unpub. So yeah, we can make that distinction. Good call.

Hi @clintkitson,

Totally get that, but just making sure that the /data is still configurable after change, sounds like you have that covered.

Covered isn't fair since I'm kind of not doing anything. Just an extra bind mount. :) But yeah, it will work.

Hi @codenrhoden,

In a way, we would now be using /var/lib/rexray/volumes as our private mount area, which is very consistent with what we've done in other drivers.

That's exactly what we're going to be doing. Before we used it for both the private mount area and the Docker volume area. Except now since Docker is subject to the private mount area, they can no longer be the same.

@akutz I don't see anything in here that jumps out at me -- looks like you've covered all the bases, but only testing will find out!


Inheritance in Config Server in "native" profile with yml not working with Angel.SR3

Hello,
i use the Inheritance feature in the config server for yml config but since Angel.SR3 is not working anymore.
my setup:
application.yml
test:
  url: 123

customserver:
  host: <IP_ADDRESS>

---

spring:
  profiles: dev

customserver:
  host: <IP_ADDRESS>

app.yml (no content)

When i start the config server and call http://localhost:8888/app/dev i just get the server.host and not the "test.url" from application.yml.
if i change the version back to 1.0.0.RELEASE, it works correctly.
Thanks in advance
Marcel
P.s: i create an example project under https://github.com/marcelalburg/SpringConfigServerAngelInheritanceBug
after starting, you can access http://localhost:8888/app/dev and there is no "test.url"

Those files are config files for the config server itself. So you are running in the "native" profile, and then the server builds a new Environment from those files for each request. The meaning of "native" might changed a bit since 1.0.0 because a bug was fixed which meant that the server was sending its own configuration (i.e. the one from "application.yml" in your sample). The native profile isn't very flexible, but I believe the behaviour you observe is correct. You would be much better off with a local git repository and the default profile.

Mhh, this means, there is no possibility to have a "global" config (with profiles) for all applications and some special properties for exact one app ?
i mean, it's a real usecase. two application use the same database maybe - in this "new case" i've to duplicate the properties ...
Thanks
Marcel

You can do it with spring.cloud.config.server.native.searchLocations (setting it to a value that includes your two files but isn't in the default path for Spring Boot). The native profile is really only for testing though, so I'd use the git repository if I were you.

Thanks, i'll try it. But in my opinion, not everybody likes to put his properties into a git. But i can be false. Thanks
Marcel


NIFI-4345 Added a MongoDB controller service and a lookup service.

Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

@mattyb149 Ok should be good to go now.

@markap14 Updated based on your feedback and added the ability to merge either a single field or the entire Mongo result minus the _id field.

@markap14 Do you think you could take a look at the updates?

@MikeThomsen I have reviewed the updated PR. It looks like all of my concerns are addressed. Thanks for the new iteration!  I'm a +1 as long as @mattyb149 's concerns are all addressed. Thanks!

I took the liberty of changing the Lookup Key property to Lookup Value Field, and updated variables and unit tests and such. +1 LGTM, built and ran unit tests, also tried a flow with LookupRecord with various settings (Lookup Value Field set and not set, Insert Entire Record and Insert Record Fields, etc.).  Thank you for the feature! Merging (after rebasing with my addition) to master


[Feature][core-base] Translate seatunnel-core-base scala code to java

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description
Translate seatunnel-core-base scala code to java.
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

thanks @yx91490


Extend personal-best dive try for magnification 10^500

See also #19 and the methods of #100

I think you could go directly to values of e-1000 for deep zoom, since the calculation time does only scale (significantly) with the number of iterations but not with the zoomlevel any more. I suggest to perform small images like 64 by 64 to get a feeling for the iterations. And the precision of the central point.
at zomlevels higher then e-1000 the central point of the image (20_zoom_very_deep) needs to be refined, I guess

go directly to values of e-1000 for deep zoom,

Indeed, but I want two intermediate pics for the gallery, one around $10^{300}$ and a secont around $10^{500}$. Then I'll go for a cool pic around $10^{1000}$.

@S-Streulicht then we need to find new iteration points... since I've got a couple around $10^{100}$, but I'm experiencing difficulty getting thousand-order points of interest.

@S-Streulicht then we need to find new iteration points... since I've got a couple around $10^{100}$, but I'm experiencing difficulty getting thousand-order points of interest.

Hm, Finding points there could be 3 trivial ones on the real axis:
The very left one atthe tip of the mandelbrot and then veryright one at the butt I dident test it but the exact point should be easyto find in an iterative way pic twp oint on the real axis onel outode mandelbrot one inside than use a binary search to narrow the point.
The third one 9s abit more tricky but I expect there is tonth of stuf to be found in the net: at some point the mandelbrot isself similar to itself no mater the deep you zoom it is always lpoking the same If i remember it is not realy one point but it changes with the zoomlevel

Finding points

Hi @S-Streulicht I'm working on a visual semi-automated discovery program. When the GPU accelration comes in, this should allow for point-and-calculate that should work relatively quickly.

Hi @S-Streulicht here is a very, very, very preliminary pic of MandelbrotDiscovery in action, as I zoom into a point having 60 digits of magnification.

Hi @S-Streulicht I ended up going outsinde for a bike ride and hit the tavern then this is the point I ended up with on that particular run. Cool find. Although I was hoping for a hundred orders.

Finding points

Hi @S-Streulicht I'm working on a visual semi-automated discovery program. When the GPU accelration comes in, this should allow for point-and-calculate that should work relatively quickly.

@ckormanyos can you define "interesting point"?
yust a point which can be zoomed in very deep, or does it need to have some picture feature?
If it can be a random point yust next to the mandelbrotset, like the üictures of your galerie we have some interesting feature set:

The center should have the higest iteration
zooming in deper and deeper you will find a mini mandelbrot set.

Finding the point of interest:
I don't think you need the full picture to detern the center for the next zoom level, I guees it is just fine to calculate the the pixel on axist (vert and hor) in near the center you will find a platoo (maximum iterations alowed) For two 1D curves the start and the end of the platoo is easy to find. -> center of platoo is the next center of the investigation. Next step increase the iteration and calculate the pixel on axis again for just the region of the platoo. Repeed until the platoo dosen't shrink any more. At that point we have two options: First the number of iterations is just not high enough or we have a mandelbrot

For two 1D curves the start and the end of the platoo is easy to find. -> center of platoo is the next center of the investigation.

Hi @S-Streulicht so far I've only been able to find points randomly. I would like to calculate or predict them. But I've not figured out how to do that. I've heard about some orbit calculations but never pin-pointed them.

@S-Streulicht yes, indeed, we have collected a few.  I can find fluffy around order $10^30$ with any given random search. Also, at times I find some stark harsh B/W pics with magnifications as high as I want to go in order.
But I'm experiencing difficultiies finding interesting pics with hundreds of orders that offer fluffy details. I do not know what actual equations govern such orbits.
I don't know if this makes any sense?

can you define "interesting point"?

Hi @S-Streulicht that is very subjective and I've changed my views as I dive. I seek points having many details, not just stark contr4asts, yet require a bit-float to obtain.
This recent pic I got was great. See pic below. I would have liked a hundred orders, but that does not really matter. Then I'd like some pics that are not exactly symmetrical, but have some kind of curl too.


MODE-2553 Adds sequencer for several video formats

Initial implementation of video sequencer based on humble-video library.

see this comment on things that are still required for this to be included in the ModeShape distribution.

any status update on this ? ModeShape 4.6.Final will be released next week and most likely will be the last 4.x release which will include new features.
Everything else will get moved to ModeShape 5.

@RobSis: are there any plans of updating this in the near future ? We're evaluating the state of our open enhancement requests so we need to know whether to consider or close this

Hello. Yes!
Didn't have a lot of time, but I'm working on it and it should be ready in next week.
Thank you for notice!

Since 4.x branch doesn't accept new features, i'm closing this pull request and create a new one against the master.


TensorFlow 2.4 package

As of today, (2021-02-02), there is no build / package for TensorFlow 2.4. The most recent minor version of TensorFlow that is supported / built is 2.3. Do you have any plans of building a version for 2.4?
How hard would it to contribute that, if you don't have time?
Final question :laughing:, why do you have to maintain separate packages? I couldn't find anything in the readme.

We need to maintain separate packages because tf upgrades its proto versions in different tf versions. We have to upgrade proto version to be compatible with tensorflow. I recently found this rename_proto.sh which might help us to avoid this incompatibility.
it is usually not hard to add a new version unless tensorflow does anything very different in a new version in terms using proto. what is needed  is to add a file like this.
Your contribution is welcome.

ok cool. Thanks for the quick reply!

Done :+1:, can you have a look?


refactor(MarkSnapshot*): don't return the same pointer

MarkSnapshot* functiond were returning the same pointer as they received in snapshot paramter. There is no need to return pointer if we pass pointer as paramereter.
Given the pointer was returned it could puzzle devs to expect that it's a newly allocated memory instead. This old-new pointer was even used inconsistently in code, it's just safer to remove it and make clear that original pointer is being used for data updates.
Maintainers will complete the following section

[ ] Commit messages are descriptive enough (hints)
[ ] Code coverage from testing does not decrease and new code is covered (check the PR coverage on codecov)
[ ] Controllers diagrams are updated when PR changes controllers code  (if applicable)

/retest


[Bug Report] Display error when el-input using slot="append(or prepend)" in Chrome

Element UI version
2.2.2
OS/Browsers version
Windows 7 x64 / Chrome 65.0.3325.181
Vue version
2.5.16
Reproduction Link
https://jsfiddle.net/liuyuan/2m53badL/
Steps to reproduce
See reproduction link.
What is Expected?
An el-input should always be centered with the label of the outer el-form-item component.
What is actually happening?
When the el-input component is in the el-form-item component, and the el-form-item component has the label attribute, and the el-input component using slot="append(or prepend)". The label of the el-form-item component and the el-input component will not be horizontally centered in Google Chrome, but it's well in IE11.

Fixed in #10189


Load pending and expired schedules

Pending schedules are loaded once, a few minutes after startup.
Poll every few minutes to pick up expired schedules.

This PR was successfully deployed to production by CircleCI build 119 at 2017-02-13 14:46 GMT.


Add content for .NET Standard 2.1

We should have the first batch of content in place by the time .NET Core 3.0 Preview 1 is released. Right?
/cc @terrajobst

There isn’t going to be a release of .NET Standard 2.1 in preview 1 — all bits are going out first time in Preview 2.

Moving this to the backlog since we don't have what to document at this moment.