time_new_pos_embed

Hi Khaled,
I am playing with your code a bit and I struggle to understand these few lines below:
        # Adding Time/Freq information
        if first_RUN: print(" self.time_new_pos_embed.shape", self.time_new_pos_embed.shape)
        time_new_pos_embed = self.time_new_pos_embed
        if x.shape[-1] < time_new_pos_embed.shape[-1]:
            if self.training:
                toffset = torch.randint(1 + time_new_pos_embed.shape[-1] - x.shape[-1], (1,)).item()
                if first_RUN: print(f" CUT with randomoffset={toffset} time_new_pos_embed.shape",
                                    time_new_pos_embed.shape)
                time_new_pos_embed = time_new_pos_embed[:, :, :, toffset:toffset + x.shape[-1]]
            else:
                time_new_pos_embed = time_new_pos_embed[:, :, :, :x.shape[-1]]
            if first_RUN: print(" CUT time_new_pos_embed.shape", time_new_pos_embed.shape)
        else:
            warnings.warn(
                f"the patches shape:{x.shape} are larger than the expected time encodings {time_new_pos_embed.shape}, x will be cut")
            x = x[:, :, :, :time_new_pos_embed.shape[-1]]
        x = x + time_new_pos_embed

Especially the slicing of time_new_pos_embed with toffset. I understand the slicing in the first else and the second else but I don't get why the slicing is randomized at training. If it's a position embedding surely it shouldn't be random right?
Many thanks in advance.
Antoine

Hi Antoine,
You're right, the randomized slicing during training works by taking a substring from the time position embedding in order to learn time position embeddings for a longer clips. For example, Here the models passt-s-f128-20sec-p16-s10-ap.474-swa.pt and
passt-s-f128-30sec-p16-s10-ap.473-swa.pt can accept as input audio clips of 20 seconds or 30 seconds, while being trained only on 10-second clips of audioset.

Thank you for your swift reply!
Hmm... not sure I get it!
What I understood is that you have different models that can take clips in up to different lengths each (10s, 20s, 30s). Their input size varies accordingly (128x998, 128x2000, 128x3000, ...).
If I build a model with the configuration associated to passt-s-f128-20sec-p16-s10-ap.474-swa.pt, do we agree that I will only be able to fine-tune or infer on clips that are AS or LESS long than 20sec (but not more)?
In the first else, you do:
time_new_pos_embed = time_new_pos_embed[:, :, :, :x.shape[-1]]
which makes sense to me as if I pass a clip that is 10sec for example (while working with the model that's able to process up to 20sec clips) so half the size of what the model was trained on, I want to associate the time position embeddings from 0 to 10sec worth of patches.
In the second else, you do:
x = x[:, :, :, :time_new_pos_embed.shape[-1]]
which is to handle the case where the input clip is longer than what the model was trained on. So it makes sense here to trim x up to time_new_pos_embed.shape[-1]  as x is longer.
What I struggle to understand is the use of randomization at training time.
toffset = torch.randint(1 + time_new_pos_embed.shape[-1] - x.shape[-1], (1,)).item()
time_new_pos_embed = time_new_pos_embed[:, :, :, toffset:toffset + x.shape[-1]]

why are you using a random offset here? Shouldn't it work like time_new_pos_embed = time_new_pos_embed[:, :, :, :x.shape[-1]] ? I would expect to always pass embeddings starting from 0. It seems here that you could associate embeddings related to later patches, to earlier patches. Or doesn't it work like this?
Let's say we have x as :
x1 x2 x3 x4 x5
And our model is able to take in up to 10 patches:
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
Our time_new_pos_embed is initialized as:
e1 e2 e3 e4 e5 e6 e7 e8 e9 e10
Here I would associate like this:
x1+e1 x2+e2 x3+e3 x4+e4 x5+e5
But the code seems to suggest that during training, this can also happen, on a random basis:
x1+e4 x2+e5 x3+e6 x4+e7 x5+e8
or
x1+e2 x2+e3 x3+e4 x4+e5 x5+e6
etc...
Obviously the above illustration doesn't reflect the right tensors dimensions but I tried to lay down my thinking as best as I could.
Thanks a lot again in advance for your help.
Antoine

Hi Antoine,
I think you're completely right. The unclear part I believe - is the
randomization. Let's assume that we have training data compromised only of
10-second audio clips , but we need a model to give the predictions on
20-second clips.  During training, If we cut the trainable time position
encoding by taking only those correspond to the first 10 seconds, then the
remaining time pos encoding wouldn't be trained, and the model cannot
predict on longer audio. The simple approach implemented here is to sample
a substring of the time encoding always corresponds to 10-seocnds ( to
match the actual training audio length). However, the start of this
10-second encodings is randomly chosen within the the 20-second encodings.
So  during training the model always sees only 10-second audio and
positional encoding. But these encodings randomly shift within all the
possible encodings. Let me know if this makes sense.
There are probably better and more suitable ways to accomplish this, but I
chose this simple sampling method.
Best,
Khaled
On Thu, Mar 7, 2024, 09:45 Antoine101 @.***> wrote:

Thank you for your swift reply!
Hmm... not sure I get it!
What I understood is that you have different models that can take clips in
up to different lengths each (10s, 20s, 30s). Their input size varies
accordingly (128x998, 128x2000, 128x3000, ...).
If I build a model with the configuration associated to
passt-s-f128-20sec-p16-s10-ap.474-swa.pt
https://github.com/kkoutini/PaSST/releases/download/v0.0.5/passt-s-f128-20sec-p16-s10-ap.474-swa.pt,
do we agree that I will only be able to fine-tune or infer on clips that
are AS or LESS long than 20sec (but not more)?
In the first else, you do:
time_new_pos_embed = time_new_pos_embed[:, :, :, :x.shape[-1]]
which makes sense to me as if I pass a clip that is 10sec for example
(while working with the model that's able to process up to 20sec clips) so
half the size of what the model was trained on, I want to associate the
time position embeddings from 0 to 10sec worth of patches.
In the second else, you do:
x = x[:, :, :, :time_new_pos_embed.shape[-1]]
which is to handle the case where the input clip is longer than what the
model was trained on. So it makes sense here to trim x up to time_new_pos_embed.shape[-1]
as x is longer.
What I struggle to understand is the use of randomization at training time.
toffset = torch.randint(1 + time_new_pos_embed.shape[-1] - x.shape[-1], (1,)).item()
time_new_pos_embed = time_new_pos_embed[:, :, :, toffset:toffset + x.shape[-1]]
why are you using a random offset here? Shouldn't it work like time_new_pos_embed
= time_new_pos_embed[:, :, :, :x.shape[-1]] ? I would expect to always
pass embeddings starting from 0. It seems here that you could associate
embeddings related to later patches, to earlier patches. Or doesn't it work
like this?
Let's say we have x as :
x1 x2 x3 x4 x5
And our model is able to take in up to 10 patches:
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
Our time_new_pos_embed is initialized as:
e1 e2 e3 e4 e5 e6 e7 e8 e9 e10
Here I would associate like this:
x1+e1 x2+e2 x3+e3 x4+e4 x5+e5
But the code seems to suggest that during training, this can also happen,
on a random basis:
x1+e4 x2+e5 x3+e6 x4+e7 x5+e8
or
x1+e2 x2+e3 x3+e4 x4+e5 x5+e6
etc...
Obviously the above illustration doesn't reflect the right tensors
dimensions but I tried to lay down my thinking as best as I could.
Thanks a lot again in advance for your help.
Antoine
—
Reply to this email directly, view it on GitHub
https://github.com/kkoutini/PaSST/issues/42#issuecomment-1982975796, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AML2GDW6T7S2YAMKNJSFP6LYXASLPAVCNFSM6AAAAABEJC45P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBSHE3TKNZZGY
.
You are receiving this because you commented.Message ID:
@.***>

Ok I think I understand your logic and why you chose to do this!
I thought that when you said your models accept inference on clips up to 20s or 30s, it meant they were respectively trained on strict 20s or 30s clips. But you are saying that those models were trained on variable length clips below these limits.
Regardless, It is a bit counterintuitive for me to randomize the time position encodings as I would tend to think that if you associate an similar index encoding to a different patch time randomly each time during training, it is not gonna be able to learn any positioning relationships. Or is it?
For me, e1 should always be associated to x1, e2 to x2, and so on and so forth.
It may not be a problem for stationnary sounds for which mel-spectrograms will be similar from 0 to 10s and 10 to 20s for example but what about acoustic signatures like a plane taking-off where you'll see a distinctive evolution of harmonics through the 20s (meaning the 10 last seconds are complementary to the first 10 seconds to classify this sound)?
I am trying to think about cases where this approach may prove problematic. Sorry if it's a bit fuzzy...
Have you tried with AND without randomization?
The results you mentioned in your paper are really good so I guess it must work as is.
Cheers
Antoine

Regardless, It is a bit counterintuitive for me to randomize the time position encodings as I would tend to think that if you associate an similar index encoding to a different patch time randomly each time during training, it is not gonna be able to learn any positioning relationships. Or is it?

The encoding are always for 10-consecutive seconds, corresponding to 10 seconds of audio. Of course, you are right it won't be as good as if you traine on 20 second input. But giving the limitation of having only 10-second training, you can this way train each crop of 10-second encodings to represent relative position. Keep in mind that audioset 10-second clips are often clipped from longer audios.

Have you tried with AND without randomization?
I did not try without randomization, because the remaing encodings (e11-e20) would not be learned during training

Yeah sorry I thought about it multiple times and you're right.
So the 10-seconds encoding is our fixed context, so to say. So we would not perform that well for sound events that are longer than this, or that need to be longer than 10s to be "recognized". But that would likely never be the case, as only a few seconds are sufficient in most cases (ex: a few dog barks).
And training all encodings randomly make sense to have a model capable of infering on longer audio.
Thank you for your reply, as always!


Custom Animations with Jetpack Compose in Android

Proposed title of article
Custom Animations with Jetpack Compose in Android
Introduction paragraph (2-3 paragraphs):
Jetpack Compose is a modern framework for creating native UI with composable functions on Android.
It makes Android UI development easier and faster.
 Composable functions make it easier to reuse code.
In order to achieve a seamless and clear user experience in a modern mobile app, animations are required. Many Jetpack Compose Animation APIs are composable functions, similar to layouts and other UI elements, and are powered by powerful and extensible APIs.
Key takeaways:
By the end of this tutorial, the reader will be able to:

Understand what Jetpack Compose Animations are
Use default Animations provided by Compose APIs
Differentiate between high-level and low-level Animation APIs
Implement custom Animations

References:
N/A

Good afternoon and thank you for submitting your topic suggestion. @Janenjoki
Your topic form has been entered into our queue and should be reviewed (for approval) as soon as a content moderator is finished reviewing the ones in the queue before it.

Hi @Janenjoki
This sounds like a really helpful article. I however think that a lot of the information you intend to cover has already been covered on the official docs. Please ensure you provide more original content so as to avoid the closing of articles in the future.
topic approved :rocket:

To help avoid a backlog queue (and longer review times) we are currently only accepting ONE article TOPIC/PR approved at a time 👍. We can re-open this topic - once the 1st one has been merged/published.


NOTICE: Detuning locks due to high load per core

Describe the bug
We're seeing numerous log entries related to "Detuning locks".
1:12:S 25 Nov 2020 04:47:25.563 # NOTICE: Detuning locks due to high load per core: 97.49%
1:12:S 25 Nov 2020 04:47:55.668 # NOTICE: CPU pressure reduced

These logs are being printed when running KeyDB under Kubernetes on a moderately busy cluster. The pods have more than sufficient CPU requests.
What are the implications of the locks changing from __ATOMIC_RELEASE to __ATOMIC_RELAXED?

The acquire/release semantics aren’t important for the detune. The meaning of this message is that KeyDB is attempting to gracefully handle a CPU overcommit situation.
KeyDB uses spinlocks for fast synchronization. When CPU load is above 100% per core the kernel may unwittingly give CPU time to a blocked thread that is spin waiting instead of the active thread. This results in much higher latency.
Instead when this situation is detected KeyDB will spin for a shorter time before putting a thread to sleep. This results in better throughput when CPU time is limited. However the “proper” fix is to ensure CPU time is not so constrained.
If you aren’t on the latest version I’d recommend updating as we’re much more efficient with CPU usage in the latest version. This may solve your CPU load issues without upgrading hardware.

Does this mean that when there is perceived CPU pressure KeyDB is less consistent when running in multi-master?

No this does not affect consistency - just performance. If your clients are getting acceptable latency then this log may be ignored. If not then the correct fix is to reduce cpu load on the system.


Make admin password label actually show up.

I swear I fixed this before, but maybe it was a different element.

:+1:


Issue where after repairing/extracting materia/using consumables HQ mats aren't selected to continue craft

I noticed while crafting where a supply of 1 NQ mat and 1 HQ mat whenever stopping to extract materia, use a consumable, or repair gear the HQ mat wouldn't be automatically selected/applied. It had to be manually selected and then crafting resumed as normal until the next instance of materia/consumables/repairs happened.

Can confirm this is still happening


styling problem with allDay event header?

Do you want to request a feature or report a bug?
bug?
What's the current behavior?
Looks like maybe there's a styling problem in .rbc-allday-cell? For some reason, although .rbc-allday-cell is styled with width: 100%, it is larger than the remaining space in the rbc-row - the gutter is correctly fixed width at the left side, but the remaining portion of the rbc-row seems to be too big (see below - red box added for illustration). This causes busy views to overflow and the header columns to become unaligned vertically...
This is not a problem with the row immediately above it (the one with the day/date headers) - the only difference in the markup between these rows is the insertion of the .rbc-allday-cell and the .rbc-row-bg and .rbc-row-content divs, so I presume there's a (minor?) css problem there somewhere?
I tried to figure this out but became stumped (I'm not great at css!) - I'm happy to work on this but I need some advice on where to invest more time. Seems like it is some odd interaction of flex and the intervening divs below rbc-row?

What's the expected behavior?
I'd expect the allday-cell columns to line up exactly with the rest of the grid and not overflow.

I'm not 100% sure this is the right fix, setting overflow: hidden on the .rbc-allday-cell fixes my problem. Happy to do a PR if desired. Feel free to close.

@ehahn9 i'd make sure it's not still present with the diff from #733 there is a bunch of small styling things there

There looks to be a fix to this issue so closing this issue for now.


Analyze Rollbar logs [3h]

We should analyze any existing errors in the logs.

Go over Rollbar and confirm the connection is still here

Analyze errors and open related issues. If an issue is not clear or actionable, escalate.

//cc @adamhstewart

Suspected error:

Problem at Prenatal stats data, Nameba HC:

This is probably happening due to duplicate MUAC measurement.
Such duplicates are deleted during night.
Synced Nameba HC on my device, and stats work well.
No problem here.


Multiple files CLI

lint-staged is very useful with svgo (it will send to svgo` any new or changed SVG).
But lint-staged pass all file names in single command TOOL FILE1 FILE2 FILE3.
svgo falls in this case. Also when you change only two files lint-staged called it with svgo one.svg two.svg which even broke uncommit files :(.
@GreLI do we have any solution for this? If not. just give me CLI API — I will send PR.

Actually, I'm working on it right now.

JFYI, for now current CLI API is
svgo <input-file> <output-file>

(and I want to change that.)

Hi, @GreLI!
Any progress on this issue? lint-staged and svgo are a great couple, but it's annoying to commit not more than one SVG file per commit.
Thanks!

I took a stab at this in #786, people are reaching for the same things for pre-commit

Finally, It has been done it in e3ddf1d96ab88beb9f496a122dee6ebcb4a0c907.


Seed tests

Closes #865
I think it was too early to write these tests as many issues exist and the usecase is not done yet.
Therefore I will mark this as draft and wait before finishing the tests.
Basics

[x] I added a line to /doc/CHANGELOG.md
[x] The PR is rebased with current master.

[x] References to issues, e.g. close #X, are in the commit messages.
[ ] The buildserver is happy.

Checklist

[ ] I have installed and I am using pre-commit hooks
[ ] I fully described what my PR does in the documentation
(not in the PR description)
[ ] I fixed all affected documentation
[ ] I fixed the introduction tour
[ ] I wrote migrations in a way that they are compatible with already present data
[ ] I fixed all affected decisions
[ ] I added unit tests for my code
[ ] I added code comments, logging, and assertions as appropriate
[ ] I translated all strings visible to the user
[ ] I mentioned every code or binary not directly written or done by me in reuse syntax
[ ] I created left-over issues for things that are still to be done
[ ] Code is conforming to our Architecture
[ ] Code is conforming to our Guidelines
(exceptions are documented)
[ ] Code is consistent to our Design Decisions

Review

[ ] I've tested the code
[ ] I've read through the whole code
[ ] Examples are well chosen and understandable

I think it was too early to write these tests as many issues exist and the usecase is not done yet.
Therefore I will mark this as draft and wait before finishing the tests.

#836 is now merged, so it is ready to be tested now. Pls rebase :rocket:

Seed delete doesnt exist.
Seed edit doesnt exist.
Validation is unfinished.

It is not possible to continue efficiently for now.

Seed delete doesnt exist.
Seed edit doesnt exist.

Do you mean the locators? The UI elements are there and work for me:

The only weirdness is that for "delete" it seems to somewhat refresh the whole UI and you get greeted again. If this is a problem for e2e tests, please create a release-critical issue. Otherwise I think it is only minor annoyance. I added it to #872

Validation is unfinished.

Yes, this is tracked in #868

I am sorry, somehow I didnt see the latest version. Now it works.

@badnames can you please help out with these 4 locators?

Similar https://github.com/testing-library/react-testing-library/issues/377

Its the best if I finish this after #872 is done and maybe the search is also enabled. I think there is still some open questions that will change how the tests will look, which I have asked in https://github.com/ElektraInitiative/PermaplanT/issues/872#issuecomment-1723288292

I think it is best to get this merged asap, with what currently works, so that it is also the responsibility of the people who change something to look for regressions.
When the search gets added, we add more tests.

rebased on #973, hence the messy commits and changed files

I removed two timeline tests because they were broken in some other PR.

Is this to be merged before #1063?

I dont know. I dont think it interferes with it.


Add Firebase Component + Account Management

Add Firebase Component + Account Management
:recycle: Current situation & Problem
The current PAWS application does not include a user login functionality and does not connect to a cloud storage provider.
:bulb: Proposed solution
This PR adds a Firebase-based login & account sign up mechanism and the Firestone component to automatically upload observations and other FHIR resources to Firebase.
Related PRs

https://github.com/StanfordBDHG/CardinalKitTemplateApplication/pull/8

Testing

Adapts the UI test accordingly.

Code of Conduct & Contributing Guidelines
By submitting creating this pull request, you agree to follow our Code of Conduct and Contributing Guidelines:

[x] I agree to follow the Code of Conduct and Contributing Guidelines.

@CS342/2023-paws A short reminder that this PR is still open 👋


Add fmt test to make sure formatting works for all parser test cases

This also makes (most of) the requisite changes so this test passes:

Add formatting for tuple literals, skipped in previous PRs (related to #4465)
Fix a case where we added a literal space instead of calling buf.spaces(1), and triggered a debug_assert - important for making sure the formatter never adds trailing whitespace
Changes the formatter to just output the text in the case of MalformedIdent, to preserve the input better
Fixes multiline strings to always start on their own line, making indentation choices easier for the formatter (alternatively, we could track text columns in the formatter - but that gets a little more complicated)
Temporarily, because I'm lazy, don't enforce the formatting-stability requirement on these tests - i.e. don't require that fmt(input)==fmt(fmt(input)) ... yet.

The current failure looks like another CI flake. 😞

or by dumping the formatted *.expr.roc contents to files on disk.

Done!
FWIW, there aren't currently any tests for multiline tuples or tuples with comments - but I don't expect there to be bugs here because the formatting impl shares code with the formatting of lists, and we have plenty of tests for that.

I eyeballed all the formatting differences versus what's currently in the parse test snapshots, and they all look as expected, with the exception that in some cases we're missing parens around patterns. This is expected, since we don't have a Pattern::ParensAround currently. The test asserts that they all parse to the same AST, so we have significant protections here.

I went ahead and added a parse test (and therefore fmt test, now!), for a multiline tuple.


Migrate to react-native-test-app

Summary

Test Plan

What's required for testing (prerequisites)?
What are the steps to reproduce (after prerequisites)?
Compatibility

OS
Implemented

iOS
✅❌

Android
✅❌

Checklist

[ ] I have tested this on a device and a simulator
[ ] I added the documentation in README.md
[ ] I updated the typed files (TS and Flow)

Thank you for this PR 🎉
In the bob library, there is a discussion about it here: https://github.com/callstack/react-native-builder-bob/issues/272 . Right now, I would like to follow the official pattern to make our upgrades more easier.


Determine the maximum number of coin pouches based on Ardounge diary completion

Coin pouches can stack higher if the Medium/Hard/Elite Ardounge diaries have been completed.

@Diabolickal would you be able to take a look at this PR?


STORM-1078: Updated RateTracker to be thread safe

, more accurate, and very fast. Removed sub-sampling of rate calculation in disruptor, because sub-sampling was more expensive than just doing the calculation.

+1
It's easier to understand. Great work.

+1


Update PeerDependencies of React

The React versions should be updated in peerDependencies.

We cannot update to React 15.4.2 as LiterallyCanvas specifies 15.1.

I don't plan to do this upgrade myself. Should be easy for anyone to
submit a patch. I will test and merge.
On Wed, Jan 11, 2017, at 06:35 AM, Mark Boyall wrote:

We cannot update to React 15.4.2 as LiterallyCanvas specifies 15.1.
— You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub[1], or mute the
thread[2].

Links:

https://github.com/literallycanvas/literallycanvas/issues/427#issuecomment-271869811
https://github.com/notifications/unsubscribe-auth/AACclQUyNBhf17Wmvyk54kGTTz4pbRuvks5rRNq_gaJpZM4JeVz3


merge package from arlac77/npm-package-template

package.json

chore(engines<EMAIL_ADDRESS>

:tada: This PR is included in version 3.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


HERE: API updates - appId and appCode not supported anymore

Due to recent updates, appId and appCode are not supported anymore (for new Here projects, old projects still work for now, but will be depreciated).
We need to make a standard fetch with url and token for example:
https://geocode.search.hereapi.com/v1/geocode?q=<YOUR_SEARCH>&apiKey=<YOUR_API_KEY>

Thanks to report the issue going to fix this soon

Duplicate of #286

Resolved by #302


Added support for baked collision checking

There was a logic error before where stop() was being called when returning from InitPlan. This reset() the baked KinBody pointer, which caused a SEGFAULT in Plan(). I don't see why stop() is necessary at all, so I removed it entirely.
@cdellin: What was the motivation behind adding stop()? Is it safe to proceed without it?

The pointer to the baked kinbody must be removed before the planner returns, since it owns raw pointers into internal collision checker datastructures which are not guaranteed to persist past the end of the planning call (from what I remember).  That was the point of the boost scoped exit logic.  I don't understand why resetting an empty smart pointer would cause a SEGFAULT, so I'm not convinced the proposed change actually solves the problem, and I'm afraid removing the stop method introduces other bugs as well. If this is working, I'm ok with keeping it temporarily, but let's definitely not merge until we take a closer look. I will have some cycles tonight.

A "planning call" consists of InitPlan followed Plan. The old logic deleted the baked KinBody when InitPlan returns, then attempted to use the KinBody in Plan to perform collision checks during planning. This SEGFAULTs because the baked KinBody no longer exists.
I don't understand how this could create memory issues because (as far as I can tell) we don't allocate anything that needs cleanup. If cleanup is necessary, than the baking implementation in PrPy is also broken.

Then the problem is that the old code didn't correctly call start() in PlanPath. The PrPy implementation relies on the baking object going out of scope in Python, and the baked kinbody therefore getting destructed.

For another point of comparison, https://github.com/personalrobotics/comps/pull/9 only reset() the pointers: (1) if an error occurs in InitPath or (2) after planning finishes in PlanPath. This is better because it doesn't SEGFAULT in normal operation, but is still dangerous because of https://github.com/personalrobotics/comps/issues/10.

Then the problem is that the old code didn't correctly call start() in PlanPath.

If this is truly necessary, then I am fine with restoring stop() and moving both start() and stop() to PlanPath.

The PrPy implementation relies on the baking object going out of scope in Python, and the baked kinbody therefore getting destructed.

This Python garbage collector is deterministic and may not destruct the KinBody when it leaves scope. If this is indeed necessary, then we must explicitly del the object. It is probably best to do this using a  context manager or an explicit try-finally block.

I still don't understand why all of this is necessary. What data structure could possibly go out of scope when the planner returns? We don't allocate any data structures in the planner!

Yeah, looks like the start() call in PlanPath was accidentally removed in a merge in the past couple weeks. Let me try to narrow down when the offending merge was. (See https://github.com/personalrobotics/or_ompl/blob/00e53f3aaf5a1d24b3ff664daf3c5ba798d18973/src/OMPLPlanner.cpp#L348 for the original code.)

The planner constructs a baked kinbody object which internally points into the collision checker. If after the planner returns, the collision checker is removed from the environment and destructed, then that kinbody must no longer be used (and actually, the non-destructed kinbody may prevent the collision checker from being destroyed properly).
I agree that we should use a context manager to handle baking in PrPy to ensure that thr kinbody outlives the collision checker.

I also can see that the comps code is incorrect -- I can fix it when.I get in front of a laptop again (-:.

I looked at the implementation of BakedKinBody and don't see the issue. It does store raw BroadPhaseCollisionManager *, but it also stores a BroadPhaseCollisionManagerPtr smart pointer to each of those objects. Everything else uses smart pointers.

If after the planner returns, the collision checker is removed from the environment and destructed, then that kinbody must no longer be used (and actually, the non-destructed kinbody may prevent the collision checker from being destroyed properly).

Even if I am wrong about the ownership semantics, I think you are overstating the issue. This is only a problem if the KinBody outlives the collision checker. My changes tie the lifetime of the baked KinBody to the lifetime of the Planner. Two potential fixes:

Modify the baked KinBody hold a shared_ptr to the collision checker.
Modify the Planner to hold a shared_ptr to the collision checker.

Both of those changes guarantee that the KinBody will be destructed before the collision checker without introducing this (obviously very error-prone) manual resource management.

I agree your proposed fixes will address the immediate issue,  but I really don't like the way they assert ownership over the collision checker.  This will create unexpected behavior when the user attempts to destroy the collision checker,  but it is not destructed because some random planner still holds a pointer that it's not even using.  I will put together a better solution to this tonight.

Yup, bug was introduced by a merge I made on July 21 (https://github.com/personalrobotics/or_ompl/commit/d99d6e779a9e671681f5bb11bb4be394a8c9f0fb).

I just applied the quick fix to address that merge problem from last week, so it certainly shouldn't segfault any longer (I just tested with some log files).  I'll implement better logic later tonight.

@cdellin Any update on that new logic? 😁

No update yest (-: although I expect that it would look like an RAII-style handler that accomplishes the same thing that the BOOST_SCOPE_EXIT stuff does now.

This is sufficiently un-busted that I am merging the current version. I created an issue #103 to track the minor double-baked issue.

I'm happy this got merged (and thanks for making the separate PR for the double-baked issue).  @mkoval I'm still curious about my comment here: https://github.com/personalrobotics/or_ompl/commit/c475b3bfcaa4a6d1c2397622e9986e185722aa3c. What do you think?


Website on production mode doesn't work while re-building

Hi I just registered in github to write this issue...
I love nuxt js so much and its front page hot reloading save me some time but.. it wasted me so much time while compiling backend js file... it's usually 10~20 seconds... Is it normal?
And after I build and start production mode, if I npm run dev for debugging on another port, the website does not work properly and it says some files are missing (I installed nuxt.js font awesome module and after npm run dev all font awesome icons becomes tofu. And bootstrap js will not work)
Of course I can debug on my PC locally so it will not be so big problem but...
While the website is on the production mode, if I make a change and enter npm run build, same problem occurs while it is building. Though problem will not exist after building is done, our users will not be able to click anything and see bunch of tofu boxes.
You will not notice because there is cache so just try go to Chrome incognito mode while running npm run build and you will see some error message on developer console.
I installed nuxt v1.0.0 rc3 and updated to rc8. And started with nuxt-express template which use backpack. So I am not sure this is a problem of backpack or nuxt itself.
Thank you for reading.

Hi @lingopalm, thanks for your support.
I'm closing this issue to refer it on the proper repo (express-template).
https://github.com/nuxt-community/express-template/issues/76


Read entire NULL object column

Hi, When I try to read the following dataframe from an s3 bucket:
dataframe = pd.DataFrame({
    "col": [1, 2],
    "col_string_null": [None, None],
    "col_date_null": [None, None],
})

as an Iteratable and as dataframe
import awswrangler as wr
df = wr.s3.read_parquet(path='s3://<S3_BUCKET>/df_null_columns.parquet', boto3_session=boto3.Session())
gen = wr.s3.read_parquet(path='s3://<S3_BUCKET>/df_null_columns.parquet', chunked=True,boto3_session=boto3.Session())
next(gen)

I get the following error for the iterable
packages/awswrangler/_data_types.py", line 188, in pyarrow2athena
    raise exceptions.UndetectedType("We can not infer the data type from an entire null object column")
awswrangler.exceptions.UndetectedType: We can not infer the data type from an entire null object column

Is there a way in which i can get the iterable from a dataframe containing entire NULL object columns?
Thanks!

I think the only way for reading dataframes with entire null columns is to explicitly specify all other columns and therefore ignore the null columns. For your  example this would look like this:
import awswrangler as wr
df = wr.s3.read_parquet(path='s3://<S3_BUCKET>/df_null_columns.parquet', boto3_session=boto3.Session())
gen = wr.s3.read_parquet(path='s3://<S3_BUCKET>/df_null_columns.parquet', chunked=True,boto3_session=boto3.Session(), columns=["col"])
next(gen)
>>   col
  0    1
  1    2

However this means you would need to know beforehand which columns are null which you can do by running:
wr.s3.read_parquet_metadata(path="s3://<S3_BUCKET>/df_null_columns.parquet")
>> ({'col': 'bigint', 'col_string_null': 'null', 'col_date_null': 'null'}, None)

I think the only way for reading dataframes with entire null columns is to explicitly specify all other columns and therefore ignore the null columns. For your  example this would look like this:
import awswrangler as wr
df = wr.s3.read_parquet(path='s3://<S3_BUCKET>/df_null_columns.parquet', boto3_session=boto3.Session())
gen = wr.s3.read_parquet(path='s3://<S3_BUCKET>/df_null_columns.parquet', chunked=True,boto3_session=boto3.Session(), columns=["col"])
next(gen)
>>   col
  0    1
  1    2

However this means you would need to know beforehand which columns are null which you can do by running:
wr.s3.read_parquet_metadata(path="s3://<S3_BUCKET>/df_null_columns.parquet")
>> ({'col': 'bigint', 'col_string_null': 'null', 'col_date_null': 'null'}, None)

Thanks alot for this!

Thanks alot for this!

I can see some space for improvements here. This schema inference that is failing to find out the null type is only necessary if the argument validate_schema=True (non default). So the commit above fix it and will be shipped on version 2.4.0.

I can see some space for improvements here. This schema inference that is failing to find out the null type is only necessary if the argument validate_schema=True (non default). So the commit above fix it and will be shipped on version 2.4.0.


add support to config SuccessfulJobsHistoryLimit (#618)

SuccessfulJobsHistoryLimit will control the number of cronjobs history after execute successfully.
This is very useful when some monitor service like prometheus will be set the alarm if the job status is not allow to be completed.

Thanks for your contribution! It looks great, just need some documentation. It would also be nice to add this new property to a test, even if just to an existing one.
About documenting this new property: could you please either change an existing example to include this option or create a PR against the docs repository, documenting this new property?
https://github.com/jaegertracing/documentation/ , and the content file is this one: content/docs/next-release/operator.md

Thanks for your review, I will add some it to the test and change documentation as your suggestion

@peacecoder do you have news about the tests? Would love to get this one merged!

@peacecoder do you have news about the tests? Would love to get this one merged!

Hi  @jpkrohling , I'am so sorry about late response. I change the doc firstly, please help me to review https://github.com/jaegertracing/documentation/pull/302
I will add some test after works.

@peacecoder do you have news about the tests? Would love to get this one merged!

Hi @jpkrohling ,
I have add some test and documentation about this change, please help me to review it. thanks.
https://github.com/jaegertracing/documentation/pull/302

@peacecoder could you please run make generate and update this PR? The CI job is failing because a model has been changed:
Build failed: a model has been changed but the deep copy functions aren't up to date. Run 'make generate' and update your PR.

Thanks, pass all ci checks.

Thank you for your contribution!

My pleasure, and thank you for helping me review the code.


Sanitize only sanitizes integers not decimals, despite being valid JSON

The following test case fails:
  it 'sanitizes decimals' do
    data = {
      "secret" => -123.432
    }

    result = @processor.process(data)

    expect(result["secret"]).to eq(Raven::Processor::SanitizeData::INT_MASK)
  end

Since a ruby Decimal is a valid JSONNumber the above example should work! This applies to other times of Numeric like BigDecimal.
Similarly booleans are also not sanitized, but this would be harder. Maybe BOOL_MASK = false, or alternatively just use the STRING_MASK.

the above example should work

I mean, it is called Raven::Processor::SanitizeData::INT_MASK for a reason 😆
SanitizeData is only intended to catch credit card numbers. Matching floats and booleans is out-of-scope, you should write your own processor for what you need or match on the key, not the value.

@nateberkopec In that case I would suggest changing the documentation and or name of the sanitize_fields param (maybe sanitize_string_fields and sanitize_integer_fields). Seems artificially restrictive, to only sanitize some types of values but others.

That's reasonable. https://github.com/getsentry/sentry-docs/pull/341 fixes.
We can't sanitize types other than Integer and String because a) it would be extremely slow and b) will inevitably break when people put strange types into the sanitizer.


[Bug] Equality test: failures => '0' is not of type integer

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
When I run an equality test from dbt_utils  on a seed and a model/table I get the error that '0' is not an integer.
It works in version 0.20.2 but not anymore in 0.21.0 or 1.0.0rc1.
Expected Behavior
Running with dbt=0.20.2
Found 64 models, 15 tests, 0 snapshots, 0 analyses, 361 macros, 0 operations, 25 seed files, 77 sources, 0 exposures
13:36:02 | Concurrency: 2 threads (target='unittest')
13:36:02 |
13:36:02 | 1 of 1 START test dbt_utils_equality_testcsv_A__ref_test_............ [RUN]
13:36:03 | 1 of 1 PASS dbt_utils_equality_testcsv_A__ref_test_.................. [PASS in 0.17s]
13:36:03 |
13:36:03 | Finished running 1 test in 0.57s.
Completed successfully
Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
Steps To Reproduce
Create following 3 files in corresponding folders for seeds and models.
testcsv.csv
"A"
1
test.sql
select 1 as A from dual

test.yml
version: 2
seeds:
  - name: testcsv
    columns:
    - name: A
    tests:
      - dbt_utils.equality:
          compare_model: ref('test')
          compare_columns:
            - A

dbt seed
dbt run
dbt test

Relevant log output
Running with dbt=0.21.0
Found 64 models, 15 tests, 0 snapshots, 0 analyses, 376 macros, 0 operations, 25 seed files, 77 sources, 0 exposures

13:31:58 | Concurrency: 2 threads (target='unittest')
13:31:58 |
13:31:58 | 1 of 1 START test dbt_utils_equality_testcsv_A__ref_test_............ [RUN]
Unhandled error while executing test.dib.dbt_utils_equality_testcsv_A__ref_test_.145311c1ec
'0' is not of type 'integer'

Failed validating 'type' in schema['properties']['failures']:
    {'type': 'integer'}

On instance['failures']:
    '0'
13:31:58 | 1 of 1 ERROR dbt_utils_equality_testcsv_A__ref_test_................. [ERROR in 0.15s]
13:31:58 |
13:31:58 | Finished running 1 test in 0.49s.

Completed with 1 error and 0 warnings:

'0' is not of type 'integer'

Failed validating 'type' in schema['properties']['failures']:
    {'type': 'integer'}

On instance['failures']:
    '0'

Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

Environment
- OS: Linux
- Python: 3.8.8
- dbt: 0.21.0 | 1.0.0rc1

What database are you using dbt with?
other (mention it in "Additional Context")
Additional Context
DB: Exasol with dbt-exasol: 1.0.2
I actually tested the scripts on the exasol and they do not output a string. So I can only see it to be converted somewhere inside the code of dbt
First opened in dbt-utils but was refered here by @joellabes .
I would need a hand. Can't figure out where it goes wrong or why.
I guess it failes at this place in dbt-core but since it depends on this dbt-utils.equality test I posted it here:
https://github.com/dbt-labs/dbt-core/blob/719b2026ab847ab887d3638c81a351b0f6f96158/core/dbt/task/test.py#L212
Some more informative compiled versions:
cat target/compiled/dib/unittests/test.yml/schema_test/dbt_utils_equality_testcsv_A__ref_test_.sql :
-- setup
with a as (
    select * from UT_DOCKER.testcsv
),
b as (
    select * from UT_DOCKER.test
),
a_minus_b as (
    select A from a
    except
    select A from b
),
b_minus_a as (
    select A from b
    except
    select A from a
),
unioned as (
    select 'a_minus_b' as which_diff, a_minus_b.* from a_minus_b
    union all
    select 'b_minus_a' as which_diff, b_minus_a.* from b_minus_a
)
select * from unioned

And from RUN folder
cat target/run/dib/unittests/test.yml/schema_test/dbt_utils_equality_testcsv_A__ref_test_.sql :

select

    count(*) + coalesce(abs(
        sum(case when which_diff = 'a_minus_b' then 1 else 0 end) -
        sum(case when which_diff = 'b_minus_a' then 1 else 0 end)
    ), 0)
 as failures,

    count(*) + coalesce(abs(
        sum(case when which_diff = 'a_minus_b' then 1 else 0 end) -
        sum(case when which_diff = 'b_minus_a' then 1 else 0 end)
    ), 0)
 != 0 as should_warn,

    count(*) + coalesce(abs(
        sum(case when which_diff = 'a_minus_b' then 1 else 0 end) -
        sum(case when which_diff = 'b_minus_a' then 1 else 0 end)
    ), 0)
 != 0 as should_error
    from (

-- setup

with a as (
    select * from UT_DOCKER.testcsv
),
b as (
    select * from UT_DOCKER.test
),
a_minus_b as (
    select A from a
    except
    select A from b
),
b_minus_a as (
    select A from b
    except
    select A from a
),
unioned as (
    select 'a_minus_b' as which_diff, a_minus_b.* from a_minus_b
    union all
    select 'b_minus_a' as which_diff, b_minus_a.* from b_minus_a
)
select * from unioned
    )

I removed some empty lines for length reasons.

@TimoKruth It does sound like this is something happening on the dbt-exasol side of things, since we're not seeing this on other databases. The result of the failure calculation:
count(*) + coalesce(abs(
        sum(case when which_diff = 'a_minus_b' then 1 else 0 end) -
        sum(case when which_diff = 'b_minus_a' then 1 else 0 end)
    ), 0)

Is being returned as '0' instead of 0.
Questions:

Is this happening for any other tests (unique, not_null, etc), or just for dbt_utils.equality?
Does this specific error go away if you configure the fail_calc to use count(*) instead?
Are you able to determine the data type of failures, when you run the full query in Exasol (not using dbt)?

One possible solution could look like doing what we did for should_warn + should_error (boolean coercion) in v0.21.1 (https://github.com/dbt-labs/dbt-core/pull/4091), and try to coerce failures to be an integer here:
https://github.com/dbt-labs/dbt-core/blob/d046ae0606422db24865ce5eaeaa00933c72b72a/core/dbt/task/test.py#L39-L60
But I'd ideally like to get at the heart of the matter, by figuring out what's different in Exasol / the dbt-exasol adapter.

Questions:

Is this happening for any other tests (unique, not_null, etc), or just for dbt_utils.equality?
I could only recognize it for equality. We run equal_rowcount aswell pretty often and that works.

Does this specific error go away if you configure this test's fail_calc to use count(*) instead? (docs)
<- Will test and edit this ->

Are you able to determine the data type of failures, when you run the full query in Exasol (not using dbt)?
When I run it in exasol it outputs a number as failures. Specifically a decimal.

Okay I found a hack... I will discuss this further with dbt-exasol.
Hacks
For Equality:
{% set set_diff %}
    cast(count(*) + coalesce(abs(
        sum(case when which_diff = 'a_minus_b' then 1 else 0 end) -
        sum(case when which_diff = 'b_minus_a' then 1 else 0 end)
    ), 0) as INT)
{% endset %}

For Equal_rowcount:
{{ config(fail_calc = 'CAST(coalesce(diff_count, 0) as INT)') }}


chore: update tslint

I also went and added a few rules to tsconfig that are new in TS2 that were previously in tslint.

I am closing this. I will migrate it by hand to the new site.


KVMHAMonitor thread blocks indefinitely while NFS not available

ISSUE TYPE

Bug Report

COMPONENT NAME

KVM Agent

CLOUDSTACK VERSION

<IP_ADDRESS>-41120rc2

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

Also see comment thread on PR #2722
We installed an RC release which includes PR #2722 on a test system expecting the host to get marked as Disconnected after using iptables to drop NFS requests, but instead the host gets marked as  Down. My investigation shows that the line storage = conn.storagePoolLookupByUUIDString(uuid); blocks indefinitely. So, kvmheartbeat.sh is never executed, a host investigation is started, the host with blocked NFS is marked as Down and finally all VMs on that host are rescheduled and result in duplicate VMs.
I pulled a thread dump and found the KVMHAMonitor thread will hang here until NFS is unblocked.
java.lang.Thread.State: RUNNABLE
      at com.sun.jna.Native.invokePointer(Native Method)
      at com.sun.jna.Function.invokePointer(Function.java:470)
      at com.sun.jna.Function.invoke(Function.java:404)
      at com.sun.jna.Function.invoke(Function.java:315)
      at com.sun.jna.Library$Handler.invoke(Library.java:212)
      at com.sun.proxy.$Proxy3.virStoragePoolLookupByUUIDString(Unknown Source)
      at org.libvirt.Connect.storagePoolLookupByUUIDString(Unknown Source)
      at com.cloud.hypervisor.kvm.resource.KVMHAMonitor$Monitor.runInContext(KVMHAMonitor.java:95)
      - locked <1afb3370> (a java.util.concurrent.ConcurrentHashMap)
      at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
      at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
      at java.lang.Thread.run(Thread.java:748)

 Locked ownable synchronizers:
      - None

STEPS TO REPRODUCE

EXPECTED RESULTS

The host still runs kvmheartbeat.sh and shows as `Disconnected`

ACTUAL RESULTS

The host heartbeat hangs and get marked as `Down` via host investigation

After performing more tests, virtlockd does help prevent from getting duplicate VMs.  With Host HA disabled, the blocked host goes to a Down state but the VMs continue to run on the blocked host and new ones don't start on other hosts because of the lock on the root disk.
But, if Host HA is enabled, the blocked host gets rebooted because of the bug above. The VMs on the host get rescheduled on that same host when it comes up, but it still should not have been rebooted because of blocked NFS.

This morning I confirmed the behavior on 4.9 is different than 4.11. When there's a long lasting (say 15 minutes) NFS hang the agent stays Up and when NFS operations resumes everyone's happy.   Note we did disable the automatic reboot in the heartbeat script for that to work.  This saved us from massive reboots and VM outages before when we had a network maintenance that cut all KVM host from NFS for 22 minutes.

Confirmed we see similar behavior on 4.11.2rc3 and the agent went in Down state.   Agent logs:
810986-e702-36ea-a87b-fd48064ecb12
2018-10-23 13:14:40,391 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:f8cd7cf7) No existing libvirtd connection found. Opening a new one
2018-10-23 13:14:40,392 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:f8cd7cf7) Can not find a connection for Instance i-4-24-VM. Assuming the default connection.
2018-10-23 13:14:40,399 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:f8cd7cf7) Trying to fetch storage pool 4e49054a-463f-306f-9678-b0d9b02af9a1 from libvirt
2018-10-23 13:14:51,496 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:3a0df8e5) Trying to fetch storage pool 0e233ec5-ea14-439e-bfde-a8c7566d254c from libvirt
2018-10-23 13:14:51,498 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:3a0df8e5) Asking libvirt to refresh storage pool 0e233ec5-ea14-439e-bfde-a8c7566d254c
2018-10-23 13:15:25,027 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:581a1d95) Trying to fetch storage pool 0e233ec5-ea14-439e-bfde-a8c7566d254c from libvirt
2018-10-23 13:15:25,029 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:581a1d95) Asking libvirt to refresh storage pool 0e233ec5-ea14-439e-bfde-a8c7566d254c
2018-10-23 13:15:25,590 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:581a1d95) Trying to fetch storage pool 3e810986-e702-36ea-a87b-fd48064ecb12 from libvirt
2018-10-23 13:15:25,592 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:581a1d95) Asking libvirt to refresh storage pool 3e810986-e702-36ea-a87b-fd48064ecb12
2018-10-23 13:21:28,804 WARN  [kvm.resource.KVMHAChecker] (Script-3:null) (logid:) Interrupting script.
2018-10-23 13:21:28,806 WARN  [kvm.resource.KVMHAChecker] (pool-15160-thread-1:null) (logid:c3d5dcaf) Timed out: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i <IP_ADDRESS> -p /vol/t500_0_fls3_pool36_root -m /mnt/d05f1c9d-9454-3707-a6c4-781398af198d -h <IP_ADDRESS> -r -t 60 .  Output is:
2018-10-23 13:21:32,826 WARN  [kvm.resource.KVMHAChecker] (Script-7:null) (logid:) Interrupting script.
2018-10-23 13:21:32,827 WARN  [kvm.resource.KVMHAChecker] (pool-15161-thread-1:null) (logid:c3d5dcaf) Timed out: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i <IP_ADDRESS> -p /vol/t500_0_fls3_pool36_root -m /mnt/d05f1c9d-9454-3707-a6c4-781398af198d -h <IP_ADDRESS> -r -t 60 .  Output is:
2018-10-23 13:21:36,846 WARN  [kvm.resource.KVMHAChecker] (Script-4:null) (logid:) Interrupting script.
2018-10-23 13:21:36,847 WARN  [kvm.resource.KVMHAChecker] (pool-15162-thread-1:null) (logid:4a3cb34f) Timed out: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i <IP_ADDRESS> -p /vol/t500_0_fls3_pool36_root -m /mnt/d05f1c9d-9454-3707-a6c4-781398af198d -h <IP_ADDRESS> -r -t 60 .  Output is:
2018-10-23 13:24:44,205 INFO  [cloud.agent.Agent] (Agent-Handler-1:null) (logid:5a5a7500) Lost connection to host: <IP_ADDRESS>. Attempting reconnection while we still have 5 commands in progress.
2018-10-23 13:24:44,206 INFO  [utils.nio.NioClient] (Agent-Handler-1:null) (logid:5a5a7500) NioClient connection closed
2018-10-23 13:24:44,206 INFO  [cloud.agent.Agent] (Agent-Handler-1:null) (logid:5a5a7500) Reconnecting to host:<IP_ADDRESS>
2018-10-23 13:24:44,207 INFO  [utils.nio.NioClient] (Agent-Handler-1:null) (logid:5a5a7500) Connecting to <IP_ADDRESS>:8250
2018-10-23 13:24:44,207 INFO  [utils.nio.Link] (Agent-Handler-1:null) (logid:5a5a7500) Conf file found: /etc/cloudstack/agent/agent.properties
Note sometimes you will see the agent successfully go in Disconnect state but the host HA framework might still fire after the kvm.ha.degraded.max.period timer and that is not expected.   In any case we want to avoid massive KVM host resets via IPMI for storage related problems because this is more damaging than waiting to primary storage to come back.

When you block NFS on a host, eventually all the agentRequest-Handler and UgentTask threads gradually hang as well.  It seems after the last UgentTask handler thread hangs, is about the time the host is marked as Down.
threaddump.txt

@rhtyd

Correct.  Only VM-HA enabled would get restarted and create a duplicate
when the host goes down.  Still,  I don't think this a good behavior to
fire VM-HA (because of host Up --> Down state) under any scenarios caused
by transient storage disconnection.   If  the host goes down after 5
minutes,  VM-HA restarts VM about one minute later, and then if the NFS
issue gets resolved you have almost 100% probability of root disk
corruption and you don't know where the 2 VMs are since Cloudstack only
remembers the last copy it started.
On Mon, Oct 29, 2018 at 3:21 AM Boris Stoyanov - a.k.a Bobby <
<EMAIL_ADDRESS>wrote:

hi @csquire https://github.com/csquire @somejfn
https://github.com/somejfn, thanks for this issue!
I think it's correct that the host goes into 'Down' state after loosing
it's grip on the storage, since this is basically making it inoperable.
Going into 'Disconnected' state would only mean the connection between
management and host is compromised.
On the other hand duplicated VMs is definitely something that needs to get
addressed, prior marking the host as 'Down' when we have a VM-HA enabled.
Just to be sure, can you please confirm you don't see these duplicated VMs
on a non VM-ha enabled instances? I'd like to narrow down this issue and
make sure it's in the VM-HA logic.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/cloudstack/issues/2890#issuecomment-433810696,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AOflqLPAw0oQ2_po-4BHKpzHRDZMUc9Hks5upqxxgaJpZM4XNOD8
.

I think the leanest way to fence the resource would be, prior to setting the host down to iterate all it's VMs and shut them down, only then to proceed and mark the host as 'Down', once were there, there's no issue with VM-HA starting a new instance on a separate host.
I guess this needs further investigation and a fix as described.

With NFS not available and since those are hard mounts,  even a "virsh
destroy" would not work.  Libvirtd will block until the NFS mount issue is
resolved.     I think that ideally the cloudstack-agent would do every task
in a non-blocking way and not be affected by primary storage hiccups.   For
instance,  to avoid the thread pool blocking on libvirtd tasks,  why no
implement a configurable timeout on thosetasks with sensible defaults ?   I
don't see a good reason a call to libvirtd take more than a few seconds
(beside known long lasting tasks such as live migration)
As for fencing,  afaik the host HA framework was created for the purpose or
reliable fencing... but will cause more damage than good if the end result
is to reboot all KVM hosts via IPMI (compared to just wait for NFS to come
back)
On Mon, Oct 29, 2018 at 9:18 AM Boris Stoyanov - a.k.a Bobby <
<EMAIL_ADDRESS>wrote:

I think the leanest way to fence the resource would be, prior to setting
the host down to iterate all it's VMs and shut them down, only then to
proceed and mark the host as 'Down', once were there, there's no issue with
VM-HA starting a new instance on a separate host.
I guess this needs further investigation and a fix as described.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/cloudstack/issues/2890#issuecomment-433906239,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AOflqMVZGtW3QGGlL-DfyzMy6duL7EXQks5upwAhgaJpZM4XNOD8
.

@csquire I could not reproduce the issue. I guess it depends on how you're simulating NFS/shared-storage failure and what your iptables rules are. In my case I simply shutdown the NFS server instead of iptables rules and observed following:
.
2018-10-30 08:04:04,842 WARN  [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) Timed out: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i <IP_ADDRESS> -p /export/testing/primary -m /mnt/05c30f6d-5725-3369-a8d6-b5fb37c7ba8f -h <IP_ADDRESS> .  Output is:
2018-10-30 08:04:04,843 WARN  [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) write heartbeat failed: timeout, try: 5 of 5
2018-10-30 08:04:04,843 DEBUG [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) [ignored] interupted between heartbeat retries.
2018-10-30 08:04:04,843 WARN  [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) write heartbeat failed: timeout; stopping cloudstack-agent
2018-10-30 08:04:04,843 DEBUG [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) Executing: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i <IP_ADDRESS> -p /export/testing/primary -m /mnt/05c30f6d-5725-3369-a8d6-b5fb37c7ba8f -c
2018-10-30 08:04:04,845 DEBUG [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) Executing while with timeout : 60000
2018-10-30 08:04:05,007 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py get_rule_logs_for_vms
2018-10-30 08:04:05,009 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Executing while with timeout : 1800000
2018-10-30 08:04:05,109 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) (logid:) Execution is successful.
2018-10-30 08:04:05,109 DEBUG [kvm.resource.LibvirtConnection] (UgentTask-5:null) (logid:) Looking for libvirtd connection at: qemu:///system
2018-10-30 08:04:05,114 DEBUG [cloud.agent.Agent] (UgentTask-5:null) (logid:) Sending ping: Seq 1-157:  { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 11, [{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{},"_hostVmStateReport":{"r-5-VM":{"state":"PowerOn","host":"centos7-kvm1"},"s-1-VM":{"state":"PowerOn","host":"centos7-kvm1"},"v-2-VM":{"state":"PowerOn","host":"centos7-kvm1"},"i-2-4-VM":{"state":"PowerOn","host":"centos7-kvm1"}},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}] }
2018-10-30 08:04:05,166 DEBUG [cloud.agent.Agent] (Agent-Handler-3:null) (logid:) Received response: Seq 1-157:  { Ans: , MgmtId:<PHONE_NUMBER>376, via: 1, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}] }
2018-10-30 08:04:06,762 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:0aa26d2d) Request:Seq 1-3303108851699548248:  { Cmd , MgmtId:<PHONE_NUMBER>376, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.GetVncPortCommand":{"id":4,"name":"i-2-4-VM","wait":0}}] }
2018-10-30 08:04:06,762 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:0aa26d2d) Processing command: com.cloud.agent.api.GetVncPortCommand
2018-10-30 08:04:06,762 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:0aa26d2d) Looking for libvirtd connection at: qemu:///system
2018-10-30 08:04:06,765 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) (logid:0aa26d2d) Seq 1-3303108851699548248:  { Ans: , MgmtId:<PHONE_NUMBER>376, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.GetVncPortAnswer":{"address":"<IP_ADDRESS>","port":5903,"result":true,"wait":0}}] }
2018-10-30 08:04:09,871 DEBUG [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) Exit value is 143
2018-10-30 08:04:09,872 DEBUG [kvm.resource.KVMHAMonitor] (Thread-34:null) (logid:) Redirecting to /bin/systemctl stop cloudstack-agent.service
2018-10-30 08:04:09,873 INFO  [cloud.agent.Agent] (AgentShutdownThread:null) (logid:) Stopping the agent: Reason = sig.kill
2018-10-30 08:04:09,874 DEBUG [cloud.agent.Agent] (AgentShutdownThread:null) (logid:) Sending shutdown to management server
2018-10-30 08:04:09,876 DEBUG [utils.nio.NioConnection] (Agent-NioConnectionHandler-1:null) (logid:) Location 1: Socket Socket[addr=/<IP_ADDRESS>,port=8250,localport=43646] closed on read.  Probably -1 returned: Connection closed with -1 on reading size.
2018-10-30 08:04:09,876 DEBUG [utils.nio.NioConnection] (Agent-NioConnectionHandler-1:null) (logid:) Closing socket Socket[addr=/<IP_ADDRESS>,port=8250,localport=43646]
2018-10-30 08:04:10,875 DEBUG [kvm.resource.LibvirtConnection] (AgentShutdownThread:null) (logid:) Looking for libvirtd connection at: qemu:///system

I could see the kvmheartbeat.sh try 5 times every 60 seconds and eventually failing to shutdown the agent by this: /bin/systemctl stop cloudstack-agent.service. On the management server side, I saw the host state go from Up to Disconnected:

The only change in behaviour I see is that the KVM host is not rebooted but only the agent gets shutdown.

Based on the triaging exercise, I've moved this to <IP_ADDRESS> as further discussion is pending. I've taken the least risk approach to revert part of the change in behaviour and submitted - https://github.com/apache/cloudstack/pull/2984

If you just shutdown the NFS server,  the NFS client will get immediate
response (TCP reset) so this is not the same as blocking the NFS server IP
with iptables and DROP rules like I do for testing the network outage:
iptables -I INPUT -s nfs_server_ip -j DROP ; iptables -I OUTPUT -d
nfs_server_ip -j DROP
It may take more than one attempt to see the tread pool block.
On Tue, Oct 30, 2018 at 4:47 AM Rohit Yadav<EMAIL_ADDRESS>wrote:

Based on the triaging exercise, I've moved this to <IP_ADDRESS> as further
discussion is pending. I've taken the least risk approach to revert part of
the change in behaviour and submitted - #2984
https://github.com/apache/cloudstack/pull/2984
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/cloudstack/issues/2890#issuecomment-434217528,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AOflqLrd7IVAIUdSuy-hW7NLhwXPDnfbks5uqBISgaJpZM4XNOD8
.

@rhtyd closing as this is two years old this weekend and #2890


Run better for DART

Change some environment variable for Dart in a Box
Make image more efficiently
Separate log files better

@yanzv, this addresses some of the issues you've noticed.  Hopefully the logs will be more helpful now.

Great.  Thanks!  I will test a new image soon.


Bug: meta is not properly applied in the loopHooks

In 7.0.0-rc.14 the function signature of loopHooks should have been changed:

Changed the function signature for all loopHooks accept a single object of named meta properties instead of a variable length of meta arguments. For example, hooks.prepareRow.push((row, instance) => void) is now written hooks.prepareRow.push((row { instance }) => void)

And in the code for the prepareRow hook there is a new signature used:
https://github.com/tannerlinsley/react-table/blob/63d54b1e876d5a04d67e1e8c9e5f1198697cd382/src/hooks/useTable.js#L503-L505
However, in the loopHooks definition the second meta argument is not applied:
https://github.com/tannerlinsley/react-table/blob/63d54b1e876d5a04d67e1e8c9e5f1198697cd382/src/publicUtils.js#L111-L113
So the meta won't be available in the hooks at the moment.

There is also a typo in the changelog entry:

is now written hooks.prepareRow.push((row { instance }) => void)

missing a coma, should be:

is now written hooks.prepareRow.push((row, { instance }) => void)

Yeah, I ran into this bug developing the new usePivotColumns hook. I've the fix applied locally. I'll also fix the changelog. :)


backspace after removing chips initially does nothing

Describe the bug
Add two or more chips, then remove the left-most chip by tapping the X on the chip. Now focus the text field and press the backspace key -- nothing happens. Nothing will happen for each press of the backspace key that corresponds to the number of chips removed by tapping the X button. That is, add 4 chips then X the first three, now backspace does nothing for three presses.
Library version, Flutter version, platform
Latest code on main on chrome and windows, probably macos as well.

Fixed in commit 72b2d74


Preview2 fails

Using dotnet sdk preview2 1.0.0-preview2-003112
Repro:
dotnet new --lang f#
dotnet restore
dotnet build

Error:
 internal error: Could not load type 'System.Reflection.PortableExecutable.ContentId' from assembly 'System.Reflection.Metadata, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

with dotnet-compile-fsc restored:
      "dotnet-compile-fsc/1.0.0-preview2-003071": {
        "type": "package",
        "dependencies": {
          "Microsoft.DotNet.Cli.Utils": "1.0.0-rc2-002678",
          "Microsoft.DotNet.Compiler.Common": "1.0.0-rc2-002678",
          "Microsoft.DotNet.ProjectModel": "1.0.0-rc4-003139",
          "Microsoft.FSharp.Compiler.netcore": "1.0.0-alpha-160509",
          "Microsoft.NETCore.App": "1.0.0"
        },

@enricosada - there was a breaking change in System.Reflection.Metadata during Ask Mode of .NET Core 1.0:
https://github.com/dotnet/corefx/commit/a9442a6fe0d0d616ce372018c86ff435cf5337c2
Microsoft.FSharp.Compiler.netcore is going to have to be recompiled (and fixed) on the latest contracts in order to fix this up.
@ericstj @tmat @KevinRansom

Who is on this?

I need to rebuild fsharp for the latest coreclr.  I have been having connection difficulties.  But I am on it.
From: Scott Hanselman<EMAIL_ADDRESS>Sent: Monday, June 27, 2016 7:30 PM
To: dotnet/netcorecli-fsc<EMAIL_ADDRESS>Cc: Kevin Ransom<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [dotnet/netcorecli-fsc] Preview2 fails (#12)
Who is on this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/dotnet/netcorecli-fsc/issues/12#issuecomment-228833435, or mute the threadhttps://github.com/notifications/unsubscribe/AE76FgRJxAp7d4Xi3CZg2jZ_ffBOHqgBks5qQBa7gaJpZM4I3l9m.

https://github.com/dotnet/cli/pull/3627 /cc @piotrpMSFT @enricosada

@KevinRansom check pr https://github.com/Microsoft/visualfsharp/pull/1290, i started to fix deps and build tool now use RTM version of .NET Core (with preview2 of sdk)

fixed by version 1.0.0-preview2-003131 of .net core sdk (ref announcement )


Updating $rootScope does not update hosting app's view

I'm trying to implement a service that exposes a global loading screen provided by the hosting app. The service works fine in the hosting app, in that my view is updated when I trigger the service from the hosting app. However, if I trigger the service from a widget, it seems the $rootScope I use to trigger the loading screen is updated, but the view tied to it is not. Plunker here: https://plnkr.co/edit/7hskrPayP2vyAkJEeIPX
You'll see if you use the buttons in App1, the Loading var in $rootScope is changed accordingly, and the view updates, but if you use the buttons from App2, the console logs confirm that the var does change but the view does not.

Turns out I didn't add widgetsProvider.addServiceToShare in my config. Once I added that, it worked.


[device_info_plus] add Windows support

Description
Add support for Windows in package:device_info_plus. This draft PR is intended for early feedback and clarity on where we stand on Windows support.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]).
This will ensure a smooth and quick review process. Updating the pubspec.yaml and changelogs is not required.

[X] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[ ] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See [Contributor Guide]).
[ ] All existing and new tests are passing.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] The analyzer (flutter analyze) does not report any problems on my PR.
[X] I read and followed the [Flutter Style Guide].
[X] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[X] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
[ ] No, this is not a breaking change.

Looking good so far. I'll spare the nit-picks for later. :smile: Are there any issues or concerns?
Perhaps we could include the computer name? https://github.com/timsneath/win32/blob/main/example/sysinfo.dart#L72

I went with the decision to migrate the device_info_plus packages to null safety, since the null-safe package:ffi API changed significantly in 1.0.0, and we're going to migrate sooner or later anyway.
However, I'm having trouble running the example without --no-sound-null-safety, and it's unclear why.
Melos is giving me a hard time on Windows, and since I'm overriding dependencies manually while working, I might have missed  to fully migrate something somewhere.

Did the migration require any manual changes? I'm wondering if it's not too much trouble, we could get them in first to reduce a bit of noise and focus on one thing.

I did it by hand, since it's not much code. I'll make an attempt of the migration separately.

Moved the null safety migration to #136. Let's do that first.

This is amazing, @jpnurmi @Schwusch just make sure when it's merged, the website and readme file are up to date too.

This is amazing, @jpnurmi @Schwusch just make sure when it's merged, the website and readme file are up to date too.

Thanks, I'm unsure to do that though. @jpnurmi halp!

Thanks, I'm unsure how to do that though. @jpnurmi halp!

Sorry I was busy tweeting! :D Anyway, no worries, the website gets automatically built and deployed.


Game Crashing On Game Over

Describe your problem here.
My Script Crashes The Game When In The Song And Game Over Happens.
In The Script, I Already Have This :
setPropertyFromClass('GameOverSubstate', 'characterName', 'bf-wrath'); --Character json file for the death animation
setPropertyFromClass('GameOverSubstate', 'deathSoundName', 'fnf_loss_sfx'); --put in mods/sounds/
setPropertyFromClass('GameOverSubstate', 'loopSoundName', 'gameOver'); --put in mods/music/
setPropertyFromClass('GameOverSubstate', 'endSoundName', 'gameOverEnd'); --put in mods/music/

The Character .json Is Set To The Right Name Across All Files.
For The Death Animations In The .json I Have:
	{
		"offsets": [
			37,
			11
		],
		"indices": [],
		"fps": 24,
		"anim": "firstDeath",
		"loop": false,
		"name": "BF NOTE DOWN MISS0"
	},
	{
		"offsets": [
			37,
			5
		],
		"loop": true,
		"fps": 24,
		"anim": "deathLoop",
		"indices": [],
		"name": "BF idle dance"
	},
	{
		"offsets": [
			37,
			69
		],
		"loop": false,
		"fps": 24,
		"anim": "deathConfirm",
		"indices": [],
		"name": "BF HEY!!"
	}

All Names For This Pointing To The .xml Are Correct, I Have Tested With The Built-In character Editor And The Animations Play Just Fine. But Once I Get Into The Song And Die It Crashes, I Can Hear The Sound Effects Begin To Play Though So It Shouldn't Be A Problem With The Script?
Using The "GameOverSubState" Script I Can't Even Seem To Get The Original BF's Game Over To Play Instead, even When All The OG BF Files Are Still In The Assets Folder.
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
I Didn't Touch Source Code At All, Via Lua I've Added A New Song With Custom Note Types, Custom Events, And A New Difficulty.

Is that your entire JSON file? If so, it's missing a few things. I know from experience that broken JSON files crash the game, so it could be that.

{ 	"animations": [ 		{ 			"offsets": [ 				7, 				19 			], 			"loop": false, 			"fps": 24, 			"anim": "singLEFTmiss", 			"indices": [], 			"name": "BF NOTE LEFT MISS" 		}, 		{ 			"offsets": [ 				-15, 				-19 			], 			"loop": false, 			"fps": 24, 			"anim": "singDOWNmiss", 			"indices": [], 			"name": "BF NOTE DOWN MISS" 		}, 		{ 			"offsets": [ 				-36, 				27 			], 			"loop": false, 			"fps": 24, 			"anim": "singUPmiss", 			"indices": [], 			"name": "BF NOTE UP MISS" 		}, 		{ 			"offsets": [ 				-44, 				22 			], 			"loop": false, 			"fps": 24, 			"anim": "singRIGHTmiss", 			"indices": [], 			"name": "BF NOTE RIGHT MISS" 		}, 		{ 			"offsets": [ 				-3, 				5 			], 			"loop": false, 			"fps": 24, 			"anim": "hey", 			"indices": [], 			"name": "BF HEY!!" 		}, 		{ 			"offsets": [ 				15, 				18 			], 			"loop": false, 			"fps": 24, 			"anim": "hurt", 			"indices": [], 			"name": "BF hit" 		}, 		{ 			"offsets": [ 				-4, 				0 			], 			"loop": true, 			"fps": 24, 			"anim": "scared", 			"indices": [], 			"name": "BF idle shaking" 		}, 		{ 			"offsets": [ 				37, 				5 			], 			"loop": true, 			"fps": 24, 			"anim": "deathLoop", 			"indices": [], 			"name": "BF idle dance" 		}, 		{ 			"offsets": [ 				37, 				69 			], 			"loop": false, 			"fps": 24, 			"anim": "deathConfirm", 			"indices": [], 			"name": "BF HEY!!" 		}, 		{ 			"offsets": [ 				-10, 				-16 			], 			"loop": false, 			"fps": 24, 			"anim": "dodge", 			"indices": [], 			"name": "boyfriend dodge" 		}, 		{ 			"offsets": [ 				294, 				267 			], 			"loop": false, 			"fps": 24, 			"anim": "attack", 			"indices": [], 			"name": "boyfriend attack" 		}, 		{ 			"offsets": [ 				-40, 				-40 			], 			"loop": false, 			"fps": 24, 			"anim": "pre-attack", 			"indices": [], 			"name": "bf pre attack" 		}, 		{ 			"offsets": [ 				0, 				0 			], 			"loop": true, 			"fps": 24, 			"anim": "", 			"indices": [], 			"name": "" 		}, 		{ 			"offsets": [ 				-5, 				0 			], 			"loop": true, 			"fps": 24, 			"anim": "idle", 			"indices": [], 			"name": "BF idle dance" 		}, 		{ 			"offsets": [ 				5, 				-6 			], 			"loop": true, 			"fps": 24, 			"anim": "singLEFT", 			"indices": [], 			"name": "BF NOTE LEFT0" 		}, 		{ 			"offsets": [ 				-10, 				-50 			], 			"loop": true, 			"fps": 24, 			"anim": "singDOWN", 			"indices": [], 			"name": "BF NOTE DOWN0" 		}, 		{ 			"offsets": [ 				-29, 				27 			], 			"loop": true, 			"fps": 24, 			"anim": "singUP", 			"indices": [], 			"name": "BF NOTE UP0" 		}, 		{ 			"offsets": [ 				-48, 				-7 			], 			"loop": true, 			"fps": 24, 			"anim": "singRIGHT", 			"indices": [], 			"name": "BF NOTE RIGHT0" 		}, 		{ 			"offsets": [ 				37, 				11 			], 			"loop": false, 			"fps": 24, 			"anim": "firstDeath", 			"indices": [], 			"name": "BF NOTE DOWN MISS0" 		} 	], 	"no_antialiasing": false, 	"image": "characters/bf-wrath", 	"position": [ 		0, 		350 	], 	"healthicon": "bf-wrath", 	"flip_x": true, 	"healthbar_colors": [ 		0, 		255, 		60 	], 	"camera_position": [ 		0, 		0 	], 	"sing_duration": 4, 	"scale": 1 }

@Draconic-Hybrid
That Is Only A Small Section Of The .json Showing The Game Over Anims
Here Is The Entire .json For The Character :
https://drive.google.com/file/d/1s7Drb1kkxblH4mrcOivPa6Jd5yMji-G8/view?usp=sharing

That's strange. The only culprit I can think of is that animation without a name or a xml directory.

If you have any code running under onUpdate, try adding an inGameOver check, otherwise it tends to crash when switching states

How Do I Write A inGameOver Check?

@frantastic24
The Only Code I Have With The "onUpdate" Function Is For The Winning Icons,
Which Doesn't Have Anything Under It For "InGameOver"

It's a simple if statement

So
function onUpdate()
if inGameOver then
setPropertyFromClass('GameOverSubstate', 'characterName', 'bf-wrath'); --Character json file for the death animation
setPropertyFromClass('GameOverSubstate', 'deathSoundName', 'fnf_loss_sfx'); --put in mods/sounds/
setPropertyFromClass('GameOverSubstate', 'loopSoundName', 'gameOver'); --put in mods/music/
setPropertyFromClass('GameOverSubstate', 'endSoundName', 'gameOverEnd'); --put in mods/music/
end
end
?

I never said to move your code under onUpdate, I said to add the check for the code you already have under it

Oh Ok Thanks :p


Improve ASC Docs in Readme

What I want to do is to improve the ASC commands documentation in the Readme.md.
What I need is a script that runs each subcommand of ASC with the -h option and generates a MD file from it. Similar to the one the readme shows currently (but that one is outdated).
Sine I don't want to update this file each time I add a new sub command I want to auto-generate the readme.
Please just ignore the other commands the Assist tool contains. I only need a good documentation for ASC at the moment.

Duplicate of https://github.com/Blackjacx/Assist/issues/27


Ingress stays in 'Initializing' status

Rancher versions:
rancher/rancher: 2.1.0
Infrastructure Stack versions:
healthcheck: None
ipsec:
network-services: Canal
scheduler:
kubernetes (if applicable): 1.11.3
Docker version: (docker version,docker info preferred)
Containers: 30
Running: 27
Paused: 0
Stopped: 3
Images: 14
Server Version: 17.03.2-ce
Storage Driver: overlay
Backing Filesystem: extfs
Supports d_type: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.67-rancher2
Operating System: RancherOS v1.4.1
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 31.42 GiB
Name: cl01-w1
ID: MELF:YYLU:7C4E:L5V4:HCPE:7FVX:R5N6:GYPQ:EWEM:ZMRL:LI3S:BPA5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
provider=vmwarevsphere
Experimental: false
Insecure Registries:
artifactory.mydomain.com
<IP_ADDRESS>/8
Live Restore Enabled: false
Operating system and kernel: (cat /etc/os-release, uname -r preferred)
4.14.67-rancher2
Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
vSphere
Setup details: (single node rancher vs. HA rancher, internal DB vs. external DB)
HA rancher
Environment Template: (Cattle/Kubernetes/Swarm/Mesos)
Kubernetes
Steps to Reproduce:
deploy nginx workload with TCP/80 on ClusterIP
deploy ingress to nginx Service with Specify a hostname to use
Results:
Ingress stays in 'Initializing' status

@eb10005  Thanks for the report!
I guess it is the same issue with https://github.com/rancher/rancher/issues/16027?

It looks similar, but I'm using the default ingress that gets deployed with Rancher/Kubernetes.  Is this a bug in Kubernetes 1.11.3?  Any way to fix/workaround?

Have you tried this https://github.com/rancher/rancher/issues/16027#issuecomment-428310589?

Same here, ingress creation stuck Initializing
Rancher versions:
rancher/rancher: 2.1.1
{
"annotations": {
"field.cattle.io/ingressState": "{}"
},
"baseType": "ingress",
"created": "2018-10-21T20:30:55Z",
"createdTS":<PHONE_NUMBER>000,
"creatorId": "user-prgtz",
"id": "monitoring:test",
"labels": {
"cattle.io/creator": "norman"
},
"links": {
"remove": "…/v3/project/c-5797b:p-ljjnf/ingresses/monitoring:test",
"self": "…/v3/project/c-5797b:p-ljjnf/ingresses/monitoring:test",
"update": "…/v3/project/c-5797b:p-ljjnf/ingresses/monitoring:test",
"yaml": "…/v3/project/c-5797b:p-ljjnf/ingresses/monitoring:test/yaml"
},
"name": "test",
"namespaceId": "monitoring",
"projectId": "c-5797b:p-ljjnf",
"rules": [
{
"host": "test.r2.lab",
"paths": {
"": {
"serviceId": null,
"targetPort": 3000,
"type": "/v3/project/schemas/httpIngressPath",
"workloadIds": [
"deployment:monitoring:grafana"
]
}
},
"type": "/v3/project/schemas/ingressRule"
}
],
"state": "initializing",
"status": {
"loadBalancer": {
"type": "/v3/project/schemas/loadBalancerStatus"
},
"type": "/v3/project/schemas/ingressStatus"
},
"transitioning": "no",
"transitioningMessage": "",
"type": "ingress",
"uuid": "36547f57-d570-11e8-9bf2-005056a4419f"
}

Try to disable nginx ingress and re-enable it. Did the trick for me.

Also experiencing this problem, Rancher v2.1.1

I had this issue. In my case, it was operator error, but documenting in case it helps others.
I was trying to run the cluster node on the same server as rancher itself , but the ingres can never start as port 80/443 is already in use by rancher.
I solved it by deploying the node on a different machine, but you could possibly solve it and run all on the same machine by using ports other than 80/443 when you launch the initial container.
docker ps -a to find the ingres container, then docker logs xxxx helped diagnose.

@ybizeul @gaui @singularo what ingress controller do you have in the cluster?

@alena1108 traefik

@gaui Can you try this https://github.com/rancher/rancher/issues/16027#issuecomment-429627605?

At first I just had nginx, in my case I just had to flip nginx controller off and on then all was good.
Traefik is a different story and doesn’t look like it can be implemented by just running the helm chart.
Did some tests with a friend yesterday and he thinks the way it works, an actual L4 load balancer is expected so I need to do some tests with MetalLB.
What he couldn’t understand is why adding externalIP to traefik parameters ends up working (but ingress are still Initializing).

Edit your cluster then check "Disabled" on Nginx Ingress setting
Save, and enable it again.
Worked for me, let us know how that goes.

@ybizeul, thanks.  I just noticed the following message on the cluster edit:
Note: Currently Amazon EKS will not create an ingress controller when launching a new cluster. If you need this functionality you will have to create an ingress controller manually after cluster creation.

Any idea what the best way to create a new ingress controller would be?

We've added the logic for populating ingress state based on the status/Load balancer field with commit aa102c67526504553c7bed623665e04718167e1a.  As we can only truly say that the ingress is functional when controller sets status/Load balancer with the ip address/hostname. Nil field value would indicate that the controller failed to populate it by some reason. There is a known issue for traefik provider side that is supposedly got resolved in 1.7 version, but based on the last comments, still requires some extra config on traefik controller side:
containous/traefik#2173
Way to fetch k8s ingress address info:
Kubecl output - Address field is the one we are looking for
> kubectl get ingress
NAME         HOSTS    ADDRESS        PORTS     AGE
ingressfoo  foo.com  <IP_ADDRESS>   80        47s

The same field presented in kubectl get ingress -o json output:
"status": {
                "loadBalancer": {
                    "ingress": [
                        {
                            "ip": "<IP_ADDRESS>"
                        }
                    ]
                }
            }

Hello
I Have the same probeme, i think the probleme is  in the nginx-ingress-controller, th logs show :

NGINX Ingress controller
Release: 0.16.2-rancher1
Build: d36d4cc
Repository: git@github.com:rancher/ingress-nginx.git
F1103 10:51:39.712022 6 main.go:72] Port 80 is already in use. Please check the flag --http-port
Any idea ?

Port is either in use (check by running netstat -plant | grep \:80) or you are running into https://github.com/kubernetes/ingress-nginx/issues/2781

In my case, neither the ingress-nginx namespace nor the workloads are created.
But workload rke-ingress-controller-delete-job spawns multiple pods on same node. Error message in log is always the same:
error: You must provide one or more resources by argument or filename.
Example resource specifications include:
'-f rsrc.yaml'
'--filename=rsrc.json'
'<resource> <name>'
   '<resource>'

Is there any other way to force Rancher to create Nginx Ingress workloads if disabling/enabling in cluster settings doesn't work?

@loganhz
I've set in traefik configuration:
--kubernetes.ingressendpoint.publishedservice=traefik/traefik

and I'm getting:
time="2018-11-15T19:33:38Z" level=debug msg="Skipping updating Ingress default/coffee-ingress due to service traefik/traefik having no status set"

and it's still stuck in "Initializing"

@ybizeul, thanks. I don't see that setting in cluster edit. I just noticed the following message on the cluster edit:
Note: Currently Amazon EKS will not create an ingress controller when launching a new cluster. If you need this functionality you will have to create an ingress controller manually after cluster creation.

Any idea what the best way to create a new ingress controller would be?

Having the same issue! i don't see that settings from Edit cluster . FYI: I'm running the EKS as cloud provider

In my case, neither the ingress-nginx namespace nor the workloads are created.
But workload rke-ingress-controller-delete-job spawns multiple pods on same node. Error message in log is always the same:
error: You must provide one or more resources by argument or filename.
Example resource specifications include:
'-f rsrc.yaml'
'--filename=rsrc.json'
'<resource> <name>'
   '<resource>'

Getting the same issue on one custom cluster using rancher v2.1.3

Having same issue with traefik and metalLB

commented out the kubernetes.io/ingress.class: traefik and redeployed its now in active state.

rancher is addings its own annotations to ingress:
  field.cattle.io/publicEndpoints: '[{"addresses":["X.X.X.X"],"port":80,"protocol":"HTTP","serviceName":"default:prometheus-server","ingressName":"default:prometheus-server","hostname":"prometheus.hello.int","path":"/","allNodes":true}]'

Hello
I Have the same probeme, i think the probleme is in the nginx-ingress-controller, th logs show :
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: 0.16.2-rancher1
Build: d36d4cc
Repository: git@github.com:rancher/ingress-nginx.git
-------------------------------------------------------------------------------
F1103 10:51:39.712022 6 main.go:72] Port 80 is already in use. Please check the flag --http-port

Any idea ?

I have the same message in my logs. Did you solve this somehow?

Just found out it works if I move away my rancher/rancher container from port 80+443 to 8080+8443.
Its logical. Using a single node install rancher blocks the port 80 used for load balancer by itself. Had to recreate the whole "cluster" setup to do this.

And recreating the ingress doesn't work..
Anybody any ideas?

Try to disable nginx ingress and re-enable it. Did the trick for me.

Hey i have the same problem.
And recreating the ingress doesn't work..
Anybody any ideas?

So after a lot of digging, it turns out your storage driver might be the issue. I tried turning my Ingress NGINX off/on, only to have nginx-ingress-controller getting stuck at F0407 22:51:52.909611 7 main.go:72] Port 80 is already in use. Please check the flag --http-port.
I went searching for it, and down the rabbit hole I ran into: ttps://github.com/rancher/rancher/issues/16172. I happen to be using the aufs Storage Driver (since it's an old docker installation), which apparently is no longer supported. To check if you're running the aufs driver, run docker info.
As far as I know the only way to upgrade it is to uninstall docker completely, including removing images / volumes etc. since there's no way to switch them from one to the other. Though I'm not entirely sure of this. The aufs driver is no longer supported anyway, so this might be a good motivation to upgrade it.

My run in with this problem was that I had the default Pod Security Policy set to "restricted" which denied the pods deployed by the ingress daemonset the ability to bind with the host ports 80/443. When I switched the Pod Security Policy for the "System" project to "unrestricted" everything deployed correctly.

There are a lot of different issues of why people might have their ingress stuck. If you still have this issue and could create a new issue if you are still facing this issue with more details, that'd be great!

If you're having this issue while following the Rancher getting started guide, and you're using Minikube for your cluster, make sure that you've enabled the ingress addon in minikube! Took longer to figure out than I'm willing to admit, but at least I got a lesson in Kubernetes debugging.


GH-3350 update pom and docker to support jdk17

GitHub issue resolved: #2046
GitHub issue related: #3350
Briefly describe the changes proposed in this PR:

updated some build plugins that don't work on jdk17 (versions already covered by CQ https://dev.eclipse.org/ipzilla/show_bug.cgi?id=20318 )
updated base docker image to jre11 which we need because we are targeting 11 with our compiler (using the docker image is the only way I know to test that I didn't break the war file)

PR Author Checklist (see the contributor guidelines for more details):

[ ] my pull request is self-contained
[ ] I've added tests for the changes I made
[ ] I've applied code formatting (you can use mvn process-resources to format from the command line)
[ ] I've squashed my commits where necessary
[ ] every commit message starts with the issue number (GH-xxxx) followed by a meaningful description of the change

@JervenBolleman I couldn't add you as a reviewer for this PR, but thought you might like to take a look.

Do we need a CQ for the docker base image update?

Do we need a CQ for the docker base image update?

I don't think we did one for the original, so let's leave it for now.

updated base docker image to jre11 which we need because we are targeting 11 with our compiler (using the docker image is the only way I know to test that I didn't break the war file)

What I typically do is build the war using mvn -Passembly package and then drop it in a locally running tomcat server. I could imagine direcly running a docker image being easier though.
Could you share (perhaps as a writeup in the developer docs) how you use the docker image to test? What I mean is literally just the commands you use to produce the docker image and then run it. It's probably straightforward enough but we have almost zero documentation around our docker images, and this might be a good place to start.

I use the bash script called build.sh and then I run "docker-compose up".

I think there are some readme files in the same directory. Probably the biggest improvement would be to move the docker stuff to its own directory to increase visibility.

Do we need a CQ for the docker base image update?

IIRC this wans't necessary, because docker builds are not 'official' Eclipse deliverables (though memory could serve me wrong)

I think there are some readme files in the same directory. Probably the biggest improvement would be to move the docker stuff to its own directory to increase visibility.

Back in the days I wrote some stuff on the dev mailinglist as well...
Which reminds me I'll have to push the images to hub.docker.com... (at one point we were looking into doing this automatically)

Related: #3389


Admin onboarding tutorial tour

Summary

Ticket Link

Related Pull Requests

Has server changes (please link here)
Has mobile changes (please link here)

Screenshots

Release Note

@matthewbirtch @esethna We can test new onBoarding tutorial here (old one will no longer be available even for regular users). Right now it will show as soon as user login will add few conditions to that.

@AshishDhama I just tried to test using the sysadmin user and it opened right to the 'Customize your experience' tour point. I assume that's because this user previously completed the first 4 tour points?
For now, I'll create a new user to go through the steps, but I'll need instructions for resetting the sysadmin user. Let me know!

@matthewbirtch Closing this in favour of https://github.com/mattermost/mattermost-webapp/pull/9774, Will add all the changes requested by you here in that.


plugins,osbs: handle deprecation of openshift_uri -> openshift_url

[x] update also other plugins which use osbs-client

LGTM


Questions about epoch

Hello, I'm sorry to bother you again.
In the project, max_epoch is set to 4000. Although the setting is large, the training and convergence are fast. But according to my past experience, isn't the epoch usually set between 100~300? I have set the epoch to 200 on other models, but the training is slower than yours, and they achieve a similar degree of accuracy.
So, I would like to ask what is the epoch here. Is it Iterations or some other parameter? I feel very confused. Could you give me some help?

Got it!
Thank you very much


Filter by nested array value

I have a json structure like this:
"users": [
    {
      "id": 1,
      "first_name": "Erek",
      "urbs":[
        {
          "id": 12
        },
        {
          "id": 8
        }
      ]

    },
    {
      "id": 2,
      "first_name": "Ladonna",
      "urbs": [
        {
          "id": 7
        }
      ]
    },
    {
      "id": 3,
      "first_name": "Lezley",
      "urbs": [
        {
          "id": 10
        }
      ]
    },
    {
      "id": 4,
      "first_name": "Rick",
      "urbs":[
        {
          "id": 14
        },
        {
          "id": 12
        }
      ]

    }
]

Is there a way to get all users where the urbs array contain an object with id = 12 ?
I've tried many things, and this is the closest I got to my goal:
http://localhost:3000/users?urbs.0.id=12
This works, but only when id = 12 is in the first position of the array.
I have no way of knowing the position of the value inside the array.

Unfortunately for me the similar thing what you have done above doesnt work.
My JSON is:
{
  "products": [
    {
      "productName": "Pipe",
      "productCode": "1-",
      "productAvailable": true,
      "productPrice": 122,
      "productRating": 4,
      "productDescription": "blahblah.",
      "productImageURL": "170767/1340130900.svg",
      "productComments": [
        {
          "id": 0,
          "dislikes": 0,
          "likes": 0,
          "comment": "asd"
        },
        {
          "id": 1,
          "dislikes": 0,
          "likes": 0,
          "comment": "asdaddsadadasd"
        }
      ]
    }
 ]
}

Having URL like this http://localhost:3000/products?productComments.0.id doesnt work at all :(

Same here, it works only when asking for a specific position in the array

any solution ever found?

any solution ever found?

Any solution till today?


parsedate.isAfter() should be validated on the basis of format used?

ERROR: minDate() date parameter is after options.maxDate: 2017-09-21T12:24:17+05:30
I am getting this error at the scenario where I am trying to give minDate as first date component's date.
at initial, the values are null for both the date component then it throws me this error as at that time, null become today's date and current time while the computation in the library.
due to which minDate becomes greater than the maxDate which is already computed.

So, in this case, I don't want it to be checked on the basis of time to get rid of this error in the current case.

what is the format of the parsedate?


Add support for TLSA

Is it possible to add support to send TLSA queries (RFC 6698, https://tools.ietf.org/html/rfc6698)?
It could be great to discover the MX record for a bunch of domains and then follow up if there are TLSA records configured for them for a list of known ports.

We'd be happy to accept a pull request that adds support for TLSA queries.

PR: #161 adds support for TLSA and SRV.


TypeError: this.props.onClick is not a function

In Page.js after event (click) in devtools I see error:
"use strict";Object.defineProperty(exports,"__esModule",{value:!0});var _createClass=function(){function e(e,t){for(var r=0;r<t.length;r++){var a
and handler was implented as onChange={handlePageChange} not as in desription onChange={:: this.handlePageChange}
Maybe I'm doing something wrong.

+1

That is I see, maintainers don't resolve this problem yet.

please provide more details. I just checked, everything is working for me.
try:
 git clone git@github.com:vayser/react-js-pagination.git
cd react-js-pagination
npm i
npm start
go to localhost:8000, works well for me, without errors which you mentioned above

I also experienced the same exact error
dist/Pages.js line 73:
e.preventDefault(), r || this.props.onClick(a);

BTW, I resolved this error on my end by doing this
constructor(props) {
super();
this.handlePageChange = this.handlePageChange.bind(this);

}
handlePageChange(pageNumber) {
console.log(active page is ${pageNumber});
this.setState({activePage: pageNumber});
}
<Paginate ...
onChange={ this.handlePageChange }
/>

Sorry my answer is not in a neat format.. ^^ but hope that can help somehow

@neilmaledev you are right. I have updated readme to avoid such issues at the future