Show experiment loading progress

The current UI shows a spinning circle when a user is loading an experiment, so it's unclear whether the experiment is loading successfully, or how much longer it's going to take.
Since the frontend knows both an experiment's last simulated tick and how many ticks of data it's already received, it'd be nice to show some indication of this progress.

Unfortunately, due to all states being fetched in one go, this is not feasible at the moment. As soon as we have a better caching mechanism (both in the API and on the frontend), this should, however, definitely be implemented.


Tensorflow 2.4 about 10% slower than 2.3 for training RNN

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google Colab
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary (pip)
TensorFlow version (use command below): difference between 2.3 and 2.4
Python version:
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

Describe the current behavior
In TensorFlow 2.4, a basic RNN model runs about 10% slower than the same model in TensorFlow 2.3. This appears on both the CPU and GPU. I've created Colab notebooks to demonstrate the issue.
The step time reported by the progress bar also appears longer (and when I did some of my own timing, I also saw the step as longer), so I don't think it has to do with callbacks or any sort of overhead outside the step function.
Describe the expected behavior
I would expect them to be the same, if not better in 2.4 (since it has newer CUDA, etc.).
I know that TensorFlow 2.4 no longer has XLA enabled by default. I tried enabling it on my local machine, and found that it didn't help (in fact it made things slower in TF 2.4). Furthermore, if I enable it on Colab with os.environ["TF_XLA_FLAGS"] = "--tf_xla_auto_jit=2" I see slower speeds on GPU for both 2.3 and 2.4. So I don't think that's the problem. (Also, the fact that there's a slowdown on CPU suggests it's not an XLA thing.)
Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
TF 2.3: https://colab.research.google.com/drive/1xnfP17Bzan7-ICI_JtMfiEdTvgwrLoEz?usp=sharing
TF 2.4: https://colab.research.google.com/drive/17rccQnDvLr4mAiGmAoJyK7t0hlrEpBoV?usp=sharing

I also did some similar experiments on a CNN, and found that there is also a slowdown there, though it seems less significant (only about 5% instead of 10%):
TF 2.3: https://colab.research.google.com/drive/10WcsVCkyxRrrclpmPWSaWAI8UslZAR_Q?usp=sharing
TF 2.4: https://colab.research.google.com/drive/1dolfWxeq_PV3UGvTFMo6E9sC1doeZENv?usp=sharing

I also did some similar experiments on a CNN, and found that there is also a slowdown there, though it seems less significant (only about 5% instead of 10%):
TF 2.3: https://colab.research.google.com/drive/10WcsVCkyxRrrclpmPWSaWAI8UslZAR_Q?usp=sharing
TF 2.4: https://colab.research.google.com/drive/1dolfWxeq_PV3UGvTFMo6E9sC1doeZENv?usp=sharing

Doing some profiling on a related model (one which also showed a slowdown in TF 2.3), I noticed that while the "Device compute time" has stayed about the same from 2.2 to 2.3 to 2.4 (if not decreased slightly), the "Kernel launch time" has increased (for that model, from 0.6 to 1.7 to 2.8 ms per step). Is there some reason the kernel launch time would be so much higher in 2.3 vs 2.2, and then higher again in 2.4?
I should also note that with the original SimpleRNN model I posted above, if I scale things up (using input_d=256 and units=1024), then I see similar performance between 2.3 and 2.4. So it seems to be related to the amount of work each kernel is performing (could be a longer kernel launch time, could be something else).
On the one hand, if there was a conscious decision made to trade off faster computation for longer kernel launch times (and thus improve performance on models that have fewer kernels doing more work, but degrade performance on models with many smaller kernels), I understand why this trade-off would make sense. On the other hand, there are definitely still many cases (even large models) that for one reason or another use smaller kernels.

Doing some profiling on a related model (one which also showed a slowdown in TF 2.3), I noticed that while the "Device compute time" has stayed about the same from 2.2 to 2.3 to 2.4 (if not decreased slightly), the "Kernel launch time" has increased (for that model, from 0.6 to 1.7 to 2.8 ms per step). Is there some reason the kernel launch time would be so much higher in 2.3 vs 2.2, and then higher again in 2.4?
I should also note that with the original SimpleRNN model I posted above, if I scale things up (using input_d=256 and units=1024), then I see similar performance between 2.3 and 2.4. So it seems to be related to the amount of work each kernel is performing (could be a longer kernel launch time, could be something else).
On the one hand, if there was a conscious decision made to trade off faster computation for longer kernel launch times (and thus improve performance on models that have fewer kernels doing more work, but degrade performance on models with many smaller kernels), I understand why this trade-off would make sense. On the other hand, there are definitely still many cases (even large models) that for one reason or another use smaller kernels.

Was able to reproduce the issue. Code runs a bit slower on TF v2.4 when compared with TF v2.3.
Whereas with TF-nightly, Colab doesn't detect the GPU. Please find the gist of it here. Thanks!

Was able to reproduce the issue. Code runs a bit slower on TF v2.4 when compared with TF v2.3.
Whereas with TF-nightly, Colab doesn't detect the GPU. Please find the gist of it here. Thanks!

I am also experiencing slowdown when upgrading from TF 2.3.1 to 2.4.1. Almost all compiled GPU ops are between 5 and 15% slower. A notable outlier is clip_by_value, which is more than 6x slower in my case.

I am also experiencing slowdown when upgrading from TF 2.3.1 to 2.4.1. Almost all compiled GPU ops are between 5 and 15% slower. A notable outlier is clip_by_value, which is more than 6x slower in my case.

Adding @yhliang2018 and @tomerk since this is more a like overall performance issue rather than just RNN issue.

Adding @yhliang2018 and @tomerk since this is more a like overall performance issue rather than just RNN issue.

Hi @hunse do you have a link to your profiling? Especially where it notes the change in kernel launch time)

Hi @hunse do you have a link to your profiling? Especially where it notes the change in kernel launch time)

(And ideally traces of before & after as well)

(And ideally traces of before & after as well)

Here's the profiling where I saw an increase in kernel launch time (one folder is TF 2.3, the other is TF 2.4). The profiling overview reports that TF 2.4 has more ops on the host, though I wouldn't think that that would increase kernel launch time, unless it's having to wait for other ops on the host to finish before it launches the next one.
Unfortunately, that profiling is from a network in our NengoDL package, so it's not an isolated TensorFlow example (I could pass along code for it, but it wouldn't be pure TensorFlow code, so I'm not sure if that's useful). I had been trying to isolate the slowdown in the Colab notebooks above, but it appears that that's not perfect: When I run that code on my local machine, I don't see a slowdown. So I don't think that that's fully captured the problem.

Hi @hunse, I've finally had a chance to dig in to your profiles. Checking the traces, t looks like your tf 2.4 runs are using 8 compute threads on the host (+ 3 tf data threads). On the other hand, your tf 2.3 runs are only using 4 or 5 compute threads on the host + 3 tf.data threads. The GPU compute itself looks like it has basically the same time breakdown across both so it seems like placement of ops hasn't changed at least at a quick glance, but both the host in 2.4 spends much more of its time on send/receives.
Perhaps context switching among the extra threads is slowing down your tensor data transfers? (e.g. via reducing cache coherency)
Could you try reducing the thread count in 2.4 to match what 2.3 used by default? https://www.tensorflow.org/api_docs/python/tf/config/threading
(Though off the top of my head I'm not sure what if any changes we made to the default threadpool size selection)

Thanks for the suggestion, @tomerk. I standardized the number of threads, which seems to have reduced the discrepancy between TF 2.3 and 2.4 in that case.
However, for that same network, when I compare between 2.2 and 2.4, I still see a significant difference. The difference is still there whether I use 4, 2, or 1 thread.
I've attached some profiling for 1 thread showing results from 2.2, 2.3, and 2.4. The big difference between 2.2 and 2.3 is that there's a lot more host-chip communication at the end of each step, which seems to increase the amount of time between steps.
In another model, I see more of a difference between 2.3 and 2.4 than 2.2 and 2.3 (profiling here). Even when accounting for the fact that TF 2.3 and 2.4 seem to take a bit longer to start up, when I measure just the time to run the core loop on the device it takes about 650 ms in 2.2, 665 ms in 2.3, and 697 ms in 2.4. Both 2.3 and 2.4 appear to have more host-device communication, but in 2.3 it's more spread-out and doesn't result in as much time between steps, whereas in 2.4 it all happens between steps. I'm not sure if this is the entire reason (or any of the reason) for the slowdown, but it is the most salient difference to me.

Hmm I don't seem to be able to view these profiles. Not sure if it's an issue on my side or not. Let me try getting someone else since I'm not sure I'll have the bandwidth to dig in to this.

I have observed the same phenomenon, with tensorflow2.0 GPU, I can run the same RNN program 300 seconds, but with tensorflow 2.4 GPU, the same code takes 30 minutes to run, such a huge difference. Therefore unfortunately I have to downgrade 2.4 to 2.0 again

another example: on Quadro RTX 6000 .. TF 2.4 ETA 7 hours for single iteration, TF 2.3 40 minutes. Same everything except for downgrading to 2.3 between the two runs.

I have also seen the performance degradation with CPU only too. Too me the performance decreases about 15%. There is another issue with multi-thread predicating. With 2.4 and 2.5 release, I got error like this occasionally(I do pre-warm predication for the model before launching threads):
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1915, in predict_on_batch
    self.predict_function = self.make_predict_function()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1545, in make_predict_function
    if self.predict_function is not None:
AttributeError: 'Sequential' object has no attribute 'predict_function'

This doesn't happen at all with same code on release 2.3.

@hunse , Could you try with the latest Tensorflow stable version 2.9.1 and let us know the outcome, Thanks!


adds support for convenient travis builds with forked repos and branches

I'd rather not switch to building an image and pushing to Dockerhub for each branch. In future I'm going to switch to only building and pushing an image for releases, and I don't want to push every branch I create to Dockerhub as I'm working :)

point taken, how about ability to control this with optional $DOCKER_BRANCHPUSH so that it can be explicitly enabled by travis variable ?

updated to process docker for branches only if travis variable DOCKER_MAKEBRANCH is set to true


automate esdoc releases

What is the current behavior?
After every successful release via semantic-release a manual fetch and gulp deploy will properly update the documentation available at https://github-issues-label-sync-package.superleap.xyz
What is the expected behavior?
gulp deploy should run in semantic-release post on travis and automatically update the documentation. A smooth configuration should ensure this doesn't fail, current manual testing is redundant.

Additional requirements:

remove gh-pages branch branch checks
enforce gh-pages branch push restriction


[Bug]: dbms output error, dbms_output.chararr type does not exist

ODC version
ODC421
OB version
Oceanbase312
What happened?
PL debugging, after the Oceanbase312 version of the function is run in the ODC412 version and the ODC421 version, the DBMS output is inconsistent.

What did you expect to happen?
Don't print DBMS output
How can we reproduce it (as minimally and precisely as possible)?
this issue can be reproduced on OceanBase 3.2.4, which doest not occur on later OceanBase.
declare
begin
    dbms_output.put_line('Hello,world');
end;

Anything else we need to know?
No response
Cloud
No response

This printing error is a bit strange, please take a look @yhilmare

odc uses this block to record dbms output:
DECLARE
  L_LINES       DBMS_OUTPUT.CHARARR;
  L_NUM         INTEGER := ?;
  L_LINES_ARRAY CLOB;
BEGIN
  DBMS_OUTPUT.GET_LINES(L_LINES, L_NUM);
  IF L_NUM > 0 THEN
    FOR i IN 1 .. L_NUM LOOP
      IF i = 1 THEN
          L_LINES_ARRAY := L_LINES(i);
      ELSE
          L_LINES_ARRAY := L_LINES_ARRAY || '
' || L_LINES(i);
      END IF;
    END LOOP;
  END IF;
  ? := L_LINES_ARRAY;
END;

unfortunately, dbms_output.get_lines and dbms_output.chararr is new features added on OceanBase 4.0 and is not patched to older OceanBase


Oess extension

Please review the changes on FIM to support OESS network service.

Could you please change it to merge to rel1.4 branch instead of master please.


Fix - 4 and 6 DoF EIG solution method modification

Issue related to the numerical precision of the solver method adopted for the eigenvalue problem, aiming to change the solver for a more precise and robust one. The current solver works fine for small matrices (smaller than ~30x30), but struggles to keep the error acceptable for anything bigger than that. Some models demand bigger and more poorly conditioned matrices, which demand a different solver.

Hi there @verg1lio !
The function used to calculate eigenvalues basically has this in it: https://github.com/ross-rotordynamics/ross/blob/f7cc3d118a727313bf648805b1b5e72b610e3f6d/ross/rotor_assembly.py#L1237-L1251
So we are using the scipy eigs function, which uses the Fortran ARPACK to calculate the eigenvalues.

ARPACK is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems.

As you can see in the code we have to set some arguments for the calculation of the eigenvalues when calling the la.eigs() function. I think we should start this investigation by checking if any of these arguments are having a negative impact on the calculation precision.
We also need to discuss what level of precision we want here, since changing the solver may affect calculation time and also can increase our dependencies, which could increase package installation size, time to build and test the software etc. So we will need to balance those aspects as well.
To evaluate this, I would recommend that we select some study cases (models) to use as reference when comparing the solvers in precision and calculation time. If you could share the models that you are seeing a larger error here we could start by those.

Hello @raphaeltimbo !
I start by agreeing with you here  "As you can see in the code we have to set some arguments for the calculation of the eigenvalues when calling the la.eigs() function." , since if this routine already uses sufficiently accurate enough eigenvalue solution methods it is a simple method of using the right parameters.
Also regarding the precision, the solver we generally use applies a method called Cholesky factorization, which gives the first "n" eigenvalues and associated eigenvectors, being either the largest or the smallest ones . This is a quick and reliable method that we have been using for decades with no issues in our codes. If i got it right, the "mpmath" package has this methodology, and is already a requirement in ROSS . Nevertheless, if the la.eigs() have this capability, then the problem has a trivial solution.
As for the case study I completely agree. I will generate the same problem we were having while doing the test functions and comparing the eigenvalue solutions for the 6 DoF model. Having the problem I will post it here.
Anyways, I will get back to you here, and link a pull request we made this morning in our repository . to this issue.

Hi @raphaeltimbo , good afternoon!
I believe I have identified the problem. It seems that the solver have different methods depending on the size or complexity of the eigenvalue problem it is trying to solve. See the attached images for clarification.
This first one have 36 finite elements, and falls in the "big" category for the solver, which utilizes a better method and gives constant and reliable results. Check that the Wns are constantly the same.

This last one, which we used in the example, utilizes 6 finite elements and fall in the "small" category, using classical triangulation methods which have little precision and tend to accumulate errors exponentially with the matrix size and conditioning. The results are not stable as the bigger matrix case are, and fluctuate from one execution to the other.

What intrigued me though is that these results are much more precise then the ones we were obtaining when writing the test functions about a year ago. I know that using increasingly bigger matrices will decrease the simple methods precision, but what i do not know is the threshold between methodologies.
In any event, I strongly recommend that we standardize the solver and make it independent of matrix size, possibly by changing the routine options to set a fixed method for all matrix sizes.
(@elyqg , @hiagopinacio)

Hi @verg1lio ! Could you paste here the code used to generate the 6 elements model so I can copy it and make some tests?

Yes, absolutely! I was using our example routine as a base, and it goes as below:
import ross as rs
import numpy as np
rotor6 = rs.rotor_assembly.rotor_example_6dof()
camp6 = rotor6.run_campbell(np.linspace(0,400,101),frequencies=18)
rotor_speed = 100.0 # rad/s
modal6 = rotor6.run_modal(rotor_speed)
print(f"Undamped natural frequencies:\n {modal6.wn}")

You will notice that there is a "n=6" parameter in the example (rotor_example_6dof, in rotor_assembly.py) that should be changed for a different number of elements in the model.

@raphaeltimbo
The complete code is the example itself, as follows:

    i_d = 0
    o_d = 0.05
    n = 36
    L = [0.25 for _ in range(n)]

    shaft_elem = [
        ShaftElement6DoF(
            material=steel,
            L=0.25,
            idl=0,
            odl=0.05,
            idr=0,
            odr=0.05,
            alpha=0,
            beta=0,
            rotary_inertia=False,
            shear_effects=False,
        )
        for l in L
    ]

    disk0 = DiskElement6DoF.from_geometry(
        n=2, material=steel, width=0.07, i_d=0.05, o_d=0.28
    )
    disk1 = DiskElement6DoF.from_geometry(
        n=4, material=steel, width=0.07, i_d=0.05, o_d=0.28
    )

    kxx = 1e6
    kyy = 0.8e6
    kzz = 1e5
    cxx = 0
    cyy = 0
    czz = 0
    bearing0 = BearingElement6DoF(
        n=0, kxx=kxx, kyy=kyy, cxx=cxx, cyy=cyy, kzz=kzz, czz=czz
    )
    bearing1 = BearingElement6DoF(
        n=6, kxx=kxx, kyy=kyy, cxx=cxx, cyy=cyy, kzz=kzz, czz=czz
    )

    rotor6(shaft_elem, [disk0, disk1], [bearing0, bearing1])

    rotor_speed = 100.0 # rad/s

    modal6 = rotor6.run_modal(rotor_speed)
    print(f"Undamped natural frequencies:\n {modal6.wn}")

Hi @verg1lio , thank you for providing the code!
I have carried out some tests and I will try to explain the results that I got and also give some options on what we can do to improve things.
We can see in the code that we have the v0 argument for the las.eigs() function:
https://github.com/ross-rotordynamics/ross/blob/f7cc3d118a727313bf648805b1b5e72b610e3f6d/ross/rotor_assembly.py#L1237-L1251
This value is set to None when we instantiate the rotor, and if we call la.eigs with this, we have:
v0 : ndarray, optional
    Starting vector for iteration. Default: random

So, the default value for the starting vector is random, but after the first eigenvalue calculation we set this value to:
self._v0 = np.real(sum(evectors.T))

We set this value for v0 to improve the performance for the next calculations.
If we set this to a fixed value, we can have repeatability (but we loose speed as we will see):
for i in range(5):
    rotor6._v0 = np.ones(len(rotor6.A()))
    modal6 = rotor6.run_modal(rotor_speed)
    print(f"Undamped natural frequencies:\n {modal6.wn}")

Undamped natural frequencies:
 [5.17964778e-05 3.53466379e+00 3.54867463e+00 2.24056139e+01
 2.25269639e+01 3.13659949e+01]
Undamped natural frequencies:
 [5.17964778e-05 3.53466379e+00 3.54867463e+00 2.24056139e+01
 2.25269639e+01 3.13659949e+01]
Undamped natural frequencies:
 [5.17964778e-05 3.53466379e+00 3.54867463e+00 2.24056139e+01
 2.25269639e+01 3.13659949e+01]
Undamped natural frequencies:
 [5.17964778e-05 3.53466379e+00 3.54867463e+00 2.24056139e+01
 2.25269639e+01 3.13659949e+01]
Undamped natural frequencies:
 [5.17964778e-05 3.53466379e+00 3.54867463e+00 2.24056139e+01
 2.25269639e+01 3.13659949e+01]

Another way to have repeatability is to use la.eig() instead of la.eigs(), which uses the eigenvalue routine from LAPACK instead of ARPACK:
for i in range(5):
   modal6 = rotor6.run_modal(rotor_speed, sparse=False)
   print(f"Undamped natural frequencies:\n {modal6.wn}")

Undamped natural frequencies:
[8.82902875e-05 3.53466380e+00 3.54867463e+00 2.24056144e+01
2.25269652e+01 3.13660164e+01]
Undamped natural frequencies:
[8.82902875e-05 3.53466380e+00 3.54867463e+00 2.24056144e+01
2.25269652e+01 3.13660164e+01]
Undamped natural frequencies:
[8.82902875e-05 3.53466380e+00 3.54867463e+00 2.24056144e+01
2.25269652e+01 3.13660164e+01]
Undamped natural frequencies:
[8.82902875e-05 3.53466380e+00 3.54867463e+00 2.24056144e+01
2.25269652e+01 3.13660164e+01]
Undamped natural frequencies:
[8.82902875e-05 3.53466380e+00 3.54867463e+00 2.24056144e+01
2.25269652e+01 3.13660164e+01]

Now, if we carry out some performance comparisons, we have:
# Current implementation with las.eigs()
%%timeit
for i in range(10):
    modal6 = rotor6.run_modal(rotor_speed)

435 ms ± 4.57 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# Setting initial vector to constant value
%%timeit
for i in range(10):
    rotor6._v0 = np.ones(len(rotor6.A()))
    modal6 = rotor6.run_modal(rotor_speed)

692 ms ± 8.77 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# Do not consider sparse, use la.eig()
%%timeit
for i in range(10):
    modal6 = rotor6.run_modal(rotor_speed, sparse=False)

1.1 s ± 8.81 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

So, with the current code we have a 2x increase in calculation speed.
Regarding the repeatability, I agree that it is not expected to see these values changing between each calculation, so I think we could address this in the documentation, stating that the method that we use as default could lead to this behavior and showing the option to change to sparse=False more clearly.
Also, I think we could add a 'global' config parameter, so that the user could do something like:
import ross as rs

rs.config.eigen_sparse = False

and this would be used in other parts of the code, instead of having to pass the argument sparse=False to each function (this is actually not possible to some run functions such as run_campbell)
Regarding the tests, we have two options:

set a higher rtol within the test;
set sparse=False when calling the calculation.

For cases that we have in the tests right now, I feel that we have tolerances that are acceptable within rotordynamics.

Hi again @raphaeltimbo !
I got the same impression you have. For rotordynamics simulations the results are reasonably accurate and within tolerances, and the high error we were detecting when writing the test functions is not present as it was. I could not replicate that error when trying it this week, and believe it to have been patched by the library devs.
In any event, the increase in computational time may prove problematic for a very detailed Campbell diagram generation, since it roughly doubled in time with no sensible gain in precision.
With that said, exclusively for the test functions I believe that setting the sparse=False option is the way to go, since it will allow for an effective and reliable conference of the code. I say that for in my experience many additions to the physics of a model may yield almost negligible effects in the end results, depending on the physical configuration on the system, and these might be lost to the precision error if we are not rigorous with the tests.
Then if we are agreed, I will simply add the  sparse=False option to the tests involving eigenvalue solutions in the branch and close the issue.
Thanks for the quick responses, and all the best!
Vergílio

Then if we are agreed, I will simply add the sparse=False option to the tests involving eigenvalue solutions in the branch and close the issue.

That is ok for me!

Hi there!
I have marked this issue as stale because it has not had activity for 45 days.
Consider the following options:

If the issue refers to a large task, break it in smaller issues that can be solved in
less than 45 days;
Label the issue as wontfix or wontfix for now and close it.

Closed with #826.


Date of "Datatracker url" link

Describe the issue
Needs a better date of being listed, will comunicate privetly if you want to know more.
Code of Conduct

[X] I agree to follow the IETF's Code of Conduct

https://www.rfc-editor.org/errata.php#reportnew

If you need to send private clarification use<EMAIL_ADDRESS>Otherwise, there's nothing actionable in this issue. I'm closing it.


Update filename regex to support non-ascii letters

I changed regex of filename to support non-ascii (unicode) letters.

Hi sunovivid, Thanks for the pull request. Actually, file creation for java is handled a bit different where this filename patching is done for the sub-directory created exclusively for that problem. JavaFileGenerator is the class in the same folder as DefaultFileGenerator class. Also, you can remove the whitespace and hyphen filtering too.

Thank you for your answer!
I didn't know that I missed JavaFileGenerator. Thanks for notifying me.
I added a new commit.
First, I added common getValidFileName function for both DefaultFileGenerator and JavaFileGenerator.
Second, I changed the special character filtering based on this answer.
But blacklist-filtering has some potential dangers as I think (like missing reserved special characters or OS-specific reserved words), so I added a validation on filename by delegating it to OS. If the validation failed, it returns valid filename based on whitelist-filtering.
And I found that whitelist-filtering can have a danger too. For example, whitespaces after the filename cause InvalidPathException in the function Path.gets(filename). So I added trim() but still there can exist some dangers. So if the whitelist-filtering also fails, it uses the original whitespaces and hyphen filtering. I think it is much safer.
Please tell me if the code is too complicated or not readable.

Hi sunovivid, Thanks for the extensive research, the code looks great.


MB4DLZ and MB02SZ use a non Standard BLAS routine

The routines MB4DLZ and MB02SZ  reference the DCABS1 routine, which is not available in some BLAS implementations(like ATLAS). In order to get SLICOT working with allmost all BLAS libraries, this routine needs to be replaced or inlined.

MB4DLZ and MB02SZ  have been updated in Version v5.9 and there are no more DCABS1 calls.


Fix some misspells in helpers_test.go

"commitedDigest" to "committedDigest".

Thanks!
CI is failing on some formatting errors:
content/helpers_test.go:1::warning: file is not gofmted with -s (gofmt)
content/helpers_test.go:1::warning: file is not goimported (goimports)
make: *** [check] Error 1
The command "make check" exited with 2.```

Please make sure to run `gofmt`.

Codecov Report

Merging #3008 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3008   +/-   ##
=======================================
  Coverage   43.96%   43.96%
=======================================
  Files         102      102
  Lines       10881    10881
=======================================
  Hits         4784     4784
  Misses       5362     5362
  Partials      735      735

Flag
Coverage Δ

#linux
47.58% <ø> (ø)
:arrow_up:

#windows
41.18% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 521a44b...d7033a7. Read the comment docs.

Codecov Report

Merging #3008 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3008   +/-   ##
=======================================
  Coverage   43.96%   43.96%
=======================================
  Files         102      102
  Lines       10881    10881
=======================================
  Hits         4784     4784
  Misses       5362     5362
  Partials      735      735

Flag
Coverage Δ

#linux
47.58% <ø> (ø)
:arrow_up:

#windows
41.18% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 521a44b...d7033a7. Read the comment docs.

@ehazlett, I have update it,Thank you for guiding me🐯.

  One more request:  would you mind squashing those commits into one (git rebase -i origin/master)?  Thanks!! 🤗

@ehazlett, I have squash it🐯.

Almost there :)  You just need to sign your commits using the DCO (git commit -s --amend -C HEAD should do the trick).

@ehazlett , Thank you for your reminding😸.

LGTM


Rework system decorations

What does the pull request do?
Adds Window.SystemDecorationsProperty allowing three decoration styles:

Full, system border and titlebar, equivalent to HasSystemDecorations == true
BorderOnly, system border but no titlebar
None, no decorations, equivalent to HasSystemDecorations == false

Checklist

[ ] Added unit tests (if possible)?
[x] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Avaloniaui.net with user documentation

BTW, how would that work with Wayland server decorations once we get the Wayland backend? I only see an on/off toggle in xdg-decoration protocol

TBH I don't know Linux all that well so can't say. If I get some time I'll look into how gtk does it when using the header bar.
But even with falling back to None I still think it's a useful feature given how popular the border-without-titlebar style is on Windows and Mac.

@kekekeks @grokys Can we get a decision on this PR? This is blocking me from resuming my work on owned windows since I will badly conflict with these changes.

Wayland isn't stable yet and I doubt the protocol will be finished any time soon. Also why avoiding a feature just because one technology doesn't support it (yet). You could just ignore unsupported values for now.

@MarchingCube
I did some investigation regarding xdg_decoration protocol and asked on #wayland IRC channel. It seems that it's a plain ON/OFF switch, so we'll just have to emulate the "BorderOnly" mode by drawing "system" decorations manually (we'll have to do the same for the full decorations on several DEs anyway, so it shouldn't be that much of a problem).
There is also a problem with toggling full decorations ON again after they were disabled, but it should get fixed on the wayland side (I've raised an issue in wayland-protocols repo).

Regarding the current PR: I'm not sure how future-proof is SystemDecorations enum.
    SystemDecorationsNone = 0,
    SystemDecorationsBorderOnly = 1,
    SystemDecorationsFull = 2,

seems to be very specific.
Other than that, LGTM.

Sorry for they delay, I haven't had much time to work on this lately. Thanks for checking wayland @kekekeks. I'll look into fixing the HasSystemDecorations property.

Regarding the current PR: I'm not sure how future-proof is SystemDecorations enum.
    SystemDecorationsNone = 0,
    SystemDecorationsBorderOnly = 1,
    SystemDecorationsFull = 2,

seems to be very specific.
Other than that, LGTM.

I did consider using flags for the enum, that way more options could be added in the future (e.g. border without shadow, or windowbuttons without titlebar). But then you run into problems of different combinations not working on different platforms and some not working at all and it all gets a bit messy.
None/Border/Full seemed like a good balance of usefulness and cross platform support. Open to changing it though if you'd prefer a different approach.
[Flags]
public enum SystemDecorations
{
    None = 0
    Border = 1
    Titlebar = 2
    Full = Border + Titlebar
}

Full = Border + Titlebar

Titlebar without border doesn't make much sense though
BTW, It seems that on OSX the border flag controls the shadow rather than border.

It was an idea to make it more future proof, but yes I agree it's not a great solution.
Testing on macos 10.15 it adds the rounded corners and border for me. Is that not what you're getting?

Apologies for the delay @ngarside - I've just pushed a commit to your branch which implements HasSystemDecorations in terms of SystemDecorations, hope you don't mind me pushing directly to your branch!
@kekekeks how's this looking now? Ready for merge?

Still not really convinced about the API design, other than that LGTM.
We might want to change that to use "hint" flags before 0.10 release

Any suggestions for a better API design?

If this is an relatively straightforward change

It would only affect the Window.cs, the rest would be the same.
I was thinking about something like WindowHints.KeepBorderWhenDecorationsAreDisabled, but it looks kinda ugly too.

Thanks @grokys.
@kekekeks window hints would just be flags on the window right? I can update this pr if you're decided on that API design?

I'm not quite sure about using flags since those aren't really XAML and data binding friendly. At the same time I don't want to pollute the list of Window properties. May be we should use something like
WindowDecorationHints.KeepBorderWhenDecorationsAreDisabled attached property?
@grokys

Sorry @kekekeks without a more detailed explanation of what you have in mind I can't really comment. Lets get this merged anyway and work from there.

SystemDecorations is in 0.9.6?
I can't use it in my project.
Thanks.

It's not in 0.9.x, should be in 0.10.

New features are only available threw https://github.com/AvaloniaUI/Avalonia/wiki/Using-nightly-build-feed

Thanks. Sorry for the convenient.

Is it in 0.10.
I update to 0.10 but still can’t use it.
Thank you.

Are you sure you're on 0.10, I can't find a reference to its release anywhere

I got it from nuget

Link? The latest release I can see on nuget is 0.9.10, which this won't be in as 0.9.x point releases are only for bug fixes.
If you need this feature now you'd have to use Gillibald's link above.

Sorry for careless. Yes im on 0.9.10
You know when 0.10 will release?

I don't know sorry, you'd have to ask @kekekeks or one of the other maintainers.

Feel free to use the nightly build feed http://avaloniaui.net/docs/quickstart/packages
That way you get most recent changes


Xiome chat history styles

Added styles for xiome's chat history for issue #87

i've now merged this work into master
for some reason github shows this pull request as rejected -- but it has been accepted and merged -- i think github is confused because i pushed the merged branch to master before i pushed the fixed branch back to your fork...
anyways, i've made some significant changes to this work in four additional commits, you may choose to review those changes in this diff: https://github.com/chase-moskal/xiome/compare/ab82aee45f020b67c88d3bd548ed8cf14d0eb54c..8910fd13901ab13dd8f9820210205d3ff3073f24


Get haproxy-wi version number

More of a how-to than an issue, but is there anyway to get the current version number of haproxy-wi?

See files descriptions on "Code" page

I mean get the version number of haproxy-wi from within a centos 7 machine with haproxy-wi running....

Left down corner?
Отправлено из Mail.ru для Android среда, 24 июля 2019г., 16:33 +03:00 от lcami<EMAIL_ADDRESS>:

I mean get the version number of haproxy-wi from within a centos 7 machine with haproxy-wi running....
—
You are receiving this because you commented.
Reply to this email directly,  view it on GitHub , or  mute the thread .

I need to get it from a script to display it on a different dashboard with the version number of haproxy.
For haproxy I use haproxy –v and then store the output in a variable.
Anyway to get the version number for haproxy-wi?
From: Pavel Loginov<EMAIL_ADDRESS>Sent: 24 July 2019 15:35
To: Aidaho12/haproxy-wi<EMAIL_ADDRESS>Cc: Luke Camilleri<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [Aidaho12/haproxy-wi] Get haproxy-wi version number (#128)
Left down corner?
Отправлено из Mail.ru для Android среда, 24 июля 2019г., 16:33 +03:00 от lcami notifications<EMAIL_ADDRESS>:

I mean get the version number of haproxy-wi from within a centos 7 machine with haproxy-wi running....
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub , or mute the thread .

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/Aidaho12/haproxy-wi/issues/128?email_source=notifications&email_token=AMWL6WGEHY3FNW2PYO3E3BTQBBK65A5CNFSM4IGPZ3P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WLEGA#issuecomment-514634264, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMWL6WDNYARJYP5MC5LUOCDQBBK65ANCNFSM4IGPZ3PQ.

Aaa... You about this... No, you can't
Отправлено из Mail.ru для Android среда, 24 июля 2019г., 16:36 +03:00 от lcami<EMAIL_ADDRESS>:

I need to get it from a script to display it on a different dashboard with the version number of haproxy.
For haproxy I use haproxy –v and then store the output in a variable.
Anyway to get the version number for haproxy-wi?
From: Pavel Loginov<EMAIL_ADDRESS>Sent: 24 July 2019 15:35
To: Aidaho12/haproxy-wi <<EMAIL_ADDRESS>Cc: Luke Camilleri <<EMAIL_ADDRESS>; Author <<EMAIL_ADDRESS>Subject: Re: [Aidaho12/haproxy-wi] Get haproxy-wi version number (#128)
Left down corner?
Отправлено из Mail.ru для Android среда, 24 июля 2019г., 16:33 +03:00 от lcami<EMAIL_ADDRESS><EMAIL_ADDRESS>:
I mean get the version number of haproxy-wi from within a centos 7 machine with haproxy-wi running....
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub , or mute the thread .
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub< https://github.com/Aidaho12/haproxy-wi/issues/128?email_source=notifications&email_token=AMWL6WGEHY3FNW2PYO3E3BTQBBK65A5CNFSM4IGPZ3P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WLEGA#issuecomment-514634264> , or mute the thread< https://github.com/notifications/unsubscribe-auth/AMWL6WDNYARJYP5MC5LUOCDQBBK65ANCNFSM4IGPZ3PQ> .
—
You are receiving this because you commented.
Reply to this email directly,  view it on GitHub , or  mute the thread .

Is there maybe a version number inside one of the files that I can grep?
Or is there the possibility to implement this small change?
From: Pavel Loginov<EMAIL_ADDRESS>Sent: 24 July 2019 15:38
To: Aidaho12/haproxy-wi<EMAIL_ADDRESS>Cc: Luke Camilleri<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [Aidaho12/haproxy-wi] Get haproxy-wi version number (#128)
Aaa... You about this... No, you can't
Отправлено из Mail.ru для Android среда, 24 июля 2019г., 16:36 +03:00 от lcami notifications<EMAIL_ADDRESS>:

I need to get it from a script to display it on a different dashboard with the version number of haproxy.
For haproxy I use haproxy –v and then store the output in a variable.
Anyway to get the version number for haproxy-wi?
From: Pavel Loginov<EMAIL_ADDRESS>Sent: 24 July 2019 15:35
To: Aidaho12/haproxy-wi < haproxy-wi<EMAIL_ADDRESS>Cc: Luke Camilleri < luke.camilleri<EMAIL_ADDRESS>; Author < author<EMAIL_ADDRESS>Subject: Re: [Aidaho12/haproxy-wi] Get haproxy-wi version number (#128)
Left down corner?
Отправлено из Mail.ru для Android среда, 24 июля 2019г., 16:33 +03:00 от lcami notifications<EMAIL_ADDRESS>notifications<EMAIL_ADDRESS>:
I mean get the version number of haproxy-wi from within a centos 7 machine with haproxy-wi running....
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub , or mute the thread .
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub< https://github.com/Aidaho12/haproxy-wi/issues/128?email_source=notifications&email_token=AMWL6WGEHY3FNW2PYO3E3BTQBBK65A5CNFSM4IGPZ3P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WLEGA#issuecomment-514634264> , or mute the thread< https://github.com/notifications/unsubscribe-auth/AMWL6WDNYARJYP5MC5LUOCDQBBK65ANCNFSM4IGPZ3PQ> .
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub , or mute the thread .

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/Aidaho12/haproxy-wi/issues/128?email_source=notifications&email_token=AMWL6WBM46YECCNOBVR4H4LQBBLL3A5CNFSM4IGPZ3P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WLPEQ#issuecomment-514635666, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AMWL6WCQMWRTBJGW2RY2T4DQBBLL3ANCNFSM4IGPZ3PQ.

You may parse the file: app/templates/base.html, string:
					HAproxy-WI v<IP_ADDRESS>

it might be cumbersome but this just works, thought of sharing it in case someone needs somethiung similar:
grep -i "https://github.com/aidaho12/haproxy-wi/" /var/www/haproxy-wi/app/templates/base.html | cut -d ">" -f 2 |cut -d "<" -f 1
HAproxy-WI v<IP_ADDRESS>
Thanks a lot for your help


Upstream Merge

Merged current AEL changes with the upstream Cesium changes.

We shouldn't merge our changes in the referenceFrame branch into master until it is merged upstream. I updated the referenceFrame branch instead. Thanks.


IntelliSense for self.prop = ... class member definitions in JavaScript

From https://github.com/Microsoft/vscode/issues/27825
TypeScript Version:  2.3.4
Code
var MyClass = function(){
  var self = this;
  self.foo= "";
  this.bar = "";
}

Currently our IntelliSense only suggests bar on instances of MyClass and not foo
I believe this may be a special case of https://github.com/Microsoft/TypeScript/issues/10868

The compiler has no way to track aliasing this for write purposes at the time being.


Upload plans to DMPTool

DMPTool (https://dmptool.org) stands for Data Management Planning Tool. It's tagline is:

Create, review, and share data management plans that meet institutional and funder requirements.

We should perhaps look into uploaded our resource sharing plans to DMPTool. We can also browse their public DMPs at https://dmptool.org/public_dmps.

@strasser mentioned the Digital Curation Centre (DCC) in https://github.com/lab-carpentry/blueprint-resourcesharing/pull/1#issuecomment-266153553.
So noting this resource here. They have guides such as How to Develop a Data Management and Sharing Plan


[Bug] [UI-Next] [V1.0.0-Alpha] Backend locale should be consistent with ui

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

The backend uses the cookie name language to get the locale in dolphinscheduler-ui.
What you expected to happen
above.
How to reproduce
above.
Anything else
No response
Version
dev
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I will fix it


Cost conversion to VS code editor API

Different parts of using the new Editor API (https://github.com/microsoft/vscode/issues/77131)

Implement save/saveAs - 1 day
Fire edit events - 1 day
Support undo/redo - 3 days
Change nativeEditorProvider into a WebViewCustomEditorProvider - 1 day
What to do about blank files? (Question out to VS code for answer)
Write tests that mock the new API but still open notebook editors - 3 days

Here's the API I based my costing on:
/**
 * Defines the editing functionality of a webview editor. This allows the webview editor to hook into standard
 * editor events such as `undo` or `save`.
 *
 * @param EditType Type of edits. Edit objects must be json serializable.
 */
interface WebviewCustomEditorEditingDelegate<EditType> {
	/**
	 * Save a resource.
	 *
	 * @param resource Resource being saved.
	 *
	 * @return Thenable signaling that the save has completed.
	 */
	save(resource: Uri): Thenable<void>;

	/**
	 * Save an existing resource at a new path.
	 *
	 * @param resource Resource being saved.
	 * @param targetResource Location to save to.
	 *
	 * @return Thenable signaling that the save has completed.
	 */
	saveAs(resource: Uri, targetResource: Uri): Thenable<void>;

	/**
	 * Event triggered by extensions to signal to VS Code that an edit has occurred.
	 */
	readonly onEdit: Event<{ readonly resource: Uri, readonly edit: EditType }>;

	/**
	 * Apply a set of edits.
	 *
	 * Note that is not invoked when `onEdit` is called as `onEdit` implies also updating the view to reflect the edit.
	 *
	 * @param resource Resource being edited.
	 * @param edit Array of edits. Sorted from oldest to most recent.
	 *
	 * @return Thenable signaling that the change has completed.
	 */
	applyEdits(resource: Uri, edits: readonly EditType[]): Thenable<void>;

	/**
	 * Undo a set of edits.
	 *
	 * This is triggered when a user undoes an edit or when revert is called on a file.
	 *
	 * @param resource Resource being edited.
	 * @param edit Array of edits. Sorted from most recent to oldest.
	 *
	 * @return Thenable signaling that the change has completed.
	 */
	undoEdits(resource: Uri, edits: readonly EditType[]): Thenable<void>;
}

export interface WebviewCustomEditorProvider {
	/**
	 * Resolve a webview editor for a given resource.
	 *
	 * To resolve a webview editor, a provider must fill in its initial html content and hook up all
	 * the event listeners it is interested it. The provider should also take ownership of the passed in `WebviewPanel`.
	 *
	 * @param resource Resource being resolved.
	 * @param webview Webview being resolved. The provider should take ownership of this webview.
	 *
	 * @return Thenable indicating that the webview editor has been resolved.
	 */
	resolveWebviewEditor(
		resource: Uri,
		webview: WebviewPanel,
	): Thenable<void>;

	/**
	 * Controls the editing functionality of a webview editor. This allows the webview editor to hook into standard
	 * editor events such as `undo` or `save`.
	 *
	 * WebviewEditors that do not have `editingCapability` are considered to be readonly. Users can still interact
	 * with readonly editors, but these editors will not integrate with VS Code's standard editor functionality.
	 */
	readonly editingDelegate?: WebviewCustomEditorEditingDelegate<unknown>;
}

namespace window {
	/**
	 * Register a new provider for webview editors of a given type.
	 *
	 * @param viewType  Type of the webview editor provider.
	 * @param provider Resolves webview editors.
	 * @param options Content settings for a webview panels the provider is given.
	 *
	 * @return Disposable that unregisters the `WebviewCustomEditorProvider`.
	 */
	export function registerWebviewCustomEditorProvider(
		viewType: string,
		provider: WebviewCustomEditorProvider,
		options?: WebviewPanelOptions,
	): Disposable;
}

Not sure how we validate costing :P. But yeah it's done so validating and closing.


Dry run for nox

Hello.
I have been using nox for a while now and I would love to see a possibility to dry run nox; the feature which exists in many different shell tools e.g. rsync, git clean, etc.
How would this feature be useful?

For me personally, this future will be useful mostly in two areas:

During the development of noxfile.py, especially when the same noxfile.py is used for many different python distributions and the logic is more complex.
For automated verification of CI/CD workflows using nox with e.g. dynamically created nox parameter to filter the sessions. It will allow us to create tests that can verify if nox works as expected without really need to run it.

Describe the solution you'd like
Add a new parameter -n, --dry-run to the nox.
$ nox --dry-run -s tests-3.8
nox > Would run session tests-3.8
nox > Would create virtual environment (virtualenv) using python3.8 in .nox/tests-3-8
nox > pip install -r requirements-test.txt
nox > pip install -e .[tox_to_nox]
nox > pytest --cov=nox --cov-config .coveragerc --cov-report= tests/
nox > Would run session cover
nox > Would create virtual environment (virtualenv) using python3 in .nox/cover
nox > pip install coverage
nox > Would run multiple sessions:
nox > * tests-3.8
nox > * cover

The exit code should be always 0.
From the source code we could detect dry run by checking the session property:
@nox.session(python="3.8")
def tests(session):
    """Run test suite with pytest."""
    if session.dry_run:
        ...
    else:
        ...

The solution described above has one problem. We can not prevent people from having e.g. shutil.rmtree(...) in the function without first checking session.dry_run, so whether it is really dry run depends on how noxfile.py is written. We can only vouch for nox API, not the logic inside the noxfile.py. Even though it is still very useful functionality, however above limitation should be pointed out in the documentation and help message.
Describe alternatives you've considered
Another alternative that does not have the same problem is something similar to --collect-only from pytest. In that case, we do not need to execute the session functions, just collect them.
$ nox --collect-only -s tests-3.8
nox > Would run session tests-3.8

The exit code should be always 0.
Contributing
I can start working on implementing this if I have a green light.
Best regards,
Maciej

I like this idea! A couple of notes:

nox > Would run session tests-3.8

Let's not change the logging messages everywhere as it'll create unnecessary complexity and branches. Let's just have nox output a message at the beginning and end of a run saying that it's in dry run mode and nothing is actually happening.

Even though it is still very useful functionality, however above limitation should be pointed out in the documentation and help message.

Agreed.


Prep Course Retrospective

As this is the first time we organized the Data Science Prep course, a retrospective would be very useful to know what went well and what could be better. Though there are a few lessons to learn from here and there, the course went pretty well and I am super happy that we have helped quite a few people to enter a whole new career path that they might have not been able to. So thank you everyone for your tremendous help and support (especially Chi and Pedro for the great idea)!
Results
We have 25 students who enrolled in the Prep Course ended up passing the exam. With this number alone, I think we really should continue with the course. Not only that the course would potentially feed 50% of qualified applicants to the Academy, it also opens the door for people from non-engineering background to start learning Data Science.
What went well

Marketing: With only 2 weeks to promote the course, we managed to get 150+ people signed up thanks to the amazing work from the Marketing team!
Community: I am very impressed with the level of engagement among the students. They read the wiki, followed the slack usage rules and proactively helped out fellow students without waiting for the instructors to come in
Organization: The course started and ended as planned. 5 live sessions (4 weekly + 1 bonus to solve the first wave exams) also went smoothly

What can be improved

Curriculum: After speaking with several students, I came to the conclusion that Dataquest's exercises are too basic and not enough to prepare them for the exam. We can still continue using Dataquest's learning materials, but definitely need to provide more advanced exercises that are on par with the exam
Expectation management:

Time requirement: I've received complaints from students that they have spent much more than the 30 hours mentioned in the Prep Course wiki page, which were estimated as the amount of time required to go through the Dataquest mission only. So next time, preparation time for the entrance exam needs to be factored in as well
Difficulty level: Some students found the exam to be unexpectedly hard, even though I have warned them many times that it would be much difficult than Dataquest exercises. So more emphasis on this is definitely needed

Student participation: Students are not asking enough questions! There was one week we received very few questions. Also, not many questions were raised in the live sessions. This might be due to the fact that Dataquest's exercises were so easy that students did not have the need to ask anything
Application process: We had 150 students enrolled in the Slack workspace, yet quite a lot of them have dropped out at some point during the course. I think a big part of this is because we allowed basically everyone to enter in the beginning, without having to pay for Dataquest upfront. Next time we should be stricter in the application process, not to dilute the slack work space with inactive members and to be able to derive meaningful metrics (e.g. the percentage of students that pass the test)
Curriculum: When selecting what would go into the Prep Course, I read through the exam but did not have the chance to actually do it - my bad on this one! So it is super important that the next person in charge of the Prep course to try out the exam in order to cover everything in the exam, and provide exercises that are equally hard.

If you have any comment, especially the instructors for Slack Q&A or live sessions, let me know :)

Nice retrospective Minh! Thanks so much for putting it together! I've got a few comments:

The course should be MUCH harder. I think that DataQuest will probably not be a good fit for us because their business model (monthly subscription) makes it difficult for them to make the courses as hard as they probably should be since the harder it is, the more people that drop dataquest, the less money they make.
I think that we should make our own course next year. We can probably re-use the same portal and infrastructure that we're using this time around for the Academy to make our own python course exactly as difficult as it needs to be.

I've also had some conversations with other instructors who feel like the difficulty of the entrance exam was too easy partly because we needed to cater to the students of the python course. In order to get our first real answer on this one, we'll need to see how many of the 25 students that went through the prep course drop out on the bootcamp requirements which are VERY difficult.
All in all, excellent first try! Thanks so much to Minh for making all of this happen and for all of the instructors who contributed in the live sessions and otherwise!

Thanks for the comment, Sam!

While Dataquest is a natural and good option to get started, it does not meet our requirements for the Academy, mostly because the exercises are too easy and come with step-by-step instructions. So I agree that we should create our own course next year, which should be very similar to the course you taught at Nova! This will give us the flexibility to adjust how hard we want it to be without depending on a third-party provider.

I understand that some instructors would feel this way, as by making the exam harder, we would raise the bar and reduce the number of people dropping out due to their technical skills. At the same time, we also need to make sure that those who put in enough effort during the prep course can pass the exam. However, the first part, which is to ensure all successful applicants meet our standards and manage through the Academy, is much more important and should come first.

As you said Sam, depending on the performance of those 25 prep course students as well as the ones that got into through second wave exam (since the first wave is closer what we are aiming for), we can see how hard the next exam should be. Then, the prep course should be designed accordingly (e.g. how much harder it should be, how long it should last, etc.) so that both goals are achieved.

Outstanding retrospective indeed (and congratulations on the results)!
I'm personally pro having our own set of content, especially as it would allow us to open source it and help anyone who wants to learn (DataQuest is not expensive, but the act of payment may still block some people). And indeed the level of difficulty of the materials needs to go up.
I'd challenge the idea that the test was too easy though. When I first started in DS I would barely have passed this Python test. We will see how our 25 peeps and peepettes fare :)

@minhhoang1023 really nice retrospective :)
I think you guys have already mentioned the important points:

Dataquest was waaay too easy
Using custom content would be the ideal way to move forward
Next time around we should have the exam ready before hand, so that the course can properly prepare the students for it
It's important to understand how many of the prep course students are going to make it through the academy until the end, to see if the course was enough as preparation for the academy

Regarding the exam difficulty:

the first one might have been a bit too hard, not because of the content but because of the way the questions were structured (there were many cases of all or nothing in terms of points)
the second wave might have been a bit too easy, given that: 1) it was heavily inspired on the first wave one, 2) we shared possible answers for the first wave exam, 3) the questions were better structured, i.e, it was possible to get partial points if the students didn't manage to solve a question as a whole

Just one thought about how do we evaluate the success of the prep course. Given the format we had this year, where there was too little time and too much to learn, I don't like to use the number of people who passed the academy's exam as a measure of success.
I'll even go a step further and say that having 25 people at the academy who learned Python through the prep course is much better than I was expecting...
We have many learnings to take from this prep course, and one of them is that we should definitely do it next year. And let's make sure that next year's one will be (even) better!
I think you did an amazing job with the prep course :)

@dominguezmaria Great comments! Yours pretty much summed up the main lessons for next year :)
I talked to some students and they also have the same view on the exam difficulty. The first one is hard, but not because of the question itself. If we can have the difficulty level of first wave with the structure of the second wave (e.g. partial credits, meaningful feedback), it would be just right.

Well done with this retrospective everyone! This thread will be invaluable for next year so let's remember that it exists :)

Getting some closure.
@Juliana-R if there are things from this issue that should go into Batch 5 please add them as separate issues.


ICSP added

Signed-off-by: Matthew Frahry<EMAIL_ADDRESS>

Broken until oneview-golang gets updated


[Bug]: Client Name Overrides are failing if the property is inherited(alias) from parent

Describe the bug
In the Typescript SDK Generation, I am trying to override the names of some objects/models generated. But, the compiler is not able to find the properties of the object when the properties are inherited from the parent.
Reproduction
The following code does not work. Playground Link is here.
import "@typespec/http";
import "@typespec/rest";
import "@typespec/versioning";
import "@azure-tools/typespec-azure-core";
import "@azure-tools/typespec-client-generator-core";

using Azure.ClientGenerator.Core;
using TypeSpec.Http;
using TypeSpec.Rest;
using TypeSpec.Versioning;
using Azure.Core;
using Azure.Core.Traits;

model A {
  ...KeyVaultCertificateBase
}

alias KeyVaultCertificateBase = {
 NBF?: float32
};

@@clientName(A.NBF, "nbf");
@@clientName(A.NBF, "nbf", "javascript");

But the following code works:
import "@typespec/http";
import "@typespec/rest";
import "@typespec/versioning";
import "@azure-tools/typespec-azure-core";
import "@azure-tools/typespec-client-generator-core";

using Azure.ClientGenerator.Core;
using TypeSpec.Http;
using TypeSpec.Rest;
using TypeSpec.Versioning;
using Azure.Core;
using Azure.Core.Traits;

model A {
  NBF?: float32;
}

@@clientName(A.NBF, "nbf");

In the first code, the property is inherited. Only in such cases, the compiler errors out with the following message.
C:\Users\......>npx tsp compile client.tsp
TypeSpec compiler v0.55.0

Diagnostics were reported during compilation:

C:/Users/..../propertyNameOverride.tsp:429:93 - error invalid-ref: Model doesn't have member nBF
> 429 | @@clientName(Microsoft.EventGrid.SystemEvents.KeyVaultCertificateNewVersionCreatedEventData.nBF,
      |                                                                                ^^^

Found 1 error

Please help me to resolve this issue.
Additional Info
Please note the following code works:
import "@typespec/http";
import "@typespec/rest";
import "@typespec/versioning";
import "@azure-tools/typespec-azure-core";
import "@azure-tools/typespec-client-generator-core";

using Azure.ClientGenerator.Core;
using TypeSpec.Http;
using TypeSpec.Rest;
using TypeSpec.Versioning;
using Azure.Core;
using Azure.Core.Traits;

model A {
  ...KeyVaultCertificateBase
}

model KeyVaultCertificateBase {
 NBF?: float32
};

@@clientName(A.NBF, "nbf");
@@clientName(A.NBF, "nbf", "javascript");

The code fails only when alias is used.
@xirzec @lmazuel @joheredi FYI.....
Checklist

[X] Follow our Code of Conduct
[X] Check that there isn't already an issue that request the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion.
[X] The provided reproduction is a minimal reproducible example of the bug.

This is not just an issue for tcgc @clientName, any decorator that is trying to decorate on spread alias properties defined as above is problematic. See this @encodedName example

@markcowl link to symbol refactor

This may be fixed by https://github.com/microsoft/typespec/pull/2640

dup of this  #4764


(ci) Add linux-arm64 support

Similar to #242, this adds support for linux-arm64
Closes #241

Tested in qemu; the generated .tar.gz from this PR is installable and seems to start fine (albeit be incredibly slow, because of the full system emulation involved when simulating an arm64 on top of amd64/x64 hardware). Merging now, it'll be interesting to see how this performs on real/Raspberry Pi 4 hardware.

Raspberry Pi 4 Model B Rev 1.1:
$ cat /proc/cpuinfo
...

Hardware	: BCM2835
Revision	: c03111
Serial		:<PHONE_NUMBER>3f612b
Model		: Raspberry Pi 4 Model B Rev 1.1

$ uname -a
Linux rpi4 5.4.0-1046-raspi #50-Ubuntu SMP PREEMPT Thu Oct 28 05:32:10 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

$ ./perlang -V
Perlang 0.1.0-dev.193 (built from git commit 002448e) on .NET 6.0.1

  Number of detected (v)CPUs: 4
  Running in 64-bit mode: True
  Operating system info: Unix <IP_ADDRESS>6

$ time ./perlang -v
0.1.0-dev.193

real	0m0.289s
user	0m0.223s
sys	0m0.052s

$ time python3 -V
Python 3.8.10

real	0m0.010s
user	0m0.001s
sys	0m0.009s

No Python 2.7 installed on this one.

Thanks! :+1: For reference, here are also some timings from an Oracle ARM/Ampere-based cloud instance (running in eu-frankfurt-1, availability domain AD-2).
ubuntu@instance-20220111-2050:~$ cat /proc/cpuinfo
processor	: 0
BogoMIPS	: 50.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x3
CPU part	: 0xd0c
CPU revision	: 1

ubuntu@instance-20220111-2050:~$ uname -a
Linux instance-20220111-2050 5.11.0-1022-oracle #23~20.04.1-Ubuntu SMP Fri Nov 12 15:45:47 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

ubuntu@instance-20220111-2050:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

ubuntu@instance-20220111-2050:~$ $HOME/.perlang/nightly/bin/perlang -V
Perlang 0.1.0-dev.193 (built from git commit 002448e) on .NET 6.0.1

  Number of detected (v)CPUs: 1
  Running in 64-bit mode: True
  Operating system info: Unix <IP_ADDRESS>2

ubuntu@instance-20220111-2050:~$ time $HOME/.perlang/nightly/bin/perlang -v
0.1.0-dev.193

real	0m0.081s
user	0m0.079s
sys	0m0.000s

ubuntu@instance-20220111-2050:~$ time python3 -V
Python 3.8.10

real	0m0.001s
user	0m0.000s
sys	0m0.001s

Similarly as on your Ubuntu 20.04 system, no Python 2 was available. But man, these Ampere-powered machines are pretty fast!!! :sunglasses:


possible issue with az cli command

The steps to create windows cluster using PS works prefect for self-sign cert case. Also I used the same PS steps to create linux cluster. This confirm Linux vnet-linuxcluster.json and vnet-linuxcluster.parameters.json are good.
Now, when I try to create Linux cluster with self-sign cert using az command and templates, I ran into error "The primary certificate parameters names in the parameters file should be specified with'sourceVaultValue','certificateThumbprint','certificateUrlValue',if the secondary certificate parameters are specified in the parameters file, the parameters names should be specified with'secSourceVaultValue','secCertificateThumbprint','secCertificateUrlValue'"
I tried using az command on windows machine, linux machine, cloud shell and I get the error in every case if I am try to use the template with self-sign cert.
However, I was able to use az command to create cluser with self sign cert using the command from the az doc "az sf cluster create -g group-name -n cluster1 -l westus --cluster-size 4 --vm-password Password#1234 --certificate-output-folder MyCertificates --certificate-subject-name cluster1" this command did not use the template.
I am just wondering if we should highlight the case of self sign cert and get script that is working

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 86106acd-e6ec-2931-b5fa-344d42479095
Version Independent ID: 912adec9-666a-e9d5-e814-a24d75add164
Content: Create a Linux Service Fabric cluster in Azure
Content Source: articles/service-fabric/service-fabric-tutorial-create-vnet-and-linux-cluster.md
Service: service-fabric
GitHub Login: @rwike77
Microsoft Alias: ryanwi

Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.


Create an About page

Describe the Feature you want to end

I want to add an about page here.
Expected behavior

Screenshots (optional)

Additional context (optional )

Can I contribute under Hacktoberfest'22?

pls assign me this issue to create an about page @zugzwang03... I want to contribute for hacktoberfest2022


[Impeller] Vulkan swapchain recreation is incorrect when rotating screen.

I/ViewRootImpl@def1a0[MainActivity](19287): MSG_RESIZED_REPORT: frame=(112,0,2280,1080) ci=(0,63,126,0) vi=(0,63,126,0) or=2
I/ViewRootImpl@def1a0[MainActivity](19287): MSG_RESIZED_REPORT: frame=(112,0,2280,1080) ci=(0,63,126,0) vi=(0,63,126,0) or=2
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
I/ViewRootImpl@def1a0[MainActivity](19287): ViewPostIme pointer 0
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR
E/flutter (19287): [ERROR:flutter/impeller/base/validation.cc(49)] Break on 'ImpellerValidationBreak' to inspect point of failure: Could not present queue: SuboptimalKHR

based on https://community.khronos.org/t/vk-suboptimal-khr-is-it-safe-to-use-it-as-window-resize-detection/107848 it looks like this happens when the surface properties change so this is expected?

Handling the rotation change and recreating the swapchain is easy enough, however there seem to be two caveats:

We also need to take the transform and plumb it through to the CTM for rendering to be correct.
Rotating back to portrait doesn't seem to always trigger the suboptimalkhr which means we might need to poll.

The expectation was that we wouldn't be in the business of figuring that out post acquisition. If the surface is suboptimal, just treat the present result as a success. There is going to be another frame incoming shortly anyway where the AcquireNextDrawable call will fail and the swapchain impl recreated.
Perhaps the fix is as simple as just adding a fallthrough in the present that returns success?

xref: https://developer.android.com/games/optimize/vulkan-prerotation

Discussed with @jonahwilliams this AM.
This problem still persists, because VK_SUBOPTIMAL_KHR is provided inconsistently (it only works at all on Android 10+, and there are reports that some devices that should support this enum do not). To focus on fidelity over performance at the moment, we're likely going to adopt polling - in the future we'll want to pre-rotate images (https://github.com/flutter/flutter/issues/129580).

FYI it appears to work fine on a Pixel 5, I might need to use a Samsung phone to see this particular issue?

@htoor3 I think you accidentally re-closed this?

@matanlurey hmm that's strange, that was definitely not on purpose! It appears that I updated my fork of the engine and GH automations auto closed the issue?


[KO Number] Korean Ordinal refinements

all test cases pass.

@aitelint Can you resolve the conflicts to merge? Thanks.

conflicts solved


Runner bloqued when trying to delete vm's

Codecov Report

Merging #91 (7a20a27) into main (5a882a2) will decrease coverage by 0.19%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main      #91      +/-   ##
==========================================
- Coverage   70.83%   70.64%   -0.20%
==========================================
  Files          12       12
  Lines         727      729       +2
==========================================
  Hits          515      515
- Misses        212      214       +2

Impacted Files
Coverage Δ

srcs/runners_manager/vm_creation/openstack.py
26.53% <0.00%> (-0.56%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a882a2...7a20a27. Read the comment docs.


Add validation to disk's name in CreateVMWizard Storage step

This PR adds validations to disk name such as in NIC name in Network Step.
The rules for the validation taken from the documentation.
In case of error it will show error message, mark the field in red and not let user save the edited disk and move to the next step:

If you have any questions please feel free to ask
Fixes: #1329

Hello contributor, thanks for submitting a PR for this project!
I am the bot who triggers "standard-CI" builds for this project.
As a security measure, I will not run automated tests on PRs that are not from white-listed contributors.
In order to allow automated tests to run, please ask one of the project maintainers to review the code and then do one of the following:

Type ci test please on this PR to trigger automated tests for it.
Type ci add to whitelist on this PR to trigger automated tests for it and also add you to the contributor white-list so that your future PRs will be tested automatically. ( keep in mind this list might be overwritten if the job XML is refreshed, for permanent whitelisting, please follow #3 option )
If you are planning to contribute to more than one project, maybe it's better to ask them to add you to the project organization, so you'll be able to run tests for all the organization's projects.

ci add to whitelist

@sgratch , @bamsalem , @rszwajko - The VM name regexp needs to be changed back.  VMs can have names that do not contain latin alpha characters.  In fact, the regexp is a copy from webadmin's VM name validation.
I'm not sure if disk names can contain non-latin alpha characters.  I did not test that, or lookup the validation code, yet.

@sgratch , @bamsalem , @rszwajko - The VM name regexp needs to be changed back.  VMs can have names that do not contain latin alpha characters.  In fact, the regexp is a copy from webadmin's VM name validation.
I'm not sure if disk names can contain non-latin alpha characters.  I did not test that, or lookup the validation code, yet.

@bamsalem @sjd78 @rszwajko

is there any reason for using VM_NAME_REGEX.test(nameCandidate) within isVmNameValid function and /^[a-zA-Z0-9_-.]{1,255}$/.test(nameCandidate) within isDiskNameValid function? We can check the length limitation separately and share the same regex...

You are all right, based on the fact that disk alias string is auto generated from the vm name ({vmName}_Disk1) then we should support UTF for both disk name and vm name.
And by checking the code:
disk: https://github.com/oVirt/ovirt-engine/blob/e48484ccb3c1c8fcaed81023b234295a109ebd8b/frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/storage/DisksAllocationModel.java#L486-L490
vm: https://github.com/oVirt/ovirt-engine/blob/9059df4d26f5d327c87495ee9e19cb1876ad92b7/frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/vms/VmModelBehaviorBase.java#L1665-L1667
Both use I18NNameValidation().
So in our web-ui code we should use VM_NAME_REGEX.test(nameCandidate) for both validations

Regarding the message we display different messages because the message in Create VM wizard must be shorter than DiskImageEditor, that more descriptive

I'm totally ok with having the shorter message version for both dialogs. They both cover the same info and it's better to maintain one message instead of two (if validation login will be ever changed).

@bamsalem @sjd78 @rszwajko

is there any reason for using VM_NAME_REGEX.test(nameCandidate) within isVmNameValid function and /^[a-zA-Z0-9_-.]{1,255}$/.test(nameCandidate) within isDiskNameValid function? We can check the length limitation separately and share the same regex...

You are all right, based on the fact that disk alias string is auto generated from the vm name ({vmName}_Disk1) then we should support UTF for both disk name and vm name.
And by checking the code:
disk: https://github.com/oVirt/ovirt-engine/blob/e48484ccb3c1c8fcaed81023b234295a109ebd8b/frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/storage/DisksAllocationModel.java#L486-L490
vm: https://github.com/oVirt/ovirt-engine/blob/9059df4d26f5d327c87495ee9e19cb1876ad92b7/frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/vms/VmModelBehaviorBase.java#L1665-L1667
Both use I18NNameValidation().
So in our web-ui code we should use VM_NAME_REGEX.test(nameCandidate) for both validations

Regarding the message we display different messages because the message in Create VM wizard must be shorter than DiskImageEditor, that more descriptive

I'm totally ok with having the shorter message version for both dialogs. They both cover the same info and it's better to maintain one message instead of two (if validation login will be ever changed).

I changed it to use VM_NAME_REGEX again in diskName and vmName.
I changed the code according to your review @sgratch @sjd78 @rszwajko...

I'm totally ok with having the shorter message version for both dialogs. They both cover the same info and it's better to maintain one message instead of two (if validation login will be ever changed).

Now the same message displayed in CreateVm and editVm:

I changed it to use VM_NAME_REGEX again in diskName and vmName.
I changed the code according to your review @sgratch @sjd78 @rszwajko...

I'm totally ok with having the shorter message version for both dialogs. They both cover the same info and it's better to maintain one message instead of two (if validation login will be ever changed).

Now the same message displayed in CreateVm and editVm:

@bamsalem please rebase

@bamsalem please rebase

@sgratch
rebased.

LGTM


323 create discussions page

A basic component for the createPost page with two fields :

title of the post
editor with post content

and a button to save the tutorial in database.

looks like it should be mostly good, I'll review this evening...

Made one minor change (rename db to afs as I'm currently refactoring how the database works), in another update I'll move the code that interacts with the database out of the component and to the discussion store but for now it works fine.


Why unlink and unlinkall not triggering delete function

I think it would be much better if unlink() and unlinkall() in  the ActiveRecord trigger the delete function in the Model .

When does the issue occur?
when I use unlinkAll('posts', true) in the model to delete all related posts in beforeDelete() function and the related item has children that must be deleted before I delete the Item
for example :
single category has many posts , single post has many details
in Category::beforeDelete() $this->unlinkAll('posts', true);
What do you see?
DB error : Cannot delete or update a parent row: a foreign key constraint fails. thats because post has details that must be deleted before
What was the expected result?
I expected that unlinkAll will trigger delete() function in Post model so I could delete related post details

Can't be done since there are many ways dealing with that. You can solve it in PHP code or you can solve it on DB constraints level by specifying ON DELETE CASCADE for foreign keys.

@samdark
I believe you are addressing a use case rather than the underlying "issue".
For example, using bedezign/yii2-audit to track ActiveRecord changes. As far as I can tell, AuditTrailBehavior reacts to ActiveRecord::EVENT_AFTER_DELETE which is not fired when the relation is unlinked.
Example "Junction Table":
        return $this->hasMany(TasksAccountsAssigned::className(), ['TaskId' => 'Id']);
    }
    public function getAccountsAssigned() {
        return $this->hasMany(Accounts::className(), ['Id' => 'AccountId'])
            ->via('tasksAccountsAssigned');
    }

so then
$this->unlinkAll('accountsAssigned', true);
doesn't produce records in the Audit extension, which normally produces Deletes.
Leads me to believe that ActiveRecord::EVENT_AFTER_DELETE is not getting fired on Delete of TasksAccountsAssigned which is issued by the unlinkAll(... , true).
If there is truly no way for unlink to fire ActiveRecord::EVENT_AFTER_DELETE, then how would you propose I go about my issue. If it's not too much trouble ;)

@IGaMbLeRI what you've described is not related to this issue. The issue was about not deleting at all, not about triggering events. For events triggering you may create a separate issue.


G2P model?

I am very impressed by your work. I am wondering what package you used or model you used to convert any language to IPA.

Mostly a combination of Epitran (https://github.com/dmort27/epitran) and CharsiuG2P (https://github.com/lingjzhu/CharsiuG2P)


Retry executing DP builds to avoid conn reset errors

We get a lot of connection reset errors during maven builds, like
[ERROR] Failed to execute goal on project receiver: Could not resolve dependencies for project dev.knative.eventing.kafka.broker:receiver:jar:1.0-SNAPSHOT: Failed to collect dependencies at io.vertx:vertx-web-client:jar:4.0.0.CR1: Failed to read artifact descriptor for io.vertx:vertx-web-client:jar:4.0.0.CR1: Could not transfer artifact io.vertx:vertx-web-client:pom:4.0.0.CR1 from/to central (https://repo.maven.apache.org/maven2): Transfer failed for https://repo.maven.apache.org/maven2/io/vertx/vertx-web-client/4.0.0.CR1/vertx-web-client-4.0.0.CR1.pom: Connection reset -> [Help 1]

and this makes our test grid red and it's difficult to see what's actually failing.
Proposed Changes

Retry executing DP unit tests to avoid conn reset errors

/lgtm
/approve


Add minimal load-time DLL support on Windows, support dllimport storage class

This patch is required for minimal support of load-time dynamic linking on Windows. For this to work,

Prepare a directory, distinct from the current Crystal installation's CRYSTAL_LIBRARY_PATH, which we will use to place the .lib and .exp files. Set CRYSTAL_LIBRARY_PATH to point to it;
Build the minimal dynamic libraries needed by the runtime. Run the following in Powershell: (this is based on the CI job)git clone --config core.autocrlf=false -b v8.2.0 https://github.com/ivmai/bdwgc bdwgc
git clone --config core.autocrlf=false -b v7.6.10 https://github.com/ivmai/libatomic_ops bdwgc\libatomic_ops
cd bdwgc
cmake . -DBUILD_SHARED_LIBS=ON -Denable_large_config=ON -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL -DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF
cmake --build . --config Release
cd ..

Invoke-WebRequest https://cs.stanford.edu/pub/exim/pcre/pcre-8.43.zip -OutFile pcre.zip
7z x pcre.zip
Move-Item pcre-* pcre
cd pcre
cmake . -DBUILD_SHARED_LIBS=ON -DPCRE_BUILD_PCREGREP=OFF -DPCRE_BUILD_TESTS=OFF -DPCRE_SUPPORT_UNICODE_PROPERTIES=ON -DPCRE_SUPPORT_JIT=ON -DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF
cmake --build . --config Release
cd ..

Copy ./bdwgc/Release/gc.lib and ./pcre/Release/pcre.lib to the CRYSTAL_LIBRARY_PATH we just set. Same for the associated .exp files;
Build a test program:> echo puts ARGV[0].match(/../) > pcretest.cr
> crystal build pcretest.cr -Dpreview_dll -Dwithout_iconv

Place gc.dll and pcre.dll in the same directory as the pcretest.exe executable;
Run it!> pcretest 123
Regex::MatchData("12")

Without this patch, the compilation step would fail with:
_main.obj : error LNK2019: unresolved external symbol pcre_malloc referenced in function __crystal_main
_main.obj : error LNK2019: unresolved external symbol pcre_free referenced in function __crystal_main
G-C-.obj : error LNK2019: unresolved external symbol GC_stackbottom referenced in function .2A.GC.3A..3A.current_thread_stack_bottom.3A.Tuple.28.Pointer.28.Void.29..2C..20.Pointer.28.Void.29..29.

This PR thus adds the dllimport storage class to any external lib variables, through LLVM.
The -Dpreview_dll flag is also added because currently the compiler defaults to static builds on Windows, even without the --static build flag, and everything would immediately break if the current command line suddenly starts using dynamic linking. With a minimal working example we can later sort out the details for dynamic linking on Windows.

The instructions to build PCRE2, which is necessary since Crystal 1.8.0, are:
git clone --config core.autocrlf=false -b pcre2-10.42 https://github.com/PCRE2Project/pcre2.git pcre2
cd pcre2
cmake . -DBUILD_STATIC_LIBS=OFF -DBUILD_SHARED_LIBS=ON -DPCRE2_BUILD_PCRE2GREP=OFF -DPCRE2_BUILD_TESTS=OFF -DPCRE2_SUPPORT_UNICODE=ON -DPCRE2_SUPPORT_JIT=ON -DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF
cmake --build . --config Release
cd ..
copy pcre2\Release\pcre2-8.lib .
copy pcre2\Release\pcre2-8.dll .


chore(halyard): remove references to halyard files

Resolves Jira: [DOCS-440]
We'll definitely want to spend some time w/ the netlify preview to make sure things are looking good

@dogonthehorizon You mean to leave the Halyard content in these files, right?

https://deploy-preview-1470--armory-docs.netlify.app/armory-enterprise/installation/guide/upgrade-spinnaker/
https://deploy-preview-1470--armory-docs.netlify.app/armory-enterprise/installation/guide/upgrade-oss-to-armory/
https://deploy-preview-1470--armory-docs.netlify.app/armory-enterprise/installation/guide/install-on-aws/

@aimeeu the AWS and upgrade OSS guides yes, since we have no alternative at this time. The upgrade spinnaker page no, I'll remove that now.


Some apps crashing or not getting push notifications

I'm re-installing CM13 with no GAPPS and I'm discovering some problems with apps I usually use from Google Play (which I know is expected). Unfortunately it seems to be that app developer's simply use Google Play services without serious thought if they needed it at all.
Currently I detected 3 affected apps:
Signal

pops up warning stating that it needs Google Play services to work properly
blocks from activating the app, rendering it useless

ProtonMail

pops up warning stating that it needs Google Play services to work properly every time the app is restarted
doesn't receive push notifications

Revolut

pops up warning stating that it needs Google Play services to work properly every time the app is restarted
app hangs and crashes when clicking on Top Up

If there is any relevant info that I could post here to help with this issue, please tell me so. Thanks!

I was able to extract the Manifest for Revolut and ProtonMail, here's the references I found for Google Services:
Revolut
<action android:name="com.google.android.c2dm.intent.RECEIVE"/>

<meta-data android:name="com.google.android.gms.analytics.globalConfigResource" android:resource="@xml/analytics"/>

<meta-data android:name="com.google.android.gms.version" android:value="@integer/google_play_services_version"/>

ProtonMail
<action android:name="com.google.android.c2dm.intent.RECEIVE"/>

<action android:name="com.google.android.gms.iid.InstanceID"/>

<meta-data android:name="com.google.android.gms.version" android:value="@integer/google_play_services_version"/>

<provider android:authorities="ch.protonmail.android.google_measurement_service" android:exported="false" android:name="com.google.android.gms.measurement.AppMeasurementContentProvider"/>

<action android:name="com.google.android.gms.measurement.UPLOAD"/>

<service android:enabled="true" android:exported="false" android:name="com.google.android.gms.measurement.AppMeasurementService"/>

Funny enough, I got Revolut Top Up to work, just need to be connected to a WiFi network as it seems. Doesn't work every time, but some parts of the app only work if connected through WiFi while the others through Cellular connection. Strange as this doesn't seem to make sense...

Surprised Signal and ProtonMail have GMS dependancies. Looks like they're both working on making it  to F-droid but haven't arrived yet. I know its not really an answer but something else that may work soon.
(Signal)[https://fdroid.eutopia.cz/]
(ProtonMail)[https://f-droid.org/forums/topic/protonmail/]

ProtonMail is planning on open-sourcing their mobile apps (both iOS and Android) but have no planned date for that. As for the F-Droid store I'm guessing they don't even have it on their roadmap, at least from the response they sent me and given the fact that they depend on Google Services.
As for Signal I think they'll never publish to the F-Droid, because they prefer to have the security the Play Store provides and using it as the only mean to get and publish the app. An example of that is the Silence fork that exists in both Play Store and F-Droid, but Signal only exists in the Play Store.

Hello, I've been using Signal and ProtonMail on CM13 with microG without issues for a long time, it's kind of surprising that you've been experiencing issues.

@metaphorex What versions of both apps are you running? Does the Google Play Services pop up appear to you as well? What are the settings for your microG app? Maybe I'm missing something...

@bytemybits I'm running 1.4.1 version of ProtonMail and 3.17.0 of Signal which are the newest I believe. I'm also running version 0.2.4-7 of microG GmsCore. In microG settings I've enabled device checkin and Google Cloud Messaging, I've also allowed all permissions except location. I don't think that version of microG or any of the apps matte, because I've been using all of them for quite a long time now and I've never had any issues nor popups at all, while I've had with some other apps.

@bytemybits Are all microG self-check boxes ticked? Do you have BlankStore or FakeStore installed?

@ArchangeGabriel I have Device Check-in and Cloud messaging enabled (I'm getting push notifications in slack and WhatsApp for example) and for the Self-check I have everything checked except:

Play Store (Phonesky)
System supports location provider
Network-based location enabled
All these three puzzle me as I did install FakeStore and have location services working (tested it with Google Maps). Also I'm running version 0.2.4-7 of microG GmsCore.

IIRC FakeStore (Phonesky) is required for Play Services to be recognized.
I found that the the Spoof package signature permission has to be granted manually for the FakeStore package via Settings > Apps > (wrench icon) > App permissions > Spoof package signature

@NormalRa I do have FakeStore installed but I don't see Spoof package signature listed in the App permissions. As CM doesn't natively support Signature Spoofing I installed Needle as per instructions in the [Wiki] (https://github.com/microg/android_packages_apps_GmsCore/wiki/Signature-Spoofing) page.
Do I still need the [Xposed Framework] (http://forum.xda-developers.com/showthread.php?t=3034811)?

@bytemybits is the problematic point in Installed pacakges "Play Store (Phonesky) installed" or "Play Store (Phonesky) has correct signature"?

@mar-v-in The first one, the second doesn't even exist (guessing because the first one ins't active). So that means I do have to install Xposed right?

No, this is not related to signature spoofing then. The system does not see the FakeStore package installed. Try uninstalling and reinstalling it.

@bytemybits Apologies, I should have mentioned that the procedure is valid only for cm-13 built with the signature spoofing patch applied.

@mar-v-in that did the trick! The setting is active in microG and I no longer have the issues I described. Revolut still crashes when doing Top Up but at this point I think it's an app bug, not an issue with microG.
Thank you all very much!


e2e: remove specs interdependency

005-mobile-chat-options.spec.ts expects that a chat message from spec 001 is present. This can cause random failures if spec 001 has not finished before 005 starts. Also, specs should be independent so that they can be run individualy.
This change consists in adding one step to spec 005 where the chat message is created. The test then continues from the 2nd ship by reacting to that message.
Tested locally by running npm run e2e:debug
We should wait for CI to run off of this branch to confirm.
PR Checklist

[ ] Includes changes to desk files
[x] Describes how you tested the PR locally (test ship vs livenet)
[ ] If a new feature, includes automated tests
[x] Comments added anywhere logic may be confusing without context

Test failing for obscure reason.
005-mobile-chat-options.spec.ts:14:5 › Post chat message as group guest
Error: Timed out 5000ms waiting for expect(received).toBeVisible()
Call log:
  - expect.toBeVisible with timeout 5000ms
  - waiting for getByText('hi there')

Investigating.

This should still be addressed but isn't working as is. Going to close for now, referenced issue will remain open in Linear.


Added Sliding Window Problem

Solution for issue #270
The sliding window problem has been added with Problem statement, code, test cases and complexities.
The code is written in C++.

change the readme please @matrix2131


Progress bar is corrupt.

I've tried everything to fix this-- I've gone into the code, redownloaded the script four times, even changed PS versions. I can't get any of my work done! Any help would be appreciated.
picture of the error:

picture of the progress bar code:

I also have this issue.

I have that same issue, and @soychicka's solution doesn't work for me.

** @soychicka**'s solution doesn't work for me either. I tried it but it just ended up telling me there was no progress_bar.json file but it was right there! Help please!

We've had a lot of changes recently, if this is still an issue for some of you please file a new issue with the OS/PS versionn


correct the CF link

Dear Richard
I respect your respecting the existing text! I think that's the right thing to do regarding the conventions, but since this new version may well supersede the existing 1.6 document, or be maintained together with it, I think it makes sense to correct the link.
Best wishes
Jonathan

Perfect :+1:
Thanks for the fix. :smile:


Peer-to-Peer across NAT

Good article:
http://www.brynosaurus.com/pub/net/p2pnat/
Thanks

The title caught my eye, so I will give my 2 cents. Just pasting a link for a feature request seems to be not very helpful or inspiring. But more importantly how would hole punching NAT become a Godot feature? That is an architectural feature of the game and seems completely outside the scope of a game engine. But maybe I am wrong and let me know how if I am.
If you are serious about this feature request I would bet you might need to explain in detail what you would want to see from Godot. I am watching because I may learn something.

I believe (and I'm not the only one) that Godot does lack higher-level networking. For now, it'd be somewhat strange to smash in this feature, but IIRC there'll be a revamp in the network module to include higher-level stuff (which will likely include NAT hole punching).
It is a feature a game engine can provide. Construct 2, for example, does have NAT traversal.
Though don't expect this to be included soon in Godot.

we'll add better higher level networking eventually

I think UPnP protocol support should be added to Godot. With UPnP you can get external IP address and set port forwarding on most routers.

Reopening, as this  is about UPnP too.