Discuss Entity sets

This is a first pass at trying to provide some Set functionality to help handle the complexity of organizing different sets of material properties and boundary conditions.

Former mfem_ext helper functions were templated to output both mfem::Array (which is currently used in serac) and std::vector to be used with the Set abstraction. (This relies on #437.)
Set operations are computed and converted back to a dense representation.

Two simple tests were implemented. One just evaluates different types of set operations. The second test creates a flag by using Sets to facilitate creation of a flag.
Internally a list of attributes is converted to an unordered_map<attribute, index list>. So Set<int>({1, 2, 4, 3}) gets converted to  { {1, {0}},    {2, {1}},   {4, {2}},    {3, {3}}}.
This allows us to grab a list of elements that have a particular attribute and apply set operations to those lists. Alternatively we can selectively assemble or perform other operations using this list as input.
Note: It might be worthwhile in the future to convert this implementation to better take advantage of c++20 ref_views.

Codecov Report

Merging #419 (16fe53d) into develop (275f060) will increase coverage by 0.25%.
The diff coverage is 95.74%.

@@             Coverage Diff             @@
##           develop     #419      +/-   ##
===========================================
+ Coverage    92.60%   92.85%   +0.25%
===========================================
  Files           48       48
  Lines         2392     2380      -12
===========================================
- Hits          2215     2210       -5
+ Misses         177      170       -7

Impacted Files
Coverage Δ

...tegrators/displacement_hyperelastic_integrator.cpp
84.37% <ø> (ø)

src/serac/coefficients/coefficient_extensions.cpp
62.50% <66.66%> (-17.81%)
:arrow_down:

src/serac/coefficients/coefficient_extensions.hpp
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 275f060...16fe53d. Read the comment docs.

@jonwong12 , should we start looking at this now that the templated MFEM extensions branch is merged?

Codecov Report

Merging #419 (01edcaf) into develop (c18fb17) will increase coverage by 0.42%.
The diff coverage is 97.98%.

:exclamation: Current head 01edcaf differs from pull request most recent head b93f1c3. Consider uploading reports for the commit b93f1c3 to get more accurate results

@@             Coverage Diff             @@
##           develop     #419      +/-   ##
===========================================
+ Coverage    92.71%   93.14%   +0.42%
===========================================
  Files           47       48       +1
  Lines         2430     2553     +123
===========================================
+ Hits          2253     2378     +125
+ Misses         177      175       -2

Impacted Files
Coverage Δ

src/serac/infrastructure/set.hpp
97.39% <97.39%> (ø)

src/serac/coefficients/coefficient_extensions.hpp
98.82% <98.79%> (+3.36%)
:arrow_up:

...c/serac/physics/utilities/finite_element_state.cpp
100.00% <100.00%> (ø)

src/serac/physics/solid.cpp
94.96% <0.00%> (-0.02%)
:arrow_down:

src/serac/physics/solid.hpp
100.00% <0.00%> (ø)

src/serac/infrastructure/input.hpp
100.00% <0.00%> (ø)

src/serac/physics/thermal_solid.cpp
97.22% <0.00%> (ø)

src/serac/physics/thermal_solid.hpp
100.00% <0.00%> (ø)

src/serac/physics/thermal_conduction.hpp
100.00% <0.00%> (ø)

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c18fb17...b93f1c3. Read the comment docs.

I'm still having some issues understanding how this would be used in a real simulation. Could you describe a realistic use case (materials, boundary conditions, etc)?

@jamiebramwell Great question. I think being able to go to and from attribute lists to subsets is generally useful. I think it would be useful for dispatching GPU kernels.
I'm not sure about realistic scenarios, but if you had an example of a multi-material problem that would be difficult in MFEM, perhaps I could sketch out how it could be used.


OS X 10.10 (Yosemite) + MacVim + vim-airline + fullscreen = blank space where airline should be

From @GoogleCodeExporter on March 16, 2015 9:25
What steps will reproduce the problem?
Under OS X 10.10:

1. Build MacVim with homebrew
2. Install vundler and use vunder to install vim-airline.
3. Now when you run MacVim, you should see the airline header just fine until
you go into fullscreen mode with the MacVim window, at which point the header
line is blacked out.

What is the expected output? What do you see instead?
Expected: Airline header should be visible in fullscreen mode (this worked
correctly in Mavericks)
Instead: see attached screen shot

What version of MacVim and OS X are you using (see "MacVim->About MacVim"
and  "Apple Menu->About This Mac" menu items, e.g. "Snapshot 40, 10.5.6
Intel")?

MacVim: Custom Version 7.4 (73)
OS X: Yosemite Beta 10.10, version 3

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 22 Sep 2014 at 1:47
Attachments:

macvim_bug.png

Copied from original issue: douglasdrumond/macvim#512

From @GoogleCodeExporter on March 16, 2015 9:25
Following up with some more info:

If I open a new tab with :tabnew, this forces airline to redraw and suddenly it
appears again. I still think this is a MacVim / Yosemite issue (since this
didn't happen with Mavericks), but there is a workaround.

Original comment by<EMAIL_ADDRESS>on 24 Sep 2014 at 6:04

From @GoogleCodeExporter on March 16, 2015 9:25
Yes I noticed this in each public beta of Yosemite (10.10). The problem repros
with a vanilla vim config. FWIW, VimR -- if I understand correctly, it uses
some kind of macvim framework -- doesn't have this problem.

Original comment by<EMAIL_ADDRESS>on 7 Oct 2014 at 6:56

From @GoogleCodeExporter on March 16, 2015 9:25
This happens without airline, too. The first line of the file is missing, which
makes editing pretty annoying. Same diagnostics as n.harris. Willing to do any
texting/experimenting for more information.

Thanks!

Original comment by<EMAIL_ADDRESS>on 3 Oct 2014 at 9:14

From @GoogleCodeExporter on March 16, 2015 9:25
any solution in view?

Original comment by<EMAIL_ADDRESS>on 18 Oct 2014 at 6:34

From @GoogleCodeExporter on March 16, 2015 9:25
:tabnew does not work for me unfortunately. I can however temporarily fix the
issue by changing 'lines' to 999. From the manual:

    You can use this command to get the tallest window possible:
       :set lines=999

Note that you will have to enter that every time you switch between windowed
and fullscreen mode. Unfortunately it does not seem possible to really automate
this.

Original comment by<EMAIL_ADDRESS>on 18 Oct 2014 at 11:53

From @GoogleCodeExporter on March 16, 2015 9:25
As a workaround you can decrease/increase the font size and the bar will appear.

Original comment by<EMAIL_ADDRESS>on 2 Nov 2014 at 11:47

From @GoogleCodeExporter on March 16, 2015 9:25
Is anyone going to fix this issue. This is very annoying!

Original comment by<EMAIL_ADDRESS>on 27 Oct 2014 at 6:15

From @GoogleCodeExporter on March 16, 2015 9:25
I have same problem on Yosemite, both :tabnew or :set lines=999 work as
workaround, but have to do it on every switch

Original comment by<EMAIL_ADDRESS>on 18 Oct 2014 at 1:45

From @GoogleCodeExporter on March 16, 2015 9:25
This is very annoying. Every time I start vim and i have to increase or
decrease the font size.
Kindly fix this issue asap (Macvim + yosemite)

Original comment by<EMAIL_ADDRESS>on 6 Nov 2014 at 5:42

From @GoogleCodeExporter on March 16, 2015 9:25
I'm experiencing the problems mentioned above, and in addition to that, I also
get the same line rendered numerous times (see attachment)

This only happens in fullscreen mode, windowed mode works fine.

Original comment by<EMAIL_ADDRESS>on 9 Nov 2014 at 11:14
Attachments:

[Screen Shot 2014-11-09 at 23.11.43.png](https://storage.googleapis.com/google-code-attachments/macvim/issue-512/comment-10/Screen Shot 2014-11-09 at 23.11.43.png)

From @GoogleCodeExporter on March 16, 2015 9:25
I have this problem too in Yosemite,

Original comment by<EMAIL_ADDRESS>on 12 Nov 2014 at 12:49

From @GoogleCodeExporter on March 16, 2015 9:25
+1 here.. for both the missing line at the top and the multiple repeated lines
on the body when I scroll

Original comment by<EMAIL_ADDRESS>on 12 Nov 2014 at 2:06

From @GoogleCodeExporter on March 16, 2015 9:25
#13

This is a good way.

Original comment by<EMAIL_ADDRESS>on 30 Dec 2014 at 7:25

From @GoogleCodeExporter on March 16, 2015 9:25
I have the missing line problem. Using a config from skwp's dotfile repo
"yadr".  Thanks<EMAIL_ADDRESS>for the font size workaround, saved me
hours of switching to a new editor. Still annoying to have an extra step after
fullscreen, but usable until we get a real fix.

Current Process (workaround):
[Command] + [Control] + [F]
[Command] + [-]
[Command] + [=]

Desired Process:
[Command] + [Control] + [F]

Original comment by<EMAIL_ADDRESS>on 16 Nov 2014 at 7:30

From @GoogleCodeExporter on March 16, 2015 9:25
This mapping works well for me.

nnoremap <silent> <F5> :set lines=999 columns=999<CR>

Original comment by<EMAIL_ADDRESS>on 9 Jan 2015 at 5:12

From @GoogleCodeExporter on March 16, 2015 9:25
Also experienced this, but an easy quickfix is to simply turn of the native
fullscreen support (cmd+, for preferences and then remove the checkmark under
advanced).

It does mean that macvim will occupy the active space instead of creating a new
fullscreen one, but i find that a minor problem

Original comment by<EMAIL_ADDRESS>on 10 Feb 2015 at 8:24

From @GoogleCodeExporter on March 16, 2015 9:25
I found a fix from
http://shorts.jeffkreeftmeijer.com/2014/full-screen-macvim-on-yosemite/
It works with me.

Original comment by<EMAIL_ADDRESS>on 16 Feb 2015 at 10:45

This appears to be the same issue as reported in PR #8, which has a proposed fix.

Closing as duplicate of #8


Latest version 0.2.1 is not yet available in pypi to use

Hi Team,
Since we wanted to use the functionality of multiple commands using pyvyos, why it is not still  available in the pypi registry?

And please add examples how to use to handle lists of operations within a single payload functionality.
Can you please look into that.
Regards,
Anu

Second this, the latest 0.2.1 has not been updated on the pypi registry

Hi, I will update it this week.

hello, while I don't update pip you can install 0.21:
pip install git+https://github.com/robertoberto/pyvyos.git@8ae0b4f237eb20d9ad9f511154df8eb554bde398

done https://pypi.org/project/pyvyos/ updated


Repo links to empty repo

There is a 404 for Linux deb package. This has been known for weeks now.
The repo links on the install instructions page point to empty packages
All Packages files are 0 bytes in length (empty)
As a result, Teams is uninstallable from repo

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0c86f401-cff4-93b3-ff69-b9787a87e644
Version Independent ID: c0808bd3-cf63-0625-4c3c-65469c26fb3e
Content: Get clients for Microsoft Teams - Microsoft Teams
Content Source: Teams/get-clients.md
Service: msteams
GitHub Login: @dstrome
Microsoft Alias: dstrome

Duplicate of https://github.com/MicrosoftDocs/OfficeDocs-SkypeForBusiness/issues/9772


Display issues during free kicks

A free kick has three states: 0, where the robots can stop any infringements and stand up, 1, where the robots can walk to their positions, and 2, where the robot can kick.
In the display, however

state 0 is not indicated at all
state 1 is shown as state 0
state 2 is correctly shown as state 2
In addition, the remaining time for state 1 is only shown on the left, but for state 2, the time is also shown on the right. This should probably be unified.
This is rather low priority because it only affects the display, however, during the public streaming at the competition, it might be confusing for spectators when no indication of state 0 is shown.

While I agree that this issue does not impact game play, it makes it really difficult to follow the situation for teams and lead to a difficult understanding of the situation by both teams and developers.

@omichel the remaining time in free kick [0] still seems to be missing


[SPARK-30325][CORE] markPartitionCompleted cause task status inconsistent

What changes were proposed in this pull request?
Fix task status inconsistent in executorLost which caused by markPartitionCompleted
Why are the changes needed?
The inconsistent will cause app hung up.
The bugs occurs in the corer case as follows:

The stage occurs during stage retry, scheduler will resubmit a new stage with unfinished tasks.
Those unfinished tasks in origin stage finished and the same task on the new retry stage hasn't finished, it will mark the task partition on the current retry stage as succesuful in TSM successful array variable.
The executor crashed when it is running tasks which have succeeded by origin stage, it cause TSM run executorLost to rescheduler the task on the executor, and it will change the partition's running status in copiesRunning twice to -1.
'dequeueTaskFromList' will use copiesRunning equal 0 as reschedule basis when rescheduler tasks, and now it is -1, can't to reschedule, and the app will hung forever.

Does this PR introduce any user-facing change?
No
How was this patch tested?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

cc @dongjoon-hyun @wangshuo128 @squito

@Ngone51 Please look at this patch, thanks!

I think I can confirm this is a bug and it's caused by we adding the sched.markPartitionCompletedInAllTaskSets logic, that when a task attempt from one TSM succeeded it shall mark the partition as completed for all the TSMs targeting the same Stage. Unfortunately the missing part is we didn't try to kill the running task attempts when we mark the partitions as completed, thus when the running task attetmpts failed with ExecutorLost it would revert the completed partition result (which is not necessary).
To me the best solution here would be to kill all the running task attempts in the TSM inside method markPartitionCompleted, this would resolve the issue without any side affect.
Also cc @squito @cloud-fan @Ngone51

I think I can confirm this is a bug and it's caused by we adding the sched.markPartitionCompletedInAllTaskSets logic, that when a task attempt from one TSM succeeded it shall mark the partition as completed for all the TSMs targeting the same Stage. Unfortunately the missing part is we didn't try to kill the running task attempts when we mark the partitions as completed, thus when the running task attetmpts failed with ExecutorLost it would revert the completed partition result (which is not necessary in this case).
To me the best solution here would be to kill all the running task attempts for the completed partition in the TSM inside method markPartitionCompleted, this would resolve the issue without any side affect.
Also cc @squito @cloud-fan @Ngone51

Expecting your code review!

OK to test

I'm not sure killing tasks can work. There is no guarantee that a task can always be killed successfully. And even if we can, we may send out the kill request, and immediately get the executor lost event before the task is killed.
I think we should accept the fact that a running task may be useless as its corresponding partition is completed, and deal with it well. e.g. when seeing executor lost, don't reschedule tasks whose corresponding partitions are already completed.

seems speculative task has the same issue? We may have multiple running tasks for one partition.

I'm not sure killing tasks can work. There is no guarantee that a task can always be killed successfully. And even if we can, we may send out the kill request, and immediately get the executor lost event before the task is killed.
I think we should accept the fact that a running task may be useless as its corresponding partition is completed, and deal with it well. e.g. when seeing executor lost, don't reschedule tasks whose corresponding partitions are already completed.

I think it doesn't matter, if driver  immediately get the executor lost event before the task is killed, the TSM will  handleFailedTask and will not scheduler it;
Btw, app process the task success or failed status in handleSuccessfulTask or handleFailedTask if the task finished before killed; In handleSuccessfulTask, we mark it as Killed(another stage succeeded), in handleFailedTask, we will not reschedule the task.

seems speculative task has the same issue? We may have multiple running tasks for one partition.

markPartitionCompleted kill logic is like speculative task, when we finished the other mutiple running tasks, we ignore and mark it Killed(another attempt succeeded) in handleSuccessfulTask or won't scheduler it in handleFailedTask.

@cloud-fan I mean add the running task attempts to killedByOtherAttempt, thus when they failed,  the completed partitions won't be affected.

I was of the opinion that the when a task is started by a previous stage and still in progress, no subsequent retries from other stage attempt must be made, unless it is fate is known.
To know if the partition is already assigned to some task, the MapStatus entry for the partition could denote the intermediate step.(As of now MapStatusEntry is either null or filled, kind of boolean. I think we can have the third stage).
I am not sure if there is any issue in making such assumption. I can try to elaborate the design if needed.
By this proposed model, we can have the compute resources also saved(no need to start a redundant computation if one stage attempt is already working on it). However, we allow speculation as its within same stage attempt.

I think the direct problem is, the task status is inconsistent: a task can satisfy both successful(task.index) and task.running. Such tasks will be handled twice in executorLost and mess up the internal status.
A simple fix can be: change task.running to !successful(task.index) && task.running in executorLost

@cloud-fan The task status is inconsistent indeed, however we need to avoid Resubmitted instead of handleFailedTask,

change task.running to !successful(task.index) && task.running in executorLost
change this will cause Resubmit and rescheduler the task again, handleFailedTask won't resheculer it, WDYT ?

I don't think it's safe to not reschedule it. Looking at the comment in executorLost, we want to reschedule because the shuffle files are all lost in this executor. The special case is killedByOtherAttempt.contains(tid), which means a speculative task has finished on another executor. For the stage attempt, there is no guarantee that 2 tasks of the same partition will be run on different executors.

@cloud-fan I think you are right ! I will process it.

@cloud-fan PTAL, I have a deep think and remove the kill logic, and in handleSuccessfulTask I think we needn't handle this case, it can overwrite the shuffle meta info, beause if the executor keeping the partition shuffle data was lost, we can Resubmit this partition in current stage instead of reschedule the partition in the next stage by FetchFailedException
cc @HyukjinKwon  @jiangxb1987 @Ngone51

ok to test

LGTM. Can you update the PR description?

Test build #115765 has started for PR 26975 at commit 5e73417.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20558/
Test PASSed.

LGTM. Can you update the PR description?

@cloud-fan thanks for your review!

@srowen @maropu @vanzin @HyukjinKwon @dongjoon-hyun
PLAT.

Test build #115773 has started for PR 26975 at commit 623639a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20567/
Test PASSed.

Test build #115765 has finished for PR 26975 at commit 5e73417.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115765/
Test PASSed.

Test build #115773 has finished for PR 26975 at commit 623639a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115773/
Test PASSed.

Is it possible to add a new test case in TaskSetManagerSuite ?

There are multiple corner cases not handled by current solution:
Image we have two TSMs (M1 and M2) working on the same Stage, and for the corresponding tasks are notated as T1 and T2 for a specific partition:

T1 and T2 might be scheduled on different executors (E1 and E2), T1 has been finished but T2 is still running. Then E2 get lost, in the approach suggested by this PR, the partition in M2 will be marked as not successful and a new pending task would be added, which is actually not necessary because the shuffle files are on E1;
T1 and T2 might be scheduled on the same executor, T1 has been finished but T2 is still running. Then the executor get lost, since T2 is still running the partition will not be marked as not successful. After a while maybe another task finished and mark the TSM as finished, but actually the shuffle files get lost, thus it lead to a new regression.

I haven't get a solution here. I'm thinking whether we can put the successful task information into taskInfos inside markPartitionCompleted, if this is possible then the second problem I mentioned above could probably get resolved.

There are multiple corner cases not handled by current solution:
Image we have two TSMs (M1 and M2) working on the same Stage, and for the corresponding tasks are notated as T1 and T2 for a specific partition:

T1 and T2 might be scheduled on different executors (E1 and E2), both tasks have been finished. Then E2 get lost, in the approach suggested by this PR, the partition in M2 will be marked as not successful and a new pending task would be added, which is actually not necessary because the shuffle files are on E1;
T1 and T2 might be scheduled on the same executor, T1 has been finished but T2 is still running. Then the executor get lost, since T2 is still running the partition will not be marked as not successful. After a while maybe another task finished and mark the TSM as finished, but actually the shuffle files get lost, thus it lead to a new regression.
I haven't get a solution here. I'm thinking whether we can put the successful task information into taskInfos inside markPartitionCompleted, if this is possible then the second problem I mentioned above could probably get resolved.

For case 1, if T2 successfully completed after T1, will T2 overriders T1's MapStatus?
For case 2, I haven't understand yet:

since T2 is still running the partition will not be marked as not successful.

So, the partition has marked as successful by T1?

After a while maybe another task finished and mark the TSM as finished

Assuming there's a T0 from M0 finished, but mark which TSM as finished? TSM1 or TSM2 or both?

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20719/
Test PASSed.

Test build #115928 has started for PR 26975 at commit 92e30c3.

@jiangxb1987 @Ngone51
For case 1, I think it is acceptable, if T1 and T2 both finished, the E1 and E2 have the same chance to get lost, if E2 get lost, we can Resubmit at current stage; however if E1 got lost, we can perceive only by next stage got FetchFailedException to rescheduler the partition, it is high cost.
For case 2, I has the save question as @Ngone51

Note that, a task roughly has 2 states: running and finished. a partition has 2 states as well: successful or not.
TaskSetManager.successful tracks the successfulness of each partition, and is kind of synced among all TSMs, via TaskSetManager.markPartitionCompleted.
It's obviously a bug that TaskSetManager.executorLost checks the status of task and partition separately. A task may satisfy both conditions and be handled twice.
For case 1: I think it's OK as the new task will override the map status (the new task has a bigger epoch). It's a waste of resources, but it's better than hang.
For case 2: I don't think it can happen. If T1 finished first, the partition in TSM2 will be marked as successful too.

Test build #115928 has finished for PR 26975 at commit 92e30c3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115928/
Test PASSed.

Let me expand on case 2:
If T1 finished first, the partition in TSM2 (notated as P1) will be marked as successful too. Then the executor get lost, since T2 is still running, we won't change successful(P1) to false.
Then, possibly another partition in TSM2 could be marked as successful by other tasks, then TSM2 think all the partitions has been finished, but actually P1 has been lost and not computed again.

Let me expand on case 2:
If T1 finished first, the partition in TSM2 (notated as P1) will be marked as successful too. Then the executor get lost, since T2 is still running, we won't change successful(P1) to false.
Then, possibly other partitions in TSM2 could be marked as successful by other tasks, then TSM2 think all the partitions has been finished, but actually P1 has been lost and not computed again.

There another two cases in this situation as follows:

T1 and T2 run on different executors, it doesn't matter.
T1 and T2 run on same executor, T2 will not retry since T1 has succeeded.
Think like this situation:
A stage has finished and then an executor holding the stage's shuffle file got lost, we can't rescheduler since it has finished, we will retry by next stage got FetchFailedException.
This case like this we disscussed, we won't reschedule the task in the finished TSM, I think it is similar.

So, I think this is reasonable, what do you think?

If T1 finished first, but T2 is still running, and executor is lost. There are 2 cases:

the stage is already finished, then we will hit fetch failure later and retry stage.
the stage is still running. Then both TSM1 and TSM2 call executorLost. TSM1 will resubmit a task for P1. TSM2 may mark itself as finished. This is OK as we still have a task submitted for P1.

@jiangxb1987 @Ngone51 @cloud-fan

retest this please

Test build #116652 has started for PR 26975 at commit 92e30c3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21431/
Test PASSed.

Test build #116652 has finished for PR 26975 at commit 92e30c3.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116652/
Test FAILed.

retest this please

Test build #116679 has started for PR 26975 at commit 92e30c3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21458/
Test PASSed.

Test build #116679 has finished for PR 26975 at commit 92e30c3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116679/
Test PASSed.

thanks, merging to master!


Capture all logging

This makes it possible to redirect all logging during tests by making kubestatus logging (and an errant "Deploy succeeded!" puts) use a settable logger.

We don't have any disagreement on what this PR is actually changing, so I'm going to merge it. Let's continue the discussion re: whether one or both loggers should include timestamps and/or other info elsewhere.


feat(documention): add form-textarea stories

Preview environments are getting posted here, as soon as they are ready!


Foundation macros

We've employed a better macro system for asm2bf, there are some foundation macros missing  though.
I'll strive to make some macros that resemble the old C preprocessor system, while trying to keep the fresh wind blowing.
Possibly one will be able to enable so-called legacy mode, where lua code will try to behave as close as possible to the original C preprocessor behaviour.
Foundation macros are necessary, because right now there isn't even a way to include a source code chunk.

[X] #include
[X] #call


feat: New react-popper API

New API is render props based and it looks like this.
I also replaced PropTypes with Flow types.
You can play with it running yarn demo, or use it in your project installing<EMAIL_ADDRESS>Feedbacks welcome!
<Manager>
  <Target>
    {({ getTargetRef }) => (

        Target

    )}
  </Target>
  <Popper placement={this.state.placement}>
    {({ getPopperRef, style, placement }) => (

        Popper
        <Arrow>
          {({ getArrowRef, style }) => (

          )}
        </Arrow>

    )}
  </Popper>
</Manager>

edit:
cool, I just tried this new code with React 15, it actually works. You just have to wrap Target and Popper in a div:
<Manager>

    <Target />
    <Popper />

</Manger>

To do:

[x] Update TypeScript definitions (need help!)
[x] Ship Flow types definitions
[x] Add tests

I updated the TypeScript d.ts file but it's failing because HTMLElement can't be assigned to HTMLButtonElement... Any clue on how to tell TS to accept ANY HTML element?

This is looking great! I've been meaning to look at a newer API similar to what you have to solve https://github.com/souporserious/react-popper/pull/70. How do you feel about moving all of the render props up into one component? So something like:
<Popper>
  { ({ getTargetProps, getPopperProps }) =>
    <Fragment>

Target

Popper

    </Fragment>
  }
</Popper>

I was thinking this could be a little simpler and get rid of all of the context stuff, but maybe the API is fine the way it is 😅.

Thanks for the feedback Travis.
What would it be the advantage of having functions that return props instead of objects?
About the single component API, I think it's a cool idea to reduce boilerplate, I hope it's not too different from what people are used to, what do you think?

@souporserious may you enable Travis CI tests?

I guess I was thinking getPopperProps() would generate a unique id or something so you could have multiple poppers with one target. Might be better to just keep them in components since it's already setup for that.
Pertaining to the Travis CI tests, do I just need to signup for an account and add it in the settings section of this repo? 😇 sorry, I've never setup CI testing 💩

Yes you should find the instructions in the settings of the repo.
Good point about multiple Poppers, at the moment we lost this ability with the new-new API 🤔

I guess we may have something like this then, so that we reduce the boilerplate but we keep the ability to have multiple poppers into a single Manager?
<Manager>
  <Target>
    {({ getTargetRef }) => (

        Target

    )}
  </Target>
  <Popper>
    {({ popperProps, arrowProps }) => (

        Popper

    )}
  </Popper>
</Manager>

Here's an explanation for why prop getters are better: “How to give rendering control to users with prop getters” https://blog.kentcdodds.com/how-to-give-rendering-control-to-users-with-prop-getters-549eaef76acf

Thanks for the reply @kentcdodds! The challenge here is to connect the "popper" and "arrow" elements together.
I can have a getPopperProps() prop getter that allows me to create multiple poppers into a single "context", but if I want to associate an arrow to a specific popper things get tricky.
<Popper>
  ({ getReferenceProps, getPopperProps }) =>
	<Fragment>

arrow?

arrow?

	</Fragments>
</Popper>

Yeah I think this is perfect. If you want to go the other route, you could look at how it's done in react-measure now. Basically we keep a list of each call to bind and reuse the stored functions, but in react popper this would require naming each popper call which could be more tedious over just using a component that can take care of it. So I guess what I'm saying is I think what you've originally proposed is great 😇.


Opacity values < 1 being changed

Hello!
I've noticed that the plugin changes opacity values from .#number to 0.#number, for example:
opacity: .75 becomes opacity: 0.75
SCSS lint demands that the former is in place, so it would be useful if it didn't manipulate these values.
Cheers,
Dave

+1

👍 THE PROJECT NEEDS UPDATES ⚠


Add missing metrics into statsd-reporter.

Description

StatsD metrics reporter is missing quite a number of metrics. This PR updates all the missing metrics.

Key changed/added classes in this PR

defaultMetricDimensions.json

This PR has:

[x] been self-reviewed.
[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[x] been tested in a test Druid cluster.

@gianm looks like everything is green. :)


setTime does not update slider knob position

Calling setTime on timelineSliderControl does not cause the position of the slider knob to update. Clicking prev/next will cause the knob to move correctly to the ticks before/after the updated time; however, the play button will play it from the original start time, not from the updated time.
Also, it seems that the setTime method needs to be called on both timelineSliderControl and timeline in order to update both. Perhaps setting the time of the control should automatically update the timeline, since one typically wouldn't want the timeline and slider to be out of sync.

I have the same problem, setTime works well and displays the data of the corresponding date, but it is not reflected in the timeline, which is a problem for me.

I managed to solve it by modifying the plugging code by adding this line in setTime
setTime: function(t) {
$('.time-slider').prop("value",t);//this line
this._sliderChanged({
type: "change",
target: {
value: t
}
})
},
the problem is that it would damage the program if you have more than two lines of time (I think), but for me it's enough


1st issue

@Terrygulang  Can you respond this issue?

@Terrygulang Can you respond this issue?

close the 1st issue.


Add  a license

Want to use alexa-scraper.py in a project and was wondering if it was possible to add a license to permit me to so?

Apologies for the delay. MIT license added.


How to covert Fastspeech2 to Onnx with dynamic input and output ?

How Can I get dynamic input in torch model to onnx model ?
I give input with dynamic_axes, but the output in inference is not dynamic.
My code
        input_names = ['speakers', 'texts', 'src_lens', 'max_src_len']
	output_names = ['output', 'postnet_output', 'p_predictions', 'e_predictions', 'log_d_predictions', 'd_rounded',
	                'src_masks', 'mel_masks', 'src_lens', 'mel_lens']
	dynamic_axes = {
		"texts": {1: "texts_len"},
		"output": {1: "output_len"},
		"postnet_output": {1: "postnet_output_len"},
		"p_predictions": {1: "p_predictions_len"},
		"e_predictions": {1: "e_predictions_len"},
		"log_d_predictions": {1: "log_d_predictions_len"},
		"d_rounded": {1: "d_rounded_len"},
		"src_masks": {1: "src_masks_len"}
	}

	texts_len = 10
	speakers = torch.tensor([0])
	texts = torch.randint(1, 200, (1, texts_len))
	text_lens = torch.tensor([texts_len])
	max_len = torch.from_numpy(np.array(texts_len)).to(device)
	torch.onnx.export(model, args=(speakers, texts, text_lens, max_len), f="./FastSpeech_2.onnx",
	                  input_names=input_names, output_names=output_names, dynamic_axes=dynamic_axes, opset_version=11)

In code , I use  src_lens=10, it is ok.
But in iinference with this onnx model，when I input with src_lens=50 or other, I get this error:
2022-01-18 16:29:38.644831855 [E:onnxruntime:, sequential_executor.cc:346 Execute] Non-zero status code returned while running Split node. Name:'Split_2888' Status Message: Cannot split using values in 'split' attribute. Axis=0 Input shape={27,256} NumOutputs=10 Num entries in 'split' (must equal number of outputs) was 10 Sum of sizes in 'split' (must equal size of selected axis) was 10
Traceback (most recent call last):
  File "torch2onnx_2.py", line 497, in <module>
    onnx_mode_test()
  File "torch2onnx_2.py", line 471, in onnx_mode_test
    ort_outs = ort_session.run(None, ort_inputs)
  File "/root/anaconda3/envs/tts_fffan/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 192, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Split node. Name:'Split_2888' Status Message: Cannot split using values in 'split' attribute. Axis=0 Input shape={27,256} NumOutputs=10 Num entries in 'split' (must equal number of outputs) was 10 Sum of sizes in 'split' (must equal size of selected axis) was 10

seems that the input len must be 10 , and it can't be dynamic
Does somebody help me ?

did you covert to onnx successful ？

@Tian14267 I have converted to onnx successful. However, when I runtime to inference model with onnx , Model required ['p_control', 'onnx::Cast_6'] while input_name don't have it

@Tian14267 The same problems with you. Besides, when I feed the input with same length but different text token,  the onnxruntime inference result is worse because of the mel output length seems like related with input data when trace onnx model. This problem has confused me for several weeks, does anyone can give me some ideas? tks!

@Tian14267 Have you found any way to fix it? i have the same problem as you

Because the output of duration_prediction of different tokens with same length is different. if we convert fastspeech2 to a single onnx model, we finally met the problems above. To fix this,  I split the fastspeech2 into three submodels when convert to onnx, and put the length_regulator inference out of onnxruntime. finally, build the sub models inference pipeline with onnxruntime.  hope it useful for you!


Not Defined

I am unable to see the following information.

Due in
Due at
latitude
longitude
Next Train

and all of the values returned are Not defined. I have attached my configuration below as well as the output of the test script.
sensor:
  - platform: gtfs_rt
    trip_update_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates'
    vehicle_position_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions'
    route_delimiter: '-'
    departures:
    - name: Grovely (Outbound)
      route: BNFG
      stopid: '600385'
      icon: mdi:train
      service_type: Train
    - name: Bowen Hills (Outbound)
      route: BNFG
      stopid: '600005'
      icon: mdi:train
      service_type: Train

INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: Grovely (Outbound), route id: BNFG, direction id: 600385
INFO:sensor:trip_update_url: https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates
INFO:sensor:vehicle_position_url: https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions
INFO:sensor:route_delimiter: ,
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: Grovely (Outbound)
INFO:sensor:   Route: BNFG
INFO:sensor:   Stop ID: 600385
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:train
INFO:sensor:   Service Type: Train
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Train: not defined
INFO:__main__:Adding Sensor: Name: Bowen Hills (Outbound), route id: BNFG, direction id: 600005
INFO:sensor:Sensor Update:
INFO:sensor:   Name: Bowen Hills (Outbound)
INFO:sensor:   Route: BNFG
INFO:sensor:   Stop ID: 600005
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:train
INFO:sensor:   Service Type: Train
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Train: not defined

Hi,
You are using the incorrect route id - it should be FGBN instead of BNFG

@andrewdrob10 were you ever able to figure this out? I have the same problem and cannot figure it out.
trip_update_url: 'https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb'
vehicle_position_url: 'https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb'
departures:
- name: '27N Upper James Northbound'
  route: '5083'
  stopid: '2807'
  service_type: 'Bus'

And output from the test script:
INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: 27N Upper James Northbound, route id: 5083, direction id: 2807
INFO:sensor:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:sensor:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:sensor:route_delimiter: None
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: 27N Upper James Northbound
INFO:sensor:   Route: 5083
INFO:sensor:   Stop ID: 2807
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:bus
INFO:sensor:   Service Type: Bus
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Bus: not defined

Hi, I believe @andrewdrob10 problem was an invalid combination of route and stop id.
The "not defined" message basically means that there is no data for the specified route and stop id combination in the TripUpdate message that was received.  This could be because:

It is an invalid combination; or
It is a valid combination but there are no matching services running at the moment, and therefore not in the realtime feed

I just had a look at the hamilton.ca feed and it is completely blank, because it is 1:00am in California.  I will look again in a few hours

Hi @mark1foley. Yes I figured that was the case but I cannot for the life of me figure out what I am doing wrong. The stop is 100% correct as it matches the right lat/lng on a map. The route might be incorrect but I've tested it with the other route id with no success. I assume there are two route id's for both directions, one being 5032 and the other 5083.
I'm getting my id values from the GTFS (static) data that I found here: https://googlehsrdocs.hamilton.ca/
Looking at the pdf schedule, service should be resumed now so the feed shouldn't be empty anymore.
Thanks for helping me out.

Hi @StarScream159 , just to be sure (i.e. eliminate my code) I downloaded the RT file from https://opendata.hamilton.ca/GTFS-RT and decoded it using the Google-supplied protoc tool and here are the results.
dump.txt
I agree that 5032 and 5083 are the correct route ids for route '27'.  However, stop id 2807 does not appear to be on that route.  I can successfully use test.py for stop id '1771':
INFO:main:Adding Sensor: Name: 27N Upper James Northbound, route id: 5083, direction id: 1771
INFO:main:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:main:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:main:route_delimiter: None
INFO:main:header: None
INFO:main:Successfully updated vehicle positions: 200
INFO:main:Successfully updated trip data: 200
INFO:main:Sensor Update:
INFO:main:   Name: 27N Upper James Northbound
INFO:main:   Route: 5083
INFO:main:   Stop ID: 1771
INFO:main:   Direction ID: 0
INFO:main:   Icon: mdi:bus
INFO:main:   Service Type: Bus
INFO:main:   unit_of_measurement: min
INFO:main:   Due in: 0
INFO:main:   Due at: 06:12
INFO:main:   Latitude: 43.25556945800781
INFO:main:   Longitude: -79.87076568603516
INFO:main:   Next Bus: not defined
So we now know that GTFS-RT service is working and the code is ok.  It's just a matter of getting the stop id correct.  They don't make it easy by putting stop numbers on the pdf timetables that are not the stop id in the stops.txt file.  But using the attached dump.txt (or alternatively run test.py with the -d DEBUG parameter) you can see the valid stop ids for route 5083.  You will need to match up these stop ids with the descriptions in the stops.txt file to find the one you want.
Hope that helps.

That does indeed help! Thanks so much for the dump from the RT file (I didn't know how to do that).
The PDF stop numbers are phone codes that you can use when you call a system standard phone number, then enter the stop code and an automated operator will tell you the time to the next bus. So yeah it's a little misleading. Weird the id's from the stops.txt file don't match up though.
While I suspect this issue is unique to my transit provider, I'll update this ticket when I figure out a solution.

I really appreciate your help, but I think I will give up on this. None of this is making any sense and I feel like it is due to my transit provider. Even your example routeId 5083 and stopId 1771 doesn't work for me now:
INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: 21N Upper Kenilworth Northbound, route id: 5083, direction id: 1771
INFO:sensor:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:sensor:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:sensor:route_delimiter: None
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: 27N Upper James Northbound
INFO:sensor:   Route: 5083
INFO:sensor:   Stop ID: 1771
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:bus
INFO:sensor:   Service Type: Bus
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Bus: not defined

Even if I use DEBUG and get a list of stops for a trip, using one of the stops doesn't work either:
DEBUG:sensor:   Received Trip ID 1829835 Route ID: 5083 direction ID 1 Start Time:  Start Date:
DEBUG:sensor:      Stop: 355796 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 355977 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 355976 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 2801 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 2817 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>

So using stopId 2817 which is 8 minutes in the future (when I did these tests), running it with that value produces:
INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: 21N Upper Kenilworth Northbound, route id: 5083, direction id: 2817
INFO:sensor:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:sensor:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:sensor:route_delimiter: None
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: 27N Upper James Northbound
INFO:sensor:   Route: 5083
INFO:sensor:   Stop ID: 2817
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:bus
INFO:sensor:   Service Type: Bus
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Bus: not defined

I can't make heads or tails of this and I've tried for hours. It's time to move on. Again though, appreciate the help and work you've put into this.

I really appreciate your help, but I think I will give up on this. None of this is making any sense and I feel like it is due to my transit provider. Even your example routeId 5083 and stopId 1771 doesn't work for me now:
INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: 27N Upper James Northbound, route id: 5083, direction id: 1771
INFO:sensor:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:sensor:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:sensor:route_delimiter: None
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: 27N Upper James Northbound
INFO:sensor:   Route: 5083
INFO:sensor:   Stop ID: 1771
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:bus
INFO:sensor:   Service Type: Bus
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Bus: not defined

Even if I use DEBUG and get a list of stops for a trip, using one of the stops doesn't work either:
DEBUG:sensor:   Received Trip ID 1829835 Route ID: 5083 direction ID 1 Start Time:  Start Date:
DEBUG:sensor:      Stop: 355796 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 355977 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 355976 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 2801 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>
DEBUG:sensor:      Stop: 2817 Stop Sequence: 0 Stop Time:<PHONE_NUMBER>

So using stopId 2817 which is 8 minutes in the future (when I did these tests), running it with that value produces:
INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: 27N Upper James Northbound, route id: 5083, direction id: 2817
INFO:sensor:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:sensor:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:sensor:route_delimiter: None
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: 27N Upper James Northbound
INFO:sensor:   Route: 5083
INFO:sensor:   Stop ID: 2817
INFO:sensor:   Direction ID: 0
INFO:sensor:   Icon: mdi:bus
INFO:sensor:   Service Type: Bus
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Bus: not defined

I can't make heads or tails of this and I've tried for hours. It's time to move on. Again though, appreciate the help and work you've put into this.

Hi @mark1foley,
Sorry to re-open such an old issue (and one I've apparently hijacked lol).
I've moved and thought I'd give this a go again.
Still trying to make heads-or-tails of what I'm getting from the DEBUG output with my new route and stop id's and it just isn't making sense. Hopefully, if you don't mind, you can shed some light on it.
Data from: https://opendata.hamilton.ca/
Route Name: UPPER KENILWORTH
Route Id: 5121 or 5230 (I assume one for North Bound and one for South Bound)
Stop Name: UPPER KENILWORTH opposite ELLIOTT
Stop Id: 2845 (I'm 99% sure this is the correct stop for the correct direction)
I got the above values from unziping the static GTFS data and importing the data into excel and looking at the Id columns.
RouteId 5121 I don't see any details about it in the DEBUG output when I write it to a file. Meaning I can't find lines that have that RouteId listed:
DEBUG:sensor:   Received Trip ID 1892170 Route ID: 5219 direction ID 0 Start Time:  Start Date:

There are none like this that have that RouteId. They are all different RouteId's.
So trying RouteId 5230 I see results, further I see more then 1 entry with a differing direction ID... 1/0:
DEBUG:sensor:   Received Trip ID 1896212 Route ID: 5230 direction ID 1 Start Time:  Start Date:
DEBUG:sensor:   Feed Route ID 5230 changed to 5230
...
DEBUG:sensor:   Received Trip ID 1896172 Route ID: 5230 direction ID 0 Start Time:  Start Date:
DEBUG:sensor:   Feed Route ID 5230 changed to 5230

I've copied the output from the DEBUG command with the following config to a pastepin for reference.
trip_update_url: 'https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb'
vehicle_position_url: 'https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb'
route_delimiter: '-'
departures:
  - name: '21 Upper Kenilworth'
    route: '5230'
    stopid: '2845'

https://pastebin.com/GgmuS8WB
Correlating the StopId's listed in the DEBUG output with the values from the static-GTFS data I can see that the correct DirectionId for my stop would be "1". So with that I added the "directionid" parameter to my test's but it didn't change the output:
homeassistant:/config/custom_components/gtfs_rt# python3 ./test.py -f test_translink.yaml -d INFO
INFO:root:Input file configuration is valid.
INFO:__main__:Adding Sensor: Name: 21 Upper Kenilworth, route id: 5230, direction id: 2845
INFO:sensor:trip_update_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_TripUpdates.pb
INFO:sensor:vehicle_position_url: https://opendata.hamilton.ca/GTFS-RT/GTFS_VehiclePositions.pb
INFO:sensor:route_delimiter: -
INFO:sensor:header: None
INFO:sensor:Successfully updated vehicle positions: 200
INFO:sensor:Successfully updated trip data: 200
INFO:sensor:Sensor Update:
INFO:sensor:   Name: 21 Upper Kenilworth
INFO:sensor:   Route: 5230
INFO:sensor:   Stop ID: 2845
INFO:sensor:   Direction ID: 1
INFO:sensor:   Icon: mdi:bus
INFO:sensor:   Service Type: Service
INFO:sensor:   unit_of_measurement: min
INFO:sensor:   Due in: -
INFO:sensor:   Due at: not defined
INFO:sensor:   latitude: not defined
INFO:sensor:   longitude: not defined
INFO:sensor:   Next Service: not defined
homeassistant:/config/custom_components/gtfs_rt#

No matter what I get not defined even though in the DEBUG output I can see StopId 2845 with TripId 1896212. Why isn't the test script finding it?
Thanks for reading.


Doesn't work with nextjs 13.1

/**
 * @type {import('next').NextConfig}
 */
const config = {
  reactStrictMode: true,
  experimental: {
    swcPlugins: [
      ['@effector/swc-plugin', {}],
    ],
  },
}

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: LayoutError', /Users/k.mironov/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.37/src/impls/core/mod.rs:265:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'thread 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/node_modules/next/dist/client/router.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: out of bounds memory access
    2: heap_get_oob', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/node_modules/next/dist/client/router.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: call stack exhausted
    2: stk_ovf', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/node_modules/next/dist/client/dev/amp-dev.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: call stack exhausted
    2: stk_ovf', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/node_modules/next/dist/client/next-dev.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: out of bounds memory access
    2: heap_get_oob', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: LayoutError', /Users/k.mironov/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.37/src/impls/core/mod.rs:265:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/projects/client-react/pages/_app.page.tsx")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: call stack exhausted
    2: stk_ovf', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: LayoutError', /Users/k.mironov/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.37/src/impls/core/mod.rs:265:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/node_modules/next/dist/pages/_error.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: call stack exhausted
    2: stk_ovf', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: LayoutError', /Users/k.mironov/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.37/src/impls/core/mod.rs:265:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/projects/client-react/pages/_document.page.tsx")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: call stack exhausted
    2: stk_ovf', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/projects/client-react/sentry.server.config.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: out of bounds memory access
    2: heap_get_oob', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/projects/client-react/pages/_app.page.tsx")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: out of bounds memory access
    2: heap_get_oob', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.236.14/src/plugin.rs:228:14
thread '<unnamed>' panicked at 'failed to invoke plugin: failed to invoke plugin on 'Some("/Users/tim/projects/timescenery/node_modules/next/dist/pages/_error.js")'

Caused by:
    0: failed to invoke<EMAIL_ADDRESS>as js transform plugin at<EMAIL_ADDRESS>    1: RuntimeError: out of bounds memory access

It's probably caused by a version mismatch between SWC version used in nextjs and SWC crates this plugin build with. Probably bumping to
swc_core = { version = "0.52.8", features = [".."] }

Should help

Adding from the example from https://github.com/effector/swc-plugin/issues/18
Example https://github.com/vvysokiy/swc-jest-decorators/tree/issue-19
"@effector/swc-plugin": "0.0.8",
"@swc/core": "^1.3.26",
"@swc/helpers": "^0.4.14",
"@swc/jest": "^0.2.24",

Simple test
describe('describe', () => {
  test('test', () => {
    expect(1).toEqual(1);
  })
});

Error
 FAIL  tests/test.spec.tsers/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc-0.241.5/src/plugin.rs:228:14
  ● Test suite failed to run

    failed to handle: failed to invoke plugin: failed to invoke plugin on 'Some("/Users/vladimirvysokij/Documents/projects/swc-jest-decorators/tests/test.spec.ts")'

    Caused by:
        0: failed to invoke `@effector/swc-plugin` as js transform plugin at<EMAIL_ADDRESS>        1: RuntimeError: out of bounds memory access
        2: heap_get_oob

It seems v0.0.8 breaks the compilation

I tried to recompile a lib locally with newer versions of SWC deps. That didn't help. But I didn't manage to build it with Rust stable channel because one of the deps (not a swc failed). So i installed latest nightly, and don't know now it panics because of nightly rust or because of deps mismatch.

I tried to recompile a lib locally with newer versions of SWC deps. That didn't help. But I didn't manage to build it with Rust stable channel because one of the deps (not a swc failed). So i installed latest nightly, and don't know now it panics because of nightly rust or because of deps mismatch.

Check "@effector/swc-plugin": "0.0.7" for your build

Related and might be interesting to others https://github.com/swc-project/swc/issues/6807

Should be fixed by #26.
Can you please try it out?

The error changed, now it's

thread '' panicked at 'position 0 does not resolve to a source location', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_common-0.29.19/src/source_map.rs:1086:17

I will try to isolate broken code later one and post it here.

I've found a root cause. This happens only when @effector/swc-plugin goes after other plugins in swcrc. If i change the order and make  effector plugin goes first - no error.
Do the effector plugin somehow directly consume or manipulate sourcemaps?
PS i'm the author of @lingui/swc-plugin and I know that there are no direct source map manipulations and all spans provided correctly (DUMMY_SPAN for new nodes and existing spans for transformed nodes), so sourcemaps after this plugin should be correct if no error in the swc itself.
.swcrc
{
  "$schema": "https://json.schemastore.org/swcrc",
  "jsc": {
    "parser": {
      "syntax": "typescript",
      "tsx": true
    },
    "transform": {
      "react": {
        "runtime": "automatic",
        "importSource": "@emotion/react"
      }
    },
    "target": "es2020",
    "loose": false,
    "externalHelpers": false,
    "keepClassNames": false,
    "experimental": {
      "plugins": [
        ["@lingui/swc-plugin", {}],
        ["@effector/swc-plugin", {}],
      ]
    }
  },
  "minify": false
}

npx swc ./component.tsx -d dist-swc --config-file="./.swcrc"

// component.tsx
import Script from 'next/script';
import { ConsentManager } from '@segment/consent-manager';
import { Trans, t } from '@lingui/macro';
import { PreferenceDialogTemplate } from '@segment/consent-manager/types/types';
import { css } from '@emotion/react';
import { zIndex } from '@/styles/z-index';

type Props = {
  segmentKey: string;
  isEnabled: boolean;
  bannerAsModal: boolean;
};

const bannerCss = css`
  position: fixed;
  bottom: 0;
  z-index: ${zIndex.segment};
  width: 100%;
  display: block;
`;

export const SegmentSnippet = (props: Props) => {
  const template: PreferenceDialogTemplate = {
    headings: {
      allowValue: t`Allow`,
      categoryValue: t`Category`,
      purposeValue: t`Purpose`,
      toolsValue: t`Tools`,
    },
    checkboxes: {
      noValue: t`No`,
      yesValue: t`Yes`,
    },
    actionButtons: {
      cancelValue: t`Cancel`,
      saveValue: t`Save`,
    },
    cancelDialogButtons: {
      cancelValue: t`Yes, Cancel`,
      backValue: t`Go Back`,
    },
    categories: [
      {
        key: 'functional',
        name: t`Functional`,
        description:
          t`To monitor the performance of our site and to enhance your browsing experience.`,
        example: t`For example, these tools enable you to communicate with us via live chat.`,
      },
      {
        key: 'marketing',
        name: t`Marketing and Analytics`,
        description:
          t`To understand user behavior in order to provide you with a more relevant browsing experience or personalize the content on our site.`,
        example:
          t`For example, we collect information about which pages you visit to help us present more relevant information.`,
      },
      {
        key: 'advertising',
        name: t`Advertising`,
        description:
          t`To personalize and measure the effectiveness of advertising on our site and other websites.`,
        example:
          t`For example, we may serve you a personalized ad based on the pages you visit on our site.`,
      },
      {
        key: 'essential',
        name: t`Essential`,
        description: t`We use browser cookies that are necessary for the site to work as intended.`,
        example:
          t`For example, we store your website data collection preferences so we can honor them if you return to our site. You can disable these cookies in your browser settings but if you do the site may not work as intended.`,
      },
    ],
  };

  const bannerContent = (

      <Trans>
        We use cookies (and other similar technologies) to collect data to improve your experience on
        our site. By using our website, you’re agreeing to the collection of data as described in our{' '}

          Website Data Collection Policy

        .
      </Trans>

  );
  const bannerSubContent = t`You can change your preferences at any time.`;
  const preferencesDialogTitle = t`Data Collection Settings`;
  const preferencesDialogContent =
    t`We use data collected by cookies and JavaScript libraries to improve your browsing experience, analyze site traffic, deliver personalized advertisements, and increase the overall performance of our site.`;
  const cancelDialogTitle = t`Are you sure you want to cancel?`;
  const cancelDialogContent =
    t`Your preferences have not been saved. By continuing to use our website, you՚re agreeing to our Website Data Collection Policy.`;

  return (
    <>


Cherry pick https://github.com/microsoft/MixedRealityToolkit-Unity/pull/7694 into stabilization

This change didn't quite make it over, so let's gooooo

/azp run


Enable-JAzRole: The requestor does not have permissions for this request

I've been using Enable-JAzRole without issues in one tenant but trying it against a different tenant results in:
Enable-JAzRole 'Owner -> MG01 (59938e56-554d-4cf5-84c3-529766c42548)' -Justification 'Daily elevation'

Enable-JAzRole: The requestor d3493a29-cc6e-46f1-b5ca-876bb0e53bf5 does not have permissions for this request. Please use $filter=asTarget() to filter on the requestor's assignments.

Activating the same role from the Azure portal works fine. Any idea what might be the reason for the above error?
Side question:  What happens if the "Require MFA on activation" setting is enabled for a role?

@janegilring  hmm not sure why asTarget() didn't get added to your command,
In terms of MFA, as long as you are in a connect-azaccount or connect-mgaccount where you did an MFA then it'll be fine, it actually won't prompt again for MFA at command run, that's just how the API works. However, if you are in a non-mfa account, it will fail with 'MFARule' policy error.

That works fine:
Get-JazRole

PrincipalEmail                            RoleDefinitionDisplayName ScopeDisplayName ScopeType       EndDateTime
--------------                            ------------------------- ---------------- ---------       -----------
                                          Contributor               IT               managementgroup
                                          Owner                     IT               managementgroup
                                          Owner                     Production       managementgroup
<EMAIL_ADDRESS>User Access Administrator IT               managementgroup

Good call. Updated to 0.5, seems like the error has changed:
Enable-JAzRole 'Owner -> IT (59938e56-554d-4cf5-84c3-529766c42548)'
                                                                                Exception: /home/vscode/.local/share/powershell/Modules/JAz.PIM/0.0.5/Private/Resolve-RoleByName.ps1:16:23                                                                                Line |                                                                                16 |  … ot $Role) { throw "RoleGuid $roleGuid from $RoleName was not found as …                                                                                |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                | RoleGuid 59938e56-554d-4cf5-84c3-529766c42548 from Owner -> IT (59938e56-554d-4cf5-84c3-529766c42548) was not found as an eligible role for this user
´´´

Get-JAzRole  (and tab completion) works fine:
´´´
Get-JAzRole
                                                                                PrincipalDisplayName                                    RoleDefinitionDisplayName ScopeDisplayName                  ScopeType       EndDateTime
--------------------                                    ------------------------- ----------------                  ---------       -----------
some-group           Owner               IT                                managementgroup
´´´

Also tested this problem in another environment, Get-JAzRole works fine:
 Get-JAzRole

PrincipalDisplayName  RoleDefinitionDisplayName ScopeDisplayName ScopeType    EndDateTime
--------------------  ------------------------- ---------------- ---------    -----------
Admin - Jan Egil Ring Owner                     democrayon       subscription

Tab-completion works fine as well, but Enable-JAzRole fails:
Enable-JAzRole 'Owner -> democrayon (641eac54-cc6a-48d1-98ea-6834911db638)'
Exception: /home/vscode/.local/share/powershell/Modules/JAz.PIM/0.0.5/Private/Resolve-RoleByName.ps1:16:23
Line |
  16 |  … ot $Role) { throw "RoleGuid $roleGuid from $RoleName was not found as …
     |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | RoleGuid 641eac54-cc6a-48d1-98ea-6834911db638 from Owner -> democrayon (641eac54-cc6a-48d1-98ea-6834911db638) was not found as
     | an eligible role for this user

Interesting, it works when piping a role into Enable-JAzRole:
´´´
Get-JAzRole | Select -First 1 | Enable-JAzRole -Justification 'Daily elevation'
´´´

Show me your Get-JAzRole | fl entry for the role you want to enable, there might be a matching problem in the lookup.

Get-JAzRole | fl
Condition                          :
ConditionVersion                   :
CreatedOn                          : 5/16/2022 7:21:54 PM
EndDateTime                        :
ExpandedPropertiesPrincipalId      : 27a72d40-6214-4198-97b4-8c6fd1bfd243
ExpandedPropertiesPrincipalType    : User
ExpandedPropertiesRoleDefinitionId : /subscriptions/b7f543e7-29f0-4e13-8b16-e8e94170be88/providers/Microsoft.Authorization/roleDefinitions/8e3af
                                     657-a8ff-443c-a75c-2fe8c4bcb635
Id                                 : /subscriptions/b7f543e7-29f0-4e13-8b16-e8e94170be88/providers/Microsoft.Authorization/roleEligibilitySchedu
                                     les/641eac54-cc6a-48d1-98ea-6834911db638
MemberType                         : Direct
Name                               : 641eac54-cc6a-48d1-98ea-6834911db638
PrincipalDisplayName               : Admin - Jan Egil Ring
PrincipalEmail<EMAIL_ADDRESS>PrincipalId                        : 27a72d40-6214-4198-97b4-8c6fd1bfd243
PrincipalType                      : User
RequestId                          : /subscriptions/b7f543e7-29f0-4e13-8b16-e8e94170be88/providers/Microsoft.Authorization/roleEligibilitySchedu
                                     leRequests/641eac54-cc6a-48d1-98ea-6834911db638
ResourceGroupName                  :
RoleDefinitionDisplayName          : Owner
RoleDefinitionId                   : /subscriptions/b7f543e7-29f0-4e13-8b16-e8e94170be88/providers/Microsoft.Authorization/roleDefinitions/8e3af
                                     657-a8ff-443c-a75c-2fe8c4bcb635
RoleDefinitionType                 : BuiltInRole
Scope                              : /subscriptions/b7f543e7-29f0-4e13-8b16-e8e94170be88
ScopeDisplayName                   : democrayon
ScopeId                            : /subscriptions/b7f543e7-29f0-4e13-8b16-e8e94170be88
ScopeType                          : subscription
StartDateTime                      : 5/16/2022 7:21:54 PM
Status                             : Provisioned
Type                               : Microsoft.Authorization/roleEligibilitySchedules
UpdatedOn                          : 5/16/2022 7:21:54 PM

I must have broke something, not working for me either, let me look into it

I must have broke something, not working for me either, let me look into it

Yup I was swallowing the object, whoops. Test again with 0.0.6 which should be out shortly and let me know if it still breaks.

Works like a charm 👍

I ran into this again today with the exact same error as initially reported:
Enable-JAzRole: The requestor d3493a29-cc6e-46f1-b5ca-876bb0e53bf5 does not have permissions for this request. Please use $filter=asTarget() to filter on the requestor's assignments.
This was due to the PIM role having the "Require MFA on activation" setting is enabled, and there is no MFA enforcement using Conditional Access in this tenant. It is rather enforced via PIM, hence I wasn't prompted for MFA which lead to the above error.
In order to force an MFA authentication, I found this:
# Install msal.ps
Install-Module msal.ps

# Get token for MS Graph by prompting for MFA
$MsResponse = Get-MSALToken -Scopes @("https://graph.microsoft.com/.default") -ClientId "1b730954-1685-4b74-9bfd-dac224a7b894" -RedirectUri "urn:ietf:wg:oauth:2.0:oob" -Authority "https://login.microsoftonline.com/common" -Interactive -ExtraQueryParameters @{claims='{"access_token" : {"amr": { "values": ["mfa"] }}}'}

Connect-MgGraph -AccessToken $MsResponse.AccessToken

After this authentication, Enable-JAzRole ran without issues.
Maybe it is possible to check whether a role requires MFA, and if so add the extra query parameters when authenticating? On the other side, this may be an edge case - as most tenants are enforcing MFA via Conditional Access.

I would say it's quite a rarity to be using PIM but not use MFA, so I'm open to PRs to this regard but I'm not going to actively work on it.


Changes to ULL virtual env script and grammar-tester script

Made fixes to the virtual environment script. Added squre bracket filtering when .ull file is used as input. Few bugs fixed.

Conflicting files -- environment.yml.
Looks like conflicting with my changes (removed igraph :(

environment.yml changes conflict resolution:

anaconda added to channels
changes by @OlegBaskov 2018-07-14 added to environment.yml by @alexei-gl 2018-04-16:

dependencies:
- atomicwrites=1.1.5=py36_0
- attrs=18.1.0=py36_0
- more-itertools=4.2.0=py36_0
- pluggy=0.6.0=py36hb689045_0
- py=1.5.3=py36_0
- pytest=3.6.2=py36_0


Include a magic number in the TCP stream handshake

Higher layers may want to multiplex Timely traffic on a port that is also used for other protocols (e.g., HTTP). Sending a magic number immediately after establishing the stream makes it easy to sniff out Timely traffic.

In principle each timely instance should be talking to the same binaries, which we could also try to check. I wonder if it makes sense to try and generate a magic number at build time, to confirm that the binaries are talking to similar instances of themselves... ?

In principle each timely instance should be talking to the same binaries, which we could also try to check. I wonder if it makes sense to try and generate a magic number at build time, to confirm that the binaries are talking to similar instances of themselves... ?

Indeed it would! I was planning to do that at a higher layer of the stack (i.e., Material), since folks often frown upon doing anything too magical in a library, and Material will need to do its own version check anyway. But there's no harm in the redundancy, if you think its worth doing in Timely too.
Did you mean generating a magic number in build time in addition to or instead of the permafixed magic number? IMO its worth having both. The permafixed magic number reduces the burden on the TCP multiplexer, since it can do a blind equality comparison on the first u64 in the stream. By contrast, the PostgreSQL wire protocol, annoyingly starts a connection with one of five possible versions, with the possibility of expansion at any point; HTTP starts with one of "GET"/"HEAD"/"OPTIONS"/"POST"/"PUT'/....
You could argue that perhaps we shouldn't be multiplexing multiple protocols over the same port—that is what ports are for, after all—and you would be right, but experience has shown that when one application needs to speak multiple protocols, it is much easier on everyone involved if you can stuff all the protocols into one port.

This looks good to me, modulo maybe wanting to tweak things in the future (e.g. per-build magic number in there too, to make sure struct layouts and usize sizes are the same; that sort of thing.

Yes, please feel free to modify as you see fit! All that matters to me is committing to some backwards-compatible scheme before Materialize ships.


usonic: update container image

The usonic repo is moved from https://github.com/microsonic/usonic to https://github.com/oopt-goldstone/usonic.
I fixed the CI of the oopt-goldstone/usonic and all container images are now available from ghcr.io/oopt-goldstone.

@KazuyaAnazawa @HarshaF1 Please review. I confirmed that the image is buildable but haven't tested it on the actual hardware.

@KazuyaAnazawa Thanks for checking. Could you check if gscli's interface command work on Galileo1?

@ishidawataru
Sure, the result is as follows. It seems interface command works fine.
(Currently I cannot confirm if oper-status can be up because no module is inserted on Galileo1. I can check it later.)
root@localhost:~# gscli
> show interface brief
+--------------+-------------+--------------+-----------+
|     name     | oper-status | admin-status |   alias   |
+--------------+-------------+--------------+-----------+
| Ethernet1_1  |    down     |     down     |  QSFP1-1  |
| Ethernet2_1  |    down     |     down     |  QSFP2-5  |
| Ethernet3_1  |    down     |     down     |  QSFP3-9  |
| Ethernet4_1  |    down     |     down     | QSFP4-13  |
| Ethernet5_1  |    down     |     down     | QSFP5-17  |
| Ethernet6_1  |    down     |     down     | QSFP6-21  |
| Ethernet7_1  |    down     |     down     | QSFP7-25  |
| Ethernet8_1  |    down     |     down     | QSFP8-29  |
| Ethernet9_1  |    down     |     down     | QSFP9-33  |
| Ethernet10_1 |    down     |     down     | QSFP10-37 |
| Ethernet11_1 |    down     |     down     | QSFP11-41 |
| Ethernet12_1 |    down     |     down     | QSFP12-45 |
| Ethernet13_1 |    down     |     down     | PIU1-1-49 |
| Ethernet14_1 |    down     |     down     | PIU1-2-53 |
| Ethernet15_1 |    down     |     down     | PIU2-1-57 |
| Ethernet16_1 |    down     |     down     | PIU2-2-61 |
| Ethernet17_1 |    down     |     down     | PIU3-1-65 |
| Ethernet18_1 |    down     |     down     | PIU3-2-69 |
| Ethernet19_1 |    down     |     down     | PIU4-1-73 |
| Ethernet20_1 |    down     |     down     | PIU4-2-77 |
| Ethernet21_1 |    down     |     down     |  CPU1-81  |
| Ethernet22_1 |    down     |     down     |  CPU2-83  |
+--------------+-------------+--------------+-----------+
> interface Ethernet1_1
interface(Ethernet1_1)> ad up
interface(Ethernet1_1)> q
> vlan range 3-5
Selected VLANs: [3, 4, 5]
vlan(3-5)> q
> interface Ethernet1_1
interface(Ethernet1_1)> switchport mode access vlan 3
interface(Ethernet1_1)> interface-type SR4
interface(Ethernet1_1)> q
> interface Ethernet4_1
interface(Ethernet4_1)> auto-negotiate enable
interface(Ethernet4_1)> show running-config
interface(Ethernet3_1)> q
> interface Ethernet2_1
interface(Ethernet2_1)> no shutdown
interface(Ethernet2_1)> switchport mode trunk vlan 4
interface(Ethernet2_1)> q
> show interface brief
+--------------+-------------+--------------+-----------+
|     name     | oper-status | admin-status |   alias   |
+--------------+-------------+--------------+-----------+
| Ethernet1_1  |    down     |      up      |  QSFP1-1  |
| Ethernet2_1  |    down     |      up      |  QSFP2-5  |
| Ethernet3_1  |    down     |     down     |  QSFP3-9  |
| Ethernet4_1  |    down     |     down     | QSFP4-13  |
| Ethernet5_1  |    down     |     down     | QSFP5-17  |
| Ethernet6_1  |    down     |     down     | QSFP6-21  |
| Ethernet7_1  |    down     |     down     | QSFP7-25  |
| Ethernet8_1  |    down     |     down     | QSFP8-29  |
| Ethernet9_1  |    down     |     down     | QSFP9-33  |
| Ethernet10_1 |    down     |     down     | QSFP10-37 |
| Ethernet11_1 |    down     |     down     | QSFP11-41 |
| Ethernet12_1 |    down     |     down     | QSFP12-45 |
| Ethernet13_1 |    down     |     down     | PIU1-1-49 |
| Ethernet14_1 |    down     |     down     | PIU1-2-53 |
| Ethernet15_1 |    down     |     down     | PIU2-1-57 |
| Ethernet16_1 |    down     |     down     | PIU2-2-61 |
| Ethernet17_1 |    down     |     down     | PIU3-1-65 |
| Ethernet18_1 |    down     |     down     | PIU3-2-69 |
| Ethernet19_1 |    down     |     down     | PIU4-1-73 |
| Ethernet20_1 |    down     |     down     | PIU4-2-77 |
| Ethernet21_1 |    down     |     down     |  CPU1-81  |
| Ethernet22_1 |    down     |     down     |  CPU2-83  |
+--------------+-------------+--------------+-----------+
> show vlan details
  vid  name    members      mode
-----  ------  -----------  ------
    3  -       Ethernet1_1  access
    4  -       Ethernet2_1  trunk
    5  -       -            -
interface Ethernet1_1
  admin-status up
  interface-type SR4
  switchport mode access vlan 3
  quit
!
interface Ethernet2_1
  admin-status up
  switchport mode trunk vlan 4
  quit
!
interface Ethernet4_1
  auto-negotiate enable
  quit
!
vlan 3
  quit
vlan 4
  quit
vlan 5
  quit
!

Changes look good to me. We a 7716-32x box in our lab, I am building an image now.
Since we have a stand alone box, not sure how much of functional testing I can do. But I will check the basic configs and update the logs here.

@HarshaF1 Thanks for checking.

GNMI and xlate-oc pods are crashing , buts that's not related to this PR I suppose .

This is expected. xlate-oc currently only supports the GFT platform.
We should disable the service on other platforms. I'll open a PR for that once this PR is merged.

Interfaces are coming up and traffic looks good
root@localhost:~# gscli
> sho inbr
invalid command 'inbr'. available commands: ['datastore', 'tech-support', 'logging', 'running-config', 'interface', 'chassis-hardware', 'transponder', 'aaa', 'tacacs', 'version', 'ufd', 'portchannel', 'vlan']
> show interface brief
+--------------+-------------+--------------+------------+
|     name     | oper-status | admin-status |   alias    |
+--------------+-------------+--------------+------------+
| Ethernet1_1  |    down     |      up      |  QSFP1-1   |
| Ethernet2_1  |    down     |      up      |  QSFP2-5   |
| Ethernet3_1  |    down     |     down     |  QSFP3-9   |
| Ethernet4_1  |    down     |     down     |  QSFP4-13  |
| Ethernet5_1  |    down     |     down     |  QSFP5-17  |
| Ethernet6_1  |    down     |     down     |  QSFP6-21  |
| Ethernet7_1  |    down     |     down     |  QSFP7-25  |
| Ethernet8_1  |    down     |     down     |  QSFP8-29  |
| Ethernet9_1  |    down     |     down     |  QSFP9-33  |
| Ethernet10_1 |    down     |     down     | QSFP10-37  |
| Ethernet11_1 |    down     |     down     | QSFP11-41  |
| Ethernet12_1 |    down     |     down     | QSFP12-45  |
| Ethernet13_1 |    down     |     down     | QSFP13-49  |
| Ethernet14_1 |    down     |     down     | QSFP14-53  |
| Ethernet15_1 |    down     |     down     | QSFP15-57  |
| Ethernet16_1 |    down     |     down     | QSFP16-61  |
| Ethernet17_1 |    down     |     down     | QSFP17-65  |
| Ethernet18_1 |    down     |     down     | QSFP18-69  |
| Ethernet19_1 |    down     |     down     | QSFP19-73  |
| Ethernet20_1 |    down     |     down     | QSFP20-77  |
| Ethernet21_1 |    down     |     down     | QSFP21-81  |
| Ethernet22_1 |    down     |     down     | QSFP22-85  |
| Ethernet23_1 |    down     |     down     | QSFP23-89  |
| Ethernet24_1 |    down     |     down     | QSFP24-93  |
| Ethernet25_1 |    down     |     down     | QSFP25-97  |
| Ethernet26_1 |    down     |     down     | QSFP26-101 |
| Ethernet27_1 |    down     |     down     | QSFP27-105 |
| Ethernet28_1 |    down     |     down     | QSFP28-109 |
| Ethernet29_1 |     up      |      up      | QSFP29-113 |
| Ethernet30_1 |    down     |     down     | QSFP30-117 |
| Ethernet31_1 |     up      |      up      | QSFP31-121 |
| Ethernet32_1 |    down     |     down     | QSFP32-125 |
| Ethernet33_1 |    down     |     down     |  CPU1-130  |
| Ethernet34_1 |    down     |     down     |  CPU2-132  |
+--------------+-------------+--------------+------------+
> exit
root@localhost:~# bcmsh
psps
ps
                 ena/        speed/ link auto    STP                  lrn  inter   max   cut   loop
           port  link  Lns   duplex scan neg?   state   pause  discrd ops   face frame  thru?  back
       ce0(  1)  up     4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce1(  5)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce2(  9)  up     4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce3( 13)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce4( 17)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce5( 21)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce6( 25)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce7( 29)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No
       ce8( 34)  down   4  100G  FD   SW  No   Forward          None   FA    CR4  9122    No

@HarshaF1 Thanks a lot!


Introduced LogicalPlanCounter to count the subplans in a logical plan.

A logical plan counter to count all the subplans within a logical plan.

Torn between whether to keep the LogicalCounterVisitor subclass as a child of LogicalWalkerVisitor to avoid code duplication or whether to make it separate in order to make the walk method protected.
I'm also don't quite get what the 'visit' function does. I see it declared but I don't know where it is implemented.


Migrate tests to<EMAIL_ADDRESS>

Closes:https://github.com/metarhia/do/issues/31

@tshemsedinov, do i need to upgrade this to<EMAIL_ADDRESS>Or open new PR?


Set up content query index for the search functionality

@AngelaFabregues why its that needed? We already have https://netcentric.biz/query-index.json which has the entire page.

https://netcentric.biz/query-index.json does not have the entire page, it has only a few fields, we need the full text content for ingestion, we could add new fields to the existing index or just create a new one

I see, yes I suggest to extend the current existing index config

Why extending instead of a dedicated one? those fields will be very search specific like "full text", "boosted" etc. with just all text content in it. I don't see why we should clutter the existing index with those values, will only bloat the payload for other usages.

query-index.json already has all pages, only fields are missing, if I understood it correctly? So why not just adding them instead of a second index config to index the same amount of pages.
Technically both approaches are fine. Maybe this is just personal preference.

exactly, all pages but not all content, my preference would be separate because the content is quite specific for search and I don't see any other consumer interested in those values, it will bloat the payload for others.

Hi @mhaack ,
it is necessary for the search functionality only.
Thanks @nc-andreashaller .


How to get Tenant directory path as string?

Hi all,
is there any function to get the current Tenant directory base path.
When I use the prefix of the tenancy-directory I just get the /storage/app/tenancy/tenant/ but I miss the UUID. What do you think, should we add some function to the current file system?
I need the path as string a I need to set this as parameter to vendor packages.
Thank you.

Information

hyn/multi-tenant version: latest
laravel version: latest 5.6

Try this when a Tenant was identified
app(\Hyn\Tenancy\Website\Directory::class)->path();

app(\Hyn\Tenancy\Website\Directory::class)->path();
Return is a boolean and i need the path

@cesarZaabra, no it doesn't: https://github.com/tenancy/multi-tenant/blob/5.x/src/Website/Directory.php#L78


Fix misalignment of table output for multi-byte chars (#268)

This PR suggests a fix for #268. The problem is that std::string::length() is used in TableOutput to determine the length of table cell values. Since std::string is not UTF8-aware, it returns the number of bytes, not characters. The two are different if the string contains special characters.
This fix adds a Utility::UTF8Length method to obtain the number of characters of a UTF8 string, and uses it in TableOutput to compute the display length of cell values.
Microsoft Reviewers: Open in CodeFlow

If I understand this code correctly, it converts the UTF-8 string into an UTF-16 string and counts the number of wchar_t.
If I'm not mistaking, this would suffer from the same issue as the current implementation, pro problem just occurs on more exotic chars like Chinese or Emojis.
Counting the number of grapheme clusters would be the way to go, I think.
See: https://mortoray.com/2016/04/28/what-is-the-length-of-a-string-a-tricky-question/

If I'm not mistaking, this would suffer from the same issue as the current implementation, the problem just occurs on more exotic chars like Chinese or Emojis.

I was wondering about this, too, and you are probably right.
In any case, the way UTF-8 is currently handled in the code base (in plain std::string) does not seem ideal because it is very likely that similar mistakes will slip into the code in the future again. C++20 has std::u8string but I guess the project cannot use it yet.

Unfortunately u8string won't help either, as @nikeee points out what we really want to count are grapheme clusters.  I was aware of this when I wrote it, but I:

Mistakenly thought that it would simply result in a wider column (not thinking that different strings would have different disparities in encoded vs cluster count).
Didn't find a simple Windows API to handle it.
Ok, I was aware that there were issues, but not to the depths of terminology / semantics around Unicode and characters...

I think that the right thing to do is to implement "character" breakpoint iteration counting from ICU.  Since we use the version of ICU that ships with Windows, that means using the C API as described http://userguide.icu-project.org/boundaryanalysis
In addition to UTF8Length, the TableOutput as written would also need "UTF8Substring" for truncating strings properly.  Also, since we are on the subject, is the ellipses (...) a language independent construct for indicating truncation?
I do appreciate this fix for my haste, but I don't want to sweep it under the rug by fixing only some of the problem.

I think that the right thing to do is to implement "character" breakpoint iteration counting from ICU. Since we use the version of ICU that ships with Windows, that means using the C API as described http://userguide.icu-project.org/boundaryanalysis
In addition to UTF8Length, the TableOutput as written would also need "UTF8Substring" for truncating strings properly.

I changed the implementation to use the code you linked. I also added a UTF8Substring method.
Independent from that, in bb92fd9, I changed the three dots into a single ellipsis character. This wastes a little less space and allows to show two more characters of the abbreviated string. But feel free to revert this if you preferred the three dots.

Also, since we are on the subject, is the ellipses (...) a language independent construct for indicating truncation?

According to Wikipedia, ellipsis seems common in other languages, as well. But some languages use variations of it, e.g. in Chinese they use six instead of three dots.

/azp run

Looks great @chausner
I would probably move the ICU setup boilerplate into a separate type just so its only written once.  And you can use wil::unique_any_t to define a smart pointer without having to use scope_exits.
Yes, Wikipedia does seem to suggest that it is used in other languages, although it also points to varying ways in which it is represented.  I think that your move to use the single character ellipsis may actually prove to be the right thing with regards to this in any case, both because it shows more characters and because it may just render as something more appropriate in the user's chosen font.

I made the changes that I was suggesting on your branch.  Let me know how you feel about them.
/azp run

/azp run

I made the changes that I was suggesting on your branch. Let me know how you feel about them.

Just the one typo in the log message, otherwise looks good to me. 👍

/azp run

Resolves #268


run.dlang.io: dmd-nightly not using nightly build

Evidenced by selecting dmd-nightly and running this:
pragma(msg, __VERSION__);

Which prints 2098L, while dmd-beta prints 2099L

It's using the nightly build but the newer images hang when compiling D code (for some unknown reason). That causes run.dlang.io to retain the latest working image.

The nightly should be fixed now. I opened a PR with various CI/CD implevemnts: https://github.com/dlang-tour/core/pull/772