silently autenticate user

To silently authenticate a user , I used : NM.auth().authenticateWithCachedToken().subscribe() which works fine.
Now , it's deprecated and the  alternative ChatSDK.core() didn't have any similar method, how can I proceed ? thanks

guys any suggestion? thank you

You can just call:
ChatSDK.auth().authenticate().subscribe(...


Switch Ign-math to Eigen3

Should solve: PX4/PX4-Autopilot#16349
But there are other commits, which is not meged here yet.

I dont know how to resolve confilcts here...

I dont know how to resolve confilcts here...

I have resolved the conflicts (emerging probably from cherrypicking of some commits), by this web editor.  The conflicts are not functional.
I think this is ready for merging. @Jaeyoung-Lim  ?

I have resolved the conflicts (emerging probably from cherrypicking of some commits), by this web editor.  The conflicts are not functional.
I think this is ready for merging. @Jaeyoung-Lim  ?


Fix sync directory meta repeatedly

What changes are proposed in this pull request?
Fix the issue: A file with a lastSync time newer than its directory causes the meta sync of the directory triggered repeatedly
Why are the changes needed?
Fix https://github.com/Alluxio/alluxio/issues/15265
Does this PR introduce any user facing changes?
No

Thanks for the pull request. I wonder if a better solution would be to track the oldest time any of the children have been synced and update the root last sync path time with that? I agree in the current solution sync is being called to often, but switching this to true as in your pull request may case the files to be synced to little, i.e. a file could become as old as 2x the sync time given that the root directory sync time could be updated just before the sync time on the file became too old, then the root directory will not be synced again for the full sync timeout.

@tcrain thanks for your review.

I wonder if a better solution would be to track the oldest time any of the children have been synced and update the root last sync path time with that?
A good point. Just to confirm: update the directory's last sync time to the oldest time of its children when listStatus triggered, right?

switching this to true as in your pull request may case the files to be synced to little
Yes, you are right.
Generally "read file" follows "list directory" in our current use cases, and "read file" updates the file's last sync time. So it is better than syncing directory meta every time.

Yes, my initial thought would be for the function to return the last sync time, instead of true/false. Then the root directory will update its sync time with the oldest time of the values returned.

Yes, my initial thought would be for the function to return the last sync time, instead of true/false. Then the root directory will update its sync time with the oldest time of the values returned.

I'm not sure how the parent sync time should be associated with the children sync time because in many corner cases these two are separate. For example I do chown on the parent dir and that's irrelevant to the children sync time. I'm thinking if we try to assume a relationship between parent sync time and children sync time, we get into the swamp of handling all parent-children scenarios.
Another thought I'm having is, if you return a timestamp from processSyncPath(), it is hard to distinguish SKIPPED vs FAILED without using hacky logic.
Due to my reasons above, I prefer using an enum for the return value. WDYT @tcrain ?

May I know if there is any update from your discussions? @jiacheliu3 @tcrain

May I know if there is any update from your discussions? @jiacheliu3 @tcrain

I think let's start with defining an enum and returning that in processSyncPath() instead of a boolean. If a timestamp makes more sense, I can add that in a separate PR in the near future. @secfree Please let me know if that's confusing :)

May I know if there is any update from your discussions? @jiacheliu3 @tcrain

I think let's start with defining an enum and returning that in processSyncPath() instead of a boolean. If a timestamp makes more sense, I can add that in a separate PR in the near future. @secfree Please let me know if that's confusing :)

@jiacheliu3 Thanks for your reply.
If using enum, may I know how will you update the "last sync time" of the directory if some files under the directory are skipped?

May I know if there is any update from your discussions? @jiacheliu3 @tcrain

I think let's start with defining an enum and returning that in processSyncPath() instead of a boolean. If a timestamp makes more sense, I can add that in a separate PR in the near future. @secfree Please let me know if that's confusing :)

@jiacheliu3 Thanks for your reply. If using enum, may I know how will you update the "last sync time" of the directory if some files under the directory are skipped?

IMO I think the parent's last sync time should not be affected by children last sync time. For example the user performs a full sync on /dir, then after a while force-checks one individual file /dir/file(by -f). I think the parent's last sync time should not be refreshed by the child.
When a child is checked, UfsSyncPathCache.shouldSyncPath() checks parents of this child. So if the user performs a full sync on /dir, then checks /dir/file, the latter sync will be skipped correctly.
So if some files under the directory are skipped during the sync, that should count as a successful sync in InodeSyncStream.sync(), and we update the last sync time on the directory correctly.
What do you think? Did I miss anything?

So if some files under the directory are skipped during the sync, that should count as a successful sync in InodeSyncStream.sync(), and we update the last sync time on the directory correctly.

Then the solution will have the same effect as this PR, and it has the issue mentioned by tcrain "a file could become as old as 2x the sync time given that the root directory sync time could be updated just before the sync time on the file became too old, then the root directory will not be synced again for the full sync timeout." Right?

I think the parent's last sync time should not be refreshed by the child.

"track the oldest time any of the children" does not trigger parent "last sync time" refreshing by the child. It updates "last sync time" of the parent referring to children's "last sync time".

When a child is checked, UfsSyncPathCache.shouldSyncPath() checks parents of this child. So if the user performs a full sync on /dir, then checks /dir/file, the latter sync will be skipped correctly.

It seems that this is not implemented yet, it needs this feature https://github.com/Alluxio/alluxio/issues/15486

So if some files under the directory are skipped during the sync, that should count as a successful sync in InodeSyncStream.sync(), and we update the last sync time on the directory correctly.

Then the solution will have the same effect as this PR, and it has the issue mentioned by tcrain "a file could become as old as 2x the sync time given that the root directory sync time could be updated just before the sync time on the file became too old, then the root directory will not be synced again for the full sync timeout." Right?

Great call! I think the trouble is from the fact that the last sync time does not consider if the sync is recursive. The user may trigger a single-path ls or a recursive one on the directory and the TS gets updated the same way. I have the feeling that we have many different scenarios to consider. The only way to make 100% sure is to list every case and we scrutinize them all. There should be <10 cases so I'd think it's still manageable and worthwhile.

I think the parent's last sync time should not be refreshed by the child.

"track the oldest time any of the children" does not trigger parent "last sync time" refreshing by the child. It updates "last sync time" of the parent referring to children's "last sync time".

Sure you can do that. I'm just a bit concerned about too many new objects flying around and about the performance. Also a bit careful reasoning is needed, when you distinguish a recursive sync and a single-path sync. You can totally go ahead and implement that, but the performance of the sync will very much impact the total performance of Alluxio.

When a child is checked, UfsSyncPathCache.shouldSyncPath() checks parents of this child. So if the user performs a full sync on /dir, then checks /dir/file, the latter sync will be skipped correctly.

It seems that this is not implemented yet, it needs this feature #15486

This is in
https://github.com/Alluxio/alluxio/blob/f8755551bf777edcf101ca00ecb036586090b98a/core/server/master/src/main/java/alluxio/master/file/meta/UfsSyncPathCache.java#L103
I haven't looked at #15846 thoroughly but that can just be a bug in UfsSyncPathCache.

I think the parent's last sync time should not be refreshed by the child.

"track the oldest time any of the children" does not trigger parent "last sync time" refreshing by the child. It updates "last sync time" of the parent referring to children's "last sync time".

I'm just a bit concerned about too many new objects flying around and about the performance. Also a bit careful reasoning is needed, when you distinguish a recursive sync and a single-path sync. You can totally go ahead and implement that, but the performance of the sync will very much impact the total performance of Alluxio.

I also concern the performance, and I am not clear that if there are problems to use the oldest "last sync time" of the FIRST level children for recursive mode.
Now in my case, this issue(sync directory meta repeatedly) made listStatus on Alluxio have about 10x time cost than on HDFS. The performance won't be worse.
I think I can implement the the idea based on the oldest last sync time first and compare the performance. I will also check if it works in recursive mode.  @jiacheliu3 is this ok?

When a child is checked, UfsSyncPathCache.shouldSyncPath() checks parents of this child. So if the user performs a full sync on /dir, then checks /dir/file, the latter sync will be skipped correctly.

It seems that this is not implemented yet, it needs this feature #15486

This is in ... but that can just be a bug in UfsSyncPathCache

Thanks for your help. I will verify it.

@secfree UfsSyncPathCache is a Map<Path, SyncTime> where SyncTime actually just contains the last sync time and last recursively sync time. I think we can just use that as the return value of processSyncPath().
Currently processSyncPath() returns:
true -> path synced
false -> path skipped or sync interrupted or sync failed

We can change that to:
SyncTime.UNSYNCED -> skipped; also if the sync is interrupted, we can also return this
A real SyncTime where the sync and recursive-sync time are managed -> sync success, recursively or not
If the sync failed, you can define another static constant like SyncTime.FAILED=Long.MIN_VALUE if that makes it easier. It's not very easy to throw exceptions with threadpool.submit()

It may not be straightforward how to find the smallest value and update the parent last sync time accordingly. I prefer not to keep all results and sort them later, because imagine you have a flat directory with 1 million files... If you can keep an atomic variable for the parent sync time and update that in every sync job, that can be a good start.
To better avoid contentions, I randomly imagine there's some data structure like LongAdder, where the variable can have multiple versions, and those versions are merged on the getter. But you need close to zero contention when updating that, instead of using CAS. That would be an optimal data structure when you need a global min from concurrent threads. Not sure if you can use a ConcurrentHashMap<ThreadId, LocalMin> and we find the final minimal when the sync finishes.

I found a strange point.
For the listStatus operation

InodeSyncStream.sync() calls syncInodeMetadata() for the directory before calling processSyncPath() for its children
syncInodeMetadata() calls syncExistingInodeMetadata() which may do prefetchChildren(), this gets the latest metadata for its direct children but does not update children's last sync time
processSyncPath() checks last sync time to decide if it needs to do syncInodeMetadata() for the children

It seems that step 3 is not needed if prefetchChildren() happened at step 2, right? @jiacheliu3 @tcrain

I updated the code of this PR. Now it updates children's meta if already prefetched.
This solution can resolve this issue and should not import new issues.
Please help review @jiacheliu3 @tcrain , thank you

I have a different idea:

processSyncPath() keeps returning a boolean value

true -> path synced or skipped
false -> sync interrupted or sync failed

processSyncPath() also updates "last sync time" of the path if path sync succeed. This relates to listStatus should update files' lastSyncTime under the directory for HDFS #15486

InodeSyncStream.sync() checks failedSyncPathCount. If failedSyncPathCount=0, it gets the min value of "last sync time" of its children cached in UfsSyncPathCache(just iterate children without sorting).

What do you think about this? @jiacheliu3

On your points:

processSyncPath() keeps returning a boolean value
I think a enum is much better than a boolean in being more explicit in the state. Actually even if there are 2 states, I prefer State.Success and State.Failed to a boolean.

processSyncPath() also updates "last sync time" of the path if path sync succeed.
I didn't find that. Could you point me to it? By the fact that I didn't find it in a quick search, that alone means the logic is too complicated and needs a refactor...

InodeSyncStream.sync() checks failedSyncPathCount.
I don't find that in your code change, is that still applicable?

I found a strange point.
For the listStatus operation

InodeSyncStream.sync() calls syncInodeMetadata() for the directory before calling processSyncPath() for its children
syncInodeMetadata() calls syncExistingInodeMetadata() which may do prefetchChildren(), this gets the latest metadata for its direct children but does not update children's last sync time
processSyncPath() checks last sync time to decide if it needs to do syncInodeMetadata() for the children

It seems that step 3 is not needed if prefetchChildren() happened at step 2, right? @jiacheliu3 @tcrain
If we can skip step 3, then we do not need to care children's last sync time.

I think processSyncPath is on the dir, while prefetchChildren is on the children? Why do we not need 3 when we have 2? Could you elaborate?

On your points:

InodeSyncStream.sync() checks failedSyncPathCount.

I don't find that in your code change, is that still applicable?

Please ignore this idea, it is already outdated. I should have deleted it. Sorry for it.

processSyncPath() also updates "last sync time" of the path if path sync succeed.

I didn't find that. Could you point me to it? By the fact that I didn't find it in a quick search, that alone means the logic is too complicated and needs a refactor...

I planed to do this with #15486 together, please also ignore this.

I think processSyncPath is on the dir, while prefetchChildren is on the children? Why do we not need 3 when we have 2? Could you elaborate?

If you turn on debug log you can find that: listStatus on a directory will

prefetch the metadata of all children first
Then call processSyncPath for each children concurrently

That is the idea of my latest PR. If listStatus already prefetched metadata, we can update children's metadata without checking their last sync time.

As the changes and discussions of this PR have been incorporated into #16081, I close this one.


Bug report: MToon Appears Black on New Version

Hi, Reporting a bug on the MToon shader on the new version
Currently the MToon Doesn't work and Model appear as black as [Screenshot (1)]

from what I have suspect it might be that there is no Material Output Node at the end [Screenshot (2)]

I have been using the Addon in version 2.16 and the shader works fine

Thanks for reporting!
MToon seems to work fine in my Windows11+Blender3.6.0 environment. Please let us know your OS and Blender version so we can investigate.
Also, there is a tutorial on how to use MToon, so check it out https://vrm-addon-for-blender.info/en/material-mtoon/

The shader seems to work great! I just have to manually plug it in MToon, sorry for the misunderstanding. it's just that the older version plug the surface automatically


Install on other platforms

Team,
Is is possible to get this as a docker bundle, or install directions for standard linux, like ubunutu?   (but complete instructions)
Thanks,
Eric

Eric,
I don't see why not. I'll see if I can't put one together in the near future. I'm in the middle of a move right now though, so it might be a little while. If you come up with something, feel free to add it and send a pull request.
In the near term, there are some pretty good tutorials out there for deploying Ruby on Rails apps to Digital Ocean droplets. Also, refer to our wiki (https://github.com/mcpa-stlouis/hack-the-arch/wiki/Documentation) page for more detailed config stuff.
Hope that helps.
Sent from my iPhone

On Sep 12, 2016, at 16:25, Eric Shulze<EMAIL_ADDRESS>wrote:
Team,
Is is possible to get this as a docker bundle, or install directions for standard linux, like ubunutu? (but complete instructions)
Thanks,
Eric
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks,  but I'm having issues getting it to install.
Processing by StaticPagesController#home as HTML
PG::UndefinedTable: ERROR:  relation "settings" does not exist
LINE 5:                WHERE a.attrelid = '"settings"'::regclass
^
:               SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"settings"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
Rendered static_pages/home.html.erb within layouts/application (3.5ms)
Completed 500 Internal Server Error in 13ms (ActiveRecord: 0.8ms)
ActionView::Template::Error (PG::UndefinedTable: ERROR:  relation "settings" does not exist
LINE 5:                WHERE a.attrelid = '"settings"'::regclass
^
:               SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"settings"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
):
1:
2:   Welcome to <%= competition_name %>
3:
4:
5: 	<%= link_to image_tag("mcpa-black.png", width: '80%', alt: "MCPA St. Louis logo",
app/helpers/settings_helper.rb:17:in competition_name'   app/views/static_pages/home.html.erb:2:in _app_views_static_pages_home_html_erb___52037221604440625_44715900'
Any ideas?

Ah, yeah.  You need to initialize the database: bundle exec rake db:seed.

(of course followed by a bundle exec rake db:migrate)

Thanks,  I tried those, and now get this:
Processing by StaticPagesController#home as HTML
PG::UndefinedTable: ERROR:  relation "settings" does not exist
LINE 5:                WHERE a.attrelid = '"settings"'::regclass
^
:               SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"settings"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
Rendered static_pages/home.html.erb within layouts/application (3.4ms)
Completed 500 Internal Server Error in 13ms (ActiveRecord: 0.8ms)
ActionView::Template::Error (PG::UndefinedTable: ERROR:  relation "settings" does not exist
LINE 5:                WHERE a.attrelid = '"settings"'::regclass
^
:               SELECT a.attname, format_type(a.atttypid, a.atttypmod),
pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"settings"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
):
1:
2:   Welcome to <%= competition_name %>
3:
4:
5: 	<%= link_to image_tag("mcpa-black.png", width: '80%', alt: "MCPA St. Louis logo",
app/helpers/settings_helper.rb:17:in competition_name'   app/views/static_pages/home.html.erb:2:in _app_views_static_pages_home_html_erb__4003633606754379878_53107620'

That's the same error... is there something else?
Something is going on with your database.  You might want to check your db permissions. Was the rake migrate command successful?

Yes,  All the commands were successful.   I'm using Postgres for the DB.
On Mon, Sep 12, 2016 at 8:57 PM, Paul Jordan<EMAIL_ADDRESS>wrote:

That's the same error... is there something else?
Something is going on with your database. You might want to check your db
permissions. Was the rake migrate command successful?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mcpa-stlouis/hack-the-arch/issues/13#issuecomment-246550693,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AUM9AIqevB_NNj7GvRnesK4OjXNe5Ljzks5qpgL2gaJpZM4J6_i-
.

--
Eric Shulze
(920) 540-4735
<EMAIL_ADDRESS>

Try a bundle exec rake db:reset.

It might also be worth asking how you're running the server.  Are you running bundle exec rails s?

Eric,
In that last commit, I added a docker-compose and Dockerfile for deploying HTA inside a docker container.  Let me know if that works for you or if you got it to work another way.  If I don't hear anything in the next day or so, I'm going to close the issue.
-Paul

Dockerfile for building image created.  Nothing heard from submitter.  Assuming problem is with his database being improperly configured.  Closing issue.


Test: Changed the story title of the styled/example.

やったこと

Changed the story title to styled/example.

Since the packages are different, we should split the scope.

The VRT test failure appears to be random. Therefore, this PR intentionally ignores it by updating the snapshot. This problem should be fixed.
The failed job: https://github.com/pixiv/charcoal/actions/runs/8244776811/job/22589694612


[BUG/ERROR] Chunk - Class 'SuperClosure\Serializer' not found

FatalThrowableError in ChunkedReadJob.php line 74:
Class 'SuperClosure\Serializer' not found

More info ?

closed... solution :
just add dependency jeremeamia/superclosure in https://github.com/jeremeamia/super_closure
$ composer require jeremeamia/superclosure
thanks... 👍

I just encountered this same error.
ChunkedReadJob.php line 74:
Class 'SuperClosure\Serializer' not found

I've required jeremeamia/superclosure to resolve this but Laravel-Excel should be doing that, not the user.


Structure View Zoom In problem

Once the chemical structure is displayed, it can zoom out properly when the panel is reduced, but can't zoom in as the panel is expanded.

Fixed in 2016-10-31 update

Confirmed fixed. Close.


memory not release

before open picker, the app use memory 30m
then open the picker, select one image,then the view push to the filter view
memory use increase to 110m
then click cancel button on the left navigation to dismiss controller view
memory use 100m
the 70m(100m - 30m) memory use will never release.
did some one also had this problem?

Could you try launching the picker more than once to see if it just keeps growing. Ideally profiling with Instruments.
During the first launch is normal that some internal SDK caching/initialization takes place, thus consuming more memory. This may get released later or on low memory conditions automatically.
As for memory leaks or circular retentions, I think I have checked many times in the past.

I tried launching the picker more than once, it didn't keep growing.
Because of I'm new to iOS developing, I may try profiling with Instruments
as soon as possible.
I'll try to see that if memory get released later.If I find some useful,
I'll contract you.
Thanks for your replying.
The ImagePicker is wonderful ! Thanks for your effort for that.
2015-01-22 9:21 GMT+08:00 Ernesto Rivera<EMAIL_ADDRESS>
Could you try launching the picker more than once to see if it just keeps
growing. Ideally profiling with Instruments.
During the first launch is normal that some internal SDK
caching/initialization takes place, thus consuming more memory. This may
get released later or on low memory conditions automatically.
As for memory leaks or circular retentions, I think I have checked many
times in the past.
—
Reply to this email directly or view it on GitHub
https://github.com/CyberAgent/iOS-NBUImagePicker/issues/12#issuecomment-70955579
.

as the screenshot above, is the profiling with Instruments.
may be as you said, internal SDK caching/initialization took the place

What I usually try is to:

Profile in the simulator with a Release configuration.
Use the app normally repeating actions several times.
"Simulate Memory Warning" in the Simulator. This will purge most internal caches and give you an idea of how well your app behaves.


zinmanga.lat

Source name
Zinmanga
Source link
https://zinmanga.lat/manga
Source language
English
Other details

Acknowledgements

[X] I have checked that the extension does not already exist by searching the extension listing and verified it does not appear there.
[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a meaningful title with the source name.
[X] I will fill out all of the requested information in this form.

Source is not down

Source is not down

Domain is parked.


feat(toast) Use html output element

What is it?

[x] Feature / enhancement
[ ] Bug
[ ] Docs / tests

Description
As described in #239, using output html element improve the a11y to announce a message after a user action or time-sensitive information (which is usually what a Toast is for).
This is a first PR to start playing with the project a little.
Checklist:

[x] My code follows the developer guidelines of this project
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[ ] Added new tests to cover the fix / functionality (Not applicable)

Thanks for the change.
We, the team, need to discuss this before approval because this is one way to implement a11y on a toast.
Another way can be found here: https://react-spectrum.adobe.com/react-aria/useToast.html#anatomy.

Sure no problem. Though I would recommend not to set the role to alert by default. Not all messages in a toast are time sensitive (ex: notifications, confirmation from the server, ...). That's why I think the status role (provided by the output element by default) should be used for most cases and let the user switch to alert for time sensitive message (errors, invalid form, ...)

Closing due to inactivity, but there's a good chance this will be used in the future Toast component of Qwik UI headless!


[Suggestion] allow jumping between hierarchy views

Inside the hierarchy view I'd like to jump to other hierarchy
views I click, instead of going back to the detail description.
For example:
http://swiftdoc.org/v2.1/type/String/hierarchy/
The gray boxes should take me to another hierarchy view
The current/red box functionality should stay the same (detail description).
Thanks!

Of course, that's how it should have worked all along! :facepalm:

ah cool!

Implemented in https://github.com/SwiftDocOrg/swiftdoc-parser/commit/fe45d094137e1c8656344820fcf682a8cf13c084. Thanks again, @masters3d! :tada:


How to use alignedGrids?

I'm wondering if alignedGrids is supported? I see it as a prop but am not able to make it work.

The key was to pass alignedGrids GridOptions[] instead of { api: GridApi, columnApi: ColumnApi}[].


cff2 vf fixes. Make calc_bounds work, fix subsetting bug.

Fixed psCharstrings so that calc_bounds will run. I would guess no-one has tried to use a BoundsPen on a CFF2 VF before - thanks to Chris Chapman. It now returns a result only for the default instance.
Fixed bug in subsetting: removed assert that a Subr is not empty after subsetting or de-hinting. CFF2 Charstrings do not have terminal "return' op.

A note on the calc_bounds fix. There were several problems. First, the op_blend handler for T2OutlineExtractor needed to leave the default font args on the stack, rather than popping off all blend args.  So, the T2Decompiler op_blend handler does that - just don't override it. Then the  T2Decompiler op_blend handler had a bug - it was deleting from the top of the stack, rather than the bottom. Finally, in order for  T2Decompiler op_blend handler to work in the context of a T2OutlineExtractor, we needed to pass the 'private' argument down the chain of super'd init() functions.

@miguelsousa @anthrotype  Can you review this soon?

@readroberts if this is related to calc_bounds @cjchapman is a better reviewer. You need to push the request button tho.


GroupByBenchmark: Fix queryable index generation, improve memory use.

With the old code, all on-disk segments were the same. Now they're different.
This will end up altering benchmark results for queryMultiQueryableIndex,
likely making them slower (since values won't group as well as they used to).
The memory changes will help test with larger/more segments, since we won't
have to hold them all in memory at once.

👍

:+1:


Only set creator when creating task

There was a bug:

UserA creates task
UserB saves task
UserB is now the "created_by".

In other words, "created_by" was acting more like "last modified by".
I'm still pretty new to Django, so please let me know if you have any feedback on this fix. Thanks!

I figured out a simpler way to do it (  #81 ), so I'm closing this PR.
As always, let me know if I'm doing GitHub wrong.

I can reproduce the bug, good find. The bug is that the template sends in the created_by variable at all. But if you remove that line from the template, the task isn't created at all. If you add an else block to if form.is_valid() in the TaskList detail view, you can see the error coming through.
The fix will involve 1) Remove that line from the template code - it shouldn't be there; 2) Modify the view code so that it sets created_by to request.user but ONLY if this is a new task.
I can fix this, but want to give you the chance since it seems like you're an eager contributor :)
As for "doing github wrong," yeah, you want to keep a single PR focused on a single issue. You should keep making changes to the same branch as the PR until it's accepted (there was no need to create a new PR for this - just keep changing your code until we agree it's right). But no worries - we can work on either branch.

Thanks for the feedback! You said to "Modify the view code so that it sets created_by to request.user but ONLY if this is a new task." Are you thinking of modifying the form data here just before form.is_valid(), or something else? If it's ok with you, I would like to see the solution you have in mind if/when you get the chance.
(Sidenote - I'm trying to figure out what flaws that the fix in #81 might have. Is it the risk of a user intentionally modifying created_by ?)

Not quite - I would do it right after this line, when you've got a handle on the task object. Note the argument commit=False which means the form bound to a model object but did not push it to the database. And that means it doesn't yet have an ID! So you can do:
`if not new_task.id:
new_task.created_by = request.user
new_task.created_date = timezone.now()
....

Note I'm moving created_date into this `if` since it's suffering the same bug, so you can fix them both at once.

Also `new_task` is a bad name for the var, since this is for handling new OR existing tasks. So I would rename `new_task` to `task`.

It looks like there are three places that save tasks:

external_add.py (line 44 ish)
task_detail.py (line 101 ish)
list_detail.py (line 55 ish)

task_detail.py, AFAICT, is used if-and-only-if you are editing a task.
So I think list_detail.py adds tasks and cannot edit existing tasks.
Sidenote: I added another commit to remove the seemingly unneeded new_task.created_date = timezone.now() (This mimics external_add.py, which also doesn't set created_date. It looks like the model handles setting created_date because default=timezone.now.)
Thoughts?

These changes will work great, thanks @james1293 . Last step here is to add a test. Would you like to do that, or should I? A good example of the test setup you'll need at todo/tests/test_views.py::test_no_javascript_in_task_note , which shows how to post data to a form "as" a user. So your test should make a task owned by user1 (the users are created in conftest.py), then user2 will log in and make a change to it, then you can assert that the created_by is unchanged. Are you up for it?

I'll give it a shot next week, thanks for the specific suggestions!

Test added.


Adding support for publishing messages with an expiration.

Added WithPublishOptionsExpiration which permits message expiry when the TTL has been reached
Introduced new message var to permit the confidiontal injection of amp.Publishing options in the future.

I needed to add support for publishing messages with an expiration. Therefore find a basic PR for this. Additionally as more options are supported I thought it would be a good idea to add some conditional to the injection of amqp.Publishing options. I was not able to determine a method to write tests for this, but if you have suggests I would gladly write some.


Unnecessary stack trace print

Stack traces should be reserved for critical errors within the plugin requiring attention from the developers.

From #1283:

Instead of printing a stack trace, only send the user friendly error message detailing the problem with the syntax.

So we could possibly pass on the value of Throwable#getCause or the result of #getMessage.
The point of having the stack trace there is that it at least logs to the console more detailed info about why the filter could not be parsed.

That is one option, or implementing a Logging Level configuration option which dictates whether user error stack traces should be printed or ignored.

I don't see why you'd ever want to totally suppress them.
I'm merely suggesting that if we totally remove any trace of the underlying error from the console, it might be a good idea to include a brief description of the issue in the message sent to the user, as you originally suggested.
e.g.
--- common/src/main/java/me/lucko/luckperms/common/locale/message/Message.java
+++ common/src/main/java/me/lucko/luckperms/common/locale/message/Message.java
@@ -128,7 +128,7 @@
     /*
      * Commands
      */
-    VERBOSE_INVALID_FILTER("&4{}&c is not a valid verbose filter.", true),
+    VERBOSE_INVALID_FILTER("&4{}&c is not a valid verbose filter. &7({})", true),
     VERBOSE_ON("&bVerbose logging &aenabled &bfor checks matching &aANY&b.", true),
     VERBOSE_ON_QUERY("&bVerbose logging &aenabled &bfor checks matching &a{}&b.", true),
     VERBOSE_OFF("&bVerbose logging &cdisabled&b.", true),
Index: common/src/main/java/me/lucko/luckperms/common/commands/misc/VerboseCommand.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- common/src/main/java/me/lucko/luckperms/common/commands/misc/VerboseCommand.java
+++ common/src/main/java/me/lucko/luckperms/common/commands/misc/VerboseCommand.java
@@ -79,8 +79,7 @@
             try {
                 parsedFilter = VerboseFilter.parse(filter);
             } catch (InvalidFilterException e) {
-                e.printStackTrace();
-                Message.VERBOSE_INVALID_FILTER.send(sender, filter);
+                Message.VERBOSE_INVALID_FILTER.send(sender, filter, e.getCause().getMessage());
                 return CommandResult.FAILURE;
             }

Agreed.
Please, however, consider Logging Levels for future implementations as it may be possible to forcibly output stack traces on a higher Logging Level for a larger amount of user input errors for debugging purposes.

Luck is limited to the levels provided by the platform. And considering the plugin supports many different platforms, there isn’t really an consensus on anything below the info level. Even for the higher ones there are discrepancies.


[JS] Topic: Iterables, Sets and Maps

Description 📜
Clear explanation of iterables sets and maps with correct use of comments and code.
Fixes # <1833>

Type of change 📝

[ ] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)
[ ] Vi#deo (Animations, screen-recordings, presentations and regular explanatory films are all possibilties etc)
[x] Documentation (Content Creation in the form of codes or tutorials)
[ ] Other (If you choose other, Please mention changes below)

Domain of Contribution 📊

[ ] Android Dev #(Flutter)
[ ] Android Dev #(Java)
[ ] Android Dev #(Kotlin)
[ ] Backend Dev #(Java)
[ ] Backend Dev #(.NET)
[ ] Backend Dev #(PHP)
[ ] Backend Dev #(Python)
[ ] C/#CPP
[ ] Competitive #Programming
[ ] Cyber #Security
[ ] DS#A
[ ] Data#base
[ ] Datascience with #Python
[ ] Datascience with #R
[ ] Frontend Dev #HTML/CSS/JS
[ ] Frontend Dev #React/Angular/Vue
[ ] Go#lang
[ ] Interview #Prep
[ ] Java_#Domain
[x] JavaScript
[ ] ME#RN
[ ] Machine #Learning
[ ] Open #Source
[ ] Python_#Domain
[ ] Ru#st
[ ] Statis#tics
[ ] UI/#UX

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

@Creatoon @dakshsethi @s4-2 Please review. This PR is a recreation of #1881 .

Sir I have modified the README.md file as per the changes you said. Please review it @paramsiddharth

Looks good, but let's break it down into 2 subfolders for Maps and Sets
Where they each contain their own README.md with proper explanation
The main README.md can be kept short.

Looks good, but let's break it down into 2 subfolders for Maps and Sets
Where they each contain their own README.md with proper explanation
The main README.md can be kept short.

Ok sir, working on the changes.

@dakshsethi Sir please review. Made the changes.
@paramsiddharth

@Creatoon @dakshsethi @s4-2 Please review.

@paramsiddharth @dakshsethi please review sir. All changes made.

@Creatoon @s4-2 Please review the PR.


Column CustomHeadRender reuse the sortToggle function

Column CustomHeadRender rehuse the sort function
Right now I want to create a custom head and use the sort function provided by the table.
Current Behavior
right now is not possible,
Proposal
I noticed the sort function is passed on the customHeadRender function so I made a few changes on the MUIDataTableHead component
line 55
column.customHeadRender({ index, ...column }, (columnIndex) => this.handleToggleColumn(columnIndex))

This give us the oportunity of continue using the sortToggle using the current index or other column index. :) I changed the name to columnIndex because the name index is shared inside of the map function on columns array.
    const columns = ["Name", "Title", "Location", "Age", {
        name: "Salary",
        sortDirection: 'asc',
        options: {
          customHeadRender: ({index, ...column}, sortColumn) => {
            return (
              <TableCell onClick={() => sortColumn(index)} key={index}>
                {column.name}
              </TableCell>
            );
          }
        }
    }];

I can create the PR to include this small functionality.

Any updates on this?

Thanks @diegochavez . But if we use this as it is then we lost the sort icon which is provided by default on datatable. We can customize it as follow
const columnHeadRender = ({index, name,...column}, sortColumn) => (
  <th style={columnStyle} onClick={() => sortColumn(index)} key={index}>
    {name}
    <TableSortLabel active={column.sortDirection !== null} direction={column.sortDirection || "asc"}/>
  </th>
)

We can get this TableSortLabel component from
 import TableSortLabel from '@material-ui/core/TableSortLabel';

If someone wants to open a PR for this behavior, I will take a look.

I would find this useful as well; for a tooltip I can use hint but I'd like to just override the basic Sort tooltip and skip the icon without re-implementing the rest of the header.

A quick update to this: a recent update in mui-datatables changed the default value of an unsorted sortDirection to none (as opposed to null).
Anyone relying on the above example should update the code accordingly:
<TableSortLabel active={column.sortDirection !== "none"} ... />

If you continue to use null, it should still work, but you will get a console deprecation error warning you to change the behavior.

hi guys, how about the arrow icon? his direction doesn't change if i use ss sorting

to change the styling of table header, use customHeadLabelRender instead :
customHeadLabelRender: (columnMeta) => (         <span           className="tableheader"         >           {columnMeta.name}         </span>       ),


Nx angular dynamic mfe got NG0203 error

Describe the bug
I had no issuse since i used nx mfe with angular 15,
but i tried to migrate to angular 18 and got this error when navigating from shell (host) app to another mfe
Error: NG0203: inject() must be called from an injection context such as a constructor, a factory function, a field initializer, or a function used with runInInjectionContext. Find more at https://angular.dev/errors/NG0203
i also saw this topic
https://github.com/nrwl/nx/issues/19121
and then i tried it on my two envirements which is a windows system and debian linux, and in both cases got the same error.
Reproduction
clone this repo
https://github.com/mehrabix/nx-angular-mfe
do npm i
run this command:
npx nx serve shell --devRemotes="first-microfront,second-microfront"
Used Package Manager
npm
System Info
System:
    OS: Linux 6.1 Debian GNU/Linux 12 (bookworm) 12 (bookworm)
    CPU: (4) x64 Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
    Memory: 2.24 GB / 7.53 GB
    Container: Yes
    Shell: 5.9 - /usr/bin/zsh
  Binaries:
    Node: 22.5.1 - /home/linuxbrew/.linuxbrew/bin/node
    npm: 10.8.2 - /home/linuxbrew/.linuxbrew/bin/npm
    pnpm: 9.5.0 - /home/linuxbrew/.linuxbrew/bin/pnpm
  Browsers:
    Chrome: 126.0.6478.114

Validations

[X] Read the docs.
[X] Read the common issues list.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Module federation issue and not a framework-specific issue.
[X] The provided reproduction is a minimal reproducible example of the bug.

Hi,
I got some clues. Clone this repository (it's mentioned in the Angular MFE tutorial: https://nx.dev/recipes/angular/dynamic-module-federation-with-angular):
https://github.com/Coly010/nx-ng-dyn-fed
Run:
npx nx serve employee --devRemotes="dashboard,todo,login"
The Angular version is 18.0.4, and it works perfectly.
Then, run this in order to migrate to Angular 18.1.2. It will migrate successfully.
After that, run:
npx nx serve employee --devRemotes="dashboard,todo,login"
again.
It will not work anymore, and you will get the NG0203 error.

@mehrabix
Can you open this issue on the Nx Repo? It’s not directly related to this package.

Hi @Coly010
I don't think it's related to Nx.
as i mentioned it's work on Angular 18.0.4 and not 18.1.2
Anyway
i did it. Thanks
https://github.com/nrwl/nx/issues/27162

This bug most certainly is not related to Nx. I'm doing the same thing with an Angular 17 remote and receiving the same error. In my case, however, my host is a single spa MFE using systemjs externals, @angular-architects/module-federation to load this remote, and I'm trying to use this example to point the angular dependencies to the systemjs map. Now, before I tried figuring out this remote plugin stuff, I noticed the remoteEntry file can't even figure out how to share the angular dependencies so I've been trying to reverse engineer the cause of this and haven't found anything yet. This enhanced mod fed plugin is missing something the webpack version isn't.
extra-webpackconfig.js
const modfedOptions = {
    library: { type: "module" },
    filename: "remoteEntry.js",
    name: 'CarrierPigeonApp',
    exposes: {
        './Component': './src/app/app.component.ts', // this is a standalone component
    },
    shared: //{ ...deps },
    share({
        // '@angular/core': {
        //     singleton: true,
        //     strictVersion: true,
        //     requiredVersion: 'auto',
        //   }
        ...shareAll({ singleton: true, strictVersion: true, requiredVersion: 'auto' }),
    }),
    // runtimePlugins: [require.resolve('./npm-runtime-global-plugin')],
    dts: false,
    disableManifest: true
};

module.exports = (config, options) => {
//   const singleSpaWebpackConfig = singleSpaAngularWebpack(config, options);

    config.experiments = {
        outputModule: true
    };
    config.output = {
      ...config.output,
      uniqueName: "CarrierPigeonApp",
      // depending on what imports the remote entry determines if this succeeds or not
      publicPath: "auto",
    };
    config.optimization = {
        ...config.optimization,
        runtimeChunk: false
    };
    config.plugins = [
        ...config.plugins,
        new ModuleFederationPlugin(modfedOptions)
    ];

  return config;
};

angular.json
"build": {
          "builder": "@angular-builders/custom-webpack:browser",
          "options": {
            "outputPath": "dist/carrier-pigeon-app",
            "index": "src/index.html",
            "polyfills": [
              "zone.js"
            ],
            "tsConfig": "tsconfig.app.json",
            "assets": [
              "src/favicon.ico",
              "src/assets"
            ],
            "scripts": [],
            "main": "src/main.ts",
            "customWebpackConfig": {
              "path": "extra-webpack.config.js",
              "libraryName": "CarrierPigeonApp",
              "libraryTarget": "umd"
            }
          },
          "configurations": {
            "production": {
              "budgets": [
                {
                  "type": "initial",
                  "maximumWarning": "500kb",
                  "maximumError": "1mb"
                },
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "2kb",
                  "maximumError": "4kb"
                }
              ],
              "outputHashing": "none"
            },
            "development": {
              "optimization": false,
              "extractLicenses": false,
              "sourceMap": true,
              "outputHashing": "none"
            }
          },
          "defaultConfiguration": "production"
     }

main.ts
import('./bootstrap')
	.catch(err => console.error(err));

bootstrap.ts
import { bootstrapApplication } from '@angular/platform-browser';
import { appConfig } from './app/app.config';
import { AppComponent } from './app/app.component';

bootstrapApplication(AppComponent, appConfig)
  .catch((err) => console.error(err));

package.json dependencies

The shared resource map compiled using the enhanced mod fed plugin

The shared resource map compiled using webpack's mod fed plugin

As you can see, swapping out mod fed plugins leads to different shared mappings which is probably a separate bug altogether, but should give a hint that this plugin has a problem with Angular projects.

Figured out my problem was using angular-architects' shareAll function. For some reason I needed to specify my shared dependencies the manual way and everything just worked afterwards

Angular Architects might be incorrect for V2


Extendability problem

Hello,
first of all thanks for the interesting and great project! :)
I want to use Bulma CSS in my hobby project. Bulma uses section as core element. I can't implement this:
class Section extends HTMLWidgetBase {
...

because HTMLWidgetBase is marked as @internal. The only way to add it is to copy the Rad source code and use it locally.
It will be great if (at least) HTMLWidgetBase is exposed to other packages. Thanks!

Thanks for bringing this up.
Another issue:
To implement a widget that renders section element, we've to return section from correspondingTag. This is something we cannot do as we can return only a valid DomTagType value from correspondingTag and there's no entry for section in the DomTagType enum.
In order to fix this we've to first add entry for section in the enum DomTagType and then you can:

Option 1: Use HTMLWidgetBase (but like you said this requires making HTMLWidgetBase public).

Option 2: Use a existing HTML widget (e.g span).
class Section extends Span {
  const Section({
    super.key,
    super.ref,
    super.id,
    super.title,
    super.style,
    super.className,
    super.hidden,
    super.innerText,
    super.child,
    super.children,
    super.onClick,
    super.additionalAttributes,
  }));

  @override
  DomTagType get correspondingTag => DomTagType.section; // important!
}

runApp(
  app: Section(innerText: 'hello'),  // <section> hello </section>
  appTargetId: 'out',
);

On framework side:
Here's what I think we should do:

Add all the missing entries(not just section) to the enum DomTagType.
Add Section widget to framework HTML widgets(so that you don't have to implement it).
Maybe add HTMLWidgetBase to public API. I'm not sure whether we should do that. I mean its functionality is easily accessible(as shown in option-2) through any HTML widget(span, div, ...). But if you still want it, we can make it public.

If you or anyone else would like to work on any specific part, please feel free to do so (but do let us know here so that I don't start working on the same thing).

I've made all the changes, including adding missing entries, making HTMLWidgetBase public, and adding Section widget.
The new version (v1.7.0) is live. Enjoy! 🚀


Parallelized GCCA

Reference Issues/PRs
#19
What does this implement/fix? Explain your changes.
Adds partial_fit() functionality to GCCA for batch loading and parallelizes the initial SVDs of views with joblib.
Any other comments?

Looks like there are a couple extraneous files, like checkpoint.model and .vscode settings


Added TS sample for 'Team member added' section

This is not the same as the C# sample (I don't know where that sample came from that is included in the 'Team member added' section).
This TS sample came from the master branch of the botbuilder-js repo:
https://github.com/microsoft/botbuilder-js/tree/master/libraries/botbuilder/tests/teams/conversationUpdate/src

Docs Build status updates of commit cd46d9c:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md
:warning:Warning

Details

msteams-platform/includes/bots/teams-conversation-bot-sample.md
:warning:Warning

Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md

Line 32: [Warning] Invalid link: 'A Member added to team'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Team-Member-Added'.
Line 33: [Warning] Invalid link: 'A Member was removed from team'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Team-Member-Removed'.
Line 476: [Warning] Invalid link: 'Reaction to bot message'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Reaction-to-a-bot-message'.
Line 477: [Warning] Invalid link: 'Reaction removed from bot message'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Reactions-removed-from-bot-message'.

msteams-platform/includes/bots/teams-conversation-bot-sample.md

Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 149e0dc:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md
:warning:Warning

Details

msteams-platform/includes/bots/teams-conversation-bot-sample.md
:warning:Warning

Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md

Line 32: [Warning] Invalid link: 'A Member added to team'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Team-Member-Added'.
Line 33: [Warning] Invalid link: 'A Member was removed from team'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Team-Member-Removed'.
Line 507: [Warning] Invalid link: 'Reaction to bot message'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Reaction-to-a-bot-message'.
Line 508: [Warning] Invalid link: 'Reaction removed from bot message'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Reactions-removed-from-bot-message'.

msteams-platform/includes/bots/teams-conversation-bot-sample.md

Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 1e9e79b:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md
:warning:Warning

Details

msteams-platform/includes/bots/teams-conversation-bot-sample.md
:warning:Warning

Details

msteams-platform/bots/how-to/conversations/subscribe-to-conversation-events.md

Line 32: [Warning] Invalid link: 'A Member added to team'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Team-Member-Added'.
Line 33: [Warning] Invalid link: 'A Member was removed from team'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Team-Member-Removed'.
Line 501: [Warning] Invalid link: 'Reaction to bot message'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Reaction-to-a-bot-message'.
Line 502: [Warning] Invalid link: 'Reaction removed from bot message'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'Reactions-removed-from-bot-message'.

msteams-platform/includes/bots/teams-conversation-bot-sample.md

Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).
Line 3: [Warning] Invalid file link:(~/foo.md).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


MVC with RIBS

Hi @dev4jam,
I have an application which is already built using MVC and now I'm planning to start using RIBs and migrate the app step by step. So I created a RIB for example ProfileRIB and I want to added it to my TabBarController. is there a way I can combine RIBs with MVC or the whole application have to be built using RIBs?
Thanks

Hi @mohammad19991. Definitely, it's possible. In fact, I've done this several times. One concern though - you'll need to implement several protocols from RIBs.

but for me it still not clear what needs to be done, for example I created ProfileRIB and now I want to add it to my TabBarController:
let builder = ProfileBuilder(dependency: AppComponent()).build(with: context)
let profileViewController = builder.viewControllable.uiviewController
tabbar.viewControllers = [vc1, vc2, profileViewController]

doing this will cause memory leak and crash the app because am not attaching this (child)RIB to a parent RIB.

@mohammad19991 you may try to implement your Profile RIB as your own LaunchRouter (just name it differently) and confirm to LaunchRouting protocol. And override launch method to embed your viewcontroller into the tabbar.


Test bumping hyperkube to v1.8.4

Pay no attention to this.

coreosbot run e2e

Pass when bump applied after d2e6328b4dded4285b88d0de63bf6b89ca418854 ^

Pass when bump applied after c637bd531b91399f8d5caa3a65e20fce36def178 ^

Pass when bump applied after a905d0e4a645a054e172dbd9a333dd01bc3deb3b ^


Error: must provide image ID and image name to use an image: invalid argument

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description

Steps to reproduce the issue:

Install Podman 1.3.2 on RHEL 7.7 with basic repo, extras and beta, and make a test user without root rights.

Login with the test user.

Try to run multiple containers.

Describe the results you received:
Second container gives error that i need to provide Image ID.
[test@rhel77 ~]$ podman run -d nginx
25042d922c25eaf726518a87460b51d12b5ecc791b2e988349738832935c86c1
[test@rhel77 ~]$ podman run -d nginx
Error: must provide image ID and image name to use an image: invalid argument
Describe the results you expected:
That the second container would start.
[test@rhel77 ~]$ podman run -d nginx
25042d922c25eaf726518a87460b51d12b5ecc791b2e988349738832935c86c1
[test@rhel77 ~]$ podman run -d nginx
d12b5ecc791b2e988349738832935c86c125042d922c25eaf726518a87460b51
Additional information you deem important (e.g. issue happens only occasionally):
When I stop the only container active I can start one again.
Output of podman version:
podman version
Version:            1.3.2
RemoteAPI Version:  1
Go Version:         go1.10.8
OS/Arch:            linux/amd64

Output of podman info --debug:
podman info --debug
debug:
  compiler: gc
  git commit: ""
  go version: go1.10.8
  podman version: 1.3.2
host:
  BuildahVersion: 1.8.2
  Conmon:
    package: podman-1.3.2-1.git14fdcd0.el7.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 1.14.0-dev, commit: 4780496c1f6949e21c807e17b57c183e9424c90a-dirty'
  Distribution:
    distribution: '"rhel"'
    version: "7.7"
  MemFree: 174981120
  MemTotal:<PHONE_NUMBER>
  OCIRuntime:
    package: runc-1.0.0-83.dev.gitf56b4cb.fc28.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  SwapFree:<PHONE_NUMBER>
  SwapTotal:<PHONE_NUMBER>
  arch: amd64
  cpus: 1
  hostname: rhel77.michiel
  kernel: 3.10.0-1049.el7.x86_64
  os: linux
  rootless: false
  uptime: 139h 46m 20.66s (Approximately 5.79 days)
registries:
  blocked: null
  insecure: null
  search:
  - registry.access.redhat.com
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.centos.org
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 18
  GraphDriverName: overlay
  GraphOptions: null
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 2
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

Additional environment details (AWS, VirtualBox, physical, etc.):

This is strange, I don't see an issue on Fedora.
If you attempt to run nginx a third time, does it continue to give you the same error?
Of course I am using a slightly newer version.
podman version
Version:            1.4.0
RemoteAPI Version:  1
Go Version:         go1.12.5
OS/Arch:            linux/amd64

Can't reproduce with 1.3.2, rootless, on F30. I'll try 7.7 later, but this one seems like it could be an environment issue.

@rhatdan I'm using podman 1.3.2, runc 1.0.0-rc8+dev and slirp4netns-<IP_ADDRESS>
[test@localhost ~]$ podman ps
CONTAINER ID  IMAGE                           COMMAND               CREATED         STATUS             PORTS  NAMES
5cf40bbb4d91  docker.io/library/nginx:latest  nginx -g daemon o...  23 seconds ago  Up 23 seconds ago         infallible_napier
[test@localhost ~]$ podman run -d nginx
Error: must provide image ID and image name to use an image: invalid argument
[test@localhost ~]$ podman stop 5cf4
5cf40bbb4d91f7ec92a66e61bda2c98f6c98ffbf2c8e68f628daa3f84a138950
[test@localhost ~]$ podman run -d nginx
7468b471f8995ce268748902feca34c4b606e913f14b0210f044a1b018076171
[test@localhost ~]$ podman run -d nginx
Error: must provide image ID and image name to use an image: invalid argument
[test@localhost ~]$ podman run -d nginx
Error: must provide image ID and image name to use an image: invalid argument
[test@localhost ~]$ podman run -d nginx
Error: must provide image ID and image name to use an image: invalid argument
@mheon Just installed it on another system. Same error occurs.

Alright, so it's consistent, but seemingly only on RHEL7
@michielhemme To confirm, are these fresh installs of Podman, or upgrading existing installations that have run Podman before?

@mheon, the podman installation is fresh out of the repositories. It is being delivered with runc-1.0.0-59-dev and trying to start a container then results in:
[test@localhost ~]$ podman run -d nginx
Error: container create failed: cannot specify gid= mount options for unmapped gid in rootless containers
: internal libpod error
after upgrading runc to the latest version, I get the other error. That the first container can start but not the second one.

Could this be a vfs storage issue?  That is one difference.  RHEL7 does not support fuse-overlay.

Can't replicate with VFS on F30 - still seems RHEL specific

@giuseppe Any ideas?

the 7.7 kernel is missing some ioctl's we are using.  We will need to backport 4dca13e70423e657aec98999d9c6803075359517

it applies cleanly on top of v1.3.2, and it seems to solve the problem on RHEL 7.7

Is that in 1.4? We're expecting a 1.4 with 7.7, so if it's available there
we should be set
On Tue, Jun 18, 2019, 04:05 Giuseppe Scrivano<EMAIL_ADDRESS>wrote:

it applies cleanly on top of v1.3.2, and it seems to solve the problem on
RHEL 7.7
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/containers/libpod/issues/3345?email_source=notifications&email_token=AB3AOCF4U4SMD7KEDHLGJNTP3CJLPA5CNFSM4HYTJ2G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX5RO6A#issuecomment-502994808,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB3AOCEXVZCB6YE46IUGLPLP3CJLPANCNFSM4HYTJ2GQ
.

yes, it is included in 1.4

Closing since it is in Master and soon to be everywhere.


Feature request: Layer2 scanner in diagnostic tools

I'm asking if is it possible to have netdiscover, or arpscan, into OPNsense, it could be helpfull in Interfaces/Diagnostic tools to see which IP are available in a LAN.
I do not have any preferences about netdiscover or arpscan even if I use netdiscover frequently, arpscan is good enough.
I can also develop it by myself and then ask you a pull request, here in github, first I need to know if this feature is needed by others users.
It could be great having also a generic arpping and a arpwatch process, if possibile :)
These could alert when ip collisions happens.
let me know if interested,
bye

Installing it in OPNsense isn't difficult at all, I've just typed:
pkg add http://pkg.freebsd.org/freebsd:11:x86:64/latest/All/arp-scan-1.9.txz
Together with arpwatch another good tool, easy to integrate with a plugin interface, could be arpalert.
These tools could build together a good layer2 security system, isn't it?
`arp-scan can be used as simple as it come
root@OPNsense:~ # arp-scan --interface=em0 <IP_ADDRESS>/24
Interface: em0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.9 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)
<IP_ADDRESS>	00:27:19:d5:16:a2	TP-LINK TECHNOLOGIES CO., LTD.
<IP_ADDRESS>	00:00:74:ab:bc:fc	RICOH COMPANY LTD.
<IP_ADDRESS>	c4:6e:1f:b9:15:e1	(Unknown)
....
`

Hi @peppelinux, a pull request would be probably the best option as the time of the developers is limited. For your linked script: Please note that there is no bash on OPNsense.
Here you can find a How To: https://docs.opnsense.org/development/examples/helloworld.html
In your case, I would recommend you to run the tool async (detached) with configd.
You can look at any existing plugin if you like.

Yes, no bash in OPNsense :)
Actually I installed arp-scan via ssh but when time will permit I could code this plugin and then ask to you a merge.
Thank you Fabianz, OPNsense is a very good product.

It would be better if you would ask @fichtner for review and merge.

I read the sourcetree,
It seems to me that I should not replicate "Diagnostic: Packet probe" tool but only the tools with the new UI approach. I'd like to reuse part of Packet probe UI InputFields for example, like the interface selection (select menu).
I also think that Arp scanner could be included in the Core

I've found diag_packet_capture.php in /usr/local/www as a legacy component.
I think that this kind of tool is very similar of what I'm going to develop so I think that I will use diag_packet_capture.php like a sort of template and the arp scanner will be, hopefully, coded in the core of OPNsense.

@peppelinux please note that diag_packet_capture.php is based on legacy code, better not to use as a template. New tools will likely remain in the package repository, there's no real reason to extend the core system for every install.

You can look at the quagga diagnostics pages if you like. They may behave very similar.

@AdSchellevis ok, so I have to avoid this approach and code in the new coding style.
Ok, I will, let's code this plugin
@fabianfrz Thank you, I'll start from there

system -> diagnostics (top process list) could also be of interest, should be quite easy to read.
Controller + template:
https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/controllers/OPNsense/Diagnostics/ActivityController.php
https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/views/OPNsense/Diagnostics/systemactivity.volt
API calls:
https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/controllers/OPNsense/Diagnostics/Api/ActivityController.php
and the link to configd:
https://github.com/opnsense/core/blob/master/src/opnsense/service/conf/actions.d/actions_system.conf#L1-L5

@AdSchellevis Thank you, those examples looks very good for my situation, no problem for MVC pattern, I'm a Python Django coder. Just a question, a curiosity, I read that applications are, most of them, simple python scripts that works like a wrapper, managing I/O at the OPN's api/backend.
According to this, could it be more accurate to evaluate the possibility to upgrade python interpreter to 3.6 (at least, time is changing and python is now a very important component) and coupled with this, to install SCAPY python network framework, to do a lot of diagnosys and packet forgery directly in OPNsense framework (through scapy).
Could it be a good approach?

@peppelinux sorry, we're stuck on 2.7.x at the moment (which is still supported). 2.7/3.x aren't very compatible unfortunately.
A switch might be something for next year, but it turns out to be quite some work to keep all compatible on both versions and the gain isn't very large unfortunately (I did some performance tests some time ago).
You could use https://pypi.python.org/pypi/netaddr , which is installed by default. Scapy should be compatible with 2.7 too, if you need it, you could ask in the tools repository for a build https://github.com/opnsense/ports/tree/master/net/scapy
(maybe @fichtner can deliver a package when he has some time available)

Thank you, brilliant support, nothing more to say :)
I'm coding arpscanner according to WOL plugin example (I can easily follow its small code).
Scanner result will be available in the bootstrap dialog, like WOL actually does.
After the first pull request I think that we'll decide how to emprove, as design and usage too.

A mini MVC plugin is certainly mdns-repeater

Thank you @fichtner but I cannot see or install mdns-repeater plugin from OPNsense web UI interface.
I'd like to view it before using it as a template.
Is there any of the default installed application, in the current stable version of OPNsense, that is MVC compliant? I want to use a working example and not write it totally from scratch.

If you want to work on plugins you should work with the repository :)
(from the OPNsense box run these)
opnsense-code plugins
cd /usr/plugins/net/mdns-repeater
The "official" plugin will be available for OPNsense itself with 17.7.2 later today.
Cheers,
Franco

@fichtner thank you very much, I'm learning a lot of things with your help.
I'm coding with mdns-repeater I've found in this git repo. You know that arp-scan is a system package dependencies.
Makefile, in the root of the plugin folder, should contain the declaration of the system dependencies.
I'm actually start to code it :)

sure, you put your package dependencies on this line:
https://github.com/opnsense/plugins/blob/master/net/mdns-repeater/Makefile#L5

@fichtner good to hear that my code can be used as a template 👍

Do we have a setup script that put the plugin's  files/folder in their /usr/local/opnsense/mvc/app/* tree or we have to put them manually, each by one?

make install

This is the wrapper I coded for run, as a system command, the arp-scan tool:
#!/usr/bin/env python2.7

"""
    Copyright (c) 2017 Giuseppe De Marco<EMAIL_ADDRESS>    All rights reserved.

    Redistribution and use in source and binary forms, with or without
    modification, are permitted provided that the following conditions are met:

    1. Redistributions of source code must retain the above copyright notice,
     this list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above copyright
     notice, this list of conditions and the following disclaimer in the
     documentation and/or other materials provided with the distribution.

     IN NO EVENT SHALL THE
    AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
    SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
    INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
    CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
    ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
    POSSIBILITY OF SUCH DAMAGE.
"""
from subprocess import Popen, PIPE
import json
import re

_DEBUG=False

RFC1918_NETWORKS = ["<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/16",
                    "<IP_ADDRESS>/8"]

class ArpScanner(object):
    def __init__(self, ifname, network_list):
        """
        netif='eth0'
        network_list=['<IP_ADDRESS>/24', ...]
        """
        self.ifname         = ifname
        self.network_list   = network_list if network_list else RFC1918_NETWORKS
        self.outputs        = {} # raw outputs from system command
        self.result         = {} # filtered output containing only what needed

    def start(self):
        # first loop will not block the others
        for net in self.network_list:
            os_command = ["arp-scan", "-I", self.ifname, net]
            self.outputs[net] = Popen(os_command, stdin=PIPE, stdout=PIPE, stderr=PIPE)
            output, err = self.outputs[net].communicate()
            returncode = self.outputs[net].returncode
            if _DEBUG: print(returncode, output, err)
            self.outputs[net] = returncode, output, err

            regexp = re.findall('([0-9\.]+[\t])+([\dA-F]{2}(?:[-:][\dA-F]{2}){5})', output, re.I)
            for netfound in regexp:
                self.result[net] = netfound[0].replace('\t', ''), netfound[1]

        # second lood will fetch resultset
        # no way, exception with parallel execution on the same ifname
        # ERROR: failed to send packet: No buffer space available\n
        # for net in self.network_list:

    def view_outputs(self):
        for res in self.outputs:
            print(res)
            print('return code: {}\n'.format(self.outputs[res][0]))
            print(self.outputs[res][1])

    def get_json(self):
        return json.dumps(self.result)

if __name__ == '__main__':
    import argparse
    parser = argparse.ArgumentParser()

    # This is the correct way to handle accepting multiple arguments.
    # '+' == 1 or more.
    # '*' == 0 or more.
    # '?' == 0 or 1.
    # An int is an explicit number of arguments to accept.
    parser.add_argument('-i', nargs='?', required=True, help="network interface")
    parser.add_argument('-r', nargs='+', help="""multiple network ranges,
    as: <IP_ADDRESS>/24 <IP_ADDRESS>/12
    If not specified it will scan all the RFC 1918 local area networks.""")
    args = parser.parse_args()

    if args.r:
       plural = '' if len(args.i) == 1 else 's'
       print('Network{} to scan: {}'.format(plural, ' '.join(args.r)))
    print("On interface: {}".format(args.i))

It can also return a json object, according to API needs, like this:
{"<IP_ADDRESS>/24": ["<IP_ADDRESS>", "00:27:19:d5:1e:56"]}
soon as possible I will let you read the code directly on my fork

It could be runned as a standalone program as well.
arpscanner_wrapper.py -i wlp2s0 -r <IP_ADDRESS>/24

If you want to run it not in parallel, is there something that would be against <IP_ADDRESS>/12 to reduce the invocations?

Good question,
with arp-scan we have a problem, every scan do a sort of lock on the memory/interface/buffer.
At first time, this morning, I coded a distribution in different forks BUT arp-scan works on the first run but the others, in parallel, returns 1 with this message:
ERROR: failed to send packet: No buffer space available\n
Using scapy or someother tool could improve the things but actually arp-scan must be serialized.
Another good strategy could be to store the stdout of scapy on a buffer and read it. in realtime, with "Audit" feature of OPNsense UI, this will prevent to wait for a very long network scan over all the RFC1918 networks. Probably this will be the next use-case that I'd like to expose to you.

@fabianfrz the only way could be splitting it in different /CIDR

is there a command to reload OPNsense web definition, templates and other stuff without restart the entire VM/server?
When I made changements in the src/mvc tree on a working OPNsense installation, do I have to execute a  command to flush and reload cache/definitions/stuffs or at each http request all the things will be reloaded, automatically as usual in PHP? (I never use falcon and the web internals of OPNsense are unknow to me at the moment)

if your plugin is already installed:
# make upgrade

is there a OPNsense SHELL cheatsheet on the wiki?
command = goal in a short format
@fichtner Thank you. I'm looking how many files we are producing for this MVC, they're a lot :)

sure :) the initial setup is a bit tedious still, but for the sake of privilege separation and clean coding it's necessary
https://github.com/opnsense/plugins#a-brief-description-of-how-to-use-the-plugins-repository

@peppelinux
Probably:

2 Controllers (1 Frontend, 1 Backend)
Your worker script
A config file for configd
1 HTML template (volt)
1 ACL file (not required but recommended)
1 XML File for linking into the menu

Today's small steps:
https://github.com/peppelinux/plugins
Tomorrow I have to check how to set a Select InputFields (not that textfield actually in the form) for select one interface only, not multiple, at time.
I'm not working all the time on this  little project 'cause we are developing other things for my job but we are tryng to do some experiments on OPNsense and, actually, it's driving quite well in production (college/academy). Very thanks for all the good work you've done

Hi @peppelinux,
Looks good so far. :)
I will add arp-scan to our packages tomorrow so that the plugin can be merged and build in the future.
There is, however, no rush to finish this, take your time and thanks in advance for your efforts and kind feedback!
Cheers,
Franco

FYI https://github.com/opnsense/tools/commit/2002f25d

@fichtner thank you, I'll do my best
@fabianfrz, I've found a patch regarding of what you told me yesterday:
arp-scan -I eth2 <IP_ADDRESS>/12

I pasted in the code the networks classified by /24 classes, because of a previous script that used netdiscover, it accepts only /8-16-24. So, all that 172.16.x.0/24 networks in the arpscanner script will be
replaced by one <IP_ADDRESS>/12

A lot of things are more clear to me know, tomorrow I will introduce ajax call for selecting interfaces (is there any suggestions about this, shortcut or whatever) and I would like to implement realtime "audit" from now instead of produce a rounder box, with the scan output, after an http response. I think that for a euristic scan, like most of them, I need the audit feature because of scanning time on /16 networks.
I do not seen any doc about audit default template, is it hacky or you still have to produce a template either for this?

You can store the progress into a file which you can query via the API. If the file exists, you also know that a scan is running.

See the firmware page.

@fabianfrz, thank you. File is always good, I will in this traditional way.
Is there a strategy to avoids concurrent reading/writing on the same file from different http_requests? I think that I will have to deal with it.
Is there any possibility, in the future, to use redis to store concurrent resultset and flush them after a timeout of N minutes/hours?

you have no HTTP request which will write that file. What you can do is checking if your script is already running and abort if it is (so the file will not be written again).

@fabianfrz I also dealed in the past with this kind of concurrent executions. This example works with backup schedule, it simply avoids that to identic jobs could start in concurrency:
https://github.com/peppelinux/LinuxNotes/blob/master/rsync_storage_flock
I think that putting a file descriptor with a wrapper around it could be a solution if there will be the possibility to develop something to be normalized for all the other similar tasks, without any preliminary lock file or search in the system processes.
It works in the script wrapper, It assign to the kernel a FD to himself with always the same ID. If another job execution is still present as the same FD in the kernel the scripts will not run its system processes.
This could introduce us to introduce two other feature:

the possibility to share running scan between users (a user can see the current scan, on a interface, running from the click of another user)
the possibility to STOP a running scan (if it was created with wrong arguments or if it will be escaled with another priorities)

just some thoughts, thank you fabianfrz, I'll do something that will work as we expect,
any further sharing of knowledge could come back to us in a good way :)

We already use flock in the firmware updates to share the progress output of the command line tools and seamless GUI page refresh / auto-attach:
https://github.com/opnsense/core/blob/5cc1515bfd15beda0e89d5b7804768532f77918d/src/opnsense/scripts/firmware/launcher.sh
Some other parts use daemon command + pid files to track job progress, that's also easily possible.

I'm tryng to fetch interfaces from opnsense internal functions/tools (otherwise I should wrap ifconfig command in PHP).
Is there a actions reference documentation for configd?
I need to use it in my model methods, I would like to use configd instead of hardcode something huge :)

Everything you need is in the configuration file. If you use a "Interface Field" in the model, you will get them to the frontend.

@fabianfrz I used "Interface Field" but I can only read "nothing selected" into the select field.
This means that a api call should fill this select, I mean.
I'd also like to include and recycle some of the old functions I read into the legacy code, like "get_interface_list() ", but I cannot get them working into Phalcon approach.

Yes the box has to be filled using the API request - This is what the mapDataToFormUI call does. https://github.com/opnsense/plugins/blob/master/net/mdns-repeater/src/opnsense/mvc/app/views/OPNsense/MDNSRepeater/index.volt#L32-L36
Please note that the field names between the form and the API result MUST match.
I would recommend you to install the mdns-repeater plugin from the plugins page and then open the page while you are having the network tab of the developer tools of your browser open.
You will see the result of the API requests, which where one will include the list of the interfaces.

@fabianfrz Thank you, I just added:
        formatTokenizersUI();
        $('select').selectpicker('refresh');

Into Helloworld template I found in documentation
        mapDataToFormUI(data_get_map).done(function(data){
            // place actions to run after load, for example update form styles.
        });

Now it works, I can view and select the interfaces names like "lan", when I'll post it I'll have to retrieve the ifname of lan (it would be em0) and use it with arp-scan. Is there a API method to get all interfaces objects with all their attributes?

see src/etc/inc/interfaces.inc in core - there is a function called get_real_interface in line 3899. @fichtner can explain you how to load it properly.

we don't load legacy files / functions from the mvc.. if you really must create bounce scripts to be called via the backend to return your required information to the mvc.
there may be some functions in there that support native reading from the library code, but since we largely haven't migrated interface management that side is still underdeveloped.

@fabianfrz yes I was tryng to use it but includes doesn't works and Phalcon throws to me some exceptions...
@fichtner, I'm now following mdns responder but I cannot run arpscanner with "lan" as argument, I need some internals API calls, or legacy one, or python/shell script to be runned in thearpscanner api controller. I need to forge a good strategy for create this primary build environment for this and some other utility, I'm actually tryng to link the pieces for this

you can still access the config directly (https://github.com/opnsense/plugins/pull/266/files#diff-aeb1ac0012d65409cc950908761cd638R51) and load the real interface name from the config.

@fichtner could I use configd and configure my bounce scripts as configd service? I just have to deal with the stdout fetching from a configd call.
For getting interfaces names (as em0) instead that interfaces ZONE name (as lan) I'd use this system command
ifconfig -l -u | sed  -e "s/lo0//g"

Should I develop a custom Field Type to retrieve this information globally in the project? Because if I use custom system command to get informations I will cannot use anymore the phalcon form wizard.
@fabianfrz Do you mean "ProxySSO()"?

@peppelinux yes, the configd approach is perfectly fine and recommended :)

@fichtner please suggest to me a working example, I'm spendng too much time on this task :(

@peppelinux that's not for you - this is a comment for the author of the pull request I did review.

https://github.com/opnsense/core/blob/master/src/opnsense/scripts/routes/routes_wrapper

https://github.com/opnsense/core/blob/42e8e49b48ef429a7859fd7322252d5081bde42f/src/opnsense/service/conf/actions.d/actions_interface.conf#L87-L90
the magic is "script_output" it returns the stdout of the command

@fabianfrz I need a working example of how to retrieve interfaces from the global config of OPNsense, I mean objects, not strings

the output of any script can be json encoded data which can be mapped back to an associative array or object.

@fabianfrz so I'm going to fetch interfaces name (em0) from a custom configd command, taking its stdout and the exploding it to the arpscanner.general.interfaces, in SettingsController, when the first api call will be made, calling /api/arpscanner/settings/get.
Ok ok I will :)

Is there any example using confid with async (detach strategy)?

from php just add a second parameter true

Is there a method to pass arguments to configd actions?
...Or the only way is to save and reload values in the template.conf?
In other words configd action could take arguments by command line?
Thank you for support

Actually the arp-scan works through configd, with arguments from command line as well.
Now I have to develop the audit feature, the lock file.
I've also a bug, I cannot get to save the values from UI to templates, as hellowolrd tutorial explains. Soon as possible I will try to fix it.

@peppelinux please create a pull request so the code can be reviewed. Just prefix it with WIP:  as long as you don't want it to be merged.

Is it good for my detach/fork this?
https://github.com/opnsense/core/blob/master/src/opnsense/site-python/daemonize.py
I'm asking if could I reuse some of your code

@peppelinux not sure what you want to achieve, if you want to build a daemon, yes that's the way, if you just want to wait for a longer lasting proces, no.

configd calls are forked by default using subprocess (https://github.com/opnsense/core/blob/17.7.1/src/opnsense/service/modules/processhandler.py#L460)

@AdSchellevis Yes :) I'm looking for a smart way to manage long running process in OPNsense.
I'm going to put all the I/O in files named by the ifname. The API will read these on every ajax polling, having $ifname in _POST. When the arp-scan fork process will end will unlock its usage, if /api/arpscanner/service/stop every arp-scan process will be killed and their file destroyed. This will be a very simple approach, the father process will exit normally and the child process will take so much time as needed to end the job.
If there are not usable classes in the current OPN release I think to use multiprocessing from python stdlib. Will it be good enough? :)

like I said, the daemonize wrapper is certainly fine when creating daemons.... if you have a process which takes longer, just detach and wait, nothing magical to do there.

This works too... daemon(8) is very convenient ;)
https://github.com/opnsense/core/blob/8cfabc94d42366286e5d5c228f59ab8815f896e1/src/opnsense/service/conf/actions.d/actions_firmware.conf#L51
and
https://github.com/opnsense/core/blob/8cfabc94d42366286e5d5c228f59ab8815f896e1/src/opnsense/scripts/firmware/launcher.sh

Or tracking via pid works, too...
https://github.com/opnsense/plugins/blob/8b5af2e44a3cb27424d58c6123edc3934cb8f5c8/security/clamav/src/opnsense/service/conf/actions.d/actions_clamav.conf#L32
and
https://github.com/opnsense/plugins/blob/929cca97e6ac3b563d33b736e99ef45103b4cacf/security/clamav/src/opnsense/scripts/OPNsense/ClamAV/freshclam.sh

@AdSchellevis thank you, I think that
from subprocess import Popen

it will be good enough, multiprocessing sounds more overkill :)

Nothing more to say, the plugin works well with long scan tasks.
The only thing I will improve for next releases (and subsequent plugins) is the problem with the buffered subprocess. Open FileIO.
With python2 the best way is to scroll simultaneously, with some python3 options working better as expected. I've tried a different strategy but I'm not fortunate in python2 with that popen configuration. For the result, the user must wait until the end of the subprocess. It just works fine, it would be better with async io, but for now it is still a working and usefull tool.

can you add a small distance between the buttons and the form above?

Yes, I've just made 13px between them. You can find it in the last commit

should I close this issue.

Yes but I still not seeing the plugin into the last release. Is there something wrong?

We need a final statement from you to make a release from the last development version. It’s your plugin so your decision. 😊

Thank you Franco, excuse me if I missed something.
What I have to do now to give to you this final statement and make the release of the last development version?

you just need to tell me, fraenki, ad or franco to make it public visible. Just tell us that it is ready.

arp-scan plugin is now in the next stable release, this issue can be closed now


Always clear ContentType cache to enforce determinism

I had a check failure on another PR, and I am pretty sure it is not from that patch, so I'm making this for first discover the test bugs we have, and then fix them on a case-by-case basis after identified.

For records, this forces the following failure, and then fixes it:
def test_raise_an_exception_if_the_key_is_cached_but_the_new_key_is_the_same(self, random_public_key, mocked_http, create_mock_method):
        # We are going to:
        #     1. return a key which will not work with jwt_token provided by mocked_http
        #     2. Cache a key that is invalid
        #     3. Error when the new token is the same as the cached token

        mock_field_dicts = [
            {"key": random_public_key, "cached": True},
            {"key": random_public_key, "cached": False},
        ]

        # Pretend the key is coming from a URL
        url = 'https://example.com/'
        with override_settings(ANSIBLE_BASE_JWT_KEY=url):
            # 1. Make the get_decryption_key always return the random key (which is invalid) && 2. pretend the key is cached
            with mock.patch('ansible_base.jwt_consumer.common.auth.JWTCert.get_decryption_key', create_mock_method(mock_field_dicts)):
                # 3. Make the call.


fix resource limits and scaffolding code

This commit:

Removes the customizations made to resource limits in manager, as with
the default limits are already higher.
Fixes the watches and leaderelection Id flags which were missing in the
scaffolding.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 89.693%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1375

Relevant Lines:
1533

💛  - Coveralls


Cleanup local pod split

We don't need to pass in environment vars anymore
because of service accounts. We're always running the apb
incluster so oc login will use the service account to login.
Partially-fixes #196

 Remove the incluster pod creation path

Depends-on: https://github.com/fusor/apb-examples/pull/58

Merged https://github.com/fusor/apb-examples/pull/58, 2 ACKs on this, merging.