ENH: Added scipy.spatial.distance.directed_hausdorff

Background
This PR aims to implement a fast algorithm for the Hausdorff distance, which is used in a variety of scientific fields including computer vision (Huttenlocher et al. (1993)), computer graphics, path similarity analysis (Seyler et al. (2015)), and many others.
The concept is perhaps most clearly explained with a simple example of two 2-D point sets (or paths) that are being compared:

The directed Hausdorff distance is effectively the maximum closest distance between a point in one data set and a point in the other. So, from path 1 to path 2 the directed Hausdorff distance is very close to 1.0, because all points on that inner circle will never have a closest distance to a point in the outer data set that is much greater than that (the outlier in path 2 is never closest to a point in path 1).
Conversely, the directed Hausdorff distance from path 2 to path 1 is much greater than 1.0 because the outlier will have a minimum distance to a point in path 1 that is quite large.
This asymmetric property of the directed Hausdorff distance is typical of "maximini" functions, but it is important to note that it far more common for an end user to be interested in the symmetric or undirected Hausdorff distance. The reason I'm proposing to implement directed_hausdorff is because it has its own specific uses in some cases, and it is more general -- you can use it twice to calculate the symmetric (general) Hausdorff as follows:
d (symmetric) = max ( d (directed a->b) , d (directed b->a))

Which was indeed one of the docstring examples I wrote.
Implementation Details
There are relatively straightforward brute force approaches to calculating the directed Hausdorff distance using scientific Python:
def directed_hausdorff_brute_force(u, v):
    return max(np.amin(scipy.spatial.distance.cdist(u,v), axis = 1))

However, the above approach obviously has polynomial time complexity -- you have to calculate all possible distances. As discussed by Taha and Hanbury (2015), you can write an algorithm that (on average) performs far better, by incorporating a break into the distance calculation and shuffling the input data to increase the likelihood of a performance-boosting inner loop break. That is the approach I've used here for scipy.spatial.distance.directed_hausdorff, so let's see how it performs compared to the competition.
Benchmarking and Python Competitors
There is a Python implementation of the symmetric Hausdorff distance in MDAnalysis (another project I work on), and I was actually writing unit tests for that when I thought about how efficient it actually was, and whether we really want multiple python libraries trying to reimplement this. Let's see how brute force approaches, the MDAnalysis algorithm (adapted to the directed calculation), and the Cython version I've written (with some helpful feedback from others) compare. There's a fourth condition where the innermost loop has been hardcoded (on the assumption that only 3D data is handled -- so it has an advantage over my general implementation proposed here).

And zooming in for the competing Cython implementations I've written:

Remember that there is a stochastic component to the performance (whether the inner loop breaks often) -- which may partly explain the roughness in places.
I'm not sure whether you think it might be worth hardcoding the square distance calculation loop for say 3 to 5 dimensional (column) data and falling back to the loop for higher (less common) dimensions. For now I've just left it in the more general (and perhaps arguably more maintainable) form that iterates through columns / dimensions.
Reference: Jupyter notebook with benchmarking code
The only other Python implementation I know of is the medpy Hausdorff distance code -- though it appears to be using a brute force approach, I can't say for sure based on initial inspection.
You can easily find stack overflow questions where users are interested in using python to calculate the Hausdorff distance and the answers suggest there's no easy way to do this in scientific python.
Considerations
Of course, I still have to link this to the mailing list to see if the community is actually interested -- will do that shortly. This is my first de novo Cython module, so you may find some things you don't like in there. My integration into the rather large distance.py is sure to require at least a minor revision -- I know I don't do the same checks on the input data arrays that most other scipy distance calculations seem to do -- they have to be np.ndarray or else at the moment!
It is also worth considering that the scipy distance design is very modular, and allows for say 'euclidean' to be switched with many other distance metrics for a given function. The same could be done for directed_hausdorff -- it is currently only dealing with Euclidean, but perhaps that is the more obvious case at the moment and we might just sensibly leave the door open to other metrics (any valid norm would be possible in theory).

very nice PR summary!

and +1 for the maintainable version, special-casing 3D-5D doesn't seem needed.

I think Richard and I have addressed the first batch of revisions. I have one further question -- should I include the cythonized _hausdorff.c source? I know some packages do that on top of having a Cython-based setup process for various reasons.
As for summarizing the justification for the (directed) Hausdorff distance over the various other flavours of Hausdorff distance & general consideration of competing metrics I'll write a few points below, with the caveat that my formal training isn't in computational geometry (I just apply it for work sometimes/ read books occasionally). I've made a few others aware of this PR if they wish to chime in.
"Competing" metrics to consider:

Other flavours of Hausdorff: The paper by Besse and coworkers cited below states that "Among
Shape-Based Distances,  the  Hausdorff  distance  (Hausdorff,  1914  [11]), and  the  Frechet  distance  (Frechet,  1906[12])  are  likely  the most well known." That's probably the main reason I chose to use the "general" Hausdorff distance -- I knew about it, and it seems that's the main one others know about too. Just to pick one of your examples, the Yau-Hausdorff distance, it was first introduced in 2015 in a biological context -- while I'm working in a somewhat related field -- is scipy really the place for something that domain-specific / recent -- at least until the citations really build up, etc. The robust HD that you mention does look more interesting and more heavily cited. In one such robust approach, the Euclidean distance is replaced by the cost function that can eliminate outliers (i.e., compare images with more noise) -- this sounds like further motivation to have a 'modular' philosophy for the Hausdorff calculation -- perhaps eventually allowing end users to feed in a metric argument.
The Frechet distance is definitely worth a look -- that wikipedia article describes some of the broad uses / differences and advantages over Hausdorff. It is also used in the path similarity paper I cited at the beginning of the PR (which some of my colleagues wrote). A quick scan of the google scholar results for Frechet distance shows a fair amount of literature, and some of the better algorithms do indeed appear rather involved, but could be well worth the effort for biological path and hand-writing recognition (!) purposes. There are many subtypes: weak Frechet, discrete Frechet (pure python implementation is available in MDA, but we could probably Cythonize + choose an optimal [read: probably harder to implement] algorithm), geodesic Frechet, and homotopic Frechet.
Expanding Hausdorff and related things to use a Manhattan distance metric is worth considering because of the increased usage / relevance of automobile tracking / GPS / automation type stuff in looking at paths (i.e., Besse et al., 2015).

There's a stack exchange post that is quite relevant for this stuff as well. Other, somewhat related, things like Procrutes analysis appear to be covered by PyGeometry.
I might suggest that i.e., a Frechet implementation would fall outside the remit of this specific PR, but if you think the scipy community would be receptive to it that would definitely encourage me to start chipping away at it when possible.

Also of note: there appears to be a stochastic error in the Python 3.4 portion of the testing / build matrix (2.7 and 3.5 pass).

should I include the cythonized _hausdorff.c source?

no, we don't check in generated files anymore. Keeps the repo size and commit noise down. Cython is a dependency for development builds (we do include generated files in releasse tarballs).

Is there no way to further accelerate the process using spatial data structures (for example, kd-trees)? This seems like it might lead to a rather dramatic improvement, and it might be possible to implement it efficiently using the existing interface to these data structures.

@aarchiba I'll try to test an approach that uses kd-tree when I have some time. I'll probably link the source so you can check it if the results aren't what you expect, etc.
Also, it is worth noting that the first author of the path similarity paper I cited above (Sean) posted some interesting benchmarks in another thread using sklearn that you may want to look over. I responded below that post, but I'm biased to the Cython implementation so you should probably judge for yourself.

@aarchiba Also, please note the tests that use more than 1000 dimensions -- are you confident that a kdtree can perform comparably at that level of dimensionality? The discussion in the above thread suggests they cannot.

@tylerjereddy - Indeed, kd-trees will not be useful at dimensions above about 20. So they would supply an alternative implementation at best, and I fear to use them efficiently you'd want an algorithm that short-cut the tree traversal.

Ok, I've gone ahead and done a refined set of benchmarks testing minor code adjustments based on feedback here and elsewhere:

Based on the above results (source Jupyter notebook), I suggest:

Using typed memoryviews is clearly superior to using the old numpy buffers in Cython, and it is now the recommended way to do this -- I think we can agree I should probably push the memoryview adjustment into the branch unless scipy uses a really old version of Cython sometimes? Indeed, the numpy buffer syntax is effectively described as a historical curiosity in Kurt W. Smith's "Cython: A Guide for Python Programmers" textbook.
@cython.wraparound(False) is not helping, and is actually making things worse, so this performance decorator is not warranted (it was not included in the original PR anyway)
usingscipy.spatial.ckdtree to replace the loop nesting / point searching produces a back-and-forth performance battle with the vanilla implementation; @aarchiba what do you think? The function that uses ckdtree in the above notebook is directed_hausdorff_memoryviews_ckdtree. I didn't explicitly type the tree object -- could that or anything else be done / helpful to improve the ckdtree-based approach?

A few other notes:

Some colleagues suggested that float would provide better performance than double, but I think in the scipy context we want to use double, right?
There is something to be said for identifying "Hausdorff pairs--" the indices of the points from each array that correspond to the returned distance. My impression is that the need for this functionality is domain-specific. I'm not sure what you think about the idea of returning both the Hausdorff distance and the corresponding indices?

@tylerjereddy I've run your notebook at my own machine (Core i7-4600U). For me the performance between the 3 versions is the same. I don't see any difference. But I was able to get a little bit better performance on my machine by assigning the np.inf to a variable outside of the loops. gist

@kain88-de It seems most of the improvements at this stage are proving to be pretty minor / negligible. Your suggestion of typed memoryviews is probably still solid though since that's the recommended approach now, even if the performance effectively matches here.
I also thought about assigning break_occurred the unsigned int type, but I can't imagine this would help much, if at all.

I think we can agree I should probably push the memoryview adjustment into the branch unless scipy uses a really old version of Cython sometimes?

At the moment we require Cython >= 0.22, and we're happy to bump that up pretty aggressively if needed (reason: no Cython required for releases, just for development). So yes, go for memoryview.

Some colleagues suggested that float would provide better performance than double, but I think in the scipy context we want to use double, right?

Indeed, can't use float where there's double now, that'll lose accuracy.

I'm not sure what you think about the idea of returning both the Hausdorff distance and the corresponding indices?

Even if usage of the second return value is field-specific, now would be the time to add it. It's easy to discard a return value if not needed. Adding return values later is a pain. So if the return value is unambiguous (no changed definitions between fields for example), then why not?

typed memoryviews are now being used
the return value is now a tuple with (distance, index u, index v) -- note that because the algorithm fundamentally depends on shuffling the input data, I had to 'unshuffle' to get input-appropriate indices -- we should check the performance cost of this (see below)
I've attempted to add asv perf benchmarks for directed_hausdorff, but something isn't working -- that name is not available to the benchmark suite when imported, even after a clean re-build of my working branch (I must be missing something simple)
The docstring and tests have been updated as well -- the tests all pass (including the new index functionality), but I'd be surprised if at least a minor additional change wasn't needed for the docstring
Since we now depend on a fixed random seed to 'unshuffle' the indices returned to the user, there may be an argument for giving the user the option to provide a keyword argument with an integer value for the seed -- if their data is particularly pathological & they are really unlucky, our specific seed may not produce the best  likelihood of breaking the inner loop; conversely, this is probably very unlikely in practice and I suspect it is substantially less painful to add a kwarg down the line since a default can be set to ensure backwards compatibility

I did manage to benchmark before / after the code was modified to return a tuple:
asv continuous --bench Hausdorff 0cc120b04def22beba3f9d65e85cb82e853d79ea 963d71377b30b06fd7250868004f5937586cfd93
output:
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For scipy commit hash 963d7137:
[  0.00%] ·· Building for py2.7-Cython0.22-Tempita0.5.2-numpy1.8.2-six................................................................................................................................................................................................................................................................................................
[  0.00%] ·· Benchmarking py2.7-Cython0.22-Tempita0.5.2-numpy1.8.2-six
[ 50.00%] ··· Running spatial.Hausdorff.time_directed_hausdorff                                                                                37.84μs;...
[ 50.00%] · For scipy commit hash 0cc120b0:
[ 50.00%] ·· Building for py2.7-Cython0.22-Tempita0.5.2-numpy1.8.2-six......................................................................................................................................................................................................................................................................................
[ 50.00%] ·· Benchmarking py2.7-Cython0.22-Tempita0.5.2-numpy1.8.2-six
[100.00%] ··· Running spatial.Hausdorff.time_directed_hausdorff                                                                                53.12μs;...
BENCHMARKS NOT SIGNIFICANTLY CHANGED.

I've also managed to run the benchmarks individually now (though I have to manually terminate to prevent all other benchmarks from running):
[most recent commit / using return tuple + index code]:
asv run 0cc120b04def22beba3f9d65e85cb82e853d79ea --bench Hausdorff -e
               ============ ==========
                num_points
               ------------ ----------
                    10       52.81μs
                   100       388.23μs
                   1000       4.13ms
               ============ ==========

[commit before tuple / index return code was added]:
asv run 963d71377b30b06fd7250868004f5937586cfd93 --bench Hausdorff -e
               ============ ==========
                num_points
               ------------ ----------
                    10       36.42μs
                   100       325.97μs
                   1000       3.37ms
               ============ ==========

the unit test probing the returned indices (for the Hausdorff pair) has had its test data increased in size to properly detect issues related to shuffling / seeds
random seed fixing has been removed from the Cython code for directed_hausdorff
default values are now set for the unsigned ints involved in returning indices -- this is to prevent segfaults when u == v (the unit test for degeneracy will segfault otherwise, because we'd be indexing arrays with non-existent variables)
Max Linke has been added to the author list in the Cython source

Thank you for doing the benchmarking!
If getting at the indices is actually difficult then I would prefer it to be implemented in SciPy – otherwise people might have to copy and paste the SciPy codeto bolt on this functionality.
I don't see the performance hit as too bad, especially as I consider the indices valuable information but maybe other people have other opinions. In that case, could you switch off the code path that gets the indices with a kwarg and return None, None instead?

On 24 Oct, 2016, at 10:13, Tyler Reddy<EMAIL_ADDRESS>wrote:
I've also managed to run the benchmarks individually now (though I have to manually terminate to prevent all other benchmarks from running):
[most recent commit / using return tuple + index code]:
asv run 0cc120b04def22beba3f9d65e85cb82e853d79ea --bench Hausdorff -e
           ============ ==========
            num_points
           ------------ ----------
                10       52.81μs
               100       388.23μs
               1000       4.13ms
           ============ ==========

[commit before tuple / index return code was added]:
asv run 963d71377b30b06fd7250868004f5937586cfd93 --bench Hausdorff -e
           ============ ==========
            num_points
           ------------ ----------
                10       36.42μs
               100       325.97μs
               1000       3.37ms
           ============ ==========

--
Oliver Beckstein *<EMAIL_ADDRESS>skype: orbeckst  *<EMAIL_ADDRESS>

@orbeckst I could probably do something like that. The Hausdorff pair (index) returning functionality is bolted directly onto the loop code though, so I suspect to restore 100 % of performance we'd basically need two versions of the 'working part' of the code to be chosen by a kwarg switch. I agree that the performance hit isn't that bad though.
@rgommers I'm not sure if there's a style-preference here in the scipy context for the kwarg, or if you'd prefer to just calculate the indices in all cases.

I could also take a look at using cython.parallel.prange and releasing the gil to allow use of multiple threads. The innermost loop that iterates over dimensions may benefit for input with ~1000 dimensions for example, but in 3 dimensions the threading overhead might be a bit much.
Releasing the gil for the entire nested loop might even be possible as apparently the break statement is supported in parallel.
I suspect this would be a bit of an OpenMP rabbit hole, but might be worth a quick check. I'm not sure how much more painful this is to benchmark though. Offhand, I'm thinking that, for example, asv runs each benchmark in a single process? And I'm not sure how much more complicated it becomes to support fallback code that doesn't use OpenMP if that would also be needed (?), etc.

I suspect this would be a bit of an OpenMP rabbit hole, but might be worth a quick check

For an example of going down the rabbit hole right now see https://github.com/scipy/scipy/pull/6722.

And I'm not sure how much more complicated it becomes to support fallback code that doesn't use OpenMP if that would also be needed (?), etc.

cython.parallel.prange handles this gracefully (i.e. just becomes a normal loop)--the more annoying bit is getting the setup.py right.

+1 for releasing the GIL wherever you can. Don't write a prange loop that's on by default though - this will be annoying for users who want themselves to parallelize larger chunks of code (then they get multiple levels of parallel, which will just slow things down instead of up). If there's really a lot of benefit of parallelization, use n_jobs in the API like cKDTree does.

unit tests have been added to check that directed_hausdorff does not consume a singleton of the global RandomState
directed_hausdorff now creates its own instance of RandomState and consumes singletons of the latter local instance for shuffling rather than consuming singletons from the global hidden instance
directed_hausdorff now allows a user-specified (int or None) seed argument that may be used for the local generator; the default is 0, consistent with the example @ev-br provides above, such that the default behaviour is reproducible random shuffling that consumes local singletons only

Sadly, I've not seen any improvement (indeed, quite the opposite) when using prange to release the GIL. With the current algorithm, we only have one option I can see for using prange -- on the innermost loop. This is because the two outer loops both contain variable reads / comparisons, and those are strictly prohibited because prange does not iterate in a defined order (compiler error is produced). Loop breaking to avoid spurious distance calcs depends on knowledge of values obtained in previous iterations of the loop, and that paradigm is not compatible with an undefined-order parallel for loop implementation like prange.
The benchmarks I ran for the inner loop prange were so bad I didn't even bother to plot the results (correct values are returned, but yikes!):
[the PR implementation alternates with the prange "enhancement" version]
points_per_array: 200
result: (0.2092898405651567, 21, 41)
avg_time: 0.000483274459839
result: (0.2092898405651567, 21, 41)
avg_time: 1.30585193634
points_per_array: 733
result: (0.14679621471095292, 114, 593)
avg_time: 0.00214958190918
result: (0.14679621471095292, 114, 593)
avg_time: 12.8816492558
points_per_array: 1266
result: (0.13700819630895097, 464, 806)
avg_time: 0.0029247601827
result: (0.13700819630895097, 464, 806)
avg_time: 25.5623489221

It could very well be that using prange at the third nesting level of a loop is just asking for bad performance anyway, or that the overhead here is just far too great for whatever reason. Unless cython has an alternative parallelization scheme for loops that retains memory of previous iterations in some way (seems like it would need a completely different "locking" approach for that), I suspect attempts at modifying the algorithm to suit prange would be a bit of a fishing expedition. There probably are scenarios where switching back to brute force and using all available threads would beat this non-brute force approach, but it may require quite a large number of cores and a specific range of data / dimension sizes.
I suppose we could consider coding in a switch that uses prange with a brute force algorithm that doesn't care about loop iteration order. But, in that case we're basically just calculating a big distance matrix and trying to compete with i.e., distance.cdist or cKDTree, and presumably those are already pretty good at what they do. As seen near the start of the PR discussion though, the current algorithm easily outperforms cdist at 50k points in 3D using a single thread.

@rgommers As far as I can tell, the cmax usage is fine -- looks like you may have confused a comparison in your logic. cmax is guaranteed to change in the first iteration of the loop in all cases, and then subsequently only when a new "maximum minimum" distance is found. cmax is the (squared) directed Hausdorff distance after all -- the units tests would almost certainly fail if the most important value was not correctly handled.
I think I followed your suggestions for all the other changes, and rebased against master while I was at it.
I somehow didn't notice that there is also scipy.spatial.procrustes -- that is another kind of shape analysis. If directed_hausdorff makes it in, then the Frechet distance (discrete & maybe continuous) would probably nicely round out the shape analysis code in spatial.

Also, perhaps noteworthy that CGAL apparently does not provide code for the Hausdorff and Frechet distances, at least according to one online discussion, so we wouldn't just be duplicating their efforts.

looks like you may have confused a comparison in your logic.

indeed. Somehow the way that logic worked doesn't fit my brain. Made some suggestions to make it clearer.

Frechet distance indeed seems useful too.

I think I've dealt with all the comments -- no significant change in performance according to asv
I cut out the final np.inf comparison -- that wasn't needed; note that the algorithm pseudocode in the paper doesn't even use the break checking boolean variable that was causing confusion, but my attempts to remove that variable from the logic entirely have failed; this could reflect differences between pseudocode "intention" and cython / python logic I suppose
what do you think about adding reciprocal "See also" docstring links between this and scipy.spatial.procrustes?

I think the Travis test failures are strange -- probably just need to trigger a restart on those.

restarted, but I'd expect those to be real. haven't seen any nonreproducible segfaults before

You're right -- commit 5d2fcbdd where the cmin != np.inf check was removed causes a segfault on a linux machine. I was almost certain the unit tests passed on my mac before pushing though. I'll have to investigate further.

The tests do pass on my mac. The difference between the setups is the version of Cython used -- building with 0.23.4 causes a segmentation fault and with 0.24.1 the tests all pass. I've restored the portion of the Cython code affected -- we can always perform the streamlining of the code when another Cython version bump occurs in scipy [but, see logic below].
In short, the final cmin != np.inf check is surprisingly a Cython version dependent part of the code, which I would not have expected.
My best guess based on the Cython 0.24 release notes is that this bug fix (involving infinity checks) may be involved:

isinf() declarations in libc/math.pxd and numpy/math.pxd now reflect the actual tristate int return value instead of using bint

Note on actual logic: it seems likely that if infinity is being handled incorrectly, and reintroducing an infinity check allows the code to work, then the logic as it appears is not what it seems -- placing that final infinity check in there circumvents the Cython bug for whatever reason by introducing some new behaviour (which might also depend on the Cython bug!). In fact, it seems likely that I probably saw this segfault in the early development phases and figured infinity was somehow getting through, so I put a check in to prevent it, but actually infinity was never getting through, I just accidentally found a way to circumvent the Cython bug.
Obviously, the better long term solution is to have logical code with the bug-fixed version of Cython.

Should probably put the cmin != np.inf clause first so that if there is an infinite value the check will short-circuit before you get to the potentially-buggy cmin > cmax.

moved the cmin != np.inf clause as @person142 suggested

this needs a rebase

Changes since last review LGTM

Rebased. I note the addition of one or two blank lines where I removed the git >>>>> rebase diff flags instead of deleting the entire line using vim when dealing with merge conflicts. I can try to clean those out if needed.

I can try to clean those out if needed.

no that seems fine as is

Let's get this in! Thanks @tylerjereddy, all

One last leftover suggestion:

what do you think about adding reciprocal "See also" docstring links between this and scipy.spatial.procrustes?

that seems like a good idea, maybe do that in a separate PR


[pattgen, doc] Questions for pattgen specificification

I went through the Pattgen specification document and some issues are not clear to me (for DV development)

When a pattern is completely generated, do we need to clear ctrl.enable_ch0/ctrl.enable_ch1 before re-programming registers then re-enable those bits to generate new patterns?
Can ctrl.enable_ch0/ctrl.enable_ch1 bits be set within the progress of a pattern generation? if yes, does it terminate the current pattern then start the new pattern?

Do we need to update the specification document?
Original PR: #3235

Answers:

Yes.  For simplicity, all setting changes are ignored while the pattern is set. Updates do not get reloaded from the TL-UL block until enable is deasserted.
Deasserting enable kills the currently running pattern, re-establishing enable starts a new pattern.

Please let me know if the spec can be any clearer on these two points.
As to the question 1. The spec clearly states:
"Please note that writes to a channel's configuration registers have no effect while the channel is enabled. For operational simplicity, the configuration registers are only transferred into the internal finite state machines while a channel is disabled. Changes to the configuration registers only take effect once the channel has been disabled and re-enabled."
As to Question 2 it states, after describing the entire process:
"The entire sequence can be restarted either by resetting or disabling and re-enabling the FSM."

As for describing the halting of the device: There is another sentence which states:
The pattern can be started (or halted) on either channel by setting the corresponding CTRL_ENABLE bit to 1 (or 0) for the desired channel.

thanks for clarification


Trouble with installing the plugin on Ionic 3 Lazy Loading

Hello please help me I would like to know how to add this plugin on my project ionic3 Ionic 3 Lazy Loading  thank you. Sory for my  bad expression

Hi @durvilchat, import SelectSearchable to each "lazy" page module.
import { LazyPage } from './lazy';
import { SelectSearchableModule } from 'ionic-select-searchable';

@NgModule({
	declarations: [
		LazyPage
	],
	imports: [
		IonicPageModule.forChild(LazyPage),
		SelectSearchableModule
	]
})
export class LazyPageModule { }

Hi @eakoriakin Thank you very much


Azure Event Hubs Source Beta

Purpose of this pull request
This pull request adds new Azure Event Hub source (beta). This will eventually replace C2C version.

Select the type of change:

[ ] Minor Changes - Typos, formatting, slight revisions, .clabot
[ ] Update Content - Revisions, updating sections
[x] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - Updates, maintenance, dependencies, new packages for the site (Docusaurus, Gatsby, React, etc.)

To see the specific tasks where the Asana app for GitHub is being used, see below:

https://app.asana.com/0/0/1205849078823071

@kimsauce @jpipkin1
We can add a statement :
This source over comes the documented challenge of Azure C2C source scalability.
"Collecting data from Azure Event Hubs using this Cloud-to-Cloud collection source has a supported throughput limit of 1MB/S (86GB/day) for a named Event Hub egress rate"
Reference of existing source : https://help.sumologic.com/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/azure-event-hubs-source/

@kimsauce @jpipkin1  - I think we should go ahead and publish the doc without the statement on the change.
I am working with engineering to get some comparison number with respect to scale.  I will make those edits and submit a PR.
Can we merge the PR so we can add the Doc link to the Beta Announcement Page.

@kimsauce, I fixed a broken link that was causing the deploy test to fail with commit "Fix broken link in FedRAMP article".

@kimsauce, I fixed a broken link that was causing the deploy test to fail with commit "Fix broken link in FedRAMP article".

You beat me to it, thank you!!


How should I create or authenticate a JWT per view?

Hello,
Is there a way to generate or authenticate a token to grant access only for a specified view and be invalid on other views?
This may sound odd but I want to issue a JWT token to share some info with frontend and grant access to a sensitive view, I'm using JWT at another view too but it is less important than this one.
When I started to think about both, I found that if someone swapped one with another it will pass through verification and authentication as it's a valid JWT token issued by me but not for a specific view, so is there a way to do that?
Should I make my custom JWT authentication class, verify class and check for my required payload too? or there is something more simple? or should I use something else, but JWT is the one here because you need to share some info with the client.
Should I change secret and issue the new one with a different secret? should I use PyJWT? I don't care about linking it to a user, I'm even using AnonymousUser for the existing one, what do you think?
Thanks, everyone

This is not the point of SimpleJWT. SimpleJWT is an authentication library, but it's not widely used as an authorization library, which is what you seek. If you want per-view access, you need an authorization library; in that case, you might as well use django OAuth toolkit.
If you don't want to set up that way, you can include a token claim (sample in documentation) that specifies a "custom" code for a view. Let's say you had a view called "PingViewSet". You'd access the token claim and see if the user has access to this view. The token claim'll be unencrypted since it's in the payload, but it won't matter since the user can't edit the token (it'll fail authentication).

Thanks for the fast response!
I'm actually using django OAuth toolkit at my project but the need to share data doesn't leave any other choices, forgive me if my description made an impression that I'm seeking an authorization level but I think it's more like a new or separate JWT authentication class at my system so each one is on its own.
Your suggestion should work, I was thinking about the same to be done at verification endpoint but at the view level, I prefer if we could come up with something that works on authentication level or permission levels like overriding or writing my custom class for those.

Hm a new authentication class / new authentication.py's? Or something like DRF permission classes / Django built-in permission model? Perhaps you could provide an example; I'm just slightly confused what you mean. Although, it seems like whatever I suggested means that I do understand... This part specifically:

the need to share data doesn't leave any other choices

The default authentication class suggested in the doc just uses the PyJWT package to verify the user. I think you're trying to say something like object-level permission, right? Let's say you had a url like: example.com/post/123/ in which 123 is the post's pk. You give a token to the user but then that user gives/swaps with another user. That other user shouldn't have that token.

The following is what I assume you're talking about:
What you've described is a Man in the Middle attack in which the second user stole the first user's token. This is also why SPA (e.g. React and Vue) is not officially supported and is still a WIP for this package.
SimpleJWT is a stateless method for authentication (usage of JWT in general is stateless). The Django built-in method for authentication is session cookies, a stateful method of authentication which prevents this "swapping," which use the httpOnly flag. Currently, SimpleJWT can't prevent users from "giving away" their tokens. The only way they can be "given" away is through code error or manually giving it away.
Please let me know if I'm getting this wrong. I think it's best you give me an example of what you're trying to accomplish.

Again, Thank you for the time and effort you put into this, I really appreciate that!

What you've described is a Man in the Middle attack in which the second user stole the first user's token. This is also why SPA (e.g. React and Vue) is not officially supported and is still a WIP for this package.

This is the case where I'm exactly concerned for, I'm using JWT to give access for anonyms users to submit their feedback, it's an invitation URL link with a JWT token attached to, so whoever has the correct link will submit their feedback, now I want to use JWT to complete my user's signup, but at first glance, I thought that if a malicious person takes a JWT token from an invitation link and tried to access the signup page with it he will pass authentication as it still valid for JWTTokenUserAuthentication  to pass although it hasn't the correct payload and should fail later it should be like that with such a sensitive resource.
So as you can see it's on a view authentication not on an object level.
I will start with some code snippets first to end the confusion, I think it explains the idea better:
I was thinking for something like this
class CompleteSignup(viewsets.GenericViewSet):
    authentication_classes = [JWTCustomSignupAuthentication]

Where JWTCustomSignupAuthentication inherits from a BaseJWTAuth class or something and I write down my logic fro this, like using another secretKey to verify the token or the correct payload, ... etc.
I started to think that different secretKey will be enough to accept or reject a token immediately.

In this case, you shouldn't be using JWT at all. Verification links work by sending the user a random link which expires in which authentication isn't needed at all (check out django-allauth. It does exactly what you describe and is very extensible and feature rich). For signup, you're sending a verification via EMAIL which shouldn't be able to be MITM attacked if sent to the user. Again try django-allauth.
Are you saying that you're sending a link with credentials for automatic sign in upon entering the link?
I wasn't confused about the code implementation. The confusing part is the concept you're trying to explain.

I completely agree with you about django-allauth, but it is not a verification link.
I don't have a user at that moment, I'm sending the email to start the signup process, the user has entered some values previously like email and first&last name then proceeds to payment, on successful payment a user should receive a link start the signup process, the info entered before should be used again at the client as prefilled or sent with the new data he enters for a signup, so I have to share that info, my choices for JWT because I can do both at once (authentication and share info), I know that this can be done by all auth style but there will be more steps to achieve that, I thought that JWT is the right choice for that - or that's what I thought-.

Hm I just started using Django stripe, and even without djstripe, many payment services, even if you're custom making one, really want you to have a user first. Anyways,...
What you're describing is the same method as verification links. I believe they're hash generated. The object allauth creates contains the email and confirms is at L154 (they use django.core.signing). This is so that you can include a payload of a specified record in a database that points to the purchase (you do still have a record of the payment, right, for tax purposes?). Doesn't seem like too many steps to me.
Also, adding some random info to the JWT is harmful. The payload of a JWT is not encrypted.

What you're describing is the same method as verification links. I believe they're hash generated. The object allauth creates contains the email and confirms is at L154 (they use django.core.signing). This is so that you can include a payload of a specified record in a database that points to the purchase (you do still have a record of the payment, right, for tax purposes?). Doesn't seem like too many steps to me.

This sounds helpful, but the last point is not my use case as there is so much redundant data that I won't save to my payment record so I'm only keeping a transaction_id along with email so I can get full details when needed form payment service and avoid data redundancy, with JWT I will only include these on the fly like first name and last name and forget about it but with allauth I will have to keep everything I need for this.

Also, adding some random info to the JWT is harmful. The payload of a JWT is not encrypted.

sure but since it was sent to a user's email I shouldn't worry about MITM attack as you mentioned that, only the owner of this info will see it if he decoded that and also it is not a critical data to be exposed, just some fields.

Sorry for the relatively late reply.
The MITM attack is not during the server -> client transport (i.e. the email to the user); it's when the client uses the link to go to your website (i.e. client -> server). Additionally, you can really only do GET requests from an email, which is even worse. I understand you don't want to add redundant data to your database, and so you've resorted to JWT.
I think whatever your use-case may be is ok for you, but it's too specific and, personally, slightly not safe since you're putting the token in the URL which means it can probably be re-used during the MITM attack. That means, if a snooper is fast and clever enough, the sign up could be given to the attacker and thus credit card info as well, even if it is from a different service since you're not exactly validating if the user is really the target person or not. What your intended PR implies is that the token is inside the URL, which can easily be picked up by anyone, not even in a MITM attack.
Again, I think you can innovate your methodology to be safe, but it shouldn't be in this package. I try to be inflexible with this package (counter-intuitive for a Python package, right?) because there can be so many security vulnerabilities that arise. I'm not the biggest security expert out here, but there are certain fallacies that can occur if we're not careful.

Sorry for the relatively late reply.
Not a problem at all!, I really appreciate your time and the effort you are taking into this, thank you so much 🙏🏻.

The MITM attack is not during the server -> client transport (i.e. the email to the user); it's when the client uses the link to go to your website (i.e. client -> server).

well, that's a problem 😅, especially with your later comment as it will be in the URL so using HTTPS has no powers here but it's only at URL so a frontend can pick it, verify with the server and use it in AUTHORIZATION header later fore upcoming requests, so at worst case, if someone stole a token from a header he will access the frontend only but won't go further than that unless he intercepts a full user's session, even an attacker completes the process through frontend he will only complete the signup process for the user he can't do anything with email access and if so then this user's life's is already stolen.
I agree with you that's it's not the safest method and your concerns are valid but I think it requires an exposed user at a higher level which you can't do anything about it, and for sensitive info like credit card data, thanks to payment gateway this info won't be exposed.
I agree with you again, this can be improved and later uses cases to come but I think this can be fine to start with and enhance it over time, there are many ideas, we can combining a token with an allauth link so on user verification we can share this token safely and avoid the mentioned concerns, but for now it still reliable.
Actually, I was going to comment with an update that I did the original goal of my question somehow, I created a token class with a new token_type value and a new authentication class to authenticate that new token_type specifically and I use that authentication class at my view, so with this token and authentication based on that type I did the original goal to specify a token for some parts only. it's turned out to be useful I think

If you'd like to open source some kind of methodology, I'd be happy to take a look at it. For now, I think this functionality has a bit of a security vulnerability and, in general, is too specific to be integrated in my opinion. I'll be closing this. Feel free to keep discussing as I check my GitHub notifications for any thread where there are comments!

Thanks, I really appreciate your help.
I'm willing to contribute with some points which I think will be useful for anyone to more extend.


[Feature] Support backing up/restoring multiple PV types per backup

This has been completed a/o the v0.10 beta.


issue/1273 Add runbook support under automation account

Issue-1273
PR Checklist

[x] I have added example(s) inside the [./examples/] folder
[ ] I have added the example(s) to the integration test list for normal (~30 minutes) or long runner >30 minutes
[x] I have checked the coding conventions as per the wiki
[x] I have checked to ensure there aren't other open Pull Requests for the same update/change?

Description
This PR will enhance the terraform-azurerm-caf to support runbook creation for an automation account
Does this introduce a breaking change

[ ] YES
[x] NO

Testing
Use exapmle (examples/automation/104-automation-account-runbook) for testing

Thanks @schoenr79 for the Issue and the PR, merged the other one as it was closer and picked some of yours too :)
thanks as usual :)


Fix ORJSON encode error

Proposal to fix this issue.

Python's JSON returns a str, which need to be encoded.
ORJSON returns a bytes, which produce an error when called with `encode('utf-8').

I split the blacksheep.server.responses.json() function into two function:

json(), which now only dumps → To use with ORJSON.
json_encode(), which dumps and then encode → Equivalent to the previous implementation of json()

@Joffreybvn Hi, and thank you for the PR, we appreciate it ❤
I'm not near my laptop, and can't take a look, but I think we also should add a test for new functionality.
@RobertoPrevato I'm not sure, but can we handle this case within json function instead of creating a new one like json_encode?

Hi @myusko,
I commented this PR under the related issue -> https://github.com/Neoteroi/BlackSheep/issues/192
I prefer to not add an extra method to handle JSON responses, so I generally agree that it would be best to do an "isinstance(json_dumps_result, str)" check inside the blacksheep.server.responses.json function, yes, or even the  JSONContent class. However, also this scenario doesn't look optimal to me because it would introduce a performance fee at each function call to serialize an object.
The fact that dumps returns bytes instead of str is something unique of orjson, all other libraries to handle serialization in Python (fastjson, ultrajson, the built-in json) return str and not bytes (dumps in fact stands for "dump-string" when "dump" in the built-in library handles a file-like object).
So I would rather correct the mistake in the documentation and require users of orjson to do the decode("utf-8") step, or refactor the code to handle bytes by default (so users using orjson are happy), all other libraries require anyway encoding the result. 😃

I close this PR now also because it wouldn't help handling the case for all other built-in responses functions that can optionally return JSON content, and the built-in JSONContent class.
After thinking about the case, I decided to not modify the current API to follow the design specific to orjson or introducing a performance fee at every dumps call making isinstance(return_value, bytes). I verified with benchmarks that this would reduce the performance of JSON serialization by 6%.
I'm updating the documentation to correct the orjson example and to provide instructions for orjson users who wish to avoid the decode -> encode performance fee.


Configure Windows Service via Registry keys

Currently the windows service has a hard-coded collection frequency and set of metrics which are reported. This allows both of those to be configured by Windows registry keys. In HKEY_LOCAL_MACHINE\SOFTWARE\pcm\service several options can be given. Registry was chosen as a good configuration method as it can be easily controlled via several administrative tools, does not require file location handling and can be easily shared/copied with .reg files.
SampleRate is a DWORD value which specifies the millisecond delay between data collections. This allows sampling to increase or decrease easily without recompilation.
Collect{Core,Socket,Qpi} are DWORD values which can be set to 0 to disable collection of a category of data, or 1 to enable. This allows different metrics to be collected without recompilation, as well as making it easier to add additional metrics to the service in the future without impacting on current collection behavior.
Both SampleRate and Collect* values are optional and default to their previous hard-coded value/behavior if not configured. Configuration changes are not picked up on the fly, but instead on startup only.

@ogbrugge please review

The latest commit should have fixed all indentation and casing issues. I upgraded my IDE just before working on this and only noticed after pushing that it hadn't kept my formatting settings.

I unfortunately do not see those changes, not here and not in 145 or 146. Am I missing something?

 On your comments above, GitHub is linking to the first commit only. A fresh, local checkout of the branch is showing everything correctly aligned for me.

Indeed, I missed this change, approved now.


r-modules: cran and bioc package bump

Motivation for this change
I was working on a fix for the BIOC package sets anyway (#127790), and wanted to test it.
Things done
Bumping the CRAN and BIOC package sets.
Remove the following packages from default.nix that were removed from CRAN or BIOC

BayesXsrc
`EasyqpcR
OligoSpecificitySystem
dpa

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 137751 at fc2701c1 run on aarch64-linux 1

2 packages marked as broken and skipped:

python38Packages.cnvkit
python39Packages.cnvkit

2 packages built successfully:

python38Packages.rpy2
python39Packages.rpy2

Squashed

So sorry, for some reason I missed the notification of your update. I've now got the broken marking working and a newer package set bump over here: #139609. Would you be ok with closing this in favour of that PR?
On a related note, do you have a particular update frequency you would find most useful?

Merged #139609 in favour of this bump.

Sorry, I've been out of internet range for a little while.
No problem about dropping this PR in favour of a newer package set.
Regarding package bump frequency, monthly is fine for my purposes. I use Nix to pin the package set for projects, rather than get bleeding edge CRAN packages. When the package set was getting to around 3 months old, I was starting to notice new features in packages that I couldn't get in Nix yet, so I made overlays. The overlays were useful for Github packages too, but that's another PR : )
Updates that are more frequent than monthly won't hurt if they are automated, but I wouldn't want to spend human time fetching them or fixing broken packages. I also don't know how Nixpkgs handles automatic PRs to bump package sets, I get the impression it would be discouraged.


RYA-154 Decoupled Updater Configuration from PCJ Configuration

Decoupled PrecomputedJoinIndexer configuration from PCJ configuration.  Users who want to use PCJs (which is a query based configuration) should not also be using the PrecomputedJoinIndexer (which affects ingest).

Please change this to point to the apache master branch.


Internet-pi not accessible after moving to new ISP

My Internet-pi was running well on my Pi4b running buster until I changed to a new ISP with router that assigns the Pi a different LAN address.  Now I get 'connection refused' on the browser when opening 'localhost:3030' from Chromium on the Pi's desktop.  I see nothing in the two configuration files which references any LAN or public IP address, so I don't know how to restore Internet-pi short of a complete re-install.
Is there a command which displays the status of Internet-pi?  Speedtest from Chromium on the Pi reports my 1000/500 fiber subscription delivers 300 Mbps down during the day and 12 Mbps in the evenings.  So getting a chart of the last week's speeds would help me when contacting the ISP's tech support.
Thanks

I have tried to uninstall the current SW and repeat the installation instructions, but I now get a "ModuleNotFoundError: No module named 'resolvelib': error.  This Pi is in active use by other applications, so it would be painful to install a new version of basher as an attempt to solve this problem.
=========
============
pi@Maui-Pi85:~/internet-pi $ ansible-galaxy collection install -r requirements.yml
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current version: 3.7.3 (default, Apr  3 2019,
05:39:12) [GCC 8.2.0]. This feature will be removed from ansible-core in version 2.12. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
ERROR! Unexpected Exception, this is probably a bug: No module named 'resolvelib'
the full traceback was:
Traceback (most recent call last):
File "/home/pi/.local/bin/ansible-galaxy", line 104, in
mycli = getattr(import("ansible.cli.%s" % sub, fromlist=[myclass]), myclass)
File "/home/pi/.local/lib/python3.7/site-packages/ansible/cli/galaxy.py", line 26, in
from ansible.galaxy.collection import (
File "/home/pi/.local/lib/python3.7/site-packages/ansible/galaxy/collection/init.py", line 100, in
from ansible.galaxy.collection.concrete_artifact_manager import (
File "/home/pi/.local/lib/python3.7/site-packages/ansible/galaxy/collection/concrete_artifact_manager.py", line 38, in
from ansible.galaxy.dependency_resolution.dataclasses import _GALAXY_YAML
File "/home/pi/.local/lib/python3.7/site-packages/ansible/galaxy/dependency_resolution/init.py", line 26, in
from ansible.galaxy.dependency_resolution.providers import CollectionDependencyProvider
File "/home/pi/.local/lib/python3.7/site-packages/ansible/galaxy/dependency_resolution/providers.py", line 33, in
from resolvelib import AbstractProvider
ModuleNotFoundError: No module named 'resolvelib'
pi@Maui-Pi85:~/internet-pi $

The installation of Internet-pi failed when run on a fresh download of the Sep6,2022 legacy 'buster' on a Pi4b.
So I'm still unable to find any successful installation method

@rrobinett: Do you still face this issue?

I was able to successfully install and run on all but one Pi, and I think it has the application 'cockpit' installed which uses port 9090 and thus conflicts with Internet Pi.
So I think it best for me to close this thread and open another if II have problems after I get physical  access to the Pi and can remove cockpit.
Thanks for checking


Cask request: Expressions

Cask details
Please fill out as much as possible. Before you do, note we cannot support Mac App Store-only apps.
Name: Expressions
Homepage: http://www.apptorium.com/products/expressions
Download URL: http://www.apptorium.com/products/expressions/trial
Description: An app to play with regular expressions. Easily and nicely.

https://github.com/caskroom/homebrew-cask/issues/33655#issuecomment-300018367

I had a request from app user (https://twitter.com/derimagia/status/859251648060952577), so I added it.
The ref was added to have information where the user downloads the app from. Is it bad, really?

The ref was added to have information where the user downloads the app from. Is it bad, really?

I never said it was bad. I asked you not to do it. We try to only use canonical download URLs that users can verify, for security reasons.

OK, so let it be without query. http://www.apptorium.com/products/expressions/trial is fine. Could you reopen this request?


fixes #18, lenkf.j now uses esma_mpirun

with this, lenkf,j now succesfully executes GEOSldas.x

@mathomp4

Well, I approve that it looks like what I'd write. I'm assuming you've tested.

It's tested to run. It's to be determined effects of updating g5_modules to GEOSgcm version.


Added ref to MathJax CDN

http://blog.mollywhite.net/how-to-display-mathematical-equations-in-ghost/

Hey @mswawola! Thanks for the PR, but we'd like to keep this repo in line with what Ghost is doing. You can totally add this to your blog, but we shouldn't add it for everyone.
Thanks though!


Swap : Order Cancelled

Amount to send:
0.9999 eth
Amount to receive:
0.085533  BTC
Payment Address:
17igyGiqNgY7HUk8CYhb75HmLZ89ee8mJb
txHash: https://etherscan.io/tx/0x00a5b6a59276f3756094e202e407d99c9a6207c0d665f5ae6f3151d9861f9d6f
REF ID#: Sorry , i forgot to screenshot :(
Why did my order cancel ?
And i did not receive any BTC , then i contacted support by mail.
And now , my problem's not solved. :-1:
May I receive refund in ETH ? It's so troubling . . .
Please answer to me soon :(

@tayvano Would you help me ? please :(

Each and every time I will say the exact same thing:
Please go back to https://www.myetherwallet.com/#swap and click the "Issue with your Swap? Contact support." link at the bottom of the page.
This will send an email to both us and<EMAIL_ADDRESS>with the information we need from you in order to look into the issue. Namely:
REF ID#:
Amount to send:
Amount to receive:
Payment Address:
Thanks.

I had sent what you say for more 2 days ?

If you sent then it will be resolved as soon as we can. We are doing the best we can. In the future I advise that you (1) make sure you send the exact amount and (2) make sure you send with high enough fees so that it gets on the blockchain quickly enough. Those are the top two reasons your order wasn't automatically processed.

Maybe i did not send enough fees , sorry :( it was my first time .
I hope my problem will be solved soon. I really need that ETH now. :(
Thank you , sorry for my impatience.


Update documentation URL

PR Details
I have updated the documentation URL to point to the new Sumo Logic page. This update is dependent on the SL docs PR that is currently in review. Once it is merged, this link will be live.
Description

Related Issue

Checklist

[ ] Changes follows the best practices from the given docs.
[ ] Updated CHANGELOG.md.
[ ] Ran unit tests locally.
[ ] Updated screenshots/sample logs if any.

@himanshu219 -- Here is the PR with the correct link to our documentation page, which is now live, so please feel free to review as your schedule allows. Thank you!

@sweeneyskirt-sl  we will also have to update the link in the manifest file https://github.com/SumoLogic/sumologic-public-partner-apps/blob/master/SauceLabs/SauceLabs.manifest.json#L9.


65108: Support JMeter variables in GraphQL HTTP Request

Description
In GraphQL query/mutation variables, JMeter variables can be referenced.
A JSON String value with a JMeter variable reference (e.g, { "var1":"${jm.var1}"}) has no problem as they are replaced at runtime without breaking any JSON spec, but a non-String value with JMeter variable reference (e.g, { "nvar1":${jm.nvar1}}) causes a problem because it's not a valid JSON. We need to somehow allow to use JMeter variable references in non-String JSON values.
This PR provides a conceptually simple solution:

If there's any JMeter variable references in a non-String JSON value, then replace the JMeter variable reference with a JSON Reference-like expression temporarily and parse it into a JSON Object.
Before serializing the GraphQL variables JSON object to a string, restore the temporary JSON Reference-like expression to the original JMeter variable references.

That is,

Step 1: User input for GraphQL variables:

{
  "var1": "hello",
  "var2": ${user.count}
}

Step 2: Temporarily replace it with JSON Reference-like expressions:

{
  "var1": "hello",
  "var2": {"$ref":"jmeter#/nonStringVariables/user.count"}
}

Step 3: Parse the string into JSONObject
Step 4: Before serializing the JSONObject or its parent JSONObject, restore the JMeter variable references from the serialized json string:

{
  "var1": "hello",
  "var2": ${user.count}
}

Motivation and Context
See Bugzilla Id: 65108 .
Also see PR #651 .
How Has This Been Tested?

Related unit test is updated.
You can also test the demo JMeter script at xdocs/demos/SimpleGraphQLTestPlan.jmx:

# Run ApolloGraphQL starwars-server, an open-source demo Apollo GraphQL server.
./gradlew runGui
# open the JMeter script
# Start the test and verify the test result.

Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist:

[x] My code follows the code style of this project.
[ ] I have updated the documentation accordingly.

Just wondering: does this escape-unescape buy us much?
Would it be better to keep the string as is, and perform parsing/conversion at the runtime only?

@vlsi , "performing parsing/conversion at runtime only" seems very difficult as @FSchumacher mentioned "they would have needed more changes, than I would liked to have" in PR #651 . So I ended up this "conceptually simple" solution instead.

I am not sure, that this is a better solution to not doing any JSON parsing at all.
The problems arise (in my eyes) from the fact, that we re-used a sampler, that wasn't thought for it and can't be used without change. One idea to modify it, was to add a listener/lifecycle-handler interface to that specific sampler, where we could register a callback, that would get called at the time the sampling gets done. There we could parse the JSON if needed. But that would mean adding new interfaces and changes to the public interface of the sampler.
The idea of checking the JSON at edit time, is probably doomed, as JMeter is too flexible. A user could try to use the following code in the JSON value field: ${json_prefix} "abc": 42 ${json_suffix} with json_prefix={ and json_suffix=}. That would not be handled with the proposed patch.

Thanks for the remarks! I fully understand the pointers and there must be better designs and solutions than this quick solution, indeed.
Just two things though:

The solution includes re-formatting to remove unnecessary white spaces like most GraphQL clients does, and this is done by parsing and formatting through the Jackson library. This "re-formatting" can possibly be done in a different way, but I assume the current solution is simpler.
The prefix and suffix in the solution are not just simple prefix/suffix, but instead it's following the JSON Reference syntax pattern, which means it is very unlikely that GraphQL variable JSON data could possibly allow any JSON Reference at all, or even allowed in the future, it is very unlikely for users to refer to the "JMeter-specific" json object reference in GraphQL variables in reality.
Of course, JMeter is too flexible, so for that reason, I actually narrow down the scope only to numeric variable value expression in this solution, which I believe should be good enough practically.

Regards,
Woonsan

So should we include this fix?

Hey JMeter team / @woonsan !
wondering if there is going to be any movement on this pr , or if you have any suggestions on temporary work arounds for this issue in the mean time?
Thank you and the team for all the work you put in.

Codecov Report

Merging #660 (6f87067) into master (3c23246) will increase coverage by 0.00%.
The diff coverage is 35.08%.

:exclamation: Current head 6f87067 differs from pull request most recent head 5703117. Consider uploading reports for the commit 5703117 to get more accurate results

@@            Coverage Diff            @@
##             master     #660   +/-   ##
=========================================
  Coverage     55.40%   55.41%
- Complexity    10230    10235    +5
=========================================
  Files          1047     1049    +2
  Lines         64518    64548   +30
  Branches       7317     7319    +2
=========================================
+ Hits          35749    35768   +19
- Misses        26271    26279    +8
- Partials       2498     2501    +3

Impacted Files
Coverage Δ

...AddUndoableEditListenerPropertyChangeListener.java
0.00% <0.00%> (ø)

.../org/apache/jorphan/gui/ui/DefaultUndoManager.java
0.00% <0.00%> (ø)

...ava/org/apache/jorphan/gui/ui/TextComponentUI.java
26.82% <18.75%> (+26.82%)
:arrow_up:

...va/org/apache/jmeter/gui/util/JSyntaxTextArea.java
61.53% <100.00%> (+0.42%)
:arrow_up:

...r/protocol/http/util/GraphQLRequestParamUtils.java
76.92% <100.00%> (+1.92%)
:arrow_up:

...n/java/org/apache/jmeter/reporters/Summariser.java
84.73% <0.00%> (-0.77%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 062190a...5703117. Read the comment docs.

@woonsan can you describe how can I use the jmete variable in graphQL variables?
like I have below variable how can I use groupId as integer, same like topic Id which is currently static
{
"group_ids": ["${groupId}"]
"topic_ids": [1560],
}

@haideralii , this was originally to fix the problem reported at https://bz.apache.org/bugzilla/show_bug.cgi?id=65108. Please see the example in the ticket. And if needed, please continue discussion in either JMeter user list or the bugzilla. Thanks.


Alembic Loader as Arnold Standin

Brief description
Adding Alembic Loader as Arnold Standin function as the Openpype Plugin
Description
The user can load the published workfileswith abc format as Arnold Standin if the pusblished workfiles belong to model and pointcache family.

Additional info
Doesn't have any for the current stage
Testing notes:

Start maya in Openpype
Click "Load" from Openpype in maya
Right-click the workfiles which belongs to either model or pointcache family
Click Import Alembic as Standin
It will load the abc file as an arnold standin

I ve tested it maya2023 with arnold <IP_ADDRESS>
Model imports as ASS file normally. Small note tho: use sequence is "ON" should be considered that model being static in general so should be "OFF" by nature
Speaking of Point cache at first everything seems fine and working however I ve got weird anim problems...double checked it and manually created ass.file from the same workfile and my version had no issues. Seems like some ASS creation params being wrong set. Need revision and testing. Same frame different animation:
One more thing to consider why not allow user to "Import Alembic as ASS file" even when speaking about Animation caches not just Pointcaches in general (pyblished animation alembics) could be pretty handy and also there shouldnt be any reason to not allow it too... so user could import these families as ASS:
model (already present) pointcache (already present) animation (not present now)

One of the possible reasons for the weird anim problem could be related to the fps of the animation. I find out the standin import the abc animation with 24 fps while the standard project setting is 25fps.  The abc fps is set up to 25 in the updated PR.

The updated PR also tells the systems not to load frame if the abc file doesn't contain any animation. And also I added the animation into families.

Now it works normally, did tests in maya2022 and maya2023.
I think the problem with animation mismatch was caused by difference in framerates (25fps vs 29.97) even present in pyblished abc using OP tools. So it was not on your side!
Question is if special care should be taken. I mean taking note of fps from origin of the abc file (source animation maya scene). As I understand you read this from database and set it according to it right?
Also static models still have is sequence turned on in ASS properties. Should be addressed already right?
Thanks!

Now it works normally, did tests in maya2022 and maya2023.
I think the problem with animation mismatch was caused by difference in framerates (25fps vs 29.97) even present in pyblished abc using OP tools. So it was not on your side!
Question is if special care should be taken. I mean taking note of fps from origin of the abc file (source animation maya scene). As I understand you read this from database and set it according to it right?
Also static models still have is sequence turned on in ASS properties. Should be addressed already right?
Thanks!

I think most of the issues I have discussed with you in discord. The abc file published from the model main wont have the sequence on as it doesn't contain any animation. So yes. the static models should be addressed already.

Made another go with testing...the static model like my table prop still gets the sequence option on. Also would be great if the loaded/imported ass files keep color coding in outliner (see img with outliner colors applied when assets loaded normally via asset load command using OP loader)
it means family animation being green colored in outliner and model dark orange color. Each family has its own.

Made another go with testing...the static model like my table prop still gets the sequence option on. Also would be great if the loaded/imported ass files keep color coding in outliner (see img with outliner colors applied when assets loaded normally via asset load command using OP loader) it means family animation being green colored in outliner and model dark orange color. Each family has its own.
Hi I have made the outlinecolor for the group_name and I have set up the condition for not loading the sequence in model main.

Have made another testing and all the features are already working fine!
There is one thing which prevents the approval...
I have tried to fiddle with switching versions of loaded asset...first I must say that its great that user have this option!!!
There is really minor issue that is Use File Sequence being unchecked afterwards even it was turned on before switching version of the asset.

Otherwise all works perfectly!

Have made another testing and all the features are already working fine!
There is one thing which prevents the approval... I have tried to fiddle with switching versions of loaded asset...first I must say that its great that user have this option!!!
There is really minor issue that is Use File Sequence being unchecked afterwards even it was turned on before switching version of the asset.

Otherwise all works perfectly!

Hi Libor. I think you can try with the latest code. it will update use file sequence according to which family subset it belongs to.


Support Storefront UI v0.7.9+

Current behavior

Currenty there are some breaking changes in Storefront UI in v0.7.9 which broke our header with mega menu and search panel.
Expected behavior

Header with mega menu and search panel works correctly.
Steps to reproduce the issue

Upgrade Storefront UI lib to v0.7.9
Check mega menu and search panel - it does not look nice.

Can you handle fixing this bug by yourself?

[x] YES
[ ] NO

Environment details

Browser: any
OS: any
Code Version: latest

Additional information

It turns out that Storefront UI v0.7.10 was automatically installed with yarn. I tried in src / themes / capybara / package.json in the line "@ storefront-ui / vue": "^ 0.7.8", replace it with "^ 0.7.9", Then yarn && yarn dev. But this did not fix the menu problem. What have I done wrong?

The problem with Storefront UI library that always installs latest "shared" package (regardless of fixed "vue" package version in package.json) will be tracked in https://github.com/DivanteLtd/storefront-ui/issues/1176.


Inference model on video

How to run and inference STrack on custom video file and visualize tracking results ?

Thank you for your feedback. We will soon add the script for video inference.

Thanks very much.
Your repository has been an invaluable resource for me, and I am sure that it has been a great help to many others in the community as well. Thank you for sharing your expertise and for making this project available for others to learn from and build upon.
@Xavier-Lin

Thanks very much.
Your repository has been an invaluable resource for me, and I am sure that it has been a great help to many others in the community as well. Thank you for sharing your expertise and for making this project available for others to learn from and build upon.
@Xavier-Lin

Hi
Any update on this demo script?

The code for inference has been updated

@Xavier-Lin Thank you so much

@Xavier-Lin Hi , How to run the demo. How to pass our own video for inference?


Include example for Azure Devops

Please include an Azure Devops example

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 6e0bddf9-8d20-459b-ed02-3a7520b9fa12
Version Independent ID: fee8f15e-7567-450a-dc89-7a1c35b37301
Content: Create an App Service app using a Terraform template - Azure App Service
Content Source: articles/app-service/provision-resource-terraform.md
Service: app-service
GitHub Login: @seligj95
Microsoft Alias: msangapu

Hi @fardarter - Thank you for the feedback. The intention of these docs is quickstarts to get users up and running. DevOps is a more advanced scenario that is well documented on it's own for users specifically looking to leverage that tool. Please see the DevOps docs if that has what you're looking for. https://learn.microsoft.com/en-us/azure/devops/?view=azure-devops

Hi @fardarter - Thank you for the feedback. The intention of these docs is quickstarts to get users up and running. DevOps is a more advanced scenario that is well documented on it's own for users specifically looking to leverage that tool. Please see the DevOps docs if that has what you're looking for. https://learn.microsoft.com/en-us/azure/devops/?view=azure-devops

Hi, there isn't a terraform example in the devops docs. If you know one, please direct me to it.
Otherwise, it's like to get up and running with Azure Devops and terraform and it seems like there is no guidance around this anywhere. I'd argue quite strongly that this is a failure to offer support on MS's own products.

@fardarter
Thanks for your feedback! We will investigate and update as appropriate.

@AjayBathini-MSFT - This request would be better suited for the terraform or devops teams. Can you please help route/loop in the appropriate contacts so they can have a look?

I have added Tom, who should be the point of contact for Terraform documentation.

Unfortunately, we have not been able to review this issue in a timely manner. We sincerely apologize for the delayed response. We are closing this issue. If you feel that the problem persists, please respond to this issue with any additional information - especially regarding what you would want covered in that article. I can then get the article on our backlog and work with the DevOps writer.
Please continue to provide feedback about the documentation. We appreciate your contributions to our community.
#please-close


Add formatting for large numbers

This adds a new FormatLargeNumber component that uses the current locale to format large numbers in the UI. Defaults to showing the full number when under 1000, and show a compact number with one decimal place (and a tooltip with the full value) otherwise.
Screenshots:

Closes: https://github.com/Chia-Network/chia-blockchain-gui/issues/351
I wasn't sure if when the translation files changed if they should also be PR-ed out with the UI changes. It looks like the comments indicating the lines in code changed, so figured they might need to come along.
to: @seeden
cc: @darkwebdev

@seeden would you be able to take a look at this? Or let me know who is better to tag on this PR? Thanks!

Hi, @etr2460. Sorry for the delay. I was working on the new pool functionality.
I need one confirmation and then I will merge it

Thanks @seeden, no worries, that work is certainly more important. Let me know if you want to file an issue with some of the work and I might take a look at it. I've been enjoying working on the project, so I could see myself dedicating some more time to it in the next few weeks


Update and refactor tests

Tests should be testing a single unit (function). Currently we don't test all functions directly since some of them are tested through other functions. This doesn't follow the best practices because it's misleading when tests start to fail and it leaves out edge cases. Ideally, tests should:

Every test should test a single unit.
Every test should set up the required contract state directly prior to testing.
Every unit should be fully covered.

Refactor variables from var to let/const.

Closing for now. Feel free to reopen it if needed.


<EMAIL_ADDRESS>installation fails because of a tidy_html5 missing link

Bonjour,
I need to install php-7.2 (not newer because of some dependencies) on a new Macmini under Macos bigsur 11.4.
But the following command failed:
brew install<EMAIL_ADDRESS>
=>
...
dyld: Library not loaded: /opt/homebrew/opt/tidy-html5/lib/libtidy.5.dylib
  Referenced from<EMAIL_ADDRESS>  Reason: image not found
Warning: The post-install step did not complete successfully
You can try again using:
  brew postinstall<EMAIL_ADDRESS>...

But I was able to solve my problem by adding a link for a tidy_html5 library.
cd /opt/homebrew/opt/tidy-html5/lib

ls -l
=>
-rw-r--r--  ...   889792 15 jul 17:08 libtidy.5.8.0.dylib
lrwxr-xr-x  ...       19 10 jul 22:53 libtidy.58.dylib -> libtidy.5.8.0.dylib
-r--r--r--  ...  1216816 10 jul 22:53 libtidy.a
lrwxr-xr-x  ...       16 10 jul 22:53 libtidy.dylib -> libtidy.58.dylib
drwxr-xr-x  ...       96 15 jul 17:08 pkgconfig

ln -s libtidy.58.dylib libtidy.5.dylib

And then it works.
Maybe there is an incomplete installation of tidi_html5?
Thank for maintaining all these php environments!
-- Maurice

Yes, reinstalling the latest build of PHP 7.2 and html-tidy5 should fix the issue
brew update
brew reinstall tidy-html5<EMAIL_ADDRESS>


Idea for overall building process of the programming language and name ideas...

Hi there @Kashina69! I was just checking out your repo here and really liked the idea you have, however I notice that the process in which you're using to build your programming language isn't the most suitable as I myself have tried it(it did not end well). Sense you're using Python I think you should use the PLY library as it isn't just built for building programming languages, but it also is quite easy to learn once you get the hang of it. If you have any questions just mention me in an issue like this one and I'll be glad to assist you in building this programming language.
PS:
Some name ideas for the programming language could be:
Enpy
Pytonic
In the end, the choice is yours, happy coding!

broo thank for commenting this you can't even immagine how much happy i am seeing your comment really really gladd thank for the advice thoo and i will work on that too hwo you found my repoo tell me and also i was like this will be some another side project of mine which will be just half done but your comment motivates me to complete it
actually i knew that python is not the best thing for this like slow lang making another slow lang but i wanted to make a mvp to get my foot and faster dev speed i am a js dev but js would not be the best for fast dev time my actually plan is to convert it after making mvp to rust or golang caus i want to learn those languages and they are fast tooo really like those two languages even i haven't tryed them and i was thinking of something like regex but it was too much for mee
and what was your programming language name btw
i am planning to give it the name of RawScript suits its scripting style and i am a js dev and also its Raw like english
i am done some changes in this code and pushed it check that out too
and thxx again

Yes, of course! I am really trying to become part of this community so I want to start bit by bit. I was just exploring for new programming languages being developed due to my fascination for them and so I searched up "custom programming language language:python" and then filtered it to view the most recently updated. Yours was one of the first ones there and so I checked it out. My bad, didn't mean to say it isn't good for building programming languages, I meant it is good. Sorry for the misunderstanding. Even though Python is slow, it can still be extremely useful and powerful. Rust and Go would be good choices, however they are much more complicated than Python, so do take that into consideration.
Oh, my programming language is named TEPL(short for textual educational programming language), you can check it out here: https://github.com/TENTHER101/TEPL. It's still in development and is not useful right now, however you can check out past versions by going through the history of commits I've made and find a working one to use.
RawScript sounds like a good name, just double check it doesn't already exist!
Ok, will do!
Happy coding!


Add utils function to run an api call for each endpoint

We should have functions that automate API calls to the 3 different endpoints to make it easy and fast to use the API.

[ ] YouTube video
[ ] Local audio file
[ ] Signed URL audio

@chainyo can you expand a bit on this

@chainyo can you expand a bit on this

@aleksandr-smechov simple functions that encapsulate in one line the API request process to avoid dealing with query/body params

Fixed by #94


WebAdministrationDsc.Common: Remove unused functions

Pull Request (PR) description

WebAdministrationDsc

Removed the common function Find-Certificate in favor of the command
present in the module DscResource.Common.
Removed the function Get-CurrentUser since no code were using it.

This Pull Request (PR) fixes the following issues
None.
Task list

[x] Added an entry to the change log under the Unreleased section of the file CHANGELOG.md.
Entry should say what was changed and how that affects users (if applicable), and
reference the issue being resolved (if applicable).
[ ] Resource documentation added/updated in README.md.
[ ] Resource parameter descriptions added/updated in README.md, schema.mof and comment-based
help.
[ ] Comment-based help added/updated.
[ ] Localization strings added/updated in all localization files as appropriate.
[ ] Examples appropriately added/updated.
[x] Unit tests added/updated. See DSC Community Testing Guidelines.
[ ] Integration tests added/updated (where possible). See DSC Community Testing Guidelines.
[x] New/changed code adheres to DSC Community Style Guidelines.

This change is

Merging this as it unblocks PR's were the pipeline fails with unrelated error to the PR's changes.


Support graph-DBs

Since more data in graph shape is being used, it would be handy to make this easily possible also with spin

similar to sqlite: an embedded property graph database e.g. kuzudb https://github.com/kuzudb/kuzu
BYO Graph DB e.g. falkorDB (the successor of redisgraph, same team...) https://github.com/falkordb/falkordb or good old neo4j

imho the most important thing for a broad adaption should be choosing the ones supporting opencypher / GQL (Graph Query Language) as query language. This has just (April 2024) become the new ISO standard for graph querying (graph peer to sql) https://jtc1info.org/slug/gql-database-language/

Thanks for sharing this idea - very intriguing!
I can definitely see the attraction to having an embedded graph database in the same way as we have SQLite.  The challenge would be portability to the cloud: either we would need to partner with a cloud provider running a hosted form of the same database (perhaps a Redis host with Falkor support?), or we'd have to allow backing it onto a BYO database (which could have compatibility implications).
BYO databases avoid this problem, but my understanding is that - as with SQL - standardisation applies only to the query language, not the protocol.  (Although perhaps I am wrong.  I see Falkor works with the Redis client: is that usual?)  That would mean we would need to pick one or two, a la Postgres and MySQL, and provide a host service interface for those, until Wasm network suppoirt is pervasive enough for people to use the standard clients.
One thing I noticed: since Falkor works with Redis clients, it seems like it should already be usable via Spin's Redis API, using the execute API (e.g. https://docs.rs/spin-sdk/latest/spin_sdk/redis/struct.Connection.html#method.execute, https://fermyon.github.io/spin-python-sdk/v3/wit/imports/redis.html#spin_sdk.wit.imports.redis.Connection.execute).  Not friendly, for sure, as you'd need to parse the results of execute yourself, but perhaps a starting point?

Thanks for sharing this idea - very intriguing!

thanks for getting back on this!

I can definitely see the attraction to having an embedded graph database in the same way as we have SQLite. The challenge would be portability to the cloud: either we would need to partner with a cloud provider running a hosted form of the same database (perhaps a Redis host with Falkor support?), or we'd have to allow backing it onto a BYO database (which could have compatibility implications).

falkordb and kuzudb are fresh approaches (teams) and could be a good candidates to show interest in being part of this.
kuzu seems to already having first contacts to the wasm world

https://github.com/kuzudb/kuzu/issues/3326#issuecomment-2066888881
In the longer term, we should revisit this and come up with a more flexible solution, as mmap is also not well supported in WASM. https://github.com/kuzudb/kuzu/issues/2959

BYO databases avoid this problem, but my understanding is that - as with SQL - standardisation applies only to the query language, not the protocol. (Although perhaps I am wrong. I see Falkor works with the Redis client: is that usual?) That would mean we would need to pick one or two, a la Postgres and MySQL, and provide a host service interface for those, until Wasm network suppoirt is pervasive enough for people to use the standard clients.

yep you are absolutely right. Opencypher / GQL (Graph Query Language) is the query language. I was referring to this because one may choose DBs for first implementations/Demo (embedded and BYO) supporting this since it has the widest adaption in graph world.
For the connection there is the bolt standard which is used by many graphdbs:

neo4j, specs: https://neo4j.com/docs/bolt/current/bolt/
falkordb https://www.falkordb.com/blog/released-falkordb-4-0-a1-vector-search-index-bolt-protocol/
memgraph, see below the client box: https://memgraph.com/docs/client-libraries
neptune https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-opencypher-bolt.html
...

One thing I noticed: since Falkor works with Redis clients, it seems like it should already be usable via Spin's Redis API, using the execute API (e.g. https://docs.rs/spin-sdk/latest/spin_sdk/redis/struct.Connection.html#method.execute, https://fermyon.github.io/spin-python-sdk/v3/wit/imports/redis.html#spin_sdk.wit.imports.redis.Connection.execute). Not friendly, for sure, as you'd need to parse the results of execute yourself, but perhaps a starting point?

interesting point!

maybe also hepful as staring point:
there is a rust bolt client
https://docs.rs/bolt-client/latest/bolt_client/

better:

This project aims to provide a comprehensive set of libraries that allow for interaction with graph database servers that support the Bolt protocol... This set of libraries allows interacting with servers supporting versions 1 through 4.4 of the protocol

https://github.com/0xSiO/bolt-rs


nixos/dbus: support dbus-broker

Motivation for this change
The dbus-broker project tries to improve on what was dbus-daemon, and provide a better alternative.
This PR adds support for dbus-broker, which I hope could be the default dbus daemon for NixOS.
For more information about dbus-broker see https://dvdhrm.github.io/rethinking-the-dbus-message-bus/ or this short description from Fedora. They've used dbus-broker since F30 (was released on April 30, 2019,) which should help to make this an easy switch for us.
Things done

I've logged into a root console and checked if dbus-broker was running
Inspected /etc/systemd for the units being setup in the way I expected
Ran a Pantheon VM , launched a terminal, and checked the status of dbus-broker

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

My position for this PR is to try to merge this without changing defaults. And in another PR change this and maybe we can get a jobset to see that none of our tests are acting up, make release notes, and just general more scrutiny. LMK if that sounds like the way to go.

Result of nixpkgs-review pr 112879 at 7d69145a run on x86_64-linux 1

2 packages blacklisted:

tests.nixos-functions.nixos-test
tests.nixos-functions.nixosTest-test

1 package built:

dbus-broker

A concern with I'd have to be concerned with is that dbus-broker doesn't have apparmor support https://github.com/bus1/dbus-broker/issues/169.
Also, gdm has secret dependencies on dbus-run-session https://gitlab.gnome.org/GNOME/gdm/-/merge_requests/90#note_776136 as there is no dbus-run-session equivalent https://github.com/bus1/dbus-broker/issues/145.
So certain dbus utilities might need to be split from dbus and installed, as this change would make everything from dbus not it environment.systemPackages. It would be a good idea to maybe also hardcode those.

Also missing CAP_AUDIT_WRITE https://github.com/bus1/dbus-broker/blob/f596efcaa294f4d9b656385379373b1f60eef718/src/util/audit.c#L39 https://github.com/bus1/dbus-broker/issues/239

When testing this I see a lot of Ignoring duplicate name messages in the system logs. I assume these were silently ignored by dbus and dbus-broker is just more verbose about them?

When testing this I see a lot of Ignoring duplicate name messages in the system logs. I assume these were silently ignored by dbus and dbus-broker is just more verbose about them?

Yep, I believe this is true also.

So we already hard code dbus-run-session path in gdm since https://github.com/NixOS/nixpkgs/commit/bcf3872cba1a83235a4785bd512676d65c54e9a8 right?
From a quick grep through nixpkgs, there are quite a few other occurrences of dbus-run-session though.. is the plan to hard code all of them?

I don't use GNOME, but it should no longer launch dbus manually - instead it expects systemd to have set up the socket.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/what-are-your-goals-for-21-05/11559/7

I don't use GNOME, but it should no longer launch dbus manually - instead it expects systemd to have set up the socket.

On Fedora, GNOME will rely on systemd --user to provide a D-Bus user bus. However, this means there can only be a single user-bus per user (per definition), unlike the old session buses where a single user could have multiple of those. This is usually not an issue. But GDM might run multiple times on a single machine (due to the way that GDM tracks sessions and allows multiple parallel logins). Therefore, the gdm developers use dbus-run-session to spawn their own session bus for each instance of GDM and work around this limitation. This is a hacky workaround and the intention is to switch to a dynamically allocated user instead. But nobody is working on this, so it might stay that way for some time.
Long story short: GDM completely ignores the user bus and spawns its own session bus via dbus-run-session.

So I heard that sadly worldofpeace will no longer be actively working on nixpkgs. Could we merge this as is (maybe with a warning that its experimental and some things might not work yet)? I would very much like to use it even in the current state and could also take over the pr if this is desired.

So I heard that sadly worldofpeace will no longer be actively working on nixpkgs. Could we merge this as is (maybe with a warning that its experimental and some things might not work yet)? I would very much like to use it even in the current state and could also take over the pr if this is desired.

If this needs more work just create a new PR based on this one.

Can someone just update dbus-broker package? :thinking:
Personally I'm using dbus-broker for a long time on NixOS with a home-made module, but that old version has bugs with service activation and that's really annoying :(

@ilya-fedin - maybe just pull out the commit with the update and stick it into it's own PR? Considering there are no dependencies on it right now, that should get merged quickly.

#121664
I picked only the system-console-users from here


Loosen version constraints on dependent crates

The num crates seem to change pretty frequently, causing them to fall out of sync with other crates, and byteorder is 1.0 so pegging to a specific patch version doesn't seem necessary.

x.y.z is equal to ^x.y.z, see https://github.com/steveklabnik/semver#requirements
I believe you're thinking of =x.y.z.

Oh you're right. I suppose the only exception is for 0.0.x version crates.


[Feature] Redirect to Home Page on clicking the footer logo

Description
The web page should be redirected to home landing page when the logo/header on the Footer section is clicked.
It will provide easy navigation from different pages to the home screen and maintain consistency of the website.
Screenshots

Checklist

[X] I have checked the existing issues
[X] I have read the [Contributing Guidelines]
[X] I am willing to work on this issue (optional)
[X] I am a GSSoC'23 contributor

@abhinav-m22 ok make sure to keep "no reloading" feature


Admin API Personal Tokens

Goal
Implement personal API tokens for staff users to allow third-party clients to authenticate to the admin API.

PR'd directly on https://github.com/TryGhost/Ghost-Admin


getVersion()  method can't use

GitLabApi gitLabApi = new GitLabApi(ApiVersion.V3,"http://your.gitlab.server.com","YOUR_PRIVATE_TOKEN");
When I use gitLabApi.getVersion() ,it cuase

@LeafLuo
A 401 indicates that your token is invalid or no longer authorized on the GitLab server you specified.
Also, why are you using the V3 API, are you going at an old GitLab server?

@LeafLuo
Are you still having an issue?  If not, I'd like to close this issue.

@LeafLuo
I'm going to close this issue.  If you are still having issues please re-open and provide additional info.


linux video not playing

I'm submitting a ...

[*] bug report
[ ] feature request
[ ] support request

General infomration
Prerequisites

[*] Are you running the latest version?

Description
[Description of the bug or feature]
Steps to Reproduce

[try to play a video]
[Second Step]
[and so on...]

Expected behavior: [video to play]
Actual behavior: [video not playing]
Context (Environment)
Installation

[ ] I installed the plugin via zip from the Releases page
[*] I´m using the Netflix Repo
[ ] I´m using a different source (Please tell which)

Operating System

[*] Linux (x86/x64)
[ ] OSX (x86/x64)
[ ] Windows (x86/x64)
[ ] Linux (ARM)
[ ] Android

Additional informatin on the environment
[Ubuntu 18.04 + kodi 18.0-rc2]
Debug log
[https://pastebin.com/Ec78qVwf]
Other information
[e.g. detailed explanation, related issues, suggestions how to fix, links for us to have context, etc.]
Screenshots
[Please add a screenshot if that helps understanding your problem]
[You can erase any parts of this template not applicable to your Issue.]

Please share complete log file

Hi, pasting here since I have more or less the same setup.
Here is the pastebin link with debug mode enabled
http://paste.ubuntu.com/p/SBMFRwC7pr/

I just tried the beta 0.14.0 and it doesn't work neither
Here is the pastebin
http://paste.ubuntu.com/p/Y5ZdBGRHQw/

Hi, I ran into the same issue. Turns out, Netflix blocked was IP blocking me as I am using an IPv6 tunnel.
Defaulting to IPv4 instead of v6 fixed that issue.
I suggest displaying an appropriate error message in that case.
If it helps, decrypted_payload looks like this prior to JSON parsing:
{
  "version": 2,
  "id": xxxxxx,
  "serverTime":<PHONE_NUMBER>108,
  "from": "nq_cadmium_pbo_manifests<EMAIL_ADDRESS>  "error": {
    "pboc": true,
    "code": "BLACKLISTED_IP",
    "detail": "manifest/pbcs-434: Blacklisted IP Address [viewableId=xxxx] (nq_cadmium_pbo_manifests@1.16.1/pbo-client@1.16.1)",
    "display": "You seem to be using an unblocker or proxy. Please turn off any of these services and try again. For more help, visit netflix.com/proxy.",
    "errorDisplayMessage": "You seem to be using an unblocker or proxy. Please turn off any of these services and try again. For more help, visit netflix.com/proxy.",
    "errorActionId": 3,
    "nccpError": {},
    "errorSubCode": 0,
    "errorReasonCode": 0,
    "errorNccpCode": 0,
    "errorExternalCode": 0,
    "bladeRunnerMessage": "manifest/pbcs-434: Blacklisted IP Address [viewableId=xxxx] (nq_cadmium_pbo_manifests@1.16.1/pbo-client@1.16.1)",
    "bladeRunnerCode": "434"
  }
}


Why this library has a vc runtime

i'm using -mt option compiled.

Can you please clarify what is exactly your question?

I believe the OP is asking why the Windows port has a dependency on MSVCRT.dll
This is indeed problematic because that DLL is distributed with the C++ Runtime package which is quite large when the tcmalloc dll itself is less than half a megabyte.
For my own usage, I have changed the project so that it statically links the Microsoft CRT which means that my version of tcmalloc.dll can be deployed pretty much everywhere without any external dependencies.