Defining the watching and triggering semantics

Param has recently been given the ability to 'watch' parameters, triggering callbacks when parameters are set. There is an important subtlety to think about as currently triggering occurs when parameters are set regardless of their value.
This is rather surprising: your watching callback will be triggered even if you keep setting the parameters to the same values (either with attribute setting or set_param). In my opinion, what you really want to do is watch parameters for changes.
Given these semantics, it would be nice to let users trigger watch callbacks without value changes should they decide to. Discussing this with @jbednar, we propose the following that avoids adding a boolean flag to the watch API:

Update the semantics of the watch mechanism to be specific to parameter changes.
Add a trigger method that takes a list of parameter names to trigger the (applicable) watch callbacks using the current set of parameter values. E.g foo.trigger('a','b') would pass value Change objects to the applicable watch callbacks for the current values of parameters a and b.

Unfortunately, updating the watch mechanism is itself a non trivial task as it is hard to know when something (mutable) has changed for anything other than simple types. This is the same problem we face if we want to generalize the memoization machinery in HoloViews so it can be used in param.
Ideally, I think we would tackle the following issues in param in this order:

Move the memoization/equality comparison machinery out of holoviews and expose it in a useful way from param.
Use this machinery to implement trigger and update watch with the new semantics.
Continue work on batching parameter changes on top of the updated semantics.

In practice, I think we can skip item 1 for now and partially implement item 2, with an explicit note that the equality checking is not sophisticated for complex mutable types. Then I can continue working on item 3 which is the next priority. When we have a chance, we can then implement item 1 and finish off item 2 to handle more complex types.
Does this seem reasonable?

That all sounds good to me.

I'm not at all fond of the hashing code in holoviews and would be very hesitant porting that code to param. For pretty much any array or dataframe data that code is extremely slow, so much so that I've basically had to bypass it entirely to get usable performance out of any data based Stream classes. In fact, in practice I've found that in a vast majority of cases the hashing is so slow that it actually provides no benefit over just disabling the memoization.
So I agree with the order you lay out at the end, starting by implementing this for cases where simple equality works and can decide to expand that to more complex types at a later point, hopefully with a cleaner/better approach than currently implemented in holoviews.

Just to get my thoughts recorded somewhere, basically I believe any hashing based approach is infeasible, because it requires the whole data to be hashed before any comparison can happen. The only thing I can see being feasible is a hybrid approach that performs some simple checks before a full hash is generated, e.g. things like:

Are the objects the same type?
Does the first value of the array match?
Do the dataframe column names match?
Does the array shape match?

These kinds of checks would bypass a full hash in the vast majority of cases, and only in the worst case would it fall back.

These kinds of checks would bypass a full hash in the vast majority of cases

I agree that simple checks like that should be done where feasible.  However, isn't it necessary to do a full, deep comparison before one can make use of anything that's been memoized?  So it seems like at best such checks can help make memoization less costly when it's not useful, and they won't help any for then it is useful.

That's a good point, doing this is always going to be costly and the cost of doing it can often outweigh any benefit gained by not executing the function. Ideally you would only activate it in cases where it's actually preventing large computations from being rerun.

Ideally it would just be a hash on the underlying contents of memory -- simple to calculate, and may falsely require recomputation but would always be truly accurate when it says they match.  But that's out of scope for anything we implement ourselves, so hopefully that's what joblib is doing.

Point 1 where the semantics of watch has now been addressed in #271.
As for fast equality checking, I don't think there are any easy answers. I certainly don't disagree with anything said here and I remember not being particularly happy with any approach when I implemented it. The options are:

Don't support memoization for non-trivial types.
Accept that equality checks can be expensive (normally due to serialization) and hope that it is generally faster than the recomputation you are avoiding by memoizing. This is what we opted for in holoviews.
Somehow hash the memory as Jim just mentioned. I don't know any easy way to do this is Python but maybe some library out there can handle fast, deep equality tests for arbitrary Python objects. Even if such a library exists, it would have to become a core dependency for either holoviews/param which means even this wouldn't be an ideal solution.

One approach (which I am not necessarily recommending!) would be to say parameters shouldn't be large, complex chunks of data and should only be relatively simple literals. This is normally true but we do know that there are cases where we do want parameters that hold a fair bit of state.

Currently my thinking is that we should probably just provide an extensible way to define equality checks for arbitrary types. If a user wants their custom object to work with it they can register an equality function. Testing for equality is always going to be faster than hashing and then comparing hashes since there is large (often huge) overhead in serialization, which is, I think, entirely unnecessary here because we have both the old and new value available for comparison. Just as a simple example here's the difference between HoloViews hashing comparisons and a simple numpy.array_equal:

I think we can all agree that 1.8 seconds for a fairly small array is pretty much unusable.
Defining equality function to cover the common cases such as literals, functions, ndarrays and dataframes etc. (basically all the types that param defines explicit Parameters for) won't be difficult and anyone else can define their own equality checks for custom objects they use.

That sounds good...

Just to say that a first cut at triggering was introduced in #283 and batched watching has also been merged (though that also needs to support what properly).

@philippjfr @jlstevens has this all been dealt with? could y'all open a new issue if there are any small changes to address.


 Overhead lights should be turned off first.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
Overhead lights should be turned off first. Then sleep lights turned on.

Hi, turning OFF sleep states turns sleep lights ON? Should be turning it OFF.
If the sleep lights are turning on before the others turn off, then that's expected. The whole process is asynchronous so there is no way to guarantee order of execution on these.


QR Decomposition: Short and Fat?

Hi, it seems that Western beauty standards have crept into dask.array.linalg.qr. Only "tall and skinny" (single column of blocks).
There doesn't seem to be room for the "short and fat". It seems that a quick fix.
QR = A = [ A_1 | A_2 | ... | A_n ]

Doing a decomposition for the first square matrix...
Q R_1 = A_1

... and then...
R_i = Q'A_i

which should be a quick supplementation to the existing QR method.

Quickly...
A = da.random.random((4, 10), chunks=4)

# short and fat QR
A_sq = A[:, :A.shape[0]]
A_rest = A[:, A.shape[0]:]
q, r_sq = da.linalg.qr(A_sq)
r_rest = q.T.dot(A_rest)
r = da.concatenate([r_sq, r_rest], axis=1)

((q.dot(r) - A < 1e-14).all()).compute()  # they match up

Hi, it seems that Western beauty standards have crept into dask.array.linalg.qr

Heh, guilty as charged.  :)
Any interest in submitting a PR?

I don't mind... but any pointers to how to get started contributing and guidelines?

The quick and dirty version is something that might be done fast, but I'd like to learn the internals a little more so as to handle arrays with multiple blocks on the row dimension, that would probably entail some "joins" where there is waiting for other operations to complete. Any pointers to well written code that fits that bill for arrays?

The linear algebra code is likely to be a somewhat complex introduction to
dask.array.  I recommend starting with a pull request that implements the
simple solution, and then following up later with a more complex solution.
In practice I expect that tall-and-skinny or short-and-fat arrays are
likely to be the common case, and that rechunking the array to have a
single chunk in one dimension is likely to be the most efficient solution
in these cases.
On Thu, May 31, 2018 at 10:43 PM, Jeremy Chen<EMAIL_ADDRESS>wrote:

The quick and dirty version is something that might be done fast, but I'd
like to learn the internals a little more so as to handle arrays with
multiple blocks on the row dimension, that would probably entail some
"joins" where there is waiting for other operations to complete. Any
pointers to well written code that fits that bill for arrays?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3548#issuecomment-393741814, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszHPYwWOQAuhxeNnN127C2SPZfEcLks5t4KpXgaJpZM4UUmTN
.

I have written something (with an accompanying modification to qr which does the normal thing when the number of chunks width-wise is 1. That is at the bottom.
But I'm not too satisfied because I find myself not exactly sure how to construct an Array from a delayed(np.linalg.qr)(A) after doing the required delayed(operator.getitem) without using a ShareDict. This doesn't work...
M = np.array([[3,4,4,3],
              [4,3,4,4]])
A = da.from_array(M, chunks=2).persist()

Q_R = dask.delayed(np.linalg.qr)(A)
Q = dask.delayed(operator.getitem)(qr, 0)
R = dask.delayed(operator.getitem)(qr, 1)
print (Q.compute().dot( R.compute() ))  # this is fine
RR = Array(r.dask, r.name,
           shape=data.shape,
           chunks=(len(data.chunks[0]), len(data.chunks[1])),
           dtype=A.dtype)
print(RR.compute())
# rr.visualize()

I'm alright submitting the below where (a ShareDict is used so) output has proper dimensions.
def sfqr(data, name=None):
    """ Direct Short-and-Fat QR

    Currently, this is a quick hack for non-tall-and-skinny matrices which
    are one chunk tall and (unless they are one chunk wide) have chunks
    that are wider than they are tall

    Q [R_1 R_2 ...] = [A_1 A_2 ...]

    it computes the factorization Q R_1 = A_1, then computes the other
    R_k's in parallel.

    Parameters
    ----------

    data: Array

    See Also
    --------

    dask.array.linalg.qr - Supported by this
    """

    nr, nc = len(data.chunks[0]), len(data.chunks[1])
    cr, cc = data.chunks[0][0], data.chunks[1][0]

    if not ((data.ndim == 2) and  # Is a matrix
            (nr == 1) and         # Has exactly one block row
            ((cr <= cc) or        # Chunking dimension on rows is at least that on cols or...
             (nc == 1))           # ... only one block col
           ):
        raise ValueError(
            "Input must have the following properties:\n"
            "  1. Have two dimensions\n"
            "  2. Have only one row of blocks\n"
            "  3. Either one column of blocks or chunking dimension on rows is at most that on cols"
        )

    prefix = name or 'sfqr-' + tokenize(data)
    prefix += '_'

    m, n = data.shape
    numblocks = (1, nc)

    qq, rr = np.linalg.qr(np.ones(shape=(1, 1), dtype=data.dtype))

    dsk = sharedict.ShareDict()
    dsk.update(data.dask)

    # data = A = [A_1 A_rest]
    name_A_1 = prefix + 'A_1'
    name_A_rest = prefix + 'A_rest'
    dsk.update_with_key({
        (name_A_1, 0, 0): (data.name, 0, 0)
    }, key=name_A_1)
    dsk.update_with_key({
        (name_A_rest, 0, idx): (data.name, 0, 1 + idx)
        for idx in range(nc - 1)
    }, key=name_A_rest)

    A_1 = Array(dsk, name_A_1,
                shape=(cr, cc), chunks=(cr, cc), dtype=qq.dtype)

    # Q R_1 = A_1
    name_Q_R1 = prefix + 'Q_R_1'
    name_Q = prefix + 'Q'
    name_R_1 = prefix + 'R_1'
    dsk.update_with_key({
        (name_Q_R1, 0, 0): (np.linalg.qr, (name_A_1, 0, 0))
    }, key=name_Q_R1)
    dsk.update_with_key({
        (name_Q, 0, 0): (operator.getitem, (name_Q_R1, 0, 0), 0)
    }, key=name_Q)
    dsk.update_with_key({
        (name_R_1, 0, 0): (operator.getitem, (name_Q_R1, 0, 0), 1),
    }, key=name_R_1)

    Q = Array(dsk, name_Q,
              shape=(m, min(m, n)), chunks=(m, min(m, n)), dtype=qq.dtype)
    R_1 = Array(dsk, name_R_1,
                shape=(min(m, n), min(m, n)), chunks=(cr, cc), dtype=rr.dtype)

    # R = [R_1 Q'A_rest]
    Rs = [R_1]

    if nc > 1:
        A_rest = Array(dsk, name_A_rest,
                       shape=(min(m, n), n - min(m, n)), chunks=(cr, cc), dtype=rr.dtype)
        Rs.append(Q.T.dot(A_rest))

    R = da.concatenate(Rs, axis=1)

    return Q, R

Stuff I'd like to learn to be able to better contribute would be:

how to convert o.dask items that are arrays of known dimensions and chunking into arrays via Array or something
how to pass keyword arguments into methods (in a ShareDict)
what the proper structure for items are in the ShareDict

I had to figure out that arrays would take the form:
{
  # ...
  "arr-hashhash-etc": {
    ("arr-hashhash-etc", 0, 0): (op, (src, 0, 0), arg1, arg2, ...),
    ("arr-hashhash-etc", 0, 1): (op, (src, 0, 1), arg1, arg2, ...),
    ("arr-hashhash-etc", 1, 0): (op, (src, 1, 0), arg1, arg2, ...),
    ("arr-hashhash-etc", 1, 1): (op, (src, 1, 1), arg1, arg2, ...),
  },
  # ...
}

So a good reference would be nice.
I guess structure of the core distributed objects are defined by:

bags with npartitions
dataframes with npartitions, columns and dtypes
arrays have shape and chunks

So it would be good to know how to represent these in the "list of computations" which is the ShareDict, and how to extract items (so those 3 plus generic objects).

Thanks for the notes @convexset .  Dask array is probably the most advanced of the Dask collections (it tends to have the most advanced algorithms, and so has the most performance pressure on it) and so I can appreciate that it's a bit of a challenge to dive in.
If you haven't already you should read http://dask.pydata.org/en/latest/array-design.html and the docstring of ShareDict (apparently we don't have any online docs for this?).
Regarding your questions about delayed and Dask arrays I suggest that you don't go down this path.  Calling delayed functions on dask array objects doesn't parallelize those functions, instead it converts the dask arrays to numpy arrays when the function is called.  So in the following case:

But I'm not too satisfied because I find myself not exactly sure how to construct an Array from a delayed(np.linalg.qr)(A)

You're not constructing a Dask array, you're lazily converting A to a numpy array, then calling the np.linalg.qr call on it, and returning it as a lazy numpy arrays.
Instead, for short-and-fat QR decomposition, I wonder if we can reuse the existing solution to this problem by transposing the array, calling the tall-and-skinny QR decomposition, and then transposing the results.

Ok, so I guess my code snippet works fine because the approach using delayed would just not distribute. I'll send a PR.
Also... we can't just transpose the solution... (QR)' = R' Q'.

... I'll add tests first.

See: PR https://github.com/dask/dask/pull/3575

Also... we can't just transpose the solution... (QR)' = R' Q'.

When reading this, this was my initial thought. Should be possible. We are able to transpose arrays.

Matrix multiplication is mostly not commutative. So what we get is the orthogonal matrix on the right. It's a common idea people have when going through numerical linear algebra though.


🐛 [device/flash-not-available: The Camera Device does not have a flash unit! Make sure you select a device where device.hasFlash/device.hasTorch is true.]

What's happening?
The back camera is working for me, but I am experiencing this issue when using the front camera to take selfies. I am using getCameraDevice(devices, 'front') to obtain the front camera.  I would appreciate your guidance in fixing this or letting me know if anything is incorrect.
Reproduceable Code
const frontDevice = getCameraDevice(devices, 'front');
 <Camera
              ref={ref => {
                this.camera = ref;
              }}
              style={cameraStyle}
              device={frontDevice}
              isActive={isActive}
              photo={true}
              torch={"off"}
              enableZoomGesture={true}
            />

Relevant log output
error [device/flash-not-available: The Camera Device does not have a flash unit! Make sure you select a device where `device.hasFlash`/`device.hasTorch` is true.]

Camera Device
{
  "formats": [],
  "sensorOrientation": "landscape-right",
  "hardwareLevel": "full",
  "maxZoom": 4,
  "minZoom": 1,
  "maxExposure": 12,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 0,
  "minExposure": -24,
  "name": "1 (FRONT) androidx.camera.camera2",
  "hasFlash": false,
  "hasTorch": false,
  "position": "front",
  "id": "1"
}

Device
Xiaomi Redmi Note 8 Pro - M1906G71
VisionCamera Version
4.5.2
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information

[ ] I am using Expo
[ ] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

@mrousavy Thank you for your prompt response. As I do not intend to use a flash or torch while using the front camera, why is it still necessary that device.hasFlash and device.hasTorch be true? Also, I have used the torch={"off"} property, so why is this issue still occurring?

If you don't have a device you shouldn't render the Camera. This is literally what the error message says. Use Typescript, you cannot pass null to device

@mrousavy i am not passing null to device prop. I am passing the which is return by getCameraDevice(devices, 'front') and camera view is opened in my app but takePhoto function is not working for me. So i have changed in the options flash: 'off' like below and now its working for me.
const options = { base64: false, flash: 'off', skipMetadata: true };
let response = await this.camera.takePhoto(options);


"ethereumjs-abi" not found

When installing<EMAIL_ADDRESS>which depends on ethereumjs-abi, I get this error:
➤ YN0000: ┌ Link step
➤ YN0013: │ ethereumjs-abi@https://github.com/ethereumjs/ethereumjs-abi.git#commit=1cfbb13862f90f0b391d8a699544d5fe4dfb8c7b can't be found in the cache and will be fetched from GitHub
j [HTTPError]: Response code 404 (Not Found)
    at i.<anonymous> (/Users/paulrberg/Projects/Sablier/Magellan/Protocol/.yarn/releases/yarn-berry.js:10:403654)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  code: undefined,
  timings: undefined
}

I'm using yarn v2, so I'm not sure whether this is an issue with this new experimental version of yarn or ethereumjs-abi.

Thanks for the update on this! 👍


Write script to run both frontend and backend at same time

Presently we have to first enter backend and then do npm start and then open frontend in another terminal and do npm run dev.
It would be nice to have script in the package.json in the app level which runs both,
ex: npm start should run both backend and frontend

Hello @krishnaacharyaa ,

Should I use any third party package (ex. https://www.npmjs.com/package/concurrently) ?
If so then, please assign it me.
If not then, let me any other approach.

Sure @hemant14050
Yes you need to use concurrently package
Go through https://blog.logrocket.com/running-react-express-concurrently/ and kindly make sure if we need any more changes in our project w.r.t scripts which enhances the maintainability of the project and adds value.


Install fails

Trying to install but it does not.
Did the following steps:
1-added https://github.com/ghiglie/hassio-addons to addons store
2-Clicked ghiglie Hass.io Add-ons MySensors Gateway
3-push install
4-The install button start spinning, after a while it stops
And nothing shows up at the Addons Dashboard

19-05-26 21:52:31 INFO (SyncWorker_15) [hassio.docker.addon] Start build b64b7dc4/armv7-addon-mysgw:0.11
19-05-26 21:52:50 ERROR (SyncWorker_15) [hassio.docker.addon] Can't build b64b7dc4/armv7-addon-mysgw:0.11: The command '/bin/ash -o pipefail -c chmod a+x /run.sh &&      apt-get update && apt-get install -y --force-yes      make      g++      git      && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* ~/.cache      && git clone https://github.com/mysensors/MySensors.git --branch master $APPDIR' returned a non-zero code: 127


exception when execute an AQL query using Web interface

When I'm using the Web Interface to execute a query, I encounter the following behaviour.
LET start_end = ["mycollection/9141817", "mycollection/9210025"]

LET vertex1 = start_end[0]
LET vertex2 = start_end[1]

FOR v IN ANY SHORTEST_PATH vertex1 TO vertex2
    GRAPH 'myGraph'
    RETURN v.name

Executes without any issues. However, if try to get rid of the vertex1 and vertex2 assignments, I get an error:

LET start_end = ["mycollection/9141817", "mycollection/9210025"]

FOR v IN ANY SHORTEST_PATH start_end[0] TO start_end[1]
    GRAPH 'myGraph'
    RETURN v.name

This results in a warning Successfully aborted.
And when I explain:
Explain: missing variable #28 while planning registers (exception location: /tmp/arangodb-20160822-83950-1x83y8u/ArangoDB-3.0.5/arangod/Aql/ExecutionNode.cpp:1163). Please report this error to arangodb.com (while finalizing)

Reproduced with an example graph:
var examples = require("@arangodb/graph-examples/example-graph.js");
var graph = examples.loadGraph("traversalGraph");
db._explain(`
LET start_end = ["circles/a", "circles/d"]
FOR v IN ANY SHORTEST_PATH start_end[0] TO start_end[1]
    GRAPH 'traversalGraph'
    RETURN v.name
`)

Fixed in 3.0.6 and devel.
The issue occurred when the start or end vertices for the shortest path were non-constant expressions that required extra calculations.

3.0.10 is available for download now.


Create vendor_compromise_suspicious_recipient_cred_theft.yml

Description
A known sender in your environment is using suspicious recipients pattern, credential theft language, and links to a phishing page.

/update-test-rules

/update-test-rules

/update-test-rules

/update-test-rules

Ah, LinkAnalysis. Will have to hunt with it.

I'm ok with this, but interested in second opinions.


Sample outlets shouldn't spill over as much (or at least, not at all on the right)

Fixed in #495

Definitely fixed in 02bc1eef8a681f3f2bb36af746464f85af26983f


[varLib] support building variable COLRv1 table from set of masters

Now that we have support for 32-bit VarStore, required to variate some of the COLRv1 fields, fontTools.varLib should be updated to create a COLRv1.VarStore from a set of master fonts containing compatible COLRv1 tables.

First step would be to produce a set of masters to use for testing / merging.
Do you suggest adapting varLib to do merging? Or update your toolchain to generate variations directly? We can do the former, but would require idential-structure limitations.

yes, I was thinking varLib would do the merging, similarly to how it handles the other tables.
Indeed, COLR master tables would have to have identical structure.
nanoemoji currently can build a variable color font when given (via a config file) groups of SVG glyphs associated with a given location. It creates UFO masters for each and hands them off to ufo2ft to compile the variable font (which internally uses fontTools.varLib). But the COLR table is not varLib-merged yet, so only the outlines, metrics, etc. vary.


User Manager - Save Local User

When saving a local user (VeXaCore User) and amending their password, you get an error:

Happens only when saving a system user not all local users.
Turns out locking the username field to DISABLED instead of READONLY wasn't sending over the value to the update script. Amended to READONLY and fixed this issue.


[Enhancement]Documentation and snippets enhancements

Have explicit trigger for snippets from the query editors.

Slash command icons hover state.

Remove empty records from documentation search


How to run NNet training

Hi. I am trying now to run your code, but I meet some troubles. Firstly, I run clean_data.py for preparing data. Is my next step trainunet.py or vsnus.py? Both scripts have unresolved references to Wavenet and readDataset. Where can I find these packages? Thanks

Hi. I am trying now to run your code, but I meet some troubles. Firstly, I run clean_data.py for preparing data. Is my next step trainunet.py or vsnus.py? Both scripts have unresolved references to Wavenet and readDataset. Where can I find these packages? Thanks

These packages are available under the /modelStructure and /Dataset directories respectively.
Try replacing the import statement lines with the following ones prefixed with the directory name:
from modelStructure.wavenet2 import Wavenet
from Dataset.readDataset3 import Dataset,Testset,RandomCrop,ToTensor


interface/scipy_wrapper not compatible with scipy 1.11.

In interface/scipy_wrapper there is a BIN_OPS with __div__. In version 1.11 this is removed so it is no longer possible to import cvxpy without raising an error.
There were changes made to spmatrix in 1.11.  I assume that an equivalent spmatrix.__div__ still exists but I do not know its name.
Also, __div__ is the first binop in the list of operations so it is likely that the other binary operations have the same issue.
Thanks for considering this.

Thank you for raising this issue. We will patch it ASAP.

@pinguicula Which version of CVXPY are you one? This should have been fixed by https://github.com/cvxpy/cvxpy/pull/2159
I tested running 1.4 with SciPy 1.11 without issues.

My apologies: I am using conda and therefore running an older version.
Your linked issue seems to address this problem. I will close this issue and wait for the updated version in conda.
Thank you for your response and for your work on cvxpy!


Remove ShelleyGenesis from ShelleyLedgerConfig

The difference between Babbage and Conway was intentionally as minimal as possible. This has a couple benefits.

Clarke had orchestrated the integration of new eras. So it's nice that the first one we did without him is so minimal.

The minimality helped us (especially in Clarke's absence) notice the aspects that we found confusing.

This Issue is about one such aspect: I don't think ShelleyGenesis (as CompactGenesis) should be in the ShelleyLedgerConfig. In other words, I'm proposing removing the following function.
Ouroboros.Consensus.Shelley.Ledger.Ledger.shelleyLedgerGenesis :: ShelleyLedgerConfig era -> SL.ShelleyGenesis era

I've catalogued its uses as follows:

It's used in Ouroboros.Consensus.Cardano.CanHardFork for translating/forecasting from Byron to the first Shelley era. This is a necessary use.
It's used in Ouroboros.Consensus.Shelley.Ledger.Inspect, Ouroboros.Consensus.Shelley.Ledger.Ledger, and Ouroboros.Consensus.Shelley.ShelleyHFC to get the value of k, f (only used to compute the stability window), quorum, slotLength, and epochLength. I think all of these could and should instead use the (already-existing!) shelleyLedgerGlobals :: SL.Globals field.

The second bullet point is for enabling the hard fork combinator, which every era needs to do. The first bullet point is instead for translating from Byron to Shelley, which only the first Shelley-based era needs to do. Moreover, I don't think we want to require that any Shelley-based era could be the successor Byron, do we? So I propose:
data ShelleyLedgerConfig era = ShelleyLedgerConfig {
      shelleyLedgerGlobals                     :: !SL.Globals
    , shelleyLedgerTranslationContext          :: !(Core.TranslationContext era)
    , shelleyLedgerTranslationContextFromByron :: !(StaticMaybe (SL.ByronTranslationContext era))
    }

data StaticMaybe :: Maybe Type -> Type where
  StaticJust :: !a -> StaticMaybe (Just a)
  StaticNothing :: StaticMaybe Nothing

(see https://github.com/input-output-hk/cardano-ledger/pull/3053#discussion_r1015751613 for motivating the StaticMaybe layer)
where in the ledger we have:
type family ByronTranslationContext era

type instance ByronTranslationContext ShelleyEra = SL.ParedDownShelleyGenesis
type instance ByronTranslationContext AllegraEra = Void -- or TypeError?         (I originally wrote () here but then I objected to it on one of Bart's PRs :face_palm)
type instance ByronTranslationContext MaryEra    = Void -- or TypeError?
type instance ByronTranslationContext AlonzoEra  = Void -- or TypeError?
type instance ByronTranslationContext BabbageEra = Void -- or TypeError?
type instance ByronTranslationContext ConwayEra  = Void -- or TypeError?

As a result of this, we should be able to remove all of the oxymoronic TranslateEra ShelleyGenesis instances. We'll have to replace them with TranslateEra Globals instances (or some translate-able data that determines the Globals). For example, these new translation instances won't even have a "genesis delegates" field, and so it'll avoid the existing confusing fact that the TranslateEra ShelleyGenesis ConwayEra instance does not need to update the sgGenDelegs field!

We recently assigned Issue #3878 to @yogeshsajanikar -- we wanted an onboarding techdebt task that was directly altering the code because his onboarding work so far has been on infrastructure concerns. We wanted something for him that is more directly Consensus work.
I'm also assigning this one to him.
I think both Issues have a relatively narrow scope, won't require a burdensomely-large diff in the end, but aren't trivial. All of those are good qualities for this kind of onboarding. If Yogesh chooses one, maybe we'll give the other to Bart when Bart returns.

the epoch size is considered to be a global constant for the Shelley ledger

The Shelley ledger formal spec is written from the perspective of a single era -- it doesn't consider the HFC. And within a single era, the epoch size (and k and slot length etc) must be constant.

The external API used by the node makes use of the GetGenesisConfig query in the ledger. There seems to be no way to avoid breaking that query without keeping the ShelleyGenesis in ShelleyLedgerConfig. The relevant query is here.
I have asked the Node and Daedalus teams whether the query is actually used. The threads are here and here.

Hmm. Looks like that query GetGenesisConfig is only used to implement the internal node query QueryGenesisParameters, which is in turn only used to implement Cardano.CLI.Shelley.Run.Query.runQueryKesPeriodInfo, which only calls the following functions on the result.

protocolParamMaxKESEvolutions
protocolParamSlotsPerKESPeriod
protocolParamSystemStart

Those are all selectors from this record type: https://github.com/input-output-hk/cardano-node/blob/df1dea590d9ad7a6caed58147ad8792ff98711db/cardano-api/src/Cardano/Api/GenesisParameters.hs#L37

protocolParamMaxKESEvolutions ::  Int
protocolParamSlotsPerKESPeriod :: Int
protocolParamSystemStart :: UTCTime

Those fields are also all present in the similar data type in the Ledger https://github.com/input-output-hk/cardano-ledger/blob/9a25a43f3970941a2cd64029f7ce667b48832635/libs/cardano-ledger-core/src/Cardano/Ledger/BaseTypes.hs#L585

maxKESEvo :: !Word64
slotsPerKESPeriod :: !Word64
systemStart :: !SystemStart  (which holds a UTCTime)

My proposed ShelleyLedgerConfig above does contain those data, via shelleyLedgerGlobals.
So this doesn't seem fully blocked to me. It's a matter of:

confirming my analysis here (eg there's no other use of QueryGenesisParameters? Or no other client tool that relies on GetGenesisConfig? Maybe db-sync does?)
determining the development cost of instead serving these data through a narrower query (eg perhaps a narrower GetKesInfo query would replace GetGenesisConfig?)
convincing folks it's worth it (I really want to simplify the HFC story here, but I could technically be convinced it's not worthwhile)

cc @bartfrenk @dnadales

One idea would be to keep the GetGenesisConfig query while still removing ShelleyGenesis from the ledger cfg.
This could be done by adding it as a field to the BlockConfig (static block configuration actually does not seem like a bad place for it, and the Shelley BlockConfig already contains stuff from ShelleyGenesis; it is also already an argument to mkShelleyBlockConfig). Also, answerBlockQuery has access to the BlockConfig via ExtLedgerConfig/TopLevelConfig.
This would minimize the scope of the issue (shrinking the Shelley ledger cfg to what actually needs translating) without having to submit upstream PRs (IIRC, a PR to wallet would otherwise also be necessary). At the same time, it does not prevent tweaking/removing GetGenesisConfig in the future if we think that is desirable; we can still do that separately.

Bah! I could have sworn I already wrote a reply here... I'll attempt to replicate what I thought I wrote, but I'm a bit rushed at the moment.
We discussed this on a call. I think Esgen's suggestion is strictly-speaking an improvement (because it avoids the unnecessary translations---he emphasized on the call that we never translate BlockConfigs.) However, I would prefer we consider it a contingency plan. The current plan is more involved, but I think it has lots of value, including that exercise-like elements of @bartfrenk touching other repos and interacting with other teams, narrowing an (apparently) overly-broad query, and removing the duplication (Globals vs (Compact)ShelleyGenesis) within the *Config data types.

My plan is to do this in two steps, and separate blocks of PRs:

Remove ShelleyGenesis from the LedgerConfig, while keeping the query (using the aforementioned BlockConfig). This touches both ouroboros-consensus and cardano-ledger. There is already a draft PR for ledger here: https://github.com/input-output-hk/cardano-ledger/pull/3053.
Update cardano-node, wallet and potentially other repo's to remove usage of GetGenesisConfig and then remove it from ouroboros-consensus.

The benefits to this approach, as I see them, are that we keep a sense of progress, and that we separate the work involving mostly simple changes to the codebase, from the work (and the delays) involved in interacting with other teams.
The drawback that there is overhead in re-implementing the GetGenesisConfig query from the BlockConfig, but I think that is fairly minor.

Sounds good. It'll end up with some excessive churn in the commit diffs, but that's ultimately fine if it simplifies the work for you 👍

I updated the description of this ticket to reflect what I just now commented on here: https://github.com/input-output-hk/cardano-ledger/pull/3053#discussion_r1015751613

Update: we've discussed a bit on Slack, Crucially, @amesgen noticed the relation to a change I made in this neighborhood when drafting the Conway PR. I think that decoupled the two in a way that unblocks Bart here, but hopefully @bartfrenk can confirm that and write a follow-up explanation here 👍

Marked this as "Ready" again. Current status:
The relevant upstream PRs (https://github.com/input-output-hk/cardano-ledger/pull/3164, https://github.com/input-output-hk/cardano-ledger/pull/3224) have landed and have been in the cardano-ledger releases for quite some time.
In particular, ShelleyGenesis is no longer parameterized by era (only by crypto), so its weird TranslateEra instances are already gone (see https://github.com/input-output-hk/cardano-ledger/pull/3224), so the main motivation of this issue has already been resolved.
The remaining work is to remove ShelleyGenesis (in the form of CompactGenesis) from ShelleyLedgerConfig, as we don't actually need all of it (as is already clear from the usage of CompactGenesis), and it is a bit weird to have a ShelleyGenesis in eg the Conway ledger config. This work is complicated a bit by the existing GetGenesisConfig query, so Bart's approach was to do this in two steps. See the commits by Bart in #4091 for how the first step would look like.


Doc: fix typo of schema

Hi, first of all, thanks for a nice library!
This pull request fixes a typo: scheme -> schema.

Thank you!


Update fiscal.yaml

Request for withdrawal according to the discussion at the UPOS meeting on Oct. 27, 2022

Finalized fiscal data

Finalized fiscal data


[FIX] Change regex maching rule for the flat file handler

Description
An error 404 was raised for urls like
http://mgmt.ii.inmanta.com:8888/console/resources/exec%3A%3ARun[longevity-2022.1.ii.inmanta.com],
because the flat file handler pattern match was accepting all non-space characters as valid file extensions and therefore look for a file named http://mgmt.ii.inmanta.com:8888/console/resources/exec%3A%3ARun[longevity-2022.1.ii.inmanta with file extension .com], which doesn't exist.
Solution
Instead of disallowing spaces, only accept alphanumeric characters.
closes #307
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[ ] Attached issue to pull request
[ ] Changelog entry
[ ] Type annotations are present
[ ] Code is clear and sufficiently documented
[ ] No (preventable) type errors
[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )

Reviewer Checklist:

[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Code is clear and sufficiently documented
[ ] Correct, in line with design

And I looked into \s within [^...] blocks here

And I looked into \s within [^...] blocks here


Question/Maintenence

I what used this several years ago, maybe 2017 or so, before it was an extension iirc. I have a desire for continuous queries on PostgreSQL again and so I have been looking around.
I have experience working on the OpenTSDB project. Would there be any interest in someone taking over maintenance and potentially improving or continuing this project? Or is there a better alternative for PostgreSQL that would render this unnecessary?

@johann8384 I would gladly help you however I could.

Check out https://docs.timescale.com/timescaledb/latest/how-to-guides/continuous-aggregates/


Fix achievements notifications animation

This fixes #3243. Achievement notification animation now slides in from
off-screen left and then hides back to off-screen left.

Thanks @taptapdan!


Allow re-opening a closed tab panel

As a user, I want to be able to re-open a closed panel so that I can restore with it's state if I accidently close a tab.
Suggested changes:

Remember the last 15 (arbitrary number?) closed panels PER dashboard, their location and their the state when closed. This can be ephemeral between sessions, doesn't need to be persisted.
Add a configurable shortcut (win: alt+shift+t / mac: opt+shift+t) to re-open the closed tab
Add a context menu action to the panel tabs, with a right click menu option for "Re-open closed panel". It should appear in it's own group just above the close group, and display the appropriate shortcut.

Re-opening a closed tab should restore state (ie. filters, sorts, linker etc should be resorted). We may not be able to re-open in the same place if the stack the tab was part of is removed, so it should instead re-launch in a new position if required (but ideally opens in the same location).

@dsmmcken Context menu on the panel tab? Shouldn't text be "Re-open closed panel" instead? What about if you remove the last panel from the dashboard, what do you right click on then?

I suppose we could conditionally show a button on the "Dashboard is Empty" screen (as button centered below that text) if there's panels available to be re-opened. Does that screen exist now in DHC?


Accesing Event handler for select2entity bundle?

Is it possible to access event handlers defined in select2  js for the  dropdown element:
`$eventname= $(.select2example).select2()
$eventname.on("select2:select", function(e){
ajaxupdatessamplevariable(e.params.data);
console.log(e);
           });`

I needed to access the event handler using element id (which has a format of bundlename_formtypename[optional]formname ).
Is there a better way to access it?


Update README.md with macos build & install tip

Usually I'd bundle this into another PR but the MacOS implementation of bringToFront is ugly and might take a few weeks to land.

Little update on this stale PR:
I'll clean up the Mac implementation of bringToFront this week & get it PR'd, either reusing this one or opening a new one.

For some reason, using a local JWM version with publicly released bringToFront code (see merged #269) is failing to correctly steal focus in Mac. The code runs but simply fails to steal focus -- calls to isFront afterwards always returns false. Perhaps this is a permissions issue that the local build works around?
I'm not sure yet but I'll need to PR bringToFront once I figure out a fix. In the meantime, I'm going to close this PR & bundle it into #286 to reduce PR churn.


Update macos-cheatsheet.md

fix some texts and alignments
Change Summary
Provide summary of changes with issue number if any.
Checklist
If you haven't fulfilled the below requirements or even delete the entire checklist, your PR won't be reviewed and will be closed without notice.
General

[ ] This Pull Request is all my own work. (You'll be blacklisted if you are caught for plagiarism.)
[ ] I've read CONTRIBUTING.md
[ ] I've made some valid changes to the CheatSheet, and they are not just minor changes.

Changes

[ ] I've read and followed the Solution Template

Note: To mark the checkbox, put an x inside the [ ] (like this: [x]).

@loretogutierrezjr As you have been previously assigned for this issue and these are some minor fixes, I won't be adding the hacktoberfest tag. You can contribute to any other issues that seem valid 🙂


fix: redirect using url config key

🔗 Linked issue

❓ Type of change

[ ] 📖 Documentation (updates to the documentation or readme)
[x] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
This PR changes the redirect to use the site.url config.
According to the docs, redirectToCanonicalSiteUrl redirects to the default canonical url, which is supposed to be site.url, not site.site

:man_facepalming: thanks for spotting this and the fix!


help  UnicodeDecodeError when using the Tokenizer data generator

Description
Hi , I'm generating my own problem and I'm using the tokenizer.encode from the Tokenizer data generator but I'm having problems when using t2t-decoder. I get an error:
UnicodeDecodeError: 'utf8' codec can't decode byte 0xa3 in position 0: invalid start byte.  I thought it might be related to issue #74 but I have the latest version of Tensor2Tensor.  I could not find any solution yet. Is there something else I need to take into account when using the tokenizer.encode to avoid this?
Thank you,
Daniela

Please provide Python, T2T, and Tensorflow versions you're using.

Python Version 2.7.14
T2T Version: 1.6.3
Tensorflow Version: 1.7.0

I also tried it with
Python Version 2.7.13
T2T Version: 1.6.0 and 1.6.1
Tensorflow Version: 1.7.0
and I get the same error.


Don't let 'console.log' format objects

Digging through memory dumps and this appears to be ~50% of our "memory leak" because console.log holds on to the objects passed to it forever for whatever dumb reason.

After a lot of digging through the Chrome Debugger, I found two memory leaks that this PR tackles:
Objects passed to console.log live forever under certain circumstances.
As far as I can tell, this behavior is enabled by the Chrome debugger itself to let you inspect objects logged to the console and dig out where/how it was created, and so on. Eliminating this eliminated ~50% of all of the memory leakage I could detect while connected to the debugger but it does nothing actually in staging.
I still recommend keeping this part of the PR around to 1) prevent people from falling down the same rabbit hole I did as it was pointing at some of our own objects that actually weren't at fault and was leading me the wrong direction on what's going wrong and 2) I don't know if there's any other condition that can enable this behavior that would still be bad in production, so let's just make it impossible.
The TypeORM ConnectionManager never deletes any connection
Because we name each connection by a UUID so they don't collide, that means there's never a collision within the ConnectionManager and they stay around forever. This was the majority of the leakage. We also forgot to connection.close the ctx.orm connection objects at the end of RPC calls which would have also kept them around basically forever (but fixing just that didn't clear it up).
Now our memory usage in the staging environment looks like this:

The latest deploy is flat, so the memory leak is fixed.


Hide category view pager in case of no categories

Description (required)
In case of no category of image skip to next question in review activity
Fixes #2698 Improve peer review feature quality
Changes made
On category view pager in case of zero categories enlisted swipe to next view pager.

Codecov Report

Merging #2728 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #2728      +/-   ##
=========================================
- Coverage    2.72%   2.72%   -0.01%
=========================================
  Files         267     267
  Lines       12814   12818       +4
  Branches     1140    1141       +1
=========================================
  Hits          349     349
- Misses      12439   12443       +4
  Partials       26      26

Impacted Files
Coverage Δ

...r/free/nrw/commons/review/ReviewImageFragment.java
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 239f749...cb73953. Read the comment docs.

@domdomegg , can you share how did you tested? I also tried to test this PR but I couldn't. Because I couldn't find any upload with no category at all. There are auto added categories, so I couldn't test. @vanshikaarora how did you tested this when you are working on? Can you please share the exact successful file?

@domdomegg , can you share how did you tested? I also tried to test this PR but I couldn't. Because I couldn't find any upload with no category at all. There are auto added categories, so I couldn't test. @vanshikaarora how did you tested this when you are working on? Can you please share the exact successful file?

On beta there are a lot of uncategorized images as I don't think they are automatically added. For example the one in the GIF MobileTest<PHONE_NUMBER>40.jpg

But there are categories on mentioned file @domdomegg , see bottom of the page https://commons.wikimedia.beta.wmflabs.org/wiki/File:MobileTest_190321-112640.jpg

Yeah, I'm not sure how but it seems to ignore automatic categories. Another example in screenshot above: https://commons.wikimedia.beta.wmflabs.org/wiki/File:Test_93.jpg

Maybe relevant logs
2019-03-27 22:12:43.714 30186-30233/fr.free.nrw.commons.beta D/CustomApiResult: API response for method https://commons.wikimedia.beta.wmflabs.org/w/api.php is
     <?xml version="1.0" encoding="UTF-8"?><api batchcomplete=""><warnings><main xml:space="preserve">Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application.</main><revisions xml:space="preserve">The parameter "rvgeneratexml" has been deprecated.
    Because "rvslots" was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used.</revisions></warnings><query><pages><page _idx="64070" pageid="64070" ns="6" title="File:Test 93.jpg"><revisions><rev parsetree="<root><h level="2" i="1">== <template><title>int:filedesc</title></template> ==</h>
<template lineStart="1"><title>Information
</title><part><name>description</name><equals>=</equals><value>
</value></part><part><name>source</name><equals>=</equals><value><template><title>own</title></template>
</value></part><part><name>author</name><equals>=</equals><value>[[User:Whym test1|Whym test1]]
</value></part><part><name>date</name><equals>=</equals><value><template><title>According to EXIF data</title><part><name index="1"/><value>2019-02-05</value></part></template>
</value></part></template>
<h level="2" i="2">== <template><title>int:license-header</title></template> ==</h>
<template lineStart="1"><title>self</title><part><name index="1"/><value>cc-by-sa-3.0</value></part></template>

<template lineStart="1"><title>Uploaded from Mobile</title><part><name>platform</name><equals>=</equals><value>Android</value></part><part><name>version</name><equals>=</equals><value>2.9.0-debug-stash-upload~119384e40</value></part></template>
<template lineStart="1"><title>Uncategorized</title><part><name>year</name><equals>=</equals><value>2019</value></part><part><name>month</name><equals>=</equals><value>February</value></part><part><name>day</name><equals>=</equals><value>5</value></part></template></root>" contentformat="text/x-wiki" contentmodel="wikitext" xml:space="preserve">== {{int:filedesc}} ==
    {{Information
    |description=
    |source={{own}}
    |author=[[User:Whym test1|Whym test1]]
    |date={{According to EXIF data|2019-02-05}}
    }}
    == {{int:license-header}} ==
    {{self|cc-by-sa-3.0}}

    {{Uploaded from Mobile|platform=Android|version=2.9.0-debug-stash-upload~119384e40}}
    {{Uncategorized|year=2019|month=February|day=5}}</rev></revisions></page></pages></query></api>

@domdomegg no template added on my tests when there is an automatic category. Actually, nothing happens when I skip the image under this condition, and this is expected behaviour I think. But I never experience a entirely non categorized image.
Maybe @nicolas-raoul can know. It is possible to find an image with literally no categories (If no, this PR has no impact on real life). Or every image will have some categories, at least auto added ones.

@neslihanturan I am not sure sorry, the easiest to to just try, I guess :-) Or check the many images that get uploaded without categories every day.

Please feel free to reopen this PR again if you really feel that this is needed. currently it seems like a caution for an impossible case


refactor(plot): refactor callback scheduling in Plot

Description
This PR refactors callback_sched in Plot package.
Related links

JIRA ticket: https://tier4.atlassian.net/browse/T4PB-24805
To reproduce: https://drive.google.com/drive/folders/1ir_2ixQzZkjzo_PP_PYj3BUO-exyRQko?usp=share_link

Notes for reviewers
Pre-review checklist for the PR author

[x] I've confirmed the contribution guidelines.
[ ] The PR is ready for review.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR has been properly tested.
[ ] The PR has been reviewed.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] (Optional) The PR has been properly tested with CARET_report verification.
[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

@hsgwa
I have confirmed with CARET_report that there is no problem.
Could you please review this PR?


Update KerwisChinaAerospacePack.netkan

Hey, sorry if I'm missing something obvious, are you part of the dev team of this mod? Or are these changes based on observations as a user?
Most resource links are automatically fetched from SpaceDock, no need to hardcode them in the netkan:
https://github.com/KSP-CKAN/CKAN-meta/blob/a3f08f3e00b02fceac24733c3230fb08db8046c8/KerwisChinaAerospacePack/KerwisChinaAerospacePack-1.0.2.ckan#L14-L20
The ksp_version_max can be taken from SpaceDock as well.
The license on the forum thread is still CC BY-NC-SA 4.0, so we shouldn't loose the version specificity there.
What's the reason for the "development" release_status? The mod doesn't seem to be marked WIP or similar.
Please don't remove the tags, every mod should have them.
I'm not sure about the supports: - name: RealismOverhaul. It makes the mod show up during the RO installation process, and given that RO has delicate relationships and can break quickly, I'm not sure if we should offer it to users there.

are you part of the dev team of this mod? Or are these changes based on observations as a user?

I did not directly involved in the development of this mod, however, I am helping the devs updating this under their consent since English is not their first language and they are not familiar with CKAN.

Most resource links are automatically fetched from SpaceDock, no need to hardcode them in the netkan: https://github.com/KSP-CKAN/CKAN-meta/blob/a3f08f3e00b02fceac24733c3230fb08db8046c8/KerwisChinaAerospacePack/KerwisChinaAerospacePack-1.0.2.ckan#L14-L20
The ksp_version_max can be taken from SpaceDock as well.
The license on the forum thread is still CC BY-NC-SA 4.0, so we shouldn't loose the version specificity there.
Please don't remove the tags, every mod should have them.

I will fix these in the next commit.

What's the reason for the "development" release_status? The mod doesn't seem to be marked WIP or similar.

The author told me that it should be in development since there are more parts to be added in future updates.

I'm not sure about the supports: - name: RealismOverhaul. It makes the mod show up during the RO installation process, and given that RO has delicate relationships and can break quickly, I'm not sure if we should offer it to users there.

I didn't think about that, thanks for mentioning it, I will remove it in the next commit.

The license on the forum thread is still CC BY-NC-SA 4.0, so we shouldn't loose the version specificity there.

Checks won't pass the "License should match spec" thing if the version number was added.
However, the checks will pass if I run netkan.exe --verbose .\KerwisChinaAerospacePack.netkan on my local machine.
It would be greatly appreciated if you can help fix this @DasSkelett .

Checks won't pass the "License should match spec" thing if the version number was added.

You need to replace the space with a dash. Just revert it to how it was before your changes:
https://github.com/KSP-CKAN/NetKAN/blob/19d5c820ea6164756c781266a5e52f3a122eae3a/NetKAN/KerwisChinaAerospacePack.netkan#L4

The author told me that it should be in development since there are more parts to be added in future updates.

What type of updates, just general feature updates? development in CKAN means that the mod is in its very first stages, with many bugs to be expected and important core features missing. Mod authors usually declare that on the forum thread, and/or use version numbers below 1.0.
Reading the forum thread it doesn't sound like that, but of you have better inside info, I'll trust those.

You need to replace the space with a dash. Just revert it to how it was before your changes:

Thanks, I didn't notice it was there before.

I will inform the author about the software versioning convention issue so they will respect that in future projects.

They don't need to change their versioning system for CKAN, please don't bother them for that. That was just a data point in my observation about the stableness of the mod.
Anyway, changes almost look good, I'm just gonna move the ksp_version_min into an override to make it work properly (I thought it would merge it with SpaceDock's data as is, but apparently it doesn't)

I will inform the author about the software versioning convention issue so they will respect that in future projects.

They don't need to change their versioning system for CKAN, please don't bother them for that. That was just a data point in my observation about the stableness of the mod.
Anyway, changes almost look good, I'm just gonna move the ksp_version_min into an override to make it work properly (I thought it would merge it with SpaceDock's data as is, but apparently it doesn't)

Thanks a lot for your help!


Hi there, does Bagua support model parallel across different nodes? If so, would you mind providing a simple example in examples folder? Thank you.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Currently Bagua by itself does not support model parallel. It is on our roadmap though.
At the moment, you can use PyTorch's rpc to do distributed model parallelism. Also, if your model uses both data parallel and model parallel, you can still use Bagua to accelerate the data parallel part.


txBuilder − Manage Buy Offer − Offer ID should be optional

It's all in the title. Currently the interface forces user to enter an offerId parameter, which doesn't match the network behavior.

Hmm it doesn't say it's optional on the doc https://www.stellar.org/developers/guides/concepts/list-of-operations.html#manage-buy-offer

We will update it so that if a user didn't specify a value in Offer ID input, it will be set to 0.

Yes it does... Just look at the JS doc, that's the API you use :
https://stellar.github.io/js-stellar-sdk/Operation.html#.manageBuyOffer

You're right.
We will update so that:

offerId's default value is 0 (this isn't clear on laboratory at the moment)
Add an optional attribute


Disable xz benchmark.

XZ tarballs were backdoored.

Context: https://github.com/google/oss-fuzz/issues/11760

CVE-2024-3094 is the relevant CVE.

Wow, thanks both for fixing this and providing the context!


Standardize logging

How should we standardize logging in kf?
packages should all use the same pattern to write their info/warning/error messages.

I think in line with this, we should consider how kubectl can be easily scripted by giving it a go template.


[fix] [broker] [branch-2.9] Upgrade rocksDB version to <IP_ADDRESS> to keep in sync with BookKeeper's RocksDB version

…cksDB version

Fixes #xyz

Master Issue: #xyz

PIP: #xyz

Motivation

Upgrade rocksDB version to <IP_ADDRESS> to keep in sync with BookKeeper's RocksDB version
Modifications

Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository:

As discussed on the mailing list https://lists.apache.org/thread/w4jzk27qhtosgsz7l9bmhf1t7o9mxjhp, there is no plan to release 2.9.6, so I am going to remove the release/2.9.6 label


Updated & cleaned up all of the Web.gitattributes

Looks really nice! Thanks :)

Your welcome & thank you for sharing the original templates :)


chore(website): Update website demos

This updates the website demos to use electric 0.10.x
I've also added a bundler and lint/formatting to the project so it can be ran from the main electric repo without the website as well - obviously they will not be 100% in sync but it at least gives a way to run them independently (and I can write some basic e2e tests for them as well)
Main change is getting the client API up to speed (e.g. use the connect API).
Merging this should fail the website build as it will require updating the website repo as well https://github.com/electric-sql/website/pull/124

Looks good overall!
I tried building and running this locally, outside of the website repo.
Had to uncomment dummy variables after seeing complaints about them in the browser log. On the screenshot, you can see it working but there are still two errors in the console.

Another weird but non-critical issue I saw was that, for some reason, the yarn db:migrate command gets stuck for ~45 seconds regardless of whether there are any migrations to be applied. The first time I ran it, I saw this in the terminal:
$ yarn db:migrate
yarn run v1.22.21
$ npx electric-sql with-config "npx pg-migrations apply --database {{ELECTRIC_PROXY}} --directory ./db/migrations"
Applying<PHONE_NUMBER>-initial_data_model.sql
Applied<PHONE_NUMBER>-initial_data_model.sql
1 migrations applied
Done in 50.91s.

and this in Electric's log
12:27:00.468 pid=<0.3730.0> proxy_session_id=3 [warning] Not validating user "postgres"
12:27:00.471 pid=<0.3730.0> proxy_session_id=3 [info] Initialising injector in capture mode %Electric.Postgres.Proxy.Injector.Electric{}
12:27:00.506 pid=<0.3730.0> proxy_session_id=3 [info] Upstream connection is ready to accept queries
12:27:49.395 pid=<0.3419.0> pg_producer=postgres_1 [info] Migrating version <nil> -><PHONE_NUMBER>
12:27:49.395 pid=<0.3419.0> pg_producer=postgres_1 [info] Applying migration<PHONE_NUMBER>: "CREATE TABLE demos (\n    id text NOT NULL,\n    name text NOT NULL,\n    inserted_at text NOT NULL,\n    updated_at text NOT NULL,\n    electric_user_id text NOT NULL,\n    CONSTRAINT demos_pkey PRIMARY KEY (id)\n);\n\n\n"
12:27:49.395 pid=<0.3419.0> pg_producer=postgres_1 [info] CREATE TABLE demos (...)

Notice the time gap between [info] Upstream connection is ready to accept queries and the line that follows it.
The next time I executed yarn db:migrate again, I saw this:
$ yarn db:migrate
yarn run v1.22.21
$ npx electric-sql with-config "npx pg-migrations apply --database {{ELECTRIC_PROXY}} --directory ./db/migrations"
No migrations required
Done in 50.99s.

and only this new output in Electric log:
12:31:08.408 pid=<0.3739.0> proxy_session_id=7 [warning] Not validating user "postgres"
12:31:08.411 pid=<0.3739.0> proxy_session_id=7 [info] Initialising injector in capture mode %Electric.Postgres.Proxy.Injector.Electric{}
12:31:08.416 pid=<0.3739.0> proxy_session_id=7 [info] Upstream connection is ready to accept queries

I'm really not sure what's up with this but am not considering this a blocker for the PR.

@alco thank you for reviewing and testing it out!
re: dummy variables - indeed that's why for now I've just added a note in the README about them - I really didn't want to use webpack, I'll find a better workaround in a subsequent PR for easier building :)
re: other errors - the rest of the backend for the intro demos is in the website repo, I could move it into the monorepo as well and use some one-off secret keys etc for testing. Because it is not deployed you'll see some failures when it tries to reach the API.
re: migration delay - I'm experiencing it as well and I'm not experiencing something similar on other projects, I can triage it a bit more separately from this PR

re: other errors - the rest of the backend for the intro demos is in the website repo, I could move it into the monorepo as well and use some one-off secret keys etc for testing. Because it is not deployed you'll see some failures when it tries to reach the API.

Let's not go that far. I've added a note to the readme warning people about this (pushed a new commit to your branch).

re: migration delay - I'm experiencing it as well and I'm not experiencing something similar on other projects, I can triage it a bit more separately from this PR

It's only worth it investigating if we run migrations as part of the deployment step in prod and the delay appears in that environment as well.

BTW, @magnetised might have a guess as to where to look for the source of the delay.

It's only worth it investigating if we run migrations as part of the deployment step in prod and the delay appears in that environment as well.

Migrations are done manually not as an automated deployment step, so not an immediate issue for this.
Will merge this so I can start the website redeployment.


Flask Extension that doesn't require user code changes

Hi,
Can a flask extension be installed using pip and run without user code changes at all?
I want to write a flask extension that uses flask's before_request and after_request hooks.
The user will just need to pip install flask_<my_extension> and that's it.
Thanks in advance.

No. Flask extensions are not plugins but libraries developers can use in their Flask applications.
It sounds more like you want some kind of plugin system...

Please dont write such an extension that automatically hooks itself into Flask, it makes everything harder to debug, if you have some behavior you don't know where it came from.


Question: Complex expression mapping

Consider the following models:
namespace ModelMapping.Application
{
    public class Person
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
    }
}

namespace ModelMapping.Domain
{
    public class Person
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
    }
}

how can I map the following expression?
static Expression<Func<IQueryable<Domain.Person>, IOrderedQueryable<Domain.Person>>> MyFunction(Expression<Func<IQueryable<Application.Person>, IOrderedQueryable<Application.Person>>> expression) { }

I use this expression like this:
Expression<Func<IQueryable<Application.Person>, IOrderedQueryable<Application.Person>>> expression = persons => persons.OrderBy(p => p.Name);

Thank you.

Hi
I'm not sure what exactly you want to achieve but you can take a look at this issue https://github.com/cezarypiatek/MappingGenerator/issues/34
Generating a mapping expression should be supported. MappingGenerator is generating recursive mappings so I would suggest paying special attention to the mapping of related entities in order to avoid n+1 problem when you are operating on IQueryable<>

Hi @cezarypiatek,
No, issue #34 doesn't solve my problem at all.
What I need to map, beyond models are Where expressions such as
Expression<Func<SourceModel, bool>>

and Queryable expressions, such as
Expression<Func<IQueryable<SourceModel>, IOrderedQueryable<SourceModel>>>

to generate in short the mappings for:
Expression<Func<TargetModel, bool>> // and
Expression<Func<IQueryable<TargetModel>, IOrderedQueryable<TargetModel>>>

I want to get rid of AutoMapper too, it's causing me a lot of trouble with these mappings. In short I'm trying to make my research project less dependent as possible from 3rd party frameworks and AutoMapper is one of them.
Thanks.

Can you provide an example of code that should be generated?

@cezarypiatek, I have no idea, I just use AutoMapper to map between the expressions from my Application to Domain models just to transport lambda queries, such as where and order by, for example:
// where
person => person.Age >= 18;

// order by
persons => persons.OrderBy(person => person.Name);

I'm sorry, I'm really no expert on Mapping expressions.
Thanks,
[],

I guess there are three  hidden operations performed under the hood by AutoMapper:

filtering (Where())
sorting (OrderBy())
projections (Select())

The first two (filtering and sorting) you have to do by yourself and the third one (select()) can be generated by MappingGenerator. The mapping expression it's the exact thing that #34 explains.
In your case, the example code can looks as follows
IEnumerable<ModelMapping.Domain> SimulateAutoMapperMagic(IContext<ModelMapping.Domain> input)
{
    return input.Where(person => person.Age >= 18)
        .OrderBy(person => person.Name)
        .Select(/* Here comes the expression generated by MappingGenerator */)
        .ToList();
}

@rmszc81  can you confirm that my previous commend did or did not answer your question? Is there anything else that I can help you with?

@cezarypiatek sorry, I really forgotten to answer you.
yes, your answer solve a part of my problem but, as my project is working extensively based on the generics technique, I'm being forced to use a framework for this time.
I will research more and see what I can do. For now, I will close the thread since my question is not really easy to answer due to my needs.
Thank you very much!
Regards,
[],
@rmszc81


DNU doesn't like semi-colons anymore.  It uses multiple args for the same flag.

…

Oh yeah.  I made this on to main to try to get it as a patched version against 0.10.1

@adamhathcock Thank you for your submission.  We have an upcoming release (within the next 2 weeks) will would likely be within the same timeframe as a hotfix release onto the main branch.  At this time, I think it would make more sense to target the develop branch for this PR, and we can include it in the 0.11.0 release.
I have left some comments, if you could fix them up, that would be great!  Thanks


Add Dockerfile and docker-compose for development and test (#281)

Add Dockerfile and docker-compose for development

Update Docker-Compose files as per .env changes

Replace npm with yarn in Dockerfile

Replace npmrc with yarnrc

Remove yarn installation steps from Dockerfile

Add docker-compose test

Coverage remained the same at 89.542% when pulling 490b2850f36589ee31c3504c0c1cfabbec664e4f on develop into 823fa8ec39cff3b30e69860cf02f927c85a7f687 on master.


The 'jf audit' command scans dev dependencies after running npm install

Describe the bug
It was reported that the 'jf audit' results in the CLI were displaying vulnerabilities not shown when scanning the same package in the Xray UI.
Current behavior
At first, the behavior was not reproducible, the 'jf audit' command showed no vulnerabilities like the UIG:
~/node_modules/sql-formatter$ jf audit
17:25:09 [Debug] JFrog CLI version: 2.50.1
17:25:09 [Debug] OS/Arch: linux/amd64
17:25:09 [🔵Info] Log path: /home/test/.jfrog/logs/jfrog-cli.2023-10-16.17-25-09.20962.log
💬The full scan results are available here: /tmp/jfrog.cli.temp.-1697477133-1999686231
Vulnerable Dependencies
┌─────────────────────────────────────────────┐
│ ✨ No vulnerable dependencies were found ✨ │
└─────────────────────────────────────────────┘
A few days later in the same environment, tested 'jf audit' again on the same package and saw a different result (the same result the reporter of the behavior saw):
jf audit
14:11:30 [Debug] JFrog CLI version: 2.50.1
14:11:30 [Debug] OS/Arch: linux/amd64
14:11:30 [🔵Info] Log path: /home/test/.jfrog/logs/jfrog-cli.2023-10-20.14-11-30.2407.log
💬The full scan results are available here: /tmp/jfrog.cli.temp.-1697811129-1629128009
Vulnerable Dependencies
┌────────────┬────────────┬────────────┬────────────┬────────────┬──────────┬──────┬────────────────┐
│ SEVERITY   │ DIRECT     │ DIRECT     │ IMPACTED   │ IMPACTED   │ FIXED    │ TYPE │ CVE            │
│            │ DEPENDENCY │ DEPENDENCY │ DEPENDENCY │ DEPENDENCY │ VERSIONS │      │                │
│            │            │ VERSION    │ NAME       │ VERSION    │          │      │                │
├────────────┼────────────┼────────────┼────────────┼────────────┼──────────┼──────┼────────────────┤
│ 💀Critical │ release-it │ 15.11.0    │ vm2        │ 3.9.19     │          │ npm  │ CVE-2023-37903 │
├────────────┼────────────┼────────────┼────────────┼────────────┼──────────┼──────┼────────────────┤
│ 💀Critical │ release-it │ 15.11.0    │ vm2        │ 3.9.19     │          │ npm  │ CVE-2023-37466 │
├────────────┼────────────┼────────────┼────────────┼────────────┼──────────┼──────┼────────────────┤
│ 🔥High     │ release-it │ 15.11.0    │ semver     │ 7.5.1      │ [5.7.2]  │ npm  │ CVE-2022-25883 │
│            │            │            │            │            │ [6.3.1]  │      │                │
│            │            │            │            │            │ [7.5.2]  │      │                │
└────────────┴────────────┴────────────┴────────────┴────────────┴──────────┴──────┴────────────────┘
At first the behavior didn't seem consistently reproducible but after a while found that is was reproducible after running an 'npm install' command.
Reproduction steps
Test on fresh/clean env:

export JFROG_CLI_LOG_LEVEL=DEBUG

npm i sql-formatter

cd node_modules/sql-formatter/

jf audit

Review the debug logs to see that it only scans the 10 prod dependencies:
[Debug] Running 'npm ls --ignore-scripts --json --all --long --package-lock-only' command.
[Warn] error while running '/usr/bin/npm ls --ignore-scripts --json --all --long --package-lock-only': exit status 1
npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>[Debug] Created 'npm' dependency tree with 10 nodes. Elapsed time: 2.1 seconds.
[Debug] Unique dependencies list:
[
"npm://railroad-diagrams:1.0.0",
"npm://sql-formatter:13.1.0",
"npm://nearley:2.20.1",
"npm://moo:0.5.2",
"npm://randexp:0.4.6",
"npm://discontinuous-range:1.0.0",
"npm://ret:0.1.15",
"npm://argparse:2.0.1",
"npm://get-stdin:8.0.0",
"npm://commander:2.20.3"
]
[Info] Scanning 10 npm dependencies...

To reproduce (In same env as above with first 3 steps completed):

npm install  (It will show error but I fixed the error and noted it has no impact on results)

jf audit

Review debug logs to see that it scanned 900+ dependencies and the output of the 'npm ls ...' command is significantly different:
[Debug] Running 'npm ls --ignore-scripts --json --all --long' command.
[Debug] npm 'ls --ignore-scripts --json --all --long' standard output is:
{
"_from": "sql-formatter",
"_id"<EMAIL_ADDRESS>"_inBundle": false,
"_integrity": "sha512-/nZQXuN7KzipFNM20ko+dHY4kOr9rymSfZLUDED8rhx3m8OK5y74jcyN+y1L51ZqHqiB0kp40VdpZP99uWvQdA==",
"_location": "/sql-formatter",
...
[Debug] Created 'npm' dependency tree with 906 nodes. Elapsed time: 9.8 seconds.
[Debug] Unique dependencies list:
[
"npm://string.prototype.trim:1.2.8",
"npm://@types/http-cache-semantics:4.0.4",
...
"npm://natural-compare-lite:1.4.0",
"npm://@babel/plugin-syntax-dynamic-import:7.8.3",
"npm://wildcard:2.0.1",
"npm://@typescript-eslint/scope-manager:5.62.0",
"npm://flatted:3.2.9",
"npm://@typescript-eslint/parser:5.62.0",
"npm://p-try:2.2.0",
"npm://@babel/highlight:7.22.20",
"npm://is-symbol:1.0.4"
]
[Info] Scanning 906 npm dependencies..

Expected behavior
The issue seems to be with the use of the 'npm ls' command. After running 'npm install' it can cause inconsistent behavior. Adding the '-prod' flag to the command seems to resolve the behavior and should result in more consistent scans.
JFrog CLI version
2.50.1
Operating system type and version
Ubuntu 20.04.6 LTS
JFrog Artifactory version
7.71.4
JFrog Xray version
3.85.5

Closing this since it seems there is a command option to scan only prod dependencies:
--dep-type             [Default: all] [npm] Defines npm dependencies type. Possible values are: all, devOnly and prodOnly
Example:
jf audit --dep-type=prodOnly

Closing this since it seems there is a command option to scan only prod dependencies:
--dep-type             [Default: all] [npm] Defines npm dependencies type. Possible values are: all, devOnly and prodOnly
Example:
jf audit --dep-type=prodOnly


Intellisense replaces correct path when manually typed

VSCode Version: Code 1.9.1 (f9d0c687ff2ea7aabd85fb9a43129117c0ecf519, 2017-02-08T23:31:51.320Z)
OS Version: Darwin x64 15.6.0 (OSX El Capitan 10.11.6)
Extensions:

Extension
Author
Version

vscode-eslint
dbaeumer
1.2.5

python
donjayamanne
0.5.8

tslint
eg2
0.8.1

docthis
joelday
0.3.11

Steps to Reproduce:

Create a project with two files: ThisProject/src/app/actions/comment.ts and ThisProject/src/app/components/CommentsPage.tsx
Add a tsconfig.json that sets "paths": { "app": ["./src/app"], "app/*": ["./src/app/*"] }
Edit CommentsPage.tsx
Type in import * as commentActions from 'app/actions/comment';

Expected: Intellisense does nothing
Actual: When I type the last ', Intellisense replaces 'app/actions/comment' with 'app/*'
http://imgur.com/a/Chw0J

Can you please share your vscode settings. I see you have "editor.autoClosingBrackets": false but I'm not able to reproduce this issue.
There appear to be a couple of possible issues:

the app/* suggestion is hanging around after typing the rest of the path. I'm not sure of the root cause here.
TypeScript should not return app/* as a module suggestion in the first place.  @mhegazy, is the tsconfig.json used here actually valid? It was unclear to me where glob patterns are supported in the paths compiler option

Looks like a typescript bug. I would port this over.

Thanks @mhegazy. I've opened https://github.com/Microsoft/TypeScript/issues/14282 to track app/* being returned as a module name in the suggestions.
I'm still interested in why the suggestion list remains showing though, and the root cause of that is probably somewhere on the VSCode side

Closing as we need more info to investigate this on the VSCode side. @andytuba please try disabling all extensions. If you stills see the suggestions lingering about while typing, share your VSCode settings and I can take a look

I updated extensions and disabled extensions and could still reproduce it.  I did notice that after disabling extensions and restarting the app, it seemed to work fine for a little bit.
settings.json:
// Place your settings in this file to overwrite the default settings
{
    "window.zoomLevel": 0,
    "editor.autoClosingBrackets": false,
    "editor.rulers": [79, 100]
,
"files.associations": {
    "*.pyx": "python"
},
"files.autoSave": "afterDelay"
}

Would it be useful to assemble a package with a miniature project and my settings?

@andytuba Yes, if you could do that it would help the investigation. Also, are you using any custom keybindings or a different keyboard layout by chance?

Closing again as we still need more info. @andytuba Please provide a small example project and your settings if you are still running into this


Support for Python 3.8

https://docs.python.org/3/library/hmac.html

Deprecated since version 3.4, will be removed in version 3.8: MD5 as implicit default digest for digestmod is deprecated. The digestmod parameter is now required. Pass it as a keyword argument to avoid awkwardness when you do not have an initial msg.

Hi,
You made a tiny mistake on line 314 in prudp.py. You're passing hashlib.md5 in the 'msg' parameter instead of 'digestmod', which doesn't work.
If you fix this I can accept your pull request.

Good catch! I changed all instances of hmac.new to use a named parameter for digestmod instead, so it cannot happen again.


Enriching docs

Improving documentation, restructuring links, adding better styling let's go!

maybe unify emoji of zoom on one side?

Merging for indexing
continue to

https://github.com/kodadot/docs/issues/93


chore: fix some slow maxHeartbeats and backwards compatability flags

This PR fixes some set_option maxHeartbeats and all set_option backwards._ situations. In these possible ways:

writing the proof in a better way
overwriting the default aesop_cat with a more efficient proof
simply removing an unnecessary tag.

This saves quite a bit of time.
closes #12534
closes #12535

!bench

The technical debt counter says that there is one more instance of set_option.*backward left. But that is in the file technical-debt-metrics.sh itself, so it doesn't really count

It's not aesop that's to blame, but it is simp. The aesop_cat calls that I've replaced all spent most of their time on normalizing the goal with simp.
The problem seems to be that definitional equality is very time consuming to check for some definitions. Additionally, the goals that simp is used on may contain large proofs that it also tries to simplify.


nixos/scion: init

Description of changes
Adds nixosModules and an extensive VM Test for the SCION internet architecture which implements a 5 autonomous system network topology based upon upstream documentation
VM test topology

Thanks to @roberth for help with figuring out how to structure this
Goals for a future PR, not this PR

module options for bootstrapping PKI similar to how openssh may generate /etc/ssh/ssh_host*
module options for connecting immediately to https://www.scionlab.org/ (probably a sensible default for scion.enable = true

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

VM test topology

Could you add this to a README in the test directory?

@roberth Done, I've added it in nixos/tests/scion/freestanding-deployment/README.rst

maybe add the test to passthru.tests on the scion package?


Feature/cred migrator

This PR introduces the wallet_migrator library crate, currently just for the purpose of migrating anoncreds related objects from their libvdrtools implementation to indy-credx. This is accomplished by creating a new Indy wallet, so the old one remains as a backup.
The migrator is flexible enough though to support migrating anything in terms of an Indy wallet.
In the future the crate could be extended to assist with migrating from an Indy wallet to a different one too.
The PR also adds a testing feature (unfortunately necessary) to accommodate current tests in aries-vcx so that wallets get migrated in the middle of the tests to ensure things keep working post migration.
Stubs are already in place for the indy-credx to anoncreds-rs migration, but that will be fully implemented in a future PR.

Codecov Report

Merging #867 (5425723) into main (1222ef7) will decrease coverage by 5.32%.
The diff coverage is 70.65%.

@@            Coverage Diff             @@
##             main     #867      +/-   ##
==========================================
- Coverage   48.96%   43.65%   -5.32%
==========================================
  Files         429      430       +1
  Lines       34225    34288      +63
  Branches     7594     7582      -12
==========================================
- Hits        16759    14967    -1792
- Misses      12208    14715    +2507
+ Partials     5258     4606     -652

Flag
Coverage Δ

unittests-aries-vcx
43.65% <70.65%> (-5.29%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

aries_vcx/src/core/profile/profile.rs
0.00% <0.00%> (ø)

aries_vcx/src/utils/devsetup.rs
39.09% <ø> (-26.46%)
:arrow_down:

aries_vcx_core/src/anoncreds/credx_anoncreds.rs
0.00% <ø> (-56.49%)
:arrow_down:

aries_vcx_core/src/wallet/indy_wallet.rs
22.44% <0.00%> (-35.72%)
:arrow_down:

libvdrtools/src/lib.rs
91.07% <ø> (ø)

libvdrtools/src/controllers/wallet.rs
47.00% <61.53%> (+11.94%)
:arrow_up:

libvdrtools/indy-wallet/src/export_import.rs
39.62% <62.06%> (+39.62%)
:arrow_up:

libvdrtools/indy-wallet/src/lib.rs
57.68% <69.04%> (+0.74%)
:arrow_up:

aries_vcx_core/src/indy/wallet.rs
35.61% <73.07%> (-2.05%)
:arrow_down:

cred_migrator/src/lib.rs
84.21% <84.21%> (ø)

... and 2 more

... and 83 files with indirect coverage changes

@Patrik-Stas The requested test scenario was added. I'll be monitoring the CI to check on the latest changes, but apart from that I think the points you mentioned in your review are covered now.


Fix #5426

Override changeSessionId from StandardSession to copy all items in old session to new in Redis layer.
This leaves intact all logic around when Session gets written to redis (Memory vs Redis and Default vs After Update) and ensures underlying cache is exactly the same as prior to ID change.

The above comments want to use a rename approach to update map to new key (which is admittedly superior). However, invoking rename at the changeSessionId level would result in same symptoms as original issue. This is because eventually setId gets invoked within StandardSession which would inform the Manager to delete and then resave the session. In the delete, the map gets deleted. I have new approach which overrides setId to do the rename.

Thanks for contribution!


feat: support comQuery request globallock hint

ref: https://github.com/cectc/dbpack/issues/104
Ⅰ. Describe what this PR did
Ⅱ. Does this pull request fix one issue?
Ⅲ. Why don't you add test cases (unit test/integration test)?
there are unit test cases
Ⅳ. Describe how to verify it
check ci results
Ⅴ. Special notes for reviews

Codecov Report

Merging #137 (1a1bce4) into dev (43cf2a7) will increase coverage by 0.46%.
The diff coverage is 61.29%.

@@            Coverage Diff             @@
##              dev     #137      +/-   ##
==========================================
+ Coverage   48.62%   49.08%   +0.46%
==========================================
  Files          62       62
  Lines        6425     6454      +29
==========================================
+ Hits         3124     3168      +44
+ Misses       3017     2995      -22
- Partials      284      291       +7

Impacted Files
Coverage Δ

pkg/filter/dt/exec/query_global_lock.go
62.85% <60.00%> (+62.85%)
:arrow_up:

pkg/filter/dt/exec/prepare_global_lock.go
58.44% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 43cf2a7...1a1bce4. Read the comment docs.

LGTM


Using [GraphQLNonNullType] Attribute on Non-String Scalars Breaks Filtering

Given the following GraphQL Query and backing type
public class QueryType
{
    [UseFiltering]
    public IEnumerable<TestObject> TestGet()
    {
        yield return new TestObject { Id = 1, Name = "Test 1" };
        yield return new TestObject { Id = 2, Name = "Test 2" };
        yield return new TestObject { Id = 3, Name = "Test 3" };
        yield return new TestObject { Id = 4, Name = "Fourth Test" };
        yield return new TestObject { Id = 5, Name = "Fifth Test" };
        yield return new TestObject { Id = 6, Name = "Sixth Test" };
        yield return new TestObject { Id = 7, Name = "More Tests" };
    }
}

public class TestObject
{
    [GraphQLNonNullType] public int? Id { get; set; }
    [GraphQLNonNullType] public string Name { get; set; }
}

Attempting to run the following query
query {
  testGet(where: {name_starts_with:"Tes"})
  {
    id,name
  }
}

Gives an field required error for every integer filter parameter:
{
  "errors": [
    {
      "message": "`id` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_gt` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_gte` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_lt` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_lte` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_not` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_not_gt` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_not_gte` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_not_lt` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    },
    {
      "message": "`id_not_lte` is a required field and cannot be null.",
      "locations": [
        {
          "line": 2,
          "column": 18
        }
      ]
    }
  ]
}

The error is not produced if only the string fields are marked with the [GraphQLNonNullType] attribute.
It appears that the [GraphQLNonNullType] attibute is being erroneously copied from the base type into the Filter type.
Reproduced in HC 10.4.3 and 11.0.0-preview.127

Hi @albanfellows
These cases are under test. Can you add create a minimal repro of the bug?
https://github.com/ChilliCream/hotchocolate/blob/master/src/HotChocolate/Filters/test/Types.Filters.Tests/FilterInputTypeTest.cs#L221-L238

https://github.com/ChilliCream/hotchocolate/issues/1222

Using the workaround described in #1222 does work.
i.e.
public class TestObject
{
    [GraphQLType(typeof(NonNullType<IntType>))]
    public int? Id { get; set; }
    [GraphQLNonNullType] public string Name { get; set; }
}

I will have a go at creating a test case.

Reproduction project at https://github.com/albanfellows/HC_Issue_1961

Hi @albanfellows
These cases are under test. Can you add create a minimal repro of the bug?
https://github.com/ChilliCream/hotchocolate/blob/master/src/HotChocolate/Filters/test/Types.Filters.Tests/FilterInputTypeTest.cs#L221-L238

If the [GraphQLNonNullType] attribute is added to either Pages or Chapters properties of the Book class the FilterAttribute_NonNullType test will fail on the members of the BookFilter input type having the wrong type (Int! instead of Int)

@albanfellows  is this still a thing in V11 filtering?

@albanfellows  is this still a thing in V11 filtering?

@albanfellows is this still a thing in V11 filtering?

@PascalSenn I can confirm that the original bug (from 10.4.3) no longer exists in 11.0.7.
The new filtering in 11.0.7 also works correctly.
I have updated the Sample project to HC11.0.7.

@albanfellows is this still a thing in V11 filtering?

@PascalSenn I can confirm that the original bug (from 10.4.3) no longer exists in 11.0.7.
The new filtering in 11.0.7 also works correctly.
I have updated the Sample project to HC11.0.7.

Very nice! Thanks for the feedback, i will close this issue in this case

Very nice! Thanks for the feedback, i will close this issue in this case


Tests failing, ports problem

Hey @rfay,
I cannot get the port and tests fixed, could you take a look?
The test is calling a curl on a localhost URL (taken from other DDEV add-ons), but it cannot reach the Swagger UI instance.
I'm not sure if I set up the ports correctly in the compose yaml.
I added multiple curl test.
# Restarted test-swagger-ui
# Your project can be reached at https://test-swagger-ui.ddev.site/ https://<IP_ADDRESS>:32778/
# Failed to execute command curl -s http://localhost:8080: exit status 7
# ddev get ddev/ddev-swagger-ui with project test-swagger-ui in /home/runner/tmp/test-swagger-ui (/home/runner/tmp/test-swagger-ui)
not ok 2 install from release

curl command outside on https://test-swagger-ui.ddev.site works in test and loading the addon in my project works fine, too.
Would be very kind if you can take a look.

Also note the ddev-addon-template instructions about how you can actually ssh into github's world, https://github.com/ddev/ddev-addon-template#how-to-debug-tests-github-actions

But the test instance port is random, right?
I just wanted to check if the swagger ui webpage is reachable and confirm swagger ui started.
Port 8080 is in addona compose yaml

What do you want to curl? You can curl the project with curl https://test-swagger-ui.ddev.site/

Thats right, but that ddev curl on localhost i found at other addons does not work with my addon, so i think something is odd or wrong configured with the ports in the compose file.

I want to curl the swagger ui webpage served by my addon.

Oh, https://github.com/reloxx13/ddev-swagger-ui/blob/1bc266dac1dd411913ce7ca52dd1a04b931a3dc6/docker-compose.swagger-ui.yaml#L3-L10
You really can't use ports normally in a docker-compose.*.yaml because then people can only run one project (they'll conflict on 8080).
Assuming swagger-ui is an HTTP service, you would do it with HTTP_EXPOSE and HTTPS_EXPOSE, like in https://github.com/ddev/ddev-phpmyadmin/blob/393b83e7f8fc586d749eeb828762a33a3a4f23ca/docker-compose.phpmyadmin.yaml#L20-L21

See https://ddev.readthedocs.io/en/latest/users/extend/custom-compose-files/

Heya,
okay i remove the port and reduced the test to only test https://test-swagger-ui.ddev.site/
Thank you very much for your hints 👍🏻


Added Hover Effect to Book Now button

Related Issue
Fixes:  #80
Description
Added Hover and Glow effect to Book Now button
Type of PR

[x] Feature enhancement

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes. Make sure to attach before & after screenshots in your PR.]
Without Hover:

With Hover:

Checklist:

[x] I have made this change from my own.

[x] My code follows the style guidelines of this project.

[x] I have performed a self-review of my own code.

[x] My changes generate no new warnings.

[x] I have tested the changes thoroughly before submitting this pull request.

[x] I have provided relevant issue numbers and screenshots after making the changes.
Please add GSSOC EXTD,Hacktoberfest and level labels to the PR

@PriyaGhosal I have resolved the conflicts

@ahinagangopadhyay Your pull request has been successfully merged! Don't stop here—continue contributing by addressing more issues and aiming to become a top contributor. Exclusive swag awaits for those who go the extra mile!
Great contribution to GSSOC'24 EXTENDED! 🎉


As a validator I want to see the list of domains that have been validated

A second tab has been added in the conformance rules report to include all domains instead of showing only 5 on the first tab. The second tab is called Dataset details and contains the following columns:

Dataset (name of the dataset: ae.xpt)
Label (long name of the dataset: Adverse Events)
Location
Modified Time Stamp (last time an update was made to the dataset
Size (in kb)
Number of records
report-template-update.xlsx

Location should just contain directory location of the file

I have validated the changes made for this acceptance criteria. I generated reports using validate and test functionality of CORE. The reports generated from the validate function passed the AC check for all test cases (i.e validation with: single dataset, multiple dataset, more than one dataset path).
However the Reports generated from the test functionality had one problem. The location for the dataset is not correctly reported in these generated reports. As can be observed in the screenshot, instead of a path to the dataset only a dot is present. This behavior was only observed in the reports generated using the test functionality.

The generated reports are also attached for reference
CORE-Report-2023-06-26T13-26-45_SingleDataset.xlsx
CORE-Report-2023-06-26T13-31-18_MultipleDatasets.xlsx
CORE-Report-2023-06-26T13-33-45_TwoTimesdpOption.xlsx
CORE-Report-2023-06-26T13-40-27_TestFunction.xlsx
CORE-Report-2023-06-26T13-47-37_TestFunction.xlsx

@RamilCDISC this is expected due to tests datasets not having a directory in the json

After the clarification, verified datasets. AC is successfully validated and pass from QA.