[REVIEW] Compressed sparse sliced/indexed setting

Closes #3115
Closes #2676
Closes #2677
This PR builds upon #3486, also porting over over from Scipy the functions necessary to set values in both major and minor axes, using integers, slices, and arrays.

Some light profiling is showing this is consistently the bottleneck in just about all the tests I've done so far: https://github.com/cupy/cupy/pull/3533/files#diff-91edca6879a46de14928e1d9107fd8adR896.

@asi1024,
With the 8.0.0 RC coming up, I'm really feeling pressure to get this PR in, along with #3486.

@asi1024 same here, do you think it would be possible to do a good thorough review on this PR so we can work on any necessary changes as quickly as possible to ensure it makes the cut into RC1?

I'm looking suggestion anyone might have on the best way to parallelize this for loop from the _insert_many function. Profiling is showing it to be the largest bottleneck so far.
        # Collate old and new in chunks by major index
        indices_parts = []
        data_parts = []
        ui, ui_indptr = cupy.unique(i, return_index=True)

        # This part is modified slightly from the Scipy counterpart
        # because Cupy doesn't not have an `append` function for
        # ndarrays.
        to_add = cupy.array([j.size], ui_indptr.dtype)
        ui_indptr = cupy.hstack([ui_indptr, to_add])

        new_nnzs = cupy.diff(ui_indptr)

        prev = 0
        for c, (ii, js, je) in enumerate(zip(ui, ui_indptr, ui_indptr[1:])):

            # append old entries for each row
            start = self.indptr[prev]
            stop = self.indptr[ii]
            indices_parts.append(self.indices[start:stop])
            data_parts.append(self.data[start:stop])

            # handle duplicate j: keep last setting
            uj, uj_indptr = cupy.unique(j[js:je][::-1], return_index=True)
            if len(uj) == je - js:
                indices_parts.append(j[js:je])
                data_parts.append(x[js:je])
            else:
                indices_parts.append(j[js:je][::-1][uj_indptr])
                data_parts.append(x[js:je][::-1][uj_indptr])
                new_nnzs[c] = len(uj)

            prev = ii

Now that I think about it, I might be able to combine i and j into a single int64 array where the high order bits are i and the low order bits are j. Then we have all the info we need in a single array and we can call unique once to get all the  unique columns per row and then do a scatter_add on those.

With the changes described above, _insert_many() is now orders of magnitude faster thank Scipy's _insert_many(). Most of the pytests are now faster end-to-end than Scipy but there are some cases where we are slower, all appearing to be completely unrelated to _insert_many(). I'm going to do more profiling of the algorithm end-to-end to better isolate those bottlenecks.

@asi1024, I have opened the following 3 PRs in addition to this one: #3657, #3658, #3639.
#3657 Includes the main scaffolding ported from Scipy but raises a NotImplementedError for all features not related to slicing.
#3658 Includes the changes from the previous PR but includes sparse row indexing
#3659 Includes the changes from #3658, but includes sparse column indexing.
This PR includes all the changes from #3659 and adds inner indexing (arrayXarray).
I built the new PRs incrementally in this way in hopes the deltas could be more easily reviewed. I will keep rolling the changes backwards into this PR so that they can be merged one after the other.
Will this approach work for you?

@leofang, if you get a moment, I'm hoping to get your thoughts on https://github.com/cupy/cupy/pull/3533/files#diff-91edca6879a46de14928e1d9107fd8adR920 (in case it doesn't expand for you it's cupyx/scipy/sparse/compressed.py line 920).
Packing 32-bit elements into a 64-bit to sort them together is a common trick but it won't work when the inputs are 64-bit indices. I just got to thinking about it with the knowledge that we are working towards support for 64-bit indices. I just haven't been able to think a better alternative to parallelize insertions. We could do this for now and raise an error if the insertions are being done on a sparse array with 64-bit index arrays, then add a better solution later when we find one, perhaps. I'm really looking for your opinion on this one.

Hi @cjnolet Sorry for my silence. I just took a quick look at _insert_many(). The bit packing trick is very fancy, but I wonder if a lexsort could be easier? Maybe I don't get the gist here...?
        keys = cupy.stack([j, i])
        order = cupy.lexsort(keys)
        new_i = i[order]
        new_j = j[order]

But I am not sure yet how to pick out the unique values.

Thanks @leofang! This works. I just needed to write a small kernel sets a filtering mast for removing all duplicates but the last one (the ones with the highest lexsort indices). This is great, because now it can fully support 64-bit indices.

Amazing work! thanks a lot 😃

btw, you have just included all the docs in the PR 😂

@emcastillo, I figured it was only a matter of time before I did that! I'll remove them from the history.

I have one more thing to do on this PR and I think it'll be ready for review:
Currently, the sort is taking most of the insertion time. Some of the recent changes enable us to now construct the output indices and data arrays in place. We can do this since the elements to construct the output arrays are already in sorted order so we no longer need 2 separate kernels for populating the existing and new values. We should be able execute a single kernel to populate the values in sorted order, thus removing the need for expensive sort.

@emcastillo, this should be ready to begin reviewing. Now that we have the scaffolding for the sparse slicing in place, I don't believe this is being blocked anymore, unless you prefer to wait for all the sparse advanced indexing PRs first.

Here's the script I've been using for benchmarking the sparse setitem:
import unittest

import cupy

from cupy import sparse
from cupy import testing

import pytest

@testing.parameterize(*testing.product({
    'format': ['csr', 'csc'],
    'density': [0.5],
    'dtype': ['float32', 'float64', 'complex64', 'complex128'],
    'n_rows': [15000],
    'n_cols': [15000]
}))
@testing.with_requires('scipy')
class TestSetitemIndexing(unittest.TestCase):

    def _run(self, maj, min=None, data=5):

        for i in range(2):
            a = cupy.sparse.random(self.n_rows, self.n_cols,
                                   format=self.format,
                                   density=self.density)

            # sparse.random doesn't support complex types
            # so we need to cast
            a = a.astype(self.dtype)

            if isinstance(maj, cupy.ndarray):
                maj_h = maj.get()
            else:
                maj_h = maj

            if isinstance(min, cupy.ndarray):
                min_h = min.get()
            else:
                min_h = min

            import time

            if min is not None:
                expected = a.get()

                cupy.cuda.Stream.null.synchronize()
                cpu_time = time.time()
                expected[maj_h, min_h] = data
                cpu_stop = time.time() - cpu_time

                actual = a

                gpu_time = time.time()
                cupy.cuda.Stream.null.synchronize()
                actual[maj, min] = data
                cupy.cuda.Stream.null.synchronize()
                gpu_stop = time.time() - gpu_time
            else:
                expected = a.get()

                cupy.cuda.Stream.null.synchronize()
                cpu_time = time.time()
                expected[maj_h] = data
                cpu_stop = time.time() - cpu_time

                actual = a

                gpu_time = time.time()
                cupy.cuda.Stream.null.synchronize()
                actual[maj] = data
                cupy.cuda.Stream.null.synchronize()
                gpu_stop = time.time() - gpu_time

        print("maj=%s, min=%s, format=%s, cpu_time=%s, gpu_time=%s"
              % (maj, min, self.format, cpu_stop, gpu_stop))

        if cupy.sparse.isspmatrix(actual):
            actual.sort_indices()
            expected.sort_indices()

            cupy.testing.assert_array_equal(
                actual.indptr, expected.indptr)
            cupy.testing.assert_array_equal(
                actual.indices, expected.indices)
            cupy.testing.assert_array_equal(
                actual.data, expected.data)

        else:

            cupy.testing.assert_array_equal(
                actual.ravel(), cupy.array(expected).ravel())

    def test_major_slice(self):
        self._run(slice(5, 10000), data=5)
        self._run(slice(5, 4), data=5)
        self._run(slice(4, 5, 2), data=5)
        self._run(slice(5, 4, -2), data=5)
        self._run(slice(2, 4), slice(0, 2), [[4], [1]])
        self._run(slice(2, 4), slice(0, 2), [[4, 5], [6, 7]])
        self._run(slice(2, 4), 0, [[4], [6]])

        self._run(slice(5, 9))
        self._run(slice(9, 5))

    def test_major_all(self):
        self._run(slice(None))

    def test_major_scalar(self):
        self._run(10)

    def test_major_fancy(self):
        self._run([1, 5, 4])
        self._run([10, 2])
        self._run([2])

    def test_major_slice_minor_slice(self):
        self._run(slice(1, 5), slice(1, 5))

    def test_major_slice_minor_all(self):
        self._run(slice(1, 5), slice(None))
        self._run(slice(5, 1), slice(None))

    def test_major_slice_minor_scalar(self):
        self._run(slice(1, 5), 5)
        self._run(slice(5, 1), 5)
        self._run(slice(5, 1, -1), 5)

    def test_major_slice_minor_fancy(self):
        self._run(slice(1, 10, 2), [1, 5, 4])

    def test_major_scalar_minor_slice(self):
        self._run(5, slice(1, 5))

    def test_major_scalar_minor_all(self):
        self._run(5, slice(None))

    def test_major_scalar_minor_scalar(self):
        self._run(5, 5)

    def test_major_scalar_minor_fancy(self):
        self._run(5, [1, 5, 4])

    def test_major_all_minor_all(self):
        self._run(slice(None), slice(None))

    def test_major_all_minor_fancy(self):
        self._run([1, 2, 3, 4, 1, 6, 1, 8, 9], [1, 5, 2, 3, 4, 5, 4, 1, 5])
        self._run(slice(None), [0, 3, 4, 1, 1, 5, 5, 2, 3, 4, 5, 4, 1, 5])

    def test_major_fancy_minor_fancy(self):
        self._run([1, 5, 4], [1, 5, 4])
        self._run([2, 0, 10], [9, 2, 1])
        self._run([2, 0], [2, 1])

    def test_major_fancy_minor_all(self):
        self._run([1, 5, 4], slice(None))

    def test_major_fancy_minor_scalar(self):
        self._run([1, 5, 4], 5)

    def test_major_fancy_minor_slice(self):
        self._run([1, 5, 4], slice(1, 5))
        self._run([1, 5, 4], slice(5, 1, -1))

    def test_major_bool_fancy(self):
        rand_bool = cupy.random.random(self.n_rows).astype(cupy.bool)
        self._run(rand_bool)

    def test_major_slice_with_step(self):

        # positive step
        self._run(slice(1, 10, 2))
        self._run(slice(2, 10, 5))
        self._run(slice(0, 10, 10))

        self._run(slice(1, None, 2))
        self._run(slice(2, None, 5))
        self._run(slice(0, None,  10))

        # negative step
        self._run(slice(10, 1, -2))
        self._run(slice(10, 2, -5))
        self._run(slice(10, 0, -10))

        self._run(slice(10, None, -2))
        self._run(slice(10, None, -5))
        self._run(slice(10, None, -10))

    def test_major_slice_with_step_minor_slice_with_step(self):

        # positive step
        self._run(slice(1, 10, 2), slice(1, 10, 2))
        self._run(slice(2, 10, 5), slice(2, 10, 5))
        self._run(slice(0, 10, 10), slice(0, 10, 10))

        # negative step
        self._run(slice(10, 1, 2), slice(10, 1, 2))
        self._run(slice(10, 2, 5), slice(10, 2, 5))
        self._run(slice(10, 0, 10), slice(10, 0, 10))

    def test_major_slice_with_step_minor_all(self):

        # positive step
        self._run(slice(1, 10, 2), slice(None))
        self._run(slice(2, 10, 5), slice(None))
        self._run(slice(0, 10, 10), slice(None))

        # negative step
        self._run(slice(10, 1, 2), slice(None))
        self._run(slice(10, 2, 5), slice(None))
        self._run(slice(10, 0, 10), slice(None))

    def test_fancy_setting(self):
        self._run(0, 0, 5)
        self._run([0, 5, 10, 2], 0, [1, 2, 3, 2])

        self._run([[True], [False], [True]], data=5)
        self._run([[True], [False], [False], [True], [True], [True]], data=5)
        self._run([True, False, False, True, True, True], data=5)

I'm running it now and will post the results.

@asi1024, here are the benchmark numbers for a 1500x1500 sparse matrix @ 50% density. As usual, I executed this benchmark on a Threadripper 16-core w/ a GV100 GPU.
setitem_benchmark_08020.zip

pfnCI, test this please

Jenkins CI test (for commit afc3bc2c7526c4cddc45f27cbb359f26e60a637c, target branch master) failed with status FAILURE.

pfnCI, test this please

Jenkins CI test (for commit ac4284c3a6b78f456e5d9e6283f009520f55a3cf, target branch master) succeeded!

Blocked by #3826

@asi1024 looking more closely at this PR, I'm not sure how much it is actually blocked by #3826.
We were told this would be included in the 8.1.0 release and back-ported to 8.0.0. Is it possible to start a review?

pfnCI, test this please

pfnCI, test this pleast

Jenkins CI test (for commit 39748b03b9bc93c11b318bd4fac326845f01d7f8, target branch master) failed with status FAILURE.

Odd the bot says this failed, but the checks below show this is passing. @asi1024 do you know what might be going on here? 🙂

pfnCI, test this please.

Jenkins CI test (for commit 31638ab0116ea106659cd866299b9a8d92d8d417, target branch master) failed with status FAILURE.

Could you fix Travis failures?

pfnCI, test this please

Jenkins CI test (for commit f3712524540eedbc2276e5d13ff9779b82bc5e00, target branch master) succeeded!

How about this comment? https://github.com/cupy/cupy/pull/3533#discussion_r486794594

Thinking through this a little more, what do you think about mapping the rows to warps, 1 row per warp, and setting the warp size based on the average degree of the CSR?
That will allow us to increase the parallelism over each column while avoiding the additional array of size nnz. What do you think?
I'm really okay with either solution. The one just described is going to require a RawKernel though, I think.

@asi1024, I removed _csr_sample_offsets and updated the code to use _csr_sample_values. Can you PTAL when you get a moment?

pfnCI, test this please

Jenkins CI test (for commit 0c2f4dd1dcac423b19156eb7f98ad44fe150af84, target branch master) succeeded!

pfnCI, test this please

Jenkins CI test (for commit 0c2f4dd1dcac423b19156eb7f98ad44fe150af84, target branch master) succeeded!

@asi1024 @

pfnCI, test this please

pfnCI, test this please

Jenkins CI test (for commit b2c7655a566285354886fd79a5899f11ad8f45c8, target branch master) succeeded!

@cjnolet Sorry for my late response.

Thinking through this a little more, what do you think about mapping the rows to warps, 1 row per warp, and setting the warp size based on the average degree of the CSR?
That will allow us to increase the parallelism over each column while avoiding the additional array of size nnz. What do you think?
I'm really okay with either solution. The one just described is going to require a RawKernel though, I think.

Now I have another idea that does not require an additional array. Can I show it in another PR?
I want to improve the parallelism of _get_arrayXarray before the improve of setiem, because _get_arrayXarray is called in setiem code path. However, we should not leave this PR unmerged for a long time.
So I plan to create another PR for the improvement after this PR is merged.

pfnCI, test this please

Jenkins CI test (for commit 69d4e59fb4b119601429fd443f2285b8c8c70601, target branch master) succeeded!

pfnCI, test this please

Jenkins CI test (for commit 9830997d6dac2645318532d7f05fe9637bcabdc2, target branch master) failed with status FAILURE.

pfnCI, test this please

Jenkins CI test (for commit 7303e30f9d694ea83f2ca1066484b1f52a3c08e0, target branch master) failed with status FAILURE.

pfnCI, test this please

pfnCI, test this please

Jenkins CI test (for commit d08c38dc6355ff76d639c9d8a83f5183314363a4, target branch master) failed with status FAILURE.

Completely unrelated test failure. That's strange.

pfnCI, test this please

Jenkins CI test (for commit d08c38dc6355ff76d639c9d8a83f5183314363a4, target branch master) succeeded!

LGTM! Thanks!

Jenkins CI test (for commit d08c38dc6355ff76d639c9d8a83f5183314363a4, target branch master) succeeded!


SpeechAPI samples and docs do not show how to pass credentials

I am trying to do some test using the Google Speech API but from outside Google Cloud. In the older beta version I was able to specify a credentials file but now I am unable to find this option in the SpeechClient class.
How can I go about specifying the authentication keys using the Google Speech API Java library? I believe a sample code should be added to demonstrate this (both passing flat credentials and using a credentials file)
P.S I already asked the same exact question on SO but 4 days later and still no one answered

Hi @redserpent7,
Thanks for reaching out!
You can authenticate using a service-account and setting the environment variable:
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
Instructions on creating a service account and setting the environment variable can be found at:
https://cloud.google.com/docs/authentication/getting-started

@redserpent7 did this help?

@frankyn thanks but I am really interested in setting the credentials path as a configuration variable. Namely I want to do the exact same thing described here but in Java

@gguuss do you know how to set credentials programmatically?

@redserpent7, I was able to set credentials using the following flow. I'd still like your input @gguuss when you have a moment.
CredentialsProvider credentialsProvider = FixedCredentialsProvider.create(ServiceAccountCredentials.fromStream(new FileInputStream("/path/to/service-account.json")));

ChannelProvider channelProvider = SpeechSettings.defaultChannelProviderBuilder().setCredentialsProvider(credentialsProvider).build();

SpeechSettings settings = SpeechSettings.defaultBuilder().setChannelProvider(channelProvider).build();

// Instantiates a client
SpeechClient speech = SpeechClient.create(settings);

@frankyn -- you're doing it correctly, in the builder you just provide an API key or service account credential. For the samples, we are using the Application default credentials because it's simpler and more secure configuration.

Thanks @gguuss! @redserpent7 let us know if this worked for you.

@frankyn Perfect thank you so much. And I hope you update the documentation to show that

Great! Let us know if anything else comes up! We will update the documentation with this information soon.

I posted this answer to your stackoverflow question as well.

Using Gax:grpc 0.21.1 I got .setCredentialsProvider marked as deprecated and my json auth that is working well when loaded with GoogleCredentials on gcloud storage throws
The Application Default Credentials are not available using the previous snippet.
Do I miss smth ?

@frankyn I'm trying your solution, but it doesn't work for me.
SpeechSettings.defaultChannelProviderBuilder is undefined
SpeechSettings.defaultBuilder is deprecated
and SpeechSettings.defaultBuilder,setChannelProvider is undefined.
Your solution is only a couple of months old, I can't image it's out of date.

@Solido I gleaned this from a comment inside of SpeechClient.Java, it works for me:
		FileInputStream credentialsStream = new FileInputStream(fileName);
		GoogleCredentials credentials = GoogleCredentials.fromStream(credentialsStream);
		FixedCredentialsProvider credentialsProvider = FixedCredentialsProvider.create(credentials);

		SpeechSettings speechSettings =
				SpeechSettings.newBuilder()
					.setCredentialsProvider(credentialsProvider)
					.build();

		SpeechClient speech = SpeechClient.create(speechSettings);

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json is not working @frankyn
I have already set env variable using the command, but client library is still throwing same Exception

@redserpent7 @Charles92011 thanks,Very useful to me


what is the 'simultaneousUploads' mains?

simultaneousUploads default is 3, or I just set the value 4, but it does not start simultaneous upload of 4 files.

Acctually, it works one by one, if 'simultaneousUploads' mains the number of chuncks upload simultaneous, what should I do that I can start 4 files upload at a time

The simultaneousUploads refers to how many parallel uploads are happening, but in terms of chunks, not files. The default, 3, means we will attempt to upload 3 chunks at a time, but all chunks from a single file will be uploaded before the next file is attempted.
If you want to do 4 files at a time, the easiest way is to start create 4 resumable instances and upload a file with each.


Write down a user guide on how to use Kyma Companion

Ask documentation team to test guide and best-practices on using the companion, provide their observations, and suggest improvements


schedulers: adjust hot region pending influence

Signed-off-by: nolouch<EMAIL_ADDRESS>
What problem does this PR solve?
The store's heartbeat interval is short, no need to use a complicated way.
What is changed and how it works?
remove the liner estimation.
Release note
Check List
Tests

Unit test

/rebuild


[MM-35946] Remove memoization to prevent errors on detecting unread/mention state

Summary
Remove memoization as a possible cause for not removing the unread/mention mark
Also this can't be cherry-picked directly, so once we merge this we'll need to do a separate PR to master replicating.
Ticket Link
MM-35946
Device Information
This PR was tested on: windows 10, MacOS X
I haven't been able to reproduce with this build, but this doesn't mean it might still happen if there are other causes, so I'd like to have as much feedback as possible on this one.
Release Note

Fix badge not updating properly on some setups

@coltoneshaw I added you as a reviewer to test this PR that fixes https://mattermost.atlassian.net/browse/MM-35946. This will be included in a v4.7 dot release. Guillermo was looking for an extra set of eyes for testing the bug fix.

@Willyfrog @esethna Do we want to share this with the affected customer for testing, or is this easier for them to test on RC-1?

if the customer repros consistently, then yes, as I have a hard time reproducing the problem.
the only downside is that they might not update to the dot release afterwards

@Willyfrog Colton asked about test builds in the ticket https://mattermost.atlassian.net/browse/MM-35946?focusedCommentId=112852.

@Willyfrog Colton asked about test builds in the ticket https://mattermost.atlassian.net/browse/MM-35946?focusedCommentId=112852.

thanks! I'll contact him to provide them

if anyone else wants to test, builds are in https://community-daily.mattermost.com/core/pl/kfxenwm3e7n47dsur7yz46buwc

@Willyfrog Have you received any feedback from testers yet?

@Willyfrog Have you received any feedback from testers yet?

partial only, I'm still waiting on a bit more feedback.

feedback received and it was all good, so merging

Cherry pick is scheduled.

Error trying doing the automated Cherry picking. Please do this manually
fatal: No such remote 'upstream'

+++ Returning you to the master branch and cleaning up.

cherry picked directly into master by mistake, will revert if it causes any errors


Draft: Fix CI pipeline

Just trying to get this to work... Feel free to close and delete if we don't really need the CI

Oops. Accidentally merged into upstream instead of my fork


Add logic to change logging location

Adds process that:

Creates /var/log/forescout
Conditionally-nuke /usr/lib/forescout/bin/log (condition: if already exists and is a directory)
Creates a new /usr/lib/forescout/bin/log as symlink pointing to /var/log/forescout

Closes #11

Presumably moving the state-blocks to the desired file will still allow the blocks'  existing require statements to function "as is" (since SaltStack will merge the files, first)?
@lorengordon

I think I'd like to move these additions to a config state instead of overloading the package state. The directory structure would be forescout-secure-connector/config, with the following files:

Presumably moving the state-blocks to the desired file will still allow the blocks' existing require statements to function "as is" (since SaltStack will merge the files, first)?

The include block uses explicit list-ordering for execution. So it will run the states in the order you specify. I presume it will be .package, then .config, then .service...


Black World

Describe the bug
When I Enter Work Every Thing Is Rendered, Black
Expected behavior
Everything Should Be Normal
Observed/actual behavior
World Renders Black And White
Steps To Reproduce
Launch Minecraft
Create A New World
Enter The World
The World Is Dim( If Resource Pack Used World Is Black )
Modpack version
alpha 2 3.0.0
Launcher
TLauncher
Install method
Fresh install/new profile
Additional context
with resource packs

without resource packs

If Resource Pack Used World Is Black

I cannot fix this, it most be done by the pack creator.
https://github.com/Fabulously-Optimized/fabulously-optimized/wiki/Resource-pack-issues

but without resource pack the sam happens

ok

I had a similar issue.
It was caused by Lithium I think.
Restarting the game fixed it.
Maybe you need to update your gaphical drivers.
Hope I could help!

so i wouldn't be playing Minecraft now cause my graphics card is out of support

or removing lithium could fix this

launcher log
tlauncher_12_12_2021_16_32_42.log

log
latest.log

I downloaded a fresh/untouched FO 1.18 zip, and I had the same black world issue. It's not because of any settings. I'm sure it is caused by Lithium!!

Ah, in that case you both can use 3.1.0-alpha.1, which does not have Lithium right now.
But in the long term it would be helpful if you tested it with only Lithium enabled, tried out the development versions of it and reported the bug to them with your system details. The Discord is: https://caffeinemc.net/discord

I tried it without Lithium (and the issue was fixed) but we really need a good replacement! (Starlight?)

Nah, Lithium is good for most users due to it's integration and testing with other CaffeineMC mods. It is rather an issue that should be reported and fixed.

you're right!

Nah, Lithium is good for most users due to its integration and testing with other CaffeineMC mods. It is rather an issue that should be reported and fixed.

https://github.com/CaffeineMC/lithium-fabric/issues/312

Yeah.. but you should still test and report crashes with only Lithium and also Lithium's 1.18.1 dev version (in the discord I linked above) to make sure it is caused by just that.

@Madis0 Maybe it's because of the version?

1.18 Fabulously.Optimized.MultiMC.3.0.0-alpha.2: 0.7.6-rc1
1.17.1 Fabulously.Optimized.MultiMC.2.5.0 - 0.7.5

Yeah.. but you should still test and report crashes with only Lithium and also Lithium's 1.18.1 dev version (in the discord I linked above) to make sure it is caused by just that.

alright

ONLY lithium seems to work fine on FO 1.18

Okay, in that case it could be a mod conflict, so disabling mods one by one would be preferred.
But more specifically I suggest testing 3.1.0-alpha.1 (1.18.1) with the in-development Lithium and

disabling Colormatic
disabling Sodium

Okay.

But Sodium doesn't make sense, it's the same dev xD

Same dev team, yes, but well.. both Sodium and Lithium are clearly alpha at this point, so you never know :D

I just installed the newest FO 1.18.1 alpha modpack

seems to work fine

3.1.0-alpha.1 lacks Lithium though, hence I suggested using the dev version.

so, should I add Lithium in my 1.18.1 FO instance?

Correct.

I can't manage to find it...

Where's the download?

https://discord.com/channels/602796788608401408/886002128844374146/919214269172703283

Thanks!

Seems to work fine, I guess

with F3

Apparently, the problem only seems to occur on SOME worlds.

Apparently, on some worlds, it seems to work fine

Here's a (rare) example of the issue

On the broken worlds, the issue perists when using a

Perhaps shaders are the actual issue? Or you could test with only Lithium, Sodium, Iris.

Even after optimizing the broken world, the issue keeps occuring

I loaded the exact same broken world with vanilla, the problem still exists

some more screenshots of vanilla Minecraft with a broken shadow-world

By the way, my skin/hand is not broken, it's just the way my skin is built with overlays/etc.

some more vanilla screenshots with the broken world

It doesn't have to do with the seed. I re-created the broken world's seed on vanilla

FOUND A FIX!

Erase cached data and optimize the world (I know that I'm doing) fixed it for me!

Seems like everyone involved got it fixed, closing the issue.

Its fixed in multiplayer but the problem still persists in singleplayer mode

Optimizing the world + deleting caches didn't work?

i have also made a new world and checked it still continues

but still let me try to optimize it again

Did you try the local worlds without the resource pack?

yes

but still let me try to optimize it again

I optimized with erase cached data but still no progress

Alright.
@Rajbir98, please try a clean install of 3.1.0-alpha.3 - no extra mods, resource packs or setting changes. Then enable launcher logs, create a world and try to load some chunks. Then post your launcher log to Pastebin.

problem solved thx for guiding me


dart:ffi asFunction / lookupFunction static function vs instance method

Currently Pointer.asFunction and DynamicLibrary.lookup are instance methods.
However, this enables users to call these dynamically.
Pointer<Int8 Function(Int8)> p = fromAddress(memoryAddress);
dynamic q = p;
int Function(int) f = q.asFunction();
f(123);

Unfortunately we cannot at compile-time discover this, which means that we cannot generate trampolines in AOT mode.
We have two possible solutions:

throw a runtime exception on a dynamic invocation
make these functions static: f = asFunction(q)

The second option is a bit less ergonomic, but safer.

@lrhn @mraleph

Let it crash? Or throw in asFunction if the function is not available?
If this was done using static extension types instead then dynamic invocation would not work. Much of the ffi functionality is compiler magic that works similar to this, so I'd not be worried if it just doesn't work when used through dynamic. It's already so very magical.
The trampoline you create, is it for a particular function, or can it be reused for multiple functions with the same type, just with different native addresses? If it was the latter, you could create it when you see Pointer<Int8 Function(Int8)>.

We will keep using instance methods for now and use extension methods once available, to ease migration. (Throwing an exception on a dynamic call).

This should be addressed now that we have extension methods.


graphql: handle unknown property in MapFilter

Handling unknown GraphQL filter property in MapFilter func.
It is required to fallback into selector filtering when the specified property is not available in the search response.
E.g. ?filter=(eq,location,US) --> should filter using the selector.

/lgtm

rebased

New changes are detected. LGTM label has been removed.

handled rebase conficts.


please add alignment and direction support for this great package

please add alignment and direction support for this great package

We did add alignment, see https://github.com/singerdmx/flutter-quill/blob/master/lib/widgets/text_line.dart#L64
But it seems not working. Still trying to figure this out.
Need help if someone can contribute a pull request if possible.

We did add alignment, see https://github.com/singerdmx/flutter-quill/blob/master/lib/widgets/text_line.dart#L64
But it seems not working. Still trying to figure this out.
Need help if someone can contribute a pull request if possible.

Actually found the fix https://github.com/singerdmx/flutter-quill/commit/486b823d35df31b5c9856dbb2c9ccf757ad087a6
Need to add toolbar button next.
Then figure out the text direction.

Actually found the fix https://github.com/singerdmx/flutter-quill/commit/486b823d35df31b5c9856dbb2c9ccf757ad087a6
Need to add toolbar button next.
Then figure out the text direction.

@T-Shamspour We currently don't have plan regarding text direction.
It seems a bare use.
But if you need it, you can submit a pull request.

@T-Shamspour We currently don't have plan regarding text direction.
It seems a bare use.
But if you need it, you can submit a pull request.

No plan to support alignment. Dropping it

alignment is important, please consider to support it @singerdmx

Hi, is there any chance this feature can be be considered again?
This could be very helpful fur me and many others.
without this feature bullet lists and numerical lists are on the opposite side of the text

I would also like to have this feature, to build an editor for RTL languages. @singerdmx why aren't you considering it? it would be very useful for devs in the other parts of the world.
There are 12 languages that are written from the right to the left:
Arabic
Aramaic
Azeri
Divehi
Fula
Hebrew
Kurdish
N'ko
Persian
Rohingya
Syriac
Urdu

You can submit PR

This is the last effort we are going to spend on this
https://github.com/singerdmx/flutter-quill/commit/4f9ee27f7dc8baf64d36784d4d74c6feb1e18ece
If it has bugs, please figure it out and submit fix


plata.shop.actions.export_as_csv_action: Point users to xlsxdocuments' export_select instead?

https://pypi.org/project/xlsxdocument/
Opinions?
cc @fiee

Never tried that. I’m only using https://pypi.org/project/xlwt/ in two projects, but it’s outdated.

Less code means less to test, less to maintain.
And it's not really about adding features, but about replacing code that's being used in a single place with code that's used in many places.


fix: resolve compile errors when feature optimism is on

Steps to verify

Go to crates/optimism/bin/Cargo.toml, enable feature optimism:

-default = ["jemalloc"]
+default = ["jemalloc", "optimism"]

Run cargo clippy or cargo build --package op-reth

@kien-rise We should focus on block-building for the first pevm integration into RISE Stack so let's not base on this branch. I meant to create a new branch without realizing the name conflict with this existing one...
So basically, let's try our best to not use a Reth fork in the RISE Stack. If we must, let's create a new branch with minimal changes to support the pevm block builder first.

So basically, let's try our best to not use a Reth fork in the RISE Stack. If we must, let's create a new branch with minimal changes to support the pevm block builder first.

Alright, will use this reth fork: https://github.com/risechain/reth/commits/pevm-integration/
Future PRs (if any) will target this branch (pevm-integration).
Note that the "minimal changes" is not so small. As long as we use our modified revm, the code changes on reth is inevitable.


Add a NodeMapping that automatically maps a role to the name used by kubelet

kubelet reports itself as system:node:
modifies authenticator to be able to discover an ec2 instance's private dns
and use it for the node name when the role is part of a NodeMapping
allows a role to be specified in the config file to assume for the describe
instances call, in case it is in a separate account.

Addresses issues/57

A couple fixes in progress:

Get region from metadata if not specified in config.
Allow assume role per mapNodes in case of multiple node groups in separate accounts.

Not going to add a fix for node groups in multiple separate accounts unless someone asks for it, since the aws cloudprovider can only currently create loadbalancers in a single separate account.

/cc @rtripat @saranbalaji90

Per conversation with @rtripat, we think if the mapping includes a system:node, system:bootstrappers or system:proxier group, or a username of "system:nodes:*",  the authenticator should validate at least that the InstanceId passed in the UserId of the sts GetCallerIdentity request, which is settable by the user, is an InstanceId that belongs to an instance in the node account by calling ec2 DescribeInstances.  This will help to prevent users from impersonating nodes

Squashed commits and addressed @rtripat last feedback.

Closed in favor of https://github.com/heptio/authenticator/pull/65


BCB6 in fact and literally unfriendly

[C++ Error] tinyxml2.h(1905): E2247 'XMLDocument::PushDepth()' is not accessible
[C++ Error] tinyxml2.h(1908): E2247 'XMLDocument::PopDepth()' is not accessible
Workaround:
Add this
before void PushDepth();
i.e.
...
friend class DepthTracker;
void PushDepth();
... and all be fine.

Pull request?


Info results returned from git showing as error in build step

Hi,
My build definition has a step which executes a powershell script (snippet below). The script pushes the code into GitHub. It's all working perfectly, however the output produced by calls to git are being displayed in the log as errors (see attached screenshot). This also means the build failed even though everything worked as expected.
I found the same questions being asked here but no solution was found. As you can see I have tried " | Write-Host" and "exit 0" but neither of these work.
Is there a way to have these return messages resolved as info and not considered to be errors?
param(
    [string] $GitRepoUrl
)

Write-Host "GitRepoUrl: $GitRepoUrl"

Set-Location $Env:BUILD_SOURCESDIRECTORY

Write-Host ">>git init"
git init | Write-Host

Write-Host ">>git remote add origin $GitRepoUrl"
git remote add origin $GitRepoUrl | Write-Host

Write-Host ">>Start-SshAgent -Quiet"
Start-SshAgent -Quiet | Write-Host

Write-Host ">>ssh-add c:\git\id_rsa.pub"
ssh-add c:\git\id_rsa.pub | Write-Host

Write-Host ">>ssh -T<EMAIL_ADDRESS>ssh -T<EMAIL_ADDRESS>| Write-Host

Write-Host ">>ssh -Tv<EMAIL_ADDRESS>ssh -Tv<EMAIL_ADDRESS>| Write-Host

You need to redirect stderr to stdout for calls that you know result in non error case stderr output (git writes progress, status to stderr).
Does this help?
https://stackoverflow.com/questions/8925323/redirection-of-standard-and-error-output-appending-to-the-same-log-file

@patnolan Any luck with this?

Hi @bryanmacfarlane thanks for checking in.
After a lot of mucking around and trying heaps of different options, the only option that seemed to work was using the following syntax to pipe to file
git commit --message "Build $Env:BUILDVERSION" *>> $Env:BUILD_SOURCESDIRECTORY\CommitToGit_output.txt


Strings 16 characters or longer are improperly inserted into MS Access Database

Observed Behavior:
I'm trying to insert data into an MS Access .mdb file.  I'm on Linux using unixODBC with the commercial Easysoft ODBC-Access driver.  When I try to insert strings into a text column in the database if the strings are longer than 15 characters then instead of them being inserted normally the are inserted with a NULL character between each of the characters that should be there.
When I try the same thing using the unixODBC utility isql the strings are inserted correctly.
Environment:

Python: 3.4
pyodbc: 4.0.21
OS: Ubuntu Linux 14.04
DB: MS Access
driver: Easysoft ODBC-Access

To Reproduce:
import pyodbc

test_values = [(''), ('abcdefghijklmno'), ('abcdefghijklmnop')]

print('Version: {}'.format(pyodbc.version))

create_statement = """
CREATE TABLE test_text (
    cola TEXT
);
"""

insert_statement = """
INSERT INTO test_text
VALUES (?)
;
"""

select_statement = """
SELECT
    cola,
    char_length(cola),
    length(cola)
FROM
    test_text
;
"""

drop_statement = """
DROP TABLE test_text
;
"""

with pyodbc.connect('dsn=emptymdb') as conn:
with conn.cursor() as cur:
        cur.execute(create_statement)
        for string in test_values:
	cur.execute(insert_statement, string)
        cur.execute(select_statement)
        for row in cur.fetchall():
	print(row)
        #cur.execute(drop_statement)

Output from above python code:
Version: 4.0.21
('', 0, 0)
('abcdefghijklmno', 15, 15)
('a\x00b\x00c\x00d\x00e\x00f\x00g\x00h\x00i\x00j\x00k\x00l\x00m\x00n\x00o\x00p\x00', 32, 1)

Viewing this in isql:
SQL> insert into test_text values ('0123456789abcdefghijklmnopqrstuv');
SQL> select cola, char_length(cola), length(cola) from test_text;

-----------------------------------+------------+------------+
| COLA                             |char_length |length      |
-----------------------------------+------------+------------+
|                                  | 0          | 0          |
| abcdefghijklmno                  | 15         | 15         |
| a                                | 32         | 1          |
|<PHONE_NUMBER>abcdefghijklmnopqrstuv | 32         | 32         |
-----------------------------------+------------+------------+

This appears to be related to character encoding (UTF-16 being interpreted as UTF-8 or other single-byte encoding). Can you provide an ODBC trace?

Here is an ODBC trace of the above test case.
odbc.log

From the correspondence I've had with the driver vendor I think it may be an issue with the difference between UCS2 and UCS4 builds of Python.  I'm not really sure though.

Can you try a newer pyODBC (e.g. .22 or .23)?

Can you try with 4.0.24, which was released today with fix for Unicode?  I don't have that driver.

I will try with 4.0.24.  I won't be able to work on this till next week but its still on my todo.

Closed due to inactivity. Feel free to re-open with current information if necessary.


concmd console command missing when rejoining servers

I have noticed on some occurrences that when I join servers, the command "wire_expression2_concmd" is present, but once I leave a server and then rejoin the command is missing. And because the command is missing, you can not run concmd() in expression 2 as the function cant verify if the user has concmd enabled or not.

Yep, I have noticed issues like this (not this specific one, but similar ones, because I never use concmd and have it always set to 0). I'm 99% sure this is caused by the fact that clientside E2 extension files used to be loaded only after you first open the E2 editor for the first time each session. And as you can see here
https://github.com/wiremod/wire/blob/master/lua/entities/gmod_wire_expression2/core/cl_console.lua
this console command is created on the client in one such file.
However, pretty recently, I fixed this https://github.com/wiremod/wire/commit/f374d5bbcef78667aaf4a985f50a9553e3c98112 so you'll need to tell the owner of the server you play on to update wiremod to the latest version to fix this. Unfortunately, updating your wiremod on your own computer won't help because the server's version overrides yours while you play on said server.
Until then, you'll simply have to open the E2 editor (and then close it again or whatever) every time you want to use console commands.

Ah I see now, thanks for the reply


Feature/Doc Request: zarf.yaml currently requires docker.io/ prefix to be specified for images from docker hub

Short Description of Minor Issue:
I'm investigating tooling / workflow for an Ephemeral Transport Registry (for air gap registry imports with a good UX) and Zarf looks like a great candidate, I discovered If I have this:
kind: ZarfPackageConfig
metadata:
  name: ephemeral-transport-registry
components:
- name: images
  required: true
  images:
  - docker.io/busybox
  - docker.io/rancher/backup-restore-operator:v1.2.0
  - rancher/cis-operator:v1.0.6
  - rancher/configmap-reload:v0.3.0-rancher4
  - rancher/coredns-coredns:1.6.2

Then only the docker.io/... will get uploaded
and the imperative url will be ignored
(so only 2/5 images will get loaded into zarf's registry)

So my feature request / ask is for either:

zarf cli to get smart enough to add docker.io/ prefix to images from docker hub
or document the limitation / requirement that images use full domain names.

The following is additional details of an interesting use case / example usage:
What I'm trying to do is as follows:

generate an arbitrary images.txt list of container images
imperatively generate a declarative zarf.yaml config from images.txt
use zarf to declaratively populate an ephemeral registry

# provision fresh centos7 and ssh in

# become root
sudo su -

# install zarf 0.13.3 and zarf's dependencies artifact tar
cd ~
curl -L https://zarf-public.s3-us-gov-west-1.amazonaws.com/release/v0.13.3/zarf > zarf-cli
chmod +x zarf-cli
sudo mv zarf-cli /usr/bin/zarf
curl -L https://zarf-public.s3-us-gov-west-1.amazonaws.com/release/v0.13.3/zarf-init.tar.zst > zarf-init.tar.zst #538mb, takes about 1m20sec

# init zarf
cd ~
zarf init --components=management --host=<IP_ADDRESS> --confirm

# wait for zarf registry to become healthy
time while [[ $(kubectl get pods -l app=docker-registry -n=registry -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "waiting for registry to be ready (about 2 mins)..." && sleep 5; done; echo "Registry is ready"

# Create test data (that won't require registry auth for the sake of convenience)
cat > images.txt <<EOF
docker.io/busybox
docker.io/rancher/backup-restore-operator:v1.2.0
rancher/cis-operator:v1.0.6
rancher/configmap-reload:v0.3.0-rancher4
rancher/coredns-coredns:1.6.2
EOF

# If images in images.txt required auth could use this commented out stuff
# Pre-login to Authenticated Registry
# export REGISTRY1_USERNAME=REPLACE_ME
# export REGISTRY1_PASSWORD=REPLACE_ME
# zarf tools registry login registry1.dso.mil -u $REGISTRY1_USERNAME -p $REGISTRY1_PASSWORD

# Imperatively generate declarative zarf.yaml config from images.txt
TEMPLATIZED_IMAGES_LIST=$(cat images.txt | sed 's/^/  - /' )
cat > zarf.yaml <<EOF
kind: ZarfPackageConfig
metadata:
  name: ephemeral-transport-registry
components:
- name: images
  required: true
  images:
$TEMPLATIZED_IMAGES_LIST
EOF

# Eyeball validation of zarf config
head zarf.yaml

time zarf package create --confirm
time zarf package deploy zarf-package-ephemeral-transport-registry.tar.zst --confirm
# gives info about the 2 that worked, warn about the 3 the failed

# verify contents of registry
zarf tools registry catalog <IP_ADDRESS>

I think this is a duplicate of #44.  This is currently fixed in a PR that's turned into a giant PR so I'm currently breaking it down into a smaller one to get this fix among others in sooner than later.


add docs for arr.$remove and arr.$set

In http://vuejs.org/api/ , there is no api for arr.$remove and arr.$set.
But they are add by this file: src/observer/array.js
When I saw $remove usage in some example code, I feel confused but api page dont see anything.

It's here: http://vuejs.org/guide/list.html#Caveats

I know guide page has it. But it also belong to api page.

I know guide page has it. But it also belong to api page.

Moved to vuejs.org repo: https://github.com/vuejs/vuejs.org/issues/265


Session Killed when Accessing /login in browser

Dear All
I am new comer in laravel framework.
I am trying to create authentication in laravel.
I am using php artisan make:auth to generate authentication automatically.
I am happy when everything going well.
However, when i have been login and i access login page with typing 'localhost:8000/login', i still access my login page and my session destroyed.
My problem, why i can access my login page when i have been login ?
Anyone here could fix the problem i have ?
(Sorry for my bad english and my bad grammar)
Thnks.....

Please direct questions to the forums.


How to prevent an element in the enqueue come from the same sample as the query?

How to prevent an element in the enqueue come from the same sample as the query， especially when the dataloader‘s param "shuffle" is True?
Thank you
https://github.com/HobbitLong/CMC/blob/783bf95af490f0977028c33529e979fa55b9cfff/NCE/NCEAverage.py#L178

Hi, @HebutAIer ,
That case can only happen at the beginning of each epoch. It's pretty sparse in general.
It should not affect the performance much if your dataset is big enough, e.g., ImageNet. But if your dataset is small, you might consider customizing your dataloader a bit to avoid it.


Accept associations as inputs for array filters

This pull request will fix #301.

The specs are failing because the specs are using eql instead of eq. We started doing that in e454f1ee2c1ce1743c73fed275a100db1446816d. Collection proxies aren't eql to arrays, which is the root cause of #301.
Since we didn't really switch to eql for any particular reason, I think I'll change them over.

Hmm, now the build is failing because sqlite3 won't build on JRuby.

Can we just mock instead?

We can! That worked well. Can you review these changes?

:+1:


Select NOT exported assets

When exporting, you have an option to select what assets are exported. I was in a situation when I wanted to do the opposite: I had a resource directory that was supposed to be dynamic i.e. not part of a project, but loaded as external resource. For convenience it was located inside project directory for testing purposes, but each time I wanted to export the project, I had to move that directory out so it's not included.
Now, I could select assets that get imported, but it was terribly annoying, because every time I added something new to the actual project, I had to select it for export, otherwise the exported game was crashing when I forgot to do that. So there should be an option to EXCLUDE some files, I mean particular files (or directories, when this option finally gets added to 3.x), just like we have option to include only particular files.

it seems similar issue for 2.1 #3646
there also was similar issue for 3.x #8461
and it's fixed by 85945e4367c6c0759f855a3dfefe5941c3c07cd3
do these help for your issue?

OMG, the .gdignore file fixes the issue.
Good to know that something like this already exists.


tools/ci: Skip copy gn temporary files

Summary
Save the docker image size
Impact
Dockerfile
Testing
ci

ci fail to update the docker image:
https://github.com/apache/nuttx/actions/runs/7137890244


[Question] can you tell me what is these text

Minecraft Version

[ ] 1.12.2 (End of support)
[ ] 1.16.5 (End of support)
[ ] 1.18.2
[ ] 1.19.2
[X] 1.20/1.20.1
[ ] 1.21/1.21.1

What happened?
When i right click any solid block and right click the maid again there is a flowing text "maid big container" rendered on the block, it looks buggy and i want to know how can i reproduce it with clean instance so i can find the problematic compatiblity problem

Relevant log output
No response
Contact Details
<EMAIL_ADDRESS>

I have never encountered such a problem, and this is the first time I have received similar feedback
I am afraid this is not a problem with the maid mod

How do you make those text apear ingame with just maid mod? I want to atleast knowing what could i do to let it apear first so i can look into it

closing as this is not related to touhou maid mod

okay, it is weird behavior with Chest Tracker mod, go to its config and disable container names fixes it


How can I use astream_chat method with chatEngine,

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
How can I use astream_chat method with chatEngine, How can I use it with API (fastapi) to stream my response to there where it should be asynchronous.

How can I yield response from this function to the api


Remove margin from header element

Fixes https://github.com/VEuPathDB/web-eda/issues/1172
(addresses https://github.com/VEuPathDB/web-eda/issues/1172#issuecomment-1315433250)
For some reason I wasn't able to see the extra margin noted in that comment---it only appears on QA, not my dev site

There might be a global style rule for h5 that isn't included in local dev sites.


Volume module

The volume module is now complete and stable enough that I believe it warrants opening a PR. The code is a bit rough in places so I'm going to refactor it over the next couple of weeks before it gets merged in. This hopefully also gives a chance for people to have a poke at it.
The initial version of the module provides the following:

A button on the bar which indicates volume level by icon
A popup with controls for output device switching, output device volume changing, and individual output stream (ie per program) level controls.
Support for Pulseaudio (inc pipewire-pulse).

To do still:

[x] Fix a few edge cases (such as popup state when nothing is playing)
[x] Add name and class elements to widgets
[x] Ability to show more on the bar (such as current vol level)
[x] More options
[x] Write documentation
[x] Update example config & styles

Resolves #50

I'm thinking of adding an audio device switcher using pactl subscribe to track events. Will wait for 1st cut of volume module to be merged. I want to add a way to show current audio device (headphone/speaker etc, next to volume icon.) with a way to customize the icons shown based on regex.

It sounds like this may be most of the way to that already. The popup supports device switching, and you can display the current device name next to the icon on the bar.
The only part of that is missing is regex support. Generally I'm a little cautious of adding regex in as it can be a bit heavy-handed, and less intuitive to configure. That doesn't mean it's out the question, but I think it'd be worth opening an issue to discuss what you're after before jumping into anything.


Crash inside alsa library when running example

I tried running the blocking api example [1] in a test project and it crashed. It crashes inside an alsa lib function after calling stream.get_stream_write_available in the example code. This is under Gentoo linux x86_64 with Rust 1.0 stable and rust-portaudio 0.4.17
output and crash dump here:
https://gist.github.com/darkstalker/d8e41c0fa6686ff290f4
[1] https://github.com/jeremyletang/rust-portaudio/blob/master/examples/blocking.rs

Closing as it is old issue and dump is no longer present.

corrected link of the dump (username change)
https://gist.github.com/wolfiestyle/d8e41c0fa6686ff290f4


Show history button doesn't work on big screen

Is this a chomecast receiver bug or literally a large screen issue?

It's an issue when chromecast is enabled but not necessarily if it's casting. I think it might have something to do with the fact that the 'start/stop casting' button overcasts the 'show history' button?


Feature Request: Option for output diff to be in landscape mode as opposed to portrait

Right now the diff image is outputted as top middle and bottom but it would be really useful if there was an option to also output it as left middle and right instead.

Just realized this was already an option. My B


[Feat] - launch to camera directly

is that possible to intent to camera after click some UI button and return the image to crop activity?
(can the start picker chahge to optional function?)

Hey @Hawksbillcat welcome to the library.
Good news, is possible ^^. We have many ways of doing this, the simple one is:
private val cropImage = registerForActivityResult(CropImageContract()) {
    // deal with result.
}

// code ... code

// on you UI click call
private fun startCrop() {
    cropImage.launch(
        options {
            ...
        }
    )
}

In the above case we let the library do everything.
But if you want to have only camera (without gallery) you can have this code:
private val cropImage = registerForActivityResult(CropImageContract()) {
    // deal with result.
}
private val pickImage = registerForActivityResult(PickImageContract()) { photoUri ->
    cropImage.launch(
        options(photoUri) {
            ...
        }
    )
}

// code ... code

// start camera
private fun startPickImage() {
    pickImage.launch(PickImageContractOptions(includeGallery = false))
}

All this code you can find at crop image sample file.
We have a java one too

Will move this to discussion as a Q&A


Documentation sections of Standard.Database.Data.Table.Table.join are not parsed correctly

This task is automatically imported from the old Task Issue Board and it was originally created by Mateusz Czapliński.
Original issue is here.

Steps to reproduce

Start ./run ide watch.
Open the built IDE in Chrome.
Open the Chrome Developer Console, go to the "Network" tab.
Create a new project in the IDE.
In the "Name" panel of the Dev. Console, find the 2nd (of 3) entry named "<IP_ADDRESS>" and click it.
In the right panel of the Dev. Console, click "Messages".
In the "Enter regex (...)" input, enter the following expression: Standard.Database.Data.Table.Table.*Efficiently joins. This should filter out just a single search/suggestionsDatabaseUpdates response in the "Data" panel.
Right-click the search/suggestionsDatabaseUpdates response, select "Copy message".
Paste the copied JSON message body into a text file.
In the JSON, search for documentationSections of the join method.

Expected
There should be a separate "type":"keyed" section with "key":"Arguments".
Actual
The contents of the "Arguments" section in the documentation is contained in HTML format in "type":"paragraph" entry in documentationSections:
        "documentationSections":[{
          "type":"tag",
          "name":"UNSTABLE",
          "body":""
        },
        {
          "type":"paragraph",
          "body":"
Efficiently joins two tables based on either the index or a key column.
Arguments:
- other: the table being the right operand of this join operation.
- on: the column(s) or expression(s) of `self` that should be used as
  the join key. If this argument is not provided,
           the index of `self`
  will be used.
- drop_unmatched: whether the rows of `self` without corresponding
  matches in `other` should be dropped from the result.
- left_suffix: a suffix that should be added to the columns of `self`
  when there's a name conflict with a column of `other`.
- right_suffix: a suffix that should be added to the columns of `other`
  when there's a name conflict with a column of `self`.
The resulting table contains rows of `self` extended with rows of
`other` with matching indexes. If the index in `other` is not unique,

the corresponding rows of `self` will be duplicated in the result.

"
        }]

Additional Notes

The documentation source: https://github.com/enso-org/enso/blob/7e2998bd27964b888c4f9dd03c7d99fe9694edfd/distribution/lib/Standard/Database/0.0.0-dev/src/Data/Table.enso#L522-L542
Note: this issue was found while working on #182584311 and trying to add an Icon: join section in the docs of the join method. This "Icon" section got also incorrectly merged into the "paragraph" section.
A similar block in distribution/lib/Standard/Table/0.0.0-dev/src/Data/Column.enso, which does not have an UNSTABLE tag, seems parsed properly into sections.

Blockers:
#183417662 resolved
Comments:
Update
The parser parses the docstring with an extra indent
"documentation":" UNSTABLE\n\n     Efficiently joins two...

Unstable line is parsed correctly (one space), but the next lines contain an extra indent (more than one space)
Then docparser treats that extra indent as a code block (Dmitry Bushev - Aug 1, 2022)

The new parser should be ready in a month form now. We can postpone fixing an old one, and just wait for the new implementation. (Dmitry Bushev - Aug 1, 2022)

We will not fix the old parser


Bonfire Slasher Flick

This bonfire in my opinion is too easy and could easily be a waypoint.

All "blank page" type challenges will always be bonfires and not everyone finds it equally easy. Thanks and happy coding!


HUD positioning

Anchor verb menu and inventory to the bottom left and right, respectively, to make the HUD less dependent on screen sizes.
The inventory takes up too much screen space in room2 and needs adjusting.

The issue with dialog appearing behind items should now be fixed by moving dialog_player into the dialog CanvasLayer in main.tscn, instead of its previous placement in game.tscn/game_am.tscn, where it was not in a CanvasLayer. It could have been placed within a CanvasLayer in game.tscn/game_am.tscn too, but I felt it made sense to have both kinds of dialog be in the same node, and dialogs are already modular by being configurable within their respective ResourcePreloaders.
The issue with tooltips on inventory items appearing behind the inventory should be fixed by moving tooltip further down the scene tree in hud.tscn.

I tested this, and the HUD has dimensions 1920x0, so the tooltips aren't visible. When enlarging the HUD the diff looks like this:
diff --git a/device/globals/game.tscn b/device/globals/game.tscn
index 6bb43ba..0434ff9 100644
--- a/device/globals/game.tscn
+++ b/device/globals/game.tscn
@@ -2,7 +2,7 @@

 [ext_resource path="res://globals/game.gd" type="Script" id=1]

-[node name="game" type="Node" index="0"]
+[node name="game" type="Node"]

 script = ExtResource( 1 )
 fallbacks_path = "res://demo/fallbacks.esc"
@@ -19,6 +19,9 @@ transform = Transform2D( 1, 0, 0, 1, 0, 0 )

 [node name="hud" parent="hud_layer" index="0" instance_placeholder="res://ui/hud.tscn"]

+margin_top = -1080.0
+_sections_unfolded = [ "Rect" ]
+
 [node name="wait_timer" type="Timer" parent="." index="1"]

 process_mode = 1

Although position became (0, 0) and the size 1920x1080.
With tooltip_follows_mouse the tooltip does indeed get rendered underneath the inventory items. Dragging the tooltip to the bottom of the node tree, it would appear items are rendered behind the semi-transparent background of the inventory area.
So I'd say this solution may be in the correct direction but does not work now.

I tested this, and the HUD has dimensions 1920x0, so the tooltips aren't visible.

Yeah, I remember this now. I had initially made it span the entire screen and set mouse/filter to IGNORE, which works nicely in-game, however it makes moving items around in the editor impossible, without first hiding the hud. I then decided to just make it span the bottom of the screen, since that works nicely for positioning the verb menu and the inventory, but I forgot about tooltips.
I'll see about just using a CanvasLayer instead.

it would appear items are rendered behind the semi-transparent background of the inventory area

Yeah, I thought so too, but I knew it wasn't the case, since I could interact with them. I mentioned on IRC about a week ago that I was wondering if this was some kind of optical illusion, but I didn't check up on it. Turns out the actual issue was that I was modulating both the inventory and its children, so the items just blended in with the background. I've fixed the problem in my last commit, which I plan on adding into the previous commits where appropriate.
The last commit also fixes tooltips by going back to the previous solution of making the container fill the entire viewport. It's set to hidden so that it doesn't cover nodes in the editor, but is set to visible on _ready. I don't know how I feel about that solution, but at least it makes it testable.

The following issues reported on IRC should now be fixed:

Inventory items blending with inventory: use self-modulate instead of modulate, otherwise all children will also be affected
Tooltip not visible: use global position because having a bottom anchored hud make tooltip appear below viewport

2 little conflicts before I test this PR asap.

@StraToN: I've rebased onto master now, so it's ready for testing. The easiest way to test the layering is to enable Escoria/Ui/Tooltip Follows Mouse from the project settings and make sure that text stays on top of everything in the room, including items in the inventory.
We should also look into improving the contrast of the default dialog and tooltips, but that's outside the scope of this PR.

All good for me, tests pass fine.

Thanks, @StraToN. :slightly_smiling_face:


Triangle class

with this pull request we start working on triangles and triangles meshes

Hi Giovanni! how are you? i finished developing the triangle class and generated the first images, in particular i generated the famous BLACK TRIANGLE.

Now I continue with the mesh of triangles and bounding boxes

A demonstration of the box class, I have yet to write the tests to verify its full correctness
https://github.com/GiovanniPellegrini/not_intellij_idea/assets/149505243/3b66eb5e-7d04-4419-973e-231d251933eb

Hi Giovanni, i have made the first images with triangle meshes now i start working on bounding boxes in order to optimize our code!

P.S the second image is a gourd on a grass plane, i know that it's not so photorealistic...

Hi Giovanni, i finished to develop triangle meshes with Bounding boxes, here an example

With the bounding box activated the code takes less than half of the time for rendering


Set error-message into RestException

Motivation
Right now, RestException(code, message) doesn't set given message into the exception and restException.getMessage() doesn't return correct configured message-string which doesn't log correct information when broker tries to log message string using restException.getMessage().
Modifications
Set correct message-string into RestException.
Result
RestException.getMessage() returns correct message-string.

SUCCESS
--none--


Should the typings include AjvValidator?

I'm using Typescript with the AjvValidator in a custom query builder. I noticed that the type definitions don't include Objection.AjvValidator. Is that intentional? If so, it's easy enough to add it locally.
I realize custom query builders are currently broken with Typescript (#319), but I'm hoping to offer a fix for that. (I'm porting working JS code.)

After resolving this, I believe that the type does belong in objection.js. The signature is specific to this module, requiring people to reverse engineer the code to get it right.
Here's a fix. I'll shortly be working on a fix for #319 and can send a PR for everything at once.
import * as ajv from 'ajv';

declare namespace Objection {
  /*...*/
  export type AjvConfig = {
    onCreateAjv(ajv: ajv.Ajv): void;
    options?: ajv.Options;
  };
  const AjvValidator: new(config: AjvConfig) => ajv.Ajv;
  /*...*/
}

I've also got a fix for adding the createValidationError() tyepdef. Looks like I'll be collecting mods until I've got everything ported.

AjvValidator is a class, not a function that returns an instance of Ajv.

To answer the original question, yes the typings should include AjvValidator. My previous comment was about your suggested typings.

Got it. I see what the problem is. The typedef for Model.createValidator() is missing too, so Typescript inferred its return value. My code never calls the created object, so everything works.
Okay, looks like I've got more to add thanks.
BTW, the new keyword in a typedef defines a constructor rather than a function per se, so the above code effectively includes the Ajv class without redefining it. (Might just be your word choice I'm responding to, in which case I apologize.)

@jtlapp A constructor that returns the wrong type though. new AjvValidator() returns an instance of AjvValidator which is a wrapper around ajv.Avj. AjvValidator doesn't inherit ajv.Ajv.

Yeah, got all that. Sorry, I quit when the first thing I threw in satisfied Typescript, not realizing that satisfying Typescript didn't mean the types were right yet.
Okay, I'm going to PR new typings for just the validation stuff. Most of it was missing, but a bit of what was there was also wrong. Plus I may have found a bug in AjvValidator. Will explain below...

Okay, a number of issues arose for possible discussion during this lucky-numbered PR:

The methods of Validator share a ctx of {}. Rather than create a named type for this, I just called it an object, because it only appears within the new type ValidatorArgs.
$beforeValidate() was typed to return JsonSchema. Objection does not carry forward the return value, so I changed this to void.
$validate() was incorrectly typed, but obviously so. Corrected.
$beforeValidate() is called from within AjvValidator, but $afterValidate is not called from this subclass. Is this a bug? Is either the calling or the failure-to-call a bug?
It might be useful for the docs to note that $validate() takes precedence over the validator returned by createValidator(). I learned this tracing execution in a debugger.
I had several occurrences of calls to beforeValidate() on a custom Validator (not subclassing AjvValidator) whose args argument contained only { ctx, json } and not the full { ctx, json, model, options } that I normally expected. Returning to it later to confirm a recreation, I was unable to recreate it. I guess it's possible I was doing something wrong at the time.

I think that's it. I only started working with Typescript a few months ago and have only gotten into external type declarations recently, so please do double-check my stuff. Thanks!
Oh, there's a bit of irony here. After all this, I find myself inclined to rip out the custom validator. I need to validate at more points than just object creation, so I might as well do it manually with shared validation code. I might still use Ajv though.

Oh, another item:

AjvValidator has more methods than just those that Validator exposes. None are prefixed with an underscore, so I don't know whether which are intended to be externally exposed, which not. I only added the constructor to the AjvValidator subclass.

The docs illustrate returning a value from $beforeValidate(). Either the docs are wrong or the code is wrong. The code does not heed a return value.
The docs illustrate returning a value from the Validator beforeValidate() and afterValidate() methods. Again, the code ignores the return values.

@jtlapp

Objection does use the return value of $beforeValidate
$afterValidate is called from the base class, and thus inherited
What? $validate() uses the validator created using createValidator.
Same as 1
The docs show that a value should be returned for future compatibility. Currently the return value is not used, but in the future we might add a return value. Maybe the return value can be marked as any at this point or something.

I stepped through the debugger. $beforeValidate's value was ignored and $afterValidate was not called when createValidate returned a new Objection.AjvValidator.
Checking now, I see that I was using origin/master for the tests, not the recent pub.
Re 5: When I override $validate, createValidator does not get called. That's an obvious consequence if $validate is calling createVaidator.
The problem with the Pojo definition is that it can't be customized for static type checking.
I'll try again with the latest master and get back with you, this time pointing into the code.

$beforeValidates return value was always used.

Same goes with $afterValidate being called. It does work, and has always worked.
https://github.com/Vincit/objection.js/blob/master/tests/unit/model/Model.js#L208
https://github.com/Vincit/objection.js/blob/master/tests/unit/model/Model.js#L239
These are all called by Validator or its subclasses. If you implement a custom validator, you need to take care of them being called.

Okay. Gimme time to check things. I spent most of a day on this last time, coming up with typings.

Ah poo on $beforeValidate. I see why I was mistaken. It's AjvValidator's beforeValidate who's return value is ignored (as expected).
However, when returning AjvValidator, VSCode will not run my $afterValidate override on a Model. It never gets stepped into, can't breakpoint within it.
And yet the code suggests that it should be called. Still looking into this.

Aaaaand... I'm missing a super.$afterValidate call from a descendant model.
Cancel items (2), (4), and (8). I'm so sorry to waste your time.
With (9), it's only $afterValidate that doesn't return a value.

Looks like our discussion disappeared after my last PR bringing things current. I opted to do the following to get static type checking on declared properties and pojo behavior otherwise:
  export interface ValidatorContext extends Pojo { }

  export type ValidatorArgs = {
    ctx: ValidatorContext,
    model: Model,
    json: Pojo,
    options: ModelOptions
  }

ValidatorContext can be opened to have specific properties constrained, leaving all non-named properties remain unconstrained:
declare module 'objection' {
    export interface ValidatorContext {
        foo: number;
    }
}

The latest also adds the JsconSchema return to back to $beforeValidate.


how to use this library to do discovery apis

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Hello, I am trying to implement google market place using this library I came here from this doc page
https://cloud.google.com/marketplace/docs/partners/commerce-procurement-api/reference/rest
but I am stuck on getting any details how to use it. can you guide me to the docs?

Commerce procurement API isn't yet available in the google-api-php-client-services, an upstream library of this repository.
Pls consider calling APIs via REST apis (manually constructing calls) and meanwhile we will explore adding this library to the available packages.


Create node dynamically from kafka topic

Hello,
I use the kafka streams plugin to ingest data into neo4j database from kafka topics.
It working fine when we map a kafka topic to a given node, for example :
streams.sink.topic.pattern.node.person-topic.to.testdb2=Person{!id}

The topic data looks like :
[{id:1, name : Leo},{id: 2, name: Kevin}]

But I want to create node label dynamically from another kafka topic, I want to extract the label node from a data attribute. For example I want to create 2 differents node Developer and Player from the same kafka topic based on an attribute called type.
The data looks like :
[{id:1, name : Leo, type : football_player }, {id: 2, name: Kevin, type : software_engineer}]

I want to create 2 nodes :

Player with attributes {id:1, name : Leo, type : football_player }
Developer with attributes {id: 2, name: Kevin, type : software_engineer}

I tried with using the following Cypher template :
streams.sink.topic.cyher.person-topic.to.testdb2=MERGE (p:{event.type}) ON CREATE set p.id = event.id, p.name = event.name

I got the following exception :
exception=org.neo4j.graphdb.QueryExecutionException: Invalid input '{': expected whitespace or a label name (line 1, column 34 (offset: 33))

is there a way to create node dynamically from the same kafka topic using kafka streams ?
Regards,
Aissa

Hi @aissaelouafi,
you should use the APOC apoc.create.node in your query. Please see the APOC doc at the following: https://neo4j.com/labs/apoc/4.2/overview/apoc.create/apoc.create.node/.
In your case you have to replace the query with the following:
streams.sink.topic.cyher.person-topic.to.testdb2=call apoc.create.node(event.type, {id: event.id, name: event.name}) YIELD node RETURN node

You should also change a bit the format of your json events, so that event.type would be an array:
{id:1, name : Leo, type : [football_player] }

Hope this helps!
Mauro

Hi @mroiter-larus,
Thanks a lot for you answer and your help as usual.
Unfortunately we dont have the control over the input data, we can't modify the type.
The main idea is to be able to create a map function (based on if conditions for example) between the type attribute and the node label.
for example :

Create a Player node for the following events : [{id:1, name : Leo, type : football_player },{id:2, name : Rafa, type : tennis_player}, {id:3, name : LeBron, type : basketball_player } ]
Create a Developer node for the following events : [{id:1, name : Kevin, type : software_engineer },{id:2, name : Nico, type : devops}, {id:3, name : Julien, type : web_developer } ]

The idea is to create dynamically the node based on a condition on an attribute, in this case type attribute
if type in ['football_player', 'tennis_player', 'basketball_player']:
   node = 'Player'
elif type in ['software_engineer', 'devops', 'web_developer']:
   node = 'Developer'

I dont know if it's possible to do that using cypher template.
Regards and thanks a lot again for you help @mroiter-larus.

Hi @aissaelouafi,
for conditional executions you can use APOC apoc.do.case, https://neo4j.com/labs/apoc/4.3/overview/apoc.do/apoc.do.case/
For example:
streams.sink.topic.cyher.person-topic.to.testdb2=call apoc.do.case([event.type IN ['football_player', 'tennis_player', 'basketball_player'], 'CREATE (n:Player) SET n.id=event.id, n.name=event.name RETURN n as node', event.type IN ['software_engineer', 'devops', 'web_developer'], 'CREATE (n:Developer) SET n.id=event.id, n.name=event.name RETURN n as node'], 'CREATE (n:General) RETURN n as node', {event:event}) YIELD value RETURN value.node

Regards,
Mauro

Hi @mroiter-larus,
Thanks a lot for the provided example.
I have the following exception :
exception=org.neo4j.graphdb.QueryExecutionException: There is no procedure with the name `apoc.do.case` registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

I think that I need to add this procedure in the allowlist procedures, right ?
Thanks a lot again.

Hi @aissaelouafi,
try defining the following two properties:
streams.check.apoc.timeout=<ms to await for APOC being loaded, default -1 skip the wait>
streams.check.apoc.interval=<ms interval awaiting for APOC being loaded, default 1000>

please see here for further details.
The problem is most likely that the plugin starts before the APOC procedures has been fully loaded.
Regards,
Mauro

Hi @aissaelouafi,
any news?
Mauro

Hi @mroiter-larus,
Sorry I was busy about something else.
I just tried, I added the parameters and it's working well now.
Thanks a lot for you help as usual.
Regards,
Aissa

Hi @aissaelouafi,
great! Happy to help!
Regards,
Mauro

Hi @mroiter-larus,
Sorry to distrub you again but I still have a question.
I want to create the relationships between the nodes created before dynamically using another kafka topic that contains the relationships between the node.
The input data looks like :
[{'parent_id':1, 'child_id':2, 'type':'brother'},
{'parent_id':4, 'child_id':5, 'type':'friend'}]

I can't have the node label on the input json, I tried to use a MATCH query to get nodes dynamically and to create the relationships but It's not working, this is the cypher query :
streams.sink.topic.cypher.relationships-topic.to.test-db2=MATCH (child),(parent) WHERE child.id=event.child_id AND parent.id=event.parent_id CREATE (parent)-[r:event.type]->(child) RETURN r

I don't know if we can perform this kind of cypher queries and if it's efficient in term of performance because we can have many thousands of nodes.
Thanks a lot for your precious help again.
Regards,
Aissa

Hi @aissaelouafi,
in order to increase the query performance you should add the node's label you want to match (but if you can't have the node label on the input json it obviously can't be done) and create also an index for the property id of that nodes:
CREATE INDEX index_name_parent FOR (n:LabelParent) ON (n.id)
CREATE INDEX index_name_child FOR (n:LabelChild) ON (n.id)

Please read here for further details about indexes.
Furthermore, since you can't add labels, to safely perform the query you've shared, you must be sure about the uniqueness of the ids. I mean, you must be sure that parent node with id=1 can't exists together with a child node with id=1, otherwise your query will match all the nodes with id=1 and you'll never know which is the parent and which is the child.
About the query, be careful doing MATCH (child), (parent)... because this will generate a cartesian product between the two nodes. You can also include your where condition into the match clause, like that:
MATCH (child: {id: event.child_id}) MATCH (parent: {id: event.parent_id}) CREATE...

I noticed also that your Streams configuration is pointing to two different databases, testdb2 and test-db2:
streams.sink.topic.cyher.person-topic.to.testdb2=call apoc.do.case([event.type...
streams.sink.topic.cypher.relationships-topic.to.test-db2=MATCH (child),(parent)...

This might be a problem because the query which is responsible for relationships creation will never works if the nodes does not exists in the database.
Mauro

Hi @aissaelouafi,
the exception is caused by a syntax error because the label were missing. My bad. You just need to add the label into the match clauses:
MATCH (child:LabelChild {id: event.child_id}) MATCH (parent:LabelParent {event.parent_id}) CREATE (parent)-[r:{event.type}]->(child) RETURN r

The problem here is that you don't know which label has to be specified. I mean, you don't know the nodes label to which events into relationships-topic are referring to. Is it correct?
If so, you can't create the relationship like that.
The only thing you can do here is create two different topics:

one for events which creates relationships between Player nodes
one for events which creates relationships between Developer nodes

so that you can configure two different Sink cypher queries, one for each type of relationships. For example:
streams.sink.topic.cypher.relationships-player-topic.to.test-db2=MATCH (child:Player {id: event.child_id}) MATCH (parent:Player {event.parent_id}) CREATE (parent)-[r:{event.type}]->(child) RETURN r
streams.sink.topic.cypher.relationships-developer-topic.to.test-db2=MATCH (child:Developer {id: event.child_id}) MATCH (parent:Developer {event.parent_id}) CREATE (parent)-[r:{event.type}]->(child) RETURN r

Mauro

Hi again @mroiter-larus,
As I told you before I tried this query without specifiying the node label, it's working in the browser but I dont know if it's supported by the kafka sink plugin.
MATCH (child),(parent) WHERE child.id=id1AND parent.id=id2 CREATE (parent)-[r:TEST]->(child) RETURN r

But the issue as you said is the query will generate a cartesian product between the two nodes. I dont know if there is a way to include the filter on id without speicifying the label node.
Regards
Aissa

@aissaelouafi
the query is supported by the plugin. This is not the problem. The problem is that without specifying the label, Neo4j will scan all the nodes of the graph every time (could be very expensive if you have millions) and you'll generate a lot of useless relationships.
Maybe there is a way to avoid cartesian product. Please checkout the following: https://community.neo4j.com/t/avoid-cartesian-product-when-create-relationships/13624. But i don't know if this could help you, because you'll avoid cartesian product but not the creation of useless relationships.
As i told you before, to safely perform the query you've shared, you must be sure that parent node with id=1 can't exists together with a child node with id=1, otherwise your query will match all the nodes with id=1 and you'll never know which is the parent and which is the child.
Regards,
Mauro

@mroiter-larus,
I hope that you had a great weekend.
Thanks for you the link, I will check how we can avoid the cartesian product.
Concerning the useless relationships, I don't think it's going to happen because we are using the uuid as an identifiant for both child and parent nodes, So the parent node with id=1 can't exists together witg a child node with id=1.
Concernant the cypher query to match nodes without specifiying the node label, I can use this one ?
streams.sink.topic.cypher.relationships-topic.to.test-db2=MATCH (child),(parent) WHERE child.id={event.child_id} AND parent.id={event.parent_id} CREATE (parent)-[r:{event.type}]->(child) RETURN r
Thanks a lot again.
Regards,
Aissa

Hi @aissaelouafi,
hope you had a great weekend too.
I'm sorry, I told you can use that query but it is not totally correct. You don't need the curly braces for accessing parameter values event.parent_id, event.child_id and event.type. Furthermore, you can't create the relationships "dynamically" like that:
CREATE (parent)-[r:{event.type}]->(child)

you need to use apoc.create.relationship as follow:
streams.sink.topic.cypher.relationships-topic.to.test-db2=MATCH (child),(parent) WHERE child.id=event.child_id AND parent.id=event.parent_id CALL apoc.create.relationship(parent, event.type, {}, child) YIELD rel RETURN rel

Please see the documentation about the APOC: https://neo4j.com/labs/apoc/4.1/overview/apoc.create/apoc.create.relationship/
Regards,
Mauro

Hi @mroiter-larus,
It's working well. Many thanks for your precious comments and help.
I wish you all the best !
Regards,
Aissa

Hi @aissaelouafi,
glad that it's working fine!
Regards,
Mauro

Hi @mroiter-larus,
I hope that you are doing well, I have another issue with the relationship creation apoc.
I used this cypher query to create the relationship :
streams.sink.topic.cypher.relationships-topic.to.test-db2=MATCH (child),(parent) WHERE child.id=event.child AND parent.id=event.parent CALL apoc.create.relationship(parent, event.type_display, {}, child) YIELD rel RETURN rel

I have this data :
{
  "parent": "626a4f1b1be46810704d8055464bcbf1",
  "type_display": "Hosted by ::",
  "child": "4021d02f1b24ac10123c3035464bcb1e"
}

The idea is to match the label node based on id, but the relationships are not created even if I check that both labels exists with parent and child id.
I have the impression that the request is not taken into account by the plugin. I can't see any message in logs regarding the query but sometimes when I restart the neo4j server I can see the created relationships.
The total number of nodes in the database is : 736670 nodes
The number of node with parent id is : 1 node
The number of node with child id is : 1 node
I added this parameters to the streams.conf file :
streams.sink.enabled=true
streams.sink.enabled.to.test-db2=true
streams.sink.errors.log.enable=true
streams.sink.errors.log.include.messages=true

kafka.enable.auto.commit=true
kafka.key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
kafka.value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
kafka.security.protocol=SSL

kafka.ssl.truststore.location=/apps/neo4j/neo4j-enterprise/conf/syslogng.jks
kafka.ssl.truststore.password=test123
kafka.ssl.keystore.location=/apps/neo4j/neo4j-enterprise/conf/syslogng.jks
kafka.ssl.keystore.password=test123

kafka.ssl.endpoint.identification.algorithm=HTTPS
streams.check.apoc.timeout=20000
streams.check.apoc.interval=10000
streams.sink.poll.interval=5000
kafka.streams.async.commit=true

kafka.group.id=neo4j_group
dbms.jvm.additional=-Djavax.net.debug=ssl:handshake

I don't understand why the query is not executed, when I tried the request manually I can see that the relationships are well created.
If you have an idea it can be very helpful.
Thanks again for your help as usual @mroiter-larus
Regards,

Hi @mroiter-larus,
I think it's related to a query plan caching.
Sometimes I can get the relationship with this message :
Discarded stale query from the query cache after 62549 seconds. Reason: NodesAllCardinality changed from 29300.0 to 308214.0, which is a divergence of 0.9049361807056137 which is greater than threshold 0.043763100387390126. Query: UNWIND $events AS event MATCH (child),(parent) WHERE child.id=event.child AND parent.id=event.parent CALL apoc.create.relationship(parent, event.type_display, {}, child) YIELD rel RETURN rel

Do you have an idea about this behavior ?
Regards,
Aissa


the data flow line in topologic graph is missing

Please answer these questions before submitting your issue.

Why do you submit this issue?
[x] Question or discussion
[ ] Bug
[ ] Requirement
[ ] Feature or performance improvement

Question

What do you want to know?
I implement some custom agents to collect data. in the trace graph, i can see the full stack of the traces. in topologic graph, i can see the server nodes. but the flow line between server nodes are disappeared.

Bug

Which version of SkyWalking, OS and JRE?
base on skywalking agent 6.1.0 and skywalking backend 6.1.0

Which company or project?

What happen?
If possible, provide a way for reproducing the error. e.g. demo application, component version.

Requirement or improvement

Please describe about your requirements or improvement suggestions.

here is the screenshot:

when switch into this tab. 2 reqeust had been made:
request 1:
request body:
{"query":"query queryTraces($duration: Duration!) {\n  topo: getGlobalTopology(duration: $duration) {\n    nodes {\n      id\n      name\n      type\n      isReal\n    }\n    calls {\n      id\n      source\n      target\n      callType\n      detectPoint\n    }\n  }}","variables":{"duration":{"start":"2019-07-08 1316","end":"2019-07-08 1331","step":"MINUTE"}}}

response body:
{"data":{"topo":{"nodes":[{"id":"1","name":"User","type":"USER","isReal":false},{"id":"2","name":"mservice1","type":"Dubbo","isReal":true},{"id":"3","name":"webapp1","type":"N/A","isReal":true},{"id":"5","name":"<IP_ADDRESS>:6379","type":"Redis","isReal":false},{"id":"6","name":"<IP_ADDRESS>:6382","type":"Redis","isReal":false},{"id":"7","name":"<IP_ADDRESS>:3306","type":"Unknown","isReal":false},{"id":"8","name":"<IP_ADDRESS>:3306","type":"Unknown","isReal":false}],"calls":[{"id":"3_6_30","source":"3","target":"6","callType":"Jedis","detectPoint":"CLIENT"},{"id":"2_5_30","source":"2","target":"5","callType":"Jedis","detectPoint":"CLIENT"},{"id":"2_8_1001","source":"2","target":"8","callType":"N/A","detectPoint":"CLIENT"},{"id":"3_5_30","source":"3","target":"5","callType":"Jedis","detectPoint":"CLIENT"},{"id":"2_2_3","source":"2","target":"2","callType":"Dubbo","detectPoint":"CLIENT"},{"id":"2_7_1001","source":"2","target":"7","callType":"N/A","detectPoint":"CLIENT"},{"id":"2_6_30","source":"2","target":"6","callType":"Jedis","detectPoint":"CLIENT"},{"id":"3_2_3","source":"3","target":"2","callType":"Dubbo","detectPoint":"CLIENT"},{"id":"1_2_3","source":"1","target":"2","callType":"","detectPoint":"SERVER"},{"id":"1_3_0","source":"1","target":"3","callType":"","detectPoint":"SERVER"}]}}}

request 2:
request body:
{"query":"query queryTrace(\n    $duration: Duration!,\n    $ids: [ID!]!,\n    $idsS: [ID!]!,\n    $idsC: [ID!]!)\n      {\n        \n  sla: getValues(metric: {\n    name: \"service_sla\"\n    ids: $ids\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  }\n  nodeCpm: getValues(metric: {\n    name: \"service_cpm\"\n    ids: $ids\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  }\n  nodeLatency: getValues(metric: {\n    name: \"service_resp_time\"\n    ids: $ids\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  },\n        \n  cpmS: getValues(metric: {\n    name: \"service_relation_server_cpm\"\n    ids: $idsS\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  }\n  latencyS: getValues(metric: {\n    name: \"service_relation_client_resp_time\"\n    ids: $idsS\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  },\n        \n  cpmC: getValues(metric: {\n    name: \"service_relation_client_cpm\"\n    ids: $idsC\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  }\n  latencyC: getValues(metric: {\n    name: \"service_relation_client_resp_time\"\n    ids: $idsC\n  }, duration: $duration) {\n    values {\n      id\n      value\n    }\n  }\n      }","variables":{"duration":{"start":"2019-07-08 1316","end":"2019-07-08 1331","step":"MINUTE"},"ids":["1","2","3","5","6","7","8"],"idsC":["3_6_30","2_5_30","2_8_1001","3_5_30","2_2_3","2_7_1001","2_6_30","3_2_3"],"idsS":["1_2_3","1_3_0"]}}

response body:
{"data":{"sla":{"values":[{"id":"2","value":9218},{"id":"3","value":10000}]},"nodeCpm":{"values":[{"id":"2","value":20},{"id":"3","value":21}]},"nodeLatency":{"values":[{"id":"2","value":179},{"id":"3","value":374}]},"cpmS":{"values":[{"id":"1_2_3","value":1},{"id":"1_3_0","value":21}]},"latencyS":{"values":[]},"cpmC":{"values":[{"id":"3_6_30","value":63},{"id":"2_5_30","value":110},{"id":"2_8_1001","value":390},{"id":"3_5_30","value":8},{"id":"2_2_3","value":7},{"id":"2_7_1001","value":4},{"id":"2_6_30","value":7},{"id":"3_2_3","value":24}]},"latencyC":{"values":[{"id":"3_6_30","value":19},{"id":"2_5_30","value":0},{"id":"2_8_1001","value":0},{"id":"3_5_30","value":0},{"id":"2_2_3","value":80},{"id":"2_7_1001","value":1},{"id":"2_6_30","value":0},{"id":"3_2_3","value":319}]}}}

What do you mean custom agents? If it is built by you, how could we know why you miss something?

@wu-sheng  are there any way or guide to tell how to debug the custom agent? compare compare zipikin reporter, skywalking probe looks like more difficult to debug.

@wu-sheng are there any way or guide to tell how to debug the custom agent? compare compare zipikin reporter, skywalking probe looks like more difficult to debug.

Assuming you can read Chinese, there're multiple blog articles that you may need, http://skywalking.apache.org/zh/blog/

@kezhenxu94 非常感谢

@kezhenxu94 非常感谢

English only though we provide Chinese docs :)

@kezhenxu94 非常感谢

English only though we provide Chinese docs :)

I prefer Chinese actully


Set up Build stage in CI/CD pipeline on IBM Cloud

As a developer
I need the automated CI/CD pipeline to run a build every time I push changes to IBM Cloud
So that I can test that the application still builds properly before deploying any changes to production
Details and assumptions

This story is part of coverage for #55; assumptions from that story apply here as well.
The build stage should run our unit tests via nosetests

Acceptance criteria
Given the build stage is set up
When I push changes to the application
Then the build stage of the pipeline will build the application on IBM Cloud and run our unit tests against the latest code

All three jobs (Checkout, Test, Build Image) are now passing in this stage of the pipeline on IBM Cloud


DUPLO 14752 GCP:CloudSQL: Force replacement should not be shown for database version upgrade

Overview
This PR handles state difference on GCP cloud sql resource creation or update of database_version. For version change in-place update would be carried out for database change resource will be replaced
This PR does the following:

Implemented replaceOn function of isForceNew function type which checks change of database

Testing performed

[ ] Using unit tests
[ ✓] Manually, on my local system
[ ] Manually, on a remote test system

Describe any breaking changes

...

Task linked: DUPLO-14752 TF: GCP:CloudSQL: Force replacement should not be shown for database version upgrade


initial smart wallet example setup

Opening as a draft to get feedback
but this should be a working example of coinbase smart wallet integration
https://github.com/wslyvh/nexth/assets/24376928/a8d90964-82f5-47f3-b252-3412867b287c
https://github.com/wslyvh/nexth/assets/24376928/a5481b9e-0e07-4c0e-9767-231b0474d7ae
https://github.com/wslyvh/nexth/assets/24376928/3a270824-2335-4869-b3e7-8b4971b47b9a
Removed the Coinbase connector specific connect and disconnect buttons in favor of telling the user to use the coinbase wallet in web3Modal and added links to smart wallet repo and docs

Lmao I didn't push the package.json and now I can't get it working again 😭 sad day

This is great! Love it
I wonder a little bit if we should add it as an example page or as a more detailed, full example implementation.
My thinking so far is that example/pages should demonstrate basic functionalities (send ether, transfer tokens, etc.) without requiring custom configurations, etc.
More detailed examples like this, which require different packages/configuration/etc. might be easier to maintain as fork or branch. There are a couple of other branches as examples (e.g. ethers instead of viem, using signature-recovery,etc.). We should probably keep a list of active examples on the readme so its easier to find tho.
In this case, I think you could replace the default connect/sign in flow, include a simple contract, NFT minting and provide the full E2E example and functionalities of smart wallets.
What do you think?

This is great! Love it
I wonder a little bit if we should add it as an example page or as a more detailed, full example implementation. My thinking so far is that example/pages should demonstrate basic functionalities (send ether, transfer tokens, etc.) without requiring custom configurations, etc.
More detailed examples like this, which require different packages/configuration/etc. might be easier to maintain as fork or branch. There are a couple of other branches as examples (e.g. ethers instead of viem, using signature-recovery,etc.). We should probably keep a list of active examples on the readme so its easier to find tho.
In this case, I think you could replace the default connect/sign in flow, include a simple contract, NFT minting and provide the full E2E example and functionalities of smart wallets.
What do you think?

Sounds good to me! I'll get working on this soon. I was actually thinking the smart contract example should have a lot more documentation to it so i think breaking it out into it's own branch and getting more detailed is a great idea.

Hi @hone1er just checking in and see if this is still something you'd like to contribute?
I believe there have been a lot of advancements in smart contract wallets. Would love to see a Nexth example if you're up for it :)

Hi @hone1er just checking in and see if this is still something you'd like to contribute? I believe there have been a lot of advancements in smart contract wallets. Would love to see a Nexth example if you're up for it :)

Hey @wslyvh ! Yeah my bad i've been working on some buildathon stuff and trying to get freelance work where i can. Lost my job recently lol if no one gets to some smart wallet examples before i get back to this i'd definitely still like to contribute
I've been working with the smart wallet a bit more lately plus wagmi and viem have experimental features already to make working with smart wallets easier so it should be a pretty smooth setup now

Hi @hone1er I would love to see a proper Smart contract wallet example, but closing the PR for now.. Happy to chat, once you're ready for it :)