This is a version of Richard's function that uses maybe a bit less memory.

I am not sure that this is worth it. This is meant to address @kishoreg's concerns.

Do you have numbers? The idea was that filling up the slices would avoid most of the work. You don’t do any container intersections unless you know you have a full slice of N containers.

Let’s benchmark this and take it if it’s better, no need for two versions. I’m not all that enthusiastic about memory improvements at the cost of throughput because the algorithm already avoids a lot of allocation compared to naive_and. If this improves things then it’s progress.

The idea was that filling up the slices would avoid most of the work. You don’t do any container intersections unless you know you have a full slice of N containers.

I think it is exactly the same algorithm. The difference is that you construct a potentially large array where your store the containers, and then reload them from this array.

Do you have numbers?

We don't have a benchmark right now where your approach uses a lot of memory, except for the allocation of the temporary 1024-longs array.
For that to happen, you need for numContainers to be large at the same time as bitmaps.length is large. So take one constant but multiply repeated bitmap, containing many containers. So numContainers equals to 4000 and bitmaps.length equals to 100,000. Then your approach will allocate a block of 300 MB each and every time it is called, unlike my approach.
I am not pushing this PR. I just want to illustrate what is possible.
If I wanted to push it, I would design a benchmark to make it look good. But that's not very interesting (creating the benchmark after the fact to make your code look good).
@kishoreg is concerned about memory usage. I'd invite him and other people who have these concerns to have a look at my PR.

Coverage increased (+0.01%) to 91.9% when pulling 00d151ea519f0ed94d2bb1d6b76186d2f96f49a3 on dlemire/workandmemoryshy into 0c3c4895291134fb288062d578cb2f39f6b35009 on master.

@richardstartin Let me put it another way. I started answering @kishoreg about design trade-offs and figured it would be easier to show what I meant with code.
We can merge, or not, this function. I have no vested interest. I just wanted to show that we could take your excellent approach and drastically reduce the worst-case memory usage.
I agree that your approach will do fine memory-wise in practice, I expect... But maybe  @kishoreg has other concerns that I cannot foresee.

Let me add also that creating a benchmark that shines where the memory usage of your opponent blows up, is something I'd rather not check into RoaringBitmap.

We could also create hybrids.

@lemire I think it's best to merge your changes (i.e. the merging of the last two loops and removal of the 2D containers array) into the existing method, rather than create a new one.

To be honest, I don't see a reason not to apply the change to the old method, which will probably be deleted at some point anyway.
Reusing the buffer is obviously also applicable to the original method. If you want two versions of the method, can you make workShyAnd take a buffer as input too, and then provide an overload of and which takes a buffer?
So:
RoaringBitmap and(RoaringBitmap... bitmaps) {
  return workShyAnd(new long[1024], bitmaps);
}

RoaringBitmap and(long[] bitmap, RoaringBitmap... bitmaps) {
    return workShyAnd(bitmap, bitmaps);
}

@richardstartin  I am running a benchmark:
$ java -jar jmh/target/benchmarks.jar FastAggregation

Benchmark                                        (constructionStrategy)  (count)  (probability)  (seed)   (size)   Mode  Cnt    Score     Error  Units
FastAggregationRLEStressTest.and                        CONSTANT_MEMORY      100           0.01   99999  1000000  thrpt    5  867.029 ± 170.413  ops/s
FastAggregationRLEStressTest.and                        CONSTANT_MEMORY      100            0.1   99999  1000000  thrpt    5  102.334 ±  17.711  ops/s
FastAggregationRLEStressTest.and                        CONSTANT_MEMORY      100            0.5   99999  1000000  thrpt    5  350.416 ±  36.901  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5  936.893 ±  90.405  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5  107.553 ±   3.822  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5  342.889 ±  27.903  ops/s
FastAggregationRLEStressTest.andBuffer                  CONSTANT_MEMORY      100           0.01   99999  1000000  thrpt    5  920.579 ±  61.322  ops/s
FastAggregationRLEStressTest.andBuffer                  CONSTANT_MEMORY      100            0.1   99999  1000000  thrpt    5   91.479 ±  35.229  ops/s
FastAggregationRLEStressTest.andBuffer                  CONSTANT_MEMORY      100            0.5   99999  1000000  thrpt    5  300.711 ± 105.753  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5  777.514 ± 424.695  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5   93.305 ±  11.589  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5  297.298 ±  26.440  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy         CONSTANT_MEMORY      100           0.01   99999  1000000  thrpt    5  811.430 ± 340.511  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy         CONSTANT_MEMORY      100            0.1   99999  1000000  thrpt    5   84.778 ±  36.852  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy         CONSTANT_MEMORY      100            0.5   99999  1000000  thrpt    5  272.822 ±  32.180  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5  848.879 ± 180.828  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5   94.064 ±  10.410  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5  290.661 ±  29.712  ops/s
FastAggregationRLEStressTest.andMemoryShy               CONSTANT_MEMORY      100           0.01   99999  1000000  thrpt    5  912.764 ± 195.919  ops/s
FastAggregationRLEStressTest.andMemoryShy               CONSTANT_MEMORY      100            0.1   99999  1000000  thrpt    5  104.556 ±  15.336  ops/s
FastAggregationRLEStressTest.andMemoryShy               CONSTANT_MEMORY      100            0.5   99999  1000000  thrpt    5  347.633 ±  70.697  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5  873.314 ± 118.928  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5  106.296 ±   6.539  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5  341.304 ±   8.545  ops/s

To be honest, I don't see a reason not to apply the change to the old method, which will probably be deleted at some point anyway.

I can delete your method and override mine as you propose. It is trivial.
(Note that you also have write access to the branch in question. Feel free to hack it directly if you want.)

I think it's best to merge your changes (i.e. the merging of the last two loops and removal of the 2D containers array) into the existing method, rather than create a new one.

Please confirm that this is what you want me to do.

Actually, once you reuse the words buffer in the old method, I'm not sure this is the best of the two approaches, this is only better when the intersection itself is very large. So maybe it's better to keep the two separate. I have a change I will apply to your branch.

This is a space/time tradeoff. With this approach, I think the user may give up up to 5% throughput to avoid allocating an array which will often be tiny, and the allocation is reduced anyway because there are no container transitions. Let's add the new method and users who care about memory can call the one which allocates less.

E.g. see the benchmark derived from druid. If we're being objective I don't think we should shy away from constructing a benchmark to demonstrate the worst case for any algorithm.
Benchmark                                        (constructionStrategy)  (count)  (probability)  (seed)   (size)   Mode  Cnt      Score      Error  Units
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5   4855.770 ▒  373.631  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5  11604.443 ▒   40.282  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5   6752.779 ▒  120.192  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5    639.729 ▒   22.134  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5     66.867 ▒    1.848  ops/s
FastAggregationRLEStressTest.and                     CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5    218.860 ▒   10.424  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5   4721.572 ▒  269.369  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5  10329.444 ▒ 2976.924  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5   6608.043 ▒  229.798  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5    567.643 ▒   96.512  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5     66.356 ▒    0.860  ops/s
FastAggregationRLEStressTest.andMemoryShy            CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5    137.922 ▒    4.764  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5   5132.889 ▒  264.818  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5  11980.194 ▒  156.344  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5   6186.253 ▒   51.083  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5    613.217 ▒   25.316  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5     60.029 ▒    1.385  ops/s
FastAggregationRLEStressTest.andBuffer               CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5    222.801 ▒    1.932  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5   5301.013 ▒  290.699  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5  10887.899 ▒   73.470  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5   6274.564 ▒   40.254  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5    603.944 ▒   11.274  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5     59.613 ▒    0.230  ops/s
FastAggregationRLEStressTest.andBufferMemoryShy      CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5    219.355 ▒   13.373  ops/s

This is a space/time tradeoff. With this approach, I think the user may give up up to 5% throughput to avoid allocating an array which will often be tiny, and the allocation is reduced anyway because there are no container transitions. Let's add the new method and users who care about memory can call the one which allocates less.

This sounds good.
Can you have a look at the comments that I flagged. I think that you introduced two lies. :-)

resolved

@richardstartin If you are happy, I will merge.

Looking at allocation rate, this doesn't really make much difference on the cases we're measuring

Benchmark                                                                     (constructionStrategy)  (count)  (probability)  (seed)   (size)   Mode  Cnt       Score         Error   Units
FastAggregationRLEStressTest.and:▒gc.alloc.rate                                   CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5      68.985 ▒       3.039  MB/sec
FastAggregationRLEStressTest.and:▒gc.alloc.rate                                   CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5     945.149 ▒      73.921  MB/sec
FastAggregationRLEStressTest.and:▒gc.alloc.rate                                   CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5     568.803 ▒      33.396  MB/sec
FastAggregationRLEStressTest.and:▒gc.alloc.rate                                   CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5      10.904 ▒       0.366  MB/sec
FastAggregationRLEStressTest.and:▒gc.alloc.rate                                   CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5       5.742 ▒       0.101  MB/sec
FastAggregationRLEStressTest.and:▒gc.alloc.rate                                   CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5      19.524 ▒       0.306  MB/sec
FastAggregationRLEStressTest.andMemoryShy:▒gc.alloc.rate                          CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5      61.712 ▒       7.650  MB/sec
FastAggregationRLEStressTest.andMemoryShy:▒gc.alloc.rate                          CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5     943.880 ▒      30.908  MB/sec
FastAggregationRLEStressTest.andMemoryShy:▒gc.alloc.rate                          CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5     571.208 ▒       8.122  MB/sec
FastAggregationRLEStressTest.andMemoryShy:▒gc.alloc.rate                          CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5       8.051 ▒       1.129  MB/sec
FastAggregationRLEStressTest.andMemoryShy:▒gc.alloc.rate                          CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5       5.443 ▒       0.176  MB/sec
FastAggregationRLEStressTest.andMemoryShy:▒gc.alloc.rate                          CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5      18.445 ▒       0.580  MB/sec
FastAggregationRLEStressTest.andBuffer:▒gc.alloc.rate                             CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5      75.860 ▒       2.154  MB/sec
FastAggregationRLEStressTest.andBuffer:▒gc.alloc.rate                             CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5     966.632 ▒      21.480  MB/sec
FastAggregationRLEStressTest.andBuffer:▒gc.alloc.rate                             CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5     537.416 ▒       9.896  MB/sec
FastAggregationRLEStressTest.andBuffer:▒gc.alloc.rate                             CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5      11.204 ▒       0.197  MB/sec
FastAggregationRLEStressTest.andBuffer:▒gc.alloc.rate                             CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5       5.209 ▒       0.116  MB/sec
FastAggregationRLEStressTest.andBuffer:▒gc.alloc.rate                             CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5      19.745 ▒       1.215  MB/sec
FastAggregationRLEStressTest.andBufferMemoryShy:▒gc.alloc.rate                    CONTAINER_APPENDER       10           0.01   99999  1000000  thrpt    5      78.094 ▒       2.064  MB/sec
FastAggregationRLEStressTest.andBufferMemoryShy:▒gc.alloc.rate                    CONTAINER_APPENDER       10            0.1   99999  1000000  thrpt    5     872.449 ▒      61.661  MB/sec
FastAggregationRLEStressTest.andBufferMemoryShy:▒gc.alloc.rate                    CONTAINER_APPENDER       10            0.5   99999  1000000  thrpt    5     361.426 ▒     206.814  MB/sec
FastAggregationRLEStressTest.andBufferMemoryShy:▒gc.alloc.rate                    CONTAINER_APPENDER      100           0.01   99999  1000000  thrpt    5       8.075 ▒       0.353  MB/sec
FastAggregationRLEStressTest.andBufferMemoryShy:▒gc.alloc.rate                    CONTAINER_APPENDER      100            0.1   99999  1000000  thrpt    5       4.849 ▒       0.280  MB/sec
FastAggregationRLEStressTest.andBufferMemoryShy:▒gc.alloc.rate                    CONTAINER_APPENDER      100            0.5   99999  1000000  thrpt    5      18.247 ▒       0.916  MB/sec

I did not expect a difference in our benchmark.

can you point me to the benchmark code

@kishoreg if you’re trying to use this in Pinot and it’s not working well it would be interesting to have some benchmark data


Fix the issue with the filter functions in the proxy class

How to demo

The GetDataProxy class get the data from the database in increments (user can specify the offset and number of the data in each offset).
The GetDataProxy class fetches data from the api when the database doesn't have enough data.
Several filters can be performed in the filter method.

DOne


Archive canister duplicate transactions

Found this implementation bug on accident. My archive canister ran out of cycles, when cycles were readded I noticed a duplicate batch of transactions in the archive.
The theory is, it got up to 2k transcations in the ledger and when it tried to send them the archive was out of cycles so it kept trying to send them after every transaction.
Once the archive was refilled, each transaction will try to send that bulk. So you basically need just 2 transactions one after each other with less than 2-3 seconds to get the ledger to send it twice.
I'm working on a solution to backfill and reindex my archive now. To prevent this in the future, I think some kind of check could be added the the append_transactions function that checks for the existing indexes/batch.

Worked on a deduplicator to solve any incidents like this. Please reach out to me if anyone has the same incident.
Will now work on a fix on the append_transactions to avoid it ever happening again. Will send a PR here when it's ready,.

Fixed by: https://github.com/NatLabs/icrc1/pull/27


Unable to download files

Hello,
I've currently set up my config like so:
{
      resolve: '@fs/gatsby-plugin-drive',
      options: {
        folderId: '1EHtphzknN_vt9QtVjl1AziHu2_p0VEYL',
        keyFile: path.resolve(__dirname, 'client_secret.json'),
        destination: path.join(__dirname, 'src/content'),
        exportGDocs: true,
        exportMimeType: 'application/pdf',
      }
    },

I've got two documents in that folder that I'd like to use. And I have shared those docs with the specified the email but the download seems to be failing. Here's the error:
🚗  Started downloading content
⠸ onPreBootstrap

 ERROR

UNHANDLED REJECTION filesToDownload is not iterable

  TypeError: filesToDownload is not iterable

  - gatsby-node.js:42
    [karanr.dev]/[@fs]/gatsby-plugin-drive/gatsby-node.js:42:22

  - new Promise

  - gatsby-node.js:38 recursiveFolders
    [karanr.dev]/[@fs]/gatsby-plugin-drive/gatsby-node.js:38:10

  - gatsby-node.js:30
    [karanr.dev]/[@fs]/gatsby-plugin-drive/gatsby-node.js:30:5

  - task_queues.js:93 processTicksAndRejections
    internal/process/task_queues.js:93:5

Any idea what's going wrong?

+1.
From what I can tell the array should be the result of: await googleapi.getFolder(folderId, token); but it seems as if the call doesn't return any files/an array. My guess is that googleapi.getFolder() returns an error due to some googleApi setup that we both have, which isn't handled in onPreBootstrap. If that's the case then it might be nice to get the error message in the console rather than the TypeError. But I could be off on this.

Looking in his GoogleAPIs code (src/googleapis.js) he has:
const getFolder = (folderId, token) => {
  return new Promise((resolve, reject) => {
    request(
      {
        uri: `https://www.googleapis.com/drive/v3/files`,
        auth: {
          bearer: token,
        },
        qs: {
          q: `'${folderId}' in parents`,
        },
      },
      (err, res, body) => {
        if (err) {
          reject(err);
        } else {
          resolve(JSON.parse(body).files);
        }
      }
    );
  });
};

when I removed this:
          q: `'${folderId}' in parents`,
        },
      },

I was able to get the files.
I don't know what that query string is supposed to do but it's not right.
I'm using Postman and it doesn't even work there.
I suggest we remove this and we can get the files we want out.

Fixed that issue but now the recursion is all wrong.
He is not responding it seems.
I might take this project as a fork in my own repo and make the needed adjustments as my team needs this ASAP

@stcalica do let me know if you have a fork that's working. I'd like to give it a whirl as well.

@karanrajpal14  I'll have to get around to it.
I'll need to figure out why recursion was needed and how to filter out other files.
Also, another limitation is the export endpoint of Google Drive seems to have a limitation of 10MB.
And I need to pull some videoes. If I can't get that working then I'll have to use something else for my project and put this on the backburner. Will update what I'm doing this week!

@stcalica If I remember correctly, recursion is needed because folders are treated as files. So for each folder, we need to fetch all of the files and folders contained in it, and so on. Hence the in parents query.
I'd be happy to review a pull request and publish a new version, unfortunately I don't have time at the moment to work on this myself.

@fabe  yeah I think they changed their API from when this was written. I can't find a query similar to that anymore in updated docs.
When I ran mine it showed everything flatten. As it showed all my files even ones nested in the folders and then the folders as files. We might just need to remove the folders now since it seems to extract all. But need to do more investigation as my testing directory was large.

@stcalica how have you decided to move ahead with this? I'm thinking on just patching the small change in my local modules and moving ahead with it since I don't really have a very complex folder structure.

@karanrajpal14 yeah I'd do that for now. Might tackle it as one of my "quarantine" goals. I ended up using something else to store my photos and abandoned videos for now since Google Drive won't even store them anyway :/

@stcalica looks like I'm going to be poking around this during the quarantine as well. Mind sharing the src/googleapis.js as a whole after you made the changes? I tried making the change you mentioned and it did not work for me. Could you share your changes so that I can try them out?
Also, what are you using to fetch the photos from Drive?

@karanrajpal14
My changes were to remove the query string only.
From above:
https://github.com/fabe/gatsby-plugin-drive/issues/12#issuecomment-596751582
I'll be playing around with this next week just to help a bit.
But google drive was not the tool for me.

I have encountered this issue after changing the sharing settings on one of the GDrive folderId subfolders. My local netlify dev setup worked properly however this error was thrown when deploying on Netlify.
At first I thought there was some issue with my latest commit however I tried redeploying an older successful build and it failed too.
Finally fixed it by removing and recreating the GDrive folder with a different folderId, and updating the plugin's gatsby-config.js settings accordingly.


New-StoredCredential should also accept securestring as password

I currently do this boilerplate:
try { $cred = Get-StoredCredential $url } catch {}
if ($cred -eq $null) {
    $user = Read-Host -Prompt "Username ($Env:USERDOMAIN\$Env:USERNAME)"
    $pass = Read-Host -Prompt "Password" -AsSecureString
    $cred = New-Object System.Net.NetworkCredential('', $pass)
    New-StoredCredential -Target $url $cred.password
    Remove-Variable cred
}

It would also be easier if Get-StoredCredential would use non terminating error so we could use -ea o instead of try/catch block:
$c = Get-StoredCredential $url -ea 0
if (!$c) {...}

Thanks for the feed back.  I have fixed these things in the development branch.
I have removed all the terminating errors, so all the cmdlets will now correctly respect the Erroracction preference set,  for this specific case where a requested credential is not found, I have actually removed the error completely the command will complete successfully but return no data which would be the expected behavior in powershell.
I have also added an additional parameter to the New-StoredCredential command which will accept a secure string as the password.
This is all currently sitting in the development branch which you are welcome to download and compile if you are comfortable with it.  I will roll these changes in to the next release but have some more work to do before I can do that.

Awesome, thanks. I can compile ofc but this is not a deal breaker so I will wait.


Add support for using custom kickstart file for Anaconda

Describe the bug
We have a workflow for automating the builds of our servers which use predefined Kickstart scripts for CentOS and RHEL servers that I would like to use with osbuild.
I was looking at building a custom ISO file that I could provide our kickstart file into, that would work as a bootable ISO to run the pre-defined Kickstart for the Anaconda installation.
The pre-defined Kickstart scripts we have are pre-validated and used regularly. They include full steps for our disk partitioning (both single and dual-disk layouts), registration for RHEL servers, and triggering post scripts to run Puppet configuration jobs.
I have tried manually creating images using the Red Hat hosted Image Builder tool, but the current customization options are not enough for our requirements.
Allowing us provide a kickstart file would simplify our automated server build process.
Environment

CentOS 7, RHEL 8, RHEL 9
Red Hat Image Builder https://console.redhat.com/beta/insights/image-builder

To Reproduce
Steps to reproduce the behavior:

Try creating any ISO file to build a server using automation tools
Install fails at "Kickstart insufficient"

Expected behavior
osbuild-composer allows us to specify a kickstart file that is passed through to Anaconda to use as the OS installation settings, allowing more advanced automated installations.
Additional context
We are hoping that osbuild will evolve to allow more customization so we can create ISO files to use alongside Terraform vSphere provider to created automated workflow. Creating the VM via Terraform, assigning it the ISO which starts at VM first-boot, and Anaconda installation completes without interruption, and triggers the post Puppet configuration jobs.
Creating VMDK files requires too many additional manual steps and does not fit in the IaC server build model we are working with.

I was talking to a Red Hat consultant who happened to mention Ignition files, which after a quick search turned up this document for RHEL 9.
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/composing_installing_and_managing_rhel_for_edge_images/assembly_using-the-ignition-tool-for-the-rhel-for-edge-simplified-installer-images_composing-installing-managing-rhel-for-edge-images
If the full Anaconda is not preferred, would Ignition files be a possible replacement?
Mainly need more flexibility for disk layouts here, especially the ability to specify a layout for a second disk.

+1


[WIP] SoftButtonManager implementation

Fixes #782
This PR is NOT ready for review.
Risk
This PR makes minor API changes.
Testing Plan

To be added later

Summary
This PR implements the SoftButtonManager as described in the proposal.
CLA

[x] I have signed the CLA

Codecov Report

:exclamation: No coverage uploaded for pull request base (feature/screen_manager@9829fc9). Click here to learn what that means.
The diff coverage is 5.33%.

@@                    Coverage Diff                    @@
##             feature/screen_manager     #827   +/-   ##
=========================================================
  Coverage                          ?   42.13%
  Complexity                        ?     3033
=========================================================
  Files                             ?      385
  Lines                             ?    18349
  Branches                          ?     1893
=========================================================
  Hits                              ?     7731
  Misses                            ?    10245
  Partials                          ?      373

Impacted Files
Coverage Δ
Complexity Δ

...om/smartdevicelink/api/screen/SoftButtonState.java
0% <0%> (ø)
0 <0> (?)

...m/smartdevicelink/api/screen/SoftButtonObject.java
0% <0%> (ø)
0 <0> (?)

.../com/smartdevicelink/api/screen/ScreenManager.java
12.96% <11.11%> (ø)
1 <0> (?)

.../main/java/com/smartdevicelink/api/SdlManager.java
48.12% <40%> (ø)
24 <0> (?)

.../smartdevicelink/api/screen/SoftButtonManager.java
6.66% <6.66%> (ø)
1 <1> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9829fc9...c461f4c. Read the comment docs.


Terminal echo is enabled during system()

If I do : call system("sleep 5"), and then immediately send escape sequences or regular character they are printed to the screen, littering the place where the cursor happened to be.
Is this the intended behavior? If system() meant only for non-interactive executions, then why terminal echo is activate while it executes?
Tested with vim -u NONE, and vim-enhanced-8.1.055-1.fc28.x86_64 build on Fedora Linux 28. Also reproduced by building from source obtained from this repo.

My bad - should have prefixed the command with 'silent'


ALVR Controller mapping request

Hello, depending of the game we need this or that configuration to go in the menu.
Is it possible to add some kind of preset depending of the game we are playing ?
Also a return to default button should be nice to have (I have lost my default configuration).
Anyone has a screenshot of the controller settings view by default ? thanks.

I'm having the same problem. I hope someone could help.

I had to fiddle about with settings. the mapping still isnt great, but the triggers work now.
I dont know if its to do with steamVR but the controllers are still set to look like Vive.
It's as though the controller buttons arent replicating into a customisable format (doesnt recognise certain button presses). I guess this guy is getting smashed with reports.
Also it took dropping the VR and reconnecting from SteamVR to help fix a few problems.
Do you guys have both controllers active at the same time?

I can't say, I am using a Oculus Go which only have one controller. I gess you have the Quest with 2 controller.

I can't say, I am using a Oculus Go which only have one controller. I gess you have the Quest with 2 controller.

Thats right. i assumed you were in the same boat.
Did this work ok in alpha v4? from what i'm reading in posts, when they've tried to integrate the second controller (Alpha 5-quest), things got wierd.
also i think i had to change the buttons in ALVR Server - controller tab
in addition, you can change the controller type in SteamVR to an oculus touch. unsure if that's a rift-S thing or if thats a generic term for oculus controls. from there you can 'test' controls.
But like i said, initially i had to push both X and Y and the menu came up. all i can say is trial and error and something might happen.
This is an Alpha thing so there's bound to be a bunch of ... unusual things happen.


network dns error

Wrong repo and termux is known to have wonky DNS.

thus is start qemu command
qemu-system-x86_64 -hda  nixos.vmdk -cdrom /storage/emulated/0/Download/latest-nixos-minimal-x86_64-linux.iso -boot d -m 8g -netdev user,id=n1,hostfwd=tcp::2222-:22,dns=<IP_ADDRESS> -device e1000,netdev=n1 -vnc :1 -monitor stdio

@everyone


added changelog template (#5608)

Pull request opened by the merge tool on behalf of #5608

Merged into branches iso5 in 71ac285ec03e1045d92cc3ee708c3ce3c2ccee28


Add TypeScript SDK deprecation notice

@jessetrinity : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@gcrev93

#label:"aq-pr-triaged"

#sign-off

@jessetrinity : Thanks for your contribution! The author(s) have been notified to review your proposed change.

How do PRs here get merged? Does the bot do it?

How do PRs here get merged? Does the bot do it?

It gets merged by the approved users (I am not one).

On the public repo, it is an automated process after it's approved and sign-off on by the owner (me). If you used the private repo, which is preferred for employees, then I would approve it and see that it gets through to publish.

#sign-off

@ktoliver : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Device overheating issues - Video Calls

Device overheats during video calls. Problem noted in both Android and iPhone devices.
The video quality is already set to low :- Resolution: 320 × 180; frame rate: 15 fps; bitrate: 140 Kbps
No other video enhancements are being used.
Agora version being used :- agora_rtc_engine: ^5.3.0

Noticed this as well.

Which device you are using? media encoding/decoding naturally costs more than normal mobile activities. The impact of low-end mobile phones will be more obvious.

iPhone 8, One plus 6, One plus Nord are the devices on which the problem was noted without doubt. Will include more devices if applicable.
Also, only 4 people are in a video call at a time. I believe these devices should have no problem processing video calls with such small number of participants.

For me it's Samsung Galaxy S10E (Exynos 9820 CPU) which I use for debugging via USB tethering.
I also use iPhone XS, and I never see it heat anywhere as much, even though the 2 devices have comparable CPUs from the same generation, which might indicate the problem may be Android-specific.
I mostly work with 6.0.0, and at one point while running a test call on 6.0.0-beta.1, I got a notification in the Android tray saying something like "charging stopped because your device is too hot".
I didn't get that specific message since then, and later versions of 6.0.0 seem to do better, but there are other indications that the 6.0.0 isn't as efficient as 5.x.x yet:

CPU usage with 6.0.0 is consistently 25—29%, whereas with 5.x.x it is mostly stays within 15—22% range, on the same codebase and same device.
I get 30% less FPS on the same device on 6.0.0 than on 5.3.0 (I already opened a ticket CUS-26828 about it).

I think it's better to raise a ticket to the agora support, cause you may need to share some sentence data so that we can analyze it further.


Transaction Isolation with HTTPRepositories

HTTPRepositories still do not properly support transaction isolation. See the following pseduo example code:
try (HTTPRepositoryConnection repCon=.......) {
    repCon.begin();
    repCon.add(statement1);
    repCon.getStatements(statement1); //here we will get no result
    repCon.commit();
    repCon.getStatements(statement1); //here we will get a result
}

This issue already should have been fixed according to https://openrdf.atlassian.net/browse/SES-1652 but it seems to still exist in RDF4J 2.2.1

It looks like a test still exists for that sequence, unless it is incompatible with IsolationLevels.READ_UNCOMMITTED:
https://github.com/eclipse/rdf4j/blob/master/testsuites/store/src/main/java/org/eclipse/rdf4j/repository/RepositoryConnectionTest.java#L331
And it isn't disabled by the relevant compliance test (as some tests are in some cases):
https://github.com/eclipse/rdf4j/blob/master/compliance/store/src/test/java/org/eclipse/rdf4j/repository/http/HTTPStoreConnectionTest.java

Just tried it out again and it indeed works. Don't know what my issue was before, this ticket can be closed


issue with dynamic route

dynamic routes, ones with /:customerId don't work when the route is typed directly  (they work when i click link within app).. here's the github repo https://github.com/bellicose100xp/react-input , I hope it's the gulp-connect issue again, but i cannot find anything, is there's something i'm missing.
React.render((
    <Router history={history}>
        <Route path="/" component={App}>
            <IndexRoute component={Homepage} />
            <Route path="/customer/:customerId" component={Customer} />
            <Route path="test" component={Test}/>
        </Route>
    </Router>

The issue seems to be that when I type the dynamic routes directly, the css & script link in index.html see that dynamic route as a baseline so instead of looking for css/bundle.css it's looks for customer/:customerId/css/bundle.css same for bundle.js file.

You need to add a / in front of every static resource.
In index.html change <link rel="stylesheet" href="css/bundle.css"> to <link rel="stylesheet" href="/css/bundle.css"> and <script src="scripts/bundle.js"></script> to <script src="/scripts/bundle.js"></script>.

Thanks for the help, @edvinerikson.

thank you, that resolves the issue, interestingly I had tried ./ but that didn't work.


nixos-generate-config: Properly handle btrfs subvolumes

Currently, nixos-generate-config assumes that if two mount points come from the same device one is a bind mount. This assumption fails for btrfs subvolumes (which differentiate the devices by a mount option)

Isnt't this the same as #1696 ?

I'm not sure it's the same problem but the solution may well be the same.

@shlevy What problems does the assumption cause? And in what way is this assumption made?

This was fixed way back in 87b6232f3a4c9903a432fffd8bc627f2c445a10d

Thanks!


Tab-completed paths aren't highlighted

I have distinct highlighting set for path separators. When tab-completing a path, this highlighting doesn't appear. So, if I manually type
cd ~/Dropbox

then highlighting will show up as expected. But if I then press <TAB>
cd ~/Dropbox/

the path highlighting disappears. At first I thought this was an issue with the trailing /, but if, for example, I press <Ctrl-L> to clear the terminal after tab-completing the above path, syntax highlighting on the path will reappear (including on the trailing slash). So perhaps the problem is that <TAB> isn't calling the highlight function?

I have distinct highlighting set for path separators. When tab-completing a path, this highlighting doesn't appear. So, if I manually type
cd ~/Dropbox

then highlighting will show up as expected. But if I then press <TAB>
cd ~/Dropbox/

the path highlighting disappears. At first I thought this was an issue with the trailing /, but if, for example, I press <Ctrl-L> to clear the terminal after tab-completing the above path, syntax highlighting on the path will reappear (including on the trailing slash). So perhaps the problem is that <TAB> isn't calling the highlight function?

Is z-sy-h loaded last in your zshrc?
What's the output of bindkey $'\t' and of print -r -- $widgets[${"$(bindkey $'\t')"#* }]?

I see the same problem here. This is with a completely empty .zshrc, save for loading antigen and the requisite "antigen bundle" statement for z-sy-h. In addition I've noticed that, when I also turn on zsh-autopair (again, only the two plugins are present in .zshrc), when opening a new pair of quotes or braces, the entries are not highlighted, but are immediately highlighted when text is entered in between the matched pair, or the cursor moves in any way.
To answer the question you posed to rjww:
%  bindkey $'\t'
"^I" expand-or-complete
% print -r -- $widgets[${"$(bindkey $'\t')"#* }]
completion:.expand-or-complete:_main_complete
This is with only z-sy-h loaded.
I can confirm that the problem does not appear in version 0.6.0-rc2-dev; I had updated to 0.7.0-dev yesterday, and noticed it directly.
Anyway, a very minor cosmetic problem, but I guess it bothered me enough to head to github to see if someone else had noticed it, or to see if I perhaps was going crazy :-)

% print -r -- $widgets[${"$(bindkey $'\t')"#* }]
completion:.expand-or-complete:_main_complete

This means the expand-or-complete widget isn't z-sy-h-wrapped.  That's not exactly surprising, but now we know what the immediate problem is.  The expected output is something like user:_zsh_highlight_widget_orig-s0.0000030000-r28743-expand-or-complete (the numbers vary).
Could you bisect?
You may also try the feature/redrawhook branch, which is supposed to be immune to this kind of issue (sorry, forgot to mention it earlier).

Well, I had a go at bisecting, with amusing results. First: the feature/redrawhook branch worked flawlessly; many thanks for the pointer.
I couldn't replicate the "good" result anymore, even with version 0.6.0-rc2-dev (when used with antigen). I noticed, however, that sourcing  zsh-syntax-highlighting.zsh manually would fix the problem immediately. This was even true with an empty .zshrc (save for the antigen bundle statement for z-sy-h): the problem persisted, but sourcing the plugin file (again) fixed it. I recently moved to antigen to manage the few plugins I do use; I had assumed that it, ultimately, simply sourced the plugin scripts in order. In any event, adding an extra "source zsh-syntax-highlighting.plugin.zsh" at the end of my .zshrc has things working as well as before. Ironically, I moved to antigen to make my life simpler.
I'm a little puzzled. In any event, until I can figure out exactly what antigen is doing, it seems reasonable to conclude that the problem does not lie with z-sy-h itself; apologies for wasting your time with this. Thanks again!

You're welcome. Thanks for the information about antigen interaction :-)

I'll close this as a downstream issue, but if you get to the bottom of this do drop us a line.


⚠ St. Lawrence Global Observatory CKAN has degraded performance

In cb08170, St. Lawrence Global Observatory CKAN (https://catalogue.ogsl.ca/en/) experienced degraded performance:

HTTP code: 200
Response time: 17957 ms

Resolved: St. Lawrence Global Observatory CKAN performance has improved in 74cf01d after 5 minutes.


What is %C stand for

In your manual page, I saw "%C" which means "The cache status of the object the server served." as SPECIFIERS. But I still don't understand what it is refer. I try to use it as Cache-Control(like "no-cache, must-revalidate, max-age=0"), but it didn't work, because in "THE CACHE STATUS OF THE OBJECT SERVED" of the output html, it said "No data on this panel."
I'm using server litespeed and the log is the same as apache, so what is the "%C"SPECIFIERS in Apache LogFormat? I need help, much thanks.
PS: I'm Chinese, so my English maybe poor, sorry.

The cache status pertains to various states that cached data can take on, such as HIT, MISS, STALE, among others. I don't see any cache status in your log though.

That's it. I missunderstood it. Very thanks!


Don't allow implementationOnly with package visibility.

Follow up to #1472, looks like we didn't realize this gate should also be updated.

@gjcairo fyi.


how do you work out why company registration is always being rejected?

we are attempting to register the company and have what we believe to be the correct information - but it keeps getting rejected...

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 726a5c98-ba0b-38a0-d86e-2d5df4745d1a
Version Independent ID: 643673cc-0b57-86f6-e00d-13aad2ef8d20
Content: Opening a developer account - UWP app developer
Content Source: windows-apps-src/publish/opening-a-developer-account.md
Product: uwp
GitHub Login: @JnHs
Microsoft Alias: wdg-dev-content

For help with this please open a support request at http://aka.ms/storesupport - under "Contact us" choose "Dashboard issue" and "Account setup & management".


Console, select (with options) return key, not value.

$actionName = $this->select('You want add or remove role?', ['remove' => 0, 'add' => 1]);
var_dump($actionName)

expected '1' or '0' (as int), but it return raw input ('add' or 'remove' as string)
In class (vendor)
namespace yii\helpers;

use yii\console\Markdown as ConsoleMarkdown;

/**
 * BaseConsole provides concrete implementation for [[Console]].
 *
 * Do not use BaseConsole. Use [[Console]] instead.
 *
 * @author Carsten Brandt<EMAIL_ADDRESS> * @since 2.0
 */
class BaseConsole
{
//...

    public static function select($prompt, $options = [])
    {
        top:
        static::stdout("$prompt [" . implode(',', array_keys($options)) . ',?]: ');
        $input = static::stdin();
        if ($input === '?') {
            foreach ($options as $key => $value) {
                static::output(" $key - $value");
            }
            static::output(' ? - Show help');
            goto top;
        } elseif (!array_key_exists($input, $options)) {
            goto top;
        }

        return $input;
    }

//...
}

This function return raw stdin(), but expect "value from options array", should it?
Is this logic correct?

U have the options inverted?

$input = static::sdin() // string
Function select return $input.
$options is array [key=>value].
So when user input stdin - it should return value from $options by key (stdin), same way as dropdown works.
In our case we output cases for select
        static::stdout("$prompt [" . implode(',', array_keys($options)) . ',?]: ');
and return this key, but it should return value from options array by this key.
PS: Option inverted? Maybe, but it change nothing, because function select doesn't return "value by key" it just return "key".

As @TerraSkye noted, you have options inverted. Worth improving docs though.

Only docs? What? Are we looking on save code? Again look what function select return and what you input. And what it shows in console.

Tell me, where in this code is return $options "value"  ?
    public static function select($prompt, $options = [])
    {
        top:
        static::stdout("$prompt [" . implode(',', array_keys($options)) . ',?]: ');
        $input = static::stdin();
        if ($input === '?') {
            foreach ($options as $key => $value) {
                static::output(" $key - $value");
            }
            static::output(' ? - Show help');
            goto top;
        } elseif (!array_key_exists($input, $options)) {
            goto top;
        }
        // HERE SHOULD BE SOMETHING LIKE
       // return $options[$input]; <<<-----------------------------

        return $input;
    }

I see that it returns only "key" that exists in $options array.

static::output(" $key - $value");

$key here is what's inputed and what's used. $value is what is displayed to end user by help command.

Ok, i got it. But I thought that you did it like dropdown.


I am trying to deploy the application with this script from circle CI.

It is failing after this...What could be the possible reason?

Do you have the full log in the CLI? what happens next?
If it's stalling in the in-progress --- next step could be to potentially check the AWS code deploy logs.


Azure Service Bus (Batch Publish)

In the Azure Service Bus extensions for MassTransit I not found any way to batch publish. Have any way to do this?
I've created a lab project and measured the duration time between AzureServiceBus native library and MassTransit.
Scenario: Sessions: 2, total messages: 200 (100 per session)
Results:
Batch publishing with Azure Service Bus (native library):
elapsed time 00:00:04.6412575
Publishing with MassTransit:
elapsed time 00:00:34.3397763

There isn't a way to batch publish with MassTransit.

@ircnelson - from what I know, the new .Net Core Azure Service Bus library does not support true batching over AMQP (receive batching is faked via Prefetch). On the sender side there seems to be no batching at all via AMQP and wondering how this would be implemented in MT?

@MedAnd - In my tests I used Azure Service Bus Core. Sorry for not keep it clearly.
I confess that I unknown what happen in the bare metal.
I only used SendAsync(Message[]) overload that the library provides.


PLoRA disappears when applying lora.

hello.
When I apply lora to the decoder, the existing model structure, PLoRA, is converted to linear.
I don't think this will utilize the existing PLoRA that you have learned.
Is this okay?

Before applying lora

After applying lora

This issue has been fixed by the commit 3c522e7. You can pull the latest version of fine-tune code to check if the problem still exists.

@yunsangju

@yuhangzang @myownskyW7
hello.
When I proceed with the code below, PLoRA still disappears.
for name, param in model.model.named_parameters():
            param.requires_grad = False
        lora_config = LoraConfig(
            r=lora_args.lora_r,
            lora_alpha=lora_args.lora_alpha,
            target_modules=lora_args.lora_target_modules,
            lora_dropout=lora_args.lora_dropout,
            bias=lora_args.lora_bias,
            task_type='CAUSAL_LM',
        )

        model = get_peft_model(model, lora_config)


Issue #740. Reconnect  Accumulo options to turn off statistics

Confused about this pull request.  It seems like it has a commit that I did in it, that's unrelated ...?


Create easy-arceuus-runecrafting

Contextual highlights and screen flashing for Arceuus Runecrafting.
Highlights the clickbox of the next thing to click when doing arceuus runecrafting. (essence, altars, chisel, shortcuts, etc.)
Configurable colors, flashing, flashing speed, etc.
Optionally tint the screen in different colors when idle and/or active.
Running out of steam to make handling of soul runes and lower agility levels bulletproof for v1; hopefully that's fine. I've called out the limitations in the README.

The runelite-github-app bot may need to be re-triggered; I was shadowbanned at the time this PR was created.

just git commit --amend --no-edit && git push origin head -f on the phub repo branch you pr'd.

Should be good now; still need a maintainer to approve the bot workflow.


Unable to install extensions in a conventional manor

Describe the bug
I am unable to add a extension to ungoogled chromium via chrome://extensions with developer mode
I beleive this is because un-chrome does not have a proper extension dir?
To Reproduce
Steps to reproduce the behavior:
https://i.imgur.com/a8rvQhQ.png

Download a CRX file from any source,
go to chrome://extensions
enable dev mode
drag and drop teh CRX to the page

Expected behavior
I expected the plugin/extension to install normally like it does on actual chrome
Screenshots
Environment (please complete the following information):
https://i.imgur.com/a8rvQhQ.png
Additional context
Currently, i have to extract the CRX with 7zip (turns out they are just standard zip files)
And "Load Unpacked" every time i want to install a extension,
The attached GIF shows the issue in a side by side comparison
My Thoughts?
I think we could apply a patch to a but load of files to replace the string
clients2.google.com/service/update2/crx with something of our choosing, so we dont have to worry about a chrome webstore, maybe allow CRX installs from github?
https://cdn.merith.tk/bandicam 2019-11-13 17-28-04-002.gif < this may take a few seconds to load, i have slow internet

#834

This is a duplicate. Thanks @qvint.


Adapter - Persistence - JDBC - Stored Procedures

Add a module adapter-persistence-jdbc-sp. It should implement persistence using Postgres but not using JPA, instead using JDBC. Use raw stored procedures to execute commands.
For reference when creating the module, see adapter-persistence-jpa and adapter-persistence-redis.
So here you'll be using JDBC + Stored procedures.

hi @valentinacupac, i'm working on this issue 👍

Thanks @keremh, I assigned it to you.


[BUG]: Compliance Detail Request Flow Notification

Describe the bug
A clear and concise description of what the bug is.
Component (please tell us which flow or app you are experiencing issues with):

Compliance and Report Component
Version [e.g. 1.29]
Flow- Compliance Detail Request

Bug
I have multiple apps for which I never received any email related to Compliance Detail Request . When I shared the App with a security group, then I was notified ; which is an expected behavior . Our concern is , when we create an app for the first time ; we expect the business justification email to be send at that time itself along with Welcome Email or separately so that user fill the details at that time itself .  This notification is triggered only when app is shared broadly or with a group which makes this flow of no use. What if users are creating multiple apps but not sharing it with a group or more than 20 users, in that case they will never get the notification for filling business justifications and hence they would never fill the details . Second , it is almost impossible to give premium licenses to all users . Is there no other way to handle this ?
Screenshots

There is no "one size fits all" solution for a Center of Excellence (CoE). Some companies will want a very restrictive set of rules on their organization in hopes of mitigating the unknown, while others will want to let users personally explore without limitations. Because of this, the CoE Starter Kit doesn't come equipped with a set of design patterns for everyone. Therefore, if your organization wants a more restrictive implementation, it must be implemented in addition to these tools.
The CoE Starter Kit is a set of templates to help you get started, and - when you are ready - can be extended to fit your specific requirements, read about extending the solution here https://docs.microsoft.com/en-us/power-platform/guidance/coe/setup#extending-the-starter-kit
You can read about the Sample Audit Process here https://docs.microsoft.com/en-us/power-platform/guidance/coe/example-processes and modify the necessary flows to send notifications according to your requirements, and take inspiration from the Developer Compliance Center but replace it with a data source that doesn't require a premium connector - you could even use Microsoft Forms for this!
The CoE Starter Kit team provides these templates to share best practises and provide a starting point and helping hand to organizations getting started with setting up a CoE, but detailed implementations would be the responsibility of the organizations - sometimes this is also accomplished by working with a Microsoft Partner https://powerapps.microsoft.com/en-us/find-a-partner/


Build failed on the windows

When I try to run on the windows, these errors happen.
SEVERE: error[E0433]: failed to resolve: could not find Resource in rlimit
SEVERE: error[E0425]: cannot find function getrlimit in crate rlimit
Exception: Build process failed.

Thank you for reporting @H3mnz. Seems to be happening because some of our utility functions (to increase the number of open files) are not available on Windows.
Since this is not strictly necessary on desktop platforms I'll see if it's possible to omit this functionality.

I don't actually have a Windows machine to test this. @H3mnz can you please try using the branch from https://github.com/Foundation-Devices/tor/pull/30?


[SPARK-31064] [SQL] New Parquet Predicate Filter APIs with multi-part Identifier Support

What changes were proposed in this pull request?
Parquet's org.apache.parquet.filter2.predicate.FilterApi uses dots as separators to split the column name into multi-parts of nested fields. The drawback is this causes issues when the field name contains dots.
The new APIs that will be added will take array of string directly for multi-parts of nested fields, so no confusion as using dots as separators.
Why are the changes needed?
To support nested predicate pushdown and predicate pushdown for columns containing dots.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Existing UTs.

Thank you for narrowing down.

Test build #119442 has started for PR 27824 at commit f59b2b4.

BTW, previously, you create a type for PartialFunction[ParquetSchemaType, (String, Any) => FilterPredicate]. Did you decide to drop that in this PR?

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24178/
Test PASSed.

In general, the logic looks fine to me. Let's see the Jenkins result.

Test build #119444 has started for PR 27824 at commit 60ff198.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24180/
Test PASSed.

Test build #119442 has finished for PR 27824 at commit f59b2b4.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public final class SparkFilterApi

Test build #119444 has finished for PR 27824 at commit 60ff198.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119444/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119442/
Test FAILed.

Retest this please.

retest this please

Test build #119473 has started for PR 27824 at commit 60ff198.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24204/
Test PASSed.

Test build #119473 has finished for PR 27824 at commit 60ff198.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119473/
Test PASSed.

Thanks all for reviewing. Merged into master / 3.0


feat(steps): handle unknown category for all encoders

Unknown categories are currently ignored in the current encoding implementations. While we should consider adding an option to handle this in the future, it's not a high priority at the moment.
Open an issue to record this for future consideration.
The current implemenations:

CategoricalEncode will convert unknown category to None
OneHotEncode will convert all encoded cols to 0, see following example.
CountEncode will convert unknown category to 0

For example:
>>> import ibis
>>> import ibisml as ml
>>> import pandas as pd
>>>
>>> t_train = ibis.memtable(
...         {
...             "time": [
...                 pd.Timestamp("2016-05-25 13:30:00.023"),
...                 pd.Timestamp("2016-05-25 13:30:00.023"),
...                 pd.Timestamp("2016-05-25 13:30:00.030"),
...                 pd.Timestamp("2016-05-25 13:30:00.041"),
...                 pd.Timestamp("2016-05-25 13:30:00.048"),
...                 pd.Timestamp("2016-05-25 13:30:00.049"),
...                 pd.Timestamp("2016-05-25 13:30:00.072"),
...                 pd.Timestamp("2016-05-25 13:30:00.075"),
...             ],
...             "ticker": ["GOOG", "MSFT", "MSFT", "MSFT", None, "AAPL", "GOOG", "MSFT"],
...         }
...     )
>>> t_test = ibis.memtable(
...         {
...             "time": [
...                 pd.Timestamp("2016-05-25 13:30:00.023"),
...                 pd.Timestamp("2016-05-25 13:30:00.038"),
...                 pd.Timestamp("2016-05-25 13:30:00.048"),
...                 pd.Timestamp("2016-05-25 13:30:00.049"),
...                 pd.Timestamp("2016-05-25 13:30:00.050"),
...                 pd.Timestamp("2016-05-25 13:30:00.051"),
...             ],
...             "ticker": ["MSFT", "MSFT", "GOOG", "GOOG", "AMZN", None],
...         }
...     )
>>> step = ml.OneHotEncode("ticker")
>>> step.fit_table(t_train, ml.core.Metadata())
>>> res = step.transform_table(t_test)
>>> res

AMZN in the 5th row is unknown, it will be translated to all 0s
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ time                    ┃ ticker_AAPL ┃ ticker_GOOG ┃ ticker_MSFT ┃ ticker_None ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ timestamp               │ int8        │ int8        │ int8        │ int8        │
├─────────────────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│ 2016-05-25 13:30:00.023 │           0 │           0 │           1 │           0 │
│ 2016-05-25 13:30:00.038 │           0 │           0 │           1 │           0 │
│ 2016-05-25 13:30:00.048 │           0 │           1 │           0 │           0 │
│ 2016-05-25 13:30:00.049 │           0 │           1 │           0 │           0 │
│ 2016-05-25 13:30:00.050 │           0 │           0 │           0 │           0 │
│ 2016-05-25 13:30:00.051 │           0 │           0 │           0 │           1 │
└─────────────────────────┴─────────────┴─────────────┴─────────────┴─────────────┘

CategoricalEncode will convert unknown category to None

I haven't looked into whether this is correct or not, since the step doesn't have any tests; we should definitely add one, and then can identify the correct behaviors. 😅

OneHotEncode will convert all encoded cols to 0, see following example.

Having a separate category for unknown (i.e. rest of the encoding column values are all 0) could be a nice option to provide/give the user more flexibility, but may not matter for a lot of model types (e.g. GBDT).

CountEncode will convert unknown category to 0

This is intentional, as the count should be 0 for something that has not been seen.
To me, seems like the immediate action item is to add a test for CategoricalEncode to make sure it's functionality is correct, and to (at lower priority) make the OneHotEncode unknown category handling a bit more flexible.

@jitingxu1 closing this; if want to make OneHotEncode unknown category handling a bit more flexible, feel free to create a new issue, but it doesn't seem to be a priority at this time.


Image assets

How you guys use images in components with Este?
When I add image.png into /assets folder and then add  into component, it doesn't load. How should I do this?

It's simple express static middleware, check it https://github.com/este/este/blob/1718941b494308bea3fafe717c31880bc9e5add0/src/server/frontend/index.js#L16

I've added my image into assets img folder:
este/assets/img/image.png
Inside este/src/browser/app/Header.react.js component I have:

But this doesn't work. What am I doing wrong?

The path in este master express.static('../assets/img' does not work for me. This does work for me:
server/config.js
const config = {
  basePath: path.normalize(path.join(__dirname, '../..')),

server/frontend/index.js
const assetsDir = path.join(config.basePath, 'assets');
app.use('/img', express.static(path.join(assetsDir, 'img'), {maxAge: '200d'}));

And then it is:

Thank you @rkoberg that works!

+1 @rkoberg  big thanks!

@rkoberg PR please?

OK. Do you want it with the basePath added to src/server/config? Or just correct the path in src/server/frontend/index.js

The most simple minimal PR. Do we need basePath anything elsewhere? I don't think so, so fixing paths + some explaining comment is good enough.

Hey, should I commit an image and add html for it? ( https://placeholdit.imgix.net/~text?txtsize=6&txt=50×50&w=50&h=50 or whatever you like )

Yes please, add some small image to assets. Thank you!

Fixed in #598. Thank you!


Prepare README for 2.0

This PR updates the readme.md to reflect the API changes introduced mostly in https://github.com/rackt/redux-simple-router/pull/141 by @taion.
It also adds a little section for server-side usage - explaining that the memoryHistory has to be hydrated with the current location from the server for redux-simple-router to determine the correct initialState.

Thanks a lot for this! I'll merge one you remove that section. That's not the right place for an implementation detail; for server-rendering I'd rather link to proper example once we get there.

Removed, rebased, squashed :)
Server rendering works pretty neat for me once I figured out that I have to hydrate a memoryHistory and pass that to match for redux-simple-router to determine a non-empty initial state. But you are right - perhaps I'm able to create a failing test for this in the following days.

Personally I update the state after creating the server-side store with a routeActions.push action. We should have better docs about all this but let's get 2.0.0 out first. Thanks!

@jlongster routeActions is undefined in the current npm package. happy to file a separate issue…

Nope, I'm about to release 2.0.0 so we updated the readme in preparation for it.

I sorta figured that out as the day went on :). Using 2.0 on dev right now in anticipation of the release, so I should be all ready.

Is the 2.0.0. release scheduled for today?
The lack of routeActions on the npm release was a really unexpected hold up on my development.
Should I wait for 2.0.0. (as in a few hours) or must I use an older version?
(I know this question would be better off in a separate issue, but since it has been brought up here and @jlongster said it should not be dealt with since 2.0.0 is about to be released, i figured i'd should just post it here)

Yes it will be released today.


Merged with ExoPlayer dev-v2 on 20181203

Hi @tresvecesseis @portizb - please find a merged & tested update to your dev-v2-rtsp branch.
I was able to play a Dash Cam video stream, but found connection to be unstable / delayed. The connection will keep dropping and restarting.
(Happy to provide you with more debug logs or even a hardware sample).

Updated once more to reflect latest changes. @portizb @tresvecesseis happy to hear your comments


Provide explicit selection of licenses when uploading image

I've received feedback from a few users (on Google Play and on Commons) that they did not like not being able to select the license when uploading. Users can currently select a license in Settings, but this isn't very clear.
Thoughts on putting a license selection field in the title/desc screen?

Why not. It could also act as a reminder that the file must be compliant with that license.

Note: This should probably be done alongside #254

I just realized that this has been previously discussed at #140 . There are a few potential options I guess:

No explicit license selection on upload screen, but do the one-time popup thing explaining what the default license is and how to change license selection
License selection on upload screen but it defaults to the selection that user made previously (so no need for him to select it every upload)
Have a link to Settings page ("Change license here") below the small license text in upload page (and maybe make that text bigger)

@nicolas-raoul , any thoughts on which option is preferred? I'm leaning towards option 2 or option 3 personally.

Option 2 sounds perfect to me :-)


App crashes when opening books

Whenever I open an ebook (ePub3, confirmed by both Sigil and Calibre), it shows a cream page with the bookmark options on the bottom screen. When I press any button(s) or tap anywhere on the touch screen, it crashes. Luma says the crash was in the '3ds ebook reader' (or something similar) process, so it's definitely this causing it. In Luma's crash exception handler screen it also says it is currently performing 'Translation - Section'.
Sorry if this is unclear, I'll do it again and clarify below.
Shaymin

An excerpt from the exception handler screen:
Processer: ARM11 (core 0)
Exception type: data abort
Fault status: Translation - Section
Current process: eBook Re (000400000EB11100)
I presume the 'eBook Re' is due to limited characters. The issue could be related to eBook size; is there a limit?
Shaymin


Convert Bezier integrals from static differential to using derivative

Completed

Merge derivative function into jacobian
Use differential inside Bezier curve integral methods
Add a new @testitem section for BezierCurve with analytic solution
Add jacobian to the Documenter site API page

I was just thinking: Could we rename derivative to jacobian, i.e. add a method to jacobian, such that we can reuse differential within the Bezier specialization instead of needing norm(derivative(...))? Then, of course, the new jacobian method (which is now derivative) would also get  ts instead of t (where we would obtain t by t = only(ts)) and would need to return a one-element vector.

Didn't see you were making edits and just pushed a couple of commits. I have to step away but will check back in a bit to try to sort it out.

Oh, I'm sorry. I didn't want to disturb your development :grimacing:

No worries! I'm glad to have the help, just got distracted over in VS Code and was surprised to see help had arrived.

I just added a @testitem section with new tests that have an analytic solution. I think the core issue we see with test results being non-exact is that these Bezier curves are merely splined/interpolated approximations of the ideal curves. Depending on how the error tolerances stack up, I may need to just lax the rtol or atol a little.

I was just thinking: Could we rename derivative to jacobian, i.e. add a method to jacobian, such that we can reuse differential within the Bezier specialization instead of needing norm(derivative(...))? Then, of course, the new jacobian method (which is now derivative) would also get ts instead of t (where we would obtain t by t = only(ts)) and would need to return a one-element vector.

This is a really good idea. I think part of the trouble I had conceptually with extending (partial) derivative was in trying to generalize it past 1D geometries, i.e. what does the API look like to request the partial derivative w.r.t. the N'th parametric coordinate? Whereas for the 1D case you can just ignore that bit and return a simple Vec.
In my mind, then, we've got
jacobian(geometry::Meshes.Geometry, ts)  # finite-diff fallback
jacobian(curve::Meshes.BezierCurve, ts)  # analytic solution from today's derivative
# etc (as implemented)

and can interface them the same in all integral functions.

Pending this latest commit passing CI, this should be good to go.


pipenv support

Context
One of my personal annoyances with Zappa is how it packages the project directory kind of blindly. I would like to see Zappa support (and in fact encourage) packaging projects with a Pipfile, and letting Zappa take care of creating the venv and downloading all the packages (using pipenv).
References:

https://docs.pipenv.org
https://github.com/pypa/pipenv

Thoughts @Miserlou?

Related: https://github.com/Miserlou/Zappa/pull/1435
I haven't had a chance to review yet.
Also related:
https://github.com/Miserlou/Zappa/issues/525
https://github.com/Miserlou/Zappa/issues/905

i use pyenv + pyenv-virtualenv + pip-tools in my workflow. I have a requirements/dev.(in/txt) in my local and requirements/prod.(in/txt) for deploy in production. So i change manually the packages deployed when i am going to deploy my apps, I am thinking to use a LXD container that do the deploy automatically with requirements/prod.txt meanwhile i am using requirements/dev.txt in my laptop.
If we use pipenv, something similar will be needed, only deploy [packages] from main Pipfile, ignoring the packages in [dev-packages].

For my CI jobs I use the following setup
- export PIPENV_VENV_IN_PROJECT=true
- pip install pipenv
- pipenv install
- export VIRTUAL_ENV=.venv/

This setup tells pipenvto create virtual env inside current directory at ./venv folder.
Zappa understands it and then I do a simple update
- pipenv run python manage.py migrate --settings=admin_dashboard.settings.prod
- pipenv run zappa update prod

@Miserlou, you've probably looked into this already but #1435 just changes zappa itself to use pipenv, doesn't add the option to use pipenv.


Accept output from one Katana run to parse as a diff against another run

I would like to have the output of one katana run be able to be fed into another katana run and produce a diff from the first run.
The workflow would be: 1. Run katana against site, save output 2. Pass output as a difference flag to another katana instance and produce only the differences between the first run and the second run.
Why would I like this? It would make it much easier to parse A/B testing on sites from various source IPs. I could do one run from us-east and then a second run from us-west to see what/if is different between the two runs from the two locations. On the second run from us-west I’d like to see just the diff from us-east output.

Might be a follow up of https://github.com/projectdiscovery/katana/issues/13


Activating Google Tag Manager container for wellcomecollection.org

Google Tag Manager was used previously for tagging data across wellcomecollection.org and wellcomelibrary.org. The container snippets should be deleted and replaced with a new snippet.
The old snippet in the head was:

The old snippet in the body was:

Here is a link to the snippets to install for the new wc.org only container: https://tagmanager.google.com/?authuser=0#/admin/accounts/6000872529/containers/31009043/install

In order to use Google Tag Manager, a data layer will need to be set up. Here is a link to the snippet for this:
https://developers.google.com/tag-platform/tag-manager/web/datalayer?hl=en_US

Moving this back to the backlog for now as it's a large (and important) change, see thread in Slack https://wellcome.slack.com/archives/C3TQSF63C/p1664797960502819

Thinking out loud
We currently use Universal Analytics and GA4 without Google Tag Manager and could continue to do so. However, we've agreed that Google Tag Manager will provide certain benefits that make it worth adopting. Not least that it empowers Tacey to set up and remove event tracking without the need for developers to be involved, and all that that entails.
The recommended way of using Google Tag Manager, with Google Analytics, is to install Google Tag Manager and then use it to install Google Analytics. However, we already include the google analytics script (gtag.js) for both our GA4 and Universal Analytics, and track certain events, within the website code.
In order that we don't duplicate and/or lose data, we need to proceed carefully.
I can see a few approaches (option 3 being my favourite).
1. Only use Google Tag Manager
We add the Google Tag Manager code to the website and remove the Google Analytics code at the same time.
At present the Google Tag Manager account seems to be set up to include the Universal Analytics and GA4 tags:

This would be the most risky option, in terms of losing data. For example, if the UA and GA tags in Google Tag Manager don't work as expected. We also have a lot of code within the application for tracking events and I don't know how these would be affected by doing this. We could remove this event tracking code too and try and do it in Tag Manager instead, but that seems like a lot of moving parts and may not be possible for certain things like the performance metrics.
2. Take a short term hybrid approach
It is possible to include Google Tag Manager along side Google Analytics in the website code. In this case we would need to be sure that we didn't double track things.
For example, we'd need to remove the UA and GA4 tags from our Google Tag Manager account before we implement Google Tag Manager on the website. We'd also need to make sure we didn't imnplement event tracking via Google Tag Manager for things that are already being tracked in the website code.
We could then move in the direction of removing the analytics code from the website to the scenario of only having the Tag Manager code. We could move the events we care about across to Tag Manager in a more piecemeal fashion. At the same time we could send these events to GA4 rather than UA and remove the events we no longer care about altogether. As a last step we would remove the Google Analytics scripts and implement them in Google Tag Manager and would only need to implement GA4.
3. Take a long term hybrid approach
Basically the same as option 2, but with no intention of ever removing the gtag.js analytics scripts from the website. We might also want to keep certain events, like the performance metrics in code. At the moment this is my preferred option.

@jamieparkinson and @taceybadgerbrook I would appreciate your thoughts on the above.

@gestchild there are 2 GTM accounts that I can see. GTM-53DFWQD is not a live container. It contains changes but these are the result of me trying to do training on a container that isn't live. None of these changes have been published, so we can throw this away without doing any damage.
The other account is called Wellcome Collection but it has no GTM container.

ok, GTM-53DFWQD was the one linked to at the top of this issue as the one to install. Do we still want to do that? If yes, we'll just need to tidy it up a bit and remove the analytics tags. If no, you'll need to let me know the one you want to use.
Do you know why we have 2 and can we delete one of them?

@gestchild Ok, I've cleared out everything in GTM-53DFWQD.
The other account was used to track across wellcomecollection.org and wellcomelibrary.org. The stats it produced looked wrong and so the decision was taken to discontinue use of this. There is no container to delete so I'm not sure how to delete the account.

Oops, yes, will correct original comment

I think they'll be events we'll always need to trigger in code, such as the Next.js reportWebVitals stuff

hmmm - on the contrary I think we should be updating and unifying these events, and that this is a good opportunity to do so. eg for web vitals there is a recommended way to do it with GA4 and GTM.
Another way to say this - if it's true that we'll always need some events that are triggered in code, I don't think this automatically means we'll need gtag.js to do this. Disadvantages of keeping both would obviously be increased cost of maintenance, higher risk of regressing to use gtag for "too much" in future once this context is lost, and increased overhead of JS sent to clients.

That's a fair point and I'm all for moving everything out of the code if we can, I was thinking we'd miss the Nextjs custom metrics, but we don't actually do anything with those currently so maybe that doesn't matter.

@taceybadgerbrook will you be happy setting up the Web Vitals metric tracking when the time comes.

If we got with 2, the result of the work means that as well as empowering Tacey to do tracking stuff without developers, it also moves all the responsibility onto those who know how to/want to use Google Tag Manager (which at the moment is @taceybadgerbrook and possibly @jennpb?). I just want to be clear that everyone is happy with that.

Can I summarise what I think we're agreeing?
We're moving all tracking to GA4 via GTM including

web vitals via the Core Web Vitals custom template mentioned here
component-level events that we want to keep (subject to review with @jennpb @taceybadgerbrook and @DominiqueMarshall )

And that moving the component-level events into GA4 via GTM can be done independently of setting up the GTM container + GA4?
I have some questions about what out-of-the-box events are already included in GA4, and whether there's duplication of what we currently track?

That all sounds right.

And that moving the component-level events into GA4 via GTM can be done independently of setting up the GTM container + GA4?

Yes,  I think we'll add GTM, shift over the events, then remove the gtag (analytics code) and implement that in GTM too.

Tag Manager was added in #9174 and we now use GTM to add the GA4 tag


Add extra configmap labels and annotations to istiod helm chart

Signed-off-by: Ethern Su<EMAIL_ADDRESS>Please provide a description of this PR:
I added extra labels and annotations config to dynamically loaded configmaps (i.e. istio and istio-sidecar-injector) in istio-discovery helm chart to increase the configuration flexibility. Refer to https://github.com/istio/istio/issues/40571 for detailed reason and consideration.
To help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ ] Configuration Infrastructure
[ ] Docs
[X] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Please check any characteristics that apply to this pull request.

[ ] Does not have any user-facing changes. This may include CLI changes, API changes, behavior changes, performance improvements, etc.

Hi @ethernoy. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

@ethernoy: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ethernoy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test unit-tests-arm64

unit-tests_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ethernoy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test unit-tests-arm64

unit-tests_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test unit-tests

gencheck_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ethernoy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
62e1880d6b7d13fd444e7f982e700303ca986ff6
link
true
/test unit-tests

gencheck_istio
9bef3f95c231294b11f85b72ad86ca396e748a47
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ethernoy: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Allow to run day2 only

Demo example: https://asciinema.org/a/580457

/retest

Comments fixed.

The test fails because of race condition in tests, it's fixed here: https://github.com/stolostron/cluster-templates-operator/pull/121/files#diff-72dc2ce54223c3c017525e285df9d5a5115813da973fd5a4b3502a2b12dec6c5R61

Rebased


Unable to download any sheet music from Musescore. Thanks!

Unable to download any sheet music from Musescore. Thanks!

I had the same problem as you but I played around with Tampermonkey Beta and figured out a simple solution to make it work.
Disable Tampermonkey before accessing the page with the sheet music you want, then enable Tampermonkey and CLICK REFRESH.
Download your file then disable Tampermonkey and CLICK REFRESH before leaving page to get another sheet, repeat the same steps for each file.
Works for me so far.... good luck.

Hi,
Many thanks for your comments.
In fact, I was going to try it and disabled the Greasemonkey Musescore downloader..... and it worked anyway! Strange! So, now it’s working with the disabled script, but I will keep in mind your advice.
Take care.
Jordi
De<EMAIL_ADDRESS>Enviat: divendres, 4 de desembre de 2020 5:04
Per a<EMAIL_ADDRESS>A/c<EMAIL_ADDRESS><EMAIL_ADDRESS>Tema: Re: [Xmader/musescore-downloader] Unable to download any sheet music from Musescore. Thanks! (#97)
I had the same problem as you but I played around with Tampermonkey Beta and figured out a simple solution to make it work.
Disable Tampermonkey before accessing the page with the sheet music you want, then enable Tampermonkey and CLICK REFRESH.
Download your file then disable Tampermonkey and CLICK REFRESH before leaving page to get another sheet, repeat the same steps for each file.
Works for me so far.... good luck.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/Xmader/musescore-downloader/issues/97#issuecomment-738551363, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AR7RAZVD33BEIO3K5CKP4GLSTBNUXANCNFSM4UMC6ZQA.

Please close the issue if your problem is solved.

It seems to work sometimes, sometimes not.


ESM broken

What were you trying to do?
Use pdf-lib in node 18 with esm, esbuild (esm) to bundle, and run locally with nodejs in esm.
How did you attempt to do it?
import {PDFDocument} from "@cantoo/pdf-lib"

import PdfLib from "@cantoo/pdf-lib"
const {PDFDocument} = PdfLib;

import * as  PdfLib from "@cantoo/pdf-lib"

What actually happened?
Outcome of the import patterns in order

node 18 complains that the module is CJS and destructuring isn't supported
works for node, but esbuild complains that there is no default export for es/index.js
works for esbuild, but at runtime with esm on nodejs, all of the exports are undefined (because there is no default export).

What did you expect to happen?
I expect at least import pattern 1 to work.
How can we reproduce the issue?
index.mjs:
import { PDFDocument } from "@cantoo/pdf-lib";
PDFDocument.create();

{
  "name": "pdf-lib-sample",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "devDependencies": {
    "@cantoo/pdf-lib": "^1.19.11",
  },
  "author": "",
  "license": "ISC"
}

npm i
node index.mjs

Version

=1.19

What environment are you running pdf-lib in?
Node, Other
Checklist

[X] My report includes a Short, Self Contained, Correct (Compilable) Example.
[X] I have attached all PDFs, images, and other files needed to run my SSCCE.

Additional Notes
For typescript to recognize the es/index.js import, it needs a package.json with { type: "module" } in the directory at a minimum.
For node, it needs the new "exports" block in package.json
  "exports": {
    ".": {
      "types": "./cjs/index.d.ts",
      "require": "./cjs/index.js",
      "import": "./es/index.js"
    }
  },

Additionally, the es/* directory imports need to be updated to use full file paths.

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '[redacted]/pdf-lib-sample/node_modules/@cantoo/pdf-lib/es/core/errors' imported from<EMAIL_ADDRESS>
Is fixed by changing es/index.js to:
export * from './api/index.js';
export * from './core/index.js';
export * from './types/index.js';
export * from './utils/index.js';

But needs to be repeated all of the way down.
My current resolution is to downgrade back to 1.17, which does not have this issue.

I'll be on it. Sorry for the delay.

@thantos Did the prepatch fix the issue? I'll be releasing version 1.19.13 now.

Haven't had a chance to try yet, will try later today.

I released version 1.19.13. I decided to embark the change, as the current version is broken anyway. Not much risk to break anything, right?

Assuming this is related:
Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'color.js' imported from<EMAIL_ADDRESS>

Ah, indeed. Let me fix that

Fixed in 1.19.14

Its working for me so far, thanks for the fix!


routes does not work

e.g the rsshub route:https://rsshub.app/telegram/channel/rssaid returns an rss feed
But importing https://t.me/rssaid to RSSAid  returns nothing

e.g the rsshub route:https://rsshub.app/telegram/channel/rssaid returns an rss feed But importing https://t.me/rssaid to RSSAid returns nothing

For F-Droid Rules, the application will not fetch rules without permission. You may need to click the 'Refresh' button on the settings page for Rules and then import again~

I missed an /s/ in telegram link
It needs the web version of the link

I missed an /s/ in telegram link
It needs the web version of the link

I’m glad that you solved the problem.😊 I’ll close the issue. If you have any other questions, feel free to ask.


Token Revocation rule can not work

Hi there, i just found the token revocation rule you describe above can not work : If a user changes their password, the refreshtoken and accesstoken can continue use. anything i missed?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2906fe9e-319d-3625-441d-d28c9251b450
Version Independent ID: 168fde63-9b86-41a9-f1cc-7b5a0e2e5d7a
Content: Learn about the different token and claim types supported by Azure AD
Content Source: articles/active-directory/develop/active-directory-token-and-claims.md
Service: active-directory
GitHub Login: @hpsin
Microsoft Alias: hirsin

@chenxizhang Thanks for your feedback! Please confirm that you referred to Note in this section Token Revocation which states below -
If a non-password method of authentication was used (Windows Hello, the Authenticator app, biometrics like a face or fingerprint) to attain the token, changing the user's password will not force the user to re-authenticate (but it will force their Authenticator app to re-authenticate). This is because their chosen authentication input (a face, e.g.) has not changed, and therefore can be used again to re-authenticate.

@MohitGargMSFT
Thank you
We tested in this way. Let's see if there are any problems.

The first time user login to the application, they enter their credential, and the application obtain the access_token to access the resource.

The application save the access_token, and Use this information directly in the next request.

When the access_token expired, the application use the refresh_token to obtain an new access_token

Users may modify their passwords for a variety of reasons, We expect the original token to be revoked automatically and prompt use to re-authenticate next time

We cannot see the behavior as expectation

@chenxizhang Please open a new MSDN forum post. Please refer to below link to MSDN forum to ask new question -
https://social.msdn.microsoft.com/Forums/azure/en-US/newthread?category=windowsazureplatform&forum=WindowsAzureAD
Also please post the link of the forum post here and I will work on your above scenerio though MSDN forum. Thanks.

https://social.msdn.microsoft.com/Forums/azure/en-US/f4ac66f2-946a-4b07-b04d-4d4c994e7750/token-revocation-rule-can-not-work?forum=WindowsAzureAD#f4ac66f2-946a-4b07-b04d-4d4c994e7750
@MohitGargMSFT  thank you

@chenxizhang Thanks. I will work with you through MSDN forum and move ahead to close this issue.


Update auth service

update the image file, also the ci for auth service create manifests for auth service

merge auth branch


[Feature request]: App To App OAuth for Discord

General Info

[X] I installed the latest version of Supabase Kt
[X] I checked for similar feature requests

Feature request
According to this closed issue in the Discord API Github you should be able to have oauth for Discord occur in the native android app if its installed, however currently supabase-kt only redirects to a browser to authenticate. I have had feedback from multiple users that they would prefer if it authenticated them in the Discord app itself.
Usecase
Given user chooses to Auth via Discord
When they tap sign in with Discord in my app
And they have Discord installed on the same device
Then the user will be redirected to the Discord app to authenticate
And not be brought to a web page

Have you tried opening any Discord OAuth url manually on Android just to see if this was actually fixed?


Notifications not being sent automatically (Possible to do manually)

Hi, forgot to add the backup zip in prior issue.
Able to detect changes in the GUI, however the notification does not post by it self.
I'm able to trigger manual notification via the GUI.
Logs and Backup Zip attached.
Logs from docker logs changedetection.io > log.txt :
Watching: 37d85dae-3df4-4ab1-8fda-ebbe7f040f8a https://www.ea.com/en-gb/games/apex-legends/about/closed-beta-mobile-faq
Watching: df23461b-b3be-41d5-b50e-d25ece90a369 https://play.google.com/store/apps/details?id=com.ea.gp.apexlegendsmobilefps
Saving..

Watching: 37d85dae-3df4-4ab1-8fda-ebbe7f040f8a https://www.ea.com/en-gb/games/apex-legends/about/closed-beta-mobile-faq
Watching: df23461b-b3be-41d5-b50e-d25ece90a369 https://play.google.com/store/apps/details?id=com.ea.gp.apexlegendsmobilefps
Saving..

>> Change detected in UUID df23461b-b3be-41d5-b50e-d25ece90a369 - https://play.google.com/store/apps/details?id=com.ea.gp.apexlegendsmobilefps
>>> Notifications queued for UUID from watch df23461b-b3be-41d5-b50e-d25ece90a369
>> Process Notification: AppRise notifying discord://868448<redacted>/vRYNB6UiPX8<redacted>
Saving..

>> Process Notification: AppRise notifying discord://868448<redacted>/vRYNB6UiPX8<redacted>
Saving..

>> Change detected in UUID df23461b-b3be-41d5-b50e-d25ece90a369 - https://play.google.com/store/apps/details?id=com.ea.gp.apexlegendsmobilefps
>>> Notifications queued for UUID from watch df23461b-b3be-41d5-b50e-d25ece90a369
>> Process Notification: AppRise notifying discord://868448<redacted>/vRYNB6UiPX<redacte>
Saving..

changedetection-backup-1631879606.zip

@ekenheim could you please pull master/latest and try again? it should have a lot more debug in the output too, so re-attached your docker logs changedetection.io > log.txt

I just pulled the newest image at https://hub.docker.com/r/dgtlmoon/changedetection.io
I guess I'll have to wait for another update to happen on the website before I submit those new logs - tried doing it now and it's nothing interesting in the log.txt to paste
Watching: 37d85dae-3df4-4ab1-8fda-ebbe7f040f8a https://www.ea.com/en-gb/games/apex-legends/about/closed-beta-mobile-faq
Watching: df23461b-b3be-41d5-b50e-d25ece90a369 https://play.google.com/store/apps/details?id=com.ea.gp.apexlegendsmobilefps
Saving..

I'll update the ticket once I see an update on the website, thanks for the help in the meanwhile @dgtlmoon

@ekenheim Could you test with https://changedetection.io/random.php  ?

I'm getting 404 Not Found, perhaps I got the wrong version on the docker pull

@ekenheim 404 on what exactly? you need to tell me with a lot of detail, i'm not a mind reader :)

Haha my bad @dgtlmoon , I'm getting 404 when trying to access http://192.168.1.XXX:5009/random or http://192.168.1.XXX:5009/random.php which is where I run the changedetection on my Unraid server
Not Found
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

@ekenheim Could you test with https://changedetection.io/random.php ?

You know I wrote literally https://changedetection.io/random.php
Use it to trigger a notification....

Yes, I didn't grasp the context that you wanted me to use it
Even as it was a reply to my issue... Sorry I'll blame it being Friday and my brain has shut down :)
Yes, it worked perfectly fine and triggered the Discord notification with no problems

@ekenheim right cool, so I hope the latest commit fixed it then , I'll close this issue?


pd: respond to queries with specified height

This may be the cause of IBC issues.

Pushed a fix-it commit for the buf mod update. Let's merge optimistically after CI passes, so the height logic makes it into 49.


fixed issues regarding slashing pointer

Fixed issues regarding Slashing pointer including using METRICS_HISTORY to get previous metric for a current validator, Functions for removing validator and withdrawing funds added by @galacticminter. Also fixed some lint errors.


Fix invalid reference in Jenkinsfiles

The Jenkinsfile was incorrectly referencing TRIGGER_STRING which caused a Jenkins build error when going into the Validate PR Source stage.
This change fixes that and also makes the Jenkinsfile trigger command and error congruent with LogDNA's other public repos.

test this please

@logdnabot test this

@logdnabot test this