Implement new designs for cluster create page

Implement the new designs for the cluster create page based off of @ivelisseca's feedback.
Requirements:

Clusters are created via design presets

Cluster specs match the presets that are defined in the Azure Portal

Location of the resource group is used for the location of the cluster

If there is an issue, default to EastUS

AKS version defaults to the current default AKS version available on the latest patch

If the API call fails, in the error message output, link to the Azure Portal asking the user to create the cluster there

Some design/implementation notes:

The presets are supposedly 'fixed', but in reality they will almost certainly change over time. Newer SKUs will be introduced and cluster features and best practices will evolve.
This will mean periodically revisiting this work, and ensuring we are consistent with the Portal.
The easiest way I can see to compare functionality between the Portal and the extension is to view a diff between the ARM templates generated by the Portal and those used by us (they can be stored as source-controlled resources in our repo).
Partly for that reason I'd prefer to use ARM templates to define the resources we're creating, rather than doing it via the JS SDKs for each resource.
ARM templates also have other benefits, particularly the repeatability and declarative nature which I think would ease recovery from errors resulting in partially deployed state.

Some challenges and discussion points for this:

Some parameters (we've identified K8s version so far) will change rapidly over time, so we will need to determine those dynamically when we run the deployment. K8s version is dependent on subscription and region, which are already specified by users, so this should be straightforward using the JS SDK.
Capabilities of resources depend on region and zones. For example Standard_D8ds_v5 is not allowed in all zones in southeastasia.
The Portal deals with the above issue by extensively validating the entire tree of resources and allowing the user to make changes. We can't do that without effectively re-implementing the entire Portal UX. So we'd need to make some choices: allow the user to proceed and report errors back to them? Or validate and ask them to choose a different region?
Depending on the user's permissions on the selected subscription, they may not have permission to deploy some resources. For example, when deploying a production-standard cluster with monitoring, the resources include an administrator role, which I think can only be deployed by a user with Owner permissions on the subscription. Can we validate that? Can we filter the subscriptions that will work? Do we ignore the problem and just report errors back to the user when things fail?
The Portal deploys different resources for the same preset depending on the subscription chosen. We've noticed this for the monitoring resources. Sometimes the Portal will decide to create new workspaces and Prometheus/Grafana resources, and sometimes it won't. We'll need to understand the reasoning behind this. It might be hard to predict what users would want here, because we won't know enough about their circumstances to decide whether we should use an existing workspace (if they have one) or create a new one. We could have some dedicated UX to help guide and question them about this decision (which would be an improvement on the Portal experience), or we could simply say we're not going to create monitoring resources and link them to some guidance for adding those themselves (if they've selected a preset that includes monitoring).
If we were to deploy monitoring resources ourselves, the original plan of copying ARM templates from the Portal and storing them as resources would be a large maintenance burden. The reason for this is that the Portal-generated templates for monitoring resources are not parameterized (unlike the cluster resource). In other words, each of the 12 extra resources in the template contain hard-coded subscription, resource group and cluster name strings, and would hence be unusable for anyone except the person who generated them. We'd have to either go through them manually and templatize them (not just now, but in the future when we're maintaining this and think the presets might have changed), or revert to creating these resources individually through SDKs, resulting in a confusing hybrid approach where it's hard to reason about the behaviour, and hard to view/compare history.

Just archiving most of the chat around how the dynamic kubernetesVersion,  - https://learn.microsoft.com/en-us/rest/api/aks/managed-clusters/list-kubernetes-versions?view=rest-aks-2023-08-01&tabs=JavaScript  - Unless anyone else have has better idea:
Once this gives lis of kubernetes verion, we have no har dependency and all hunky dory, jsut some thoughts.
Here is the API.
 const result = await client.managedClusters.listKubernetesVersions(location);

To make it more clear
💡 We have all the attributes in need for this funct call at the repo level with examples.
const { ContainerServiceClient } = require("@azure/arm-containerservice");
const { DefaultAzureCredential } = require("@azure/identity");
async function listKubernetesVersions() {
  const subscriptionId = process.env["CONTAINERSERVICE_SUBSCRIPTION_ID"] || "subid1";
  const location = "location1";
  const credential = new DefaultAzureCredential();
  const client = new ContainerServiceClient(credential, subscriptionId);
  const result = await client.managedClusters.listKubernetesVersions(location);
  console.log(result);
}

Pros:

This will atleast take a year to deprecate and we already do with other tools and keep them inline with the version those tool use.
More certain, easy to test and less uncertainity.
More from the source of truth and nothing hardcoded

Error Handle API Failure Is Best Move
If API for some reason fails then it means issue is with API and we should safely errorable our code for those reasons, because if list operation from sdk fails then surely create will fail. (These failures at SDK level is above layer and we should be deterministic to these failures and error handle with same certainty. Please don't misunderstand this with, I am canvassing for no retry but this is basically clean path for sdk and error-handling. If anyone has further retry idea this is great place to share or internal channel)
Very excited to see this new U/X coming into life.

Currently we are proceeding with one preset Dev/Test matching with portal and see how users/developers like this. Agreed there are challenges with ARM templates parameterizing and handling error/exception cases. Adding to @peterbom points there are few more to this

When using ARM templates and when error happens due to one of the resources, we need to get inner exception from deployments response.
Also some of the parameters requires validations ex: nodeResourceGroup should not be greater than length 80 characters - link here. Since we are going towards ARM templates to match with portal, we need to know how portal is dealing with this.


Upgrade to check-types 2.x

Major release to be compatible with check-types v2

No longer necessary - does not require check-types library


Fix Config error: indexed_colors: missing field color

Fix Config error: indexed_colors: missing field color

Already fixed by @andreasgrafen

Yup.. my bad! I didn't see the PR pending. Anyhow; thanks a lot for your contribution. ♡


Can't take Screenshot

I want take screenshot of invalid website like: https://domestic.some.app
But, I'm getting error like
Error: net::ERR_NAME_NOT_RESOLVED at https://domestic.some.app
at navigate (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:194:23)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async FrameManager.navigateFrame (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:170:21)
at async Frame.goto (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:589:16)
at async Page.goto (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1089:16)
at async /app/app.js:237:3

But I want response as

My code:
app.get('/screenshot', async (req, res) => {
  if (!req.query.url) return void res.sendStatus(404)
  const { url } = req.query
  console.log(url)
  const browser = await puppeteer.launch({
    headless: true,
    args: ['--no-sandbox', '--incognito', '--single-process', '--no-zygote'],
    defaultViewport: null,
  })
  const page = await browser.newPage();
  await page.goto(url, { waitUntil: 'networkidle0', timeout: 60000 }).then(async () => {
     const image = await page.screenshot({ fullPage: true });
     await browser.close()
     res.status(200).setHeader('Content-Type', 'image/png').send(image).end()
    }).catch(async (error) => {
      console.error(error)
      res.sendStatus(404)
   })
})

Try using try/catch to catch the navigation exception? In any case, taking a screenshot of the internal error page is not something we explicitly support.

Try using try/catch to catch the navigation exception? In any case, taking a screenshot of the internal error page is not something we explicitly support.

this happens:
Error: net::ERR_NAME_NOT_RESOLVED at https://something.ub.app
at navigate (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:194:23)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async FrameManager.navigateFrame (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:170:21)
at async Frame.goto (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:589:16)
at async Page.goto (/app/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1089:16)
at async /app/app.js:238:5

Can't I take a screenshot like that which says "This site can't be reached"


bool uniforms and legacy ES shaders

Hello,
I have this shader (test.vert):
#version 450

layout(set = 0, binding = 0) uniform UBO
{
	bool b;
}

void main()
{
	gl_Position = b ? vec4(0.0) : vec4(1.0)
}

When I try to convert it for ES 2.0 targets spirv-cross throws next exception:
Unsigned integers not supported for ES targets.
For conversion I use following tools:
glslangValidator.exe -V -o test.vert.spv test.vert
spirv-cross test.vert.spv --version 100 --es --output test.vert.es

How to workaround this?
Thanks.

Doesn't seem to be more to do here, closing.


Implement CTRL+BACKSPACE word-deletion

Actually (Terminal 1.0.1401.0), if you use CTRL+LEFT or CTRL+LEFT, you correctly move between words. However, CTRL+BACKSPACE does not delete the previous word, even though CTRL+DEL does delete the next word.
Since the latter is implemented, it would be great (and more useful) to implement CTRL+BACKSPACE.
Odly, CMD.exe has CTRL+BACKSPACE but doesn't support CTRL+DEL 😄.
Thank you.

This is typically a feature that the shell needs to implement, not the Terminal. As you've correctly identified, cmd does support deleting the previous word on ctrl+bksp.
Which shell are you using?

Oh. Right. Even if I open CMD inside Terminal, it has CTRL+BACKSPACE. The shell I'm using is the default WSL shell:

Should I take this to Microsoft/WSL?

Shortcuts in WSL depend on the shell that Linux is using. Most of them support "readline" shortcuts.

No need, you'll just need to configure it with whatever shell you're using in WSL. For me, I use bash, so I've added the following to my ~/.bashrc:
bind '"\C-h":backward-kill-word'

Amazing, thank you, @zadjii-msft. Added that and it's working.
And thank you @dreadnaut for the shortcut sheet.


bug: Generic structs error in the idl if they're populated with custom structs

The title says it all - if we do:
#[account]
struct MyAccount {
    data: GStruct<u32>,
}

#[derive(AnchorSerialize, AnchorDeserialize, Clone)]
struct GStruct<T> {
    data: T,
}

the generated idl is fine and can be used. If we do
#[account]
struct MyAccount {
    data: GStruct<MyStruct>,
}

#[derive(AnchorSerialize, AnchorDeserialize, Clone)]
struct GStruct<T> {
    data: T,
}

#[derive(AnchorSerialize, AnchorDeserialize, Clone)]
struct MyStruct {
    data: u32,
}

the generated idl is unparsable. Might open a PR for this later, but in case I forget wanted to at least have the issue. MRE for this here: https://github.com/cryptopapi997/anchor-generics-mre

Should be fixed by https://github.com/coral-xyz/anchor/pull/3016, feel free to take a look @acheroncrypto


Update to v1.6.0

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Reset the build number to 0 (if the version changed)
[x] Ensured the license file is being packaged.

Bumping up version to v1.6.0

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Closing in favor of #32


Problem with matching single line signatures

Single line signatures could have extra stuff after the final } which doesn't seem to be handled properly by this extension. Note how the Rubocop disable comment throws off the signature matching:

I believe it is the  *?$ part that is trying to just match whitespace until the end of line that is the problem. A similar problem probably exists for the multiline match regex with comments after the end keyword.


Problem with @tf.function decoration: tensor out of scope

Click to expand!
Issue Type
Bug
Source
source
Tensorflow Version
2.10.0
Custom Code
Yes
OS Platform and Distribution
Windows 10
Mobile device
No response
Python version
3.10
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
11.2
GPU model and memory
No response
Current Behaviour?
Current behavior is I get error message 'The tensor cannot be accessed from FuncGraph, because it was defined in FuncGraph which is out of scope.' due to line 'ghat[j] = ghat[j] + ghat_update[j]' (line 35). This error only occurs if decorated with tf.function.

I would expect the behavior to work in tf.function mode the same as eager mode.

Standalone code to reproduce the issue
import tensorflow as tf

class Outer(tf.keras.Model):
    def __init__(self, model):
        super().__init__()
        self.model = model
        self.mem = None
        self.loss_mem = None

    def loss(self, x):
        return 2*x-1

    @tf.function
    def gradient(self, x):
        self.mem = tf.TensorArray(tf.float32, size=10)
        self.loss_mem = tf.TensorArray(tf.float32, size=10)
        for i in tf.range(10):
            out_i = self.model.call(x)
            loss_i = self.loss(out_i)
            self.mem = self.mem.write(i, out_i)
            self.loss_mem = self.loss_mem.write(i, loss_i)

        ghat = [tf.zeros_like(i, dtype=tf.float32) for i in self.model.trainable_variables]
        for i in tf.reverse(tf.range(10), [0]):
            with tf.GradientTape() as g:
                g.watch(x)
                out_i = self.model.call(x)
            with tf.GradientTape() as g_loss:
                g_loss.watch(out_i)
                loss_i = self.loss(out_i)
            output_grad = g_loss.gradient(loss_i, out_i)
            ghat_update = g.gradient(out_i, self.model.trainable_variables, output_gradients=output_grad)
            for j in range(len(ghat)):
                ghat[j] = ghat[j] + ghat_update[j]
        return ghat

class Inner(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.out = tf.keras.layers.Dense(10)

    def call(self, x):
        return self.out(x[0])

x = [tf.zeros((32, 4, 16), tf.float32), tf.zeros((64, 10), tf.float32)]
mytest = Outer(Inner())
mytest.gradient(x)

Relevant log output
Traceback (most recent call last):
  File "C:\Users\tawit\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch.py", line 49, in <module>
    mytest.gradient(x)
  File "C:\Users\tawit\AppData\Local\Programs\Python\Python310\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users\tawit\AppData\Local\Programs\Python\Python310\lib\site-packages\tensorflow\python\framework\func_graph.py", line 790, in capture
    raise errors.InaccessibleTensorError(
tensorflow.python.framework.errors_impl.InaccessibleTensorError: <tf.Tensor 'while_1/add:0' shape=(16, 10) dtype=float32> is out of scope and cannot be used here. Use return values, explicit Python locals or TensorFlow collections to access it.
Please see https://www.tensorflow.org/guide/function#all_outputs_of_a_tffunction_must_be_return_values for more information.

<tf.Tensor 'while_1/add:0' shape=(16, 10) dtype=float32> was defined here:
    File "C:\Users\tawit\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch.py", line 49, in <module>
      mytest.gradient(x)
    File "C:\Users\tawit\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch.py", line 25, in gradient
      for i in tf.reverse(tf.range(10), [0]):
    File "C:\Users\tawit\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch.py", line 34, in gradient
      for j in range(len(ghat)):
    File "C:\Users\tawit\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch.py", line 35, in gradient
      ghat[j] = ghat[j] + ghat_update[j]

The tensor <tf.Tensor 'while_1/add:0' shape=(16, 10) dtype=float32> cannot be accessed from FuncGraph(name=gradient, id=1901677038992), because it was defined in FuncGraph(name=while_1_body_143, id=1902270847184), which is out of scope.

@ronan-keane,
Could you please take a look at the comment from another user with a similar error which helps to deliver the information. Thank you!

@tilakrayal I have looked at that example but I do not think it is applicable here. First of all, it is a different error message. Also, their case is just not really the same as what is happening here. All the output is generated from 'x' which is already an input to gradient method.
Right now I think this is a bug in tensorflow graph compilation, if this is user error please let me know.
I found out another interesting thing, which is if I comment out first code block of gradient, the error goes away. That first code block has nothing to do with the 'ghat_update' calculation, which is the part that gets the error.
import tensorflow as tf

class Outer(tf.keras.Model):
    def __init__(self, model):
        super().__init__()
        self.model = model
        self.mem = None
        self.loss_mem = None

    def loss(self, x):
        return 2*x-1

    def test_maybe(self, x):
        self.mem = tf.TensorArray(tf.float32, size=10)
        self.loss_mem = tf.TensorArray(tf.float32, size=10)
        for i in tf.range(10):
            out_i = self.model.call(x)
            loss_i = self.loss(out_i)
            self.mem = self.mem.write(i, out_i)
            self.loss_mem = self.loss_mem.write(i, loss_i)

    @tf.function
    def gradient(self, x):
        # self.test_maybe(x)

        ghat = [tf.zeros_like(i, dtype=tf.float32) for i in self.model.trainable_variables]
        for i in tf.reverse(tf.range(10), [0]):
            with tf.GradientTape() as g:
                g.watch(x)
                out_i = self.model.call(x)
            with tf.GradientTape() as g_loss:
                g_loss.watch(out_i)
                loss_i = self.loss(out_i)
            output_grad = g_loss.gradient(loss_i, out_i)
            ghat_update = g.gradient(out_i, self.model.trainable_variables, output_gradients=output_grad)
            for j in range(len(ghat)):
                ghat[j] = ghat[j] + ghat_update[j]
        return ghat

class Inner(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.out = tf.keras.layers.Dense(10)

    @tf.function
    def call(self, x):
        return self.out(x[0])

x = [tf.zeros((32, 4, 16), tf.float32), tf.zeros((64, 10), tf.float32)]
mytest = Outer(Inner())
mytest.gradient(x)

@ronan-keane,
I was able to execute the mentioned code without any issue/error. Kindly find the gist of it here.

Yep like I said that was the fix. Wondering if someone with knowledge of how tf.function compilation works could look at the first example, and the working example, and tell me whether or not tf.function is working as intended.

Your response makes no sense to me. The issue has nothing to do with loops. The issue is with tensor out of scope. If you look at the fix I used to make this code work, it seems extremely clear to me that it is a bug in tensorflow's jit compilation. If you think that is wrong, please explain why.

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.


Update Composer dependencies (2021-07-19-00-12)

Visual regression test passed!
View the visual regression test report


Modify Spinnaker Description

Already fixed in #1531. I guess your PR should have gone in first. Sorry we missed it.


App endpoints not working if they are accessed directly

Clicking on the 'Create' link will take me to https://terra.cuibonobo.com/create, but visiting https://terra.cuibonobo.com/create directly results in a 404 error.

Fixed in 8f6e343178e330354440affa96a6f12281ea30e2


Add ability to display nearby stops departures using device position, or arbitrary set position

It would be great to be able to display the next departures at nearby stops (within a given or set radius) from the device position, but also be able to do this to an arbitrary position (for helping someone travelling, or simply to display that information for some place we're are not yet).

You may want to describe the feature with more detail. At least in Digitransit, it hasn't been easy to define what this means exactly: how to cluster, sort, and filter stops, patterns and trips, and which columns to display.

Do you want to sort by time to departure, by distance to stop, or by a combination of those? (We sort by distance first, then by time.)
Do you want to group together all the platforms of a train station although distance to them varies? (I think we need this but we don't currently have it.)
Do you want to filter out stops that are farther away that only duplicate trips from closer stops? (We do but don't have an API for this.)
Do you want to display the next two trips as columns per route (pattern)? (We do.)
Do you want to display routes that don't depart until next morning or next Monday - and if not, what's the limit? (We haven't decided on the limit.)

All that aside, the new GraphQL endpoint can give you stopsByRadius, and for those, stoptimesForPatterns or stoptimesWithoutPatterns. clustersByRadius for stations seems to be missing. patternsByRadius might be nice.

@tuukka thanks for helping me to be clearer.

I want to sort by distance first, and then by time, with a "group" item for each stop.
If possible, yes. It is tricky for the user to find the proper line when bus stops and stations for each line have different names or stop locations, whereas they are in the same ~50m radius.
Sure, it would be nice, but not mandatory.
Yes, and I think you're doing this because it is very useful when you will miss the next bus due to a few minutes, and know when is the next one, or have to hurry to get it (because the next one is too late).
No, I think that the good limit would be one hour or two, not later. Or, you provide a mean to set a time when you want to know next departures.

I know, and that would make it easier. But currently due to this bug https://github.com/opentripplanner/OpenTripPlanner/issues/2170 I'm still stuck to OTP 0.18.x, as it make me unable to properly run the OTP instances I have on the same server.


7zip-zstd doesn't register right-click shortcut in file explorer

If you install the 7zip package it will register a shortcut in file explorer, but the 7zip-zstd package doesn't.
IMO this is because the 7zip package uses msi installer while the 7zip-zstd package uses 7z archive.
The release page of 7zip-zstd provides a *.exe, this installer will create a shortcut for 7zip-zstd, maybe we should use this executable instead.
By using this *.exe installer, we can also remove the 7zip dependency of 7zip-zstd, current 7zip-zstd depends on 7zip which I think is unreasonable.

Scoop won't modify your system except shortcut, so it won't create content menu items. If you install 7-zip using Scoop, you don't have content menu, right?

Am I missing something? If I install 7zip, I'll have a right-click shortcut in explorer, but when I uninstall it, the shortcut disappears.

Even if it doesn't create a shortcut, I still suggest using the executable, because by this way you don't need to install 7zip as dependency.

@HUMORCE Thanks!


Audio and video are out of sync with live HLS streaming when audio and video segments are in separate subdirectories.

Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
v4.3.4
Can you reproduce the issue with our latest release version?
Yes
Can you reproduce the issue with the latest code from main?
Yes
Are you using the demo app or your own custom app?
Both
If custom app, can you reproduce the issue using our demo app?
Yes
What browser and OS are you using?
Chrome on macOS Monterey 12.1
For embedded devices (smart TVs, etc.), what model and firmware version are you using?
n/a
What are the manifest and license server URIs?

n/a
What configuration are you using?  What is the output of player.getConfiguration()?

Unsure.
What did you do?

Create a live HLS master manifest, where video and audio segments are in separate subdirectories, with ffmpeg.
Command we use is:

ffmpeg -re -i input.mp4 -map 0:0 -vcodec copy -map 0:1 -acodec copy -f hls -hls_list_size 10 -g 90 -hls_segment_type fmp4 -movflags +faststart -hls_time 3 -hls_segment_filename stream_%v/data%02d.m4s -master_pl_name master.m3u8 -var_stream_map "v:0,agroup:av a:0,agroup:av" "stream_%v/output.m3u8"

Input video we use is 1080p h264 with aac audio in a mp4 container.
What did you expect to happen?
Since having variant streams in subdirectories is allowed in HLS, we expect a smooth playback of the master manifest.
What actually happened?

On Chrome, Shaka Player is able to load the content, however, video and audio are out of sync.
Previsouly, with v4.3.0, we were experiencing an issue where video is frozen for a few second while audio is able to play when loading a content, as well as constant pauses and a/v out of sync issues during playback .
With v4.3.4, the major problem is that the video and audio are still out of sync.

We've recently fixed a lot of issues with HLS, can you check again against the main branch? Thanks!

We've tested this again with v4.3.7. The issue is still not solved.
On Chrome browser, av sync is still off when loading the video.
Test content example : https://ireplay.tv/test/blender.m3u8

We checked again with v4.4.1. The av sync is still off...


Insiders

This pull request includes fixes for the bugs inserted in the latest updates made to add support to optimized xlsx and transcription format chunking and introduction of ChunkerFactory approach.
PR includes changes to improve the handling of different file formats, optimize chunking processes, and enhance logging for better debugging. The most important changes include removing support for certain file formats, modifying chunking logic, and adding new environment variables.
File Format Handling:

Removed support for epub, rtf, docx, doc, pptx, ppt, msg, and pdf file formats from the LangChainChunker. (chunking/chunkers/langchain_chunker.py, README.md) [1] [2] [3]

Chunking Logic:

Updated SpreadsheetChunker to use a new environment variable SPREADSHEET_NUM_TOKENS for the maximum chunk size and added detailed logging for chunk creation. (chunking/chunkers/spreadsheet_chunker.py) [1] [2] [3]
Modified LangChainChunker to download and decode blob data before chunking. (chunking/chunkers/langchain_chunker.py) [1] [2] [3]
Adjusted TranscriptionChunker to process VTT files directly from blob data and updated chunking logic. (chunking/chunkers/transcription_chunker.py) [1] [2] [3] [4]

Environment Variables:

Introduced SPREADSHEET_NUM_TOKENS in local.settings.json.template to control the maximum chunk size for spreadsheet chunking. (local.settings.json.template)

Logging and Error Handling:

Enhanced logging for error messages and chunking processes to provide more detailed information. (chunking/document_chunking.py, function_app.py) [1] [2] [3]

Retry Mechanism:

Increased the maximum number of retries for rate limit errors in AzureOpenAIClient from 3 to 10. (tools/aoai.py) [1] [2]

Approve assuming that removed file formats are still supported by document chunking function.


Wire desktop bump

Description of changes
Bump the versions for Linux and Darwin. Tested on Darwin but not Linux.
Usually this is backported to the stable channels, not sure how to request that properly? Wait for merge and then open PRs targeting stable branches?

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Backport is done by throwing a label on the PR and if that fails do it manually.

Result of nixpkgs-review pr 196593 run on x86_64-linux 1

1 package built:

wire-desktop

Downgrading to 3.28.2946 seems to bring the bootup back, without breaking my history at least. Maybe we should revert this, if more people are affected?

Can you try strace to see if it is a NixOS problem with drivers?

Maybe @arianvp or @kiwi run this on Linux and can help with testing?
Opened a revert PR just for the Linux version in #199350.

@toonn you only tested this on MacOS?

I did test it on Linux and it ran far enough to at least show the login screen.

It happens after the login screen unfortunately.

I am not using wire I just tried to be helpful.

@flokli, I don't have a Linux machine to test it on. I asked for reviews in #nix multiple times, this is after about a week of waiting for reviews by maintainers. Sandro and K900 tried to help out but aren't users.
Maybe you can take on maintainership, then at least you will get pinged and you can take care of the Linux side? Since right now you're kinda getting notified by the actual breakage, which is suboptimal to say the least.

I had @arianvp test this on a non-NixOS chromebook today (with NixGL), and it did work. There must be something odd with my machine, even though I can't pinpoint it.
I'm not sure how to proceed with this. I guess it's fine, considering more people would have complained otherwise (?)

Maybe you can take on maintainership, then at least you will get pinged and you can take care of the Linux side?

There's not much I can do, considering this is a binary build, and I run into some crashes on my machine :-/ I also can't easily try if this works with other versions of electron, or if some of the warnings run on startup are patched. Did you have any success in trying to build from source?

Ah, sorry to hear. I'll close the revert PR then. I guess you can pin the package until an update fixes it?
I did not have any success no, though that attempt was years ago. I've recently packaged another electron app so I might try this again early next year.

Yeah, i guess I'll pin it for the time being, hoping it'll keep working for a while.


[Enhancement] limit global runtime filter size

currently we don't have a good way to limit very large global runtime filter.  The global rf coordinator will skip using the whole global runtime filter if it found that the size of grf already exceeds global_runtime_filter_build_max_size when merging partial rf. But in sender side, all partial runtime filters need to be built and send to grf coordinator although their sizes already exceed global_runtime_filter_build_max_size, which is not necessary.
so there are two ideas to optimize it:

If the sender side finds that the size of bloom filter has exceeded the limit when building partial runtime filter, it can directly skip building bloom filter and only keep min/max filter.
If the size of grf'sbloom filter has exceeded the limit when merging partial rf, there is no need to abandon the entire grf. We can choose to keep only min/max filter, which may help reducing some IO.

In this PR, I implemented these two optimizations.
What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.2
[x] 3.1
[x] 3.0
[x] 2.5

[FE Incremental Coverage Report]
:heart_eyes: pass : 5 / 5 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
5
5
100.00%
[]

[BE Incremental Coverage Report]
:disappointed: fail : 4 / 76 (05.26%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exec/hash_joiner.cpp
0
1
00.00%
[560]

:large_blue_circle:
src/exec/pipeline/runtime_filter_types.h
0
7
00.00%
[211, 212, 325, 330, 331, 332, 334]

:large_blue_circle:
src/exec/hash_join_node.cpp
0
6
00.00%
[489, 491, 492, 493, 494, 498]

:large_blue_circle:
src/exprs/runtime_filter_bank.h
0
2
00.00%
[103, 104]

:large_blue_circle:
src/exec/pipeline/hashjoin/hash_join_build_operator.cpp
0
5
00.00%
[106, 108, 109, 110, 111]

:large_blue_circle:
src/runtime/runtime_filter_worker.cpp
0
19
00.00%
[132, 138, 257, 258, 259, 260, 261, 267, 270, 280, 281, 282, 283, 335, 349, 558, 559, 647, 648]

:large_blue_circle:
src/exprs/runtime_filter.cpp
1
14
07.14%
[143, 144, 145, 146, 148, 226, 227, 228, 229, 230, 232, 233, 235]

:large_blue_circle:
src/exprs/runtime_filter.h
3
22
13.64%
[306, 307, 310, 311, 329, 330, 457, 458, 460, 461, 489, 490, 817, 852, 868, 874, 875, 883, 884]

[BE Incremental Coverage Report]
:disappointed: fail : 16 / 83 (19.28%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exec/hash_joiner.cpp
0
1
00.00%
[560]

:large_blue_circle:
src/exec/pipeline/runtime_filter_types.h
0
6
00.00%
[211, 212, 325, 330, 331, 333]

:large_blue_circle:
src/exec/hash_join_node.cpp
0
6
00.00%
[489, 491, 492, 493, 494, 498]

:large_blue_circle:
src/exec/pipeline/hashjoin/hash_join_build_operator.cpp
0
5
00.00%
[106, 108, 109, 110, 111]

:large_blue_circle:
src/runtime/runtime_filter_worker.cpp
0
15
00.00%
[137, 185, 186, 257, 258, 259, 260, 266, 269, 279, 280, 281, 282, 334, 335]

:large_blue_circle:
src/exprs/runtime_filter.h
5
26
19.23%
[102, 103, 127, 128, 180, 322, 323, 324, 326, 461, 462, 464, 465, 802, 814, 856, 872, 878, 879, 887, 888]

:large_blue_circle:
src/exprs/runtime_filter.cpp
11
24
45.83%
[113, 150, 151, 152, 153, 154, 155, 234, 235, 236, 238, 239, 242]

[FE Incremental Coverage Report]
:heart_eyes: pass : 5 / 5 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
5
5
100.00%
[]

[BE Incremental Coverage Report]
:disappointed: fail : 16 / 83 (19.28%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exec/hash_joiner.cpp
0
1
00.00%
[560]

:large_blue_circle:
src/exec/pipeline/runtime_filter_types.h
0
6
00.00%
[211, 212, 325, 330, 331, 333]

:large_blue_circle:
src/exec/hash_join_node.cpp
0
6
00.00%
[489, 491, 492, 493, 494, 498]

:large_blue_circle:
src/exec/pipeline/hashjoin/hash_join_build_operator.cpp
0
5
00.00%
[106, 108, 109, 110, 111]

:large_blue_circle:
src/runtime/runtime_filter_worker.cpp
0
15
00.00%
[137, 185, 186, 257, 258, 259, 260, 266, 269, 279, 280, 281, 282, 334, 335]

:large_blue_circle:
src/exprs/runtime_filter.h
5
26
19.23%
[102, 103, 127, 128, 180, 322, 323, 324, 326, 461, 462, 464, 465, 802, 814, 856, 872, 878, 879, 887, 888]

:large_blue_circle:
src/exprs/runtime_filter.cpp
11
24
45.83%
[113, 150, 151, 152, 153, 154, 155, 234, 235, 236, 238, 239, 242]

[FE Incremental Coverage Report]
:heart_eyes: pass : 5 / 5 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
5
5
100.00%
[]

[BE Incremental Coverage Report]
:disappointed: fail : 16 / 85 (18.82%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
src/exec/hash_joiner.cpp
0
1
00.00%
[560]

:large_blue_circle:
src/exec/pipeline/runtime_filter_types.h
0
6
00.00%
[211, 212, 325, 330, 331, 333]

:large_blue_circle:
src/exec/hash_join_node.cpp
0
8
00.00%
[489, 491, 492, 493, 494, 497, 499, 500]

:large_blue_circle:
src/exec/pipeline/hashjoin/hash_join_build_operator.cpp
0
5
00.00%
[106, 108, 109, 110, 111]

:large_blue_circle:
src/runtime/runtime_filter_worker.cpp
0
15
00.00%
[137, 185, 186, 257, 258, 259, 260, 266, 269, 279, 280, 281, 282, 334, 335]

:large_blue_circle:
src/exprs/runtime_filter.h
5
26
19.23%
[102, 103, 127, 128, 180, 322, 323, 324, 326, 461, 462, 464, 465, 802, 814, 856, 872, 878, 879, 887, 888]

:large_blue_circle:
src/exprs/runtime_filter.cpp
11
24
45.83%
[113, 150, 151, 152, 153, 154, 155, 234, 235, 236, 238, 239, 242]


[Bug]: Show navbar Toggle under App settings>Navigation Turned ON Is Not Persisted

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When turning on the Show navbar toggle in App settings>Navigation, it does not show the child options (Orientation, Variant, etc.), and the setting does not persist after moving away from the Navigation pane, nor does the navigation bar show up in the deployed view.

More details (screen recording, shared app) in the Slack thread.
Steps To Reproduce

Go to App settings>Navigation.
Turn on the Show navbar toggle.
Move away from the Navigation pane.
Come back to the Navigation pane and note that the Show navbar is OFF.

Public Sample App
No response
Environment
Production
Issue video log
No response
Version
v<IP_ADDRESS>

@ame-appsmith I'm able to see it and it seems to be working. Please share a video


Baseline on the CNN/DailyMail dataset

Hi, thanks for writing this code! I had an idea for the README and wanted to share with you.
I think the tutorial on the README could be even more useful if it included your code's ROUGE score on the CNN/DailyMail dataset which is the standard for comparing English text summarization models. I've been reading several papers (example 1 example 2) on the topic and they all test out their models on this dataset.
In addition, it would be very helpful to include a tutorial on how to run this code on the CNN/DailyMail dataset since I believe the current code is based on Chinese text.
I am willing to help on this issue if you'd like!

Soory. The work is too busy. I don't have time to do this. If I finish my project. I will complete it. Thanks for u advice
from Alimail iPhone
------------------Original Mail ------------------
From:ronnyli<EMAIL_ADDRESS>Date:2019-09-04 21:11:37
Recipient:policeme/transformer-pointer-generator<EMAIL_ADDRESS>CC:Subscribed<EMAIL_ADDRESS>Subject:[policeme/transformer-pointer-generator] Baseline on the CNN/DailyMail dataset (#21)
Hi, thanks for writing this code! I had an idea for the README and wanted to share with you.
I think the tutorial on the README could be even more useful if it included your code's ROUGE score on the CNN/DailyMail dataset which is the standard for comparing English text summarization models. I've been reading several papers (example 1 example 2) on the topic and they all test out their models on this dataset.
In addition, it would be very helpful to include a tutorial on how to run this code on the CNN/DailyMail dataset since I believe the current code is based on Chinese text.
I am willing to help on this issue if you'd like!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

if u want. U can put a pull request.
from Alimail iPhone
------------------Original Mail ------------------
From:ronnyli<EMAIL_ADDRESS>Date:2019-09-04 21:11:34
Recipient:policeme/transformer-pointer-generator<EMAIL_ADDRESS>CC:Subscribed<EMAIL_ADDRESS>Subject:[policeme/transformer-pointer-generator] Baseline on the CNN/DailyMail dataset (#21)
Hi, thanks for writing this code! I had an idea for the README and wanted to share with you.
I think the tutorial on the README could be even more useful if it included your code's ROUGE score on the CNN/DailyMail dataset which is the standard for comparing English text summarization models. I've been reading several papers (example 1 example 2) on the topic and they all test out their models on this dataset.
In addition, it would be very helpful to include a tutorial on how to run this code on the CNN/DailyMail dataset since I believe the current code is based on Chinese text.
I am willing to help on this issue if you'd like!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks for the fast response! I will try to make a pull request :)


Bad error message from unsanitary port names

Type of issue: Bug Report
Please provide the steps to reproduce the problem:
Name a port with characters that are not legal Verilog and you'll get this error, see the following Scastie:
https://scastie.scala-lang.org/mHRyrwNCScCZIfElmoX8nQ
import chisel3._

class Example extends Module {
  val foo, bar = IO(Input(UInt(8.W)))
  val out      = IO(Output(UInt(8.W)))

  // Add a character to the name that will be sanitized away
  out.suggestName("out-")

  out := foo + bar
}

What is the current behavior?
You'll get this error:
[error] DeletedWrapper.scala:38: Cannot have duplicate names out and out-
[error] There were 1 error(s) during hardware elaboration.

What is the expected behavior?
I'm not sure if this should be an error or not, but at the very least, the error message should be more precise. Also ideally, if it is an error, it should give some source locator or stack trace. Note that this works with Chisel 3.5.5 and just drops the unsanitary characters: https://scastie.scala-lang.org/hJ1sDILoT3aHO0RjyehdEg
Please tell us about your environment:
Other Information
What is the use case for changing the behavior?

Note that the example in this ticket is a case where the error message is wrong. The error message is referring to cases where there is a name collision with a port, like the following where I try to give two ports the same name: https://scastie.scala-lang.org/9zqBtV0iRpiWGgf3dtRkBA. The error message could still be improved in this case, there's not really a reason to show both candidate and available name since in theory, the two things are asking for the same name (so maybe only give candidate?).


Docs: Specify & document the Gateway upgrade path if Gateway Discovery is enabled

Is there an existing issue for this?

[X] I have searched the existing issues

Problem Statement
The user needs to know how to upgrade their Gateway if they're running the KIC GD setup.
Proposed Solution
Ensure that the user can learn how to upgrade Gateway in the GD setup.
Additional information
No response
Acceptance Criteria

[ ] KIC's documentation answers the question "how do I upgrade Gateway if GD is enabled?"

Based on the backlog grooming discussion, there is no special treatment needed for the GD case, but we should relax the version check on KIC startup: https://github.com/Kong/kubernetes-ingress-controller/issues/4085
Closing as "not needed"


Latest link does not work on Windows

Command ln -s cannot create lnk files on Windows filesystems.
Need to find a fix to this.

Try the command mklink on Windows.

Fixed in commit 741c3ab15e78da5ab5b6caa6c4de72fe77a4f6c3


Fix libcd:or behaving like an "and"

libcd:or currently returns false if any condition fails. This PR changes it to return true if any condition succeeds.

Thanks! I probably messed this up while changing to the new condition system.


fix impl has_dict_page to replace old logic and fix write empty parquet row group bug

What problem does this PR solve?
Problem Summary:
Checks if the given column has a dictionary page.
This function determines the presence of a dictionary page by checking the dictionary_page_offset field in the column metadata. The dictionary_page_offset must be set and greater than 0, and it must be less than the data_page_offset.
The reason for these checks is based on the implementation in the Java version of ORC, where dictionary_page_offset is used to indicate the absence of a dictionary. Additionally, Parquet may write an empty row group, in which case the dictionary page content would be empty, and thus the dictionary page should not be read.
See https://github.com/apache/arrow/pull/2667/files
Release note

impl has_dict_page to replace old logic
only make new row group when there is data to write

Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

run buildall

run buildall

run buildall

run buildall


Reorganize configs

As we have a two stepped system
Code goes through dev-staging-prod
-> only models trained from prod code gets the additional staging->prod flow in the mcp , thus i have separated out the models

that model reallx struggled with the text, this is great though


Feedback Bug (possibly)

Hey, been running this locally, testing out feedback. Not sure if this is how you've written it to be but if the check box for "enable actions" is not ticked on powerpoint, I don't appear to get feedback back to companion. As soon as I tick that box it works fine.
I had guessed there were options to run either/or so feedback without actions and vice versa.
Thanks, Great job BTW the feedback is a game changer!

Thanks for the note.
Feedback and actions should work independent of each other - when I get a chance I'll take a look to see if I can replicate this issue.

Hi also, the feed back is working fine but showing as working in edit mode rather than show mode. I have seen it indicate each mode correctly previously...

Started indicating show mode randomly on one of the clicks about 20 slides later


Use more of DOI data returned by Crossref

Introduction

We have functionality that obtains DOI data from Crossref Work API and converts it to a Relaton bibliographic item: https://github.com/ietf-ribose/bibxml-service/blob/main/doi/crossref.py

This is used by the DOI lookup widget, as well as by /public/rfc/bibxml7/... paths.

doi2ietf-py is a Python library, based on doilit Ruby library, that converts DOI data (from obsolete Crossref DOI API) to xml2rfc format: https://github.com/ietf-ribose/doi2ietf-py

Relaton bibliographic item specification supports a wider scope of bibliographic data than the xml2rfc format, such as ISBN etc. As a result, BibXML service overall is expected to use more DOI data than doi2ietf-py/doilit.
Challenge
Some DOI data is still ignored by BibXML service, but is recognized by doi2ietf-py/doilit.
We could investigate doi2ietf-py (and optionally doilit) source and use that DOI data appropriately when constructing a Relaton bibliographic item. For example, I wonder if we could use volume, journal-issue and page, which we currently seem to ignore.
Key pointers

Crossref DOI API spec: http://api.crossref.org/swagger-ui/index.html

Spec may be incorrect, but one can try different DOIs (discoverable e.g. via bib.ietf.org) and inspect API responses directly, such as https://api.crossref.org/v1/works/10.3390/fi11030055
This is the DOI data that we get.
doi2ietf-py/doilit use a different API endpoint (https://data.crossref.org/10.3390/fi11030055, now inaccessible), so the data they get could be slightly different. See .json files in doi2ietf-py test fixtures for what they got.

Where doi2ietf-py converts DOI structure to xml2rfc: https://github.com/ietf-ribose/doi2ietf-py/blob/6ac3904972ceaeb110e7711a222e93a564aa0250/doi2ietf/utils.py
Where we construct Relaton’s bibliographic item from DOI data: https://github.com/ietf-ribose/bibxml-service/blob/main/doi/crossref.py
Relaton model spec: LutaML model (see sibling files for referenced class definitions), RNC grammar

We want to find a way to correctly use more DOI data when constructing relaton.models.bibdata.BibliographicItem, using doi2ietf-py/doilit as example
Bibliographic data model that we have specified in Python so far: relaton.models.bibdata.BibliographicItem (docs)

Separate issues can be filed for expanding Relaton model coverage in relaton-py (as well as corresponding updates of BibXML service GUI), if needed to accommodate DOI data. The first step should be to try filling in the models already defined and handled by the GUI.

Optional

doi2ietf-py is based on doilit Ruby library

Just in case the Python doi2ietf library was forgotten:
https://github.com/ietf-ribose/doi2ietf-py
It already does everything the Ruby doilit can do.

(Updated the description)

It looks like we may have to add an additional model to the Relaton-py (AKA LocalityStack, see https://github.com/relaton/relaton-models/blob/main/models/LocalityStack.lutaml) in order to accommodate the need of additional data (volume, page and issue) for the Relaton BibliographicItem.
The definition of this model can be found here.
Changes to the relaton-py BibliographicItem model and related serializer will be required.
Crossref APIs returns all the data needed. The format of the response, as previously stated, is slightly different than the response returned by the obsolete Crossref DOI API used by doilit, thus it'll require some adjustments to the way we capture some of this data.

@stefanomunarini  Understood about locality stack, and thanks for the links.
Is there any other data we can use from DOI without adding new fields? 🤔

@strogonoff the only data we can add without updating the  BibliographicItem model is date (based on the comparison with the Ruby doilit implementation).
However, this is true only if the date field of the BibliographiItem model corresponds to either the issued or the created date.

@stefanomunarini Check out the date field descriptions in linked specifications, and also see the instances of date field in our Relaton YAML datasets. You’ll see that it’s possible to use it for different types of dates.

DOI may provide more data we’re not capturing yet, but I think this could be left for later.


Remove section explanations on copy/paste page

I often have the screen reader speak 5:15s to me. However, I end up hearing the section explanations over and over again. When reading visually, it's easy enough to skip over these. But the truth is that even visually, they're unnecessary and they clutter things up.
It would be easy enough to add a checkbox labeled "hide section explanations", but I would argue even that is unnecessary. Most managers are going to understand what each section is – heck they've probably written the explanations themselves.

Duplicate of #53 and #15.


handle IMR history for non-monitored reports

default view e.g. No monitoring history

fixed in f8916ab


OA: Replace networkcontext.csv with use of DNS

Suggestion / request.
Since the IP DNS mapping can change frequently, building and keeping the networkcontext.csv file up to date is an unnecessary complication. It would be great if the default was to use DNS instead. Even if this was only implemented for local IP addresses (as defined in ipranges.csv), this would simplify installation and use.
Perhaps the option to use file based and then DNS based would help, like /etc/resolv.conf. This would allows the user to over-ride DNS for analyst readability as required by the site.
In a similar manner, 'whois' info could be make available about external domains within the UI.

Hi @thealy thanks for the suggestion, will discuss it with the team on how & when we can incorporate this option. I think will be more applicable to networkcontext rather than ipranges. We might need to think on a hybrid since many environments use DDNS causing that the name of a particular IP address might change over time. Perhaps running OA and querying the DNS while runs and store that information into a file for a particular day, could be a potential solution

Any news on this?


Enable LINQ ThenBy operator after OrderBy

Description

Enable LINQ ThenBy operator after OrderBy to use composite indexing
Refactor LINQ tests

E.g
LINQ with ThenBy
query.OrderBy(f => f.FamilyId).ThenBy(f => f.Int).ThenByDescending(f => f.IsRegistered)

SQL
SELECT VALUE root
FROM root
ORDER BY root["FamilyId"] ASC, root["Int"] ASC, root["IsRegistered"] DESC

Type of change

New feature (non-breaking change which adds functionality)

Closing issues

@khdang can you please take a look at the test failures?

@khdang can you please take a look at the test failures?


Added tutorial videos to the Adding 3D Models doc

Pros:

Now the tutorial videos are in the appropriate doc, where they belong, and not hidden on S3

Cons:

Now anyone who clones the repo gets to download the videos
Had to add a new python/sphinx dependency that's not on pypy
Had to manually copy over the tutorial videos since sphinx wasn't doing it automatically (maybe it ignores all video files?)

Dean figured out how to copy over the tutorial videos using sphinx itself, rather than manually copying them using the script file. I've implemented and committed the change. Thanks Dean!


feat(rockspec): per platform overrides

Stacked on #10

I think we can merge the PR now as-is and then make improvements/changes to the actual codebase later on. What do we think?


Drbendanillobranch

This is my basic CodeIgniter Project.

Wow. I'm just Learning GitHub. 3 days ago I didn't know how to use this stuff.


Keyboard shortcuts stop working after a while

Keyboard short cuts stop working after using VS Code for a while.

VSCode Version: 1.18.1
OS Version: ubuntu 16.04

Steps to Reproduce:

No really steps to produce this error just happens after using vs code for a while (over an hour or so). Need to reboot VS Code now so will time it and report back.

will need to test this but I've not added any extensions for a while.
Reproduces without extensions: Yes/No

Please provide more details like reporducable steps and follow our issue filling guidelines
https://github.com/Microsoft/vscode/wiki/Submitting-Bugs-and-Suggestions

Not sure I can add much more to this, the keyboard short cuts just stop working after a while (a few hours from what I can tell) and VS Code needs to be restarted. Code can still be edited and saved (using menu items and not keyboard). Nothing appears to change with VS Code it still looks and operates  as normal, but shortcuts seem to stop working. Not added any new extensions for several months and these don't appear to require any updates as far as I can see at least.
There are no steps to reproduce this, it just seems to happen at random times.
This seemed to appear in 1.18.1 update, can't say I had noticed it before 1.18.0 and certainly not in previous versions.
I am happy to close this issue, after all, this could be just my installation of VS Code and next release may resolve it.

@DuncanFaulkner please try using vscode insiders and let us know if you can still reproduce the behavior there
https://code.visualstudio.com/insiders/

Same here after an upgrade to version 1.18.1

@frlinw Do you also see it in the latest Insiders? https://code.visualstudio.com/insiders/

Have been running the insiders version all afternoon and I've not experienced the same keyboard short cut issue, will continue to run this for this week at least and will let you know the out come, so far so good.

Since there has been no update here, I am going to assume that this issue no longer repros in the latest update of 1.19.0
If you do see it again in 1.19.0, feel free to re-open this issue or open a new issue

I can confirm, it's fine in 1.19.0


Call auth.onAuthStateChange at least once when listener is registered.

Feature request
Call auth.onAuthStateChange() at least once when registering the listener.
Is your feature request related to a problem? Please describe.
When getting the current user, I have to first get the user using auth.user(), then add a listener using auth.onAuthStateChange() and implement the same code there.
Describe the solution you'd like
I feel like it would be better if auth.onAuthStateChange() is called at least once when registering the listener, this removes unnecessary duplicate or spaghetti code. Additionally, this is also how firebase does it: https://firebase.google.com/docs/reference/android/com/google/firebase/auth/FirebaseAuth.AuthStateListener
Describe alternatives you've considered
There is a solution as mentioned above, but doesn't result in the cleanest code.
Additional context
None.

V2 Has been released and this issue is still present. And with V2 it now calls for even more advanced async handling due to sync auth.user() & auth.session() being removed. I can recall many instances of users opening issues on this in V1 and personally think it should be highlighted as almost every Firebase dev migrating is used to the easy handling of getting the user object on first load.

hey @sannajammeh and @Xzoky174, can you guys please explain why it is necessary for the callback to be fired when the listener is registered aside from remaining compatible with firebase?
If the reasoning is for easy handling of getting the user object on first load, you can just use getSession in supabase-js v2 to get the current user. It doesn't seem necessary for it to be invoke when onAuthStateChange is called. Maybe i'm missing something obvious here 😅

hey @sannajammeh and @Xzoky174, can you guys please explain why it is necessary for the callback to be fired when the listener is registered aside from remaining compatible with firebase?
If the reasoning is for easy handling of getting the user object on first load, you can just use getSession in supabase-js v2 to get the current user. It doesn't seem necessary for it to be invoke when onAuthStateChange is called. Maybe i'm missing something obvious here 😅

Firstly it's the point of Firebase  migrations. onAuthStateChange always invokes with the the available auth object.
Secondly it's the ease of use. When we invoke with the object we are also able to provide a single source of truth to any global user store in our codebase. getSession fires once, onAuthStateChange should logically fire every time the auth object changes, which by extension includes when it's loaded as it goes from no-auth to auth.
Then it's the duplicated logic needed when onAuthStateChange is not invoking:

User fetching
Post auth validation
Setting JWT cookie server side
Fetching user related data (once)

All this could've been done in a few lines if it fires with the user on load. However we now have to either move this logic outside and have these points react to data changes or duplicate this logic first getSession then inside onAuthStateChange.
It's certainly not a problem at all when you only do simple things like fetching the user. Even then it's logical to have it invoke as the auth state literally changes.
I'll see if I can look through my commit history on one of my projects. I recall writing my onAuthStateChange wrapper saved 40-50 lines of duplicate logic. Either way it's up to you guys if you want to implement it. We'll keep writing wrappers 😅


Trying to optimize custom cursor..

Trying new approach for the custom cursor. Looking for improvement in comparison to master

Merging as the new custom-cursor performs much better.


[AIRFLOW-3088] Include slack-compatible emoji image.

Emoji preview:

Make sure you have checked all steps below.
Jira

[x] My PR addresses the following Airflow Jira issues and references them in the PR title.

https://issues.apache.org/jira/browse/AIRFLOW-3088

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

This adds a 64px by 64px version of your logo, suitable for adding to slack as a custom emoji.
Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:
This is a static PNG image and does not require tests.

Commits

[x] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters (not including Jira issue reference)
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Documentation

[ ] In case of new functionality, my PR adds documentation that describes how to use it.

When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.

I did not document the inclusion of the image.
Code Quality

[x] Passes git diff upstream/master -u -- "*.py" | flake8 --diff

@sethwoodworth ^

@r39132 Sorry for the delay, I wasn't getting notifications for a little while there.

Image moved to /docs/images/logo subfolder
Preview image added to PR description
Rebased, squashed, & pushed

Codecov Report

Merging #3918 into master will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3918      +/-   ##
==========================================
+ Coverage   74.95%   74.98%   +0.03%
==========================================
  Files         199      199
  Lines       15965    15965
==========================================
+ Hits        11966    11971       +5
+ Misses       3999     3994       -5

Impacted Files
Coverage Δ

airflow/models.py
88.66% <0%> (+0.04%)
:arrow_up:

airflow/jobs.py
82.48% <0%> (+0.35%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd4e3cf...6a07d3f. Read the comment docs.

Cool! @sethwoodworth
Can you add transparent in the image name just to make it more obvious airflow_64x64_emoji_transparent.png? We currently have some png that do not have a transparent background.

@sethwoodworth any updates?

@r39132 Renamed and rebased


Added WebpEncoder and encoder tests

new CI workflow added
webp encoder and tests
ffi config for version above 8.2
jpeg encoder tests
png encoder fix
png encoder tests
fill modifier and tests

Wow, great. That moves the project forward.


:globe_with_meridians: Updated /lang/se.conf

Updated Swedish language translations (some translations were missed in earlier change submitted) that work with NeoWX Material

added also into https://github.com/seehase/neowx-material/pull/33


Creating a new process on Windows

On Unix, we use os.fork() to fork the current process. The child process will be replaced by a new nvim process and the parent process will attach to that new process to run its commands.
Forking a process doesn't work on Windows though, so nvr can't be used for that on Windows.
If someone has knowledge in this domain and feels motivated enough to add support for it, I'll gladly add him as contributor who can push Win-specific commits on its own. Every now and then, I'll check for regressions on the Unix side and make a new release otherwise.
I don't have access to Windows, so I can't implement or test it. If you say a patch works on Windows, that will be good enough for me.

I bumped into this at work when trying to replicate my vim setup.
I'm currently using the following powershell function to workaround the issue. Perhaps neovim-remote could provide a shim like this for Windows users.
function vi {
  nvr --remote-tab-silent $args *> $null
  if (!$?) {
    nvim-qt $args
  }
}

The way nvr launching neovim doesn't work on windows.
For example, this tiny script will launch neovim but you can't interact with it.
import os
os.execvpe("nvim", ["nvim"], os.environ)

As the above comment suggests, the solution may need a bat file or PowerShell script to launch neovim on windows instead.

Windows seems to require either subprocess or multiprocessing module rather than os.fork, which isn't supported from what I gather.
The best work around I found is using this autoIt script, it's faster than the powershell function above.. not sure why though. It needs to be compiled. I think something can be written in autohotkey as well, and when compiled its faster than powershell.
I didn't write this script can't remember where I found it. Credit to the author.
Hope this helps someone else with windows struggles:
invr au3.zip

Hi guys, wanted to mention this script absolutely works with latest neovim. Please see steps below, (to add contextual menu item in windows, it will add a new tab inside current neovim. You can name contextual menu item "Edit with Existing Neovim" )

Make sure you've opened an instance of Neovim from powershell, like this C:\Users\david\Downloads\nvim-win64\Neovim\bin\nvim-qt.exe -- --listen '<IP_ADDRESS>:7777'
Download "invr au3.zip" (script mentioned by @dseeni https://github.com/mhinz/neovim-remote/files/4670221/invr.au3.zip)
Also download "autoIt" program (you will need this, to compile the script mentioned above)
Compile the script into an exe using the AutoIt program downloaded
To test the exe, call it from Powershell like so .'C:\Users\david\Downloads\invr.au3\invr au3\invr.exe' --remote-send ':tabnew :e C:\src\log.txt<cr>'
At this point, it should be opening up a new tab within your currently running Neovim, and editing that file (you can choose any file you want)
You can now add an entry for "edit with existing Neovim" to windows explorer, I think the regedit is HKEY_CLASSES_ROOT\*\shell\Open With Existing Neovim, the syntax is basically "C:\invr.exe --remote-send ':tabnew :e %1'" where %1 is the parameter windows will use as the file path


Added support for madvise

Codecov Report

Merging #2036 (60eccca) into main (f479840) will decrease coverage by 0.02%.
The diff coverage is 76.59%.

:exclamation: Current head 60eccca differs from pull request most recent head 71e6738. Consider uploading reports for the commit 71e6738 to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main    #2036      +/-   ##
==========================================
- Coverage   94.39%   94.38%   -0.02%
==========================================
  Files         319      319
  Lines       59161    59199      +38
==========================================
+ Hits        55846    55874      +28
- Misses       3315     3325      +10

Impacted Files
Coverage Δ

src/lib.rs
98.94% <ø> (ø)

src/directory/mmap_directory.rs
85.96% <76.59%> (-1.38%)
:arrow_down:

... and 1 file with indirect coverage changes

@PSeitz can you have a look?


Fix: crash when unknown TSL address is received

fixes #1

This was indeed an oversight. Thanks for the fix.
(Note that this was developped more as a proof-of-concept, and thus further bugs may be present.)

(Note that this was developped more as a proof-of-concept, and thus further bugs may be present.)
I appreciate it never the less :-). Thanks also for the quick merge.


Add caching of vm memory for info segments

In cheatnet, in crates/cheatnet/src/rpc.rs::execute_next_syscall_cheated there is a moment where we create a block info segment each time if it's requested. We should cache the segment instead, and either create or return the one that has been created.
Remember to add tests with stop_roll/stop_warp variants, as it might break the behavior potentially ⚠
Make sure to test the caching as well

Blocked by #303

@Arcticae can you please expand the task description?

can you please expand the task description

Done, can you verify if that's sufficient @cptartur?

I think it's okay. It will still require a bit of thinking about the exact documentation, but it should probably be clear enough.


[GITHUB-3] Fixes references to undefined variables

To resolve "this issue" - I think I know what it is - I added a script in init to force release of attached disk as soon as possible @wojtek-oledzki

otherwise... :) LGTM


Error on F5 when using Work or School Single Org with Read

From @sayedihashimi on February 1, 2017 2:5
Here's an issue from an internal partner, we have an existing email thread that a few of us are on.
I am trying to create a simple web application and register it with an Azure Active directory like I've done many times in the past. I've started to run in to problems, not sure if its because the framework version or what it may be.
Here are my steps.

Create a new Azure AD, add a new user.
Open Visual Studio 2015 Update 3 as an administrator and choose File | New | Project | Templates | Visual C# | Cloud | ASP.Net Web Application (.Net Framework). I am using version 4.6 of the .Net Framework.
Choose an MVC style app.
Select the Change Authentication button and choose Work or School accounts. Here, I will enter my full domain name like xxxxxxxxxx.onmicrosoft.com, Single Cloud Organization and Read directory data. I sign in to Azure.
The VS app finishes the creation process. I can look at my web.config file and see the changes that have been made for connectivity to Azure AD.
When I select F5 to run the app, I get this error regarding System.Spatial:

From Vittorio I heard that the “spatial” issue can be solved by simply deleting the binding, given that it isn’t really used.
Copied from original issue: aspnet/Templates#808

From @phenning on February 15, 2017 17:56
Note, that this is only an issue with the legacy templates, which are not maintained in the repo.


Cache data on circle again

Codecov Report

Merging #153 into master will decrease coverage by 0.07%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #153      +/-   ##
==========================================
- Coverage   79.26%   79.18%   -0.08%
==========================================
  Files          12       12
  Lines        1008     1009       +1
  Branches      147      147
==========================================
  Hits          799      799
- Misses        168      169       +1
  Partials       41       41

Impacted Files
Coverage Δ

celer/datasets/libsvm.py
20.89% <0.00%> (-0.32%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b463ad...04529e2. Read the comment docs.


Error Neo.ClientError.Schema.EquivalentSchemaRuleAlreadyExists on running cartography

**Error Neo.ClientError.Schema.EquivalentSchemaRuleAlreadyExists on running cartography **
Description:
Hello, I have installed neo4j and confirmed it's working. Then I run cartography using this line cartography --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password-prompt. But when running this line and providing my correct password, I get this error
Password:
INFO:cartography.sync:Starting sync with update tag '1608722184'
INFO:cartography.sync:Starting sync stage 'create-indexes'
INFO:cartography.intel.create_indexes:Creating indexes for cartography node types.
ERROR:cartography.sync:Unhandled exception during sync stage 'create-indexes'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/cartography/sync.py", line 71, in run
    stage_func(neo4j_session, config)
  File "/usr/local/lib/python3.8/dist-packages/cartography/intel/create_indexes.py", line 22, in run
    neo4j_session.run(statement)
  File "/usr/local/lib/python3.8/dist-packages/neo4j/work/simple.py", line 217, in run
    self._autoResult._run(query, parameters, self._config.database, self._config.default_access_mode, self._bookmarks, **kwparameters)
  File "/usr/local/lib/python3.8/dist-packages/neo4j/work/result.py", line 101, in _run
    self._attach()
  File "/usr/local/lib/python3.8/dist-packages/neo4j/work/result.py", line 202, in _attach
    self._connection.fetch_message()
  File "/usr/local/lib/python3.8/dist-packages/neo4j/io/_bolt4.py", line 363, in fetch_message
    response.on_failure(summary_metadata or {})
  File "/usr/local/lib/python3.8/dist-packages/neo4j/io/_common.py", line 179, in on_failure
    raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.ClientError: {code: Neo.ClientError.Schema.EquivalentSchemaRuleAlreadyExists} {message: An equivalent index already exists, 'Index( id=60, name='index_f3a83024', type='GENERAL BTREE', schema=(:GCPNicAccessConfig {id}), indexProvider='native-btree-1.0' )'.}
Traceback (most recent call last):
  File "/usr/local/bin/cartography", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/cartography/cli.py", line 348, in main
    return CLI(default_sync, prog='cartography').main(argv)
  File "/usr/local/lib/python3.8/dist-packages/cartography/cli.py", line 328, in main
    return cartography.sync.run_with_config(self.sync, config)
  File "/usr/local/lib/python3.8/dist-packages/cartography/sync.py", line 145, in run_with_config
    return sync.run(neo4j_driver, config)
  File "/usr/local/lib/python3.8/dist-packages/cartography/sync.py", line 71, in run
    stage_func(neo4j_session, config)
  File "/usr/local/lib/python3.8/dist-packages/cartography/intel/create_indexes.py", line 22, in run
    neo4j_session.run(statement)
  File "/usr/local/lib/python3.8/dist-packages/neo4j/work/simple.py", line 217, in run
    self._autoResult._run(query, parameters, self._config.database, self._config.default_access_mode, self._bookmarks, **kwparameters)
  File "/usr/local/lib/python3.8/dist-packages/neo4j/work/result.py", line 101, in _run
    self._attach()
  File "/usr/local/lib/python3.8/dist-packages/neo4j/work/result.py", line 202, in _attach
    self._connection.fetch_message()
  File "/usr/local/lib/python3.8/dist-packages/neo4j/io/_bolt4.py", line 363, in fetch_message
    response.on_failure(summary_metadata or {})
  File "/usr/local/lib/python3.8/dist-packages/neo4j/io/_common.py", line 179, in on_failure
    raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.ClientError: {code: Neo.ClientError.Schema.EquivalentSchemaRuleAlreadyExists} {message: An equivalent index already exists, 'Index( id=60, name='index_f3a83024', type='GENERAL BTREE', schema=(:GCPNicAccessConfig {id}), indexProvider='native-btree-1.0' )'.}

Due to this error, I'm not able to move further.
I have also tried to manually delete all the indexes in the neo4j database but this didn't help and the indexes were repopulated on running cryptography command once again.

Cartography release version 1.7.8
Python version: 3.8.5
OS : Ubuntu 20.04 LTS

Issue was with neo4j version. Installed v4.1 which gave me this error
Did a clean install with neo4j v3.5.25 and it worked like a charm.
Closing the issue


Trigger delay handling of virtual_awg

@peendebak @CumulonimbusCalvus
Fix delay for readout trigger by virtual_awg for scan1Dfast and scan2Dfast by passing through the sampling frequency.

@CumulonimbusCalvus  @eendebakpt
I agree with both comments by Luc. I think this code was originally written by Pieter and I only copy-pasted it to make the different scan functions consistent with each other. Pieter, could you reply on Luc's comment and implement them as suggested if you agree?

@CJvanDiepen How is the delay fixed with this PR? I see the sampling frequency is added to many functions of the virtual awg, but I do not see it being used.
Most of the changes are related to the old virtual awg object. We will probably stop supporting that in the near future, so I suggest not to put a large amount of work into this. (It does also mean I will not block a PR that is not perfect)
Making a subfunction to be reused by the different scan functions should not be that hard. Something like:
from qtt.measurements.scans import *

def sample_rate_from_instrument_handle(measurement_instrument_handle):
    """ Retrieve current sampling rate from an instrument

    Args:
        measurement_instrument_handle (tuple or str or Instrument): specification of the instrument
    Returns:
        float: current sampling rate
    """

    instrument = get_instrument(measurement_instrument_handle)

    try:
        isfpga = isinstance(
            instrument, qtt.instrument_drivers.FPGA_ave.FPGA_ave)
    except:
        isfpga = False
    try:
        ism4i = isinstance(
            instrument, qcodes.instrument_drivers.Spectrum.M4i.M4i)
    except:
        ism4i = False
    if isfpga:
        sample_rate = instrument.get_sampling_frequency()
    elif ism4i:
        sample_rate = instrument.sample_rate()
    return sample_rate

(I did not test this, just to give an idea)

@peendebak The pretrigger_period is set based on the samp_freq. Hence, this introduces a conditional delay in the data acquisition. For some sweep functions in the virtual_awg this samp_freq argument was introduced but for others it apparently was not. This PR adds it to the ones still missing it, such that the pretrigger_period is set consistently.
Pieter, your suggestion for a subfunction to obtain the sampling frequency is then perhaps a good idea for the new virtual_awg instrument? (@CumulonimbusCalvus )

@CJvanDiepen Can you test the scan updates I made on your setup, to see whether I did not made any mistake?

@CJvanDiepen @CumulonimbusCalvus This PR has many conflicts with current dev. Is the functionality still required? If so, then I suggest we open a new PR.


User Admin Permissions

This is a brain-dump / discussion thread about the following problem
Problem
(For us,) having access to the admin should not equate being able to do
everything in the admin. For instance we would like to differentiate between
librarians and super users: both should be able to access the admin, but
librarians should only see certain sections and should only be able to read
and edit records say (not delete them).
Current situation
Currently there doesn't seem to be a way to achieve this in Invenio. (This is where you chastise me for my misunderstanding of the current situation!)
Invenio-admin through its ext.py protects admin
views visibility via the configurable ADMIN_PERMISSION_FACTORY.
This only addresses the visibility problem (read permission of admin view if you will),
and not the action problem (create, update, delete associated with admin view).
Even then, it is placing the onus on the developer using invenio-admin to implement
a factory that would enforce visibility logic for all admin views of all modules
their instance happens to use.
Community
Maybe the logic is different per instances, but the structure to tie these
things together could be common. Supporting more complex permissions and different
roles has been mentioned in the past in a similar context.
A proposal
I am less experienced than others with Invenio, so take the following as you will:
Invenio-admin provides a ProtectedAdminView (not just as a factory),
from which other invenio modules' AdminViews should inherit.

class ProtectedAdminView(BaseView):
    """ProtectedAdminView"""
    def is_accessible(self):
        # Use configurable 'read' permission
        # else default functionality
        # (current_admin or start from config)
        pass

    def is_action_allowed(self, action):
        # Use configurable per action permission
        # else default functionality
        # (current_admin or start from config)
        if action == 'create':
            pass
        elif action == 'update':
            pass
        elif action == 'delete':
            pass
        if action == 'action':
            pass

The configuration for an instance could look like:
ADMIN_PERMISSION_FACTORIES = {
    # <admin_view> to be replaced by the left hand value registered in the
    # `invenio_admin.views` entry point
    '<admin_viewA>': {
        'create': 'my_project.my_module.<admin_view>_create_permission',
        'read': 'my_project.my_module.<admin_view>_read_permission',
        'update': 'my_project.my_module.<admin_view>_update_permission',
        'delete': 'my_project.my_module.<admin_view>_delete_permission',
    },
    '<admin_viewB>': {
        # another set
        # any that are not defined just use the default
    }
}

This configuration could eventually replace ADMIN_PERMISSION_FACTORY itself.
To help provide a default functionality for each of these cases, it could be
standard for invenio-modules to create an ActionNeed() for each
(<admin_view>, <action>) pair.
There might be ways to automate the creation of this code structure, but because
the permissions should be fed to the invenio_access.actions entrypoint there might be
limits to what we can do here.
Pros/Cons
The benefit is that it solves the problem (right?).
The cons are:

Requires a change in invenio-admin
Requires a change in EVERY invenio-module that exposes an admin to use ProtectedAdminView
Doesn't address custom actions on an admin view
(could be extended for that, but let's keep it simple for now)
Permissions can take some time to grok and that would add to it. Are we making permissions even messier?

Alternatives
The enabling pattern can be
used to limit admin visibility in general, but it is:

only for visibility and not actions
non user-dependent
requiring changes in all modules as well

I think invenio-accounts / invenio-accounts-rest is trying to pull off something in that vicinity but I may be mistaken / don't understand it.
Actions
Feedback / discussion on this. Is this something a lot of others want and/or in scope for the invenioRDM? I am sure you must have thought about this and
come up with a better solution, but I wanted to have a place to get this issue discussed and get myself completely corrected ;)
@inveniosoftware/architects

Initially, I thought of the admin area as really only the area for admins, as some of the views exposed are basically just database table access. Librarians and other power-users would not access the admin area (/admin/*), but rather go through normal views tailored to their needs.
Also, the admin interface is good for putting up some quick actions on the database, but it doesn't help expose REST APIs.
Example: User/role management would go through a Community Teams feature, rather than through the admin. The admin interface would display role names like community_blr_reader, community_blr_writer, community_blr_admin and would not be very user-friendly.
As said, that was the initial idea. I do see that there are good reasons to change this and allow more fine-grained permissions in the admin, but it probably requires a bit wider discussion.

It is good that you mention the User/Role management, because it is something missing from invenio-access, right?
Yes, in most cases, we can provide site pages/features that do the equivalent things. For instance, we do this already for librarian search and editing. I think the batch update feature might be served well by the admin. Maybe some of the database values in the admin can display the user-friendly values as well to help.

Just to mention that we used custom admin view solve the same problem in RERO-ILS: limit access and visibility views depending of the role of the professional woking in a given library.
Basically we used is_accessible and is_visible class method:
https://github.com/rero/rero-ils/blob/master/rero_ils/modules/admin.py
We did this using invenio_admin to avoid to re-implement this kind of logic.

Yes this is definitely the way to go for custom/own module views now. Reiterates the value in creating a setup so that someone making a custom view (module) and someone using it (instance) can easily setup their admin permissions.

Since I'm merging some things on the repo right now, I'm checking old issues as well :-)
For InvenioRDM (not Invenio Framework), we have disabled the admin. Also, the idea is to go with a custom admin interface that uses the REST API fully. The invenio-admin is not using our service layer and thus we risk have discrepancies what is allowed and not compared to the normal REST API. Also, the normal REST API provides fine grained control via the permission policies
I'm closing this issue as a won't fix for now.


Implement sniffing of CSS and JS files

This adds a couple of media type sniffers that can guess CSS and JS files from their content types and/or file extensions.
I added them as the lowest priority they could be.

Wrong branch!


Helm 3.0 helm config file location

Hi All,
We are using helm in aws EKS. The current helm version is 2.14.3 which is using tiller pod which has some security concerns because of the privilege we are giving to tiller. We saw this has been fixed in helm 3.0, we tried using 3.0 but we have some queries and need help.

We are not able to mention the name for all helm commands as it is saying --name is invaidoption.

In the older version, while installing helm it will create a .helm directory and some config files but now I don't see any config file or .helm directory, how does it finds the kubernetes API.

Apart from pulling images, will this setup require internet

Is there any tentative date the prod version will be released

A .helm directory is created but it now follows XDG for the base directory. #6244 has been opened to provide a command that will show the locations.
This setup will require Internet to connect to the Kubernetes cluster. The Kubernetes cluster will need internet access to pull any images. Helm currently does not pull images itself.
Helm v3 beta is coming any day now. Once that happens we will be focused on bug fixes in order to get the the 3.0.0 release out. We are in the final stretch to release.
What command did you expect to run with the --name flag?

Ping, any further comments before we close this one out?

Thanks for the responses, we will wait until the helm3 production version releases


Running an application that relies on restclient throwing an exception

I'm the author of an application that relies heavily on restclient, as it turns out this application is distributed through a couple OS distos. When running the application with ruby 2.4 it tends to throw this exception.
$ whitewidow
/usr/share/whitewidow/vendor/bundle/ruby/2.4.0/gems/rest-client-2.0.0/lib/restclient/request.rb:198:in `fetch': key not found: :ciphers (KeyError)
        from /usr/share/whitewidow/vendor/bundle/ruby/2.4.0/gems/rest-client-2.0.0/lib/restclient/request.rb:198:in `initialize'
        from /usr/share/whitewidow/vendor/bundle/ruby/2.4.0/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `new'
        from /usr/share/whitewidow/vendor/bundle/ruby/2.4.0/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `execute'
        from /usr/share/whitewidow/lib/modules/core/settings.rb:9:in `page'
        from /usr/share/whitewidow/lib/modules/core/settings.rb:18:in `parse'
        from /usr/share/whitewidow/lib/misc/banner.rb:36:in `version_type'
        from /usr/share/whitewidow/lib/imports/constants_and_requires.rb:82:in `<top (required)>'
        from whitewidow.rb:2:in `require_relative'
        from whitewidow.rb:2:in `<main>'

Now I've done some research and found that I may need to update the gem, after doing so I had the owner of the distro try to install the package, still getting the same error. Do you have any idea of what's going on?

Any news on this or updates? It's been almost a week

This was fixed in rest-client 2.0.1.
https://github.com/rest-client/rest-client/pull/573


Offer to join org and help maintain this

This project is an awesome idea and started out well, but it has fallen into disrepair and the current state of affairs is a mess. It looks like it's been a few years since it got any real attention, and it needs both some love on the code side of things and a lot of database culling. For example I notice there are 4 entries for a plugin I maintain, only one of which even has the right links. There are reams and reams of links that are not even vim plugins!
I am the instigator of @preservim, and currently an active maintainer of plugins such as nerdcommenter, tagbar, @vim-pandoc & friends, vim-pencil & friends, vim-ledger, vim-sile, vim-tridactyl, and others.
I would like to offer to help. I would propose being added to the org as an owner, be given whatever access is necessary to deploy the code from the repository to the domain, and get access to the active DB and any management tools there is for it.
My goals would be to:

Get the database cleaned up of cruft that is detracting from it being useful. Non vim-plugins, duplicates, etc.
Encourage the community here to actually contribute by being responsive to PRs, encouraging issue reporters to pitch in with fixes, and generally making sure that when people contribute work (such as the issue reports about bad entries) they actually get taken care of.
Gradually improving the UI, submission process, etc. so it stays cleaner in the future.

Just in case repository notifications are turned off, pinging @divad12 and @captbaritone directly as they are publicly listed on the org... there may be more relevant people but Github doesn't make it possible to see who all actually has repo ownership.

This seems reasonable to me. The most I’ve had time/motivation to do in the last few years is restart the server when it occasionally crashes.
@sophiebits pays the bills on the server that it runs on, so I’ll defer to her on access to the server.
Probably the best place to start would be to take a look at the code and try to understand where the duplicate entries are coming from, or why they are not getting correctly deduped.
Of our many failure, the most critical ones I’m aware of are:

Failures to correctly dedupe plugins
False positives in our code which tries to determine what is actually a plugin.
A lack of human effort to respond to human submissions.

Surely there are others too, but those are the ones which I’m most aware of.

Someone using my key?

Hi @divad12,
On May 6th, 2020 at 07:13 (UTC) your personal access token (For VimAwesome scraping locally on my laptop) using python-requests/2.9.1 was used as part of a query parameter to access an endpoint through the GitHub API:
https://api.github.com/user/443562/repos
Please use the Authorization HTTP header instead, as using the access_token query parameter is deprecated. If this token is being used by an app you don't have control over, be aware that it may stop working as a result of this deprecation.
Depending on your API usage, we'll be sending you this email reminder on a monthly basis for each token and User-Agent used in API calls made on your behalf.
Just one URL that was accessed with a token and User-Agent combination will be listed in the email reminder, not all.
Visit https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param for more information about suggested workarounds and removal dates.
Thanks,
The GitHub Team

Negative from my side. If you didn't do so I would revoke the token, it must have leaked somewhere.

A lack of human effort to respond to human submissions.

hi, im happy to help triage new submissions or setup a support ticket triage system through github or whatever is preferred.

@HP4k1h5 Thanks for the offer. Unfortunately at this point to date nothing has really come of this attempt to revive the project. We haven't heard back from the admin of the current infrastructure about getting access to deploy anything that does get worked on and I still don't have any access to actually work on this repo. Given that old contributions have sat around for years without attention I'm kind of unwilling to sink a bunch of time into working on a fork until it's clear there will be a path to get any improvements deployed. If and when that happens I'll ping you again.

@alerque what sort of access do you need — I'm wondering if a dump of the production DB would be useful?

I'd like to help on this also.

I'd like to help on this also.

Great! How can we help you get started?

@divad12 access to reviewing submitted plugins and ability to add them myself would be a good start.
For technical part, I can assist @alerque , since i'm not a python developer.

@kristijanhusak awesome. Send an email to<EMAIL_ADDRESS>and I'll send you a database dump where you can start reviewing plugins.

@divad12 what an email :D
Isn't there a way to do reviewing directly on vimawesome.com? Or this is just a precaution from your side?

@divad12 A dump of the production database is a start, but what we really need is access to production! This project has been lots of people offering to help going back to many PRs that never got reviewed and accepted but if there is no path to actually get our contributions into production what's the point? I am still interested in helping out with this and can coordinate and effort to cleanup and fixup the current site, but I can't do it without actually access to this repo and the production db. I don't want to sink a bunch of time working on it unless it's clear it will go somewhere.

@kristijanhusak unfortunately there is no way to do direct reviewing on vimawesome.com. It would be nice to build such a feature!
@alerque I hear your desire to help and to have access to production to be able to help, and see that, even after 8 months, you are still eager to. I'll email you with next steps.

@divad12 i'll definitely try to introduce this first. It's pain to do reviewing like this.
Can you also add me as collaborator here and allow me to merge PRs into master? Direct push to master should be forbidden anyway.

@kristijanhusak Submit a PR and I'll add you as collaborator if it looks good and gets merged. :)

@divad12 access to reviewing submitted plugins and ability to add them myself would be a good start.
For technical part, I can assist @alerque , since i'm not a python developer.

And with #391 you are now a Python developer, @kristijanhusak 🤣

@alerque I can create you a user for vimawesome so you can approve submitted plugins. Give me your email and i'll send you credentials.

What's the current status? Is vimawesome gonna be revived now? https://www.reddit.com/r/neovim/comments/mydhln/i_wanna_create_a_dead_simple_and_free_website_for/ neovim started a discussion about it.

@TornaxO7 i don't think that should be the case. Codebase is very outdated (Python 2, old react version). It's maybe better to have something rewritten from scratch and have data migrated from here to there.

Hm... that's sad to here... vimawesome was such a nice platform to find plugins :( So shouldn't that be announced on the website/README?

It's not up to me. I'm just a contributor, not an author.
Since I introduced login and review feature, i check once a month and confirm few submitted plugins, but I don't have time to go through all of them. If you want to help, I can make you a user so you can login and review some of the submitted plugins yourself.

And what's with cleaning up the database and remove non-vim-plugins?

I don't know how much I can help, but I'd like to give it a try, so yeah, you can create an account for me :)

@TornaxO7 if you can please contact me via Twitter DM (https://twitter.com/kristijan_husak) so we can set you up an account.

oof... eh I don't have a twitter account.... could you write me an email instead<EMAIL_ADDRESS>

https://www.reddit.com/r/neovim/comments/mydhln/i_wanna_create_a_dead_simple_and_free_website_for/
There has been a discussion about creating a vim-awesome-like platform. It might be worth it to interact with the maintainer, since the project is pretty new so changes can be made "easily".

Ok, to refresh the status of this issue, there's a "new" plugin-platform now: https://nvim-awesome.vercel.app/
I started a little issue wondering if we could "port" the plugins from vim-awesome to nvim-awesome, since the project is pretty new.

I'm taking up the cause of maintenance on this and cleaning up old issues. Thanks for your concern!

Thank you @cbartlett!!