Tips for speeding up OpenAI API answers?

Hi,
I've seen applications that are able to give fast responses using the Langchain & OpenAI (chat with own data).
However, in my case responses on simple questions seem to take a long time. I've been playing around with settings but I am wondering if there is anything else I can do to increase speed.
Current settings:

Chunk Size: 700
Chunk Overlap: 100
Max tokens: 150
Streaming enabled

What am I missing?
thanks!

I use GPT-3.5-Turbo. I also tried with the davinci 003 and 002 models, and they are very fast but GPT-3.5-Turbo is better for my use case.
I see online tools that use Langchain and GPT-4 with a very fast response time so I guess it is possible.

Hi, facing similar issue even though streaming is enabled. Any ideas on how to speed up.

It is not an issue anymore.
Just a suggestion: Count the tokens & price you have. With me I made the mistake of sending too much tokens

I agree with you that it's not an issue at all. LLM take plenty of time to compute the query. When you  enabled streaming, did you notice any change in computing the query. Hasn't the time taken to compute the query.


URL naming convention

Should we rename this endpoint from /config to /settings since it is not part of the config app but rather the settings app?

From meeting:

config > project
settings > config

https://github.com/thenewboston-developers/Core/pull/81

https://github.com/thenewboston-developers/Core/pull/82


Retain text highlighting when adding links

When adding a link in the editor, the current behavior removes any highlighting that may have been applied to the text. The proposed solution is to retain the text highlighting when adding links in the editor.
https://app.usebubbles.com/eMPQh6xF73vwm5i8NU5udc/retain-text-highlighting-when-adding-links
@AbhayVAshokan _a
cc: @yedhink

Agreed this needs to be fixed. @gaagul _a let's take this up.


Update maven-bundle-plugin and hamcrest-all

The repository does not currently compile under Java 14 due to some outdated dependencies.
This updates a couple which then allows the library to compile and tests to pass.

Thank you for the PR!


Update URLs in installer.sh

I've updated the URLs of the OpenJDK Build repository, Maven Central and the the Debian repository in the installer.sh, since they seem to be outdated and don't work anymore.

nice
btw this is dead (if you didnt know)


Unable to get reply from other user.

Hi,
I am unable to get the response from the user on skype UI as it only shows on the console.
How to get it On the skype UI.
private void Client_MessageReceived(Message message)
{
ConsoleWrite_Green("Messaged Received from {0}: {1}", message.GetParticipant().Result.Uri, message.Text);
//client.ReplyMessage(message.Text, message).Wait();
}
Above code only print my message on the console but i need to show it on my Skype for Business UI .

You can consider UCWA as separate instance of Skype for Business. It is not SfB client add-on. So what you do on console app may not reflect on SfB client all the time.


add to community registry

PR created at https://github.com/obsidianmd/obsidian-releases/pull/2147

Added to registry, need to update docs.


Update readme.md

This adds some more usage information based on additions from #23 which was recently closed
Co-Authored-By: Mangirdas Judeikis<EMAIL_ADDRESS>

/cc @mjudeikis @s-urbaniak
I noticed https://github.com/kcp-dev/helm-charts/pull/23 had some useful additions to the readme and a values example, so I made some similar changes here

/assign @s-urbaniak

I will check it out during weekend. Had way more in my other fork too :) thanks!

Hi! I'm trying to deploy kcp to a kind cluster but it seems missing some important parts, like:

s.Options.Extra.ShardVirtualWorkspaceCAFile, not sure where to import this file.
"command failed" err="--shards-kubeconfig is required" when starting a kcp-front-proxy

Not sure how to solve it and continue to make it works. Could you point me some ways to work on? Thanks!

@hardys this looks good enough.
Its not part of this PR, but I would change these things if possible (sorry, late to the review train on previous PR):
- --shard-virtual-workspace-ca-file=/etc/kcp/tls/server-client/ca.crt is missing in KCP pod spec
and I would change this as this allow rapid dev deployment into KIND without need to have proper DNS configured (will add how to get it running in follow-up PR)

OR we can just merge this and I will do follow-up PR with my proposed changes :) Your call
/lgtm

--shard-virtual-workspace-ca-file=/etc/kcp/tls/server-client/ca.crt is missing in KCP pod spec

That should not be required since https://github.com/kcp-dev/kcp/pull/2659 (for now I remove the standalone VW option from the templates, until we resolve issues discussed via https://github.com/kcp-dev/kcp/pull/2672) - can you confirm which KCP image you're using?

Hi! I'm trying to deploy kcp to a kind cluster but it seems missing some important parts, like:
* `s.Options.Extra.ShardVirtualWorkspaceCAFile`, not sure where to import this file.

* `"command failed" err="--shards-kubeconfig is required"` when starting a `kcp-front-proxy`

Not sure how to solve it and continue to make it works. Could you point me some ways to work on? Thanks!

@cychiang please check you have the latest version of the chart - the shards-kubeconfig issue was resolved in #24 - if you still have issues after pulling the latest, please open an issue with more details, thanks!

@ncdc or @s-urbaniak perhaps you could merge this and we'll deal with any further additions as follow-ups?

@hardys you have merge permissions now

@hardys Thanks, I will give it a try.


Native Winget Repo Server for Windows Server

Description of the new feature / enhancement
Currently, there is no good solution to host an on-prem winget repo that does not connect back to Azure.
The idea would be to create a Winget Repo Server/Service for Windows Server that could easily configured by the managing end-user.
Proposed technical implementation details
This could be done by

Writing the server/service in C# using .Net 8.0 and ms sql local db for the database
Implement it using built in web components for site management.

"ms sql local db" we might use the Windows Server internal role "Windows Internal Database".
But technically would have doubts. It was intended for once smaller WSUS and IPAM.
Over the years this WID proves giving admins hard nuts to troubleshoot and maintenance since this is a very special version of SQL LocalDB
On top of that this one is still using SQL 2012 or such, would need to revisit this information on WS 2025.
If this is still the case this would give me shivers and a reason to loop back internally :)

This is one of two issues that prevents us from using winget.
The other one is that it doesn't work under WinRM or SSH (it can be hacked to work).
Our scenario is simple. A couple of thousand servers and no internet connection.
Since MS Rest example starts with requiring Azure it's end of the line directly.
About the database. For smaller use cases i would prefer just a single textfile (after all - that is how it always has been done in the Linux world)

Can recommend winget.pro after an evaluation. Very powerful.

@matsmcp you can run https://github.com/omaha-consulting/winget.pro offline. It's an alternative implementation of a private winget repo. (I am one of the authors.)

@mherrmann looking at it :)

Is there any advancement on this one ?
We would have a use case for this one.
The docs mention the possibility to run a local version. But even there, they mention the need of a 'cosmoDb'... Is this maybe just a matter of updating the doc to fit with a MsSql perhaps ?


In IDEA configuration ignore resource dir declaration if the target is a source dir

As explained at https://github.com/gradle/gradle/issues/654 if a
directory is both declared as a source and a resource directory, then
the resource flag was added to it. This makes IDEA unusable as the
source editing is disabled for the contained source files. To overcome
that, this commit makes the resource declaration omitted if the target
is a source directory.

Running build: https://builds.gradle.org/viewQueued.html?itemId=16211832


CIFAR10 script dtype error

I receive the following TypeError when I try to import CIFAR10 data via the pylearn2 API:

Traceback (most recent call last):
File "my_binary_net.py", line 27, in
train_set = CIFAR10(which_set="train", start=0, stop=train_set_size)
File "/Users/Jackson/pylearn2/pylearn2/datasets/cifar10.py", line 85, in init
x = numpy.zeros((lenx, self.img_size), dtype=dtype)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

I fixed the error by typecasting lenx, which is originally np.float64, to int:
lenx = int(numpy.ceil((ntrain + nvalid) / 10000.) * 10000)

Pylearn2 don't have maintainer. This error is comming from newer numpy.
If you make a PR to fix, it, I can merge it, but we don't maintain it
anymore.
There is other framework on top of Theano like Lasagne, Keras and blocks
that are still alive.
On Mon, Jun 5, 2017 at 9:47 AM Jackson Huang<EMAIL_ADDRESS>wrote:

I receive the following TypeError when I try to import CIFAR10 data via
the pylearn2 API:
Traceback (most recent call last):
File "my_binary_net.py", line 27, in
train_set = CIFAR10(which_set="train", start=0, stop=train_set_size)
File "/Users/Jackson/pylearn2/pylearn2/datasets/cifar10.py", line 85, in
init
x = numpy.zeros((lenx, self.img_size), dtype=dtype)
TypeError: 'numpy.float64' object cannot be interpreted as an integer
I fixed the error by typecasting lenx, which is originally np.float64 to
int:
lenx = int(numpy.ceil((ntrain + nvalid) / 10000.) * 10000)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/lisa-lab/pylearn2/issues/1587, or mute the thread
https://github.com/notifications/unsubscribe-auth/AALC-xXsZYyDJoyp1YIR0IGrLVMNTaeoks5sBAbhgaJpZM4NwEAN
.


Advance to "Move upsert API to insert" commit.

Coverage decreased (-0.2%) to 84.507% when pulling 0d771f33defd095607b79afd726feb75016b3e2f on upsert-to-insert into 2f58db7b1736172cbb448c2fa7c0d601ec191fc8 on master.


Revert "Starship"

Reverts ICE-V-Wireless/ICE-V-Wireless#18
this weas not intended to be a pr on the upstream - apologies

I'm concerned that you've been submitting PRs and then closing them several times over the last few hours. If you don't mean for these PRs to be upstream please stop submitting them.

Yes – it was a misconfiguration on my side.
I do apologise for my error, sorry for the confusion and hassle I’ve caused.
Regards
Rob
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows
From: Eric @.>
Sent: Sunday, 16 July 2023 12:14 PM
To: @.>
Cc: @.>; @.>
Subject: Re: [ICE-V-Wireless/ICE-V-Wireless] Revert "Starship" (PR #19)
I'm concerned that you've been submitting PRs and then closing them several times over the last few hours. If you don't mean for these PRs to be upstream please stop submitting them.
—
Reply to this email directly, view it on GitHubhttps://github.com/ICE-V-Wireless/ICE-V-Wireless/pull/19#issuecomment-1636951239, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAXERXPPG7ESOPWCN5M2W6LXQNE7ZANCNFSM6AAAAAA2LUFCEQ.
You are receiving this because you authored the thread.Message ID: @.***>


enable Gratuitous ARP for SR-IOV device

The issue is to request adding GARP support in sriov-cni
What would you like to be added?
GARP packet is sent by kernel when /proc/sys/net/ipv4/conf//arp_notify is set to 1.
03:33:54.410993 00:11:22:33:44:66 > Broadcast, ethertype ARP (0x0806), length 60: Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 46

What is the use case for this feature / enhancement?
IP address allocated to VF may be re-used while SR-IOV pods are restarted/recreated in a cluster. It is desired to broadcast GARP packet while IP assignment is changed for a particular VF MAC.

So this wouldn't be possible with tuning CNI chained because we don't know the interface name in advance?
{
  "name": "mytuning",
  "type": "tuning",
  "sysctl": {
          "net.ipv4.conf.${INTERFACE_NAME}.arp_notify": "1"
  }
}

So, a potential SRIOV CNI NAD spec would look like below?:
{
  "cniVersion": "0.3.1",
  "name": "sriov-advanced",
  "type": "sriov",
  "trust": "on",
  "garp": "on",
  ...
}

@martinkennelly  I just got to know that it's not necessary to specify the interface name in CNI net-attach-def config, chained CNI like tuning CNI inherits the CNI result passed down from previous CNI which contains the args.IfName.
I need to check if it's possible to use tuning CNI for arp notificuation.
What I noticed is that tuning CNI implements its own logic for sending arp notification, which replies on specifying a MAC change in the same tuning plugin. but in sriov case, we use sriov-cni to change the MAC address.
Another thing is kernel/driver has support for generating garp packet for VF devices, while current implementation in tuning CNI doesn't make use of it. Also kernel/driver requires the device link be restarted in order to send the garp packet, not sure if tunning CNI would restart the link (down & up).

Question: shouldn't we send GARP by default ?

Question: shouldn't we send GARP by default on CmdAdd ?

Nope with current sriov-cni implementation.
Two changes are required in sriov-cni to be able to send GARP:

Enable arp_notify for the VF device
Restart VF link (down & up) while IP has been assigned to VF
Both the two are not met in sriov-cni.

@zshi-redhat im aware we are not doing that in SR-IOV CNI today,
My question is, should we do that by default ? (say if IPAM is specified)

@zshi-redhat im aware we are not doing that in SR-IOV CNI today,
My question is, should we do that by default ? (say if IPAM is specified)

I think so.

I saw mentions of using tuning plugin for that hence my previous question.
My 2 cent is that we should not rely on chained plugins for correct operation of SRIOV cni. It seems that sending GARP is a missing step in cni add call

I'm working on a PR to add garp support in sriov-cni.

For IPv6: net.ipv6.conf.<interface>.ndisc_notify=1
Most probably it is enabled by default, but better would be to set it for sure.


"az containerapp create" missing --registry-server parameter

I'm trying to deploy an application, and I keep getting an error:
(InvalidParameterValueInContainerTemplate) The following field(s) are either invalid or missing. Field 'template.containers.appName-fe.image' is invalid with details: 'Invalid value: "[acr].azurecr.io/full-name-fe:aca-deploy": GET https:?scope=repository%3Afull-name-fe%3Apull&service=[acr].azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.';.

The command used is:
az containerapp create \
    -n appName-fe \
    -i [acr].azurecr.io/full-name-fe:aca-deploy \
    --environment appName-dev \
    --ingress external \
    --target-port 3000

The login used is successful:
  CA_GH_ACTION_ACR_ACCESS_TOKEN=$(az acr login --name [acrName] --output json --expose-token | jq -r '.accessToken')
  echo "CA_GH_ACTION_ACR_ACCESS_TOKEN=${CA_GH_ACTION_ACR_ACCESS_TOKEN}" >> $GITHUB_ENV
  docker login [acr].azurecr.io -u 00000000-0000-0000-0000-000000000000 -p $CA_GH_ACTION_ACR_ACCESS_TOKEN
  CA_GH_ACTION_REGISTRY_URL=[acr].azurecr.io
  echo "CA_GH_ACTION_REGISTRY_URL=${CA_GH_ACTION_REGISTRY_URL}" >> $GITHUB_ENV

WARNING: You can perform manual login using the provided access token below, for example: 'docker login loginServer -u 00000000-0000-0000-0000-000000000000 -p accessToken'
WARNING! Using -*** the CLI is insecure. Use --password-stdin.
Login Succeeded

I have validated locally there is a missing parameter:
➜  full-name-fe git:(aca-deploy) ✗   az containerapp create \
    -g appName-dev \
    -n appName-fe \
    -i [acr].azurecr.io/full-name-fe:aca-deploy \
    --environment appName-api-dev \
    --ingress external \
    --target-port 3000
(InvalidParameterValueInContainerTemplate) The following field(s) are either invalid or missing. Field 'template.containers.appName-fe.image' is invalid with details: 'Invalid value: "[acr].azurecr.io/full-name-fe:aca-deploy": GET https:?scope=repository%3Afull-name-fe%3Apull&service=[acr].azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.';.
➜  full-name-fe git:(aca-deploy) ✗   az containerapp create \
    -g appName-dev \
    -n appName-fe \
    -i [acr].azurecr.io/full-name-fe:aca-deploy \
    --environment appName-api-dev \
    --ingress external \
    --target-port 3000 \
    --registry-server [acr].azurecr.io
No credential was provided to access Azure Container Registry. Trying to look up credentials...
Adding registry password as a secret with name "ecrazurecrio-ecrname"
/ Running ..^C
➜  full-name-fe git:(aca-deploy) ✗

All I did was add     --registry-server [acr].azurecr.io to the CLI command.
I can't find anywhere to get this action to also add this parameter. acrName and registryUrl cannot be used together. Is there some registryServer variable not documented?
The Azure setup is two resource groups; I'll use RG-A and RG-B.

RG-A contain the [acrName] registry
RG-B is the deployment target resource group
The action parameters uses resourceGroup: RG-B.
Github login is done via federated credentials, and logs in successfully in a previous step in the same job workflow.

Action Definition:
      - name: ACA Deploy
        uses: azure/container-apps-deploy-action@v1
        with:
          acrName: [acrName]
          # registryUrl: ${{ vars.REGISTRY_LOGIN_SERVER }}
          imageToDeploy: [..] # [acr].azurecr.io/full-name-fe:aca-deploy from cli above
          resourceGroup: [..] # "RG-B" mentioned above
          containerAppName: appName-fe
          environmentVariables: # {junk stuff I'm leaving out}
          targetPort: 3000
          containerAppEnvironment: appName-dev

Related: https://github.com/Azure/azure-cli-extensions/issues/5238

After talking to one of the ACA PMs, I learned that I needed to specify the --registry-server argument.

For those that run across this, I found a hack that so totally worked. It's a good there's no input validation - code injection FTW.
      - name: ACA Deploy
        uses: azure/container-apps-deploy-action@v1
        with:
          # [... snip ..]
          targetPort: 3000 --registry-server ${{ vars.REGISTRY_LOGIN_SERVER }}
         # [... snip ..]

Command that was ran within the pipline:
  az containerapp up \
    -g [rg-b] \
    -n [appName] \
    -i ecrName.azurecr.io/[repo]:aca-deploy \
    --target-port 3000 --registry-server [ecrName].azurecr.io \
     \
    --env-vars NEXTAUTH_URL=*** \

@guice Hey Philip, thanks for filing this issue (along with the very fun workaround 😄)
The action currently sets the --registry-server argument (along with --registry-username and --registry-password) based on two different sets of inputs provided

ACR provided: acrName, acrUsername and acrPassword
General registry provided: registryUrl, registryUsername and registryPassword

If you're providing acrName when calling this action, then you can also provide the acrUsername and acrPassword inputs to ensure that the --registry-* arguments are provided to the create or up command call made under-the-hood.
Is this a sufficient solution, or are you trying to avoid providing secret values for the acrUsername and acrPassword inputs with this action? Just want to ensure I understand your scenario.

or are you trying to avoid providing secret values for the acrUsername and acrPassword inputs with this action

This. I'm logging in using OIDC via federated credentials. I'm avoiding using username / password combinations.

If you're providing acrName when calling this action, then you can also provide the acrUsername and acrPassword inputs

How about allowing registryUrl with acrName and use it on --registry-server if provided? I don't know exactly all the combinations here. So, take my suggestion with a grain of salt. I do see, currently, you do not allow the combination of the two. Allowing registryUrl and acrName would remove the need for the tertiary acrName and acrUsername and acrPassword checks.


Untested code in Nodes.py

Lots of error cases are untested, which is worth improving but not urgently so.
Here are some chunks of code that should better be tested (line numbers as found in rev 0448bfbb9a1845a3cca5d07a32fdd4590f538713, HTML coverage report):

[x] Nodes.py:3480

3480        for arg in self.args:
3481            if arg.hdr_type and arg.type.is_cpp_class:   # 3481 ↛ 3484
3482                # it's safe to move converted C++ types because they aren't
3483                # used again afterwards
3484                code.globalstate.use_utility_code(
3485                    UtilityCode.load_cached("MoveIfSupported", "CppSupport.cpp"))
3486                args.append("__PYX_STD_MOVE_IF_SUPPORTED(%s)" % arg.entry.cname)

[x] Nodes.py:3772

3772            allow_null = all(ref.node.allow_null for ref in self.starstar_arg.entry.cf_references)  # 3772 ↛ exit
3773            if allow_null:    # 3773 ↛ 3774
3774                code.putln("%s = NULL;" % (self.starstar_arg.entry.cname,))

[x] Nodes.py:4324

4324        if old_type.is_pyobject:
4325            if arg.default:  # 4325 ↛ 4326
4326                code.putln("if (%s) {" % arg.hdr_cname)
4327            else:
4328                code.putln("assert(%s); {" % arg.hdr_cname)
4329            self.generate_arg_conversion_from_pyobject(arg, code)
4330            code.putln("}")
4331        elif new_type.is_pyobject:  # 4331 ↛ 4334
4332            self.generate_arg_conversion_to_pyobject(arg, code)
4333        else:
4334            if new_type.assignable_from(old_type):
4335                code.putln("%s = %s;" % (arg.entry.cname, arg.hdr_cname))
4336            else:
4337                error(arg.pos, "Cannot convert 1 argument from '%s' to '%s'" % (old_type, new_type))

[x] Nodes.py:4644

4644    def analyse_expressions(self, env):
4645        self.args = env.arg_entries
4646        if self.py_func.is_module_scope:  # 4646 ↛ 4647
4647            first_arg = 0

[ ] Nodes.py:5042

5042    def declare(self, env):
5043        if self.module_name and self.visibility != 'extern':  # 5043 ↛ 5044
5044            module_path = self.module_name.split(".")
5045            home_scope = env.find_imported_module(module_path, self.pos)
5046            if not home_scope:
5047                return None

[ ] Nodes.py:5680

5680                    if func_name == 'declare':
5681                        if isinstance(lhs, ExprNodes.NameNode):  # 5681 ↛ 5683
5682                            vars = [(lhs.name, lhs.pos)]
5683                        elif isinstance(lhs, ExprNodes.TupleNode):
5684                            vars = [(var.name, var.pos) for var in lhs.args]
5685                        else:
5686                            error(lhs.pos, "Invalid declaration")
5687                            return

[ ] Nodes.py:5805

5805                if stop_node:  # 5805 ↛ 5808
5806                    stop_node = stop_node.coerce_to(PyrexTypes.c_py_ssize_t_type, env)
5807                else:
5808                    if node.type.is_array and node.type.size:
5809                        stop_node = ExprNodes.IntNode(
5810                            self.pos, value=str(node.type.size),
5811                            constant_result=(node.type.size if isinstance(node.type.size, _py_int_types)
5812                                             else ExprNodes.constant_value_not_set))

[ ] Nodes.py:5853

5853        if start_node and not start_node.is_literal:  # 5853 ↛ 5854
5854            start_node = UtilNodes.LetRefNode(start_node)
5855            refs.append(start_node)
5856        if stop_node and not stop_node.is_literal:  # 5856 ↛ 5857
5857            stop_node = UtilNodes.LetRefNode(stop_node)
5858            refs.append(stop_node)
5859        if step_node and not step_node.is_literal:  # 5859 ↛ 5860
5860            step_node = UtilNodes.LetRefNode(step_node)
5861            refs.append(step_node)
5862
5863        for ix in range(target_size):
5864            ix_node = ExprNodes.IntNode(self.pos, value=str(ix), constant_result=ix, type=PyrexTypes.c_py_ssize_t_type)
5865            if step_node is not None:  # 5865 ↛ 5866
5866                if step_node.has_constant_result():
5867                    step_value = ix_node.constant_result * step_node.constant_result
5868                    ix_node = ExprNodes.IntNode(self.pos, value=str(step_value), constant_result=step_value)
5869                else:
5870                    ix_node = ExprNodes.MulNode(self.pos, operator='*', operand1=step_node, operand2=ix_node)
5871            if start_node is not None:
5872                if start_node.has_constant_result() and ix_node.has_constant_result():  # 5872 ↛ 5876
5873                    index_value = ix_node.constant_result + start_node.constant_result
5874                    ix_node = ExprNodes.IntNode(self.pos, value=str(index_value), constant_result=index_value)
5875                else:
5876                    ix_node = ExprNodes.AddNode(

[ ] Nodes.py:5978

5978            # Avoid coercion for overloaded assignment operators.
5979            if next(iter(lhs_types)).is_cpp_class:  # 5979 ↛ 5980
5980                op = env.lookup_operator('=', [lhs, self.rhs])
5981                if not op:
5982                    rhs = rhs.coerce_to(lhs_types.pop(), env)

[ ] Nodes.py:6482

6482        if self.in_parallel:  # 6482 ↛ 6483
6483            code.putln_openmp("#pragma omp critical(__pyx_returning)")

[ ] Nodes.py:7367

7367                if self.target.entry.scope.is_module_scope:  # 7367 ↛ 7372
7368                    code.globalstate.use_utility_code(
7369                        UtilityCode.load_cached("GetModuleGlobalName", "ObjectHandling.c"))
7370                    lookup_func = '__Pyx_GetModuleGlobalName(%s, %s); %s'
7371                else:
7372                    code.globalstate.use_utility_code(
7373                        UtilityCode.load_cached("GetNameInClass", "ObjectHandling.c"))
7374                    lookup_func = '__Pyx_GetNameInClass(%s, {}, %s); %s'.format(
7375                        self.target.entry.scope.namespace_cname)

[ ] Nodes.py:7823

7823                code.globalstate.use_utility_code(
7824                    UtilityCode.load_cached("FastTypeChecks", "ModuleSetupCode.c"))
7825                if len(patterns) == 2:  # 7825 ↛ 7826
7826                    exc_tests.append("__Pyx_PyErr_GivenExceptionMatches2(%s, %s, %s)" % (
7827                        exc_type, patterns[0], patterns[1],
7828                    ))

[x] Nodes.py:8468

8468        for pos, name, as_name, kind in self.imported_names:
8469            if name == "*":
8470                for local_name, entry in list(module_scope.entries.items()):
8471                    env.add_imported_entry(local_name, entry, pos)
8472            else:
8473                entry = module_scope.lookup(name)
8474                if entry:
8475                    if kind and not self.declaration_matches(entry, kind):  # 8475 ↛ 8476
8476                        entry.redeclared(pos)
8477                    entry.used = 1
8478                else:
8479                    if kind == 'struct' or kind == 'union':  # 8479 ↛ 8480
8480                        entry = module_scope.declare_struct_or_union(
8481                            name, kind=kind, scope=None, typedef_flag=0, pos=pos)
8482                    elif kind == 'class':  # 8482 ↛ 8483
8483                        entry = module_scope.declare_c_class(name, pos=pos, module_name=module_name)
8484                    else:
8485                        submodule_scope = env.context.find_module(
8486                            name, relative_to=module_scope, pos=self.pos, absolute_fallback=False)
8487                        if submodule_scope.parent_module is module_scope:  # 8487 ↛ 8490
8488                            env.declare_module(as_name or name, submodule_scope, self.pos)
8489                        else:
8490                            error(pos, "Name '%s' not declared in module '%s'" % (name, module_name))

[ ] Nodes.py:8554

8554        for name, target in self.items:
8555            if name == '*':
8556                for _, entry in env.entries.items():
8557                    if not entry.is_type and entry.type.is_extension_type:  # 8557 ↛ 8558
8558                        env.use_utility_code(UtilityCode.load_cached("ExtTypeTest", "ObjectHandling.c"))
8559                        break
8560            else:
8561                entry = env.lookup(target.name)
8562                # check whether or not entry is already cimported
8563                if (entry.is_type and entry.type.name == name  # 8563 ↛ 8565
8564                        and hasattr(entry.type, 'module_name')):
8565                    if entry.type.module_name == self.module.module_name.value:
8566                        # cimported with absolute name
8567                        continue
8568                    try:
8569                        # cimported with relative name
8570                        module = env.find_module(self.module.module_name.value, pos=self.pos,
8571                                                 relative_level=self.module.level)
8572                        if entry.type.module_name == module.qualified_name:
8573                            continue
8574                    except AttributeError:
8575                        pass

[ ] Nodes.py:9295

9295            code.putln("switch (%s) {" % Naming.parallel_why)
9296            if continue_:  # 9296 ↛ 9297
9297                code.put("    case 1: ")
9298                code.put_goto(code.continue_label)
9299
9300            if break_:  # 9300 ↛ 9301
9301                code.put("    case 2: ")
9302                code.put_goto(code.break_label)

I would like to contribute to this issue!
could you just me for testing with an example.

Hi @BitGeek29. Pick one of the remaining issues (I've ticked off the two that I have open PRs for just so you don't duplicate that).
Find the relevant code in Nodes.py and think about how you get might get there. If it's a branch that isn't being taken then I suggest adding a breakpoint in the debugger and running cython on a file that you think will get close.
There's basically two options:

the code isn't being tested - in this case think about what needs to change, and see if you can produce a small cython file that does reach the code (if you do that we can help you turn it into a "proper" test).
the code is impossible to reach - in this case it should be removed.

Hi @da-woods. Sorry but ,I couldn't get my head around it.
So can you show me the test codes of Nodes.py:3480, Nodes.py:3772 and Nodes.py:4324 for reference.

3480 https://github.com/cython/cython/pull/4164
3772 https://github.com/cython/cython/pull/4899
4324 https://github.com/cython/cython/pull/4900

hey @da-woods @da-woods could you give me some directions/tips to debug the code defined in Node.py. I started exploring the following code, but it is difficult to understand what is going on when OverrideCheckNode is instantiated.
def analyse_expressions(self, env):
       self.args = env.arg_entries
       if self.py_func.is_module_scope:  # 4646 ↛ 4647
           first_arg = 0

OverrideCheckNode is to do with cpdef functions. The most common case is that you have:
cdef class Something:
    cpdef some_func(self):
        pass

and the override check is to see if some Python code has inherited from that class and redefined some_func.
The less common case is for functions at module scope with Options.lookup_module_cpdef set. See https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#Cython.Compiler.Options.lookup_module_cpdef.
I suspect this case may well be untested since git grep lookup_module_cpdef doesn't show anything in the tests.

OverrideCheckNode is to do with cpdef functions. The most common case is that you have:
cdef class Something:
    cpdef some_func(self):
        pass

and the override check is to see if some Python code has inherited from that class and redefined some_func.
The less common case is for functions at module scope with Options.lookup_module_cpdef set. See https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#Cython.Compiler.Options.lookup_module_cpdef.
I suspect this case may well be untested since git grep lookup_module_cpdef doesn't show anything in the tests.

Ok, thanks! I decided to write a srctree file to test it, so I defined classes A and B in different files, then I override
the method and the module in a  setup.py file, but now I am trying to debug using these files without success. Is there any way to trigger just my srctree file during a debug process? Could you give me some tips?

Is there any way to trigger just my srctree file during a debug process? Could you give me some tips?

There's a few things you can do:

python3 runtests.py name_of_your_file just runs your test (note that name_of_your_file doesn't include .srctree)
--no-capture argument to runtests.py makes it much easier to use in a debugger
--no-cleanup and --no-cleanup-sharedlibs don't delete the files after the test finishes. That means you can change directory into TEST_TMP and import/run the modules yourself after the test has finished.

There's also a combined --debug option in runtests.py that enables everything that makes pdb usable. However:

I decided to write a srctree file to test it, so I defined classes A and B in different files, then I override
the method and the module in a setup.py file, but now I am trying to debug using these files without success.

One issue with srctree tests is that they run Python subprocesses as test commands. That cuts off external pdb runs. What I'd do is run it once with --no-cleanup and then use the resulting test directory to do your own Cython runs in it, as @da-woods suggested above. That could possibly be improved by detecting pdb and injecting a pdb command into the setup.py build runs of srctree tests automatically, but that might also be surprising and unwanted, so not sure it's worth implementing. Again, I'd just run the commands manually.

Hi! I would like to contribute to this, is it still open?

Hi! I would like to contribute to this, is it still open?

Yes still plenty of untested code. Please go ahead

I'm a first time contributor and have never contributed to anything before, do you mind giving me some tips/ pointing me in the right direction to let me write test cases for one of the codes please!

The bottom one looks to be about using break and continue in prange loops.
The main tests for that feature are in tests/run/parallel.pyx. So look at trying to create some a loop with some observable and reliable behaviour that involves break and continue. Follow the style of the existing tests in the file - these you Python's doctest module.
You can run that individual testcase with python3 runtests.py "run[.]parallel". The --no-capture` argument may be useful if you want to put breakpoints in Cython's code to check if you're reaching the right lines.
There's no need to compile/install Cython - just clone the repository.

So I have a few questions:

I noticed that on Pycharm, I am getting red lines underneath parallel, prange, threadid as shown in the picture. The message shows that pycharm cannot find a reference for the following items. My question is, is that normal or have I screwed something up?

Is there a preffered setup/ method of going through the cython files. i.e: is there a .venv that I should use or is there a particular text editor that I should use/ setup in a particular way?

The third question I have is I am uncertain as to where to write the code that I want to be tested, do i write it in the runtests.py file and then test the file? or do i write my test case in a separate file and call that one somehow?

The last question is are there any resources I can refer to finish this task? I'm going through the cython documentation to learn cython but are there any other webpages/resources that I should also study to complete this task?

Probably normal that PyCharm can't find it. I'm not 100% sure though because I don't use PyCharm.
No special venv needed really. Cython doesn't have any dependencies so I just clone the git repo and edit that.
Best test to add to is tests/run/parallel.pyx. No need to edit runtests.py - just use it to run the tests with the command line I suggested.
I don't think there's huge amounts of relevant documentation beyond the Cython documentation. I think that if you don't already know Cython as user, then trying to contribute to it is going to be hard.

See https://github.com/cython/cython/blob/master/docs/CONTRIBUTING.rst and https://github.com/cython/cython/wiki/HackerGuide#the-test-suite for some general help with getting started.


Latest patch has broken the civic/tech/policy panels

The latest path (4/14/17) has broken the functionality of the civic, tech, and policy panels. Clicking on any of those icons does nothing. You can click any of the options down the left-side panel, but actually viewing the trees is not possible.
In addition, the turn is reset to "1" and the date to 2100AD.
Any game that has this mod enabled is unplayable, including new games. New games can be created and played if the mod is disabled however.

Are you using the CQUI from the Steam workshop?
If not, make sure you are using the newest version:
https://github.com/CQUI-Org/cqui/archive/master.zip

Oh, sorry forgot to give some details.
Yes, I'm using it from the Steam Workshop and I'm running steam-os (linux), however this appears to be affecting all os versions as judging from the comments.
It is the newest version that's causing the problem.

Could you download and test the latest nightly from the link ProustIdee posted and test if the problem persists? I'm not seeing this problem on my end with the latest nightly, so it might be a problem with the steam version.

I downloaded and diff'd the directories and there the same. Are you sure that link points to the nightlys?

It's possible this could be a mac and linux issue only, but not confirmed.

I cannot replicate. Win7 Pro 64 here.

Seeing this issue on Mac, but not on my Windows box.

What version is the game on Mac/Linux? Have they received the spring patch? If not I think that might be the problem.

Fix for me was:
cd ~/Library/Application\ Support/Steam/steamapps/workshop/content/289070/870911250
git checkout d97bb439ce732e5303d1451cd3559a2063c9939f

Because the spring patch probably broke macOS

@SpaceOgre Mac/Linux haven't received the spring patch yet.
@cjcjameson any other steps?  Per another site, I did the following after the 'git checkout' command...
git checkout -b revert.spring
... no luck.  When I load a saved game, the Science/Culture/Govt issues still exist.

Yep so:

git checkout -b foobar doesn't change the code in the file tree, but it does label the version of the code which you've "checkout"ed to, for easy navigation.
Saved games get locked into using CQUI -- you can't reopen them with no CQUI
However, I don't know the details of whether rolling back to a previous commit should or shouldn't work... there might be come weird details there.
I'm almost certain that if start a new game at d97bb439ce732e5303d1451cd3559a2063c9939f, and then a fix comes out, you'll be able to continue it.

Note that in my workflow, I had been mucking around with the underlying directories a lot before coming across the minimal fix above. So I ended up nuking my install almost completely to make sure I had a fresh start (steps below). However, I don't think that should be necessary for most people...
# uninstall from Steam UI
cd ~
rm -rf Library/Application\ Support/Steam/steamapps/common/Sid\ Meier\'s\ Civilization\ VI
rm -rf Library/Application\ Support/Sid\ Meier\'s\ Civilization\ VI/
rm -rf Library/Application\ Support/Steam/steamapps/workshop/content/289070

Confirmed: I've narrowed the macOS bug to the Spring patch commit and/or one other commit (which merge-conflicts without the spring patch commit, so I didn't test it independently)
d97bb439ce732e5303d1451cd3559a2063c9939f
8018286b1fa91fcb97127e5b87c4ff6ac6c0df34
If you want to use the latest release with only those two commits removed, instead of the instructions above:
cd ~/Library/Application\ Support/Steam/steamapps/workshop/content/289070/870911250
git remote add cjcjameson https://github.com/cjcjameson/cqui.git
git checkout cjcjameson/cjcjameson_diagnosis

It is most likely 8018286b1fa91fcb97127e5b87c4ff6ac6c0df34 that is the culprit since all the new files in that commit is from the spring patch.

Looks like the issue is may be related to the naming of the new WorldTracker.lua import, which should be lowercased. Any non-lowercased files break Linux pretty much unconditionally. Yep, that's a thing that happens, so we can fix that relatively easily.
This issue shouldn't affect OSX if my theory is correct, since that OS is case-agnostic

@cjcjameson Are you testing on OSX or Linux?

I just wanted to confirm that cjcjameson suggestion above worked: I loaded my game and was able to open the panels.
By the way, I had to do a git fetch. So here's what I did to be complete:
git remote add cjcjameson https://github.com/cjcjameson/cqui.git
git fetch cjcjameson
git checkout cjcjameson/cjcjameson_diagnosis
I'm on linux by the way, and the path for linux is:
Steam/SteamApps/workshop/content/289070/870911250

@islandboy Try renaming WorldTracker.lua to worldtracker.lua. Tell me if that fixes the issue for you

@chaorace Can you explain this:
"Any non-lowercased files break Linux pretty much unconditionally"
Ok, I can test that. Do you want me to undo @cjcjameson changes?

@chaorace Wait, I don't think the lower case is the issue because with @cjcjameson patches that file is still camel-humping: "WorldTracker.lua". And the game seems to be working.

@islandboy Yes, just try the change with the base nightly
Linux is a case-sensitive kernel. That means a file named WorldTracker.lua and a file named worldtracker.lua are not equivalent in Linux-based environments. This is not the case in OSX or Windows, which ignore case and would consider these two files equivalent.
It would seem that, on Linux by some process, the internal file-tree is still case sensitive, yet considers all files in the environment lowercased, unless imported via mod, in which case the case of the filename is preserved. This means mods can very easily break Linux games unless they take care to lowercase all file names
Also, WorldInput.lua didn't exist before the commit in question, which probably explains why it doesn't affect anything despite looking the same (since it's no longer included in the file manifest)

@islandboy While I've got you, could you also provide a gist of the the contents of lua.log, which is found under the same general location as your save files (directory structure would look like "Sid Meier's Civilization 6/logs")

@chaorace I'll let me give that a shot
Ok, this is a true statement:

Linux is a case-sensitive kernel. That means a file named WorldTracker.lua and a file named worldtracker.lua are not equivalent in Linux-based environments.

But you lose me here:

It would seem that, on Linux by some process, the internal file-tree is still case sensitive, yet considers all files in the environment lowercased...

Linux is always consistent: Those two files will always be different in linux.

@chaorace Renaming that file didn't work.
By the way, I couldn't find a lua.log file in the steam directory structure. Maybe it's called a different name is linux. Is there a string I can grep for?

The internal file tree I'm referring to is the one used in the Civ 6 engine. The filetree is flat, unlike what we see in modern operating systems. This means all files, from mods and base game, are stitched together into a single working directory without subfolders. This process is probably employed so that we're free to organize the files as we see fit, independent of how they may be arranged in the base game. The issue is that the original case of the base-game files is not preserved when creating this flat working directory. They're all treated as lowercase! This is an issue because apparently this lowercasing process is skipped when importing mod files to the same working directory. Allow me to demonstrate with a flow-chart and example:
We start with a complex tree like this:
Base - Dir1 - File2
- File3
- Dir2 - File4
- File1
All lua/xml files are taken from the game directory and stitched into a working, flat directory, the tree now looks like this:
Base - File1
- File2
- File3
- File4
The files are now all converted to lowercase:
Base - file1
- file2
- file3
- file4
Mod files are now written into the working directory, replacing any files with a matching name. On Windows/OSX, with files named "File1", "file2", and "New1"  that process looks like this:
Base - file1 <- File1 overwrites
- file2 <- file2 overwrites
- file3 --No match
- file4 --No match
- New1 --New file created
However, on Linux-based OSes, that process breaks down because the lowercasing step doesn't happen first, it looks like this instead:
Base - file1 --No match
- File1 <- New file created
- file2 <- file2 overwrites
- file3 --No Match
- file4 --No Match
- New1 <- New file created
Finally, the mod files are renamed to lowercase:
Base - file1 --No match
- File1 <- Can't lowercase, file discarded
- file2 <- file2 overwrites
- file3 --No Match
- file4 --No Match
- new1 <- Renamed to lowercase
The lua.log file is not in the workshop directory, it's in the same directory where your saves are stored

I searched the entire steam directory structure and I have no lua.log file. It might be a different name in linux... I'm guessing. And I have no idea where the saves are stored. Let me see if I can find it...

Anyways, it looks like this issue affects OSX as well based on feedback from the comments, so it's not an issue specifically with the lowercasing thing. That means it's probably an issue with an engine difference between Spring/Summer patches. I'm setting up a SteamOS install on my Hyper-V server now and will be harvesting error logs myself when that finishes. Though, my only ISP option is hot garbage and I'm pulling about  1 mb/s right now, so the necessary downloads could take a few hours. If anyone could send lua.log pastes in the meantime, that may considerably expedite the process

Ok thanks!
I'm trying to look for the saves directory but the directory structure is a mess in linux...
What's in the lua.log file? Maybe I can do a string search.

It's going to definitely be called "Lua.log", probably in your "My Games" folder or a similar folder in your user home directory.

Holy crap! Found it!
ok, so on linux the games are saved here for civ6:
~/.local/share/aspyr-media/Sid\ Meier\'s\ Civilization\ VI/Saves
And I found the Lua.log. Let me send that to you.
By the way, the Lua.log can be found here:
~/.local/share/aspyr-media/Sid\ Meier\'s\ Civilization\ VI/Logs/Lua.log

Here's the gist:

Let me know if you have any issues reading the file.
I'm also attaching the file here as well:
Lua.log.gz

FTFY: https://gist.github.com/islandboy/8b14404f78e41c372ae82817113f8e52

The casing of WorldTracker.lua was not changed in that commit, atleast not what I can see. It was camel cased when I started working on that specific feature. But it should be changed to avoid problems in the future.

@SpaceOgre I agree and also, yes
Now that we've captured a stack trace of the given issue, I'm reverting the workshop release to the last weekly until we can address this issue

Just to keep you guys updated, I'm pursuing a solution using azure, which is probably going to be the most cost effective solution for me running a linux test environment (I can't run an environment like this on my machine natively because of a very hacky custom driver I use with nonstandard hardware). I've hit a minor snag however with my azure account getting flagged so I'm waiting on support to clear me before I can patch this

The Mac/Linux update was supposed to be out mid April, so could drop any day now. Not sure if you'd wanna spend the time debugging when it could resolve itself soon enough anyway :)
https://support.aspyr.com/hc/en-us/articles/216979606-Civilization-VI-STEAM-KB

It's mostly an issue of keeping the weekly current with the nightly without fragmenting the workshop userbase, I wouldn't be too upset if they dropped it soon though

@chaorace I can test the linux stuff if you want. The only caveat is that my schedule is unpredictable. But I'll do my best. I some time in the evenings when I get home from work. The weekends I have a bit more time.

Version 1.0.4 was just released for Mac. As soon as I download the update (my version is through Apple Store) I will try out the mod and see if the issue remains.

Welp, I guess this issue resolved itself then. Yay?


Fix documentation for the --stdout flag

Change description
The --stdout flag in the docs was incorrect (was std_out, but should be stdout). Fixed the docs and added a test for verification.
What problem does this solve?
The use of the more verbose flag failed with a runtime exception saying the --std_out flag was invalid.
Checklist

[x] I have added unit tests to cover my changes.
[x] I have added documentation to cover my changes.
[x] My changes have passed unit tests and have been tested E2E in an example project.

Pull Request Test Coverage Report for Build 58a15306a1018caf6ac4cb60de6dc7871f3bca9f-PR-61

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 85.147%

Totals

Change from base Build 47017a28f9a4cd21e597179427b44b05febe0f2b:
0.0%

Covered Lines:
579

Relevant Lines:
680

💛  - Coveralls

Thanks for the doc fix and the test!


fatal error: 'tr1/unordered_map' file not found

Sorry! My English is not good,so I hope you can understand the issues
Tell the author that this fails on your system:
npm ERR!     node-gyp configure build

Same here.. anybody have suggestions for a replacement of this lib?
 node-gyp configure build

  CXX(target) Release/obj.target/native/src/hashtable.o
In file included from ../src/hashtable.cpp:1:
../src/hashtable.h:7:10: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>

I would suggest the builtin Map: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map


Disable PIE for non-Ubuntu distros

In reference to my comment this commit will disable PIE for non-Ubuntu distros again. Tested on Debian 9.
Or should I better disable it for Debian only?

@ffflorian I think should be OK to just disable the way you did as IDK how many other platforms would face the same issue and then instead of catering to all, we'll just disable PIE for now. However i think sometime in the future we should move to not disabling PIE and if ppl face problems in their specific platforms, they can just do it for their builds (we'll mention it in the README or some place when we change it).

@ustulation Sounds good, thanks!


idiot-proofing: string copies instead of references

When using the inflate operator the layers are adjust at the beginning
  const cost_map::Matrix& data_source = cost_map.get(layer_source);
  cost_map.add(layer_destination, data_source);

https://github.com/stonier/cost_map/blob/indigo-devel/cost_map_core/src/lib/operators/inflation.cpp#L32-L33
but later the layer reference is used to retrieve data
 enqueue(cost_map.get(layer_source), cost_map.get(layer_destination), i, j, i, j);

https://github.com/stonier/cost_map/blob/indigo-devel/cost_map_core/src/lib/operators/inflation.cpp#L51
At which point the string reference may point to an invalid memory address, since the std::vector freely shuffles memory around as needed, when adding or removing an item from the vector.
Passing string copies instead of references avoids this issue and adds only little overhead.

Deflate is safe as it is right now, since the matrix is retrieved by the layer name before the destination layer is added.
However, if someone later shuffles the code around and adds the new layer before retrieve the source, we end up in the same situation. Hence I propose the same approach as for inflate: use string copies.

Updated description.
@stonier please merge, if this passes your review.

:neckbeard:


Add Auto Comment Feature to Improve Collaboration

Issue Description:
As an active contributor to your open-source project, I believe that implementing an auto-comment feature would greatly enhance collaboration and communication within the project. This feature would automatically generate comments in response to specific events, such as when an issue is opened, a pull request is created, an issue is assigned, or an issue is unassigned.
Feature Details:

When an issue is opened, the auto-comment should greet the author and provide a brief acknowledgement and request for additional context.
When a pull request is opened, the auto-comment should greet the author, express gratitude, and remind them to follow the project's contributing guidelines.
When an issue is closed, the auto-comment should thank the author for their contribution and encourage further engagement.
When an issue is assigned to someone, the auto-comment should notify the assignee and encourage them to start working on it.
When an issue is unassigned from someone, the auto-comment should notify the assignee about the change and suggest reassignment if they are offline.

Benefits:

Improved communication and engagement with contributors.
Provides clear instructions and acknowledgements for various events.
Enhances collaboration by setting expectations and providing reminders.
Reduces manual effort by automating comment generation.

Acceptance Criteria:

The auto-comment feature should be implemented using the "wow-actions/auto-comment" GitHub Action.
Comments should be appropriately customized for each event, mentioning relevant parties and providing the necessary information.
The auto-comment workflow should trigger on the following events: issues opened, pull requests opened, issues closed, issues assigned, and issues unassigned.
The feature should be added to the project's existing GitHub Actions workflow file.

Additional Context:
Feel free to ask any questions or seek clarification regarding the auto-comment feature. I'm excited about contributing to your project and believe this enhancement will greatly benefit its community.

@Nishitbaria sounds interesting! Go ahead, assigned it to you 🚀

@Nishitbaria hey man can we work together on this issue. Thanks

Yes why not 😄

@Nishitbaria @Stroller15 When I merge the PR, only the person who opened the PR gets the points for it in GSSoC.
Also, Nishit, it's been a while, do you want to continue working on this?

Yes I want to work on this


small-updates

not clear what num_line_x & num_line_y represent: https://github.com/Fudenberg-Research-Group/dynamic_extrusion_boundaries/blob/0fdff1920574203231e75c3548a14f926dc6f8ad/dynamic_extrusion_boundaries/util_visual.py#L7C57-L7C80

a docstring for the main function FRiP would be nice: https://github.com/Fudenberg-Research-Group/dynamic_extrusion_boundaries/blob/0fdff1920574203231e75c3548a14f926dc6f8ad/scripts/processing/frip.py#L41

I think num_line_x and y are needed in linespace function. Isn't it?


User should be able to scan a QR code

From the main page, user should be able to scan a QR code.
Checkout libraries like Instascan
Scanning a QR code successfully, shows a small microinteraction indicating success and returns the user to the main page.

@agneym Picking this up.


Windows CI

Adding Windows CI

Next time only make the PR once you think it's all good on your fork. Just keeps things cleaner on the repo

Fair ig but CI was did appear to be triggered without the PR

You have to create an Action on your own fork

Should I add Windows CI to the disabled workflows? @ac-freeman
I'm a bit curious why they're disabled...

Not sure what you mean

This only modifies the rust_action_test_and_docs.yml workflow. Should I modify the other two? It seems that they are disabled ([1], [2]).

Those are old, just ignore them

In that case you can merge this, or let me know if there's a better way to clean it up. I'm not sure which branch you want the bandwidth limiting to be in but please rebase that one if it isn't the main branch so I can work on that next.


SyntaxError: Unexpected token

We've been using Sage for a few months without issue but now when we try to run gulp watch we get this error
SyntaxError: Unexpected token }
at Object.parse (native)
at module.exports (C:\wamp\www\multisite\wp-content\themes\imgtec\node_modules\asset-builder\lib\readManifest.js:19:15)
at new module.exports (C:\wamp\www\multisite\wp-content\themes\imgtec\node_modules\asset-builder\lib\Manifest.js:28:34)
at module.exports (C:\wamp\www\multisite\wp-content\themes\imgtec\node_modules\asset-builder\index.js:17:10)
at Object. (C:\wamp\www\multisite\wp-content\themes\imgtec\gulpfile.js:11:40)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
Does anyone have any ideas how we fix this because as far as we can tell nothing changed?

Did your bower.json file change? Looks like it could be a syntax error there.

If this were a bug it would be over at https://github.com/austinpray/asset-builder


exclude in pyrightconfig.json not working

Environment data

Language Server version: 2022.9.40
OS and version: on remote linux server ubuntu 18.04
Python version (& distribution if applicable, e.g. Anaconda): 3.9

Describe the bug
I don't want the fs watcher to track the update of the log files (which updates very frequently), and here is my configuration pyrightconfig.json
{
    "verboseOutput": true,
    "exclude": [
        "**/log"
    ]
}

But it just not working.
...
[Info  - 12:59:00] (71313) SourceFile: Received fs event 'change' for path '/mnt/workspace/zhenghh/Transducer-dev/egs/aishell/exp/test-ctc-base/log/20220929-062141/events.out.tfevents.1664403701.ubuntu.45913.0'
[Info  - 12:59:00] (71313) SourceFile: Received fs event 'change' for path '/mnt/workspace/zhenghh/Transducer-dev/egs/aishell/exp/test-ctc-base/log/20220929-062141/events.out.tfevents.1664403701.ubuntu.45913.0'
[Info  - 12:59:00] (71313) SourceFile: Received fs event 'change' for path '/mnt/workspace/zhenghh/Transducer-dev/egs/aishell/exp/test-ctc-base/log/20220929-062141/events.out.tfevents.1664403701.ubuntu.45913.0'
[Info  - 12:59:01] (71313) SourceFile: Received fs event 'change' for path '/mnt/workspace/zhenghh/Transducer-dev/egs/aishell/exp/test-ctc-base/log/20220929-062141/events.out.tfevents.1664403701.ubuntu.45913.0'
[Info  - 12:59:01] (71313) SourceFile: Received fs event 'change' for path '/mnt/workspace/zhenghh/Transducer-dev/egs/aishell/exp/test-ctc-base/log/20220929-062141/events.out.tfevents.1664403701.ubuntu.45913.0'

exclude excludes files from analysis, not from the file watcher. See https://github.com/microsoft/pyright/issues/2108#issuecomment-884932599

Hello @debonte,
In this issue joaotavora/eglot#1224 issue a user is reporting that watching too many files in .venv (presumably uselessly since these files aren't changed during development) exhausts the file-watching capabilities of certain clients in certain operating systems (such as MacOSX).
If something is excluded from analysis, why not also exclude it from the file watcher?  If this is "by design" what is the rationale for this design?

There's a separate setting for excluding things from the file watcher. VS code has a setting for that : files.exclude.
It's described here:
https://code.visualstudio.com/docs/getstarted/settings

Files that are part of the project can import symbols from files in the venv. That means type analysis is affected by these files. If files in the venv change (e.g. due to a pip install or pip uninstall), type analysis needs to be performed again on any dependent files. The contents of the venv also affects language server features like completion suggestions with auto-import. When the venv files change, any cached auto-import information needs to be invalidated and recomputed.
The exclude setting in the pyrightconfig is completely unrelated to import dependency tracking. It should have no bearing on which files the fs watcher is applied to.

Certainly some files in .venv are probably worth analysing and/or watching.  But as far as I understand, Pyright requests by default that every file in the project is analysed.  I'm not a Pyright user but the bug reporter sent me this report of a server request  (this is JSON represented as Lisp, you should be able to read it).
[server-request] (id:4) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 4 :method "client/registerCapability" :params
  (:registrations
    [(:id "5f68e7c3-d6be-46f9-b8f5-18ca46acd165" :method "workspace/didChangeWatchedFiles" :registerOptions
       (:watchers
         [(:globPattern "**/pyrightconfig.json" :kind 7)
           (:globPattern "**" :kind 7)]))]))

Presumably the user's .venv file has a lot of files and maybe -- rather probably -- not all of them are Python files in the conditions you describe.  So why ask the client to spend system resources to watch them?  What files is Pyright actually interested in?

None of the files in a venv are "analyzed", but their contents (and even their presence) affects the type analysis of files within your project. They also affect which symbols are available for completion suggestions.
The addition or removal of any importable file (".py", ".pyi", ".pyc", ".so", etc.) or any directory within the site-packages directory can affect import resolution. Likewise, any of these can necessitate the addition or removal of auto-import entries. It's reasonable for users to expect that when a change is made to their venv, a language server will be smart enough to reflect those changes without restarting the language server.
Modern file systems allow a caller to install a file watcher on a directory recursively. Changes are then reported for any directory or file within that hierarchy. This doesn't require significant system resources.

The addition or removal of any importable file (".py", ".pyi", ".pyc", ".so", etc.)

If these are the files you are interested in, just code this into Pyright so that -- instead of ** -- it requests watcher in files.  Aren't LSP globs flexible enough for you?  Do you really need to watch ** i.e. ALL files??

This doesn't require significant system resources.

Depends on the implementation underlying platform.  On Linux, you're right, on Mac OSX, the kqueue mechanism doesn't scale well and hits a limit.  And there's also the case of all the wasted client->server bandwidth for files that Pyright probably couldn't care less about.

All directories and nearly every file within a venv need to be watched.

Depends on the implementation underlying platform. On Linux, you're right, on Mac OSX, the kqueue mechanism doesn't scale well and hits a limit.

I regularly use pyright and pylance with venvs that contain 10K+ files on a Mac, and I don't see problems with recursive fs watchers or watcher exhaustion.
Also, my understanding is that this is a per-process limit, not a global limit, so if some fs-specific limit is reached, it should affect only pyright, not other processes. Pyright degrades gracefully in this case. It sill runs, but you will need to restart the language server if you make changes to your venv.

And there's also the case of all the wasted client->server bandwidth

This hasn't been an issue. Files and directories within a venv don't typically change very often. Only when installing and uninstalling libraries.
Are you concerned about some theoretical problem, or are you observing an actual problem? If it's an actual problem, what symptoms are you seeing?

Are you concerned about some theoretical problem, or are you observing an actual problem? If it's an actual problem, what symptoms are you seeing?

Actual problem.  See issue I linked to: https://github.com/joaotavora/eglot/issues/1224

All directories and nearly every file within a venv need to be watched.

Hmm, just how much is in that "nearly"?  In the user's MRE, they install numpy and numpydoc only.  Just from those two packages, I ran the following command (taken from your account of the file extensions needed to track dependencies).
❯ find .venv | grep  -E -v "\.(py.?|so)$" | grep -E -v "^.venv/bin" | wc -l
3410

That list is full of C, fortran, csv files, txt files.  Does Pyright need to be notified of changes to them?  Why?

I regularly use pyright and pylance with venvs that contain 10K+ files on a Mac, and I don't see problems with recursive fs watchers or watcher exhaustion.

That's because your're either using a client that doesn't use kqueue watchers or you're using a client that actively refuses to put watches in .venv ,such as this one. https://github.com/emacs-lsp/lsp-mode/blob/c8bbdb80ed2dbfdbf55b6c4058308dab092306b5/lsp-mode.el#L366.  The latter, according to your account, will have broken dependency tracking.
I maintain another client, whose platform on Mac OS also uses kqueue. I'm asking this question before offering my users a IMHO cumbersome & error-prone option of ignoring server watches.

This hasn't been an issue. Files and directories within a venv don't typically change very often. Only when installing and uninstalling libraries.

Anyway, this isn't about .venv only.  It's not very hard to imagine projects having many more files changed very frequently.  This users's log files, for example: https://github.com/microsoft/pyright/issues/2108#issuecomment-884932599

Also, my understanding is that this is a per-process limit, not a global limit, so if some fs-specific limit is reached, it should affect only pyright, not other processes. Pyright degrades gracefully in this case. It sill runs, but you will need to restart the language server if you make changes to your venv.

The watchers are owned by the process which runs the client.  That process emacs in this case, can be connecting to many other LsP servers (for other documents in other languages).  So it affects other processes.
Here's the relevant interaction from my user's log file:
[server-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "window/logMessage" :params
  (:type 3 :message "Found 11 source files"))
[server-request] (id:4) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 4 :method "client/registerCapability" :params
  (:registrations
    [(:id "5f68e7c3-d6be-46f9-b8f5-18ca46acd165" :method "workspace/didChangeWatchedFiles" :registerOptions
       (:watchers
         [(:globPattern "**/pyrightconfig.json" :kind 7)
           (:globPattern "**" :kind 7)]))]))
[client-reply] (id:4) ERROR Fri May 12 02:33:41 2023:
(:jsonrpc "2.0" :id 4 :error
  (:code -32603 :message "Internal error"))
[server-notification] Fri May 12 02:33:41 2023:
(:jsonrpc "2.0" :method "textDocument/publishDiagnostics" :params
  (:uri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles/tree.py" :version 0 :diagnostics
    []))
[stderr] Error: Internal error
[stderr]     at handleResponse (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:565:48)
[stderr]     at handleMessage (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:345:13)
[stderr]     at processMessageQueue (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:362:17)
[stderr]     at Immediate.<anonymous> (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:334:13)
[stderr]     at processImmediate (node:internal/timers:478:21)

My understanding from this is that Pyright, upon receiving the "internal error" report by the client that the humongous file-watching request couldn't be honoured, just dies.  My client will then restart it a few times, but that will just make Pyright start up again and re-request the watchers.   I don't have Mac OS to test this.
Finally, curiously the users seems to hint that Pyright (when ran standalone) has file watching capabilities itself (but they don't seem to work also).

That's because you're either using a client that doesn't use kqueue watchers

Yeah, that would explain it. I'm using VS Code. Perhaps you would consider updating your client to use a more robust fs watcher mechanism on MacOS? Even if we were to change pyright to specify only a subset of file types (plus all directories), I think you'd find that you'd hit the same issue for all but the smallest of venvs. In other words, I don't think this is something we can fix within pyright.
I think you'll need to find a workaround for this issue in your client if you want to work with the pyright language server. One option is to report that your client isn't compatible with server-side file watchers, and pyright will then not attempt to request any file watchers from your client.

Pyright (when ran standalone) has file watching capabilities itself

Yes, if you run the CLI version of pyright in --watch mode, it uses file watchers to determine what has changed in the venv. If it's unable to install a file watcher, it still works fine, but users need to restart pyright if they modify their venv.

I think you'll need to find a workaround for this issue in your client if you want to work with the pyright language server. One option is to report that your client isn't compatible with server-side file watchers, and pyright will then not attempt to request any file watchers from your client.

You mean "report that your client isn't compatible with client-side file watcher", right??  Well, it is compatible and no other server has this problem. But yes, my client's users can do that.  But will Pyright server-side's watching capabilities kick in then?

Perhaps you would consider updating your client to use a more robust fs watcher mechanism on MacOS?

That's being worked on AFAIU, but it there's still old versions to think about.  And I think it's infinitely easier to supply an option in pyright, similar to the exclude.
Or to just not request the ** blob.  I can't think of a good reason not to fix that ** blob on Pyright's side since at the very least Pyright's insistence on keeping it will still lead to tons of useless messages from the client -> server.  On every platform, and regardless of venv or not.  It's quite possible it's a one-line fix somewhere in your code:
diff --git a/packages/pyright-internal/src/languageServerBase.ts b/packages/pyright-internal/src/languageServerBase.ts
index 7e296a021..27cba25a7 100644
--- a/packages/pyright-internal/src/languageServerBase.ts
+++ b/packages/pyright-internal/src/languageServerBase.ts
@@ -1390,7 +1390,7 @@ export abstract class LanguageServerBase implements LanguageServerInterface {
         // Set default (config files and all workspace files) first.
         const watchers: FileSystemWatcher[] = [
             ...configFileNames.map((fileName) => ({ globPattern: `**/${fileName}`, kind: watchKind })),
-            { globPattern: '**', kind: watchKind },
+          { globPattern: '*.{py,pyc,pyi,so}', kind: watchKind },
         ];

         // Add all python search paths to watch list

@erictraut the above is on MacOS Venture 13.3.1 and Pyright 1.1.307.
Now, I don't entirely understand what is going on, but, could you help me understand or tell me why my questions don't make sense?
What is this bit saying,
Search paths for /Users/tfn/Desktop/mre
  /Users/tfn/.cache/pyright-python/1.1.310/node_modules/pyright/dist/typeshed-fallback/stdlib
  /Users/tfn/Desktop/mre
  /Users/tfn/Desktop/mre/typings
  /Users/tfn/.cache/pyright-python/1.1.310/node_modules/pyright/dist/typeshed-fallback/stubs/...
<EMAIL_ADDRESS><EMAIL_ADDRESS>  /Users/tfn/Desktop/mre/.venv/lib/python3.11/site-packages

Is it that it is looking for changes to files in those directories? If so, why do we need to look for changes in /Users/tfn/.cache/pyright-python/1.1.310/node_modules?
Also with<EMAIL_ADDRESS>are we expecting changes to the Python standard libraries, e.g., for completion?
Watcher could not use native fsevents library. File system watcher disabled. Why not? Is this due to the excessive file watch requests? If yeah, should this not be handled in someway within Pyright since I am running it with nothing else and this seems like an undesirable behaviour caused solely by invoking Pyright in the Terminal.app.


python3Packages.wakeonlan: 1.1.6 -> 2.0.0

Motivation for this change
Update to latest upstream release 2.0.0
Change log: https://github.com/remcohaszing/pywakeonlan/releases/tag/2.0.0
openwebifpy is only available for Python > 3.6. Thus, wakeonlan as well.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 118765 run on x86_64-linux 1

4 packages built:

python38Packages.openwebifpy
python38Packages.wakeonlan
python39Packages.openwebifpy
python39Packages.wakeonlan

@ofborg eval

Nitpick: The license should be updated to MIT.

Changed in https://github.com/NixOS/nixpkgs/commit/f37e906a1841784de18b9cee03a7a8e274ab29af


Issues with signing the app

Failed to load signer "signer #1": /home/runner/work/mobile_app/mobile_app/android/app/build/outputs/apk/release/signingKey.jks entry "***" does not contain a key
I am base64 the pepk file, could that be the issue?

same issue here. Did you find any solution for it? @nadav-virdee  @r0adkll

Hey, dont remember exactly but I would clean the project and cache


Fix margin and alignment in dashboard repolist

Seems this has recently regressed, previously, there was a significant whitespace between icon and text, but it seems to be gone, so I added the margin and also vertically aligned the icon because it was slightly misaligned.
Before:

After:

Putting in draft, need to fix text truncation in the flexbox.

Truncation fixed. The repo name can now stretch the full box instead of the previous arbitrary 70% value:

make L-G-T-M work

please send backport

Was already backported to 1.18 in https://github.com/go-gitea/gitea/pull/22122 and 1.17 does not have the bug.


Fix timezone conversion issues

https://www.pivotaltracker.com/story/show/91500052
content designers noticed that scheduling an edition at 00:01 was showing-up as 01:01 in the publishing tool.
when the edition schedule time is submitted through the form, the ApplicationController#squash_multiparameter_datetime_attributes method squashes multi-parameter datetime into a single attribute before saving the field in the database. this logic was ignoring the local timezone, and made all field handling logic beyond this point think that publish_at field is in UTC.
using Time.zone.local preserves the timezone information, and fixes this issue.
also correcting the integration test that was masking this issue.

Code looks good to me. Testing locally to confirm fix.

It's confusing that the downtime feature didn't have the same problem, it also goes through the squash_multiparameter_datetime_attributes method, and uses time_select and date_select.

Tested scheduled publishing and downtime locally, worked as expected.

@fofr: thanks for reviewing! the only reason i can think of why downtime timestamps didn't need to be updated is because we're doing something extra for scheduled publishing. that seems unnecessary, as rails converts from local timezone to UTC before saving time fields to database.
i'm testing to find-out if that can be removed, and will create a PR shortly.


Documentation

can i make documentation for this?
started using it yesterday and expected it to be similar to discord.py, but it's vastly different, finding my way around so far and want to help with the documentation

This'd be incredibly helpful honestly, I tried to use sphinx before but I kept running into issues.

Alright, I'll work on it during the week, I'll work with the current features and update along the way.
I'll keep an eye out for the docstring changes as well

I've attempted to start writing some documentation in https://github.com/Shell1010/Selfcord/wiki, edit it to your hearts content.

what


[WIP] CHK-4719: Add EPS compatibility.

Please refer to https://github.com/bold-commerce/adobe-commerce-bold-checkout-payment-booster/pull/76


Make block filters case-insensitive

There is an issue with using checksummed addresses in block filter query: https://github.com/streamingfast/substreams-foundational-modules/issues/9
I think it makes sense to make block filter case-insensitive. At least I don't see why not.

It could be changed, but first we'll need some version-checking because it is a change that would "silenlty change the output of existing substreams". Not a prio.


Google maps Webview and Navigator API

Hello, has anyone tried using google maps inside a webview on Android? Whenever I use a Navigator the webview doesn't show anything and when I use the webview on its own it works.
The maps site is written in react js
The iOS version of the app works fine. I am using RN0.22
I am using this webview for geolocation: https://github.com/lucasferreira/react-native-webview-android

@facebook-github-bot stack-overflow


Possible rate limit or other issue

If you run terraform plan back-to-back, it seems Namecheap will throttle your requests. I get the following error (in summary) from terraform:
expected element type <ApiResponse> but have <html>
Terraform Version
Terraform v0.11.10
Namecheap v1.1.1
Affected Resource(s)

namecheap_record
namecheap_ns

Debug Output
Terraform trace output towards the end of a terraform apply I executed back to back a few times:
2018/12/09 12:30:04 [TRACE] [walkRefresh] Exiting eval tree: namecheap_record.copelander-com
2018/12/09 12:30:05 [ERROR] root: eval: *terraform.EvalRefresh, err: namecheap_record.www-copelander-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>
2018/12/09 12:30:05 [ERROR] root: eval: *terraform.EvalSequence, err: namecheap_record.www-copelander-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>
2018/12/09 12:30:05 [TRACE] [walkRefresh] Exiting eval tree: namecheap_record.www-copelander-com
2018/12/09 12:30:05 [ERROR] root: eval: *terraform.EvalRefresh, err: namecheap_record.www-remotedevforce-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>
2018/12/09 12:30:05 [ERROR] root: eval: *terraform.EvalSequence, err: namecheap_record.www-remotedevforce-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>
2018/12/09 12:30:05 [TRACE] [walkRefresh] Exiting eval tree: namecheap_record.www-remotedevforce-com
2018/12/09 12:30:05 [TRACE] dag/walk: upstream errored, not walking "provider.namecheap (close)"
2018/12/09 12:30:05 [TRACE] dag/walk: upstream errored, not walking "root"

2018/12/09 12:30:05 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: Error refreshing state: 3 error(s) occurred:

* namecheap_record.copelander-com: 1 error(s) occurred:

2018-12-09T12:30:05.199-0800 [DEBUG] plugin.terraform-provider-namecheap: 2018/12/09 12:30:05 [ERR] plugin: plugin server: accept unix /var/folders/03/blcr90b52jb9d_4qphynlz680000gn/T/plugin407438253: use of closed network connection
2018-12-09T12:30:05.199-0800 [DEBUG] plugin.terraform-provider-google_v1.19.1_x4: 2018/12/09 12:30:05 [ERR] plugin: plugin server: accept unix /var/folders/03/blcr90b52jb9d_4qphynlz680000gn/T/plugin922996050: use of closed network connection
* namecheap_record.copelander-com: namecheap_record.copelander-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>
* namecheap_record.www-copelander-com: 1 error(s) occurred:

* namecheap_record.www-copelander-com: namecheap_record.www-copelander-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>
* namecheap_record.www-remotedevforce-com: 1 error(s) occurred:

* namecheap_record.www-remotedevforce-com: namecheap_record.www-remotedevforce-com: Couldn't find namecheap record: expected element type <ApiResponse> but have <html>

2018-12-09T12:30:05.202-0800 [DEBUG] plugin: plugin process exited: path=/Users/me/my_project/.terraform/plugins/darwin_amd64/terraform-provider-google_v1.19.1_x4
2018-12-09T12:30:05.203-0800 [DEBUG] plugin: plugin process exited: path=/Users/me/.terraform.d/plugins/terraform-provider-namecheap
Error: terraform apply failed.

Expected Behavior
I know when AWS hits rate limits, terraform will exponentially back-off from hitting the API too much. I'd expect if we hit any rate limits with namecheap, we would back-off, and retry. Unfortunately they are not returning anything in their headers I can see for rate limiting which I know some APIs do.
Actual Behavior
Namecheap API seems to bomb out somehow. Getting back this error expected element type <ApiResponse> but have <html>
Steps to Reproduce
If you have a dozen of so records managed by terraform with namecheap. Run:

terraform plan over and over till the API bombs out.

References
Maybe it has something to do with rate limiting? Still don't know why it says "html" though. A capped out response from namecheap looks like this for me hitting getList dns over and over with my browser.
<ApiResponse Status="ERROR">
<Errors>
<Error Number="500000">Too many requests</Error>
</Errors>
<Server/>
<GMTTimeDifference/>
<ExecutionTime>0</ExecutionTime>
</ApiResponse>

Maybe there is a way to account for this (if this is the issue I guess, I'm still assuming this has to do with rate limiting since the first few terraform plan jobs will work fine until it bombs out) and "backoff of the API" then retry like AWS and other providers do?
Thanks again for these great go libraries.

Makes sense. I pushed off a backoff aware client until it was needed. Do you know if they return a header regarding the rate limit?
We can look to see if there are better calls to minimize the requests too.

Regarding a retry/backoff aware http.Client I'd recommend using https://github.com/hashicorp/go-retryablehttp - thoughts? We should be able to hook in a Backoff func to scrape the XML or a header value.

This is very much an RPC style API... response is a 200 even on error. No rate limiting headers in the response before or during a throttle. Not sure if we should ask support about what the limits are because I don't see anything in the docs. That library looks good, maybe if we see ApiResponse  Status="ERROR" with the Error # 500000 then we start backing off a bit. I'm going to take a look if there are examples out there of retryablehttp being used in a tf provider.
The response we can parse:
<ApiResponse Status="ERROR">
<Errors>
<Error Number="500000">Too many requests</Error>
</Errors>

If we also figure out what the response is when the ip isn't whitelisted, perhaps we can provide better feedback on that. Right now if you are on a new ip, you get back namecheap_record.example-com: namecheap_record.example-com: Couldn't find namecheap record: Could not find the record which could be confusing for a dev seeing this and not realizing it is whitelist ip related.

 Checkout the PR https://github.com/adamdecaf/terraform-provider-namecheap/pull/11
Looks like it is working well thus far. Appreciate your feedback, wondering a bit about the mutexes and if I messed anything up there.

@OGProgrammer Can you try out master? It worked for me to create a record, so I hope the retries work for you also! If so I'll make a proper release.

Tested master and it so slowww. I think you need to double check whats up with your branch. The branch I have hotfix/rateThrottling is blazing fast, and master seems to cause issues where namecheap returns some error. If you could pass the error up somehow through the namecheap go depenedency then we could get more visibility maybe?
Not sure what the issue is but I'm sticking with my branch in the meantime.

@OGProgrammer Whoops. I've fixed that in https://github.com/adamdecaf/terraform-provider-namecheap/commit/bab5a379587159efacae63b2bf52653664c82707. It works for me to create a record in 6-9s and delete one in ~6s. Hope it's fast for you too now!

Looks good now!

Awesome! I released 1.2.0 which has binaries to download
https://github.com/adamdecaf/terraform-provider-namecheap/releases/tag/1.2.0
Let me know if you have other issues, but thanks for the help on this!


SHORTEST path restriction

Using shortest on a multi rel query such as
MATCH (f:Tab)<-[:Rel* SHORTEST 0..]-(t:Tab)-[:Err*1]->(i:Tab)
WHERE EXISTS {MATCH (t)<-[:Rel* SHORTEST 0..]-(ti:Tab) WHERE ti.name = $tname } RETURN *

raises a runtime error
RuntimeError: Binder exception: Lower bound of shortest/all_shortest path must be 1.
For my use case, shortest makes perfect sense.
From documentation: Further notes on shortest path We force the lower bound of shortest path to be 1 to avoid ambiguity.
The request is to remove that limit.

I don't fully understand the question but the error is just saying change the 0 in this part [:Rel* SHORTEST 0..] to 1. A Path of length 0 is not a valid path, so we need at least 1 edge to be in the path.

In this case, I want f or t, whichever is the shortest and only that. How else can  that be achieved?

I would try to use CASE statement that checks the lengths of the variable lengths. I probably would not put the variable-length path to ti in WHERE EXISTS. Instead, maybe directly put into MATCH. Something like this:
MATCH (f:Tab)<-[e1:Rel* SHORTEST 1..]-(t:Tab)-[:Err*1]->(i:Tab), (t)<-[e2:Rel* SHORTEST 1..]-(ti:Tab)
WHERE ti.name = $tname
RETURN
  CASE
     WHEN length(e1) < length(e2) THEN t
     ELSE ti
  END AS s;

You can project more than t and ti.

thanks! I'll try that

This doesn't do what the og query does. The only alternative I can think of is to recompile kuzu and comment out the check.

Can you say a bit more? What do you mean by the "og query"? What's the query trying to compute. I wouldn't edit Kuzu to run a query. It will likely not work and compute the right thing. It's better for us to understand exactly what you're trying to do and what the right query in Cypher would be.

Also, it would be much more efficient to iterate on this if you came to Discord: https://discord.gg/VtX2gw9Rug.

Consider this graph

Here I want to be able to find shortest path around Rel2 which may have Rel1 at either side or not.
for start node node1 and end node node6, this should return node1 as the start base and node4 as the end base node.
for start node node2 and end node node6, this should return node5 as the start base and node4 as the end base node.

I can think of two options. But before that: I think you need to be careful about whether you want the edge direction in the (sn:Node)<-[:Rel1* SHORTEST 0..]-(sbn:Node) pattern to go from sn to sbn or vice versa. For the 2 examples you gave, you seem to want different things. In start=1, end=6, you seem to want the edge to from sbn to sn. In the start=2, end=6 example you want it to go from sn to sbn. I'll give my answer based on wanting these to from sn to sbn.
Option 1: 4 separate queries.
What you are asking for is a type of regular path query, where you seem to have a regex in mind on the labels of the edges in the paths you want to match. Regular path queries are not supported in Cypher, so you can expand the query to 4 possible different ways it can match (2 possibilities about whether the sbn can have an optional [Rel1*] pattern to the left X 2 possibilities for whether ebn can have an optional [Rel1*] pattern to the right):
MATCH (sn:Nodes)-[:Rel2*1]->(en:Nodes)
WHERE sn.id = $start AND en.id = $end
RETURN sn.id, en.id

MATCH (sn:Nodes)-[:Rel1* SHORTEST 1..]->(sbn:Nodes)-[:Rel2*1]->(ebn:Nodes)
WHERE sn.id = $start  AND ebn.id = $end
RETURN sbn.id, ebn.id

MATCH (sbn:Nodes)-[:Rel2*1]->(ebn:Nodes)-[:Rel1* SHORTEST 1..]->(en:Nodes)
WHERE sbn.id = $start  AND en.id = $end
RETURN sbn.id, ebn.id

MATCH (sn:Nodes)-[:Rel1* SHORTEST 1..]->(sbn:Nodes)-[:Rel2*1]->(ebn:Nodes)-[:Rel1* SHORTEST 1..]->(en:Nodes)
WHERE sn.id = $start AND en.id = $end
RETURN sbn.id, ebn.id

Again this won't return exactly the answers in your examples. Specifically, this won't return 5, 4 when start=2, and end=6 but I think it's because your answers are not accurate. If you change the direction of the sbn to sn path, then it will return 5, 4.
Option 2: Single query: Or you can use WHERE EXISTS and merge those 4 queries into a single query.
MATCH (sbn:Nodes)-[:Rel2*1]->(ebn:Nodes)
WHERE (sbn.id = $start OR EXISTS { MATCH (sn:Nodes {id: $start})-[:Rel1* 1..3]->(sbn) })  AND (ebn.id = $end OR EXISTS { MATCH (ebn:Nodes)-[:Rel1* SHORTEST 1..3]->(en:Nodes {id: $end}) })
RETURN sbn.id, ebn.id

Hope this helps. Here are the create statements I used to test these queries:
create node table Nodes(id INT64, primary key (id));
create rel table Rel1(from Nodes to Nodes);
create rel table Rel2(from Nodes to Nodes);
create (n1:Nodes {id: 1})-[r1:Rel1]->(n2:Nodes {id: 2})-[r2:Rel1]->(n5:Nodes {id: 5});
match (n1:Nodes {id: 1}) create (n1)-[r2:Rel2]->(n4:Nodes {id: 4});
match (n5:Nodes {id: 5}), (n4:Nodes {id: 4}) create (n5)-[r2:Rel2]->(n4);
match (n4:Nodes {id: 4}) create (n3:Nodes {id: 3})-[r1:Rel1]->(n4);
match (n4:Nodes {id: 4}) create (n4)-[r1:Rel1]->(n6:Nodes {id: 6});

Thanks for pointing out the mistake. I made the diagram in a hurry so direction is reversed for the last Rel1.
Also huge thanks for the time and effort spent on the suggestions! Will try them and get back.


Vary interface ordering depending on platform

#248 helped with device bays, but there are other instances:
Racks (e.g. B1.0.[1-12])
Interfaces (e.g. INT[1-14], EXT[1-11], which leads to something like this:
EXT1
INT1
INT2
EXT2
EXT3
INT3
...)

When I create INT[1-3] and EXT[1-3] on a device, they are ordered as:

EXT1
INT1
EXT2
INT2
EXT3
INT3

This is due to how interfaces are ordered by their manager, sorting on the trailing numeric ID before the alphabetic type.

For the simple case of just one integer field might it be worth sorting on sub-section A then B? Modular order usually does not apply when there isnt a module number (because its just a single integer field)

I'm having some issues as well:
Cisco switch
TenGigabitEthernet1/0/1
GigabitEthernet1/0/1
TenGigabitEthernet1/0/2
GigabitEthernet1/0/2
...

Another Cisco switch
...
 GigabitEthernet1/0/51
 GigabitEthernet1/0/52
 vlan1

An HP Switch
 vlan 1
 interface 1
...

Another HP Switch
 interface 1
 vlan 1
 interface 2
...

Juniper Router
...
xe-1/0/0
xe-1/1/0
xe-1/2/0
xe-1/3/0
GigabitEthernet2/0/0
GigabitEthernet2/0/1
GigabitEthernet2/0/2
GigabitEthernet2/0/3
GigabitEthernet2/0/4
GigabitEthernet2/0/5
GigabitEthernet2/0/6
GigabitEthernet2/0/7
GigabitEthernet2/0/8
GigabitEthernet2/0/9
GigabitEthernet2/1/0
GigabitEthernet2/1/1
GigabitEthernet2/1/2
GigabitEthernet2/1/3
GigabitEthernet2/1/4
GigabitEthernet2/1/5
GigabitEthernet2/1/6
GigabitEthernet2/1/7
GigabitEthernet2/1/8
GigabitEthernet2/1/9
xe-2/2/0
xe-2/3/0
...

can interfaces not be sorted wholly as a string? if they were, then all characters should sort as expected. that includes trailing numbers. this is a snippet of my virtual interfaces. they're not sorted this way in the menus (by last 4 digits), but, sorted alphabetically in a spreadsheet or other app, they end up as desired:
V01-0199
V01-0299
V01-0399
V01-0499
V01-0599
V01-0699
V01-0799
V01-0899
V01-0999
V01-1099
V01-1199
V01-1299
V01-1399
V01-1499
V01-1599
V01-1699
V01-1799
V01-1899
V01-1999
V01-2099
V02-0199
V02-0299
V02-0399
V02-0499
V02-0599
V02-0699
V02-0799
V02-0899
V02-0999
V02-1099
V02-1199
V02-1299
V02-1399
V02-1499
V02-1599
V02-1699
V02-1799
V02-1899
V02-1999
V02-2099

can interfaces not be sorted wholly as a string?

This leads to chaos on some platforms. For instance, a Juniper switch might show
ge-0/0/0
ge-0/0/1
ge-0/0/3
...
ge-0/0/47
xe-0/0/2

Hence the need to order on slot/number rather than by type.
It doesn't seem like there exists a one-size-fits-all solution, unfortunately. Maybe we need to implement a few different ordering options (e.g. Cisco style, Juniper style, etc.) and make it an option on the device type.

I'm actually OK with that Juniper example, all the ge interfaces come first followed by the xe interfaces
Ideally I don't want ge and xe mixing as the actual conf has them sorted out as you indicate above
Sorting of interfaces should be just as you'd see them on a switch config.

Sorting of interfaces should be just as you'd see them on a switch config.

I agree. Junos always sorts interfaces by slot and position, both in the configuration and in the output of show interfaces. It disregards interface type entirely because this can change (e.g. by using a different transceiver). NetBox replicates this ordering.
Cisco IOS, conversely, lists interfaces first by media type and then by slot/position. I can't think of any practical way to consolidate both strategies in a single function.

Maybe then as you stated we have different ones that can be tied to the devices' platform:
-Cisco IOS
-HP OS
-JunOS
...and so forth.
We'd have to ship with a few default platforms to match up with the associated sorting strategies.
This could also can-of-worms where in people will want to define their own sorting strategies (is that accomplishable?)
Just thinking the angles

any update on the racks' lexicographic sorting?

Per #373, it would be nice to add some capability to dynamically group interfaces by slot.

I've added an interface_ordering field to the DeviceType model, which allows for toggling the natural order of Interfaces (and InterfaceTemplates) per device type. The two options provided (by position and by name) seem sufficient, but if additional use cases arise we can handle them in a new issue.

After the 1.8.2 release I now get the otoin to toggle the Interface list using the tow options above, but this only works on the Device Type Page.  The Actual device page doesn't take any notice of this and does not have an optoin to toggle the sort options.

@jennec The device view inherits interface ordering from its parent device type. This can be observed by creating, for example, interfaces named Gi0/[1-4] and Te0/[1-4] on a device. If the parent device type is configured to order interfaces by slot/position, the interfaces will appear intermingled; if it is configured to order alphabetically, all the Gi interfaces will be listed first, followed by all the Te interfaces. This is confirmed to be working on v1.8.2.

@jeremystretch That doesn't match up wtih my expirience.  I have a switch setup with 54 ports.  Each labeled similarly to "04 - 10GBE (SFP+)".  The Switch Device Type is set for Alphabetical sorting.  When viewing the interfaces from the Device Type page the interfaces are sorted in reverse order (which is OK).  But when viewed from the Device page they are in, what seems like, a random order.  Perhaps it's related to the way I've named the interfaces, but that doesn;t make sense when it works on the Device Type Page but not the Device page.
Is the sort order inherited, when changed, for existing Devices using that Device Type or will it only affect new Devices created using that Device Type?

You've appended the interface type to the name, which is evading the natural ordering logic. The interfaces should be named as they appear on the device; the interface form factor is maintained in a separate field. If you name the interfaces using only their numeric IDs (as they appear on the device), they will be sorted correctly.

I can appreciate the logic in that.  But its competing against the sorting working correctly in the Device Type page but not on the Device itself.  Are there differences between the two pages?

Since the ordering logic can't work with the names you've given the interfaces, the objects effectively aren't being ordered. This can result in non-deterministic behavior.