[Bug]: fail to parse nodes with relationships defined from qdrant

Bug Description
Here is the problem I encountered: I have created Document type nodes and utilized them to establish a VectorStoreIndex using qdrant as the vector store. The process of creating the index went smoothly, and the nodes were successfully stored in qdrant. However, when I attempt to query the index, it loads the data from qdrant and attempts to parse it into a TextNode type, resulting in an error.
Interestingly, the code functions properly when no relationship is defined.
# vetore_stores/utils.py
def metadata_dict_to_node(metadata: dict) -> TextNode:
    """Common logic for loading Node data from metadata dict."""
    node_json = metadata.get("_node_content", None)
    if node_json is None:
        raise ValueError("Node content not found in metadata dict.")

    return TextNode.parse_raw(node_json)  # <--

Version
0.6.35
Steps to Reproduce
from llama_index import Document
from llama_index.schema import NodeRelationship
from qdrant_client import QdrantClient
from llama_index.vector_stores import QdrantVectorStore
from qdrant_client.http.models import Distance, VectorParams
from typing import List
from llama_index import StorageContext, VectorStoreIndex
from llama_index.vector_stores import QdrantVectorStore
from llama_index import load_index_from_storage
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

def unit_vector(idx: int) -> List[float]:
    arr = [0]*1536
    arr[idx] = 1
    return arr

def create_collection(client: QdrantClient, collection_name: str) -> None:
    resp = client.get_collections()
    found = False
    for col in resp.collections:
        if col.name == collection_name:
            found = True
            break

    if found:
        print(f"collection [{collection_name}] exist!")
        print(client.get_collection(collection_name))
    else:
        success = client.create_collection(
            collection_name=collection_name,
            vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
        )
        print(f"create collection [{collection_name}] success: {success}")

def qdrant_client() -> QdrantClient:
    qd_host = "host"
    qd_port = 1234
    qd_api_key = "key"

    # Creating a Qdrant vector store
    client = QdrantClient(
        host=qd_host,
        port=qd_port,
        api_key=qd_api_key,
        https=False
    )
    return client

uuid1 = "43ac3cc4-eef1-4f66-9d27-b8eef2681596"
uuid2 = "c5f6feb0-36b7-4442-8a6f-398c80253e1e"

def make_nodes() -> List[Document]:
    doc1 = Document(
        text="2023/06/27: learn about llamaindex",
        metadata={"docType": "diary", "mood": "happy"},
        doc_id=uuid1
    )

    doc2 = Document(
        text="2023/06/28: confused about vectore store",
        metadata={"docType": "diary", "mood": "confusing"},
        doc_id=uuid2
    )

    # defined as Document, parsed as TextNode
    doc1.relationships[NodeRelationship.NEXT] = doc2.node_id
    doc2.relationships[NodeRelationship.PREVIOUS] = doc1.node_id

    # just for saving money
    doc1.embedding = unit_vector(0)
    doc2.embedding = unit_vector(1)

    nodes = [doc1, doc2]

    return nodes

def replay():
    collection_name = "test"

    client = qdrant_client()

    client.delete_collection(collection_name)

    create_collection(client, collection_name)

    nodes = make_nodes()

    qdrant_vector_store = QdrantVectorStore(
        client=client,
        collection_name=collection_name,
    )

    storage_ctx = StorageContext.from_defaults(vector_store=qdrant_vector_store)

    vec_index = VectorStoreIndex(nodes, storage_context=storage_ctx)

    vec_index.set_index_id("vec_index")

    for t in client.scroll(collection_name, limit=5)[0]:
        print(t)
    # failed here if relationships are defined
    print(vec_index.as_query_engine(verbose=True).query("what have i done on 2023/06/27"))

    vec_index.storage_context.persist()

    loaded_vec_index = load_index_from_storage(storage_ctx, index_id="vec_index")

   # would fail if last query() was commented
    print(loaded_vec_index.as_query_engine(verbose=True).query("how i feel on June 28th"))

if __name__ == "__main__":
    replay()

Relevant Logs/Tracbacks
INFO:httpx:HTTP Request: POST http://<qdrant>/collections/test/points/scroll "HTTP/1.1 200 OK"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'POST']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
id='43ac3cc4-eef1-4f66-9d27-b8eef2681596' payload={'_node_content': '{"id_": "43ac3cc4-eef1-4f66-9d27-b8eef2681596", "embedding": null, "metadata": {"docType": "diary", "mood": "happy"}, "excluded_embed_metadata_keys": [], "excluded_llm_metadata_keys": [], "relationships": {"3": "c5f6feb0-36b7-4442-8a6f-398c80253e1e"}, "hash": "a96e9f994571730ade8b16fea52d8fc79cddd55529c8a256bde04597b8603e17", "text": "2023/06/27: learn
about llamaindex", "start_char_idx": null, "end_char_idx": null, "text_template": "{metadata_str}\\n\\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\\n"}', 'docType': 'diary', 'doc_id': 'None', 'document_id': 'None', 'mood': 'happy', 'ref_doc_id': 'None'} vector=None
id='c5f6feb0-36b7-4442-8a6f-398c80253e1e' payload={'_node_content': '{"id_": "c5f6feb0-36b7-4442-8a6f-398c80253e1e", "embedding": null, "metadata": {"docType": "diary", "mood": "confusing"}, "excluded_embed_metadata_keys": [], "excluded_llm_metadata_keys": [], "relationships": {"2": "43ac3cc4-eef1-4f66-9d27-b8eef2681596"}, "hash": "92d1382ef27fa31890d11ae25bfcb166e99283a63b1746cfded8cba580bfd103", "text": "2023/06/28: confused about vectore store", "start_char_idx": null, "end_char_idx": null, "text_template": "{metadata_str}\\n\\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\\n"}', 'docType': 'diary', 'doc_id': 'None', 'document_id': 'None', 'mood': 'confusing', 'ref_doc_id': 'None'} vector=None
DEBUG:openai:message='Request to OpenAI API' method=post path=https://api.openai.com/v1/embeddings
DEBUG:openai:api_version=None data='{"input": ["what have i done on 2023/06/27"], "model": "text-embedding-ada-002", "encoding_format": "base64"}' message='Post details'
DEBUG:urllib3.util.retry:Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.openai.com:443
DEBUG:urllib3.connectionpool:https://api.openai.com:443 "POST /v1/embeddings HTTP/1.1" 200 None
DEBUG:openai:message='OpenAI API response' path=https://api.openai.com/v1/embeddings processing_ms=112 request_id=9898e56344b07edbaae2616a3c51dca6 response_code=200
DEBUG:httpcore.connection:connect_tcp.started host='<IP_ADDRESS>' port=10809 local_address=None timeout=5.0 socket_options=None
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore.backends.sync.SyncStream object at 0x0000022B70BB8310>
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'POST']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'POST']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'POST']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Connection', b'close'), (b'Transfer-Encoding', b'chunked'), (b'Content-Encoding', b'gzip'), (b'Content-Type', b'application/json'), (b'Date', b'Thu, 29 Jun 2023 03:51:03 GMT'), (b'Vary', b'accept-encoding, Origin, Access-Control-Request-Method, Access-Control-Request-Headers')])
INFO:httpx:HTTP Request: POST http://<qdrant>/collections/test/points/search "HTTP/1.1 200 OK"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'POST']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:llama_index.vector_stores.qdrant:> Top 2 nodes:
DEBUG:llama_index.vector_stores.qdrant:Failed to parse Node metadata, fallback to legacy logic.
Traceback (most recent call last):
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\vector_stores\qdrant.py", line 187, in query
    node = metadata_dict_to_node(payload)
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\vector_stores\utils.py", line 61, in metadata_dict_to_node
    return TextNode.parse_raw(node_json)
  File "pydantic\main.py", line 549, in pydantic.main.BaseModel.parse_raw
  File "pydantic\main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for TextNode
relationships -> 2
  value is not a valid dict (type=type_error.dict)
relationships -> 2
  value is not a valid list (type=type_error.list)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\code\python\demo\qdrant_fail_to_load.py", line 116, in <module>
    replay()
  File "d:\code\python\demo\qdrant_fail_to_load.py", line 105, in replay
    print(vec_index.as_query_engine(verbose=True).query("what have i done on 2023/06/27"))
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\indices\query\base.py", line 23, in query
    response = self._query(str_or_query_bundle)
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\query_engine\retriever_query_engine.py", line 142, in _query
    nodes = self._retriever.retrieve(query_bundle)
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\indices\base_retriever.py", line 21, in retrieve
    return self._retrieve(str_or_query_bundle)
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\token_counter\token_counter.py", line 78, in wrapped_llm_predict
    f_return_val = f(_self, *args, **kwargs)
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\indices\vector_store\retrievers\retriever.py", line 84, in _retrieve
    query_result = self._vector_store.query(query, **self._kwargs)
  File "D:\software\Anaconda3\envs\py310\lib\site-packages\llama_index\vector_stores\qdrant.py", line 195, in query
    node = TextNode(
  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for TextNode
text
  none is not an allowed value (type=type_error.none.not_allowed)

Hmm surprising that it's going through the backward compatibility code path. Will look into this.

@Disiok @rxue92 The way node relationships are defined changed in 0.6.35. Please see the docs here: https://gpt-index.readthedocs.io/en/latest/core_modules/data_modules/documents_and_nodes/usage_nodes.html#defining-and-customizing-nodes
And the RelatedNodeInfo class API for reference
https://github.com/jerryjliu/llama_index/blob/77009aa67a50d42b3c0c57af84b5bca8044829ee/llama_index/schema.py#L49
It seems like the backwards compat code path has a small bug though, not sure why it's complaining that text was None

Going to close this for now. My suggestion about changing the relation type should help.
Further, even though you define Document objects, they have the exact same interface (hence, this is likely not the cause of the issue, and the issue is likely the relationships)
Feel free to reach out on discord if you have more issues!


Change token address

I'm submitting a ...
[ ] bug report
[x ] feature request
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Summary

Other information Change of the BNTY token address https://github.com/solana-labs/token-list/pull/25417

First PR # https://github.com/solana-labs/token-list/pull/11251
Latest PR # https://github.com/solana-labs/token-list/pull/25417

Hi need help merging this @keone

Need urgent help merging this, our old token got hacked and we wish to replace the token address of the old token! @keone


spanner SDK is mixing the callback(err) and throw error will cause unhandled promise rejection

Environment details

OS: COS
Node.js version: 8.11.1
yarn version: 1.9.4
@google-cloud/spanner version: 2.0.0

Steps to reproduce

transaction.end called
SessionPool.release throw ReleaseError
https://github.com/googleapis/nodejs-spanner/blob/c5500c7fb2d063bff80fe38589f7adcaad1b3645/src/session-pool.js#L339-L342
transaction.end didn't wrap with try catch and didn't passed to callback, causing unhandled promise rejection.
https://github.com/googleapis/nodejs-spanner/blob/c5500c7fb2d063bff80fe38589f7adcaad1b3645/src/database.js#L496-L498

We toke a look into the source code and found that spanner SDK is mixing the callback(err) and throw error. error handling in the underneath spanner sdk is broken and encapsulated. as API user, we could not handle those errors on the public API provided by spanner sdk.
https://github.com/googleapis/nodejs-spanner/blob/c5500c7fb2d063bff80fe38589f7adcaad1b3645/src/transaction.js#L454-L458
this line, it should use callback(new Error) which could be handled by the promisified try catch block, but they use throw new Error which could not be properly handled by promisify utils and yields this unhandled promise rejection error

I'd like to see if @callmehiphop could chime in on this one.

@stephenplusplus @JustinBeckwith usually I would say throwing off the bat is the way to go, however I think maybe we want to make an exception for the Transaction object. Transactions will almost certainly have a fair amount of nesting going on
database.runTransaction((err, txn) => {
  txn.run('SELECT * FROM MyTable', (err, rows) => {
    txn.runUpdate('UPDATE MyTable (thing) VALUES ("yas")', (err, rowCount) => {
      txn.commit(err => {});
    });
  });
});

IMO it doesn't feel practical to throw here, because then a ton of try/catch nesting would be needed and that sounds annoying/painful to work with. Any thoughts/opinions here?

I'm entirely supportive of using the errback instead of throwing. I generally find most people are surprised by throws in callback based async functions 🤷♂

If the SDK is callback based, need to follow Error-first callbacks,  otherwise it will be so painful to handle the error
database.runTransaction((err, txn) => {
    try {
        txn.runUpdate('UPDATE MyTable (thing) VALUES ("yas")', (err, rowCount) => {
            if (err) {
                // Handle Error
            }
            txn.commit(err => {});
        });
    } catch (e) {
            // Handle Error
    }
});


Webhook Configuration

I am unable to configure webhooks to the caduceus server. I have the latest release of caduceus setup, and am able to trace notification message exchange from the parodus client till caduceus.I am currently using a mock SNS (https://github.com/s12v/sns). When I configure a new webhook using the '/hook' post request, I get a 'Success' response with a 200 OK status. However when I list all the registered hooks using '/hooks' API it responds with an empty array '[]'. There is also no communication from caduceus to the webhook. I have written a webhook server for testing this, but have also tried it with 'ngrok' proxy server and both shows no communication from caduceus.
The Debug log of caduceus shows a 'not a compact JWS' error every time I make an http request to caduceus. It also keeps repeating the following log related to SOA even though I have not configured SOA in 'caduceus.yaml'. There is no other errors in the debug log.
{"caller":"sns.go:247","endpoint":"XX.XX.XX.XX:6000","error":"dial tcp [::1]:5079: connect: connection refused","level":"debug","msg":"checking if server's DNS is ready","response":"\u003cnil\u003e MsgHdr","ts":"2019-02-19T06:08:14.850261431Z"}

@Johns-b If you get an empty array from caduceus, that means that it's list is empty.  The way SNS works for scytale->caduceus is not ideal as it introduces the need to wait for 5 minutes before caduceus can start sending messages (to ensure it has the full list of notifications to fulfill).  Does the empty array [] happen after 5 minutes?
On the SOA, that is present to ensure the DNS name of the local machine is able to work with SNS (otherwise SNS gets into a bad state).  I think that it doesn't skip the check is a bug to file.

@Johns-b, I wanted to follow up with you to see if you need something else or if I can close this issue.

@schmidtw, I switched to actual AWS-SNS and  updated the Caduceus version and everything worked fine. Sorry for the delay in my reply. Thank you for your help.


Could not display the default blue Icon of Marker

Environment

Leaflet   : 1.0.3+master.162276e
Browser : Chrome Version 55.0.2883.87 (64-bit)
OS/Platform : Ubuntu 16.04

Issue
Could not display the Leaflet's default blue Icon of Marker.
The source code in src/layer/marker/Icon.Default.js as below
        var path = DomUtil.getStyle(el, 'background-image') || DomUtil.getStyle(el, 'backgroundImage');
        document.body.removeChild(el);
        if (path === null || path.indexOf('url') !== 0) {
            path = '';
        } else {
            path.replace(/^url\([\"\']?/, '').replace(/marker-icon\.png[\"\']?\)$/, '');
        }

For the replaced result string didn't store back to path variable, the Icon got a wrong URL.
The last line should be replaced by
     path =  path.replace(/^url\([\"\']?/, '').replace(/marker-icon\.png[\"\']?\)$/, '');

Can you publish a fiddle/codepen/playground that displays this behaviour?

https://playground-leaflet.rhcloud.com/fiwo/1/edit?html,output
From PR https://github.com/Leaflet/Leaflet/pull/5535

Fixed by PR https://github.com/Leaflet/Leaflet/pull/5580


[BEAM-2724] Preparing support for Structured Names in Dataflow counters

Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[x] Each commit in the pull request should have a meaningful subject line and body.
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.

Adding a CounterName class to help support structured names for counters in the future.
Also remove function calls from the apiclient.py file that are no longer used.
And also removing function calls used for aggregators, which have been dead code for a while.

Changes Unknown when pulling b1e15abc0423ea7c8013f3d745ab97b44c04b5f5 on pabloem:add-countername into ** on apache:master**.

Failure is unrelated. It was in the direct runner tests.

Coverage remained the same at 69.971% when pulling a4176f8e7739ec24d35fd90259ca732b390be688 on pabloem:add-countername into 2fa4fdecd8ef06534e369e527b15ae8193823c8b on apache:master.

Coverage increased (+0.003%) to 69.975% when pulling a4176f8e7739ec24d35fd90259ca732b390be688 on pabloem:add-countername into 2fa4fdecd8ef06534e369e527b15ae8193823c8b on apache:master.

r: @aaltay
First change to add structured counter names in Dataflow runner. Adding the CounterName class.

LGTM


Date custom field not supporting data format on entry

Environment

Python version:  3.6.6
NetBox version:  2.5.4

Steps to Reproduce

Set date format to desired format in configurations.py (screenshot below).  I've chosen 'j-M-y'. Save file.
Add custom field to dcim > device
Set type to Date and Create name, description.
Leave label, required not checked, filter logic = loose, no default and weight = 100
Select a device and enter date into custom field using format that was set int he configurations.py file.
Error occurs stating that wrong date format is being used. Only the standard system one can be used successfully, 'Y-m-d' or '2019-Feb-2'

Changes made to configurations.py file below

Custom field creation

Expected Behavior
I expected to be able to enter in the date in the device page in the format that I had specified in the configurations.py file.

Observed Behavior
The device page would only except field input using the standard format of 'Y-m-d'

Hi @robbagithub please read this: https://groups.google.com/forum/?nomobile=true#!topic/netbox-discuss/0_LOtVUYtdM


Add bypass for rest sensor if the timeout is exceeded

Description:
I had the problem that the Scrape sensor could not be configured for a specific URL. It failed with HTTPSConnectionPool(host='www.accuweather.com', port=443): Read timed out. (read timeout=10). The URL I want to scrape is https://www.accuweather.com/en/de/hamburg/20095/biking-daily-forecast/178556?day=1
My config looks like this:
sensor:
  - platform: scrape
    name: "Todays Biking Weather"
    resource: https://www.accuweather.com/en/de/hamburg/20095/biking-daily-forecast/178556
    select: 'li[class="title pos"]'
    headers:
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36

I try to fix this issue by adding a request.get() command if the timeout exceeded.
Checklist:

[X] The code change is tested and works locally.
[X] Local tests pass with tox. Your PR cannot be merged unless tests pass
[X] There is no commented out code in this PR.
[X] I have followed the development checklist

Hi @obelix05,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!

@balloob, @amelchio : Thank you very much for your input and sorry for my too quick pull request. Because of your discussion, I've gone deeper into the matter.  I would like to explain a little:
When a requests.Session() is initialized, different default_headers are set ('Accept-Encoding', 'Accept' and 'Connection'). Such a session is also initialized by requests.get(). Then the custom_headers are merged with the default_headers. For the query of my URL it is important that 'Accept' and 'Connection' are set. However, calling Request.prepare() does not set these default_headers and calling requests.Session().send() overwrites any values already set (including the default_headers). I didn't see this difference before and therefore made this pull request. Sorry for that. The solution to my problem is to set these two headers in my configuration.yaml.
      Accept: "*/*"
      Connection: keep-alive

Please excuse the inconvenience.

That's weird that the headers are different. That might actually be a bug.


Support .wmbt  2

I know the tool is for .msbt files, but I found some similarities in the .wmbt file. If anyone can provide support for this as well, if I’m not mistaken, currently we only have reading support for the Switch-Toolbox tool.
Game: WarioWare™: Move It!
File
File.zip

Obs. This file seems to have a different error.
Error:

Duplicate of #5


Role on UM listing and User details should be Team Member (not member)

Verified and looking good in development, attaching screen shot:
Private Zenhub Image
CC: @marieanik


cropTime zeroTime behaviour fixed

@arrenglover I've reviewed the PR but I cannot merge it since I'm not authorized.


Updating a resource delete to send a delete event

When a resource is deleted end up getting a WRITE followed by a DELETE without a resource event. Checking deletedOn to check for a deleted resource.

In a perfect world, would this be fixed on the backend? I'm a bit nervous that this is tech debt on the frontend side. The switch statement for the different resource events was super clear, so it'd be a shame to get rid of it.

The event with resource object is a soft delete. When hard delete happens there will not be a resource object. Another thing to note, if we treat soft delete as an update we will end up refreshing the queries which will fail. This change basically solves these 2 issues.


Allow lint rules to be configurable

Currently your lint rule set is the only one usable. It'd be nice if we could provide our on eslint file and have it use that. Maybe by specifying a path to it?
There's a big benefit of having an eslint file in the root. It allows editors or IDEs to use it while editing.

Oh, I think it's already available via options.eslint, I just haven't documented this because it's tested yet 😅
By default it uses a preset config file, you can override it though:
module.exports = {
  lint: true,
  eslint: {
    configFile: cwd('.eslintrc')
  }
}

Will document it after adding more improvements to this.

Sounds good.

See https://vbuild.js.org/docs/addons.html#eslint

I'm wondering how to do lint in v7. vbuild --eslint doesn't work now.

@NemoAlex
If you want to run eslint during building, here's a preset: https://github.com/egoist/vbuild/blob/master/packages/vbuild-preset-eslint/README.md
In fact it's as simple as mutating the webpack config via old webpack(config) {} option for new extendWebpack(config) {} option.
Personally I would use eslint or xo directly like eslint **/src/**/*.{vue,js}, since it's much faster, see here for example


UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 85: ordinal not in range(256)

2017-12-20 13:23:07,910 DEBUG: http://www.plymouthherald.co.uk:80 "GET /news/plymouth-news/hms-ocean-home-whats-next-952279 HTTP/1.1" 200 33602
/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/sql/sqltypes.py:219: SAWarning: Unicode type received non-unicode bind param value "b'Christmas came early for '...". (this warning may be suppressed after 10 occurrences)
(util.ellipses_string(value),))
Traceback (most recent call last):
File "/root/ThisIsScraper/this_is_scraper/scraper.py", line 126, in
main()
File "/root/ThisIsScraper/this_is_scraper/scraper.py", line 121, in main
process_pending_articles(db_sess)
File "/root/ThisIsScraper/this_is_scraper/scraper.py", line 108, in process_pending_articles
db_sess.commit()
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 921, in commit
self.transaction.commit()
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 461, in commit
self._prepare_impl()
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
self.session.flush()
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 2192, in flush
self._flush(objects)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 2312, in _flush
transaction.rollback(_capture_exception=True)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/util/langhelpers.py", line 66, in exit
compat.reraise(exc_type, exc_value, exc_tb)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 2276, in _flush
flush_context.execute()
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
rec.execute(self)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
uow
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/persistence.py", line 177, in save_obj
mapper, table, update)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/orm/persistence.py", line 737, in _emit_update_statements
execute(statement, multiparams)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception
util.reraise(*exc_info)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/opt/thisisscraper/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
File "/opt/thisisscraper/lib/python3.4/site-packages/pymysql/cursors.py", line 166, in execute
result = self._query(query)
File "/opt/thisisscraper/lib/python3.4/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/opt/thisisscraper/lib/python3.4/site-packages/pymysql/connections.py", line 854, in query
sql = sql.encode(self.encoding, 'surrogateescape')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 85: ordinal not in range(256)

Propose base 64 encoding content for storage?

Fixed, does not require base64 encoding of content, just don't use latin-1 character set in MySQL DB schema... When creating database use...
CREATE DATABASE thisisscraper CHARACTER SET utf8 COLLATE utf8_general_ci;


Fix download scripts

ISTIO_VERSION=1.6.0-alpha.8 ./downloadIstioCandidate.sh
Downloading istio-1.6.0-alpha.8 from https://github.com/istio/istio/releases/download/1.6.0-alpha.8/istio-1.6.0-alpha.8-linux.tar.gz ...Failed.

 Trying with ISTIO_ARCH. Downloading istio-1.6.0-alpha.8 from https://github.com/istio/istio/releases/download/1.6.0-alpha.8/istio-1.6.0-alpha.8-linux-amd64.tar.gz ...

Unable to download Istio 1.6.0-alpha.8 at this moment!
Please verify the version you are trying to download.

./downloadIstioCandidate.sh
Downloading istio-1.5.1 from https://github.com/istio/istio/releases/download/1.5.1/istio-1.5.1-linux.tar.gz ...
Istio 1.5.1 Download Complete!

Istio has been successfully downloaded into the istio-1.5.1 folder on your system.

Next Steps:
See https://istio.io/docs/setup/kubernetes/install/ to add Istio to your Kubernetes cluster.

To configure the istioctl client tool for your workstation,
add the /home/bavery/code/istio/src/istio/release/istio-1.5.1/bin directory to your environment path variable with:
         export PATH="$PATH:/home/bavery/code/istio/src/istio/release/istio-1.5.1/bin"

Begin the Istio pre-installation verification check by running:
         istioctl verify-install

Need more information? Visit https://istio.io/docs/setup/kubernetes/install/

ISTIO_VERSION=1.6.0-alpha.2 ./downloadIstioCandidate.sh
Downloading istio-1.6.0-alpha.2 from https://github.com/istio/istio/releases/download/1.6.0-alpha.2/istio-1.6.0-alpha.2-linux.tar.gz ...Failed.

 Trying with ISTIO_ARCH. Downloading istio-1.6.0-alpha.2 from https://github.com/istio/istio/releases/download/1.6.0-alpha.2/istio-1.6.0-alpha.2-linux-amd64.tar.gz ...

Istio 1.6.0-alpha.2 Download Complete!

Istio has been successfully downloaded into the istio-1.6.0-alpha.2 folder on your system.

Next Steps:
See https://istio.io/docs/setup/kubernetes/install/ to add Istio to your Kubernetes cluster.

To configure the istioctl client tool for your workstation,
add the /home/bavery/code/istio/src/istio/release/istio-1.6.0-alpha.2/bin directory to your environment path variable with:
         export PATH="$PATH:/home/bavery/code/istio/src/istio/release/istio-1.6.0-alpha.2/bin"

Begin the Istio pre-installation verification check by running:
         istioctl verify-install

Need more information? Visit https://istio.io/docs/setup/kubernetes/install/

ISTIO_VERSION=1.6.0-alpha.8 ./downloadIstioCtl.sh
Downloading istioctl-1.6.0-alpha.8 from https://github.com/istio/istio/releases/download/1.6.0-alpha.8/istioctl-1.6.0-
alpha.8-linux.tar.gz ...
Failed.

Trying with ISTIO_ARCH. Downloading istioctl-1.6.0-alpha.8 from https://github.com/istio/istio/releases/download/1.6.0-alpha.8/istioctl-1.6.0-alpha.8-linux-amd64.tar.gz ...
Download failed, please make sure your ISTIO_VERSION is correct and verify the download URL exists!

ISTIO_VERSION=1.6.0-alpha.2 ./downloadIstioCtl.sh
Downloading istioctl-1.6.0-alpha.2 from https://github.com/istio/istio/releases/download/1.6.0-alpha.2/istioctl-1.6.0-alpha.2-linux.tar.gz ...
Failed.

Trying with ISTIO_ARCH. Downloading istioctl-1.6.0-alpha.2 from https://github.com/istio/istio/releases/download/1.6.0-alpha.2/istioctl-1.6.0-alpha.2-linux-amd64.tar.gz ...
istioctl-1.6.0-alpha.2-linux-amd64.tar.gz download complete!

Add the istioctl to your path with:
  export PATH=$PATH:$HOME/.istioctl/bin

Begin the Istio pre-installation verification check by running:
         istioctl verify-install

Need more information? Visit https://istio.io/docs/reference/commands/istioctl/

./downloadIstioCtl.sh
Downloading istioctl-1.5.1 from https://github.com/istio/istio/releases/download/1.5.1/istioctl-1.5.1-linux.tar.gz ...
istioctl-1.5.1-linux.tar.gz download complete!

Add the istioctl to your path with:
  export PATH=$PATH:$HOME/.istioctl/bin

Begin the Istio pre-installation verification check by running:
         istioctl verify-install

Need more information? Visit https://istio.io/docs/reference/commands/istioctl/

@brian-avery: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
754b0daff4f56a1dcd324bb809a3b79ef5b63ef3
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@brian-avery: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
6e4c10e9ae9730d809464d8951713f23a91cbccc
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Brian, thanks for this. I pulled both scripts and ran them with older and alpha.2 specified on my Mac and they worked OK. I had to remarked out the OS= line and passed in some OS=linux TARGET_ARCH=various with various releases and all that worked fine.
Since this script goes live as soon as it's merged,I would like at least a second person to review and approve before this goes in.

In response to a cherrypick label: new pull request created: #23247

Since it was found during Internal dev testing @brian-avery pls add testing label


Vjudge login issue

I ran with & without editing the .env file with VJ usr & pass but still getting this:

ok, the issue solved when set USE_VJ=false

If you can't login to Vjudge from this scripts, just download all the solutions in module/solutions & extract there.. It'll suffice


Adds Elixir Tutorial with Plug

Hey @pragmaticivan, thanks for creating this tutorial! Did you purposely remove the It Works page like in the hosting tutorial?

Fixed @jonnilundy good catch, I missed this commit

Thanks @pragmaticivan!


support other specs for preventDefault

With custom inputs, sometimes you could get a DOM event vs. a JS event. This change accommodates both.

Please change defaultPrevented as discussed on https://github.com/Hacker0x01/react-datepicker/issues/590

Current coverage is 93.04% (diff: 100%)

Merging #591 into master will not change coverage

@@             master       #591   diff @@
==========================================
  Files            11         11
  Lines           503        503
  Methods          15         15
  Messages          0          0
  Branches         90         90
==========================================
  Hits            468        468
  Misses           12         12
  Partials         23         23

Powered by Codecov. Last update 0750e8e...980ee1a

Thanks!


poly: add initial lowering of NTT op

poly: add initial lowering NTT op
  - add implementation of the fast NTT to lower the NTT operation
  - add testing for lowering and a runner

poly: add additional root parameter to RingAttr
  - include an optional parameter that computes the 2n-th primitive
    root of a rings ideal degree and its coefficient modulus
  - allow for this to be retrieved from a precomputation or to be
    user-specified
  - Add a script that will compute the 2n-th roots of unity for the
    specified polynomial degree values and n. The values are only
    outputted if the 2n-th root of unity exists.
  - Note: even if the nth root of unity exists it will not be output if
    the 2n-th doesn't exist

I think this is ready for an initial review. @j2kun
Currently only the NTT lowering is implemented, I would like to make sure that the approach is good before doing the work for the INTT in a similar manner. There are some points that would be good to get feedback on:

Which (prime, degree) pairs should be included in the generation script.
In the current implementation, the recursive function fastNTT function is unrolled out already. It might be beneficial to make this into an SCF for loop? Or we could generate custom private functions for each degree of the iteration (similar to the implementation of the polynomial modulus)? Functionally they would be the same, but it could improve readability of the lowered code.
At the moment we construct an empty tensor for each iteration of the fast NTT and then concatenate them to construct our final result. We could reduce the uses of tensor.empty to a single one at the start and then operate directly on the single tensor. Although I haven't checked the generated LLVM code to see if it does this for us.

Sweet, no rush. Another thing to think about would be if we should automatically extend the polynomial coefficients to be a power of 2, or if we should issue an error/warning for the user to do so themselves? Right now we just assume it is a power of 2.

Sweet, no rush.
Note to self: I realise now there should also be a warning/error in the case the degree is not a power of 2.

One more inconvenience: my partial work upstreaming the polynomial dialect (https://github.com/llvm/llvm-project/pull/72081) causes a name conflict here. The fix to those namespace conflicts will probably land in this repo today. The changes that need to be made to this PR to handle this are:

Use a fully-qualified namespace ::mlir::heir::polynomial::Polynomial in the C++ API where relevant (https://github.com/google/heir/pull/616, which already landed)
Use _polynomial as the dialect name instead of polynomial in mlir code (https://github.com/google/heir/pull/617, which will land as a separate commit later)

Awesome. I really appreciate the time for all of the feedback!
Okay, sounds good. I will experiment with loop unrolling and see if it is worth it to change.
Thanks for the heads up, will deal with that once I get to rebasing.

The next steps will be to implement the INTT

I would support doing this i followup PRs, just so we can incrementally merge things and avoid any headaches rebasing in the future.

Sounds good. I will hopefully get those follow-up prs in the next couple days :)

Let me know when this is ready for a final review.

Okay, took another jab at looking into the performance. For the new test added, it takes ~40 seconds to compile with --heir-polynomial-to-llvm and < 1 second for --polynomial-to-standard for n=256 with random coefficients.
My initial suspicion was that my use of tensor.extract_slices to split the vector at each function call would have been the cause. So instead I have changed it to use tensor.extract on each element using the bit-reverse order. The other thing I wanted to see, was if creating the cMod and roots at each function call was generating too many values for --cse to deal with efficiently. So I have moved these out to be pre-computed before calling fastNTT.

Starting from c6c5b7cd5e8fd8d72b94f0b829b3e0f0d0d9b85f, I ran your test from be317d2 with -mlir-timing and see
  Total Execution Time: 25.0003 seconds

  ----User Time----  ----Wall Time----  ----Name----
    0.0060 (  0.0%)    0.0060 (  0.0%)  Parser
    0.0011 (  0.0%)    0.0011 (  0.0%)  ElementwiseToAffine
    0.1182 (  0.4%)    0.1182 (  0.5%)  PolynomialToStandard
    0.1586 (  0.6%)    0.1586 (  0.6%)  Canonicalizer
    0.0492 (  0.2%)    0.0492 (  0.2%)  CSE
    0.0000 (  0.0%)    0.0000 (  0.0%)    (A) DominanceInfo
    0.3331 (  1.3%)    0.3329 (  1.3%)  'func.func' Pipeline
    0.1313 (  0.5%)    0.1312 (  0.5%)    ConvertElementwiseToLinalgPass
    0.1132 (  0.4%)    0.1131 (  0.5%)    AffineExpandIndexOps
    0.0885 (  0.3%)    0.0885 (  0.4%)    SimplifyAffineStructures
    0.0954 (  0.4%)    0.0954 (  0.4%)  ConvertAffineToStandard
    0.2173 (  0.8%)    0.2171 (  0.9%)  'func.func' Pipeline
    0.0941 (  0.4%)    0.0940 (  0.4%)    ExpandOps
    0.1232 (  0.5%)    0.1231 (  0.5%)    ExpandStridedMetadata
   16.4575 ( 62.4%)   16.4575 ( 65.8%)  OneShotBufferize
    0.1061 (  0.4%)    0.1061 (  0.4%)  ExpandRealloc
    0.1572 (  0.6%)    0.1572 (  0.6%)  OwnershipBasedBufferDeallocation
    0.2453 (  0.9%)    0.2453 (  1.0%)  Canonicalizer
    0.8840 (  3.4%)    0.8840 (  3.5%)  BufferDeallocationSimplification
    0.1216 (  0.5%)    0.1216 (  0.5%)  LowerDeallocations
    0.1331 (  0.5%)    0.1331 (  0.5%)  CSE
    0.0000 (  0.0%)    0.0000 (  0.0%)    (A) DominanceInfo
    0.2199 (  0.8%)    0.2199 (  0.9%)  Canonicalizer
    0.4434 (  1.7%)    0.4433 (  1.8%)  'func.func' Pipeline
    0.4434 (  1.7%)    0.4433 (  1.8%)    ConvertLinalgToLoopsPass
    0.0813 (  0.3%)    0.0813 (  0.3%)  ConvertAffineToStandard
    0.0818 (  0.3%)    0.0818 (  0.3%)  ConvertBufferizationToMemRef
    0.1679 (  0.6%)    0.1679 (  0.7%)  Canonicalizer
    0.1819 (  0.7%)    0.1819 (  0.7%)  SCCP
    0.0471 (  0.2%)    0.0471 (  0.2%)  CSE
    0.0000 (  0.0%)    0.0000 (  0.0%)    (A) DominanceInfo
    0.1121 (  0.4%)    0.1121 (  0.4%)  SymbolDCE
    0.1036 (  0.4%)    0.1036 (  0.4%)  ArithExpandOpsPass
    0.6725 (  2.5%)    0.6725 (  2.7%)  SCFToControlFlow
    0.3854 (  1.5%)    0.3853 (  1.5%)  'func.func' Pipeline
    0.3854 (  1.5%)    0.3853 (  1.5%)    ExpandStridedMetadata
    0.7060 (  2.7%)    0.7060 (  2.8%)  ConvertToLLVMPass
    0.7442 (  2.8%)    0.7442 (  3.0%)  Canonicalizer
    0.4704 (  1.8%)    0.4704 (  1.9%)  SCCP
    0.2397 (  0.9%)    0.2397 (  1.0%)  CSE
    0.0000 (  0.0%)    0.0000 (  0.0%)    (A) DominanceInfo
    0.1932 (  0.7%)    0.1932 (  0.8%)  SymbolDCE
    1.0333 (  3.9%)    1.0333 (  4.1%)  Output
    0.0131 (  0.0%)    0.0131 (  0.1%)  Rest
   26.3794 (100.0%)   25.0003 (100.0%)  Total

I've never seen OneShotBufferize take so long.

Here's the mlir that is slow input to OneShotBufferize. Looks like it's ~5k single-op linalg.generic ops. https://gist.github.com/j2kun/af8ac4fbfd7df989d92fa5414c544db1

I suspect the main problem is that the IR after polynomial-to-standard consists largely of single-element tensor ops (see below), which get lowered to linalg
%449 = arith.muli %from_elements_413, %cst_15 : tensor<1xi40>
%450 = arith.remui %449, %cst_14 : tensor<1xi40>
%451 = arith.addi %from_elements_411, %450 : tensor<1xi40>
%452 = arith.remui %451, %cst_14 : tensor<1xi40>
%453 = arith.subi %from_elements_411, %450 : tensor<1xi40>
%454 = arith.addi %453, %cst_14 : tensor<1xi40>
%455 = arith.remui %454, %cst_14 : tensor<1xi40>

Can we just change these to scalar ops? Either in the original lowering for ntt, or else in a pass that identifies tensor<1xfoo> and replaces it with scalar ops? I feel like there should be an upstream pass that can do this, but I'm not able to find one quickly.

That is a very handy flag!
That seems promising, thanks for taking a look. I will test out with the scalar ops and also to see what manual fusion that we could do (also for the larger sized tensors). Doing the modulo arithmetic should be straightforward.
Checking I understand correctly, when you say to use an affine.for, that would be instead of the current recursive algorithm right?

Checking I understand correctly, when you say to use an affine.for, that would be instead of the current recursive algorithm right?

In reference to your comment https://github.com/google/heir/pull/590#issuecomment-2054587006, " the recursive function fastNTT function is unrolled out already. It might be beneficial to make this into an SCF for loop"
I'm certain that making it a loop (in any form) will alleviate the slow compile time issues. I'm not sure what sort of loop would be reasonable for this. The best would be affine.for, since that would preserve affine loop nests when lowering through linalg. I haven't thought too hard about how bad it would be to use scf.while, since I'm not aware of an upstream loop unroller for scf (ideally we could just use whatever loop we want, apply bufferization, and then unroll to get what we have now back).

Yep, then we are on the same page. I will give it a go to prioritize using affine.for

This pull request is continued in #642 with an implementation using affine.for loops.


Wrong filename by echo @% command when the file was open with startify

Here is the problem, when i have let g:startify_change_to_dir = 0 option in my vimrc(actually in neovim's init file), and when i open a file from startify startpage `:echo @%' command returns full path to the file. Maybe i missed some other option to disable/enable?
Example:
:pwd or echo getcwd()
>>> /home/user
:echo @%
>>> /home/user/.config/somefile.txt

Expected to return path related to the current working directory.
Example:
:pwd or echo getcwd()
>>> /home/user
:echo @%
>>>  .config/somefile.txt

Hmm, I can't reproduce that. It shows just .config/somefile.txt for me, as expected. The working directory is never tempered with, if g:startify_change_to_dir = 0.
What's the output of :echo getcwd() @% after opening a file from Startify?
Maybe another plugin is interfering? It might be worth to check :verb au BufEnter.

Let me check. I didn't completely check this problem. Because i just opened file with startify and without and recived different results as i described early. Thank you for some advises, going to try them now and will respond after that.

:echo getcwd() @% returns /home/user /home/user/.config/somefile.txt
And here is output of :verb au BufEnter
--- Auto-Commands ---
filetypedetect  BufEnter
    *.xpm     if getline(1) =~ "XPM2" |   setf xpm2 | else |   setf xpm | endif
	Last set from /usr/share/nvim/runtime/filetype.vim
    *.xpm2    setf xpm2
	Last set from /usr/share/nvim/runtime/filetype.vim
delimitMate  BufEnter
    *         if !exists('b:delimitMate_was_here') |   call <SID>setup() |   let b:delimitMate_was_here = 1 | endif
	Last set from ~/.local/share/nvim/site/dein/.cache/init.vim/.dein/plugin/delimitMate.vim
gitgutter  BufEnter
    *         if gettabvar(tabpagenr(), 'gitgutter_didtabenter') |   call settabvar(tabpagenr(), 'gitgutter_didtabenter', 0) | else |   call gitgutter#process_buffer(bufnr(''), 0) | endif
	Last set from ~/.local/share/nvim/site/dein/.cache/init.vim/.dein/plugin/gitgutter.vim
FileExplorer  BufEnter
    *         sil call s:LocalBrowse(expand("<amatch>"))
	Last set from /usr/share/nvim/runtime/plugin/netrwPlugin.vim

I don't any suspicious commd. You?

Rechecked. Disable all plugins and except plugin manager and startify + disable all my options and :echo getcwd() @% still returns /home/user /home/user/.config/somefile.txt
Neovim compability issue?

Hmm, I tested with Neovim master as well, same result. Technically, if you choose an entry by index or <cr>, Startify simply does :edit /home/user/.config/somefile.txt (check :StartifyDebug).
So, do you have the same behaviour with this?
$ cd
$ nvim
:edit .config/somefile.txt
:echo @%

Yes. Didn't disable g:startify_change_to_dir = 0
And yes :edit with full path output same result as file opened with startify. So there is some problem.
Hm, so why you don't have this problem. I don't get it.

Hm, maybe it's a platform thing. I'm on OSX. Are you on Linux by any chance?

@mhinz yes, im on linux.
So can we keep this open so maybe some linux + neovim user can confirm or negate the issue?

Current status: I played around with vim -u NORC and I'm pretty sure that showing an absolute path is correct when using :e ~/.config/somefile and a relative path is corret when using :e .config/somefile. So just how Vim works.
Interestingly I only get a relative path in both cases, when I have my own plugin vim-signify enabled. I'll take a closer look now.

An easy way to reproduce it:
$ nvim -u NORC
:cd
:edit ~/.config/somefile.txt
" absolute path
:echo @%
:cd .config
:cd -
" relative path
:echo @%

See https://github.com/neovim/neovim/pull/5845

So, mine neovim+startify behavior is right(as it should be)?
Is it hard(even possible) to make that when you inside some dir and open file under MRU THIS DIR that it opens not with full path. Sorry bad english, dunno how to describe properly. Like if you open MRU files it will :edit "fullpathhere", but when you open MRU THIS DIR it will execute :edit "relativepath"?

Yes we can close it if this is not startify problem.
But what's about your pull request to neovim. I think @% should return relative path. Maybe i made you think wrong or something.

Yea, well, obviously that's a subjective topic. Personally I'd prefer it to always show absolute paths, because they're exact. ;-) But yeah, as long as it's consistent, I'll accept relative paths as well.

Just for conclusion i don't know why @% isn't always return relative path. Bug or i read doc wrong, but there is expand('%:.') that seems always return right relative path.

Oh, I didn't know you needed it for scripting purposes.
Yes, expand('%:.') or, if you have an absolute filename in a variable already, fnamemodify(var, ':.').


Validator failed to produce block

Description
My teku validator has failed to produce block.
A console log shows this message:
18:33:46.054 ERROR - Validator   *** Failed to produce block  Slot: 58368 tech.pegasys.teku.core.StateTransitionException: tech.pegasys.teku.core.exceptions.BlockProcessingException: java.lang.IllegalArgumentException: process_operations: Verify that outstanding deposits are processed up to the maximum number of deposits (See log file for full stack trace)
Here is a corresponding fragment from teku.log: gist
Versions (Add all that apply)

Software version: teku/v0.12.3-dev-fe0dc3ed/linux-x86_64/oracle-java-14
Java version: java version "14.0.2" 2020-07-14 Java(TM) SE Runtime Environment (build 14.0.2+12-46) Java HotSpot(TM) 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing)
OS Name & Version: Ubuntu 18.04.5 LTS
Kernel Version: 4.15.0-112-generic #113-Ubuntu SMP
Eth1 client: Geth 1.6.5-stable-cf87713d

thanks for raising this, we'll look at it as soon as possible!

Hi @uvizhe,
Any chance you have the logs around that time, not only the error but some logs before and after? That could help us identify the state Teku was in when the issue happened.

Hi @cemozerr,
Unfortunately, I can't find necessary teku logs in ~/.local/share/teku/ anymore. Probably, I've deleted them to clean some space.

It looks like you could've failed to produce a block here if the eth1 node you were connected to failed to send 1 or more deposit events.  Added extra validation to make sure that the eth1 data we receive is complete.  Teku will now fail to start if any data is missing from our stored eth1 deposits, and we will print warnings if any ongoing requests for deposits return a response with missing data.
If your node fails to start up due to bad stored deposit data, at this point you'll have to resync your node.  If you do need to resync, be careful not to delete the validators folder inside of your data directory.


Update README.md

Add option "--recursive" in the installation guide due to the use of submodules

Thanks


Expander Fixes

Description
Closes #4390
Closes #4391
Closes #4248 (The Header is now stretched by default and will respect min/max/width)
Motivation and Context

Fixes 'Expanding' event to be named 'Expanded' which follows convention
Ensures events are fired after the state is applied
The control is now responsive. Unlike all other content controls, the Expander previously set a MaxWidth in the default style directly. This prevents the control from stretching horizontally.
The control now resizes to fit content as all other content controls such as Button, Border, Grid do.

How Has This Been Tested?
Manually tested in MUXControlsTestApp
Screenshots (if appropriate):

I think there are more issues with the default Expander style. This should be reviewed more carefully, I only fixed the ones I noticed. I'm certain all the height/width settings are not optimal. The control needs to be much more responsive in its architecture and I only did a quick pass on fixing this.

Setting a height for this control is a little strange when the expander is collapsed (see below). Should the Header take all of this space in this case?

/azp run

Personally I think all of the content control inheritied properties (Background, BorderThickness, padding, etc) should always apply to the property which is marked with the metadata tag Content, which in this case is the Expander.Content property

Yes, this was my earlier point and I think the majority of developers will expect that. It's true this is somewhat of a strange case though as a lot of devs will also want to customize the header. However, we need to follow convention here so I'll rework things to reverse the properties template bound to the header and will bind them to the content area instead.

This is ready for review and merge.

/azp run

@StephenLPeters I see some commits that are going to cause a lot of merge issues. What/when is the plan for getting this merged?

/azp run


RadioGroup orientation not working

Bug report
Current Behavior
When changing the orientation Prop it does not change the actual orientation from vertical to horizontal. Not even in the demo provided. RadioGroup demo

Expected behavior
When changing orientation of the component that it changes the orientation of the RadioGroupItems. My example is not pretty but my quick (unfinished styled) solution is to add a flex-direction to the RadioGroup.Root

Reproducible example
RadioGroup demo
Suggested solution
Yes this should be solved and I think its not hard but runs the risk of the component having opinionated styling
Additional context

Your environment

Software
Name(s)
Version

"@radix-ui/react-radio-group": "^1.2.0",

"react": "18.2.0",

Browser  All of them

@nickb-WaPo thanks for the issue. I stumbled over the same "problem", but it would seem that it's not a problem at all. You can read more here or https://github.com/radix-ui/website/issues/463. The orientation is only meant for functionality I guess, not style.


Delete user from permission?

Is there a way to remove a specific external user from a permission?  There is no delete for that, and patching with a new list of GrantedToIdentities fails.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cad8792a-e6c7-2fed-2ae8-b1489b6ad60f
Version Independent ID: ea682f8a-7249-4bf4-503c-d0e01a93f142
Content: Remove access to an item - OneDrive API - OneDrive dev center
Content Source: docs/rest-api/api/permission_delete.md
Product: onedrive
GitHub Login: @rgregg
Microsoft Alias: rgregg

Thanks for your feedback.  A feature to delete a specific user or PATCH is a feature enhancement that is on our backlog.

Please propose or vote for an enhancement via the OneDrive Developer Platform UserVoice at http://aka.ms/od-dev-uservoice. Thanks!


[FR]: Add support for dataview inline fields

Dataview Inline Fields is an additional way to specify note metadata and it will be great to support it

yes this is on the roadmap and something I want as well! thanks

looks like it has an API I can build against: https://blacksmithgu.github.io/obsidian-dataview/plugin/develop-against-dataview/


add logistic growth and robertson examples

logistic growth example: checks against correct solution
robertson example: requires solve_dense so not working atm

Thank you for submitting these, Martin. I'd like to make these into tests whilst I'm building up CI for the project. In future you may see these in the tests/ folder.


How to disable output of dictionary loading?

There always output information about dictionary loading
2023/01/07 23:04:15 Dict files path:  [./xxx]
2023/01/07 23:04:15 Load the gse dictionary: "./xxx"
2023/01/07 23:04:15 Gse dictionary loaded finished.
2023/01/07 23:04:15 Dict files path:  [./xxx]
2023/01/07 23:04:15 Load the gse dictionary: "./xxx"
2023/01/07 23:04:15 Gse dictionary loaded finished.
2023/01/07 23:04:15 Load the stop word dictionary: "./xxx"
2023/01/07 23:04:15 Dict files path:  [./xxx]
2023/01/07 23:04:15 Load the gse dictionary: "./xxx"
2023/01/07 23:04:15 Gse dictionary loaded finished.
2023/01/07 23:04:15 Dict files path:  [./xxx]
2023/01/07 23:04:15 Load the gse dictionary: "./xxx"
2023/01/07 23:04:15 Gse dictionary loaded finished.
2023/01/07 23:04:15 Load the stop word dictionary: "./xxx"

My codes as below
	segmenter, err := gse.NewEmbed(static.DictFile)
	segmenter.LoadStopEmbed(static.StopFile)
	segmenter.MoreLog = false  <-- seems no usable
	segmenter.SkipLog = true  <-- seems no usable

Is there any option to disable output such info?

You used seg.LoadDict() in the others function, You should used seg.SkipLog = true first, then load the dictionary.

Thanks.


Update AWS SDK, fixed GCC 9 on ubuntu 20.04

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Not sure why the test is failing with a segmentation fault

FYI @emersonknapp @mm318.
I may be misremembering, but thought utils-common was stuck at 1.6 for Lex and Polly breaking changes. Thoughts on this PR?

I do remember there needed to be code changes to the Lex Cloud Extension when updating between some AWS SDK versions, and I think we've already done that once before. It may apply to this new AWS SDK version bump as well.
These particular Build & Test failures are unrelated to breaking changes to downstream dependencies though. However, bumping the AWS SDK version should involve some testing of downstream dependencies though.

The main reason I did this PR is for porting the ROS packages to 20.04 and noetic. I only had to fix this other project -> https://github.com/aws-robotics/utils-ros1/pull/36 for it to work

Not sure why the test is failing with a segmentation fault

fixed segfault


Support for xml

I use this patch to inline some css into svg.

That is pretty cool. It would be good to add a test case for it.

I'll add test(s) and docs.


Protected branches API not implemented?

Hi,
I read the documentation up and down but could not find how to do what described here https://docs.gitlab.com/ee/api/protected_branches.html#protect-repository-branches. Is this not implemented yet? Thanx

Hi,
This feature is described here: https://github.com/egnyte/gitlabform/blob/master/config.yml#L122-130
Although the GitLab's API has changed much recently, it seems that this is still working.
Please try to use it and let me know!
Sorry if it is not working - I am working on adding integration tests in #72  that will check if we are not loosing some features because of GitLab changes.

I checked and it is working for me with a current-ish GitLab version, although with less features than the current API provides.

Closing this in favor of #27 . Please follow that issue @zaitsxl or reopen this ticket you have any issues with using the current feature.

Please can you give an example how you supply parameters of these https://docs.gitlab.com/ee/api/protected_branches.html#protect-repository-branches

You cannot supply these parameters, @zaitsxl , because the current version Protected branches API is not implemented in gitlabform yet. For now gitlabform uses an older API, for which docs are not available online anymore. I have just updated the docs under https://github.com/egnyte/gitlabform/blob/master/config.yml#L122 to show what is possible as of now, please check it out.

Okay that's what I was asking initially :) Thanx looking forward for that to be implemented


UIEvents.IMGUIRenderOverlays

I use MRTK On Hololens1 .when I analyze performance By Unity Proflier .I found something strange. UIEvents.IMGUIRenderOverlays is to long.it is showing when I use OnGUI ?but I do not use OnGUI .then i found FrameDebugger also show something, there are many RenderEvent about GUI.Repaint.and i do not know what i Render ? EditorMode ,it is normal. this problem lead to  my frameCount so low .can you explain it for me?  what i do to solve it?

@zhudaxing can you clarify the question about how MRTK is involved in this scenario? At first glance this seems like a HL1 + Unity perf question that may be specific to a particular application

I think i found the problem.this GUI.Repaint showing because of  developerConsole .but I set development Build false. it do not work .and other people had same condition on UWP .just like this:

there are some solution ,for example ,set solution Config Master In VS can close this console . I do not try it not yet because of build master error. and I want to know what other solution about this.

https://forum.unity.com/threads/how-can-i-remove-development-console-window-in-built-game.229171/


Enhancement Request - Close All Other Window Groups Action Icon

Enhancement 1:
So Basically, If I have a bunch of window groups currently open, and I want to close them all, except for one, I would click on a proposed new "close all but this group" icon on the window group I want to keep open.
Add On to above Enhancement:
An Add-On to this would be a "pinned window Group" mode, whoms window would be unaffected by the above action/command.  Apparently another Ticket has referenced this same idea for a "pinned window group" feature. #109

Apparently another Ticket has referenced the idea for a "pinned window group" feature. #109

Enhancement 1 is a good idea.
The second one will probably complicate too much the application.


Misc numbers at the bottom of pages

There are miscellaneous numbers at the bottom of some of the pages. It seems to be some sort of scanning number. We have decided to remove them.
See https://github.com/STAT325-S24/HistoryAmherstCollege/issues/15

Here's an example from page 65.
We all agreed (2024-03-28) that these should be removed when they appear.


Add v0.32.0 ClusterServiceVersion to upstream operators

upstream-community-operator for prometheus 0.32.0.
see #667 for merged community-operator.

[X ] Have you selected the Project Community Operator Submissions in your PR on the right-hand menu bar?
[X ] Are you familiar with our contribution guidelines?
[ X] Have you packaged and deployed your Operator for Operator Framework?

[ X] Have you tested your Operator in all supported installation modes?

Updates to existing Operators

[X ] Is your new CSV pointing to the previous version with the replaces property?
[X ] Is your new CSV referenced in the appropriate channel defined in the package.yaml ?
[ X] Have you tested an update to your Operator when deployed via OLM?

Your submission should not

[ X] Modify more than one operator
[X ] Submit operators to both upstream-community-operators and community-operators at once
[X ] Modify any files outside the above mentioned folders
[X ] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[ X] Description about the managed Application and where to find more information
[X ] Features and capabilities of your Operator and how to use it
[ X] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[X ] Human readable name and 1-liner description about your Operator
[X ] Valid category name1
[X ] One of the pre-defined capability levels2
[ X] Links to the maintainer, source code and documentation
[X ] Example templates for all Custom Resource Definitions intended to be used
[ X] A quadratic logo

@aneeshkp please review - thanks!


Mitigate against XML external entity attacks (XXE)

I was unable to forge an attack. I'll leave this open in case someone else is able to succeed.


HTML/XHTML formatting?

Is HTML/XHTML formatting supported? For example, our bot regularly pastes links like:
somewebsite

But it seems to render as the raw text. XHTML is defined in XEP-0071. Is it supported?

Nope, is there anyway to change the output on your bot?

Is there any way to support formatted text in any way?

Not at the moment unfortunately :(

@hhaidar thanks for the quick replies. Any plans for something like this in the future?

@hany probably for markdown, not sure for HTML just yet

@hhaidar markdown would be fantastic, as you can do pretty much everything. What is needed to implement this?

@hany more testing and perhaps some speed improvements. You can look at #365
Both @sibartlett and I have been kind of busy the last few months with work stuff. Hopefully end of year we can get some of these things in master.

OK, thanks @hhaidar. I'll close this issue out.

@hany If you need this for very specific patterns only, you could try to define custom replacement rules for the time being. See folder extras/replacements/.

Any word on this?  Looks like the PR #365 still needs to be merged...


translate Book

I want to take this book translated into Chinese, need to be authorized?

I think it'd be great to have our documentation translated into other languages; typically the approach we've taken is to add a link in the readme that points to a fork, that the fork owner maintains (see airbnb/javascript for an example).


Unable to disable /pause mvc endpoint (for post)

See https://github.com/spring-cloud/spring-cloud-config/issues/332 for original report made in wrong repo.

thanks for the quick fix!


XPM reader support

Request contains 4 commits:
a) 3 commits from last GSoC (2011) with my XPM input plugin (It was not accepted because i didnt write test)
b) 4th commit for current GSoC. It contains some improvements for xpm input plugin (cosmetic changes, and also added code to support reading 3byte and 12byte RGB color from file). I attached also test for my plugin.

I don't honestly know why this was not committed. Possibly my fault letting it slip through the cracks while (mistakenly?) believing we were waiting for more changes? I don't know. But I think the effort it would take to rebase to the modern code is probably not worth it, considering that in the years since it was written, I don't recall a single person asking for xpm support. But if a time traveller from 1992 shows up and wants to read xpm files with OIIO, we could revive this from the archives, I guess.


Release 8.8.6 and 9.1.0 unable to init a new Shield deployment

Describe the bug
When running initial setup of a new installation of Shield, the setup fails.
To Reproduce
Steps to reproduce the behavior:
deploy shield/8.8.6 bosh release on new VM
login to shield web ui using admin user
Select Setup
Enter new master password twice
error message: Unable to initialize the SHIELD Core
Expected behavior
Shield initializes correctly
Screenshots
If applicable, add screenshots to help explain your problem.
SHIELD versions (please complete the following information):
SHIELD Core: 8.8.6
SHIELD CLI: N/A
Additional context
/var/vcap/sys/logs/shield/shieldd
Fri Jan 27 19:45:07 UTC 2023 running any schema migrations
2023-01-27 19:45:08.327658361 +000 UTC /var/vcap/packages/shield/bin/shield-schema: INFO: deployed schema version 13
Fri Jan 27 19:45:08 UTC 2023 starting up shieldd (pid 7397)
2023-01-27 19:45:09.376825552 +000 UTC /var/vcap/packages/shield/bin/shieldd: ERROR: SHIELD's vault is %!s(vault.Status=1); please initialize or unlock this SHIELD core via the web UI or the CLI
2023-01-27 19:46:03.971995774 +000 UTC /var/vcap/packages/shield/bin/shieldd: ERROR: unable to read bootstrap.log: open /var/vcap/store/shield/bootstrap.log: no such file or directory
2023-01-27 19:46:16.119615991 +000 UTC /var/vcap/packages/shield/bin/shieldd: ERROR: POST /v2/init errored: 404 Not Found: no handler for route "secret/secret/archives/fixed_key". route entry not found.

I deployed the release 8.8.4 and was able to perform init.
Noted that vault did not mount kv store in 8.8.6 and 9.1.0 on a fresh deployment.

Hello,
I am facing with the same problem.
no handler for route "secret/secret/archives/525c8de5-987b-4cbe-a443-366055db1fa6". route entry not found

Is there a workaround for that ?
Thanks and regards,
Jean-Christophe.

It seems that vault team have fixed it with vault 1.11.6 (https://github.com/hashicorp/vault/issues/16639).
Is it possible to have a new shield release with a bump a vault from 1.11.4 to 1.11.6 ?
Thanks and regards,
Jean-Christophe.

There will be a new version to bump vault this year, exactly when depends on folks availabilities and priorities.

Hello,
Ok thanks for the answer.
Regards,
Jean-Christophe.

Hello,
Do you plan to have this new release before the end of the year ?
Thanks and regards,
Jean-Christophe.

Yes,  We are planning to have a new release before the end of year.


wiredep override does not work for repository dependencies

"wiredep override" as explained in https://github.com/taptapship/wiredep#bower-overrides works well for packages that have a bower.json file with a missing "main" element. But it does not work for packages that have no bower.json in the first place. It seems to be possible to reference git repositories as bower dependencies as follows:
"dependencies": {
    "smooth-scroll": "https://github.com/cferdinandi/smooth-scroll.git#~7.1.1",
     ...
  },
"overrides": {
    "smooth-scroll": {
      "main": [
        "dist/smooth-scroll.js"
      ]
    }

Using this kind of direct reference works with "bower install" but not with "grunt wiredep". In my case the index.html does not import the file "smooth-scroll.js" altough I expected "grunt wiredep" to look for the overrides.
How to solve this issue?

Solved. I had only missed the subfolder "dist/js"


File explorer super slow

Environment
Windows build number: 10.0.19042.631 (20H2)
Windows Terminal: 1.6.10571

RAM: 16GB
CPU: Intel Core i7-9700 3.00GHz
Disk: 512GB SSD
GPU: Intel(R) UHD Graphics 630

Any other software?

Note: this is a fresh new computer and it happens before with my previous computer
Steps to reproduce

I downloaded the Windows Terminal from GitHub Release and installed it on offline computer, the second I installed it the File Explorer because super slow that it takes about 5 seconds to open a folder when I uninstalled the Windows Terminal the File Explorer returned to his normal speed.
Expected behavior

File Explorer stay fast
Actual behavior

File Explorer very slow

Huh. If this was more widespread, then I'm sure we would have heard of this by now. It's possible that this is just explorer.exe trying to populate the context menu entries for the Terminal. Maybe it goes away after a reboot (and explorer caches the results).

How specifically did you install the terminal offline? By double-clicking the .msix, or by doing the "rename to zip, unzip" trick?
Does installing the Terminal, then rebooting the machine, then opening explorer repro the same slowness?
Is it slow every time you navigate in explorer (like, every time you change directories), or just on the initial launch of the explorer window?

Huh. If this was more widespread, then I'm sure we would have heard of this by now. It's possible that this is just explorer.exe trying to populate the context menu entries for the Terminal. Maybe it goes away after a reboot (and explorer caches the results).

The Same issue happening for a colleague of mine too

How specifically did you install the terminal offline? By double-clicking the .msix, or by doing the "rename to zip, unzip" trick?

Double clicking

Does installing the Terminal, then rebooting the machine, then opening explorer repro the same slowness?

Yes

Is it slow every time you navigate in explorer (like, every time you change directories), or just on the initial launch of the explorer window?

Every time

On a computer with a messed up windows store ( and the store reset app does nothing for me ). I had also changed permission on the windows app directory as a last ditch effort to fix the above issue. Working around it I had to create a new windows user, log in as that and then I could install WSL. It's been a journey.
I used chocolatey to install windows terminal, then proceeded to reboot at some point.
After that Explorer was absolutely useless if the internet was connected, disabling all my network adapters would immediately resolve.
I troubleshooted right click context menu changes, drivers, you name it for hours.
uninstalling windows terminal fixed it immediately. not sure why it care about an internet connection when i'm right clicking in explorer ( and why it's faster otherwise ).


Cu capture queue 1

Further streamline Metrics Cap & U metrics tests.
Goal
Remove references to Metrics::Targets.capture_ems_targets (from specs)
There is one piece of code that tests this method along with enabled and disabled targets.
All other specs are focused on what actually goes onto the queue.
So these other specs were rewritten to not depend upon capture_ems_targets,  nor the rspec example :vmware_enabled_disabled_targets, instead they are just using regular enabled targets.
Also, these specs have been rewritten to no longer simply count the number of records on the queue but rather test actual values on the queue. It is more explicit, and will make changes to the queue more transparent and straight forward in the future.
NOTE
This only changes 3 files, but the last commit removes a number of factories, so that touches 11 files.
Doing a diff without whitespace can help on a few of the commits.

fixed cops (and added 1 missing file to fix build)

fixing cops 2 (it changed it's mind back... :( )
kicking to see if that fixes a sporadic failure