NIFI-12261 Remove Hortonworks Schema References

Summary
NIFI-12261 Removes Hortonworks Schema references from Schema Registry Services, including the removal of the Schema Protocol Version property, which was specific to Hortonworks Schema Registry formats. The removal includes use of migrateProperties() to ensure that the Schema Protocol Version property is removed from existing services. This removal follows along with the removal of the Hortonworks Schema Registry components completed in NIFI-11208.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[X] Build completed using mvn clean install -P contrib-check

[X] JDK 21

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

I'm not sure this is the approach we want to take here. We have users with a mix of hortonworks / confluent setup (schema encoding following the hortonworks approach but sending the data into confluent, and vice versa). We removed the HWX Schema Registry but I think removing the schema encoding strategies does not have the same level of impact and one could wonder why we keep Confluent but not HWX/Cloudera. A cleaner approach would probably to have the schema encoding as an extension point so that it can be part of the NARs but it sounds like a significant effort that is not really worth it. Wouldn't be easier here to just rename the classes using the Cloudera name and keep both Cloudera and Confluent? I could even see value in reverting the removal of the HWX Schema Registry and just rename it given that we keep the equivalent for Confluent and we're adding support for additional Schema Registries (Glue, Apicurio, etc). Thoughts?

Thanks for the feedback @pvillard31.
I think the best solution would be to create a new Controller Service interface for writing schemas.
I agree that we don't want to artificially limit support for any given type of Schema Registry. From that perspective, having formatting capabilities specific to any particular vendor at this level makes it challenging without turning this into an extension point.
It is worth noting that the current capabilities include several older versions, and I'm not sure if there are any other more recent changes. It is also worth noting that the Registry code still uses the hortonworks namespace, which is understandable for historical reasons.
With that in mind, perhaps we should avoid any renaming, and instead target the introduction of an extension point before the 2.0.0 GA release?

Yeah I don't feel like there is any perfect solution. Some time ago, I discussed with @bbende the option of having an extension point but I think I remember that it would also raise some questions, can't exactly remember right now though.

Thanks again for the feedback @pvillard31, I will evaluate the changes necessary to create a new extensible Controller Service interface for reading Schema Registry Access information.


Missing Java validation classes

While using the artifact org.treblereel.gwt.gwtproject.widgets:gwt-widgets:202203291 I get a lot of errors like this:
 Tracing compile failure path for type 'com.google.gwt.validation.client.constraints.SizeValidatorForArrayOfObject'
      [ERROR] Errors in 'jar:file:/Users/Shared/Jenkins/.gradle/caches/modules-2/files-2.1/com.google.gwt/gwt-user/2.9.0/d68c92135dca7f9880870fb379ded95b96ac8ad9/gwt-user-2.9.0.jar!/com/google/gwt/validation/client/constraints/SizeValidatorForArrayOfObject.java'
         [ERROR] Line 29: No source code is available for type javax.validation.ConstraintValidatorContext; did you forget to inherit a required module?
      [ERROR] Errors in 'jar:file:/Users/Shared/Jenkins/.gradle/caches/modules-2/files-2.1/com.google.gwt/gwt-user/2.9.0/d68c92135dca7f9880870fb379ded95b96ac8ad9/gwt-user-2.9.0.jar!/com/google/gwt/validation/client/constraints/AbstractSizeValidator.java'
         [ERROR] Line 27: No source code is available for type javax.validation.constraints.Size; did you forget to inherit a required module?
         [ERROR] Line 27: No source code is available for type javax.validation.ConstraintValidator<A,T>; did you forget to inherit a required module?

I tried adding <inherits name="javax.validation.Validation"/> to the module that also inherits widgets via <inherits name="org.gwtproject.user.Widgets" />, but got the same error. I'm using Vertispan maven repo together with central to resolve maven dependencies. Tested with Java 8 + Gradle + both Windows and MacOS.

@zbynek thanks, i ll take a look.

@zbynek Hello, it's not my top priority ... if you use gwt2, i think it's better to use existing validation api or switch to this great project https://gitlab.com/ManfredTremmel/gwt-bean-validators.

@treblereel thanks, but I'm not looking for a validation library. I just want to resolve the compilation errors that I get when I use the widget library with Gwt2. The long term goal is migrating from Gwt2 to j2cl, I was hoping that switching widgets to this library would be a useful intermediate step.

@zbynek i have answered to you in gitter

@zbynek please check https://github.com/treblereel/gwtproject/tree/development, if issue fixed, i ll publish new release

I still had some issues related to validation, most likely because several com.google modules were still inherited and that caused conflicts between supersourced javax.validation from the old modules and the new one. Partial fix proposed in https://github.com/treblereel/gwtproject/pull/14


Put image access on main thread.

Use ?w=1 for a much nicer diff.

Woah, ?w=1 is awesome. I'll need to remember that in the future.


OCM-2180 | fix: improve info message for the command to use to list upgrades

This should help users to migrate to the correct command.
@xueli181114

/lgtm


[Platform] Provide a better error message when the developer certificate can't be used

Partially addresses https://github.com/aspnet/AspNetCore/issues/15118
Long story short, Mac OS will introduce notarization for apps, which changes the identity of dotnet.exe with regards to the OS.
That makes the keys for certs in the cert store (backed by keychain) not accessible to the app if the certificate was created with a dotnet.exe that was not notarized.
Users upgrading to 3.1 in the future will encounter this on Mac OS and they won't get any feedback about what's going on other than a failed connection screen in the browser).
To help users in this scenario we are introducing an additional error message with instructions on what to do when we detect the developer certificate is being used for HTTPS, so that at least there is a proper error message.
We are still investigating all the ramifications of this issue, hence the "partially addresses it"

@Pilchie FYI

🆙 📅

@Pilchie this is just waiting for your approval.

Approved for 3.1.0-Preview3.  Thanks @javiercn


nginx with ldap users to access elasticsearch indexes

Hello Team,
I installed nginx-auth-ldap module in my nginx server.I able to authenticate ldap users to access elasticsearch indexes.
Here is the scenario that I am trying to achieve.
I have two roles configured in my LDAP.
Level1 - Users under this role should have access to only perticular indexes.
Level2 - Users under this role should have access to all the indexes.
how to specify above permissions in my nginx.conf file
nginx.conf file as follows
events {
worker_connections  1024;
}
http {
ldap_server LDAP1 {
url "ldap://xxx.xxx.x.xxx/cn=xx,ou=yy,dc=test,dc=com?uid?sub?(objectClass=posixAccount)";
binddn "cn=xx,dc=test,dc=com";
binddn_passwd "xx";
group_attribute uid;
group_attribute_is_dn on;
require valid_user;
  }

upstream elasticsearch {
ip_hash;
server xxx.xxx.x.xxxx;
}
server {
listen 80;
auth_ldap "Please enter your ldap credentials";
auth_ldap_servers LDAP1;
location /{
  root   html;
  index  index.html index.htm;
  if ($request_method ~* "^(DELETE)$") {
    return 403;
  }

  proxy_pass http://elasticsearch;
  include /usr/local/nginx/conf/fastcgi_params;
}

}
}
Please suggest me where I am going wrong.
Thanks,
xxxxx

Hello Team,
Please let me know any updates on this issue.

You can do something like that:
ldap_server level_1 {
  ...
  require group "cn=level_1,ou=groups,dc=example,dc=com";
}
ldap_server level_2 {
  ...
  require group "cn=level_2,ou=groups,dc=example,dc=com";
}

...

location /foo {
     auth_ldap_servers level_1;
     ...
}
location /bar {
     auth_ldap_servers level_2;
     ...
}


Added typings to DT

@sonicsnes could you eventually review it or check it if I missed something?
DefinitelyTyped/DefinitelyTyped#28449

The types have been merged into DT, so eventually remember to update them with major and minor updates.

Thanks for the typings! They look good to me -- for now.
I'm not sure I'll be able to keep them extremely up-to-date since I don't use typescript myself, but I don't anticipate many api changes in the near future.

I'll keep an eye on it, too, and update them eventually with newer gamedig releases, should they introduce breaking changes! Keep the good work up!


Fix 0 Items in closet and sharing

Poshmark updated the look and field and changed a number of the elements we were looking for.  Update the elements we are looking for to match the new site.  Resolves: https://github.com/jmausolf/poshmark_sharing/issues/27

Looks like the website has reverted back to the original css, closing this pr.


[WIP] Add and use functions to test consistency of protocol implementations

Fixes #1053

Did not change MatrixGate tests due to repr being inconsistent because repr for numpy arrays doesn't behave as required.
Did not change ControlledGate tests due to their not supporting QASM output

def _numpy_array_repr(arr: np.ndarray) -> str:
    return 'np.array({!r})'.format(arr.tolist())

use that for the numpy arrays


Working for some images, not working for some even in case of a clear picture

Hi,
I tried retinaface on below picture.
https://c.gawby.com/file/mjaymg/mdg/mdm/mdu/ntc/u/j/o/ujjwal-chopra-612566686.jpg
It doesn't work, and returns
(array([], shape=(0, 5), dtype=float64), array([], shape=(0, 5, 2), dtype=float64))

I have noticed this same thing with close cropped images, I haven't been able to nail it down.. but I added a helper function to pad the image and redo the detection.. seems like just putting the image in the top left corner and filling with black seems to make detections work.. I don't know why it does this.. but in the case of no detections I run the image through this function.. originally I was centering the image in the black background and that worked as well, but it had the side effect of the coordinates being incorrect, by putting it in the top left corner there is no adjustments needed for the coordinates.. an example pad function is (assuming a cv2 image):
def padImage(img):
(w,h,c) = img.shape
maxdim = np.max([w,h])
paddeddim = int((maxdim/1024) + 1) * 1024
img2 = np.zeros((paddeddim,paddeddim,3),dtype=np.float32)
img2[0:w,0:h] = img
return img2
It does seem odd as the images of this type seem almost ideal for face detection but yet it doesn't work.  Note that this worked for insightface, not retinaface directly but I believe it uses the same methods/models and so the solution and cause is likely similar.

Try to set threshold to a smaller value. I got successful result for the given image.
Not working
RetinaFace.detect_faces("input.jpg")

Working
RetinaFace.detect_faces("input.jpg", threshold = 0.1)


Remove spurious "0" from info/data releases array response

Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
Review the contributing guidelines for this repository; remember in particular:

do not modify code without testing for regression
provide simple unit tests to test the changes
the PR must not fail unit testing
if you're adding/updating documentation of an endpoint, make sure you add/update the necessary parameters to the (template) configuration files in the ensembl-rest_private repo

Description
Remove spurious "0" from info/data releases array response.
Use case
info/data endpoint is returning "0" as part of the response because of duplication of meta table in two different bacteria dbs. Perl API modifies the species name by suffixing original name with a number but, other attributes such as release is undefined due to which rest server is considering the undefined release and returning "0" as part of releases array response.
Benefits
info/data returns the correct response
Possible Drawbacks
If applicable, describe any possible undesirable consequence of the changes.
Testing
note to submitters and reviewers: documentation-only changes may reflect
changes in other repos that can result in new or different output from
REST endpoints. In turn, these may require new tests or changes to existing tests.
Have you added/modified unit tests to test the changes?
If so, do the tests pass/fail?
Have you run the entire test suite and no regression was detected?
Changelog
Are you changing the functionality of an endpoint? If so, please give a one line summary for the public facing changelog.
eg. [/info/data] Remove spurious "0" from info/data releases array response.

Coverage increased (+0.005%) to 44.432% when pulling 4a1f7654c27b924a34ebc25e3f866728c34d7ca6 on bugfix/ignore-undefined-release into c79e196a07a033777db719403d4ba23cd16a3fb7 on main.


feat: show logged out succesfully toast on loggin out

Summary:
When the user logs out off the app a toast should be shown which says logged out successfully
Steps to reproduce:
Go to the Navigation bar and click on logout
Expected behavior:
The user gets logged out with a toast showing logged out successfully
Observed behavior:
The user gets logged out but no toast is shown.

I would like to work on this.

@abhijitramesh Where is the GCI task link? Can you please share?

@PiyushSharma99 someone is already working on this and have claimed the task and hence you are not able to find it
here is the link:
https://codein.withgoogle.com/dashboard/tasks/5071897474105344/


The chat agent 'llama_chat_agent' sometimes fails to correctly identify whether tools are needed

Sometimes I input the content that exists in the document, but chat_agent judges that it doesn't need to use tools. How can I improve it so that it can accurately determine whether to use tools?

Also, sometimes AgentExecutor has already obtained the correct answer during the first stage of execution, but it keeps iterating until it exceeds the maximum number of times.

my code:
from llama_index import SimpleDirectoryReader, ServiceContext, GPTVectorStoreIndex, PromptHelper,StorageContext,load_index_from_storage
from llama_index.llm_predictor.chatgpt import ChatGPTLLMPredictor
from langchain.chat_models import ChatOpenAI
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
import faiss
import os
from llama_index.langchain_helpers.agents import IndexToolConfig, LlamaIndexTool,LlamaToolkit
from queue import Queue
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.chains import ConversationalRetrievalChain
from langchain.agents import AgentType
from langchain.docstore import InMemoryDocstore
from langchain.vectorstores import FAISS
from llama_index.langchain_helpers.agents import create_llama_chat_agent,create_llama_agent
from llama_index.langchain_helpers.memory_wrapper import GPTIndexChatMemory
os.environ["OPENAI_API_KEY"] = 'sk-xxxx'
from langchain.memory import ConversationBufferMemory
from llama_index import download_loader
from pathlib import Path

embedding_size = 1536 # Dimensions of the OpenAIEmbeddings
index = faiss.IndexFlatL2(embedding_size)
embedding_fn = OpenAIEmbeddings().embed_query
vectorstore = FAISS(embedding_fn, index, InMemoryDocstore({}), {})

def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 2000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 2000

    q = Queue()

    llm=ChatOpenAI(temperature=0.4, model_name="gpt-3.5-turbo", verbose=True,streaming=True, callbacks=[StreamingStdOutCallbackHandler()])
    llm_predictor = ChatGPTLLMPredictor(llm=llm)
    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)

     documents = SimpleDirectoryReader(directory_path).load_data()
    service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    index = GPTVectorStoreIndex.from_documents(documents,service_context=service_context)

    index.storage_context.persist("D://storage")

    # rebuild storage context
    storage_context = StorageContext.from_defaults(persist_dir='D://storage')
    # load index
    index = load_index_from_storage(storage_context)
    #response_mode="compact",
    query_engine = index.as_query_engine(similarity_top_k=30,service_context=service_context) #query_engine = index.as_query_engine(similarity_top_k=1,streaming=True,service_context=service_context)

    retriever = vectorstore.as_retriever(search_kwargs=dict(k=1))
    #qa = ConversationalRetrievalChain.from_llm(llm,retriever)

    # memory = GPTIndexChatMemory(
    #     index=chat_history_index,
    #     memory_key="chat_history",
    #     query_kwargs={"response_mode": "compact","streaming":True,"service_context":service_context,"similarity_top_k":1}, #,"streaming":True
    #     # return_source returns source nodes instead of querying index
    #     return_source=True,
    #     # return_messages returns context in message format
    #     return_messages=True,
    # )

    #使用内存保存对话记录
    memory = ConversationBufferMemory(
        memory_key="chat_history"
    )

    tool_config = IndexToolConfig(
        query_engine=query_engine,
        name=f"AI chat bot",
        description=f"If it is a game-related question, please use tools to obtain information before answering.If this question is about a game and you don't know the answer, jst say 'sorry,i don't know'. Remember must respond in Chinese.",
        tool_kwargs={"return_direct": True}
    )

    tool = LlamaIndexTool.from_tool_config(tool_config)
    toolkit = LlamaToolkit(
        index_configs=[tool],
    )

    agent_chain = create_llama_chat_agent(
        toolkit,
        llm,
        memory=memory,
        verbose=True,
        agent_kwargs={"max_iterations":3}
    )

    while True:
        query = input("What do you want to ask? ")
        print(agent_chain.memory.chat_memory.messages)
        response_stream = agent_chain.run(query)

        if  hasattr(response_stream,'response_gen'):
                for text in response_stream.response_gen:
                    print(text, end="")
                    #todo send to client
        else:
            print(response_stream)

service_context=construct_index('D://Traning')

output examp:

> Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: AI chat bot
Action Input: "How to obtain diamonds."Thought: Do I need to use a tool? Yes
Action: AI chat bot
Action Input: "How to obtain diamonds？"目前钻石可以通过以下几种方式获得：刷新每日商店、完成每日任务、每日广告奖励、参与赛季对战并达到一定杯数、邀请新人好友进入游戏、好友助力、集结令探寻宝藏。The original answer is not related to the new context. Please provide more information or clarify the question.钻石可以通过完成游戏内的任务、活动和竞技场排名等方式获得。此外，可以通过充值来购买钻石，以便更
快地提升自身实力和获取更多的英雄和装备。但是，建议玩家在游戏中充分利用各种机会获得钻石，以避免过度依赖充值，从而保证游戏的公平性和可持续性。The game in question is called " Potato Hero" and it is a card battle game. The diamond in the game can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.The additional context is not relevant to the original question about how to obtain diamonds
in " Potato Hero". Therefore, the original answer still stands:

The diamond in the game can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.The context provided is not relevant to the question about how to obtain diamonds in " Potato Hero".
Therefore, the original answer still stands:

The diamond in the game can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.
Observation: The context provided is not relevant to the question about how to obtain diamonds in " Potato Hero". Therefore, the original answer still stands:

The diamond in the game can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.
Thought:Do I need to use a tool? No
AI: In " Potato Hero", diamonds can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.Do I need to use a tool? No
AI: In " Potato Hero", diamonds can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.

> Finished chain.
In " Potato Hero", diamonds can be obtained through completing in-game tasks, activities, and arena rankings. Additionally, players can purchase diamonds through recharging. However, it is recommended that players take advantage of various opportunities to obtain diamonds in the game to ensure fairness and sustainability.
What do you want to ask?

Both of these behaviors are (sadly) hard to control. Which tool it picks is based almost entirely on the description of the tools
Whether or not it feels like it has the correct answer is up to the LLM to decide. This can sometimes be alleviated by making some tools return_direct=True


Fix MutableHashTableOpTest.testMutableHashTableOfTensors. (Issue #23564)

np.sort sorts the last axis by default, but this axis was already sorted. It should have been sorting the first axis.

@ysuematsu  Any update please ?

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 14 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 29 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 29 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 44 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 59 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 74 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 89 days with no activity and the awaiting review label has been applied.

Nagging Reviewer @ysuematsu: You have been added as a reviewer to this pull request. Please add your review or reassign. It has been 104 days with no activity and the awaiting review label has been applied.

@MorganR please resolve conflicts.

@MorganR Gentle reminder to resolve the conflicts.

@MorganR Did you get a chance to look on conflicts? Please let us know on the update. Thanks !

This test was moved to tensorflow/python/kernel_tests/lookup_ops_test.py where it has since been fixed. Closing this pull request.


Connect device with a certificate

Closes #15

PR looks good, but if possible, please check if the new scripts works in our devcontainer.

I checked and they work.


[Fix bug] Fix result.params always be null

The length of URLSearchParams cannot be obtained with Object.keys(params).length, thus Line 32 was never be executed
https://github.com/9bany/curl-to-json/blob/9c681a60cceac0566f0e1b4bc3a87693616fe188/src/pare-json.js#L27-L34
Change it to
if(Array.from(params).length) {

works on my side
Ref: https://stackoverflow.com/questions/42156024/how-do-i-get-the-size-or-length-for-url-searchparams

LGTM ! Thanks @CW-B-W


Update names of bindings/templates to clearly distinguish release stages

Need to update the bindings/templates with clear labels/categories for things that are released/preview/experimental.
Release stages
Release:

Supported and will stay backwards compatible within semver standards. Production use cases supported.

Preview:

Close to release, but some compatibility issues may arise. Production use cases not recommended. Mostly committed to releasing this experience.

Experimental:

Will likely have large changes. Expect breaking changes, even in patch versions. Absolutely do not use in Production workloads. Not necessarily committed to releasing this experience.

Scenarios needing stage warnings
Templates
All templates receive the lowest level of support specified.
Example: if a template is using C# (Release) and SaaS File Triggers (Preview), it is preview, because Preview is the lowest support level.
Example: If a template is using Python (experimental) and Queue Trigger (Release), it is experimental, because experimental is the lowest support level.
Integrate tab
In the portal, we should message if a binding is in a stage other than release (preview/experimental).
Breakdown by category
Languages
Release

C#
JS

Preview

F#
PowerShell

Experimental

Batch
Bash
PHP
Python

Bindings
Release

Service
Trigger
Input
Output

HTTP
✓

✓*

Timer
✓

Manual
✓

Azure Blob Storage
✓
✓
✓

Azure Queue Storage
✓
✓
✓

Azure Table Storage

✓
✓

Azure Service Bus [Queues + Topics]
✓

✓

Azure Event Hub
✓

✓

Azure DocDB

✓
✓

Azure Notification Hubs

✓

Azure Mobile Table

✓
✓

(*) HTTP output response only used with HTTP Trigger request
Preview

Service
Trigger
Input
Output

SaaS File*
✓
✓
✓

SendGrid

✓

Twilio SMS

✓

(*) SaaS Files is being renamed to External Files soon
Experimental

Service
Trigger
Input
Output

SaaS Table*

✓
✓

(*) SaaS Table is being renamed to External Tables soon

Added SendGrid and Twilio to the list.

Removed trigger from SaaS table

fixed in https://github.com/Azure/azure-webjobs-sdk-templates/commit/0df02b9946d31117cef9d1f5e51828a2100bcdd2


enter should also accept selection

if currently in suggestion drop down

I'll implement this as soon as I figure out how to do it. :smile:


Oauth authentication

Hi.
I'm trying to figure out how to request an API which require oauth 2 Authentication.

/v2/oauth/token has to be used with basic Auth to get a Bearer token
every requests must contain the token
after 3600s, the token is expired and a new /v2/oauth/token call is needed

Any recommended way to implement this ? Are events the place to do this ?
Thanks

You can use guzzle_bundle.pre_transaction event:
public function onPreTransaction(PreTransactionEvent $event)
    {
        $request     = $event->getTransaction();
        $serviceName = $event->getServiceName();
        if ($serviceName == $this->serviceName) {
            if (!$this->requestHelper->isOauth2TokenReceiveRequest($request)) {
                $request = $request->withHeader('Authorization', sprintf('Bearer %s', $this->accessTokenProvider->getAccessToken()->getToken()));
                $event->setTransaction($request);
            }
        }
    }

To handle  401 Unauthorized  you can override EightPoints\Bundle\GuzzleBundle\Middleware\EventDispatchMiddleware to catch if the response is with status code >=400 (in this case GuzzleHttp\Exception\RequestException is thrown):
 // Handle the response form teh server.
                return $promise->then(
                        function (ResponseInterface $response) {
                        // Create hte Post Transaction event.
                        $postTransactionEvent = new PostTransactionEvent($response, $this->serviceName);
                        // Dispatch the event on the symfony event dispatcher.
                        $this->eventDispatcher->dispatch(GuzzleEvents::POST_TRANSACTION, $postTransactionEvent);
                        // Continue down the chain.
                        return $postTransactionEvent->getTransaction();
                    }, function (RequestException $e) {
                        $postTransactionEvent = new PostTransactionErrorEvent($e, $this->serviceName);
                        // Dispatch the event on the symfony event dispatcher.
                        $this->eventDispatcher->dispatch(GuzzleEvents::POST_TRANSACTION_ERROR, $postTransactionEvent);
                    }
                );

Thanks for your answer @piron4o!


Copy image before changing it

@ferrine pointed out that I was quietly editing images passed to this function. This should fix that.

Pull Request Test Coverage Report for Build 5

5 of 5 (100.0%) changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 97.826%

Totals

Change from base Build 3:
0.02%

Covered Lines:
135

Relevant Lines:
138

💛  - Coveralls

Pull Request Test Coverage Report for Build 5

5 of 5 (100.0%) changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 97.826%

Totals

Change from base Build 3:
0.02%

Covered Lines:
135

Relevant Lines:
138

💛  - Coveralls


[e2e]: go schedule main config-ldflags-main-dir slsa3

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/8449935074
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.schedule.main.config-ldflags-main-dir.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.schedule.main.config-ldflags-main-dir.slsa3.yml
Trigger: schedule
Branch: main
Date: Wed Mar 27 10:09:38 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/8465571313
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.go.schedule.main.config-ldflags-main-dir.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.go.schedule.main.config-ldflags-main-dir.slsa3.yml
Trigger: schedule
Branch: main
Date: Thu Mar 28 10:08:49 UTC 2024
Tests are passing now. Closing this issue.


Fix typo in Roman Numerals

Simple typo I noticed while solving it.
Unrelated to this PR, but I'm interested in contributing to this track. There's not many issues open, but is it open to new practice exercises? Or are there other things, like building analyzers?

Good find, but this file is generated from upstream here:
https://github.com/exercism/problem-specifications/blob/main/exercises/roman-numerals/description.md?plain=1
Would you mind putting a pull request together upstream?

Oh of course, I'm dumb.


x/tools/gopls: add go:embed support

Is your feature request related to a problem? Please describe.
Go embed feature introduced in version 1.16. Currently, go-vscode treats text like  //go:embed app  as a simple comment.
Describe the solution you'd like
Provide text highlighting or navigation by CTRL+Clicking on these comments
Describe alternatives you've considered
Goland provides this feature in its recent versions:

Additional context
VS code view

It would be really nice if we have this feature in my favorite code editor.

List of improvements

[ ] checks for the "embed" import
[ ] Report it as semantic for highlighting
[ ] The directive must immediately precede a line containing the declaration of a single variable. Only blank lines and ‘//’ line comments are permitted between the directive and the declaration.

We should also include a quick-fix to add the missing "embed" import.

@vikblom expressed interest in picking this up.
I think the semantic highlighting support is a bit tricky to implement, so let's separate that bit out for now.
Viktor, if you are interested you could do the following:

[ ] Update the embeddirective
analyzer to check that the embed directive immediately precedes a line containing the declaration of a single variable. Only blank lines and ‘//’ line comments are permitted between the directive and the declaration.
[ ] Update the analyzer to include a suggested fix to add the missing "embed" import.

I recommend doing this in two separate CLs. Adding the suggested fix is a little subtle (we should re-use the existing gopls.add_import custom command). So let's discuss the second item after the first is done.
I'll tentatively assign you. Let me know if you don't think you'll be able to work on this.

Well, @vikblom it turns out I cannot assign you, so I assigned myself as your proxy :)

Hi @findleyr, with the CL above merged I've started looking at suggesting a fix from the analyzer.
analysis.Diagnostic has a field for SuggestedFixes which seems straight forward enough. You mentioned gopls.add_import, best I can tell that command is backed by source.AddImport. Am I aiming for the right things?
Are there any requirement on placement for the import. I'm used to placing blank imports like _ "embed" last.

Thanks @findleyr that approach works great :+1:
I've added a TestLSP testcase with a golden file verifying that the import is added. Do you have any suggestion or tips on how to test the negative case? (That the other kind of reports (incorrect decl) does not have a code action.)

I noticed that I haven't kept all the docs up to speed, for example analyzers.md. That should probably be corrected as part of this issue.

@vikblom the analyzer doc is auto-generated, and there is a test to enforce this: cd gopls && go generate .

@findleyr I see! Then it's the analyzers Doc field that (maybe) needs an update. I was thinking it should mention the new variable check and suggested fixes, do you agree?

Hi @findleyr, is there a path forward with semantic highlighting here, given that the vscode ticket was closed by triage?
I tried out a naive textDocument/documentLink solution passing file:// URIs. Emacs could handle both files and directories. VSCode only accepted files. Does it makes sense to use that as a opt-in or base case (only files) feature?
You mentioned code action+associated command and jump-to-definition+hover above, which seems doable. However if VSCode requires specific (or new) mechanisms to handle directories, this feature might spread to codebases (vscode or vscode-go) I'm not familiar with

Hi @vikblom, very sorry for the slow response on this.
Let's not worry about semantic tokens for now, I think that's a low priority. We can treat highlighting //go: directives as a separate project.
We discussed this in our triage meeting, and I think jump-to-definition+hover is the best way to implement this feature, as it will be supported by the greatest number of LSP clients. For directories, perhaps we should just pick an arbitrary file in the directory?

An arbitrary file could be unexpected as a user. But I don't see any alternatives if this is a limitation of LSP clients. I agree it is a good place to start.
I'll try to put something together and post a CL.

Great, thanks for sticking to this!

Hi @findleyr, what do you think is the next step? Would tackling semantic highlighting make sense or do we let the issue rest for a bit?

@vikblom if we're going to highlight go:embed, I think we should highlight all go: directives. Is that something you'd be interested in?
If so, I think we should perhaps make that a separate issue, and close this one.

Sure, I'd be glad to take a look directive highlights!
Thanks again for all the help in this issue.

Great! Opened https://go.dev/issue/63538.
Will close this then.


Original Picture

Hello, my question is about the pic, in the example select but is a thumbnail (low quality and small), How to select the image, video original from the library or camera?
Thanks

Hi @josaal17
You can use asset.fullResolutionImage or asset.fullScreenImage for image. For video, do you want to get the video as NSData?

Thanks, it's works!!

I'm sorry, I had not seen the other question!, Yes i need get the video as NSData!

Hi @josaal17
The originalAsset becomes public property in 2.3.0, now you can get the raw data of the selected assets like this:
pickerController.didSelectAssets = { [unowned self] (assets: [DKAsset]) in
    print("didSelectAssets")
    print(assets.map({ $0.url}))

	if let rep = assets.first?.originalAsset?.defaultRepresentation() {
		let sizeOfRawDataInBytes = Int(rep.size())
		let rawData = NSMutableData(length: sizeOfRawDataInBytes)!
		let bufferPtr = rawData.mutableBytes
		let bufferPtr8 = UnsafeMutablePointer<UInt8>(bufferPtr)

		rep.getBytes(bufferPtr8, fromOffset: 0, length: sizeOfRawDataInBytes, error: nil)
		print(rawData)
	}

    self.assets = assets
    self.previewView?.reloadData()
}


SSL certificate for database connection

I'm trying to do the deploy to heroku step (first bit of Part 2) but having some SSL related problems. It seems that Heroku requires connections to Postgres be with an SSL certificate? Has anyone else has stumbled across this problem and managed to solve it?
This is the information I get from heroku logs
[2021-03-07 17:11:39 +0000] [132] [ERROR] Application startup failed. Exiting.
2021-03-07T17:11:39.433115+00:00 app[web.1]: [2021-03-07 17:11:39 +0000] [132] [INFO] Worker exiting (pid: 132)
2021-03-07T17:11:39.662653+00:00 app[web.1]: [2021-03-07 17:11:39 +0000] [137] [INFO] Booting worker with pid: 137
2021-03-07T17:11:40.711469+00:00 app[web.1]: [2021-03-07 17:11:40 +0000] [137] [INFO] Started server process [137]
2021-03-07T17:11:40.711604+00:00 app[web.1]: [2021-03-07 17:11:40 +0000] [137] [INFO] Waiting for application startup.
2021-03-07T17:11:40.917185+00:00 app[web.1]: [2021-03-07 17:11:40 +0000] [137] [ERROR] Traceback (most recent call last):
2021-03-07T17:11:40.917187+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 526, in lifespan
2021-03-07T17:11:40.917188+00:00 app[web.1]: async for item in self.lifespan_context(app):
2021-03-07T17:11:40.917191+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 467, in default_lifespan
2021-03-07T17:11:40.917192+00:00 app[web.1]: await self.startup()
2021-03-07T17:11:40.917193+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 502, in startup
2021-03-07T17:11:40.917193+00:00 app[web.1]: await handler()
2021-03-07T17:11:40.917193+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/contrib/fastapi/__init__.py", line 92, in init_orm
2021-03-07T17:11:40.917194+00:00 app[web.1]: await Tortoise.init(config=config, config_file=config_file, db_url=db_url, modules=modules)
2021-03-07T17:11:40.917194+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/__init__.py", line 567, in init
2021-03-07T17:11:40.917195+00:00 app[web.1]: await cls._init_connections(connections_config, _create_db)
2021-03-07T17:11:40.917195+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/__init__.py", line 385, in _init_connections
2021-03-07T17:11:40.917196+00:00 app[web.1]: await connection.create_connection(with_db=True)
2021-03-07T17:11:40.917197+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/backends/asyncpg/client.py", line 94, in create_connection
2021-03-07T17:11:40.917197+00:00 app[web.1]: self._pool = await asyncpg.create_pool(None, password=self.password, **self._template)
2021-03-07T17:11:40.917197+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 398, in _async__init__
2021-03-07T17:11:40.917198+00:00 app[web.1]: await self._initialize()
2021-03-07T17:11:40.917198+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 426, in _initialize
2021-03-07T17:11:40.917199+00:00 app[web.1]: await first_ch.connect()
2021-03-07T17:11:40.917199+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 125, in connect
2021-03-07T17:11:40.917199+00:00 app[web.1]: self._con = await self._pool._get_new_connection()
2021-03-07T17:11:40.917200+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 468, in _get_new_connection
2021-03-07T17:11:40.917200+00:00 app[web.1]: con = await connection.connect(
2021-03-07T17:11:40.917201+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connection.py", line 1718, in connect
2021-03-07T17:11:40.917201+00:00 app[web.1]: return await connect_utils._connect(
2021-03-07T17:11:40.917202+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 663, in _connect
2021-03-07T17:11:40.917202+00:00 app[web.1]: con = await _connect_addr(
2021-03-07T17:11:40.917202+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 642, in _connect_addr
2021-03-07T17:11:40.917203+00:00 app[web.1]: await asyncio.wait_for(connected, timeout=timeout)
2021-03-07T17:11:40.917203+00:00 app[web.1]: File "/usr/local/lib/python3.8/asyncio/tasks.py", line 491, in wait_for
2021-03-07T17:11:40.917204+00:00 app[web.1]: return fut.result()
2021-03-07T17:11:40.917205+00:00 app[web.1]: asyncpg.exceptions.InvalidAuthorizationSpecificationError: no pg_hba.conf entry for host "<IP_ADDRESS>", user "cslyxnkrbufuls", database "d517o756cbcm5n", SSL off

I tried adding "ssl=True" to the db_url like so

def init_db(app: FastAPI) -> None:

    register_tortoise(
        app,
        db_url=os.environ.get("DATABASE_URL") + "?ssl=True",
        modules={"models": ["app.models.tortoise"]},
        generate_schemas=False,
        add_exception_handlers=True,
    )

but it seems that it can't verify a self signed certificate..
2021-03-07T17:16:56.727597+00:00 app[web.1]: [2021-03-07 17:16:56 +0000] [77] [ERROR] Application startup failed. Exiting.
2021-03-07T17:16:56.727835+00:00 app[web.1]: [2021-03-07 17:16:56 +0000] [77] [INFO] Worker exiting (pid: 77)
2021-03-07T17:16:56.789371+00:00 app[web.1]: [2021-03-07 17:16:56 +0000] [82] [INFO] Booting worker with pid: 82
2021-03-07T17:16:57.142427+00:00 app[web.1]: [2021-03-07 17:16:57 +0000] [82] [INFO] Started server process [82]
2021-03-07T17:16:57.142517+00:00 app[web.1]: [2021-03-07 17:16:57 +0000] [82] [INFO] Waiting for application startup.
2021-03-07T17:16:57.207812+00:00 app[web.1]: [2021-03-07 17:16:57 +0000] [82] [ERROR] Traceback (most recent call last):
2021-03-07T17:16:57.207814+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 526, in lifespan
2021-03-07T17:16:57.207814+00:00 app[web.1]: async for item in self.lifespan_context(app):
2021-03-07T17:16:57.207815+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 467, in default_lifespan
2021-03-07T17:16:57.207816+00:00 app[web.1]: await self.startup()
2021-03-07T17:16:57.207816+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 502, in startup
2021-03-07T17:16:57.207816+00:00 app[web.1]: await handler()
2021-03-07T17:16:57.207817+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/contrib/fastapi/__init__.py", line 92, in init_orm
2021-03-07T17:16:57.207817+00:00 app[web.1]: await Tortoise.init(config=config, config_file=config_file, db_url=db_url, modules=modules)
2021-03-07T17:16:57.207817+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/__init__.py", line 567, in init
2021-03-07T17:16:57.207817+00:00 app[web.1]: await cls._init_connections(connections_config, _create_db)
2021-03-07T17:16:57.207818+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/__init__.py", line 385, in _init_connections
2021-03-07T17:16:57.207818+00:00 app[web.1]: await connection.create_connection(with_db=True)
2021-03-07T17:16:57.207819+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/tortoise/backends/asyncpg/client.py", line 94, in create_connection
2021-03-07T17:16:57.207819+00:00 app[web.1]: self._pool = await asyncpg.create_pool(None, password=self.password, **self._template)
2021-03-07T17:16:57.207819+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 398, in _async__init__
2021-03-07T17:16:57.207822+00:00 app[web.1]: await self._initialize()
2021-03-07T17:16:57.207822+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 426, in _initialize
2021-03-07T17:16:57.207822+00:00 app[web.1]: await first_ch.connect()
2021-03-07T17:16:57.207822+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 125, in connect
2021-03-07T17:16:57.207823+00:00 app[web.1]: self._con = await self._pool._get_new_connection()
2021-03-07T17:16:57.207823+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/pool.py", line 468, in _get_new_connection
2021-03-07T17:16:57.207823+00:00 app[web.1]: con = await connection.connect(
2021-03-07T17:16:57.207823+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connection.py", line 1718, in connect
2021-03-07T17:16:57.207824+00:00 app[web.1]: return await connect_utils._connect(
2021-03-07T17:16:57.207824+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 674, in _connect
2021-03-07T17:16:57.207824+00:00 app[web.1]: raise last_error
2021-03-07T17:16:57.207824+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 663, in _connect
2021-03-07T17:16:57.207825+00:00 app[web.1]: con = await _connect_addr(
2021-03-07T17:16:57.207825+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 632, in _connect_addr
2021-03-07T17:16:57.207825+00:00 app[web.1]: tr, pr = await asyncio.wait_for(
2021-03-07T17:16:57.207826+00:00 app[web.1]: File "/usr/local/lib/python3.8/asyncio/tasks.py", line 491, in wait_for
2021-03-07T17:16:57.207826+00:00 app[web.1]: return fut.result()
2021-03-07T17:16:57.207826+00:00 app[web.1]: File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 564, in _create_ssl_connection
2021-03-07T17:16:57.207827+00:00 app[web.1]: new_tr = await loop.start_tls(
2021-03-07T17:16:57.207827+00:00 app[web.1]: File "uvloop/loop.pyx", line 1617, in start_tls
2021-03-07T17:16:57.207827+00:00 app[web.1]: File "uvloop/loop.pyx", line 1610, in uvloop.loop.Loop.start_tls
2021-03-07T17:16:57.207827+00:00 app[web.1]: File "uvloop/sslproto.pyx", line 517, in uvloop.loop.SSLProtocol._on_handshake_complete
2021-03-07T17:16:57.207828+00:00 app[web.1]: File "uvloop/sslproto.pyx", line 499, in uvloop.loop.SSLProtocol._do_handshake
2021-03-07T17:16:57.207828+00:00 app[web.1]: File "/usr/local/lib/python3.8/ssl.py", line 944, in do_handshake
2021-03-07T17:16:57.207828+00:00 app[web.1]: self._sslobj.do_handshake()
2021-03-07T17:16:57.207828+00:00 app[web.1]: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1124)

Any ideas?

Hi! I had the same problem and asked a question on StackOverflow: https://stackoverflow.com/questions/66119608/testdriven-io-test-driven-development-with-fastapi-and-docker-heroku-getting/66359374#66359374
The solution from there kinda works, but now I got stuck on Continues Integration section where I can't get DATABASE_URL (because we pass DATABASE_TEST_URL in test job).

Hey! Thanks for this 😄  that got it working in the end (after I saw your comment about adding it also to the generate_schema).
There must be a better way to fix it, but at least it got my heroku deployment to work - though now I can't run the docker image locally since the database url locally is sqlite://sqlite.db and splitting it to get the credentials breaks.
Maybe I'll be back here once I reach the Continuous Integration part...

@idjotherwise which command are you running? heroku run python app/db.py?
Also, can you run heroku run pip freeze and copy and paste the requirements. For example:
aiosqlite==0.16.1
asyncpg==0.22.0
attrs==20.3.0
certifi==2020.12.5
chardet==4.0.0
click==7.1.2
fastapi==0.63.0
gunicorn==20.0.4
h11==0.12.0
httptools==0.1.1
idna==2.10
iniconfig==1.1.1
iso8601==0.1.14
packaging==20.9
pluggy==0.13.1
py==1.10.0
pydantic==1.8.1
pyparsing==2.4.7
PyPika==0.44.1
pytest==6.2.2
python-dotenv==0.15.0
pytz==2020.5
PyYAML==5.4.1
requests==2.25.1
starlette==0.13.6
toml==0.10.2
tortoise-orm==0.16.21
typing-extensions==<IP_ADDRESS>
urllib3==1.26.3
uvicorn==0.13.4
uvloop==0.15.2
watchgod==0.7
websockets==8.1

I'm updating the course right now and I'm still not seeing this error.

@mjhea0 The error was happening when the application tried to start, so just after deploying with
heroku container:release web --app secret-crag-02876 (with my app name).
After adding @gmurman770 's suggestion, it wasn't happening anymore, and it deployed successfully (and heroku run python app/db.py also completed fine). But then when I try to run the same docker image locally with
docker run --name fastapi-tdd -e PORT=8765 -e DATABASE_URL=sqlite://sqlite.db -p 5003:8765 registry.heroku.com/secret-crag-02876/web:latest (with my app name)
it fails because the DATABASE_URL is in a different format. I guess I can just work around that by checking first if the database is postgres otherwise register the tortoise_orm like normal, but I tend to think this goes against the purpose of the course!
Here's my pip freeze:
aiosqlite==0.16.1
asyncpg==0.21.0
attrs==20.3.0
certifi==2020.12.5
chardet==3.0.4
click==7.1.2
fastapi==0.62.0
gunicorn==20.0.4
h11==0.12.0
idna==2.10
iniconfig==1.1.1
iso8601==0.1.14
packaging==20.9
pluggy==0.13.1
py==1.10.0
pydantic==1.8.1
pyparsing==2.4.7
PyPika==0.44.1
pytest==6.2.0
pytz==2020.5
requests==2.25.0
starlette==0.13.6
toml==0.10.2
tortoise-orm==0.16.18
typing-extensions==<IP_ADDRESS>
urllib3==1.26.3
uvicorn==0.13.1

Also when running docker-compose up it fails to start with the following information from the logs:
web-db_1  | The files belonging to this database system will be owned by user "postgres".
web-db_1  | This user must also own the server process.
web-db_1  |
web-db_1  | The database cluster will be initialized with locale "en_US.utf8".
web-db_1  | The default database encoding has accordingly been set to "UTF8".
web-db_1  | The default text search configuration will be set to "english".
web-db_1  |
web-db_1  | Data page checksums are disabled.
web-db_1  |
web-db_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
web-db_1  | creating subdirectories ... ok
web-db_1  | selecting dynamic shared memory implementation ... posix
web-db_1  | selecting default max_connections ... 100
web-db_1  | selecting default shared_buffers ... 128MB
web-db_1  | selecting default time zone ... UTC
web-db_1  | creating configuration files ... ok
web-db_1  | running bootstrap script ... ok
web-db_1  | sh: locale: not found
web-db_1  | 2021-03-13 10:32:16.542 UTC [30] WARNING:  no usable system locales were found
web-db_1  | performing post-bootstrap initialization ... ok
web-db_1  | syncing data to disk ... ok
web-db_1  |
web-db_1  | initdb: warning: enabling "trust" authentication for local connections
web-db_1  | You can change this by editing pg_hba.conf or using the option -A, or
web-db_1  | --auth-local and --auth-host, the next time you run initdb.
web-db_1  |
web-db_1  | Success. You can now start the database server using:
web-db_1  |
web-db_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
web-db_1  |
web-db_1  | waiting for server to start....2021-03-13 10:32:17.431 UTC [35] LOG:  starting PostgreSQL 13.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 64-bit
web-db_1  | 2021-03-13 10:32:17.434 UTC [35] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
web-db_1  | 2021-03-13 10:32:17.439 UTC [36] LOG:  database system was shut down at 2021-03-13 10:32:17 UTC
web-db_1  | 2021-03-13 10:32:17.442 UTC [35] LOG:  database system is ready to accept connections
web-db_1  |  done
web-db_1  | server started
web-db_1  |
web-db_1  | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/create.sql
web-db_1  | CREATE DATABASE
web-db_1  | CREATE DATABASE
web-db_1  |
web-db_1  |
web-db_1  | 2021-03-13 10:32:17.748 UTC [35] LOG:  received fast shutdown request
web-db_1  | waiting for server to shut down....2021-03-13 10:32:17.749 UTC [35] LOG:  aborting any active transactions
web-db_1  | 2021-03-13 10:32:17.749 UTC [35] LOG:  background worker "logical replication launcher" (PID 42) exited with exit code 1
web-db_1  | 2021-03-13 10:32:17.750 UTC [37] LOG:  shutting down
web-db_1  | 2021-03-13 10:32:17.759 UTC [35] LOG:  database system is shut down
web-db_1  |  done
web-db_1  | server stopped
web-db_1  |
web-db_1  | PostgreSQL init process complete; ready for start up.
web-db_1  |
web-db_1  | 2021-03-13 10:32:17.874 UTC [1] LOG:  starting PostgreSQL 13.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 64-bit
web-db_1  | 2021-03-13 10:32:17.874 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
web-db_1  | 2021-03-13 10:32:17.874 UTC [1] LOG:  listening on IPv6 address "::", port 5432
web-db_1  | 2021-03-13 10:32:17.877 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
web-db_1  | 2021-03-13 10:32:17.881 UTC [49] LOG:  database system was shut down at 2021-03-13 10:32:17 UTC
web-db_1  | 2021-03-13 10:32:17.884 UTC [1] LOG:  database system is ready to accept connections
web_1     | Waiting for postgres...
web_1     | PostgreSQL started.
web_1     | INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
web_1     | INFO:     Started reloader process [1] using statreload
web_1     | INFO:     Started server process [42]
web_1     | INFO:     Waiting for application startup.
web_1     | INFO:     Starting up...
web_1     | INFO:     Initializing DB...
web_1     | INFO:     postgres://postgres:postgres@web-db:5432/web_dev
web_1     | ERROR:    Traceback (most recent call last):
web_1     |   File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 526, in lifespan
web_1     |     async for item in self.lifespan_context(app):
web_1     |   File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 467, in default_lifespan
web_1     |     await self.startup()
web_1     |   File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 502, in startup
web_1     |     await handler()
web_1     |   File "/usr/local/lib/python3.9/site-packages/tortoise/contrib/fastapi/__init__.py", line 92, in init_orm
web_1     |     await Tortoise.init(config=config, config_file=config_file, db_url=db_url, modules=modules)
web_1     |   File "/usr/local/lib/python3.9/site-packages/tortoise/__init__.py", line 567, in init
web_1     |     await cls._init_connections(connections_config, _create_db)
web_1     |   File "/usr/local/lib/python3.9/site-packages/tortoise/__init__.py", line 385, in _init_connections
web_1     |     await connection.create_connection(with_db=True)
web_1     |   File "/usr/local/lib/python3.9/site-packages/tortoise/backends/asyncpg/client.py", line 94, in create_connection
web_1     |     self._pool = await asyncpg.create_pool(None, password=self.password, **self._template)
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 398, in _async__init__
web_1     |     await self._initialize()
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 426, in _initialize
web_1     |     await first_ch.connect()
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 125, in connect
web_1     |     self._con = await self._pool._get_new_connection()
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/pool.py", line 468, in _get_new_connection
web_1     |     con = await connection.connect(
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/connection.py", line 1718, in connect
web_1     |     return await connect_utils._connect(
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 674, in _connect
web_1     |     raise last_error
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 663, in _connect
web_1     |     con = await _connect_addr(
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 632, in _connect_addr
web_1     |     tr, pr = await asyncio.wait_for(
web_1     |   File "/usr/local/lib/python3.9/asyncio/tasks.py", line 476, in wait_for
web_1     |     return fut.result()
web_1     |   File "/usr/local/lib/python3.9/site-packages/asyncpg/connect_utils.py", line 556, in _create_ssl_connection
web_1     |     do_ssl_upgrade = await pr.on_data
web_1     | ConnectionError: PostgreSQL server at "web-db:5432" rejected SSL upgrade
web_1     |
web_1     | ERROR:    Application startup failed. Exiting.

@idjotherwise
Can you try upgrading the following dependencies:
asyncpg==0.22.0
fastapi==0.63.0
requests==2.25.1
tortoise-orm==0.16.21
Then, do a new build, push the image to the registry, and release.

Ah-ha that seems to have fixed it! Everything is working as expected now (without the need for the certificate work around). Thanks @mjhea0 - and thanks for the course as well, it's top quality stuff.


Unhandled python exception in read callback: KeyError: 'confirm'

I'm getting this on Ubuntu 14.04 using your default config (minus the graphite section). Any idea what would cause this?
[2017-07-25 19:21:32] rabbitmq_monitoring: Interval: 20.0
[2017-07-25 19:21:32] rabbitmq_monitoring: Connecting to: localhost:15672 as user:guest password:guest
[2017-07-25 19:21:32] rabbitmq_monitoring: Counting messages on: ('metering.sample', 'notifications.info')
[2017-07-25 19:21:32] Initialization complete, entering read-loop.
[2017-07-25 19:21:33] Unhandled python exception in read callback: KeyError: 'confirm'
[2017-07-25 19:21:33] Traceback (most recent call last):
[2017-07-25 19:21:33]   File "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq_monitoring/init.py", line 102, in read
metric.values = [overview['message_stats'][metric_instance]]
[2017-07-25 19:21:33] KeyError: 'confirm'
[2017-07-25 19:21:33] read-function of plugin `python.collectd_rabbitmq_monitoring' failed. Will suspend it for 40 seconds.

Yes I can fix it I have recently experienced this issue as well, looks like there may have been some changes in the expected keys returned by the rabbitmq management plugin response through pyrabbit.  I'll get a quick fix up in a little bit

awesome, thanks.

You can try the code out before I publish to pypi to verify it fixes your issue, just replace "/usr/local/lib/python2.7/dist-packages/collectd_rabbitmq_monitoring/init.py" on your system with the latest in this repo.
If that fixes it I'll repackage to pypi for pip installation.

Yep, that's working.


support ov02c10 in Samsung Galaxy Book 3 Pro

In my Samsung Galaxy Book 3 Pro (960XFG), the ov02c10 sensor is bound on CSI port 0.
And I guess this commit also should fix the IPU6 camera problems in many laptops of the Samsung Galaxy Book series.
my dmesg log:
$ sudo dmesg | grep -iE 'ipu6|02c1|int34'
[    3.920345] int3472-discrete INT3472:01: power-enable \_SB.GPI0 pin number mismatch _DSM 227 resource 131
[    3.921693] int3472-discrete INT3472:01: privacy-led \_SB.GPI0 pin number mismatch _DSM 101 resource 357
[    3.923018] int3472-discrete INT3472:01: clk-enable \_SB.GPI0 pin number mismatch _DSM 246 resource 150
[    3.927463] ov02c10: module verification failed: signature and/or required key missing - tainting kernel
[    3.944194] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.949366] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.955211] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.957398] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.961490] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.966605] intel-ipu6 0000:00:05.0: enabling device (0000 -> 0002)
[    3.966619] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.966733] intel-ipu6 0000:00:05.0: Device 0xa75d (rev: 0x0)
[    3.966749] intel-ipu6 0000:00:05.0: physical base address 0x603c000000
[    3.966750] intel-ipu6 0000:00:05.0: mapped as: 0x000000000036c1fe
[    3.966808] intel-ipu6 0000:00:05.0: Unable to set secure mode
[    3.966812] intel-ipu6 0000:00:05.0: IPU in non-secure mode
[    3.966814] intel-ipu6 0000:00:05.0: IPU secure touch = 0x80000000
[    3.966816] intel-ipu6 0000:00:05.0: IPU camera mask = 0x0
[    3.966937] intel-ipu6 0000:00:05.0: Skip ipc reset for non-secure mode
[    3.966940] intel-ipu6 0000:00:05.0: IPC reset done
[    3.968589] intel-ipu6 0000:00:05.0: Direct firmware load for intel/ipu6ep_fw.bin failed with error -2
[    3.971723] intel-ipu6 0000:00:05.0: FW version: 20230925
[    3.972493] ov02c10 i2c-OVTI02C1:00: failed to check hwcfg: -517
[    3.973109] intel-ipu6 0000:00:05.0: Found supported sensor OVTI02C1:00
[    3.973156] intel-ipu6 0000:00:05.0: Connected 1 cameras
[    3.975203] intel-ipu6 0000:00:05.0: IPU6-v3 driver version 1.0
[    4.009520] intel-ipu6-psys intel-ipu6-psys0: pkg_dir entry count:8
[    4.009663] intel-ipu6-psys intel-ipu6-psys0: psys probe minor: 0
[    4.071791] intel-ipu6-isys intel-ipu6-isys0: bind ov02c10 0-0036 nlanes is 2 port is 0
[    4.072127] intel-ipu6-isys intel-ipu6-isys0: All sensor registration completed.

more details here - https://github.com/intel/ipu6-drivers/issues/228#issuecomment-2286137675

+1 to get this added as I just bought a Galaxy book 3 Pro :)


Remove use of pkg:browser in tests

tests/html/custom/element_upgrade_test.html:49:      src="/packages/browser/interop.js"></script>
tests/lib_2/html/custom/element_upgrade_test.html:49:      src="/packages/browser/interop.js"></script>

@dgrove – who could run w/ this?

@terrylucas - can you please take a look at this?


Test issue for discord integration

This is a buch or random text with a
for c in a:
	pass

hmmm didn't seem to work


Add ElidePermutations transpiler pass

Summary
This commit adds a new transpiler pass ElidePermutations which is a logical optimization pass designed to run prior to layout or any other physical embedding steps in the transpiler pipeline. It traverses the DAG in topological order and when a swap gate is encountered it removes that gate and instead permutes the logical qubits for any subsequent gates in the DAG. This will eliminate any swaps or permutation gates in a circuit not caused by routing. Additionally, this pass is added to the preset pass manager for optimization level 3, we can consider adding it to other levels too if we think it makes sense (there is little overhead, and almost 0 if there are no swaps).
One thing to note is that this pass does not recurse into control flow blocks at all, it treats them as black box operations. So if any control flow blocks contain swap gates those will not be optimized away. This change was made because mapping the permutation outside and inside any control flow block was larger in scope than what the intent for this pass was. That type of work is better suited for the routing passes which already have to reason about this.
Details and comments

I just realized that I need to update the layout in the property set as part of this pass too, it's breaking 2 tests that are checking unitary equivalence and the from_circuit() method doesn't have enough information to factor in how things get moved around in this pass

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

48 of 48 (100.0%)  changed or added relevant lines in 7 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 85.42%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
67471

Relevant Lines:
78987

💛  - Coveralls

This is still failing one test locally for me, but it's a lot closer than it was when I left it ~6 months ago. The failure is in the case where we elide swaps but do not set a layout as part of the transpiler. In that case the final_layout handling doesn't seem to be working correctly and the output unitary with Operator.from_circuit() doesn't match the input circuits. It was working in an earlier iteration of the PR, but when I changed this line: https://github.com/Qiskit/qiskit/pull/9523/commits/1d30a65260eca8a7f93b550384413f2c82fd8b3c#diff-92d475523dd55dcfa92cfc8772ecddf189dbe7243cce7e020a1e55a876c77e7eL76 from the earlier iterations, to fix the case when an initial layout is set, it started failing when an initial layout isn't set. (I mostly pushed this to save my work)

I think this is basically ready for review now.
I do have one open question which I'm not sure we need to do anything about. But in the case when we do run routing and insert swaps, right now this PR ignores the output permutation caused by ElidePermutations and the final layout returned is just the output from routing. I'm thinking we might need to do the tracking so the returned layout object in qc.layout.final_layout shows the mapping through eliision_final_layout -> final_layout.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

65 of 70 (92.86%)  changed or added relevant lines in 7 files are covered.
16 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.004%) to 87.266%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit/transpiler/passes/optimization/elide_permutations.py
39
41
95.12%

qiskit/transpiler/passes/layout/apply_layout.py
4
7
57.14%

Files with Coverage Reduction
New Missed Lines
%

crates/qasm2/src/lex.rs
2
91.39%

qiskit/extensions/quantum_initializer/squ.py
2
80.0%

crates/qasm2/src/parse.rs
12
96.67%

Totals

Change from base Build<PHONE_NUMBER>:
-0.004%

Covered Lines:
74334

Relevant Lines:
85181

💛  - Coveralls

I added a test to address my concern when both ElidePermutations and routing run together in: https://github.com/Qiskit/qiskit/pull/9523/commits/0f1df1599e0a88d6148fcfc52ae16fc06aa4b40f and it seems to work despite my concerns the final_layout attribute wasn't valid in this case. So I think this is ready to go.

I'm fairly certain this is functionally correct and will pass CI after #12057 is merged. Thanks for your comments @kevinhartman - I addressed most of the remaining ones and resolved the associated conversation.
I think the pass name ElidePermutations is fine but I'm not entirely happy with it. I suspect a lot of non-native speakers will need to check their dictionary to understand what "elide" means but as soon as this is done I think it is a fair description of what the pass does.
Alternatively, I would suggest something like ResolveMidCircuitPermutations or PushPermutationsToCircuitBoundary but both have their individual drawbacks. :-)

Please see the updated summary. Please let me know if you think that adding ElidePermutations + FinalizeLayouts to the transpiler flow (with optimization_level=3) would make more sense in a separate PR. Intuitively, removing SWAPs/PermutationGates from the virtual circuit should almost always be beneficial.

There is a test error that seems unrelated to this PR. I am done with my changes, the PR is ready for review.

Fwiw (since I'm assigned) I read through this quickly and there's nothing I immediately wanted to interject beyond what Kevin/Matt already said.

Actually, scratch that, I do have a comment - do we think it's definitely better to have FinalizeLayouts as a separate pass that a user could forget to put into a pass manager, or would it be better to shift that logic into PassManager._passmanager_backend, which handles the final_layout in general, so it's not possible to forget?

Actually, scratch that, I do have a comment - do we think it's definitely better to have FinalizeLayouts as a separate pass that a user could forget to put into a pass manager, or would it be better to shift that logic into PassManager._passmanager_backend, which handles the final_layout in general, so it's not possible to forget?

FWIW, this is what I did in the original PR. I can adjust back to this, it makes sense to me to always handle this at the pass manager level with the other layout handling.

I integrated finalize layouts in the pass manager in: https://github.com/Qiskit/qiskit/pull/9523/commits/e755bcde5894a5bf327995967143c97361d68baa


Image rendering

To support sprites we need image rendering on the renderer including the ability to use sprite sheets

Done.


Migrate travis-ci.org to .com

Update the link and URL of the badge.
Cf. https://groups.google.com/g/vim_dev/c/fj7d7pBllx0/m/3dSWNIpfBgAJ

Included in 8.2.2274

Included in 8.2.2274


Doesn't load files/folder when owncloud-server is not in apache root dir

Description
Phoenix client doesn't load files/folders if the owncloud server is not in the apache root dir.
But, if the owncloud-server is in apache root dir everything works fine.
Steps to reproduce

Configure Apache to point www dir

Clone owncloud core insider www so that server is in www/core

Access owncloud server through localhost/core

Change phoenix config in config.json as:
{  "server" : "http://localhost/core",  "theme": "owncloud",  "version": "0.1.0",  "apps" : [  	"files"  ],  "auth": {  	"clientId": "j3VLpTvmRSxqNGJWxAmn4JA3Nu9BoFbL7EoM8zAcDiPfkBCYaXqiSTiW2CFbM1LP",  	"apiUrl": "http://localhost/core/index.php/apps/oauth2/api/v1/token",  	"authUrl": "http://localhost/core/index.php/apps/oauth2/authorize"  } }

Access Phoenix and login using oauth2

See that the files/folder are not loaded in the webUI (Note: was able to see them in owncloud-server frontend)

Note: You can see the files/folders in webdav response.

c.c @individual-it @phil-davis @DeepDiver1975

this was fixed with #212


Reverse order of web dashboard

Having new tasks show up on the last page isn't useful.

I also noticed the cancel button in the dashboard is clickable for tasks with the state "SYSTEM_ERROR" - that also should be fixed