[u-u] AI - Llama 2 an open source AI that can run on aRaspberry PI

[Hugh Gamble]

 Before prompt engineering, first you have to understand how a 
generative AI LLM works.
It was trained once, at a fixed point in history, on a specific data 
set.
Don't try to ask time based questions, newer than your training date.

Commercial LLMs offer a hybrid model.
e.g. Windows copilot gives you Bing, which uses ChatGPT (3.5 or 
possibly newer now)
which also falls back to Bing.

But do, try to carry on a conversation, and argue with it.
Just remember to reset the conversation when you want to start fresh.

>
> --- Original message ---
> Subject: Re: [u-u] AI - Llama 2 an open source AI that can run on 
> aRaspberry PI
> From: Giles Orr <gilesorr at gmail.com>
> To: Colin McGregor <colin.mc151 at gmail.com>
> Cc: UU <u-u at unixunanimous.org>
> Date: Sunday, 14/01/2024 1:25 PM
>
> On Wed, 10 Jan 2024 at 21:21, Colin McGregor <colin.mc151 at gmail.com> 
> wrote:
>>
>>
>> This is a lightly edited version of a post that I made to another
>> mailing list recently (sorry, not sorry).
>>
>> As noted before this evening's meeting I am running the open source
>> (but NOT GPL) Llama 2  AI on a Raspberry PI 5. Llama 2 is a product of
>> Meta (parent company of Facebook), and as you will see in clause 2 of
>> their licence agreement, there is a poison pill clause aimed at the
>> likes of Google (for a hobbyist interested in seeing / playing with a
>> local AI there are no issues) : https://ai.meta.com/llama/license/ .
>>
>> Here is the guide I used for getting Llama 2 up and running on a
>> Raspberry PI (I am using the PI 5, but as the video notes (subject to
>> conditions) it can run on a PI 4) :
>> https://youtu.be/idZctq7WIq4?si=kekFWQHty60dp2ce
>>
>> There is an old saying "The amazing thing about a dancing bear is not
>> how well the bear dances, but that the bear can dance at all.". So,
>> yes, I will to some degree mock Llama 2 on a Raspberry PI in the
>> following, but... It is kind of amazing that you can get a functional,
>> local AI running on a machine as modest as Raspberry Pi, so the flaws
>> are very forgivable.
>>
>> I have run Llama 2 on a Raspberry Pi 5 with 4GB of RAM and one with
>> 8GB of RAM, with no obvious difference in performance. At present the
>> key issue appears to be storage I/O performance, to try to attack that
>> I currently have a PCIe to NVMe adapter on order from Pimoroni that
>> will let me use NVMe SSD storage with the PI (I want to see if/how
>> much that helps). I have also tried the 7b and 13b language models on
>> the Pi, the 7b runs (crawls?) faster than the 13b model, but the 13b
>> model gives better (but far from perfect) answers. I am currently
>> normally using the 13b model and accept the slowness. As it is,
>> obviously different questions take different amounts of time, but as
>> things stand, I will ask a question in the evening and see how things
>> are coming the next morning after a good night's sleep.
>>
>> I have been keeping a log of the questions / answers I have had with
>> Llama 2 and I will start with a question I asked both the 7b and 13b
>> models. For those who didn't take (or failed) Canadian history in
>> school, let me point out that Lester B. Pearson didn't have any
>> significant role in the Cuban Missile Crisis of the 1960s (but his
>> role in the 1950s Suez Crisis got him a Nobel Peace Prize). Also, Joe
>> Clark was Prime Minister of Canada from June 4, 1979 to March 3, 1980
>> so Pierre Trudeau both preceded and succeeded Joe Clark. These are two
>> of the issues (there are others) in the 7b answer. There is some
>> legitimate room for debate on which Prime Ministers should be on lists
>> like these, but I don't consider any of the names presented as clearly
>> wrong.
>>
>> 7b :
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>>
>>> Who was Canada's best Prime Minister?
>> Canada has had several effective and well-respected Prime Ministers
>> throughout its history, making it difficult to pinpoint a single
>> individual as the "best." However, some notable Prime Ministers who
>> have made significant contributions to the country include:
>>
>> 1. Sir John A. Macdonald (1867-1873, 1878-1880): Often considered one
>> of Canada's most important Prime Ministers, Macdonald played a crucial
>> role in shaping the country's political and economic systems. He was
>> instrumental in creating the National Policy, which helped establish
>> Canada as a major industrial power, and he also negotiated the
>> Northwest Passage Agreement with the United States, securing Canada's
>> western border.
>> 2. William Lyon Mackenzie King (1921-1930, 1935-1948): King was a
>> skilled diplomat and leader who guided Canada through two major
>> crises: World War I and the Great Depression. He implemented policies
>> that helped stabilize the economy and provided support to those
>> affected by the economic downturn. Additionally, he played a key role
>> in establishing the Canadian Broadcasting Corporation (CBC), which has
>> become an important institution for Canadian culture and identity.
>> 3. Lester B. Pearson (1963-1967): Pearson is best known for his role
>> in resolving the Cuban Missile Crisis, which brought the world closer
>> to nuclear war. He also played a crucial part in establishing the
>> United Nations, for which he was awarded the Nobel Peace Prize in
>> 1957. As Prime Minister, Pearson implemented policies that promoted
>> peace and disarmament, and he worked tirelessly to improve Canada's
>> international reputation.
>> 4. Pierre Trudeau (1968-1984): Trudeau is remembered for his
>> charismatic leadership and his efforts to promote Canadian identity.
>> He implemented a number of progressive policies, including the
>> patriation of the Canadian Constitution, which gave the province of
>> Quebec greater autonomy, and he also legalized same-sex marriage in
>> 1979. Additionally, Trudeau was instrumental in establishing the
>> Canadian Human Rights Commission, which has helped protect the rights
>> of marginalized communities.
>> 5. Jean Chrétien (1993-2003): Chrétien is credited with helping to
>> stabilize Canada's economy and promote international trade. He also
>> played a key role in the Belfast Agreement, which brought an end to
>> the Troubles in Northern Ireland. Additionally, he implemented
>> policies that helped to promote diversity and inclusion within
>> Canadian society.
>> Of course, there are many other notable Prime Ministers in Canadian
>> history, but these five individuals have had particularly significant
>> impacts on the country's political landscape and international
>> reputation.
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> 13b
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>>
>>> Who was Canada's best Prime Minister?
>>    Determining the "best" Prime Minister of any country is subjective
>> and can depend on various factors such as political affiliation,
>> policy preferences, and the current events that occurred during their
>> tenure. However, here are some of the most highly-regarded Canadian
>> Prime Ministers based on historical assessments and public opinion:
>>
>> 1. Pierre Trudeau (1968-1979, 1980-1984): Known for his charisma,
>> progressive policies, and efforts to promote Canadian unity, Trudeau
>> is often considered one of Canada's greatest Prime Ministers. He
>> implemented policies such as official bilingualism, the Canadian
>> Charter of Rights and Freedoms, and the creation of the Canadian Human
>> Rights Commission.
>> 2. John A. Macdonald (1867-1873, 1878-1891): As Canada's first Prime
>> Minister, Macdonald played a crucial role in shaping the country's
>> Confederation and establishing many of its foundational institutions.
>> He is credited with developing the National Policy, which aimed to
>> promote economic growth and unity through tariffs and transportation
>> infrastructure.
>> 3. Lester B. Pearson (1963-1968): Pearson's tenure as Prime Minister
>> saw significant social and political changes in Canada, including the
>> introduction of universal healthcare, the establishment of a new flag,
>> and the devolution of power to Quebec. He was also instrumental in
>> promoting Canadian diplomacy and peacekeeping efforts globally.
>> 4. Wilfrid Laurier (1896-1911): Laurier is often praised for his
>> leadership during Canada's early years, when the country faced
>> challenges such as population growth, economic development, and the
>> need for greater autonomy from Britain. He is credited with promoting
>> French-English relations and expanding the scope of federal powers.
>> 5. Jean Chrétien (1993-2003): Chrétien's tenure saw significant
>> economic growth, the repayment of Canada's national debt, and the
>> negotiation of several major international agreements, including the
>> Free Trade Agreement with the United States. He also played a key role
>> in promoting Canadian unity during the Quebec sovereignty referendum
>> in 1995.
>>
>> Of course, these are just a few examples of highly-regarded Prime
>> Ministers in Canadian history. Other notable leaders include William
>> Lyon Mackenzie King, Joe Clark, and Brian Mulroney, among others.
>> Ultimately, the "best" Prime Minister is a matter of personal opinion
>> and depends on one's values, priorities, and perspectives on Canadian
>> history.
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> As someone who lives near Yonge & Eglinton, I found the following
>> answer out of the 13b model ALMOST amusing:
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>>
>>> When will Toronto's Eglinton Crosstown LRT line open?
>>    The Eglinton Crosstown Light Rail Transit (LRT) line is a major
>> transit project under construction in Toronto, Canada. The project has
>> been underway since 2013 and is expected to be completed in two
>> phases. Here's the current status of the project and when you can
>> expect it to open:
>>
>> Phase 1:
>>
>> * The first phase of the Eglinton Crosstown LRT line will run from
>> Kennedy Station in the east to Brentcliffe Road in the west, with 25
>> stops along the route.
>> * This phase is expected to be completed and open to the public in 
>> spring 2023.
>>
>> Phase 2:
>>
>> * The second phase of the Eglinton Crosstown LRT line will extend the
>> line from Brentcliffe Road to the Toronto Pearson International
>> Airport, with 15 additional stops.
>> * This phase is expected to be completed and open to the public in the
>> fall of 2024.
>>
>> Please note that these dates are subject to change based on various
>> factors, including construction progress and any unforeseen delays.
>> The Toronto Transit Commission (TTC) will provide updates on the
>> project's status and opening dates as more information becomes
>> available.
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> Two questions I asked Llama 2 13b related to itself, which I assume to
>> be totally correct:
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>>
>>> Does Llama 2 learn from past questions?
>>    Yes, I can learn from past questions! As a large language model, I
>> have the ability to retain information and use it to inform my
>> responses to future questions. This allows me to improve my
>> performance over time and provide more accurate and helpful answers.
>>
>> For example, if you ask me a question that I have seen before, I can
>> draw on my past knowledge to provide a more informed response.
>> Additionally, I can use my understanding of language and context to
>> recognize when a question is similar to one I have seen before, and
>> adapt my response accordingly.
>>
>> However, it's important to note that my ability to learn from past
>> questions is limited by the data that I have been trained on. While I
>> have been trained on a vast amount of text data, there may be certain
>> topics or questions that I am not familiar with or cannot answer
>> accurately. In these cases, I will do my best to provide a helpful
>> response based on the information that I do have.
>>
>>>
>>> Do you access the internet when answering questions?
>>    Yes, I do access the internet when answering questions! As a large
>> language model, my primary function is to assist with tasks and answer
>> questions to the best of my ability. To do this, I rely on a vast
>> amount of text data that I have been trained on, as well as the
>> ability to access the internet for more up-to-date information.
>>
>> When you ask me a question, I use a combination of natural language
>> processing (NLP) and machine learning algorithms to understand the
>> context and intent behind your question. I then draw on my training
>> data and any relevant information available online to provide an
>> accurate and helpful response.
>>
>> However, it's important to note that while I can access the internet
>> for information, I do not have the ability to browse or search the web
>> in the same way that a human would. My primary focus is on providing
>> answers to specific questions based on the information that I have
>> been trained on and what is available online.
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> I find the Llama 2 answers to be interesting enough to keep the
>> program around, but I wouldn't use the answers it provides without
>> some extra checking / verification.
>>
>> Any event, I could go on, but... Any questions for me or Llama 2?
>
>
> According to 7b, Pierre Trudeau "... also legalized same-sex marriage
> in 1979."  I lived through that, and I definitely don't remember it.
> Mostly because it didn't happen.  If you're curious, see
> https://en.wikipedia.org/wiki/Same-sex_marriage_in_Canada - Wikipedia
> is far from perfect, but I think it's more accurate on that subject.
> I notice that the 13b version fixed that particular problem, but I'm
> pretty sure it only switched it out for more subtly incorrect answers.
> This is something that really alarms me about AI right now: you have
> to fact-check it like crazy to get good answers, and 99% of the users
> will instead take the answers as gospel.
>
> My apologies: I'm stepping over from the computing aspects of this
> into the philosophical ... but I do find it quite alarming.  And this
> is before we get into the wonderful potential for _deliberate_ bias,
> which will undoubtedly be exploited as we progress with AI.
>
> --
> Giles
> https://www.gilesorr.com/
> gilesorr at gmail.com
> _______________________________________________
> u-u mailing list
> u-u at mail-ml.infra-service.ca
> http://www2.infra-service.ca/mailman/listinfo/u-u

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www2.infra-service.ca/pipermail/u-u/attachments/20240114/228ad98e/attachment-0001.html>