[u-u] AI - Llama 2 an open source AI that can run on a Raspberry PI

Sun Jan 14 13:19:41 EST 2024

On Wed, 10 Jan 2024 at 21:21, Colin McGregor <colin.mc151 at gmail.com> wrote:
>
> This is a lightly edited version of a post that I made to another
> mailing list recently (sorry, not sorry).
>
> As noted before this evening's meeting I am running the open source
> (but NOT GPL) Llama 2  AI on a Raspberry PI 5. Llama 2 is a product of
> Meta (parent company of Facebook), and as you will see in clause 2 of
> their licence agreement, there is a poison pill clause aimed at the
> likes of Google (for a hobbyist interested in seeing / playing with a
> local AI there are no issues) :  https://ai.meta.com/llama/license/ .
>
> Here is the guide I used for getting Llama 2 up and running on a
> Raspberry PI (I am using the PI 5, but as the video notes (subject to
> conditions) it can run on a PI 4) :
> https://youtu.be/idZctq7WIq4?si=kekFWQHty60dp2ce
>
> There is an old saying "The amazing thing about a dancing bear is not
> how well the bear dances, but that the bear can dance at all.". So,
> yes, I will to some degree mock Llama 2 on a Raspberry PI in the
> following, but... It is kind of amazing that you can get a functional,
> local AI running on a machine as modest as Raspberry Pi, so the flaws
> are very forgivable.
>
> I have run Llama 2 on a Raspberry Pi 5 with 4GB of RAM and one with
> 8GB of RAM, with no obvious difference in performance. At present the
> key issue appears to be storage I/O performance, to try to attack that
> I currently have a PCIe to NVMe adapter on order from Pimoroni that
> will let me use NVMe SSD storage with the PI (I want to see if/how
> much that helps). I have also tried the 7b and 13b language models on
> the Pi, the 7b runs (crawls?) faster than the 13b model, but the 13b
> model gives better (but far from perfect) answers. I am currently
> normally using the 13b model and accept the slowness. As it is,
> obviously different questions take different amounts of time, but as
> things stand, I will ask a question in the evening and see how things
> are coming the next morning after a good night's sleep.
>
> I have been keeping a log of the questions / answers I have had with
> Llama 2 and I will start with a question I asked both the 7b and 13b
> models. For those who didn't take (or failed) Canadian history in
> school, let me point out that Lester B. Pearson didn't have any
> significant role in the Cuban Missile Crisis of the 1960s (but his
> role in the 1950s Suez Crisis got him a Nobel Peace Prize). Also, Joe
> Clark was Prime Minister of Canada from June 4, 1979 to March 3, 1980
> so Pierre Trudeau both preceded and succeeded Joe Clark. These are two
> of the issues (there are others) in the 7b answer. There is some
> legitimate room for debate on which Prime Ministers should be on lists
> like these, but I don't consider any of the names presented as clearly
> wrong.
>
> 7b :
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> > Who was Canada's best Prime Minister?
> Canada has had several effective and well-respected Prime Ministers
> throughout its history, making it difficult to pinpoint a single
> individual as the "best." However, some notable Prime Ministers who
> have made significant contributions to the country include:
>
> 1. Sir John A. Macdonald (1867-1873, 1878-1880): Often considered one
> of Canada's most important Prime Ministers, Macdonald played a crucial
> role in shaping the country's political and economic systems. He was
> instrumental in creating the National Policy, which helped establish
> Canada as a major industrial power, and he also negotiated the
> Northwest Passage Agreement with the United States, securing Canada's
> western border.
> 2. William Lyon Mackenzie King (1921-1930, 1935-1948): King was a
> skilled diplomat and leader who guided Canada through two major
> crises: World War I and the Great Depression. He implemented policies
> that helped stabilize the economy and provided support to those
> affected by the economic downturn. Additionally, he played a key role
> in establishing the Canadian Broadcasting Corporation (CBC), which has
> become an important institution for Canadian culture and identity.
> 3. Lester B. Pearson (1963-1967): Pearson is best known for his role
> in resolving the Cuban Missile Crisis, which brought the world closer
> to nuclear war. He also played a crucial part in establishing the
> United Nations, for which he was awarded the Nobel Peace Prize in
> 1957. As Prime Minister, Pearson implemented policies that promoted
> peace and disarmament, and he worked tirelessly to improve Canada's
> international reputation.
> 4. Pierre Trudeau (1968-1984): Trudeau is remembered for his
> charismatic leadership and his efforts to promote Canadian identity.
> He implemented a number of progressive policies, including the
> patriation of the Canadian Constitution, which gave the province of
> Quebec greater autonomy, and he also legalized same-sex marriage in
> 1979. Additionally, Trudeau was instrumental in establishing the
> Canadian Human Rights Commission, which has helped protect the rights
> of marginalized communities.
> 5. Jean Chrétien (1993-2003): Chrétien is credited with helping to
> stabilize Canada's economy and promote international trade. He also
> played a key role in the Belfast Agreement, which brought an end to
> the Troubles in Northern Ireland. Additionally, he implemented
> policies that helped to promote diversity and inclusion within
> Canadian society.
> Of course, there are many other notable Prime Ministers in Canadian
> history, but these five individuals have had particularly significant
> impacts on the country's political landscape and international
> reputation.
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 13b
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> > Who was Canada's best Prime Minister?
>  Determining the "best" Prime Minister of any country is subjective
> and can depend on various factors such as political affiliation,
> policy preferences, and the current events that occurred during their
> tenure. However, here are some of the most highly-regarded Canadian
> Prime Ministers based on historical assessments and public opinion:
>
> 1. Pierre Trudeau (1968-1979, 1980-1984): Known for his charisma,
> progressive policies, and efforts to promote Canadian unity, Trudeau
> is often considered one of Canada's greatest Prime Ministers. He
> implemented policies such as official bilingualism, the Canadian
> Charter of Rights and Freedoms, and the creation of the Canadian Human
> Rights Commission.
> 2. John A. Macdonald (1867-1873, 1878-1891): As Canada's first Prime
> Minister, Macdonald played a crucial role in shaping the country's
> Confederation and establishing many of its foundational institutions.
> He is credited with developing the National Policy, which aimed to
> promote economic growth and unity through tariffs and transportation
> infrastructure.
> 3. Lester B. Pearson (1963-1968): Pearson's tenure as Prime Minister
> saw significant social and political changes in Canada, including the
> introduction of universal healthcare, the establishment of a new flag,
> and the devolution of power to Quebec. He was also instrumental in
> promoting Canadian diplomacy and peacekeeping efforts globally.
> 4. Wilfrid Laurier (1896-1911): Laurier is often praised for his
> leadership during Canada's early years, when the country faced
> challenges such as population growth, economic development, and the
> need for greater autonomy from Britain. He is credited with promoting
> French-English relations and expanding the scope of federal powers.
> 5. Jean Chrétien (1993-2003): Chrétien's tenure saw significant
> economic growth, the repayment of Canada's national debt, and the
> negotiation of several major international agreements, including the
> Free Trade Agreement with the United States. He also played a key role
> in promoting Canadian unity during the Quebec sovereignty referendum
> in 1995.
>
> Of course, these are just a few examples of highly-regarded Prime
> Ministers in Canadian history. Other notable leaders include William
> Lyon Mackenzie King, Joe Clark, and Brian Mulroney, among others.
> Ultimately, the "best" Prime Minister is a matter of personal opinion
> and depends on one's values, priorities, and perspectives on Canadian
> history.
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> As someone who lives near Yonge & Eglinton, I found the following
> answer out of the 13b model ALMOST amusing:
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> > When will Toronto's Eglinton Crosstown LRT line open?
>  The Eglinton Crosstown Light Rail Transit (LRT) line is a major
> transit project under construction in Toronto, Canada. The project has
> been underway since 2013 and is expected to be completed in two
> phases. Here's the current status of the project and when you can
> expect it to open:
>
> Phase 1:
>
> * The first phase of the Eglinton Crosstown LRT line will run from
> Kennedy Station in the east to Brentcliffe Road in the west, with 25
> stops along the route.
> * This phase is expected to be completed and open to the public in spring 2023.
>
> Phase 2:
>
> * The second phase of the Eglinton Crosstown LRT line will extend the
> line from Brentcliffe Road to the Toronto Pearson International
> Airport, with 15 additional stops.
> * This phase is expected to be completed and open to the public in the
> fall of 2024.
>
> Please note that these dates are subject to change based on various
> factors, including construction progress and any unforeseen delays.
> The Toronto Transit Commission (TTC) will provide updates on the
> project's status and opening dates as more information becomes
> available.
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Two questions I asked Llama 2 13b related to itself, which I assume to
> be totally correct:
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> > Does Llama 2 learn from past questions?
>  Yes, I can learn from past questions! As a large language model, I
> have the ability to retain information and use it to inform my
> responses to future questions. This allows me to improve my
> performance over time and provide more accurate and helpful answers.
>
> For example, if you ask me a question that I have seen before, I can
> draw on my past knowledge to provide a more informed response.
> Additionally, I can use my understanding of language and context to
> recognize when a question is similar to one I have seen before, and
> adapt my response accordingly.
>
> However, it's important to note that my ability to learn from past
> questions is limited by the data that I have been trained on. While I
> have been trained on a vast amount of text data, there may be certain
> topics or questions that I am not familiar with or cannot answer
> accurately. In these cases, I will do my best to provide a helpful
> response based on the information that I do have.
>
> > Do you access the internet when answering questions?
>  Yes, I do access the internet when answering questions! As a large
> language model, my primary function is to assist with tasks and answer
> questions to the best of my ability. To do this, I rely on a vast
> amount of text data that I have been trained on, as well as the
> ability to access the internet for more up-to-date information.
>
> When you ask me a question, I use a combination of natural language
> processing (NLP) and machine learning algorithms to understand the
> context and intent behind your question. I then draw on my training
> data and any relevant information available online to provide an
> accurate and helpful response.
>
> However, it's important to note that while I can access the internet
> for information, I do not have the ability to browse or search the web
> in the same way that a human would. My primary focus is on providing
> answers to specific questions based on the information that I have
> been trained on and what is available online.
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> I find the Llama 2 answers to be interesting enough to keep the
> program around, but I wouldn't use the answers it provides without
> some extra checking / verification.
>
> Any event, I could go on, but... Any questions for me or Llama 2?

According to 7b, Pierre Trudeau "... also legalized same-sex marriage
in 1979."  I lived through that, and I definitely don't remember it.
Mostly because it didn't happen.  If you're curious, see
https://en.wikipedia.org/wiki/Same-sex_marriage_in_Canada - Wikipedia
is far from perfect, but I think it's more accurate on that subject.
I notice that the 13b version fixed that particular problem, but I'm
pretty sure it only switched it out for more subtly incorrect answers.
This is something that really alarms me about AI right now: you have
to fact-check it like crazy to get good answers, and 99% of the users
will instead take the answers as gospel.

My apologies: I'm stepping over from the computing aspects of this
into the philosophical ... but I do find it quite alarming.  And this
is before we get into the wonderful potential for _deliberate_ bias,
which will undoubtedly be exploited as we progress with AI.

-- 
Giles
https://www.gilesorr.com/
gilesorr at gmail.com